2023-10-30T12:18:53.953 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-10-30T12:18:53.955 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} --verbose --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026 --name leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi -- /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026/orig.config.yaml 2023-10-30T12:18:54.009 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-30T12:18:54.100 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026 branch: wip-lusov-balance-after-stop description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} email: null first_in_suite: false job_id: '7441026' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-lusov-balance-after-stop ceph: cephfs: max_mds: 3 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: false mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - Replacing daemon mds sha1: 00c1c13cea21a686b046584596fbeac5315a7d71 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 00c1c13cea21a686b046584596fbeac5315a7d71 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-balance-after-stop sha1: 00c1c13cea21a686b046584596fbeac5315a7d71 owner: scheduled_leonidus@teuthology priority: 150 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 8397 sha1: 00c1c13cea21a686b046584596fbeac5315a7d71 sleep_before_teardown: 0 subset: 464/1024 suite: fs suite_branch: wip-lusov-balance-after-stop suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 00c1c13cea21a686b046584596fbeac5315a7d71 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOIXUlZNg24Ie/pDwa4MO4k4bufpZa63YdkTdS0B3ACISQbEG5xzJxPppwdXu1r1e/5mohao6G9qZoXXF4DvJ6k= smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFC+TrPsAKqCG2WeGXqVschBg9bQSR3yot22TUIQ+OSTc5On7+Srgg955m9m+XBsGGUGsXdNNBTJIM5PraPhF/4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - mds_thrash: null - workunit: clients: all: - fs/snaps timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-10-30_06:34:16 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.770726 2023-10-30T12:18:54.101 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa; will attempt to use it 2023-10-30T12:18:54.101 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa/tasks 2023-10-30T12:18:54.102 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-30T12:18:54.104 INFO:teuthology.task.internal:Checking packages... 2023-10-30T12:18:54.127 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '00c1c13cea21a686b046584596fbeac5315a7d71' 2023-10-30T12:18:54.128 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-30T12:18:54.128 INFO:teuthology.packaging:ref: None 2023-10-30T12:18:54.128 INFO:teuthology.packaging:tag: None 2023-10-30T12:18:54.129 INFO:teuthology.packaging:branch: wip-lusov-balance-after-stop 2023-10-30T12:18:54.129 INFO:teuthology.packaging:sha1: 00c1c13cea21a686b046584596fbeac5315a7d71 2023-10-30T12:18:54.129 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-lusov-balance-after-stop 2023-10-30T12:18:54.273 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:18:54.273 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-30T12:18:54.284 INFO:teuthology.task.internal:no buildpackages task found 2023-10-30T12:18:54.285 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-30T12:18:54.342 INFO:teuthology.task.internal:Saving configuration 2023-10-30T12:18:54.361 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-30T12:18:54.373 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-30T12:18:54.400 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-30 12:13:00.703267', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIL0JoQqn41BjmHxnVkENZklMLAJDP+/dNVH37kpamTx'} 2023-10-30T12:18:54.422 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-30 12:13:00.705372', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFKgLp6O7OFkuiMJQesX3oGOdNxR25C4MNaNXDP/RD3h'} 2023-10-30T12:18:54.422 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-30T12:18:54.431 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2023-10-30T12:18:54.432 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2023-10-30T12:18:54.432 INFO:teuthology.run_tasks:Running task console_log... 2023-10-30T12:18:54.525 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc75b21cc10>, signals=[15]) 2023-10-30T12:18:54.526 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-30T12:18:54.532 INFO:teuthology.task.internal:Opening connections... 2023-10-30T12:18:54.533 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2023-10-30T12:18:54.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T12:18:54.613 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2023-10-30T12:18:54.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T12:18:54.691 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-30T12:18:54.698 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2023-10-30T12:18:54.703 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2023-10-30T12:18:54.704 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2023-10-30T12:18:54.753 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-10-30T12:18:54.753 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Ubuntu" 2023-10-30T12:18:54.754 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="22.04" 2023-10-30T12:18:54.754 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-10-30T12:18:54.754 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_CODENAME=jammy 2023-10-30T12:18:54.754 INFO:teuthology.orchestra.run.smithi012.stdout:ID=ubuntu 2023-10-30T12:18:54.755 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE=debian 2023-10-30T12:18:54.755 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-30T12:18:54.755 INFO:teuthology.orchestra.run.smithi012.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-30T12:18:54.755 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-30T12:18:54.755 INFO:teuthology.orchestra.run.smithi012.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-30T12:18:54.756 INFO:teuthology.orchestra.run.smithi012.stdout:UBUNTU_CODENAME=jammy 2023-10-30T12:18:54.756 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2023-10-30T12:18:54.777 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2023-10-30T12:18:54.782 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2023-10-30T12:18:54.783 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2023-10-30T12:18:54.832 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-10-30T12:18:54.832 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="Ubuntu" 2023-10-30T12:18:54.833 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="22.04" 2023-10-30T12:18:54.833 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-10-30T12:18:54.833 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_CODENAME=jammy 2023-10-30T12:18:54.833 INFO:teuthology.orchestra.run.smithi154.stdout:ID=ubuntu 2023-10-30T12:18:54.834 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE=debian 2023-10-30T12:18:54.834 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-30T12:18:54.834 INFO:teuthology.orchestra.run.smithi154.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-30T12:18:54.834 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-30T12:18:54.835 INFO:teuthology.orchestra.run.smithi154.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-30T12:18:54.835 INFO:teuthology.orchestra.run.smithi154.stdout:UBUNTU_CODENAME=jammy 2023-10-30T12:18:54.838 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2023-10-30T12:18:54.868 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-30T12:18:54.877 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-30T12:18:54.907 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-30T12:18:54.907 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2023-10-30T12:18:54.910 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2023-10-30T12:18:54.914 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-30T12:18:54.940 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-30T12:18:54.940 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2023-10-30T12:18:54.959 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2023-10-30T12:18:54.977 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-30T12:18:55.029 INFO:teuthology.run_tasks:Running task kernel... 2023-10-30T12:18:55.045 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-30T12:18:55.046 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-10-30T12:18:55.046 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-10-30T12:18:55.047 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-30T12:18:55.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:18:55.053 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2023-10-30T12:18:55.219 DEBUG:teuthology.task.kernel:sha1 for client.0 is eacf00cc51e204f401b4d5d4f84825a68f373c74 2023-10-30T12:18:55.220 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "eacf00cc51e204f401b4d5d4f84825a68f373c74"... 2023-10-30T12:18:55.220 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2023-10-30T12:18:55.225 INFO:teuthology.orchestra.run.smithi012.stdout:5.15.0-56-generic 2023-10-30T12:18:55.226 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs eacf00cc51e204f401b4d5d4f84825a68f373c74 2023-10-30T12:18:55.227 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-30T12:18:55.227 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2023-10-30T12:18:55.227 DEBUG:teuthology.orchestra.run.smithi012:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-10-30T12:18:55.297 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-10-30T12:18:55.412 INFO:teuthology.orchestra.run.smithi012.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-10-30T12:18:55.413 INFO:teuthology.orchestra.run.smithi012.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-10-30T12:18:55.413 INFO:teuthology.orchestra.run.smithi012.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-10-30T12:18:55.413 INFO:teuthology.orchestra.run.smithi012.stderr:hint: 2023-10-30T12:18:55.413 INFO:teuthology.orchestra.run.smithi012.stderr:hint: git config --global init.defaultBranch 2023-10-30T12:18:55.414 INFO:teuthology.orchestra.run.smithi012.stderr:hint: 2023-10-30T12:18:55.414 INFO:teuthology.orchestra.run.smithi012.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-10-30T12:18:55.414 INFO:teuthology.orchestra.run.smithi012.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-10-30T12:18:55.415 INFO:teuthology.orchestra.run.smithi012.stderr:hint: 2023-10-30T12:18:55.415 INFO:teuthology.orchestra.run.smithi012.stderr:hint: git branch -m 2023-10-30T12:18:55.416 INFO:teuthology.orchestra.run.smithi012.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-10-30T12:18:55.416 DEBUG:teuthology.orchestra.run.smithi012:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-10-30T12:18:55.481 DEBUG:teuthology.orchestra.run.smithi012:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-10-30T12:18:55.534 DEBUG:teuthology.orchestra.run.smithi012:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-10-30T12:20:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-10-30T12:20:11.318 INFO:teuthology.orchestra.run.smithi012.stderr: * [new branch] main -> origin/main 2023-10-30T12:20:11.351 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20190312 -> 20190312 2023-10-30T12:20:11.351 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20190416 -> 20190416 2023-10-30T12:20:11.351 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20190514 -> 20190514 2023-10-30T12:20:11.351 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20190618 -> 20190618 2023-10-30T12:20:11.352 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20190717 -> 20190717 2023-10-30T12:20:11.352 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20190815 -> 20190815 2023-10-30T12:20:11.352 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20190923 -> 20190923 2023-10-30T12:20:11.352 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20191022 -> 20191022 2023-10-30T12:20:11.352 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20191215 -> 20191215 2023-10-30T12:20:11.353 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200122 -> 20200122 2023-10-30T12:20:11.353 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200316 -> 20200316 2023-10-30T12:20:11.353 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200421 -> 20200421 2023-10-30T12:20:11.356 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200519 -> 20200519 2023-10-30T12:20:11.357 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200619 -> 20200619 2023-10-30T12:20:11.357 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200721 -> 20200721 2023-10-30T12:20:11.357 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200817 -> 20200817 2023-10-30T12:20:11.357 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200918 -> 20200918 2023-10-30T12:20:11.357 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20201022 -> 20201022 2023-10-30T12:20:11.358 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20201118 -> 20201118 2023-10-30T12:20:11.358 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20201218 -> 20201218 2023-10-30T12:20:11.358 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20210208 -> 20210208 2023-10-30T12:20:11.358 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20210315 -> 20210315 2023-10-30T12:20:11.358 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20210511 -> 20210511 2023-10-30T12:20:11.359 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20210716 -> 20210716 2023-10-30T12:20:11.359 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20210818 -> 20210818 2023-10-30T12:20:11.359 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20210919 -> 20210919 2023-10-30T12:20:11.359 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20211027 -> 20211027 2023-10-30T12:20:11.360 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20211216 -> 20211216 2023-10-30T12:20:11.360 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220209 -> 20220209 2023-10-30T12:20:11.360 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220310 -> 20220310 2023-10-30T12:20:11.360 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220411 -> 20220411 2023-10-30T12:20:11.360 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220509 -> 20220509 2023-10-30T12:20:11.361 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220610 -> 20220610 2023-10-30T12:20:11.361 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220708 -> 20220708 2023-10-30T12:20:11.361 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220815 -> 20220815 2023-10-30T12:20:11.361 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220913 -> 20220913 2023-10-30T12:20:11.361 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20221012 -> 20221012 2023-10-30T12:20:11.362 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20221109 -> 20221109 2023-10-30T12:20:11.362 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20221214 -> 20221214 2023-10-30T12:20:11.362 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230117 -> 20230117 2023-10-30T12:20:11.362 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230210 -> 20230210 2023-10-30T12:20:11.362 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230310 -> 20230310 2023-10-30T12:20:11.363 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230404 -> 20230404 2023-10-30T12:20:11.363 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230515 -> 20230515 2023-10-30T12:20:11.363 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230625 -> 20230625 2023-10-30T12:20:11.363 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230804 -> 20230804 2023-10-30T12:20:11.363 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230919 -> 20230919 2023-10-30T12:20:15.291 INFO:teuthology.orchestra.run.smithi012.stderr:Updating files: 48% (1545/3168) Updating files: 49% (1553/3168) Updating files: 50% (1584/3168) Updating files: 51% (1616/3168) Updating files: 52% (1648/3168) Updating files: 53% (1680/3168) Updating files: 54% (1711/3168) Updating files: 55% (1743/3168) Updating files: 56% (1775/3168) Updating files: 57% (1806/3168) Updating files: 58% (1838/3168) Updating files: 59% (1870/3168) Updating files: 60% (1901/3168) Updating files: 61% (1933/3168) Updating files: 62% (1965/3168) Updating files: 63% (1996/3168) Updating files: 64% (2028/3168) Updating files: 65% (2060/3168) Updating files: 66% (2091/3168) Updating files: 67% (2123/3168) Updating files: 68% (2155/3168) Updating files: 69% (2186/3168) Updating files: 70% (2218/3168) Updating files: 71% (2250/3168) Updating files: 72% (2281/3168) Updating files: 73% (2313/3168) Updating files: 74% (2345/3168) Updating files: 75% (2376/3168) Updating files: 76% (2408/3168) Updating files: 77% (2440/3168) Updating files: 78% (2472/3168) Updating files: 79% (2503/3168) Updating files: 80% (2535/3168) Updating files: 81% (2567/3168) Updating files: 81% (2580/3168) Updating files: 82% (2598/3168) Updating files: 83% (2630/3168) Updating files: 84% (2662/3168) Updating files: 85% (2693/3168) Updating files: 86% (2725/3168) Updating files: 87% (2757/3168) Updating files: 88% (2788/3168) Updating files: 89% (2820/3168) Updating files: 90% (2852/3168) Updating files: 91% (2883/3168) Updating files: 92% (2915/3168) Updating files: 93% (2947/3168) Updating files: 94% (2978/3168) Updating files: 95% (3010/3168) Updating files: 96% (3042/3168) Updating files: 97% (3073/3168) Updating files: 98% (3105/3168) Updating files: 99% (3137/3168) Updating files: 100% (3168/3168) Updating files: 100% (3168/3168), done. 2023-10-30T12:20:15.296 INFO:teuthology.orchestra.run.smithi012.stdout:HEAD is now at 2b304bfe Merge branch 'main' into 'main' 2023-10-30T12:20:15.317 INFO:teuthology.task.kernel:Downloading kernel eacf00cc51e204f401b4d5d4f84825a68f373c74 on client.0... 2023-10-30T12:20:15.317 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=eacf00cc51e204f401b4d5d4f84825a68f373c74 2023-10-30T12:20:15.553 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/eacf00cc51e204f401b4d5d4f84825a68f373c74/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-geacf00cc51e2/ 2023-10-30T12:20:15.554 DEBUG:teuthology.orchestra.run.smithi012:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-rc5-geacf00cc51e2_6.6.0-rc5-geacf00cc51e2-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/eacf00cc51e204f401b4d5d4f84825a68f373c74/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-geacf00cc51e2/ --input-file=- 2023-10-30T12:20:15.557 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi012.front.sepia.ceph.com to complete... 2023-10-30T12:20:17.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30 12:20:16 URL:https://4.chacra.ceph.com/r/kernel/testing/eacf00cc51e204f401b4d5d4f84825a68f373c74/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-geacf00cc51e2/linux-image-6.6.0-rc5-geacf00cc51e2_6.6.0-rc5-geacf00cc51e2-1_amd64.deb [38909658/38909658] -> "/tmp/linux-image.deb" [1] 2023-10-30T12:20:17.075 INFO:teuthology.orchestra.run.smithi012.stderr:FINISHED --2023-10-30 12:20:16-- 2023-10-30T12:20:17.076 INFO:teuthology.orchestra.run.smithi012.stderr:Total wall clock time: 1.5s 2023-10-30T12:20:17.076 INFO:teuthology.orchestra.run.smithi012.stderr:Downloaded: 1 files, 37M in 1.4s (27.2 MB/s) 2023-10-30T12:20:17.078 INFO:teuthology.task.kernel:Installing kernel eacf00cc51e204f401b4d5d4f84825a68f373c74 on client.0... 2023-10-30T12:20:17.079 DEBUG:teuthology.orchestra.run.smithi012:> sudo dpkg -i /tmp/linux-image.deb 2023-10-30T12:20:17.362 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-image-6.6.0-rc5-geacf00cc51e2. 2023-10-30T12:20:19.447 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-10-30T12:20:19.451 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-10-30T12:20:19.613 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-6.6.0-rc5-geacf00cc51e2 (6.6.0-rc5-geacf00cc51e2-1) ... 2023-10-30T12:20:23.082 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-6.6.0-rc5-geacf00cc51e2 (6.6.0-rc5-geacf00cc51e2-1) ... 2023-10-30T12:20:23.170 INFO:teuthology.orchestra.run.smithi012.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:20:43.760 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub' 2023-10-30T12:20:43.772 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-30T12:20:43.798 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2023-10-30T12:20:44.133 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:20:44.162 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:20:44.462 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-30T12:20:44.475 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-30T12:20:44.544 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-30T12:20:44.597 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-30T12:20:44.704 INFO:teuthology.orchestra.run.smithi012.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-30T12:20:44.704 INFO:teuthology.orchestra.run.smithi012.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-30T12:20:44.705 INFO:teuthology.orchestra.run.smithi012.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-30T12:20:44.752 INFO:teuthology.orchestra.run.smithi012.stderr:done 2023-10-30T12:20:44.944 DEBUG:teuthology.orchestra.run.smithi012:> dpkg-deb -c /tmp/linux-image.deb 2023-10-30T12:20:44.993 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./ 2023-10-30T12:20:44.993 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./boot/ 2023-10-30T12:20:44.993 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 4415097 2023-10-30 03:46 ./boot/System.map-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:20:45.006 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 169233 2023-10-30 03:46 ./boot/config-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:20:45.006 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11352928 2023-10-30 03:46 ./boot/vmlinuz-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:20:45.026 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./etc/ 2023-10-30T12:20:45.026 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./etc/kernel/ 2023-10-30T12:20:45.026 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./etc/kernel/postinst.d/ 2023-10-30T12:20:45.026 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./etc/kernel/postrm.d/ 2023-10-30T12:20:45.027 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./etc/kernel/preinst.d/ 2023-10-30T12:20:45.027 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./etc/kernel/prerm.d/ 2023-10-30T12:20:45.027 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/ 2023-10-30T12:20:45.027 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/ 2023-10-30T12:20:45.028 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/ 2023-10-30T12:20:45.028 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/ 2023-10-30T12:20:45.028 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/ 2023-10-30T12:20:45.028 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/ 2023-10-30T12:20:45.029 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/ 2023-10-30T12:20:45.029 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 490233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/aesni-intel.ko 2023-10-30T12:20:45.031 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-30T12:20:45.031 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-30T12:20:45.031 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-30T12:20:45.032 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 93113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-30T12:20:45.032 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 106729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-30T12:20:45.032 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 152241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-30T12:20:45.032 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-30T12:20:45.032 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-30T12:20:45.033 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 83889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-30T12:20:45.033 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-30T12:20:45.033 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-30T12:20:45.033 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-30T12:20:45.034 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-30T12:20:45.034 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-30T12:20:45.034 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-30T12:20:45.034 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 58585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-30T12:20:45.034 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-30T12:20:45.035 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-30T12:20:45.035 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-30T12:20:45.035 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kernel/ 2023-10-30T12:20:45.035 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kernel/cpu/ 2023-10-30T12:20:45.036 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kernel/cpu/mce/ 2023-10-30T12:20:45.036 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-30T12:20:45.036 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kernel/cpuid.ko 2023-10-30T12:20:45.036 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kernel/msr.ko 2023-10-30T12:20:45.036 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kvm/ 2023-10-30T12:20:45.037 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 353761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-30T12:20:45.037 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 881713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-30T12:20:45.037 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 2276817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kvm/kvm.ko 2023-10-30T12:20:45.048 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/ 2023-10-30T12:20:45.048 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/af_alg.ko 2023-10-30T12:20:45.048 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/algif_hash.ko 2023-10-30T12:20:45.049 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/algif_skcipher.ko 2023-10-30T12:20:45.049 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/ansi_cprng.ko 2023-10-30T12:20:45.049 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/anubis.ko 2023-10-30T12:20:45.049 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/arc4.ko 2023-10-30T12:20:45.049 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/asymmetric_keys/ 2023-10-30T12:20:45.050 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-10-30T12:20:45.050 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/async_tx/ 2023-10-30T12:20:45.050 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/async_tx/async_memcpy.ko 2023-10-30T12:20:45.050 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/async_tx/async_pq.ko 2023-10-30T12:20:45.051 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-30T12:20:45.051 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/async_tx/async_tx.ko 2023-10-30T12:20:45.051 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/async_tx/async_xor.ko 2023-10-30T12:20:45.051 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/async_tx/raid6test.ko 2023-10-30T12:20:45.051 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/authenc.ko 2023-10-30T12:20:45.052 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/authencesn.ko 2023-10-30T12:20:45.052 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/blake2b_generic.ko 2023-10-30T12:20:45.052 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/blowfish_common.ko 2023-10-30T12:20:45.052 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/blowfish_generic.ko 2023-10-30T12:20:45.052 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/camellia_generic.ko 2023-10-30T12:20:45.053 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/cast5_generic.ko 2023-10-30T12:20:45.053 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/cast6_generic.ko 2023-10-30T12:20:45.053 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/cast_common.ko 2023-10-30T12:20:45.053 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/ccm.ko 2023-10-30T12:20:45.053 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/cmac.ko 2023-10-30T12:20:45.054 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/crc32_generic.ko 2023-10-30T12:20:45.054 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/cryptd.ko 2023-10-30T12:20:45.054 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/crypto_engine.ko 2023-10-30T12:20:45.054 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/crypto_simd.ko 2023-10-30T12:20:45.054 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/crypto_user.ko 2023-10-30T12:20:45.055 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/cts.ko 2023-10-30T12:20:45.055 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/des_generic.ko 2023-10-30T12:20:45.055 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/dh_generic.ko 2023-10-30T12:20:45.055 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/echainiv.ko 2023-10-30T12:20:45.055 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/essiv.ko 2023-10-30T12:20:45.056 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/fcrypt.ko 2023-10-30T12:20:45.056 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/khazad.ko 2023-10-30T12:20:45.056 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/lrw.ko 2023-10-30T12:20:45.056 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/lz4.ko 2023-10-30T12:20:45.056 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/lz4hc.ko 2023-10-30T12:20:45.057 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/md4.ko 2023-10-30T12:20:45.057 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/michael_mic.ko 2023-10-30T12:20:45.057 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/pcbc.ko 2023-10-30T12:20:45.057 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/pcrypt.ko 2023-10-30T12:20:45.057 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/rmd160.ko 2023-10-30T12:20:45.058 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/seed.ko 2023-10-30T12:20:45.058 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/serpent_generic.ko 2023-10-30T12:20:45.058 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 149193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/tcrypt.ko 2023-10-30T12:20:45.058 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/tea.ko 2023-10-30T12:20:45.058 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/twofish_common.ko 2023-10-30T12:20:45.059 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/twofish_generic.ko 2023-10-30T12:20:45.059 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/vmac.ko 2023-10-30T12:20:45.059 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/wp512.ko 2023-10-30T12:20:45.059 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/xcbc.ko 2023-10-30T12:20:45.059 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/xor.ko 2023-10-30T12:20:45.059 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/xts.ko 2023-10-30T12:20:45.060 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/xxhash_generic.ko 2023-10-30T12:20:45.060 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ 2023-10-30T12:20:45.060 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/ 2023-10-30T12:20:45.060 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/acpi_extlog.ko 2023-10-30T12:20:45.060 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-30T12:20:45.061 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/acpi_pad.ko 2023-10-30T12:20:45.061 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/apei/ 2023-10-30T12:20:45.061 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/apei/einj.ko 2023-10-30T12:20:45.061 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/ec_sys.ko 2023-10-30T12:20:45.061 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/platform_profile.ko 2023-10-30T12:20:45.062 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/sbs.ko 2023-10-30T12:20:45.062 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/sbshc.ko 2023-10-30T12:20:45.062 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 103025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/video.ko 2023-10-30T12:20:45.062 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/ 2023-10-30T12:20:45.062 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/acard-ahci.ko 2023-10-30T12:20:45.063 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 88865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/ahci.ko 2023-10-30T12:20:45.063 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/ahci_platform.ko 2023-10-30T12:20:45.063 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 88289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/libahci.ko 2023-10-30T12:20:45.063 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/libahci_platform.ko 2023-10-30T12:20:45.064 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_acpi.ko 2023-10-30T12:20:45.064 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_ali.ko 2023-10-30T12:20:45.064 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_amd.ko 2023-10-30T12:20:45.064 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_artop.ko 2023-10-30T12:20:45.064 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_atiixp.ko 2023-10-30T12:20:45.065 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_atp867x.ko 2023-10-30T12:20:45.065 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_cmd640.ko 2023-10-30T12:20:45.065 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_cmd64x.ko 2023-10-30T12:20:45.065 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_cypress.ko 2023-10-30T12:20:45.065 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_efar.ko 2023-10-30T12:20:45.066 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_hpt366.ko 2023-10-30T12:20:45.066 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_hpt37x.ko 2023-10-30T12:20:45.066 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-30T12:20:45.066 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-30T12:20:45.066 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_it8213.ko 2023-10-30T12:20:45.066 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_it821x.ko 2023-10-30T12:20:45.067 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_jmicron.ko 2023-10-30T12:20:45.067 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_legacy.ko 2023-10-30T12:20:45.067 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_marvell.ko 2023-10-30T12:20:45.067 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_mpiix.ko 2023-10-30T12:20:45.067 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_netcell.ko 2023-10-30T12:20:45.068 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_ninja32.ko 2023-10-30T12:20:45.068 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_ns87410.ko 2023-10-30T12:20:45.068 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_ns87415.ko 2023-10-30T12:20:45.068 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_oldpiix.ko 2023-10-30T12:20:45.068 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_opti.ko 2023-10-30T12:20:45.069 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_optidma.ko 2023-10-30T12:20:45.069 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_pcmcia.ko 2023-10-30T12:20:45.069 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-30T12:20:45.069 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-30T12:20:45.069 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_piccolo.ko 2023-10-30T12:20:45.070 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_radisys.ko 2023-10-30T12:20:45.070 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_rdc.ko 2023-10-30T12:20:45.070 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_rz1000.ko 2023-10-30T12:20:45.070 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_sch.ko 2023-10-30T12:20:45.070 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_serverworks.ko 2023-10-30T12:20:45.071 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_sil680.ko 2023-10-30T12:20:45.071 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_sl82c105.ko 2023-10-30T12:20:45.071 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_triflex.ko 2023-10-30T12:20:45.071 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_via.ko 2023-10-30T12:20:45.071 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pdc_adma.ko 2023-10-30T12:20:45.071 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_inic162x.ko 2023-10-30T12:20:45.072 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 83345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_mv.ko 2023-10-30T12:20:45.072 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_nv.ko 2023-10-30T12:20:45.072 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_promise.ko 2023-10-30T12:20:45.072 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_qstor.ko 2023-10-30T12:20:45.072 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_sil.ko 2023-10-30T12:20:45.073 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_sil24.ko 2023-10-30T12:20:45.073 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_sis.ko 2023-10-30T12:20:45.073 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_svw.ko 2023-10-30T12:20:45.073 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_sx4.ko 2023-10-30T12:20:45.073 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_uli.ko 2023-10-30T12:20:45.074 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_via.ko 2023-10-30T12:20:45.074 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_vsc.ko 2023-10-30T12:20:45.074 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/auxdisplay/ 2023-10-30T12:20:45.074 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/auxdisplay/ks0108.ko 2023-10-30T12:20:45.074 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/bcma/ 2023-10-30T12:20:45.075 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 124473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/bcma/bcma.ko 2023-10-30T12:20:45.075 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/ 2023-10-30T12:20:45.075 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/aoe/ 2023-10-30T12:20:45.075 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 113665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/aoe/aoe.ko 2023-10-30T12:20:45.075 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/drbd/ 2023-10-30T12:20:45.076 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 965585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/drbd/drbd.ko 2023-10-30T12:20:45.076 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 173225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/floppy.ko 2023-10-30T12:20:45.076 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/mtip32xx/ 2023-10-30T12:20:45.076 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 94297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-30T12:20:45.076 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 115793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/nbd.ko 2023-10-30T12:20:45.077 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/null_blk/ 2023-10-30T12:20:45.077 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 110449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/null_blk/null_blk.ko 2023-10-30T12:20:45.077 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 98193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/pktcdvd.ko 2023-10-30T12:20:45.077 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 263769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/rbd.ko 2023-10-30T12:20:45.077 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/xen-blkback/ 2023-10-30T12:20:45.077 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 98145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-30T12:20:45.131 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/zram/ 2023-10-30T12:20:45.132 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/zram/zram.ko 2023-10-30T12:20:45.132 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ 2023-10-30T12:20:45.132 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30705 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/applicom.ko 2023-10-30T12:20:45.132 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/hangcheck-timer.ko 2023-10-30T12:20:45.133 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/hw_random/ 2023-10-30T12:20:45.133 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-30T12:20:45.133 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-30T12:20:45.133 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-30T12:20:45.134 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/hw_random/via-rng.ko 2023-10-30T12:20:45.134 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-30T12:20:45.134 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ipmi/ 2023-10-30T12:20:45.134 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-30T12:20:45.135 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 148601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-30T12:20:45.135 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-30T12:20:45.135 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 139313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-30T12:20:45.135 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-30T12:20:45.135 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-30T12:20:45.136 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/lp.ko 2023-10-30T12:20:45.136 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/mwave/ 2023-10-30T12:20:45.136 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 84097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/mwave/mwave.ko 2023-10-30T12:20:45.136 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/nvram.ko 2023-10-30T12:20:45.137 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ppdev.ko 2023-10-30T12:20:45.137 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tlclk.ko 2023-10-30T12:20:45.137 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/ 2023-10-30T12:20:45.137 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-30T12:20:45.137 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-10-30T12:20:45.138 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-10-30T12:20:45.138 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-10-30T12:20:45.138 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-30T12:20:45.138 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-30T12:20:45.138 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-10-30T12:20:45.139 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/xillybus/ 2023-10-30T12:20:45.139 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/xillybus/xillybus_class.ko 2023-10-30T12:20:45.139 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/xillybus/xillybus_core.ko 2023-10-30T12:20:45.139 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-10-30T12:20:45.140 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/clk/ 2023-10-30T12:20:45.140 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/clk/clk-palmas.ko 2023-10-30T12:20:45.140 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/clk/clk-si5351.ko 2023-10-30T12:20:45.140 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/clk/clk-twl6040.ko 2023-10-30T12:20:45.140 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/clk/clk-wm831x.ko 2023-10-30T12:20:45.141 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/cpufreq/ 2023-10-30T12:20:45.141 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-30T12:20:45.141 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-30T12:20:45.141 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-30T12:20:45.142 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/ 2023-10-30T12:20:45.142 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/ccp/ 2023-10-30T12:20:45.142 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-30T12:20:45.142 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 243025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/ccp/ccp.ko 2023-10-30T12:20:45.142 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/intel/ 2023-10-30T12:20:45.143 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/intel/qat/ 2023-10-30T12:20:45.143 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/intel/qat/qat_common/ 2023-10-30T12:20:45.143 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 438761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-30T12:20:45.143 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-10-30T12:20:45.143 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-30T12:20:45.144 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/padlock-aes.ko 2023-10-30T12:20:45.144 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/padlock-sha.ko 2023-10-30T12:20:45.144 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/virtio/ 2023-10-30T12:20:45.144 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 68857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-30T12:20:45.144 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/dca/ 2023-10-30T12:20:45.145 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/dca/dca.ko 2023-10-30T12:20:45.145 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/dma/ 2023-10-30T12:20:45.145 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/dma/dw/ 2023-10-30T12:20:45.145 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-30T12:20:45.146 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/dma/ioat/ 2023-10-30T12:20:45.146 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 122569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-30T12:20:45.146 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/ 2023-10-30T12:20:45.146 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 79953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/amd64_edac.ko 2023-10-30T12:20:45.146 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/e752x_edac.ko 2023-10-30T12:20:45.147 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 63617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/edac_mce_amd.ko 2023-10-30T12:20:45.147 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/i3000_edac.ko 2023-10-30T12:20:45.147 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/i3200_edac.ko 2023-10-30T12:20:45.147 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/i5100_edac.ko 2023-10-30T12:20:45.147 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/i5400_edac.ko 2023-10-30T12:20:45.148 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/i7300_edac.ko 2023-10-30T12:20:45.148 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 52041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/i7core_edac.ko 2023-10-30T12:20:45.148 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/i82975x_edac.ko 2023-10-30T12:20:45.148 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/ie31200_edac.ko 2023-10-30T12:20:45.148 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 56825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/sb_edac.ko 2023-10-30T12:20:45.148 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/x38_edac.ko 2023-10-30T12:20:45.149 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/ 2023-10-30T12:20:45.149 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/extcon-gpio.ko 2023-10-30T12:20:45.149 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/extcon-max14577.ko 2023-10-30T12:20:45.149 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/extcon-max77693.ko 2023-10-30T12:20:45.149 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/extcon-max8997.ko 2023-10-30T12:20:45.150 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/extcon-palmas.ko 2023-10-30T12:20:45.150 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/extcon-rt8973a.ko 2023-10-30T12:20:45.150 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/extcon-sm5502.ko 2023-10-30T12:20:45.150 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/firmware/ 2023-10-30T12:20:45.150 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/firmware/dmi-sysfs.ko 2023-10-30T12:20:45.151 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/firmware/efi/ 2023-10-30T12:20:45.151 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/firmware/efi/efi-pstore.ko 2023-10-30T12:20:45.151 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-30T12:20:45.151 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/ 2023-10-30T12:20:45.151 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-adp5520.ko 2023-10-30T12:20:45.152 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-amd8111.ko 2023-10-30T12:20:45.152 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-arizona.ko 2023-10-30T12:20:45.152 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-da9052.ko 2023-10-30T12:20:45.152 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-da9055.ko 2023-10-30T12:20:45.152 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-dln2.ko 2023-10-30T12:20:45.153 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 77849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-f7188x.ko 2023-10-30T12:20:45.153 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-generic.ko 2023-10-30T12:20:45.153 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-ich.ko 2023-10-30T12:20:45.153 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-10-30T12:20:45.153 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-kempld.ko 2023-10-30T12:20:45.154 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-lp3943.ko 2023-10-30T12:20:45.154 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-max7300.ko 2023-10-30T12:20:45.154 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-max730x.ko 2023-10-30T12:20:45.154 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-max732x.ko 2023-10-30T12:20:45.155 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-10-30T12:20:45.155 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-pca953x.ko 2023-10-30T12:20:45.155 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-pcf857x.ko 2023-10-30T12:20:45.155 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-rdc321x.ko 2023-10-30T12:20:45.155 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-sch.ko 2023-10-30T12:20:45.156 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-sch311x.ko 2023-10-30T12:20:45.156 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-tps65912.ko 2023-10-30T12:20:45.156 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-twl4030.ko 2023-10-30T12:20:45.156 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-twl6040.ko 2023-10-30T12:20:45.156 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-30T12:20:45.157 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-vx855.ko 2023-10-30T12:20:45.157 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-wm831x.ko 2023-10-30T12:20:45.157 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-wm8350.ko 2023-10-30T12:20:45.157 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-wm8994.ko 2023-10-30T12:20:45.157 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/ 2023-10-30T12:20:45.157 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-a4tech.ko 2023-10-30T12:20:45.158 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-apple.ko 2023-10-30T12:20:45.158 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-appleir.ko 2023-10-30T12:20:45.158 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-aureal.ko 2023-10-30T12:20:45.158 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-axff.ko 2023-10-30T12:20:45.159 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-belkin.ko 2023-10-30T12:20:45.159 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-cherry.ko 2023-10-30T12:20:45.159 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-chicony.ko 2023-10-30T12:20:45.159 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-cp2112.ko 2023-10-30T12:20:45.159 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-cypress.ko 2023-10-30T12:20:45.159 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-dr.ko 2023-10-30T12:20:45.160 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-elecom.ko 2023-10-30T12:20:45.160 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-elo.ko 2023-10-30T12:20:45.160 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-emsff.ko 2023-10-30T12:20:45.160 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-ezkey.ko 2023-10-30T12:20:45.160 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-gaff.ko 2023-10-30T12:20:45.161 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-generic.ko 2023-10-30T12:20:45.161 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-gt683r.ko 2023-10-30T12:20:45.161 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-gyration.ko 2023-10-30T12:20:45.161 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-30T12:20:45.161 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-30T12:20:45.162 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-holtekff.ko 2023-10-30T12:20:45.162 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-hyperv.ko 2023-10-30T12:20:45.162 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-icade.ko 2023-10-30T12:20:45.162 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-kensington.ko 2023-10-30T12:20:45.162 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-keytouch.ko 2023-10-30T12:20:45.163 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-kye.ko 2023-10-30T12:20:45.163 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-lcpower.ko 2023-10-30T12:20:45.163 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-led.ko 2023-10-30T12:20:45.163 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-lenovo.ko 2023-10-30T12:20:45.163 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-lg-g15.ko 2023-10-30T12:20:45.163 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 58185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-30T12:20:45.164 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 101369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-30T12:20:45.164 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 80993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-logitech.ko 2023-10-30T12:20:45.164 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-magicmouse.ko 2023-10-30T12:20:45.164 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-microsoft.ko 2023-10-30T12:20:45.164 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-monterey.ko 2023-10-30T12:20:45.165 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-multitouch.ko 2023-10-30T12:20:45.165 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-ntrig.ko 2023-10-30T12:20:45.165 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-ortek.ko 2023-10-30T12:20:45.165 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-penmount.ko 2023-10-30T12:20:45.165 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-petalynx.ko 2023-10-30T12:20:45.166 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 72937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-picolcd.ko 2023-10-30T12:20:45.166 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-pl.ko 2023-10-30T12:20:45.166 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-plantronics.ko 2023-10-30T12:20:45.166 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-primax.ko 2023-10-30T12:20:45.166 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-rmi.ko 2023-10-30T12:20:45.166 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-30T12:20:45.167 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-common.ko 2023-10-30T12:20:45.167 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-30T12:20:45.167 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-30T12:20:45.167 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-30T12:20:45.167 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-30T12:20:45.168 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-30T12:20:45.168 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-30T12:20:45.168 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-30T12:20:45.168 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-30T12:20:45.168 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-30T12:20:45.169 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat.ko 2023-10-30T12:20:45.169 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-saitek.ko 2023-10-30T12:20:45.169 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-samsung.ko 2023-10-30T12:20:45.169 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-30T12:20:45.169 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-sjoy.ko 2023-10-30T12:20:45.170 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-sony.ko 2023-10-30T12:20:45.170 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-speedlink.ko 2023-10-30T12:20:45.170 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-steelseries.ko 2023-10-30T12:20:45.170 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-sunplus.ko 2023-10-30T12:20:45.170 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-30T12:20:45.171 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-tivo.ko 2023-10-30T12:20:45.171 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-tmff.ko 2023-10-30T12:20:45.171 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-topseed.ko 2023-10-30T12:20:45.171 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-twinhan.ko 2023-10-30T12:20:45.171 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 71073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-uclogic.ko 2023-10-30T12:20:45.171 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-waltop.ko 2023-10-30T12:20:45.172 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 143473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-wiimote.ko 2023-10-30T12:20:45.172 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-xinmo.ko 2023-10-30T12:20:45.172 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-zpff.ko 2023-10-30T12:20:45.172 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-zydacron.ko 2023-10-30T12:20:45.173 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 267825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid.ko 2023-10-30T12:20:45.173 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/uhid.ko 2023-10-30T12:20:45.173 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/usbhid/ 2023-10-30T12:20:45.173 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 125721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/usbhid/usbhid.ko 2023-10-30T12:20:45.173 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/usbhid/usbkbd.ko 2023-10-30T12:20:45.174 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/usbhid/usbmouse.ko 2023-10-30T12:20:45.174 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 217873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/wacom.ko 2023-10-30T12:20:45.174 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hv/ 2023-10-30T12:20:45.174 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 64201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hv/hv_balloon.ko 2023-10-30T12:20:45.175 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hv/hv_utils.ko 2023-10-30T12:20:45.175 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 305465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hv/hv_vmbus.ko 2023-10-30T12:20:45.175 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ 2023-10-30T12:20:45.175 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/abituguru.ko 2023-10-30T12:20:45.175 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 61393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/abituguru3.ko 2023-10-30T12:20:45.176 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-30T12:20:45.176 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ad7414.ko 2023-10-30T12:20:45.176 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ad7418.ko 2023-10-30T12:20:45.176 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adc128d818.ko 2023-10-30T12:20:45.176 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adm1025.ko 2023-10-30T12:20:45.177 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 97977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adm1026.ko 2023-10-30T12:20:45.177 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adm1029.ko 2023-10-30T12:20:45.177 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 45553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adm1031.ko 2023-10-30T12:20:45.177 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adm9240.ko 2023-10-30T12:20:45.177 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ads7828.ko 2023-10-30T12:20:45.177 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adt7410.ko 2023-10-30T12:20:45.178 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adt7411.ko 2023-10-30T12:20:45.178 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 74433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adt7462.ko 2023-10-30T12:20:45.178 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adt7470.ko 2023-10-30T12:20:45.178 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adt7475.ko 2023-10-30T12:20:45.178 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adt7x10.ko 2023-10-30T12:20:45.179 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/amc6821.ko 2023-10-30T12:20:45.179 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 55265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/applesmc.ko 2023-10-30T12:20:45.179 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/asb100.ko 2023-10-30T12:20:45.179 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 58617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/asc7621.ko 2023-10-30T12:20:45.179 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-30T12:20:45.180 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/atxp1.ko 2023-10-30T12:20:45.180 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/coretemp.ko 2023-10-30T12:20:45.180 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/da9052-hwmon.ko 2023-10-30T12:20:45.180 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/da9055-hwmon.ko 2023-10-30T12:20:45.180 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-30T12:20:45.181 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 99601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/dme1737.ko 2023-10-30T12:20:45.181 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ds1621.ko 2023-10-30T12:20:45.181 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ds620.ko 2023-10-30T12:20:45.181 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/emc1403.ko 2023-10-30T12:20:45.181 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/emc2103.ko 2023-10-30T12:20:45.182 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/emc6w201.ko 2023-10-30T12:20:45.182 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/f71805f.ko 2023-10-30T12:20:45.182 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 94457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/f71882fg.ko 2023-10-30T12:20:45.182 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/f75375s.ko 2023-10-30T12:20:45.182 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/fam15h_power.ko 2023-10-30T12:20:45.183 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 55449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/fschmd.ko 2023-10-30T12:20:45.183 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/g760a.ko 2023-10-30T12:20:45.183 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/g762.ko 2023-10-30T12:20:45.183 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/gl518sm.ko 2023-10-30T12:20:45.183 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 45529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/gl520sm.ko 2023-10-30T12:20:45.183 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/hih6130.ko 2023-10-30T12:20:45.184 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-30T12:20:45.184 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/i5500_temp.ko 2023-10-30T12:20:45.184 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/i5k_amb.ko 2023-10-30T12:20:45.184 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ibmaem.ko 2023-10-30T12:20:45.184 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ibmpex.ko 2023-10-30T12:20:45.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ina209.ko 2023-10-30T12:20:45.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ina2xx.ko 2023-10-30T12:20:45.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 130449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/it87.ko 2023-10-30T12:20:45.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/jc42.ko 2023-10-30T12:20:45.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/k10temp.ko 2023-10-30T12:20:45.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/k8temp.ko 2023-10-30T12:20:45.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lineage-pem.ko 2023-10-30T12:20:45.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 48601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm63.ko 2023-10-30T12:20:45.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm73.ko 2023-10-30T12:20:45.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm75.ko 2023-10-30T12:20:45.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm77.ko 2023-10-30T12:20:45.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm78.ko 2023-10-30T12:20:45.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm80.ko 2023-10-30T12:20:45.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm83.ko 2023-10-30T12:20:45.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 72833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm85.ko 2023-10-30T12:20:45.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm87.ko 2023-10-30T12:20:45.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm90.ko 2023-10-30T12:20:45.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm92.ko 2023-10-30T12:20:45.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 108841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm93.ko 2023-10-30T12:20:45.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm95234.ko 2023-10-30T12:20:45.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm95241.ko 2023-10-30T12:20:45.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm95245.ko 2023-10-30T12:20:45.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ltc2945.ko 2023-10-30T12:20:45.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ltc4151.ko 2023-10-30T12:20:45.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ltc4215.ko 2023-10-30T12:20:45.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ltc4222.ko 2023-10-30T12:20:45.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ltc4245.ko 2023-10-30T12:20:45.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ltc4260.ko 2023-10-30T12:20:45.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ltc4261.ko 2023-10-30T12:20:45.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/max16065.ko 2023-10-30T12:20:45.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/max1619.ko 2023-10-30T12:20:45.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/max1668.ko 2023-10-30T12:20:45.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/max197.ko 2023-10-30T12:20:45.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/max6639.ko 2023-10-30T12:20:45.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/max6650.ko 2023-10-30T12:20:45.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/max6697.ko 2023-10-30T12:20:45.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/mc13783-adc.ko 2023-10-30T12:20:45.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/mcp3021.ko 2023-10-30T12:20:45.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-10-30T12:20:45.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/nct6683.ko 2023-10-30T12:20:45.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 119817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/nct6775-core.ko 2023-10-30T12:20:45.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60705 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/nct6775.ko 2023-10-30T12:20:45.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 59737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/nct7802.ko 2023-10-30T12:20:45.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 82337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pc87360.ko 2023-10-30T12:20:45.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pc87427.ko 2023-10-30T12:20:45.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pcf8591.ko 2023-10-30T12:20:45.194 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/ 2023-10-30T12:20:45.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-30T12:20:45.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-30T12:20:45.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-30T12:20:45.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-30T12:20:45.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-30T12:20:45.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-30T12:20:45.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-30T12:20:45.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 82577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-30T12:20:45.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-30T12:20:45.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-30T12:20:45.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-30T12:20:45.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-30T12:20:45.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/powr1220.ko 2023-10-30T12:20:45.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18705 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/sch5627.ko 2023-10-30T12:20:45.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/sch5636.ko 2023-10-30T12:20:45.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-30T12:20:45.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/sht15.ko 2023-10-30T12:20:45.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/sht21.ko 2023-10-30T12:20:45.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/shtc1.ko 2023-10-30T12:20:45.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/sis5595.ko 2023-10-30T12:20:45.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/smsc47b397.ko 2023-10-30T12:20:45.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/smsc47m1.ko 2023-10-30T12:20:45.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/smsc47m192.ko 2023-10-30T12:20:45.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/thmc50.ko 2023-10-30T12:20:45.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/tmp102.ko 2023-10-30T12:20:45.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/tmp103.ko 2023-10-30T12:20:45.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/tmp401.ko 2023-10-30T12:20:45.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/tmp421.ko 2023-10-30T12:20:45.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/via-cputemp.ko 2023-10-30T12:20:45.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/via686a.ko 2023-10-30T12:20:45.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 52457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/vt1211.ko 2023-10-30T12:20:45.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/vt8231.ko 2023-10-30T12:20:45.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83627ehf.ko 2023-10-30T12:20:45.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83627hf.ko 2023-10-30T12:20:45.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 71345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83781d.ko 2023-10-30T12:20:45.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 63289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83791d.ko 2023-10-30T12:20:45.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 71321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83792d.ko 2023-10-30T12:20:45.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 101473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83793.ko 2023-10-30T12:20:45.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 119505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83795.ko 2023-10-30T12:20:45.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83l785ts.ko 2023-10-30T12:20:45.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83l786ng.ko 2023-10-30T12:20:45.202 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-10-30T12:20:45.202 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-10-30T12:20:45.202 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/ 2023-10-30T12:20:45.202 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/algos/ 2023-10-30T12:20:45.202 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-30T12:20:45.203 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-30T12:20:45.203 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/ 2023-10-30T12:20:45.203 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-10-30T12:20:45.203 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-10-30T12:20:45.203 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-10-30T12:20:45.204 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-30T12:20:45.204 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-30T12:20:45.204 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-30T12:20:45.204 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-10-30T12:20:45.204 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 6929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-30T12:20:45.205 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-10-30T12:20:45.205 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-30T12:20:45.205 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-30T12:20:45.205 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-30T12:20:45.205 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-30T12:20:45.206 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-10-30T12:20:45.206 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-10-30T12:20:45.206 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 62817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-30T12:20:45.206 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-30T12:20:45.206 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-30T12:20:45.207 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-10-30T12:20:45.207 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-30T12:20:45.207 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-30T12:20:45.207 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-10-30T12:20:45.207 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-30T12:20:45.208 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-30T12:20:45.208 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-30T12:20:45.208 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-10-30T12:20:45.208 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-30T12:20:45.208 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-30T12:20:45.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-10-30T12:20:45.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-10-30T12:20:45.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-30T12:20:45.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-10-30T12:20:45.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-30T12:20:45.210 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-30T12:20:45.210 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-30T12:20:45.210 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-30T12:20:45.210 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-10-30T12:20:45.210 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/i2c-mux.ko 2023-10-30T12:20:45.210 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/i2c-smbus.ko 2023-10-30T12:20:45.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/i2c-stub.ko 2023-10-30T12:20:45.211 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/muxes/ 2023-10-30T12:20:45.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-10-30T12:20:45.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-10-30T12:20:45.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-10-30T12:20:45.212 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ 2023-10-30T12:20:45.212 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/ 2023-10-30T12:20:45.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 273609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-30T12:20:45.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 879969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/ib_core.ko 2023-10-30T12:20:45.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 69337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-30T12:20:45.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 324217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-30T12:20:45.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 97985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-30T12:20:45.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 282273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-30T12:20:45.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-30T12:20:45.213 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/ 2023-10-30T12:20:45.214 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/cxgb4/ 2023-10-30T12:20:45.214 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 493073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-30T12:20:45.214 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/mlx4/ 2023-10-30T12:20:45.214 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 419737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-30T12:20:45.214 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/mlx5/ 2023-10-30T12:20:45.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 739025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-30T12:20:45.215 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/mthca/ 2023-10-30T12:20:45.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 296625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-30T12:20:45.215 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/ocrdma/ 2023-10-30T12:20:45.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 153697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-10-30T12:20:45.215 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/usnic/ 2023-10-30T12:20:45.216 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 168969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-10-30T12:20:45.216 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/ 2023-10-30T12:20:45.216 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/ipoib/ 2023-10-30T12:20:45.216 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 263081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-30T12:20:45.216 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/iser/ 2023-10-30T12:20:45.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 100241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-30T12:20:45.217 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/isert/ 2023-10-30T12:20:45.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 111425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-30T12:20:45.217 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/srp/ 2023-10-30T12:20:45.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 128753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-30T12:20:45.218 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/srpt/ 2023-10-30T12:20:45.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 129529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-30T12:20:45.218 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/ 2023-10-30T12:20:45.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/evbug.ko 2023-10-30T12:20:45.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/ff-memless.ko 2023-10-30T12:20:45.219 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/gameport/ 2023-10-30T12:20:45.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-30T12:20:45.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-30T12:20:45.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/gameport/gameport.ko 2023-10-30T12:20:45.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/joydev.ko 2023-10-30T12:20:45.220 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/ 2023-10-30T12:20:45.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-10-30T12:20:45.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-10-30T12:20:45.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-10-30T12:20:45.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-10-30T12:20:45.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-30T12:20:45.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-10-30T12:20:45.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/lkkbd.ko 2023-10-30T12:20:45.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/lm8323.ko 2023-10-30T12:20:45.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/lm8333.ko 2023-10-30T12:20:45.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-10-30T12:20:45.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-10-30T12:20:45.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-10-30T12:20:45.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-10-30T12:20:45.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/newtonkbd.ko 2023-10-30T12:20:45.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-10-30T12:20:45.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/qt1070.ko 2023-10-30T12:20:45.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/qt2160.ko 2023-10-30T12:20:45.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-10-30T12:20:45.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/stowaway.ko 2023-10-30T12:20:45.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/sunkbd.ko 2023-10-30T12:20:45.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-10-30T12:20:45.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-10-30T12:20:45.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-10-30T12:20:45.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/xtkbd.ko 2023-10-30T12:20:45.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/matrix-keymap.ko 2023-10-30T12:20:45.224 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/ 2023-10-30T12:20:45.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/appletouch.ko 2023-10-30T12:20:45.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/bcm5974.ko 2023-10-30T12:20:45.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 123681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/cyapatp.ko 2023-10-30T12:20:45.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-30T12:20:45.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/gpio_mouse.ko 2023-10-30T12:20:45.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 335977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/psmouse.ko 2023-10-30T12:20:45.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/sermouse.ko 2023-10-30T12:20:45.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-30T12:20:45.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-30T12:20:45.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-30T12:20:45.226 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/rmi4/ 2023-10-30T12:20:45.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 109873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-30T12:20:45.227 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/ 2023-10-30T12:20:45.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/altera_ps2.ko 2023-10-30T12:20:45.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/arc_ps2.ko 2023-10-30T12:20:45.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/ct82c710.ko 2023-10-30T12:20:45.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-30T12:20:45.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/parkbd.ko 2023-10-30T12:20:45.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/pcips2.ko 2023-10-30T12:20:45.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/ps2mult.ko 2023-10-30T12:20:45.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/serio_raw.ko 2023-10-30T12:20:45.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/serport.ko 2023-10-30T12:20:45.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/sparse-keymap.ko 2023-10-30T12:20:45.229 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/iommu/ 2023-10-30T12:20:45.229 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/iommu/amd/ 2023-10-30T12:20:45.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-30T12:20:45.230 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/ 2023-10-30T12:20:45.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-88pm860x.ko 2023-10-30T12:20:45.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-adp5520.ko 2023-10-30T12:20:45.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 59753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-bd2802.ko 2023-10-30T12:20:45.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-blinkm.ko 2023-10-30T12:20:45.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-da903x.ko 2023-10-30T12:20:45.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-da9052.ko 2023-10-30T12:20:45.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-gpio.ko 2023-10-30T12:20:45.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-lm3530.ko 2023-10-30T12:20:45.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-lm3533.ko 2023-10-30T12:20:45.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-lm355x.ko 2023-10-30T12:20:45.232 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-lm3642.ko 2023-10-30T12:20:45.232 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-lp3944.ko 2023-10-30T12:20:45.232 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-lp8788.ko 2023-10-30T12:20:45.232 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-max8997.ko 2023-10-30T12:20:45.232 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-mc13783.ko 2023-10-30T12:20:45.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-menf21bmc.ko 2023-10-30T12:20:45.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-pca9532.ko 2023-10-30T12:20:45.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-pca955x.ko 2023-10-30T12:20:45.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-pca963x.ko 2023-10-30T12:20:45.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-ss4200.ko 2023-10-30T12:20:45.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-tca6507.ko 2023-10-30T12:20:45.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-wm831x-status.ko 2023-10-30T12:20:45.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-wm8350.ko 2023-10-30T12:20:45.234 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ 2023-10-30T12:20:45.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-30T12:20:45.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-30T12:20:45.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-30T12:20:45.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-30T12:20:45.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-30T12:20:45.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-30T12:20:45.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-30T12:20:45.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-30T12:20:45.236 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/ 2023-10-30T12:20:45.236 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/bcache/ 2023-10-30T12:20:45.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 654969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/bcache/bcache.ko 2023-10-30T12:20:45.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-bio-prison.ko 2023-10-30T12:20:45.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-bufio.ko 2023-10-30T12:20:45.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-cache-smq.ko 2023-10-30T12:20:45.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 156009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-cache.ko 2023-10-30T12:20:45.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 106897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-crypt.ko 2023-10-30T12:20:45.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-delay.ko 2023-10-30T12:20:45.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-era.ko 2023-10-30T12:20:45.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-flakey.ko 2023-10-30T12:20:45.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-log-userspace.ko 2023-10-30T12:20:45.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-log.ko 2023-10-30T12:20:45.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-mirror.ko 2023-10-30T12:20:45.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-multipath.ko 2023-10-30T12:20:45.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-queue-length.ko 2023-10-30T12:20:45.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-raid.ko 2023-10-30T12:20:45.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-region-hash.ko 2023-10-30T12:20:45.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-round-robin.ko 2023-10-30T12:20:45.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-service-time.ko 2023-10-30T12:20:45.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 109593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-snapshot.ko 2023-10-30T12:20:45.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-switch.ko 2023-10-30T12:20:45.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 181865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-thin-pool.ko 2023-10-30T12:20:45.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-verity.ko 2023-10-30T12:20:45.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-zero.ko 2023-10-30T12:20:45.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/faulty.ko 2023-10-30T12:20:45.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/linear.ko 2023-10-30T12:20:45.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/multipath.ko 2023-10-30T12:20:45.241 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/persistent-data/ 2023-10-30T12:20:45.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 199937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-30T12:20:45.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/raid0.ko 2023-10-30T12:20:45.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 113073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/raid1.ko 2023-10-30T12:20:45.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 158681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/raid10.ko 2023-10-30T12:20:45.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 395721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/raid456.ko 2023-10-30T12:20:45.243 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/ 2023-10-30T12:20:45.243 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/ 2023-10-30T12:20:45.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 178137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/mptbase.ko 2023-10-30T12:20:45.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 63385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/mptctl.ko 2023-10-30T12:20:45.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/mptfc.ko 2023-10-30T12:20:45.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/mptlan.ko 2023-10-30T12:20:45.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 123321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/mptsas.ko 2023-10-30T12:20:45.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 79041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/mptscsih.ko 2023-10-30T12:20:45.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/mptspi.ko 2023-10-30T12:20:45.244 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/ 2023-10-30T12:20:45.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/88pm800.ko 2023-10-30T12:20:45.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/88pm805.ko 2023-10-30T12:20:45.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/88pm80x.ko 2023-10-30T12:20:45.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/arizona-i2c.ko 2023-10-30T12:20:45.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 123721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/arizona.ko 2023-10-30T12:20:45.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/bcm590xx.ko 2023-10-30T12:20:45.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/cros_ec_dev.ko 2023-10-30T12:20:45.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/dln2.ko 2023-10-30T12:20:45.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/janz-cmodio.ko 2023-10-30T12:20:45.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/kempld-core.ko 2023-10-30T12:20:45.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/lm3533-core.ko 2023-10-30T12:20:45.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-10-30T12:20:45.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/lp3943.ko 2023-10-30T12:20:45.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/lpc_ich.ko 2023-10-30T12:20:45.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/lpc_sch.ko 2023-10-30T12:20:45.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/max8907.ko 2023-10-30T12:20:45.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/mc13xxx-core.ko 2023-10-30T12:20:45.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-10-30T12:20:45.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/menf21bmc.ko 2023-10-30T12:20:45.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/pcf50633-adc.ko 2023-10-30T12:20:45.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/pcf50633-gpio.ko 2023-10-30T12:20:45.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/pcf50633.ko 2023-10-30T12:20:45.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-10-30T12:20:45.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/retu-mfd.ko 2023-10-30T12:20:45.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 80425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/si476x-core.ko 2023-10-30T12:20:45.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 48457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/sm501.ko 2023-10-30T12:20:45.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-10-30T12:20:45.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/tps65010.ko 2023-10-30T12:20:45.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/tps6507x.ko 2023-10-30T12:20:45.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/viperboard.ko 2023-10-30T12:20:45.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/vx855.ko 2023-10-30T12:20:45.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/wl1273-core.ko 2023-10-30T12:20:45.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/wm8994.ko 2023-10-30T12:20:45.251 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ 2023-10-30T12:20:45.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-10-30T12:20:45.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 48489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ad525x_dpot.ko 2023-10-30T12:20:45.252 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/altera-stapl/ 2023-10-30T12:20:45.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-30T12:20:45.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/apds9802als.ko 2023-10-30T12:20:45.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/apds990x.ko 2023-10-30T12:20:45.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 45593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/bh1770glc.ko 2023-10-30T12:20:45.253 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/c2port/ 2023-10-30T12:20:45.253 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-10-30T12:20:45.253 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/c2port/core.ko 2023-10-30T12:20:45.253 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/cb710/ 2023-10-30T12:20:45.253 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/cb710/cb710.ko 2023-10-30T12:20:45.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ds1682.ko 2023-10-30T12:20:45.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/dummy-irq.ko 2023-10-30T12:20:45.254 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/echo/ 2023-10-30T12:20:45.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/echo/echo.ko 2023-10-30T12:20:45.254 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/eeprom/ 2023-10-30T12:20:45.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/eeprom/at24.ko 2023-10-30T12:20:45.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-30T12:20:45.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-30T12:20:45.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/eeprom/max6875.ko 2023-10-30T12:20:45.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/enclosure.ko 2023-10-30T12:20:45.255 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/genwqe/ 2023-10-30T12:20:45.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 152201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-10-30T12:20:45.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/hmc6352.ko 2023-10-30T12:20:45.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/hpilo.ko 2023-10-30T12:20:45.256 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ibmasm/ 2023-10-30T12:20:45.257 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 72129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-10-30T12:20:45.257 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ics932s401.ko 2023-10-30T12:20:45.257 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/isl29003.ko 2023-10-30T12:20:45.257 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/isl29020.ko 2023-10-30T12:20:45.257 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/lis3lv02d/ 2023-10-30T12:20:45.258 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-30T12:20:45.258 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-30T12:20:45.258 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/mei/ 2023-10-30T12:20:45.258 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 125209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/mei/mei-me.ko 2023-10-30T12:20:45.258 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 56521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/mei/mei-txe.ko 2023-10-30T12:20:45.259 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 310113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/mei/mei.ko 2023-10-30T12:20:45.259 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/phantom.ko 2023-10-30T12:20:45.259 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ti-st/ 2023-10-30T12:20:45.259 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 74265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ti-st/st_drv.ko 2023-10-30T12:20:45.259 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/tifm_7xx1.ko 2023-10-30T12:20:45.260 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/tifm_core.ko 2023-10-30T12:20:45.260 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/tsl2550.ko 2023-10-30T12:20:45.260 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 48697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/vmw_balloon.ko 2023-10-30T12:20:45.260 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/vmw_vmci/ 2023-10-30T12:20:45.260 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 183609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-30T12:20:45.261 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ 2023-10-30T12:20:45.261 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/bonding/ 2023-10-30T12:20:45.261 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 490129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/bonding/bonding.ko 2023-10-30T12:20:45.261 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dsa/ 2023-10-30T12:20:45.261 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dsa/b53/ 2023-10-30T12:20:45.262 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 106033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-30T12:20:45.262 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-30T12:20:45.262 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-30T12:20:45.262 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dsa/mv88e6xxx/ 2023-10-30T12:20:45.262 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 453737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-30T12:20:45.263 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dummy.ko 2023-10-30T12:20:45.263 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/eql.ko 2023-10-30T12:20:45.263 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/ 2023-10-30T12:20:45.263 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/3com/ 2023-10-30T12:20:45.263 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 45785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-30T12:20:45.263 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-30T12:20:45.264 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 130553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-30T12:20:45.264 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 68849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-30T12:20:45.264 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/8390/ 2023-10-30T12:20:45.264 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-30T12:20:45.264 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 59689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-30T12:20:45.265 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-30T12:20:45.265 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 99113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-30T12:20:45.265 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/adaptec/ 2023-10-30T12:20:45.265 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-30T12:20:45.265 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/agere/ 2023-10-30T12:20:45.266 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 69201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-30T12:20:45.266 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/alteon/ 2023-10-30T12:20:45.266 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 59585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-30T12:20:45.266 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/altera/ 2023-10-30T12:20:45.266 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-30T12:20:45.267 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/amd/ 2023-10-30T12:20:45.267 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-30T12:20:45.267 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-30T12:20:45.267 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 87297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-30T12:20:45.268 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/ 2023-10-30T12:20:45.268 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/alx/ 2023-10-30T12:20:45.268 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 97337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-30T12:20:45.268 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-10-30T12:20:45.268 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 104673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-30T12:20:45.269 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-10-30T12:20:45.269 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 90257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-30T12:20:45.269 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/atlx/ 2023-10-30T12:20:45.269 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 81569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-30T12:20:45.269 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 69729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-30T12:20:45.270 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/ 2023-10-30T12:20:45.270 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-30T12:20:45.270 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 176761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-30T12:20:45.270 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-10-30T12:20:45.270 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1421497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-30T12:20:45.271 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 128569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-30T12:20:45.271 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/genet/ 2023-10-30T12:20:45.271 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 132089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-30T12:20:45.271 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 312081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-30T12:20:45.271 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/brocade/ 2023-10-30T12:20:45.272 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/brocade/bna/ 2023-10-30T12:20:45.272 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 323513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-30T12:20:45.272 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/ 2023-10-30T12:20:45.272 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-10-30T12:20:45.272 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 193497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-30T12:20:45.272 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-10-30T12:20:45.273 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 347417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-30T12:20:45.273 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-10-30T12:20:45.273 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1044433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-30T12:20:45.273 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-10-30T12:20:45.273 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 166193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-30T12:20:45.273 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-10-30T12:20:45.274 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-30T12:20:45.274 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/cisco/ 2023-10-30T12:20:45.274 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/cisco/enic/ 2023-10-30T12:20:45.274 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 204345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-30T12:20:45.274 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/ 2023-10-30T12:20:45.274 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/tulip/ 2023-10-30T12:20:45.275 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 69449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-30T12:20:45.275 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 61489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-30T12:20:45.275 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 141225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-30T12:20:45.275 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-30T12:20:45.275 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 62817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-30T12:20:45.276 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-30T12:20:45.276 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dlink/ 2023-10-30T12:20:45.276 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-30T12:20:45.276 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 62833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-30T12:20:45.276 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 45721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dnet.ko 2023-10-30T12:20:45.277 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/ec_bhf.ko 2023-10-30T12:20:45.277 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/emulex/ 2023-10-30T12:20:45.277 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/emulex/benet/ 2023-10-30T12:20:45.277 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 281137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-30T12:20:45.277 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/ethoc.ko 2023-10-30T12:20:45.278 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/fealnx.ko 2023-10-30T12:20:45.278 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/fujitsu/ 2023-10-30T12:20:45.278 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 45345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-10-30T12:20:45.278 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/ 2023-10-30T12:20:45.278 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 90465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-30T12:20:45.279 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/e1000/ 2023-10-30T12:20:45.279 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 295577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-30T12:20:45.279 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/e1000e/ 2023-10-30T12:20:45.279 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 584497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-30T12:20:45.279 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/fm10k/ 2023-10-30T12:20:45.279 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 246209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-30T12:20:45.280 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/i40e/ 2023-10-30T12:20:45.280 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 925697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-30T12:20:45.376 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/iavf/ 2023-10-30T12:20:45.377 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 302849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-30T12:20:45.377 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/igb/ 2023-10-30T12:20:45.377 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 514681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-30T12:20:45.377 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/igbvf/ 2023-10-30T12:20:45.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 93593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-30T12:20:45.378 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-10-30T12:20:45.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 734305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-30T12:20:45.378 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-10-30T12:20:45.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 151769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-30T12:20:45.379 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 86921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/jme.ko 2023-10-30T12:20:45.379 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/marvell/ 2023-10-30T12:20:45.379 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-30T12:20:45.379 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 102289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-30T12:20:45.379 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 122649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-30T12:20:45.380 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/mellanox/ 2023-10-30T12:20:45.380 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-10-30T12:20:45.380 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 727601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-30T12:20:45.380 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 278937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-30T12:20:45.380 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-10-30T12:20:45.381 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-10-30T12:20:45.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1113993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-30T12:20:45.381 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/micrel/ 2023-10-30T12:20:45.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-10-30T12:20:45.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-10-30T12:20:45.382 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-10-30T12:20:45.382 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 95665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-30T12:20:45.382 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/myricom/ 2023-10-30T12:20:45.382 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-10-30T12:20:45.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 108649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-30T12:20:45.383 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/natsemi/ 2023-10-30T12:20:45.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 73441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-30T12:20:45.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-30T12:20:45.383 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/neterion/ 2023-10-30T12:20:45.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 150905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-30T12:20:45.384 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/nvidia/ 2023-10-30T12:20:45.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 136721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-30T12:20:45.384 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/packetengines/ 2023-10-30T12:20:45.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-30T12:20:45.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-30T12:20:45.385 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/qlogic/ 2023-10-30T12:20:45.385 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-10-30T12:20:45.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 227897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-30T12:20:45.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 87081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-30T12:20:45.385 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-10-30T12:20:45.386 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 672049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-30T12:20:45.386 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/rdc/ 2023-10-30T12:20:45.386 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-30T12:20:45.386 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/realtek/ 2023-10-30T12:20:45.386 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-30T12:20:45.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 83985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-30T12:20:45.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-30T12:20:45.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 189801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-30T12:20:45.387 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/samsung/ 2023-10-30T12:20:45.387 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-10-30T12:20:45.388 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 104241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-10-30T12:20:45.388 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sfc/ 2023-10-30T12:20:45.388 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1053921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-30T12:20:45.388 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/silan/ 2023-10-30T12:20:45.388 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-30T12:20:45.389 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sis/ 2023-10-30T12:20:45.389 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-30T12:20:45.389 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 63337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-30T12:20:45.389 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/smsc/ 2023-10-30T12:20:45.389 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 61601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-30T12:20:45.389 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-30T12:20:45.390 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 73409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-30T12:20:45.390 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 68177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-30T12:20:45.390 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/stmicro/ 2023-10-30T12:20:45.390 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-10-30T12:20:45.390 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-10-30T12:20:45.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-30T12:20:45.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-10-30T12:20:45.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 486961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-30T12:20:45.391 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sun/ 2023-10-30T12:20:45.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 103337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-30T12:20:45.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 166161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-30T12:20:45.392 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-30T12:20:45.392 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-30T12:20:45.392 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/tehuti/ 2023-10-30T12:20:45.392 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-30T12:20:45.392 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/ti/ 2023-10-30T12:20:45.393 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 86529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-30T12:20:45.393 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/via/ 2023-10-30T12:20:45.393 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 80273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-30T12:20:45.393 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 83633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-30T12:20:45.393 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/wiznet/ 2023-10-30T12:20:45.394 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-30T12:20:45.394 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-30T12:20:45.394 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/xircom/ 2023-10-30T12:20:45.394 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-30T12:20:45.394 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/fddi/ 2023-10-30T12:20:45.395 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/fddi/defxx.ko 2023-10-30T12:20:45.395 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/fddi/skfp/ 2023-10-30T12:20:45.395 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 216985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/fddi/skfp/skfp.ko 2023-10-30T12:20:45.395 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 77345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/geneve.ko 2023-10-30T12:20:45.395 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/hyperv/ 2023-10-30T12:20:45.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 235521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-30T12:20:45.396 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ieee802154/ 2023-10-30T12:20:45.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-30T12:20:45.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ifb.ko 2023-10-30T12:20:45.396 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ipvlan/ 2023-10-30T12:20:45.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 83769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-30T12:20:45.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 62209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/macvlan.ko 2023-10-30T12:20:45.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/macvtap.ko 2023-10-30T12:20:45.397 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/mdio/ 2023-10-30T12:20:45.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-30T12:20:45.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-30T12:20:45.398 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/mdio/mdio-gpio.ko 2023-10-30T12:20:45.398 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-30T12:20:45.398 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/mdio.ko 2023-10-30T12:20:45.398 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/mii.ko 2023-10-30T12:20:45.398 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/netconsole.ko 2023-10-30T12:20:45.398 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/nlmon.ko 2023-10-30T12:20:45.399 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/pcs/ 2023-10-30T12:20:45.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-30T12:20:45.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-30T12:20:45.399 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/ 2023-10-30T12:20:45.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/amd.ko 2023-10-30T12:20:45.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-30T12:20:45.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-30T12:20:45.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/bcm87xx.ko 2023-10-30T12:20:45.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/broadcom.ko 2023-10-30T12:20:45.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/cicada.ko 2023-10-30T12:20:45.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/davicom.ko 2023-10-30T12:20:45.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/et1011c.ko 2023-10-30T12:20:45.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/icplus.ko 2023-10-30T12:20:45.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/lxt.ko 2023-10-30T12:20:45.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 82049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/marvell.ko 2023-10-30T12:20:45.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 76177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/micrel.ko 2023-10-30T12:20:45.402 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/national.ko 2023-10-30T12:20:45.402 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 97857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/phylink.ko 2023-10-30T12:20:45.402 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/qsemi.ko 2023-10-30T12:20:45.402 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/realtek.ko 2023-10-30T12:20:45.402 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/smsc.ko 2023-10-30T12:20:45.403 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/ste10Xp.ko 2023-10-30T12:20:45.403 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/vitesse.ko 2023-10-30T12:20:45.403 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/plip/ 2023-10-30T12:20:45.403 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/plip/plip.ko 2023-10-30T12:20:45.403 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/ 2023-10-30T12:20:45.403 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-30T12:20:45.404 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/ppp_async.ko 2023-10-30T12:20:45.404 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-30T12:20:45.404 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-30T12:20:45.404 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-30T12:20:45.404 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/pppoe.ko 2023-10-30T12:20:45.405 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/pppox.ko 2023-10-30T12:20:45.405 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/pptp.ko 2023-10-30T12:20:45.405 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/sb1000.ko 2023-10-30T12:20:45.405 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/slip/ 2023-10-30T12:20:45.405 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/slip/slip.ko 2023-10-30T12:20:45.405 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/sungem_phy.ko 2023-10-30T12:20:45.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 64385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/tap.ko 2023-10-30T12:20:45.406 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/team/ 2023-10-30T12:20:45.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 106169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/team/team.ko 2023-10-30T12:20:45.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-30T12:20:45.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-30T12:20:45.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-30T12:20:45.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/team/team_mode_random.ko 2023-10-30T12:20:45.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-30T12:20:45.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 87017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/veth.ko 2023-10-30T12:20:45.407 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/vmxnet3/ 2023-10-30T12:20:45.408 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 144297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-30T12:20:45.408 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/vxlan/ 2023-10-30T12:20:45.408 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 297329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/vxlan/vxlan.ko 2023-10-30T12:20:45.408 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/xen-netback/ 2023-10-30T12:20:45.408 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 144649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-30T12:20:45.409 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ntb/ 2023-10-30T12:20:45.409 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ntb/ntb.ko 2023-10-30T12:20:45.409 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/nvme/ 2023-10-30T12:20:45.409 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/nvme/host/ 2023-10-30T12:20:45.409 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 264937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/nvme/host/nvme-core.ko 2023-10-30T12:20:45.409 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 102209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/nvme/host/nvme.ko 2023-10-30T12:20:45.410 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/parport/ 2023-10-30T12:20:45.410 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 133841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/parport/parport.ko 2023-10-30T12:20:45.410 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/parport/parport_cs.ko 2023-10-30T12:20:45.410 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 84465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/parport/parport_pc.ko 2023-10-30T12:20:45.410 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/parport/parport_serial.ko 2023-10-30T12:20:45.411 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pci/ 2023-10-30T12:20:45.411 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pci/hotplug/ 2023-10-30T12:20:45.411 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-30T12:20:45.411 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-10-30T12:20:45.411 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-10-30T12:20:45.412 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pci/pci-stub.ko 2023-10-30T12:20:45.412 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pci/xen-pcifront.ko 2023-10-30T12:20:45.412 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pcmcia/ 2023-10-30T12:20:45.412 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pcmcia/i82092.ko 2023-10-30T12:20:45.412 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 145809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pcmcia/pcmcia.ko 2023-10-30T12:20:45.413 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 62137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pcmcia/pcmcia_core.ko 2023-10-30T12:20:45.413 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-10-30T12:20:45.413 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pcmcia/pd6729.ko 2023-10-30T12:20:45.413 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 112889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-30T12:20:45.413 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/phy/ 2023-10-30T12:20:45.413 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/phy/broadcom/ 2023-10-30T12:20:45.414 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-10-30T12:20:45.414 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/phy/samsung/ 2023-10-30T12:20:45.414 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-10-30T12:20:45.414 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pinctrl/ 2023-10-30T12:20:45.415 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pinctrl/intel/ 2023-10-30T12:20:45.415 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-30T12:20:45.415 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/ 2023-10-30T12:20:45.415 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/ 2023-10-30T12:20:45.415 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-10-30T12:20:45.415 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/cros_ec.ko 2023-10-30T12:20:45.416 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-10-30T12:20:45.416 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-10-30T12:20:45.416 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-10-30T12:20:45.416 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-10-30T12:20:45.416 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-10-30T12:20:45.417 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/ 2023-10-30T12:20:45.417 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-30T12:20:45.417 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/acerhdf.ko 2023-10-30T12:20:45.417 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-30T12:20:45.417 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 59665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-30T12:20:45.418 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-30T12:20:45.418 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 127569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-30T12:20:45.418 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-30T12:20:45.418 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-30T12:20:45.418 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-30T12:20:45.419 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-30T12:20:45.419 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-30T12:20:45.419 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/hdaps.ko 2023-10-30T12:20:45.419 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/ibm_rtl.ko 2023-10-30T12:20:45.419 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/intel/ 2023-10-30T12:20:45.420 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-30T12:20:45.420 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-10-30T12:20:45.420 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/intel_ips.ko 2023-10-30T12:20:45.420 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-30T12:20:45.420 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-30T12:20:45.420 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-30T12:20:45.421 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-30T12:20:45.421 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-30T12:20:45.421 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-30T12:20:45.421 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-30T12:20:45.421 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-30T12:20:45.422 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-30T12:20:45.422 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/wmi.ko 2023-10-30T12:20:45.422 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/ 2023-10-30T12:20:45.422 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/ 2023-10-30T12:20:45.422 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/88pm860x_battery.ko 2023-10-30T12:20:45.423 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/88pm860x_charger.ko 2023-10-30T12:20:45.423 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 55057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/bq2415x_charger.ko 2023-10-30T12:20:45.423 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/bq24190_charger.ko 2023-10-30T12:20:45.423 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/bq24735-charger.ko 2023-10-30T12:20:45.423 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-10-30T12:20:45.424 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/da9030_battery.ko 2023-10-30T12:20:45.424 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/da9052-battery.ko 2023-10-30T12:20:45.424 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/ds2760_battery.ko 2023-10-30T12:20:45.424 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/ds2780_battery.ko 2023-10-30T12:20:45.424 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/ds2781_battery.ko 2023-10-30T12:20:45.424 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/ds2782_battery.ko 2023-10-30T12:20:45.425 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/gpio-charger.ko 2023-10-30T12:20:45.425 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/isp1704_charger.ko 2023-10-30T12:20:45.425 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/lp8727_charger.ko 2023-10-30T12:20:45.425 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/max14577_charger.ko 2023-10-30T12:20:45.425 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/max17040_battery.ko 2023-10-30T12:20:45.425 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/max17042_battery.ko 2023-10-30T12:20:45.426 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/max8903_charger.ko 2023-10-30T12:20:45.426 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/max8925_power.ko 2023-10-30T12:20:45.426 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/pcf50633-charger.ko 2023-10-30T12:20:45.426 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/sbs-battery.ko 2023-10-30T12:20:45.426 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/test_power.ko 2023-10-30T12:20:45.427 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/tps65090-charger.ko 2023-10-30T12:20:45.427 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/wm831x_backup.ko 2023-10-30T12:20:45.427 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/wm831x_power.ko 2023-10-30T12:20:45.427 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/wm8350_power.ko 2023-10-30T12:20:45.427 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/powercap/ 2023-10-30T12:20:45.428 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-30T12:20:45.428 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-30T12:20:45.428 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pps/ 2023-10-30T12:20:45.428 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pps/clients/ 2023-10-30T12:20:45.428 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-30T12:20:45.429 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-30T12:20:45.429 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pps/clients/pps_parport.ko 2023-10-30T12:20:45.429 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pps/pps_core.ko 2023-10-30T12:20:45.429 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ptp/ 2023-10-30T12:20:45.429 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ptp/ptp.ko 2023-10-30T12:20:45.429 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ptp/ptp_kvm.ko 2023-10-30T12:20:45.430 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/ 2023-10-30T12:20:45.430 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-88pm80x.ko 2023-10-30T12:20:45.430 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-88pm860x.ko 2023-10-30T12:20:45.430 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-30T12:20:45.430 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-da9052.ko 2023-10-30T12:20:45.431 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-da9055.ko 2023-10-30T12:20:45.431 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-da9063.ko 2023-10-30T12:20:45.431 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-30T12:20:45.431 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 68577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-30T12:20:45.431 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-30T12:20:45.432 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-30T12:20:45.432 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-30T12:20:45.432 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-30T12:20:45.432 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-30T12:20:45.432 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-30T12:20:45.432 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23705 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-30T12:20:45.433 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-em3027.ko 2023-10-30T12:20:45.433 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-30T12:20:45.433 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-30T12:20:45.433 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-30T12:20:45.433 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-lp8788.ko 2023-10-30T12:20:45.434 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-30T12:20:45.434 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-30T12:20:45.434 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-30T12:20:45.434 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-m48t86.ko 2023-10-30T12:20:45.434 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-max6900.ko 2023-10-30T12:20:45.435 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-max8907.ko 2023-10-30T12:20:45.435 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-max8925.ko 2023-10-30T12:20:45.435 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-max8997.ko 2023-10-30T12:20:45.435 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-max8998.ko 2023-10-30T12:20:45.435 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-10-30T12:20:45.436 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-30T12:20:45.436 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-palmas.ko 2023-10-30T12:20:45.436 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-30T12:20:45.436 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-pcf50633.ko 2023-10-30T12:20:45.436 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-30T12:20:45.436 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-30T12:20:45.437 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-30T12:20:45.437 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-30T12:20:45.437 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-rc5t583.ko 2023-10-30T12:20:45.437 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-30T12:20:45.437 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-30T12:20:45.438 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-30T12:20:45.438 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-30T12:20:45.438 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-30T12:20:45.438 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-s35390a.ko 2023-10-30T12:20:45.439 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-30T12:20:45.439 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-tps6586x.ko 2023-10-30T12:20:45.439 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-tps65910.ko 2023-10-30T12:20:45.439 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-wm831x.ko 2023-10-30T12:20:45.439 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-wm8350.ko 2023-10-30T12:20:45.439 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-x1205.ko 2023-10-30T12:20:45.440 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/ 2023-10-30T12:20:45.440 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 77617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/3w-9xxx.ko 2023-10-30T12:20:45.440 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/3w-sas.ko 2023-10-30T12:20:45.440 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 55169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/3w-xxxx.ko 2023-10-30T12:20:45.440 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 135185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/BusLogic.ko 2023-10-30T12:20:45.440 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/a100u2w.ko 2023-10-30T12:20:45.441 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/aacraid/ 2023-10-30T12:20:45.441 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 272305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-30T12:20:45.441 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 100617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/advansys.ko 2023-10-30T12:20:45.441 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/aic7xxx/ 2023-10-30T12:20:45.441 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 356265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-30T12:20:45.442 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 295801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-30T12:20:45.442 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/aic94xx/ 2023-10-30T12:20:45.442 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 189185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-10-30T12:20:45.442 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/am53c974.ko 2023-10-30T12:20:45.443 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/arcmsr/ 2023-10-30T12:20:45.443 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 106881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-30T12:20:45.443 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/atp870u.ko 2023-10-30T12:20:45.443 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/be2iscsi/ 2023-10-30T12:20:45.443 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 246985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-30T12:20:45.443 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/bfa/ 2023-10-30T12:20:45.444 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1171945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/bfa/bfa.ko 2023-10-30T12:20:45.444 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/bnx2fc/ 2023-10-30T12:20:45.444 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 237273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-30T12:20:45.444 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/bnx2i/ 2023-10-30T12:20:45.444 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 115369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-30T12:20:45.445 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/ch.ko 2023-10-30T12:20:45.445 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/csiostor/ 2023-10-30T12:20:45.445 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 308921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-30T12:20:45.445 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/cxgbi/ 2023-10-30T12:20:45.445 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-10-30T12:20:45.446 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 76145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-30T12:20:45.446 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-10-30T12:20:45.446 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 101449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-30T12:20:45.446 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 146689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-30T12:20:45.446 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 75553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/dc395x.ko 2023-10-30T12:20:45.447 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/dmx3191d.ko 2023-10-30T12:20:45.447 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/esas2r/ 2023-10-30T12:20:45.447 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 212425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-30T12:20:45.447 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/esp_scsi.ko 2023-10-30T12:20:45.447 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/fcoe/ 2023-10-30T12:20:45.447 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-30T12:20:45.448 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 158505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-30T12:20:45.448 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/fdomain.ko 2023-10-30T12:20:45.448 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/fnic/ 2023-10-30T12:20:45.448 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 263489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/fnic/fnic.ko 2023-10-30T12:20:45.448 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 201817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/hpsa.ko 2023-10-30T12:20:45.449 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/hptiop.ko 2023-10-30T12:20:45.449 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/hv_storvsc.ko 2023-10-30T12:20:45.449 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/imm.ko 2023-10-30T12:20:45.449 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/initio.ko 2023-10-30T12:20:45.449 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 266537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/ipr.ko 2023-10-30T12:20:45.450 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 101881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/ips.ko 2023-10-30T12:20:45.450 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/isci/ 2023-10-30T12:20:45.450 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 317153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/isci/isci.ko 2023-10-30T12:20:45.450 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-30T12:20:45.450 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 52961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-30T12:20:45.450 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/libfc/ 2023-10-30T12:20:45.451 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 314865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/libfc/libfc.ko 2023-10-30T12:20:45.451 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 145585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/libiscsi.ko 2023-10-30T12:20:45.451 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-30T12:20:45.451 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/libsas/ 2023-10-30T12:20:45.451 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 216665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/libsas/libsas.ko 2023-10-30T12:20:45.452 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/lpfc/ 2023-10-30T12:20:45.452 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 2197249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-30T12:20:45.452 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/megaraid/ 2023-10-30T12:20:45.452 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 73569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-30T12:20:45.452 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-30T12:20:45.452 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 310617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-30T12:20:45.453 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 84721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/megaraid.ko 2023-10-30T12:20:45.453 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/mpt3sas/ 2023-10-30T12:20:45.453 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 659777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-30T12:20:45.453 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/mvsas/ 2023-10-30T12:20:45.453 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 119561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-30T12:20:45.454 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/mvumi.ko 2023-10-30T12:20:45.454 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pcmcia/ 2023-10-30T12:20:45.454 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 81625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-10-30T12:20:45.454 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-10-30T12:20:45.454 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-10-30T12:20:45.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-10-30T12:20:45.455 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pm8001/ 2023-10-30T12:20:45.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 364105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-30T12:20:45.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 123617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pmcraid.ko 2023-10-30T12:20:45.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/ppa.ko 2023-10-30T12:20:45.456 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 55361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/qla1280.ko 2023-10-30T12:20:45.456 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/qla2xxx/ 2023-10-30T12:20:45.456 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1876625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-30T12:20:45.456 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 85289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-30T12:20:45.456 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/qla4xxx/ 2023-10-30T12:20:45.457 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 559033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-30T12:20:45.457 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/qlogicfas408.ko 2023-10-30T12:20:45.457 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/raid_class.ko 2023-10-30T12:20:45.457 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 238825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/scsi_debug.ko 2023-10-30T12:20:45.457 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 173937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-30T12:20:45.457 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 279313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-30T12:20:45.458 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 97729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-30T12:20:45.458 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 73881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-30T12:20:45.458 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-30T12:20:45.458 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/ses.ko 2023-10-30T12:20:45.458 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 115889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/st.ko 2023-10-30T12:20:45.459 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/stex.ko 2023-10-30T12:20:45.459 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/sym53c8xx_2/ 2023-10-30T12:20:45.459 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 162121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-30T12:20:45.459 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/virtio_scsi.ko 2023-10-30T12:20:45.459 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-30T12:20:45.460 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/wd719x.ko 2023-10-30T12:20:45.460 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/xen-scsifront.ko 2023-10-30T12:20:45.460 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ssb/ 2023-10-30T12:20:45.460 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 129441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ssb/ssb.ko 2023-10-30T12:20:45.460 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/ 2023-10-30T12:20:45.460 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/iscsi/ 2023-10-30T12:20:45.461 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 736601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-30T12:20:45.461 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/loopback/ 2023-10-30T12:20:45.461 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-30T12:20:45.461 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/target_core_file.ko 2023-10-30T12:20:45.461 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/target_core_iblock.ko 2023-10-30T12:20:45.462 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 930321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/target_core_mod.ko 2023-10-30T12:20:45.462 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/target_core_pscsi.ko 2023-10-30T12:20:45.462 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 122521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/target_core_user.ko 2023-10-30T12:20:45.462 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/tcm_fc/ 2023-10-30T12:20:45.462 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 75593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-30T12:20:45.463 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/ 2023-10-30T12:20:45.463 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/ 2023-10-30T12:20:45.463 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/ 2023-10-30T12:20:45.463 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-30T12:20:45.463 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-30T12:20:45.464 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-30T12:20:45.464 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-30T12:20:45.464 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-30T12:20:45.464 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-30T12:20:45.464 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-30T12:20:45.464 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-30T12:20:45.465 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-30T12:20:45.465 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-30T12:20:45.465 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-30T12:20:45.465 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-30T12:20:45.465 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-10-30T12:20:45.466 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-30T12:20:45.466 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-30T12:20:45.466 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-30T12:20:45.466 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/ 2023-10-30T12:20:45.466 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/ipwireless/ 2023-10-30T12:20:45.467 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-30T12:20:45.467 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/moxa.ko 2023-10-30T12:20:45.467 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/mxser.ko 2023-10-30T12:20:45.467 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 108601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/n_gsm.ko 2023-10-30T12:20:45.467 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/n_hdlc.ko 2023-10-30T12:20:45.468 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/nozomi.ko 2023-10-30T12:20:45.468 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/ 2023-10-30T12:20:45.468 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/8250/ 2023-10-30T12:20:45.468 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-10-30T12:20:45.468 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 59497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-30T12:20:45.468 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-10-30T12:20:45.469 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/altera_uart.ko 2023-10-30T12:20:45.469 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/arc_uart.ko 2023-10-30T12:20:45.469 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 72425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-10-30T12:20:45.469 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/jsm/ 2023-10-30T12:20:45.469 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 103073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-30T12:20:45.470 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/rp2.ko 2023-10-30T12:20:45.470 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 52817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/sc16is7xx.ko 2023-10-30T12:20:45.470 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ufs/ 2023-10-30T12:20:45.470 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ufs/core/ 2023-10-30T12:20:45.470 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 531265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-30T12:20:45.471 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ufs/host/ 2023-10-30T12:20:45.471 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-30T12:20:45.471 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-10-30T12:20:45.471 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/ 2023-10-30T12:20:45.471 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio.ko 2023-10-30T12:20:45.471 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_aec.ko 2023-10-30T12:20:45.472 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_cif.ko 2023-10-30T12:20:45.472 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_dmem_genirq.ko 2023-10-30T12:20:45.472 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_mf624.ko 2023-10-30T12:20:45.472 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_netx.ko 2023-10-30T12:20:45.472 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_pci_generic.ko 2023-10-30T12:20:45.473 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-10-30T12:20:45.473 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_sercos3.ko 2023-10-30T12:20:45.473 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/ 2023-10-30T12:20:45.473 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/c67x00/ 2023-10-30T12:20:45.473 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 73297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/c67x00/c67x00.ko 2023-10-30T12:20:45.473 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/chipidea/ 2023-10-30T12:20:45.474 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 144377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-10-30T12:20:45.474 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-10-30T12:20:45.474 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-10-30T12:20:45.474 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-10-30T12:20:45.474 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/class/ 2023-10-30T12:20:45.475 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 87089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/class/cdc-acm.ko 2023-10-30T12:20:45.475 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-30T12:20:45.475 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/class/usblp.ko 2023-10-30T12:20:45.475 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/class/usbtmc.ko 2023-10-30T12:20:45.475 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/common/ 2023-10-30T12:20:45.476 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/common/ulpi.ko 2023-10-30T12:20:45.476 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/dwc2/ 2023-10-30T12:20:45.476 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-10-30T12:20:45.476 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/dwc3/ 2023-10-30T12:20:45.476 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-10-30T12:20:45.477 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-10-30T12:20:45.477 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 444553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/dwc3/dwc3.ko 2023-10-30T12:20:45.477 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/ 2023-10-30T12:20:45.477 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/ 2023-10-30T12:20:45.477 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/u_ether.ko 2023-10-30T12:20:45.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/u_serial.ko 2023-10-30T12:20:45.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-10-30T12:20:45.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-10-30T12:20:45.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-10-30T12:20:45.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-10-30T12:20:45.479 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 92489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-10-30T12:20:45.479 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-10-30T12:20:45.479 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 122081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-10-30T12:20:45.479 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-10-30T12:20:45.479 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-10-30T12:20:45.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-10-30T12:20:45.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-10-30T12:20:45.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 85609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-10-30T12:20:45.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-10-30T12:20:45.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 58113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-10-30T12:20:45.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-10-30T12:20:45.481 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/ 2023-10-30T12:20:45.481 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-10-30T12:20:45.481 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-10-30T12:20:45.481 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-10-30T12:20:45.482 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-10-30T12:20:45.482 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-10-30T12:20:45.482 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-10-30T12:20:45.482 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-10-30T12:20:45.482 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-10-30T12:20:45.482 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-10-30T12:20:45.483 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-10-30T12:20:45.483 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-10-30T12:20:45.483 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-10-30T12:20:45.483 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 62185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-10-30T12:20:45.483 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-10-30T12:20:45.484 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 161225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/libcomposite.ko 2023-10-30T12:20:45.484 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/ 2023-10-30T12:20:45.484 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17705 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-10-30T12:20:45.484 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/bdc/ 2023-10-30T12:20:45.484 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 157681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-10-30T12:20:45.485 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-10-30T12:20:45.485 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 52721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-10-30T12:20:45.485 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 69705 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-10-30T12:20:45.485 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 56033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-10-30T12:20:45.485 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 76057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/net2272.ko 2023-10-30T12:20:45.486 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 75073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/net2280.ko 2023-10-30T12:20:45.486 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 73169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-10-30T12:20:45.486 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 65809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-10-30T12:20:45.486 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-10-30T12:20:45.486 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 81505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-10-30T12:20:45.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 156073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-10-30T12:20:45.487 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/ 2023-10-30T12:20:45.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/bcma-hcd.ko 2023-10-30T12:20:45.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 80113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/isp116x-hcd.ko 2023-10-30T12:20:45.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 87353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-10-30T12:20:45.488 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-10-30T12:20:45.488 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-30T12:20:45.488 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/sl811_cs.ko 2023-10-30T12:20:45.488 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/ssb-hcd.ko 2023-10-30T12:20:45.488 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-10-30T12:20:45.488 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/image/ 2023-10-30T12:20:45.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/image/mdc800.ko 2023-10-30T12:20:45.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/image/microtek.ko 2023-10-30T12:20:45.489 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/ 2023-10-30T12:20:45.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/adutux.ko 2023-10-30T12:20:45.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/appledisplay.ko 2023-10-30T12:20:45.490 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-10-30T12:20:45.490 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/cytherm.ko 2023-10-30T12:20:45.490 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/ehset.ko 2023-10-30T12:20:45.490 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/emi26.ko 2023-10-30T12:20:45.490 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/emi62.ko 2023-10-30T12:20:45.491 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/ezusb.ko 2023-10-30T12:20:45.491 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/idmouse.ko 2023-10-30T12:20:45.491 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/iowarrior.ko 2023-10-30T12:20:45.491 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-30T12:20:45.491 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/ldusb.ko 2023-10-30T12:20:45.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/legousbtower.ko 2023-10-30T12:20:45.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/lvstest.ko 2023-10-30T12:20:45.492 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/sisusbvga/ 2023-10-30T12:20:45.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 64841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-30T12:20:45.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-30T12:20:45.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/usb3503.ko 2023-10-30T12:20:45.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/usblcd.ko 2023-10-30T12:20:45.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-30T12:20:45.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/usbtest.ko 2023-10-30T12:20:45.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/uss720.ko 2023-10-30T12:20:45.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/yurex.ko 2023-10-30T12:20:45.494 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/mon/ 2023-10-30T12:20:45.494 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 72953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/mon/usbmon.ko 2023-10-30T12:20:45.494 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/musb/ 2023-10-30T12:20:45.494 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 301545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/musb/musb_hdrc.ko 2023-10-30T12:20:45.494 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/phy/ 2023-10-30T12:20:45.495 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/phy/phy-generic.ko 2023-10-30T12:20:45.495 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-10-30T12:20:45.495 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/phy/phy-isp1301.ko 2023-10-30T12:20:45.495 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/phy/phy-tahvo.ko 2023-10-30T12:20:45.495 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ 2023-10-30T12:20:45.495 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/aircable.ko 2023-10-30T12:20:45.496 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ark3116.ko 2023-10-30T12:20:45.496 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-30T12:20:45.496 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ch341.ko 2023-10-30T12:20:45.496 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/cp210x.ko 2023-10-30T12:20:45.496 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/cyberjack.ko 2023-10-30T12:20:45.497 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 58217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-30T12:20:45.497 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-30T12:20:45.497 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/empeg.ko 2023-10-30T12:20:45.497 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/f81232.ko 2023-10-30T12:20:45.497 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 175137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-30T12:20:45.498 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-30T12:20:45.498 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 137081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-30T12:20:45.498 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 124833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/io_ti.ko 2023-10-30T12:20:45.498 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ipaq.ko 2023-10-30T12:20:45.498 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ipw.ko 2023-10-30T12:20:45.499 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ir-usb.ko 2023-10-30T12:20:45.499 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 58233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-30T12:20:45.499 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 107521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/keyspan.ko 2023-10-30T12:20:45.499 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-30T12:20:45.499 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-30T12:20:45.500 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-30T12:20:45.500 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/mct_u232.ko 2023-10-30T12:20:45.500 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/metro-usb.ko 2023-10-30T12:20:45.500 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/mos7720.ko 2023-10-30T12:20:45.500 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/mos7840.ko 2023-10-30T12:20:45.500 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 56337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/mxuport.ko 2023-10-30T12:20:45.501 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/navman.ko 2023-10-30T12:20:45.501 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/omninet.ko 2023-10-30T12:20:45.501 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/opticon.ko 2023-10-30T12:20:45.501 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 190633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/option.ko 2023-10-30T12:20:45.501 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/oti6858.ko 2023-10-30T12:20:45.502 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/pl2303.ko 2023-10-30T12:20:45.502 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/qcaux.ko 2023-10-30T12:20:45.502 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/qcserial.ko 2023-10-30T12:20:45.502 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/quatech2.ko 2023-10-30T12:20:45.502 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/safe_serial.ko 2023-10-30T12:20:45.503 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/sierra.ko 2023-10-30T12:20:45.503 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-30T12:20:45.503 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ssu100.ko 2023-10-30T12:20:45.503 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/symbolserial.ko 2023-10-30T12:20:45.503 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 61489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-30T12:20:45.504 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-30T12:20:45.504 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/usb_debug.ko 2023-10-30T12:20:45.504 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-30T12:20:45.504 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 125609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/usbserial.ko 2023-10-30T12:20:45.504 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/visor.ko 2023-10-30T12:20:45.505 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/whiteheat.ko 2023-10-30T12:20:45.505 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/wishbone-serial.ko 2023-10-30T12:20:45.505 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-30T12:20:45.505 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ 2023-10-30T12:20:45.505 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 52913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/uas.ko 2023-10-30T12:20:45.506 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-30T12:20:45.506 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-30T12:20:45.506 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-30T12:20:45.506 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-30T12:20:45.506 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-30T12:20:45.506 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-30T12:20:45.507 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-30T12:20:45.507 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-karma.ko 2023-10-30T12:20:45.507 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-30T12:20:45.507 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-30T12:20:45.507 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-30T12:20:45.508 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-30T12:20:45.508 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-30T12:20:45.508 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 179969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/usb-storage.ko 2023-10-30T12:20:45.508 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/usbip/ 2023-10-30T12:20:45.508 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-30T12:20:45.509 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 72177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-30T12:20:45.509 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 107001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-30T12:20:45.509 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vfio/ 2023-10-30T12:20:45.509 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vfio/pci/ 2023-10-30T12:20:45.509 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 154297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-30T12:20:45.509 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-30T12:20:45.510 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 108737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vfio/vfio.ko 2023-10-30T12:20:45.510 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 77609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-30T12:20:45.510 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vhost/ 2023-10-30T12:20:45.510 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 113873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vhost/vhost.ko 2023-10-30T12:20:45.510 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-30T12:20:45.511 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 58089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vhost/vhost_net.ko 2023-10-30T12:20:45.511 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vhost/vhost_scsi.ko 2023-10-30T12:20:45.511 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/ 2023-10-30T12:20:45.511 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/ 2023-10-30T12:20:45.511 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-10-30T12:20:45.512 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/aat2870_bl.ko 2023-10-30T12:20:45.512 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/adp5520_bl.ko 2023-10-30T12:20:45.512 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/adp8860_bl.ko 2023-10-30T12:20:45.512 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/adp8870_bl.ko 2023-10-30T12:20:45.512 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/apple_bl.ko 2023-10-30T12:20:45.512 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/as3711_bl.ko 2023-10-30T12:20:45.513 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/bd6107.ko 2023-10-30T12:20:45.513 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/da903x_bl.ko 2023-10-30T12:20:45.513 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/da9052_bl.ko 2023-10-30T12:20:45.513 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/gpio_backlight.ko 2023-10-30T12:20:45.513 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/kb3886_bl.ko 2023-10-30T12:20:45.514 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/lcd.ko 2023-10-30T12:20:45.514 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/lm3533_bl.ko 2023-10-30T12:20:45.514 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/lm3639_bl.ko 2023-10-30T12:20:45.514 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/lv5207lp.ko 2023-10-30T12:20:45.514 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/max8925_bl.ko 2023-10-30T12:20:45.514 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/pandora_bl.ko 2023-10-30T12:20:45.515 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-10-30T12:20:45.515 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-30T12:20:45.515 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/wm831x_bl.ko 2023-10-30T12:20:45.515 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/ 2023-10-30T12:20:45.515 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/masters/ 2023-10-30T12:20:45.516 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/masters/ds2482.ko 2023-10-30T12:20:45.516 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/masters/ds2490.ko 2023-10-30T12:20:45.516 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/masters/matrox_w1.ko 2023-10-30T12:20:45.516 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/masters/w1-gpio.ko 2023-10-30T12:20:45.516 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/ 2023-10-30T12:20:45.517 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-30T12:20:45.517 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-30T12:20:45.517 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-30T12:20:45.517 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-30T12:20:45.517 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-30T12:20:45.518 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-30T12:20:45.518 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-30T12:20:45.518 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-30T12:20:45.518 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-30T12:20:45.518 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-30T12:20:45.518 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 56425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-30T12:20:45.519 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 102673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/wire.ko 2023-10-30T12:20:45.519 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/ 2023-10-30T12:20:45.519 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/acquirewdt.ko 2023-10-30T12:20:45.519 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/advantechwdt.ko 2023-10-30T12:20:45.519 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-30T12:20:45.520 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-30T12:20:45.520 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/cpu5wdt.ko 2023-10-30T12:20:45.520 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/da9052_wdt.ko 2023-10-30T12:20:45.520 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/da9055_wdt.ko 2023-10-30T12:20:45.520 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/da9063_wdt.ko 2023-10-30T12:20:45.520 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/dw_wdt.ko 2023-10-30T12:20:45.521 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/eurotechwdt.ko 2023-10-30T12:20:45.521 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-30T12:20:45.521 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/hpwdt.ko 2023-10-30T12:20:45.521 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/i6300esb.ko 2023-10-30T12:20:45.521 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-30T12:20:45.522 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-30T12:20:45.522 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/ib700wdt.ko 2023-10-30T12:20:45.522 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/ibmasr.ko 2023-10-30T12:20:45.522 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-30T12:20:45.523 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-30T12:20:45.523 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/it87_wdt.ko 2023-10-30T12:20:45.523 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/kempld_wdt.ko 2023-10-30T12:20:45.523 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/machzwd.ko 2023-10-30T12:20:45.523 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/mena21_wdt.ko 2023-10-30T12:20:45.524 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-10-30T12:20:45.524 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/nv_tco.ko 2023-10-30T12:20:45.524 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-10-30T12:20:45.524 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/pc87413_wdt.ko 2023-10-30T12:20:45.524 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-30T12:20:45.524 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-30T12:20:45.525 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/retu_wdt.ko 2023-10-30T12:20:45.525 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-10-30T12:20:45.525 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-10-30T12:20:45.525 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-30T12:20:45.525 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/sc1200wdt.ko 2023-10-30T12:20:45.526 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-30T12:20:45.526 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-10-30T12:20:45.526 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/softdog.ko 2023-10-30T12:20:45.526 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-30T12:20:45.526 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/twl4030_wdt.ko 2023-10-30T12:20:45.527 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/via_wdt.ko 2023-10-30T12:20:45.527 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-30T12:20:45.527 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-30T12:20:45.527 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-30T12:20:45.527 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/wafer5823wdt.ko 2023-10-30T12:20:45.528 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/wdt_pci.ko 2023-10-30T12:20:45.528 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/wm831x_wdt.ko 2023-10-30T12:20:45.528 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/wm8350_wdt.ko 2023-10-30T12:20:45.528 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/xen_wdt.ko 2023-10-30T12:20:45.528 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/ 2023-10-30T12:20:45.529 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xen-evtchn.ko 2023-10-30T12:20:45.529 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xen-gntalloc.ko 2023-10-30T12:20:45.529 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xen-gntdev.ko 2023-10-30T12:20:45.529 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xen-pciback/ 2023-10-30T12:20:45.529 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 176441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-30T12:20:45.530 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xen-privcmd.ko 2023-10-30T12:20:45.530 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 71249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xen-scsiback.ko 2023-10-30T12:20:45.530 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xenfs/ 2023-10-30T12:20:45.530 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-30T12:20:45.530 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ 2023-10-30T12:20:45.530 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/9p/ 2023-10-30T12:20:45.531 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 166425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/9p/9p.ko 2023-10-30T12:20:45.531 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/adfs/ 2023-10-30T12:20:45.531 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 61073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/adfs/adfs.ko 2023-10-30T12:20:45.531 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/affs/ 2023-10-30T12:20:45.532 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 163593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/affs/affs.ko 2023-10-30T12:20:45.532 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/afs/ 2023-10-30T12:20:45.532 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 932817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/afs/kafs.ko 2023-10-30T12:20:45.532 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/autofs/ 2023-10-30T12:20:45.532 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 101057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/autofs/autofs4.ko 2023-10-30T12:20:45.533 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/befs/ 2023-10-30T12:20:45.533 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 64545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/befs/befs.ko 2023-10-30T12:20:45.533 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/bfs/ 2023-10-30T12:20:45.533 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/bfs/bfs.ko 2023-10-30T12:20:45.533 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/binfmt_misc.ko 2023-10-30T12:20:45.534 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/btrfs/ 2023-10-30T12:20:45.534 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 3402585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/btrfs/btrfs.ko 2023-10-30T12:20:45.534 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/cachefiles/ 2023-10-30T12:20:45.534 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 306073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/cachefiles/cachefiles.ko 2023-10-30T12:20:45.534 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ceph/ 2023-10-30T12:20:45.535 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1323945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ceph/ceph.ko 2023-10-30T12:20:45.535 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/coda/ 2023-10-30T12:20:45.535 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 100801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/coda/coda.ko 2023-10-30T12:20:45.535 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/configfs/ 2023-10-30T12:20:45.535 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 108393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/configfs/configfs.ko 2023-10-30T12:20:45.535 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/cramfs/ 2023-10-30T12:20:45.536 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/cramfs/cramfs.ko 2023-10-30T12:20:45.536 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/dlm/ 2023-10-30T12:20:45.536 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 642577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/dlm/dlm.ko 2023-10-30T12:20:45.536 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/efs/ 2023-10-30T12:20:45.536 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/efs/efs.ko 2023-10-30T12:20:45.537 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/f2fs/ 2023-10-30T12:20:45.537 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1505025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/f2fs/f2fs.ko 2023-10-30T12:20:45.537 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/fat/ 2023-10-30T12:20:45.537 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/fat/msdos.ko 2023-10-30T12:20:45.537 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/freevxfs/ 2023-10-30T12:20:45.538 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/freevxfs/freevxfs.ko 2023-10-30T12:20:45.538 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/fscache/ 2023-10-30T12:20:45.538 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 226481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/fscache/fscache.ko 2023-10-30T12:20:45.538 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/fuse/ 2023-10-30T12:20:45.538 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/fuse/cuse.ko 2023-10-30T12:20:45.539 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/gfs2/ 2023-10-30T12:20:45.539 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1231529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/gfs2/gfs2.ko 2023-10-30T12:20:45.539 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/hfs/ 2023-10-30T12:20:45.539 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 141209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/hfs/hfs.ko 2023-10-30T12:20:45.539 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/hfsplus/ 2023-10-30T12:20:45.540 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 222817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/hfsplus/hfsplus.ko 2023-10-30T12:20:45.540 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/hpfs/ 2023-10-30T12:20:45.540 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 192329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/hpfs/hpfs.ko 2023-10-30T12:20:45.540 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/isofs/ 2023-10-30T12:20:45.540 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 74065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/isofs/isofs.ko 2023-10-30T12:20:45.541 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/jfs/ 2023-10-30T12:20:45.541 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 419497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/jfs/jfs.ko 2023-10-30T12:20:45.541 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/lockd/ 2023-10-30T12:20:45.541 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 280313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/lockd/lockd.ko 2023-10-30T12:20:45.541 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/minix/ 2023-10-30T12:20:45.541 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/minix/minix.ko 2023-10-30T12:20:45.542 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/netfs/ 2023-10-30T12:20:45.542 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 124649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/netfs/netfs.ko 2023-10-30T12:20:45.542 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/ 2023-10-30T12:20:45.542 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/blocklayout/ 2023-10-30T12:20:45.542 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 81449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-30T12:20:45.543 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/filelayout/ 2023-10-30T12:20:45.543 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 59449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-30T12:20:45.543 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/flexfilelayout/ 2023-10-30T12:20:45.543 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 117513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-30T12:20:45.543 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1055649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/nfs.ko 2023-10-30T12:20:45.544 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/nfsv2.ko 2023-10-30T12:20:45.544 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 120177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/nfsv3.ko 2023-10-30T12:20:45.544 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 2041689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/nfsv4.ko 2023-10-30T12:20:45.544 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs_common/ 2023-10-30T12:20:45.544 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs_common/grace.ko 2023-10-30T12:20:45.545 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs_common/nfs_acl.ko 2023-10-30T12:20:45.545 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfsd/ 2023-10-30T12:20:45.545 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1516809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfsd/nfsd.ko 2023-10-30T12:20:45.545 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nilfs2/ 2023-10-30T12:20:45.545 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 592833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nilfs2/nilfs2.ko 2023-10-30T12:20:45.546 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/ 2023-10-30T12:20:45.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-celtic.ko 2023-10-30T12:20:45.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-centeuro.ko 2023-10-30T12:20:45.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-croatian.ko 2023-10-30T12:20:45.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-cyrillic.ko 2023-10-30T12:20:45.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-gaelic.ko 2023-10-30T12:20:45.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-greek.ko 2023-10-30T12:20:45.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-iceland.ko 2023-10-30T12:20:45.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-inuit.ko 2023-10-30T12:20:45.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-roman.ko 2023-10-30T12:20:45.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-romanian.ko 2023-10-30T12:20:45.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-turkish.ko 2023-10-30T12:20:45.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_ascii.ko 2023-10-30T12:20:45.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp1250.ko 2023-10-30T12:20:45.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp1251.ko 2023-10-30T12:20:45.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp1255.ko 2023-10-30T12:20:45.549 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp737.ko 2023-10-30T12:20:45.549 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp775.ko 2023-10-30T12:20:45.549 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp850.ko 2023-10-30T12:20:45.549 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp852.ko 2023-10-30T12:20:45.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp855.ko 2023-10-30T12:20:45.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp857.ko 2023-10-30T12:20:45.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp860.ko 2023-10-30T12:20:45.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp861.ko 2023-10-30T12:20:45.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp862.ko 2023-10-30T12:20:45.551 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp863.ko 2023-10-30T12:20:45.551 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp864.ko 2023-10-30T12:20:45.551 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp865.ko 2023-10-30T12:20:45.551 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp866.ko 2023-10-30T12:20:45.551 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp869.ko 2023-10-30T12:20:45.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp874.ko 2023-10-30T12:20:45.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 94137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp932.ko 2023-10-30T12:20:45.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 142673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp936.ko 2023-10-30T12:20:45.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 165745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp949.ko 2023-10-30T12:20:45.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 118121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp950.ko 2023-10-30T12:20:45.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_euc-jp.ko 2023-10-30T12:20:45.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-1.ko 2023-10-30T12:20:45.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-13.ko 2023-10-30T12:20:45.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-14.ko 2023-10-30T12:20:45.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-15.ko 2023-10-30T12:20:45.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-2.ko 2023-10-30T12:20:45.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-3.ko 2023-10-30T12:20:45.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-4.ko 2023-10-30T12:20:45.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-5.ko 2023-10-30T12:20:45.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-6.ko 2023-10-30T12:20:45.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-7.ko 2023-10-30T12:20:45.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-9.ko 2023-10-30T12:20:45.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_koi8-r.ko 2023-10-30T12:20:45.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_koi8-ru.ko 2023-10-30T12:20:45.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_koi8-u.ko 2023-10-30T12:20:45.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-30T12:20:45.556 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_utf8.ko 2023-10-30T12:20:45.556 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ntfs/ 2023-10-30T12:20:45.556 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 219417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ntfs/ntfs.ko 2023-10-30T12:20:45.556 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/ 2023-10-30T12:20:45.556 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/cluster/ 2023-10-30T12:20:45.556 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 244409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-30T12:20:45.557 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/dlm/ 2023-10-30T12:20:45.557 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 440737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-30T12:20:45.557 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/dlmfs/ 2023-10-30T12:20:45.557 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-30T12:20:45.557 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 2592433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/ocfs2.ko 2023-10-30T12:20:45.558 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-30T12:20:45.558 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-30T12:20:45.558 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-30T12:20:45.558 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/omfs/ 2023-10-30T12:20:45.558 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/omfs/omfs.ko 2023-10-30T12:20:45.559 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/overlayfs/ 2023-10-30T12:20:45.559 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 352153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/overlayfs/overlay.ko 2023-10-30T12:20:45.559 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/pstore/ 2023-10-30T12:20:45.559 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/pstore/ramoops.ko 2023-10-30T12:20:45.559 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/qnx4/ 2023-10-30T12:20:45.560 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/qnx4/qnx4.ko 2023-10-30T12:20:45.560 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/qnx6/ 2023-10-30T12:20:45.560 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/qnx6/qnx6.ko 2023-10-30T12:20:45.560 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/quota/ 2023-10-30T12:20:45.560 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/quota/quota_tree.ko 2023-10-30T12:20:45.561 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/quota/quota_v1.ko 2023-10-30T12:20:45.561 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/quota/quota_v2.ko 2023-10-30T12:20:45.561 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/reiserfs/ 2023-10-30T12:20:45.561 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 532673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/reiserfs/reiserfs.ko 2023-10-30T12:20:45.561 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/romfs/ 2023-10-30T12:20:45.562 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/romfs/romfs.ko 2023-10-30T12:20:45.562 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/smb/ 2023-10-30T12:20:45.562 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/smb/client/ 2023-10-30T12:20:45.562 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 2668121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/smb/client/cifs.ko 2023-10-30T12:20:45.562 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/smb/common/ 2023-10-30T12:20:45.562 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/smb/common/cifs_arc4.ko 2023-10-30T12:20:45.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/smb/common/cifs_md4.ko 2023-10-30T12:20:45.563 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/squashfs/ 2023-10-30T12:20:45.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 134641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/squashfs/squashfs.ko 2023-10-30T12:20:45.563 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/sysv/ 2023-10-30T12:20:45.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 90721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/sysv/sysv.ko 2023-10-30T12:20:45.564 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/udf/ 2023-10-30T12:20:45.564 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 247737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/udf/udf.ko 2023-10-30T12:20:45.564 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ufs/ 2023-10-30T12:20:45.564 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 162497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ufs/ufs.ko 2023-10-30T12:20:45.564 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/xfs/ 2023-10-30T12:20:45.565 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 4010729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/xfs/xfs.ko 2023-10-30T12:20:45.565 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/kernel/ 2023-10-30T12:20:45.565 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/kernel/locking/ 2023-10-30T12:20:45.565 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 75489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/kernel/locking/locktorture.ko 2023-10-30T12:20:45.565 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/kernel/time/ 2023-10-30T12:20:45.566 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/kernel/time/test_udelay.ko 2023-10-30T12:20:45.566 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/kernel/torture.ko 2023-10-30T12:20:45.566 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/ 2023-10-30T12:20:45.566 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 6833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/crc-itu-t.ko 2023-10-30T12:20:45.567 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 6529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/crc7.ko 2023-10-30T12:20:45.567 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/crc8.ko 2023-10-30T12:20:45.567 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/crypto/ 2023-10-30T12:20:45.567 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/crypto/libarc4.ko 2023-10-30T12:20:45.567 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/crypto/libdes.ko 2023-10-30T12:20:45.568 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/interval_tree_test.ko 2023-10-30T12:20:45.568 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/libcrc32c.ko 2023-10-30T12:20:45.568 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/lru_cache.ko 2023-10-30T12:20:45.568 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/lz4/ 2023-10-30T12:20:45.568 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/lz4/lz4_compress.ko 2023-10-30T12:20:45.569 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/lz4/lz4hc_compress.ko 2023-10-30T12:20:45.569 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/math/ 2023-10-30T12:20:45.569 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 6617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/math/cordic.ko 2023-10-30T12:20:45.569 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/memory-notifier-error-inject.ko 2023-10-30T12:20:45.569 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/notifier-error-inject.ko 2023-10-30T12:20:45.570 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/percpu_test.ko 2023-10-30T12:20:45.570 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/pm-notifier-error-inject.ko 2023-10-30T12:20:45.570 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/raid6/ 2023-10-30T12:20:45.570 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 143433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/raid6/raid6_pq.ko 2023-10-30T12:20:45.570 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/rbtree_test.ko 2023-10-30T12:20:45.570 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/reed_solomon/ 2023-10-30T12:20:45.571 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-30T12:20:45.571 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/test-kstrtox.ko 2023-10-30T12:20:45.571 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/test-string_helpers.ko 2023-10-30T12:20:45.571 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 4760425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/test_bpf.ko 2023-10-30T12:20:45.571 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/test_firmware.ko 2023-10-30T12:20:45.572 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 6369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/test_module.ko 2023-10-30T12:20:45.572 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/test_user_copy.ko 2023-10-30T12:20:45.572 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/ts_bm.ko 2023-10-30T12:20:45.572 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/ts_fsm.ko 2023-10-30T12:20:45.572 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/ts_kmp.ko 2023-10-30T12:20:45.573 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/xz/ 2023-10-30T12:20:45.573 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/xz/xz_dec_test.ko 2023-10-30T12:20:45.573 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/mm/ 2023-10-30T12:20:45.573 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/mm/hwpoison-inject.ko 2023-10-30T12:20:45.573 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ 2023-10-30T12:20:45.574 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/ 2023-10-30T12:20:45.574 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 55737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/6lowpan.ko 2023-10-30T12:20:45.574 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/nhc_dest.ko 2023-10-30T12:20:45.574 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/nhc_fragment.ko 2023-10-30T12:20:45.574 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/nhc_hop.ko 2023-10-30T12:20:45.575 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 6985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-30T12:20:45.575 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/nhc_mobility.ko 2023-10-30T12:20:45.575 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/nhc_routing.ko 2023-10-30T12:20:45.575 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/nhc_udp.ko 2023-10-30T12:20:45.575 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/802/ 2023-10-30T12:20:45.576 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/802/garp.ko 2023-10-30T12:20:45.576 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/802/mrp.ko 2023-10-30T12:20:45.576 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/802/p8022.ko 2023-10-30T12:20:45.576 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/802/psnap.ko 2023-10-30T12:20:45.576 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/802/stp.ko 2023-10-30T12:20:45.577 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/8021q/ 2023-10-30T12:20:45.577 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 81737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/8021q/8021q.ko 2023-10-30T12:20:45.577 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/9p/ 2023-10-30T12:20:45.577 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 198337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/9p/9pnet.ko 2023-10-30T12:20:45.577 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/9p/9pnet_fd.ko 2023-10-30T12:20:45.578 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/9p/9pnet_rdma.ko 2023-10-30T12:20:45.578 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/9p/9pnet_virtio.ko 2023-10-30T12:20:45.578 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/batman-adv/ 2023-10-30T12:20:45.578 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 649297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/batman-adv/batman-adv.ko 2023-10-30T12:20:45.578 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/ 2023-10-30T12:20:45.579 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 63809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/br_netfilter.ko 2023-10-30T12:20:45.579 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 777689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/bridge.ko 2023-10-30T12:20:45.579 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ 2023-10-30T12:20:45.579 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-30T12:20:45.579 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-30T12:20:45.579 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-30T12:20:45.580 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-30T12:20:45.580 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-30T12:20:45.580 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-30T12:20:45.580 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-30T12:20:45.581 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-30T12:20:45.581 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-30T12:20:45.581 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-30T12:20:45.581 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-30T12:20:45.581 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-30T12:20:45.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-30T12:20:45.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-30T12:20:45.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-30T12:20:45.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-30T12:20:45.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-30T12:20:45.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-30T12:20:45.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-30T12:20:45.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-30T12:20:45.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 55929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebtables.ko 2023-10-30T12:20:45.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-30T12:20:45.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-30T12:20:45.584 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ceph/ 2023-10-30T12:20:45.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1047497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ceph/libceph.ko 2023-10-30T12:20:45.584 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/core/ 2023-10-30T12:20:45.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 125697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/core/pktgen.ko 2023-10-30T12:20:45.584 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dccp/ 2023-10-30T12:20:45.585 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 194209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dccp/dccp.ko 2023-10-30T12:20:45.585 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dccp/dccp_diag.ko 2023-10-30T12:20:45.585 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dccp/dccp_ipv4.ko 2023-10-30T12:20:45.585 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dccp/dccp_ipv6.ko 2023-10-30T12:20:45.585 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dsa/ 2023-10-30T12:20:45.586 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 390473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dsa/dsa_core.ko 2023-10-30T12:20:45.586 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dsa/tag_brcm.ko 2023-10-30T12:20:45.586 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dsa/tag_dsa.ko 2023-10-30T12:20:45.586 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dsa/tag_none.ko 2023-10-30T12:20:45.586 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dsa/tag_trailer.ko 2023-10-30T12:20:45.586 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/hsr/ 2023-10-30T12:20:45.587 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 99729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/hsr/hsr.ko 2023-10-30T12:20:45.587 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ieee802154/ 2023-10-30T12:20:45.587 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ieee802154/6lowpan/ 2023-10-30T12:20:45.587 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 45233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-30T12:20:45.587 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 254625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ieee802154/ieee802154.ko 2023-10-30T12:20:45.588 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-30T12:20:45.588 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/ 2023-10-30T12:20:45.588 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/ah4.ko 2023-10-30T12:20:45.588 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/esp4.ko 2023-10-30T12:20:45.588 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 48401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/fou.ko 2023-10-30T12:20:45.589 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/gre.ko 2023-10-30T12:20:45.589 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 48657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/inet_diag.ko 2023-10-30T12:20:45.589 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/ip_gre.ko 2023-10-30T12:20:45.589 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 61225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/ip_tunnel.ko 2023-10-30T12:20:45.589 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/ip_vti.ko 2023-10-30T12:20:45.590 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/ipcomp.ko 2023-10-30T12:20:45.590 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/ipip.ko 2023-10-30T12:20:45.590 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/ 2023-10-30T12:20:45.590 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-30T12:20:45.590 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-30T12:20:45.591 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-30T12:20:45.591 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/ip_tables.ko 2023-10-30T12:20:45.591 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-30T12:20:45.591 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-10-30T12:20:45.591 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-30T12:20:45.591 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-30T12:20:45.592 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-30T12:20:45.592 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-10-30T12:20:45.592 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-30T12:20:45.592 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-30T12:20:45.592 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-30T12:20:45.593 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-30T12:20:45.593 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-30T12:20:45.593 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-30T12:20:45.593 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-30T12:20:45.593 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-30T12:20:45.594 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-30T12:20:45.594 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-10-30T12:20:45.594 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-30T12:20:45.594 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-30T12:20:45.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-30T12:20:45.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-30T12:20:45.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-30T12:20:45.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_bic.ko 2023-10-30T12:20:45.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_dctcp.ko 2023-10-30T12:20:45.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_diag.ko 2023-10-30T12:20:45.596 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_highspeed.ko 2023-10-30T12:20:45.596 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_htcp.ko 2023-10-30T12:20:45.596 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_hybla.ko 2023-10-30T12:20:45.596 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_illinois.ko 2023-10-30T12:20:45.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_lp.ko 2023-10-30T12:20:45.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_scalable.ko 2023-10-30T12:20:45.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_vegas.ko 2023-10-30T12:20:45.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_veno.ko 2023-10-30T12:20:45.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_westwood.ko 2023-10-30T12:20:45.598 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_yeah.ko 2023-10-30T12:20:45.598 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tunnel4.ko 2023-10-30T12:20:45.598 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/udp_diag.ko 2023-10-30T12:20:45.598 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/udp_tunnel.ko 2023-10-30T12:20:45.598 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-30T12:20:45.599 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/ 2023-10-30T12:20:45.599 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/ah6.ko 2023-10-30T12:20:45.599 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/esp6.ko 2023-10-30T12:20:45.599 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/fou6.ko 2023-10-30T12:20:45.599 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 80897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/ip6_gre.ko 2023-10-30T12:20:45.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 93577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/ip6_tunnel.ko 2023-10-30T12:20:45.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-30T12:20:45.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/ip6_vti.ko 2023-10-30T12:20:45.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/ipcomp6.ko 2023-10-30T12:20:45.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/mip6.ko 2023-10-30T12:20:45.601 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ 2023-10-30T12:20:45.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-30T12:20:45.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-10-30T12:20:45.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-30T12:20:45.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-30T12:20:45.602 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-30T12:20:45.602 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-30T12:20:45.602 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-30T12:20:45.602 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-30T12:20:45.602 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-30T12:20:45.603 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-30T12:20:45.603 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-30T12:20:45.603 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-30T12:20:45.603 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-30T12:20:45.603 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-30T12:20:45.604 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-30T12:20:45.604 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-30T12:20:45.604 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-30T12:20:45.604 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-30T12:20:45.605 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-30T12:20:45.605 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-30T12:20:45.605 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-30T12:20:45.605 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-30T12:20:45.605 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-30T12:20:45.606 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-30T12:20:45.606 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-30T12:20:45.606 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 74073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/sit.ko 2023-10-30T12:20:45.606 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/tunnel6.ko 2023-10-30T12:20:45.606 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-30T12:20:45.606 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/key/ 2023-10-30T12:20:45.607 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 82833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/key/af_key.ko 2023-10-30T12:20:45.607 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/ 2023-10-30T12:20:45.607 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 125281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/l2tp_core.ko 2023-10-30T12:20:45.607 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-30T12:20:45.607 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/l2tp_eth.ko 2023-10-30T12:20:45.608 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/l2tp_ip.ko 2023-10-30T12:20:45.608 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/l2tp_ip6.ko 2023-10-30T12:20:45.608 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/l2tp_netlink.ko 2023-10-30T12:20:45.608 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/l2tp_ppp.ko 2023-10-30T12:20:45.608 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/lapb/ 2023-10-30T12:20:45.609 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/lapb/lapb.ko 2023-10-30T12:20:45.609 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/llc/ 2023-10-30T12:20:45.609 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/llc/llc.ko 2023-10-30T12:20:45.609 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 267777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/llc/llc2.ko 2023-10-30T12:20:45.609 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/mac802154/ 2023-10-30T12:20:45.609 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 290417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/mac802154/mac802154.ko 2023-10-30T12:20:45.610 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/mpls/ 2023-10-30T12:20:45.610 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/mpls/mpls_gso.ko 2023-10-30T12:20:45.610 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ 2023-10-30T12:20:45.610 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ 2023-10-30T12:20:45.610 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 119585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set.ko 2023-10-30T12:20:45.611 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-30T12:20:45.611 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-30T12:20:45.611 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-30T12:20:45.611 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 100297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-30T12:20:45.611 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 101441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-30T12:20:45.611 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 102817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-30T12:20:45.612 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 103481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-30T12:20:45.612 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 112849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-30T12:20:45.612 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-30T12:20:45.612 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 109593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-30T12:20:45.612 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 113129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-30T12:20:45.613 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 112585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-30T12:20:45.613 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 111585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-30T12:20:45.613 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 114465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-30T12:20:45.613 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-30T12:20:45.613 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ 2023-10-30T12:20:45.614 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 411729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-30T12:20:45.614 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-30T12:20:45.614 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-30T12:20:45.614 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-30T12:20:45.614 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-30T12:20:45.615 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-30T12:20:45.615 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-30T12:20:45.615 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-30T12:20:45.615 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-30T12:20:45.615 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-30T12:20:45.616 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-30T12:20:45.616 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-30T12:20:45.616 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-30T12:20:45.616 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-30T12:20:45.616 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conncount.ko 2023-10-30T12:20:45.617 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 408257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack.ko 2023-10-30T12:20:45.617 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-30T12:20:45.617 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-30T12:20:45.617 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-30T12:20:45.617 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 118145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-30T12:20:45.618 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-30T12:20:45.618 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-30T12:20:45.618 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 107297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-30T12:20:45.618 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-30T12:20:45.618 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-30T12:20:45.619 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-30T12:20:45.619 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-30T12:20:45.619 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-30T12:20:45.619 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-30T12:20:45.619 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_flow_table.ko 2023-10-30T12:20:45.620 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-30T12:20:45.620 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_log_syslog.ko 2023-10-30T12:20:45.620 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 95417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_nat.ko 2023-10-30T12:20:45.620 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-30T12:20:45.620 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-30T12:20:45.621 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_nat_irc.ko 2023-10-30T12:20:45.621 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_nat_sip.ko 2023-10-30T12:20:45.621 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-30T12:20:45.621 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-30T12:20:45.621 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 615313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_tables.ko 2023-10-30T12:20:45.622 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nfnetlink.ko 2023-10-30T12:20:45.622 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-30T12:20:45.622 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-10-30T12:20:45.622 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nfnetlink_log.ko 2023-10-30T12:20:45.622 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-30T12:20:45.622 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-30T12:20:45.623 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_chain_nat.ko 2023-10-30T12:20:45.623 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_compat.ko 2023-10-30T12:20:45.623 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_ct.ko 2023-10-30T12:20:45.623 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-30T12:20:45.623 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_fib.ko 2023-10-30T12:20:45.624 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-30T12:20:45.624 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_hash.ko 2023-10-30T12:20:45.624 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_limit.ko 2023-10-30T12:20:45.624 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_log.ko 2023-10-30T12:20:45.624 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_masq.ko 2023-10-30T12:20:45.625 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_nat.ko 2023-10-30T12:20:45.625 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_numgen.ko 2023-10-30T12:20:45.625 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_osf.ko 2023-10-30T12:20:45.625 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_queue.ko 2023-10-30T12:20:45.625 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_redir.ko 2023-10-30T12:20:45.625 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_reject.ko 2023-10-30T12:20:45.626 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_reject_inet.ko 2023-10-30T12:20:45.626 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_socket.ko 2023-10-30T12:20:45.626 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_synproxy.ko 2023-10-30T12:20:45.626 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_tproxy.ko 2023-10-30T12:20:45.626 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/x_tables.ko 2023-10-30T12:20:45.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_AUDIT.ko 2023-10-30T12:20:45.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-30T12:20:45.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-30T12:20:45.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-30T12:20:45.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_CT.ko 2023-10-30T12:20:45.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_DSCP.ko 2023-10-30T12:20:45.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_HL.ko 2023-10-30T12:20:45.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_HMARK.ko 2023-10-30T12:20:45.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-30T12:20:45.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_LED.ko 2023-10-30T12:20:45.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_LOG.ko 2023-10-30T12:20:45.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-30T12:20:45.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_NETMAP.ko 2023-10-30T12:20:45.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_NFLOG.ko 2023-10-30T12:20:45.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-30T12:20:45.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_RATEEST.ko 2023-10-30T12:20:45.630 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-30T12:20:45.630 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_SECMARK.ko 2023-10-30T12:20:45.630 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-30T12:20:45.630 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-30T12:20:45.630 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_TEE.ko 2023-10-30T12:20:45.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_TPROXY.ko 2023-10-30T12:20:45.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_TRACE.ko 2023-10-30T12:20:45.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_addrtype.ko 2023-10-30T12:20:45.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_bpf.ko 2023-10-30T12:20:45.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_cgroup.ko 2023-10-30T12:20:45.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_cluster.ko 2023-10-30T12:20:45.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_comment.ko 2023-10-30T12:20:45.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_connbytes.ko 2023-10-30T12:20:45.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_connlabel.ko 2023-10-30T12:20:45.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_connlimit.ko 2023-10-30T12:20:45.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_connmark.ko 2023-10-30T12:20:45.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_conntrack.ko 2023-10-30T12:20:45.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_cpu.ko 2023-10-30T12:20:45.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_dccp.ko 2023-10-30T12:20:45.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_devgroup.ko 2023-10-30T12:20:45.634 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_dscp.ko 2023-10-30T12:20:45.634 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_ecn.ko 2023-10-30T12:20:45.634 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_esp.ko 2023-10-30T12:20:45.634 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_hashlimit.ko 2023-10-30T12:20:45.634 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_helper.ko 2023-10-30T12:20:45.635 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_hl.ko 2023-10-30T12:20:45.635 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_ipcomp.ko 2023-10-30T12:20:45.635 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_iprange.ko 2023-10-30T12:20:45.635 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_ipvs.ko 2023-10-30T12:20:45.635 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_l2tp.ko 2023-10-30T12:20:45.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_length.ko 2023-10-30T12:20:45.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_limit.ko 2023-10-30T12:20:45.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_mac.ko 2023-10-30T12:20:45.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_mark.ko 2023-10-30T12:20:45.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_multiport.ko 2023-10-30T12:20:45.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_nat.ko 2023-10-30T12:20:45.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_nfacct.ko 2023-10-30T12:20:45.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_osf.ko 2023-10-30T12:20:45.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_owner.ko 2023-10-30T12:20:45.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_physdev.ko 2023-10-30T12:20:45.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_pkttype.ko 2023-10-30T12:20:45.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_policy.ko 2023-10-30T12:20:45.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_quota.ko 2023-10-30T12:20:45.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_rateest.ko 2023-10-30T12:20:45.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_realm.ko 2023-10-30T12:20:45.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_recent.ko 2023-10-30T12:20:45.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_sctp.ko 2023-10-30T12:20:45.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_set.ko 2023-10-30T12:20:45.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_socket.ko 2023-10-30T12:20:45.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_state.ko 2023-10-30T12:20:45.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_statistic.ko 2023-10-30T12:20:45.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_string.ko 2023-10-30T12:20:45.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_tcpmss.ko 2023-10-30T12:20:45.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_tcpudp.ko 2023-10-30T12:20:45.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_time.ko 2023-10-30T12:20:45.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_u32.ko 2023-10-30T12:20:45.641 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netlink/ 2023-10-30T12:20:45.641 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netlink/netlink_diag.ko 2023-10-30T12:20:45.641 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/nsh/ 2023-10-30T12:20:45.641 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/nsh/nsh.ko 2023-10-30T12:20:45.641 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/openvswitch/ 2023-10-30T12:20:45.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 380033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/openvswitch/openvswitch.ko 2023-10-30T12:20:45.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/openvswitch/vport-geneve.ko 2023-10-30T12:20:45.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/openvswitch/vport-gre.ko 2023-10-30T12:20:45.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/openvswitch/vport-vxlan.ko 2023-10-30T12:20:45.642 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/packet/ 2023-10-30T12:20:45.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/packet/af_packet_diag.ko 2023-10-30T12:20:45.643 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/phonet/ 2023-10-30T12:20:45.643 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 110201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/phonet/phonet.ko 2023-10-30T12:20:45.643 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/phonet/pn_pep.ko 2023-10-30T12:20:45.643 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/rds/ 2023-10-30T12:20:45.643 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 251481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/rds/rds.ko 2023-10-30T12:20:45.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 164937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/rds/rds_rdma.ko 2023-10-30T12:20:45.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/rds/rds_tcp.ko 2023-10-30T12:20:45.644 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/rxrpc/ 2023-10-30T12:20:45.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 819657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/rxrpc/rxrpc.ko 2023-10-30T12:20:45.644 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/ 2023-10-30T12:20:45.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_csum.ko 2023-10-30T12:20:45.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_gact.ko 2023-10-30T12:20:45.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_ipt.ko 2023-10-30T12:20:45.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_mirred.ko 2023-10-30T12:20:45.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_nat.ko 2023-10-30T12:20:45.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_pedit.ko 2023-10-30T12:20:45.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_police.ko 2023-10-30T12:20:45.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_simple.ko 2023-10-30T12:20:45.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_skbedit.ko 2023-10-30T12:20:45.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_vlan.ko 2023-10-30T12:20:45.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/cls_basic.ko 2023-10-30T12:20:45.647 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/cls_bpf.ko 2023-10-30T12:20:45.647 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/cls_cgroup.ko 2023-10-30T12:20:45.647 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/cls_flow.ko 2023-10-30T12:20:45.647 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/cls_fw.ko 2023-10-30T12:20:45.647 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/cls_route.ko 2023-10-30T12:20:45.647 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 61409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/cls_u32.ko 2023-10-30T12:20:45.648 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/em_cmp.ko 2023-10-30T12:20:45.648 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/em_ipset.ko 2023-10-30T12:20:45.648 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/em_meta.ko 2023-10-30T12:20:45.648 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/em_nbyte.ko 2023-10-30T12:20:45.648 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/em_text.ko 2023-10-30T12:20:45.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/em_u32.ko 2023-10-30T12:20:45.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_choke.ko 2023-10-30T12:20:45.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_codel.ko 2023-10-30T12:20:45.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_drr.ko 2023-10-30T12:20:45.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_fq.ko 2023-10-30T12:20:45.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_fq_codel.ko 2023-10-30T12:20:45.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_gred.ko 2023-10-30T12:20:45.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_hfsc.ko 2023-10-30T12:20:45.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_hhf.ko 2023-10-30T12:20:45.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_htb.ko 2023-10-30T12:20:45.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_ingress.ko 2023-10-30T12:20:45.651 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_mqprio.ko 2023-10-30T12:20:45.651 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_mqprio_lib.ko 2023-10-30T12:20:45.651 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_multiq.ko 2023-10-30T12:20:45.651 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_netem.ko 2023-10-30T12:20:45.651 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_pie.ko 2023-10-30T12:20:45.652 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_plug.ko 2023-10-30T12:20:45.652 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_prio.ko 2023-10-30T12:20:45.652 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 48793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_qfq.ko 2023-10-30T12:20:45.652 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_red.ko 2023-10-30T12:20:45.652 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_sfb.ko 2023-10-30T12:20:45.652 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_sfq.ko 2023-10-30T12:20:45.653 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_tbf.ko 2023-10-30T12:20:45.653 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_teql.ko 2023-10-30T12:20:45.653 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sctp/ 2023-10-30T12:20:45.653 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 929745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sctp/sctp.ko 2023-10-30T12:20:45.653 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sctp/sctp_diag.ko 2023-10-30T12:20:45.654 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sunrpc/ 2023-10-30T12:20:45.654 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sunrpc/auth_gss/ 2023-10-30T12:20:45.654 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 336529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-30T12:20:45.654 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-30T12:20:45.654 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1520161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sunrpc/sunrpc.ko 2023-10-30T12:20:45.655 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sunrpc/xprtrdma/ 2023-10-30T12:20:45.655 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 707721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-30T12:20:45.655 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/tipc/ 2023-10-30T12:20:45.655 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/tipc/diag.ko 2023-10-30T12:20:45.655 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 862641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/tipc/tipc.ko 2023-10-30T12:20:45.655 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/unix/ 2023-10-30T12:20:45.656 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/unix/unix_diag.ko 2023-10-30T12:20:45.656 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/vmw_vsock/ 2023-10-30T12:20:45.656 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 85337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-30T12:20:45.656 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 68049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-30T12:20:45.656 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 96193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/vmw_vsock/vsock.ko 2023-10-30T12:20:45.656 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10705 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-30T12:20:45.657 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-30T12:20:45.657 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/x25/ 2023-10-30T12:20:45.657 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 125177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/x25/x25.ko 2023-10-30T12:20:45.657 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/xfrm/ 2023-10-30T12:20:45.657 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/xfrm/xfrm_algo.ko 2023-10-30T12:20:45.658 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-30T12:20:45.658 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 102257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/xfrm/xfrm_user.ko 2023-10-30T12:20:45.658 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/virt/ 2023-10-30T12:20:45.658 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/virt/lib/ 2023-10-30T12:20:45.658 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/virt/lib/irqbypass.ko 2023-10-30T12:20:45.658 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 639863 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.alias 2023-10-30T12:20:45.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 623919 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.alias.bin 2023-10-30T12:20:45.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7663 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.builtin 2023-10-30T12:20:45.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23428 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.builtin.alias.bin 2023-10-30T12:20:45.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9821 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.builtin.bin 2023-10-30T12:20:45.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70004 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.builtin.modinfo 2023-10-30T12:20:45.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 146313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.dep 2023-10-30T12:20:45.660 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 232548 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.dep.bin 2023-10-30T12:20:45.660 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 210 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.devname 2023-10-30T12:20:45.660 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 72606 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.order 2023-10-30T12:20:45.660 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 879 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.softdep 2023-10-30T12:20:45.660 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 254438 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.symbols 2023-10-30T12:20:45.661 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 309363 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.symbols.bin 2023-10-30T12:20:45.661 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./usr/ 2023-10-30T12:20:45.661 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./usr/share/ 2023-10-30T12:20:45.661 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./usr/share/doc/ 2023-10-30T12:20:45.661 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./usr/share/doc/linux-image-6.6.0-rc5-geacf00cc51e2/ 2023-10-30T12:20:45.662 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 173 2023-10-30 03:46 ./usr/share/doc/linux-image-6.6.0-rc5-geacf00cc51e2/changelog.Debian.gz 2023-10-30T12:20:45.662 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 691 2023-10-30 03:46 ./usr/share/doc/linux-image-6.6.0-rc5-geacf00cc51e2/copyright 2023-10-30T12:20:45.663 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-geacf00cc51e2 2023-10-30T12:20:45.663 INFO:teuthology.task.kernel:searching for kernel 6.6.0-rc5-geacf00cc51e2 2023-10-30T12:20:45.663 DEBUG:teuthology.orchestra.run.smithi012:> egrep '(submenu|menuentry.*6.6.0-rc5-geacf00cc51e2).*{' /boot/grub/grub.cfg 2023-10-30T12:20:45.671 INFO:teuthology.orchestra.run.smithi012.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-30T12:20:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-geacf00cc51e2' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-geacf00cc51e2-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-30T12:20:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-geacf00cc51e2 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-geacf00cc51e2-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-30T12:20:45.672 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-10-30T12:20:45.672 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-rc5-geacf00cc51e2 2023-10-30T12:20:45.672 DEBUG:teuthology.orchestra.run.smithi012:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-10-30T12:20:45.715 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi012.front.sepia.ceph.com to complete... 2023-10-30T12:20:46.128 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub' 2023-10-30T12:20:46.129 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-30T12:20:46.150 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2023-10-30T12:20:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:20:46.345 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:20:46.563 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-30T12:20:46.569 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-30T12:20:46.612 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-30T12:20:46.617 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-30T12:20:46.783 INFO:teuthology.orchestra.run.smithi012.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-30T12:20:46.783 INFO:teuthology.orchestra.run.smithi012.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-30T12:20:46.783 INFO:teuthology.orchestra.run.smithi012.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-30T12:20:46.800 INFO:teuthology.orchestra.run.smithi012.stderr:done 2023-10-30T12:21:17.943 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-30T12:21:17.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-10-30T12:21:17.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T12:21:36.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-10-30T12:21:45.512 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-10-30T12:21:45.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T12:22:03.898 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-10-30T12:22:15.900 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-10-30T12:22:15.901 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T12:22:17.951 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-10-30T12:22:32.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-10-30T12:22:32.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T12:22:33.195 DEBUG:teuthology.orchestra.run.smithi012:> true 2023-10-30T12:22:33.959 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-10-30T12:22:33.960 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-geacf00cc51e2"... 2023-10-30T12:22:33.960 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2023-10-30T12:22:34.013 INFO:teuthology.orchestra.run.smithi012.stdout:6.6.0-rc5-geacf00cc51e2 2023-10-30T12:22:34.013 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-geacf00cc51e2 vs 6.6.0-rc5-geacf00cc51e2 2023-10-30T12:22:34.014 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-30T12:22:34.014 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-30T12:22:35.015 DEBUG:teuthology.parallel:result is None 2023-10-30T12:22:35.015 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-30T12:22:35.025 INFO:teuthology.task.internal:Creating test directory... 2023-10-30T12:22:35.026 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-30T12:22:35.030 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-30T12:22:35.037 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-30T12:22:35.067 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-30T12:22:35.117 INFO:teuthology.task.internal:Creating archive directory... 2023-10-30T12:22:35.118 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-30T12:22:35.122 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-30T12:22:35.144 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-30T12:22:35.167 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-30T12:22:35.168 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-30T12:22:35.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:22:35.180 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-30T12:22:35.185 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-30T12:22:35.243 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-30T12:22:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-30T12:22:35.256 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-30T12:22:35.281 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-30T12:22:35.283 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-30T12:22:35.290 INFO:teuthology.task.internal:Configuring sudo... 2023-10-30T12:22:35.291 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-30T12:22:35.295 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-30T12:22:35.316 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-30T12:22:35.371 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-30T12:22:35.372 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-30T12:22:35.376 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-30T12:22:35.382 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-30T12:22:35.431 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-30T12:22:35.483 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:22:35.484 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-30T12:22:35.556 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-30T12:22:35.561 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-30T12:22:35.612 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:22:35.613 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-30T12:22:35.681 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2023-10-30T12:22:35.686 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2023-10-30T12:22:35.791 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-30T12:22:35.799 INFO:teuthology.task.internal:Starting timer... 2023-10-30T12:22:35.800 INFO:teuthology.run_tasks:Running task pcp... 2023-10-30T12:22:35.849 INFO:teuthology.run_tasks:Running task selinux... 2023-10-30T12:22:35.895 DEBUG:teuthology.task.selinux:Excluding smithi012: OS 'ubuntu' does not support SELinux 2023-10-30T12:22:35.896 DEBUG:teuthology.task.selinux:Excluding smithi154: OS 'ubuntu' does not support SELinux 2023-10-30T12:22:35.896 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-30T12:22:35.897 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-30T12:22:35.897 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-30T12:22:35.897 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-30T12:22:35.945 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-30T12:22:35.951 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-30T12:22:36.088 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-30T12:22:36.104 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-30T12:22:36.105 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi154.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-30T12:31:14.131 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi154.front.sepia.ceph.com')] 2023-10-30T12:31:14.132 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-10-30T12:31:14.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T12:31:14.216 DEBUG:teuthology.orchestra.run.smithi012:> true 2023-10-30T12:31:14.333 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-10-30T12:31:14.334 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-30T12:31:14.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T12:31:14.412 DEBUG:teuthology.orchestra.run.smithi154:> true 2023-10-30T12:31:14.506 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-30T12:31:14.507 INFO:teuthology.run_tasks:Running task clock... 2023-10-30T12:31:14.517 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-30T12:31:14.518 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-30T12:31:14.518 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-30T12:31:14.523 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-30T12:31:14.523 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-30T12:31:14.560 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-10-30T12:31:14.560 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: Command line: ntpd -gq 2023-10-30T12:31:14.560 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: ---------------------------------------------------- 2023-10-30T12:31:14.561 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: ntp-4 is maintained by Network Time Foundation, 2023-10-30T12:31:14.561 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-10-30T12:31:14.561 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: corporation. Support and training for ntp-4 are 2023-10-30T12:31:14.561 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: available at https://www.nwtime.org/support 2023-10-30T12:31:14.561 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: ---------------------------------------------------- 2023-10-30T12:31:14.562 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: proto: precision = 0.044 usec (-24) 2023-10-30T12:31:14.562 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: basedate set to 2022-02-04 2023-10-30T12:31:14.563 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: gps base set to 2022-02-06 (week 2196) 2023-10-30T12:31:14.563 INFO:teuthology.orchestra.run.smithi012.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-30T12:31:14.564 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-30T12:31:14.564 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: restrict ::: KOD does nothing without LIMITED. 2023-10-30T12:31:14.564 INFO:teuthology.orchestra.run.smithi012.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-30T12:31:14.566 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: Listen and drop on 0 v6wildcard [::]:123 2023-10-30T12:31:14.566 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-30T12:31:14.566 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: Listen normally on 2 lo 127.0.0.1:123 2023-10-30T12:31:14.566 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: Listen normally on 3 ens1f0 172.21.15.12:123 2023-10-30T12:31:14.567 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: Listen normally on 4 lo [::1]:123 2023-10-30T12:31:14.567 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:25ea%4]:123 2023-10-30T12:31:14.567 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:14 ntpd[27870]: Listening on routing socket on fd #22 for interface updates 2023-10-30T12:31:14.574 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-10-30T12:31:14.574 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: Command line: ntpd -gq 2023-10-30T12:31:14.574 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: ---------------------------------------------------- 2023-10-30T12:31:14.575 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: ntp-4 is maintained by Network Time Foundation, 2023-10-30T12:31:14.575 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-10-30T12:31:14.575 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: corporation. Support and training for ntp-4 are 2023-10-30T12:31:14.575 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: available at https://www.nwtime.org/support 2023-10-30T12:31:14.576 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: ---------------------------------------------------- 2023-10-30T12:31:14.576 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: proto: precision = 0.045 usec (-24) 2023-10-30T12:31:14.576 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: basedate set to 2022-02-04 2023-10-30T12:31:14.577 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: gps base set to 2022-02-06 (week 2196) 2023-10-30T12:31:14.577 INFO:teuthology.orchestra.run.smithi154.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-30T12:31:14.577 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-30T12:31:14.578 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: restrict ::: KOD does nothing without LIMITED. 2023-10-30T12:31:14.578 INFO:teuthology.orchestra.run.smithi154.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-30T12:31:14.579 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: Listen and drop on 0 v6wildcard [::]:123 2023-10-30T12:31:14.579 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-30T12:31:14.579 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: Listen normally on 2 lo 127.0.0.1:123 2023-10-30T12:31:14.580 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: Listen normally on 3 enp3s0f1 172.21.15.154:123 2023-10-30T12:31:14.580 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: Listen normally on 4 lo [::1]:123 2023-10-30T12:31:14.580 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7761%5]:123 2023-10-30T12:31:14.580 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:14 ntpd[28423]: Listening on routing socket on fd #22 for interface updates 2023-10-30T12:31:15.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:15 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:15.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:15 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:15.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:15 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:15.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:15 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:15.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:15 ntpd[28423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:15.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:15 ntpd[28423]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:16.560 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:16 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:16 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:16 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:16 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:16 ntpd[27870]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:16 ntpd[27870]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:17.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:17 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:17.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:17 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:17.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:17 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:17.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:17 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:17.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:17 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:17.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:17 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:17.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:17 ntpd[28423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:17.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:17 ntpd[28423]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:18.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:18 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:18.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:18 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:18.562 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:18 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:18.562 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:18 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:18.562 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:18 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:18.562 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:18 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:18.562 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:18 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:18 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:18 ntpd[27870]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:18 ntpd[27870]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:18 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:18 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:18.564 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:18 ntpd[27870]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:18.564 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:18 ntpd[27870]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:18.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:18 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:18.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:18 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:18.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:18 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:18.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:18 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:18.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:18 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:18.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:18 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:18.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:18 ntpd[28423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:18.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:18 ntpd[28423]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:18.577 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:18 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:18.577 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:18 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:18.577 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:18 ntpd[28423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:18.577 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:18 ntpd[28423]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:19.560 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:19 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:19.560 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:19 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:19 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:19 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:19 ntpd[27870]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:19 ntpd[27870]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:19.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:19 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:19.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:19 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:19.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:19 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:19.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:19 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:19.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:19 ntpd[28423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:19.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:19 ntpd[28423]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:20.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:20 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:20.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:20 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:20.562 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:20 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:20.562 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:20 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:20.562 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:20 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:20.562 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:20 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:20.562 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:20 ntpd[27870]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:20.563 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:20 ntpd[27870]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:20.563 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:20 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:20.563 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:20 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:20.563 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:20 ntpd[27870]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:20.563 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:20 ntpd[27870]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:20.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:20 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:20.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:20 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:20.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:20 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:20.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:20 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:20.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:20 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:20.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:20 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:20.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:20 ntpd[28423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:20.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:20 ntpd[28423]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:20.577 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:20 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:20.577 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:20 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:20.577 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:20 ntpd[28423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:20.577 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:20 ntpd[28423]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:21 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:21 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:21 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:21 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:21 ntpd[27870]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:21 ntpd[27870]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:21.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:21 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:21.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:21 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:21.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:21 ntpd[28423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:21.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:21 ntpd[28423]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:21.575 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:21 ntpd[28423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:21.576 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:21 ntpd[28423]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:21.576 INFO:teuthology.orchestra.run.smithi154.stdout:30 Oct 12:31:21 ntpd[28423]: ntpd: time slew -0.006684 s 2023-10-30T12:31:21.576 INFO:teuthology.orchestra.run.smithi154.stdout:ntpd: time slew -0.006684s 2023-10-30T12:31:21.577 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:21 ntpd[28423]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-30T12:31:21.577 INFO:teuthology.orchestra.run.smithi154.stderr:30 Oct 12:31:21 ntpd[28423]: can't open /var/log/ntpstats/loopstats.20231030: Permission denied 2023-10-30T12:31:21.658 INFO:teuthology.orchestra.run.smithi154.stdout: remote refid st t when poll reach delay offset jitter 2023-10-30T12:31:21.658 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================== 2023-10-30T12:31:21.658 INFO:teuthology.orchestra.run.smithi154.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:31:21.658 INFO:teuthology.orchestra.run.smithi154.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:31:21.659 INFO:teuthology.orchestra.run.smithi154.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:31:21.659 INFO:teuthology.orchestra.run.smithi154.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:31:22.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:22 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:22.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:22 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:22.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:22 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:22.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:22 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:22.561 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:22 ntpd[27870]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:31:22.562 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:22 ntpd[27870]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:31:22.562 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:22 ntpd[27870]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:31:22.562 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:22 ntpd[27870]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:31:22.562 INFO:teuthology.orchestra.run.smithi012.stdout:30 Oct 12:31:22 ntpd[27870]: ntpd: time slew +0.001908 s 2023-10-30T12:31:22.563 INFO:teuthology.orchestra.run.smithi012.stdout:ntpd: time slew +0.001908s 2023-10-30T12:31:22.563 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:22 ntpd[27870]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-30T12:31:22.564 INFO:teuthology.orchestra.run.smithi012.stderr:30 Oct 12:31:22 ntpd[27870]: can't open /var/log/ntpstats/loopstats.20231030: Permission denied 2023-10-30T12:31:22.662 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2023-10-30T12:31:22.662 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2023-10-30T12:31:22.662 INFO:teuthology.orchestra.run.smithi012.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:31:22.662 INFO:teuthology.orchestra.run.smithi012.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:31:22.663 INFO:teuthology.orchestra.run.smithi012.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:31:22.663 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:31:22.664 INFO:teuthology.run_tasks:Running task install... 2023-10-30T12:31:22.673 DEBUG:teuthology.task.install:project ceph 2023-10-30T12:31:22.673 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71'}} 2023-10-30T12:31:22.674 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71'} 2023-10-30T12:31:22.674 INFO:teuthology.task.install:Using flavor: default 2023-10-30T12:31:22.680 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-30T12:31:22.680 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-30T12:31:22.681 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-key list | grep Ceph 2023-10-30T12:31:22.681 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-key list | grep Ceph 2023-10-30T12:31:22.747 INFO:teuthology.orchestra.run.smithi154.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-30T12:31:22.778 INFO:teuthology.orchestra.run.smithi154.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-30T12:31:22.778 INFO:teuthology.orchestra.run.smithi154.stdout:uid [ unknown] Ceph.com (release key) 2023-10-30T12:31:22.779 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-30T12:31:22.780 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-30T12:31:22.780 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=00c1c13cea21a686b046584596fbeac5315a7d71 2023-10-30T12:31:22.819 INFO:teuthology.orchestra.run.smithi012.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-30T12:31:22.874 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-30T12:31:22.874 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph.com (release key) 2023-10-30T12:31:22.875 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-30T12:31:22.875 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-30T12:31:22.876 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=00c1c13cea21a686b046584596fbeac5315a7d71 2023-10-30T12:31:23.019 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default/ 2023-10-30T12:31:23.019 INFO:teuthology.task.install.deb:Package version is 18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:31:23.084 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default/ 2023-10-30T12:31:23.085 INFO:teuthology.task.install.deb:Package version is 18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:31:23.253 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:31:23.254 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-30T12:31:23.276 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2023-10-30T12:31:23.316 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:31:23.317 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-30T12:31:23.331 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get update 2023-10-30T12:31:23.471 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-30T12:31:23.512 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-30T12:31:23.512 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-30T12:31:23.522 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-30T12:31:23.539 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-30T12:31:23.549 INFO:teuthology.orchestra.run.smithi154.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy InRelease 2023-10-30T12:31:23.571 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-10-30T12:31:23.588 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-30T12:31:23.595 INFO:teuthology.orchestra.run.smithi154.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-30T12:31:23.616 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,407 B] 2023-10-30T12:31:23.633 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy InRelease 2023-10-30T12:31:23.643 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-10-30T12:31:23.670 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-30T12:31:23.678 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-10-30T12:31:23.726 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-30T12:31:23.752 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:7 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-30T12:31:23.767 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,407 B] 2023-10-30T12:31:23.823 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-10-30T12:31:23.855 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 28.5 kB in 0s (81.4 kB/s) 2023-10-30T12:31:24.098 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 28.5 kB in 1s (48.4 kB/s) 2023-10-30T12:31:24.747 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2023-10-30T12:31:24.762 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6835-g00c1c13c-1jammy cephadm=18.0.0-6835-g00c1c13c-1jammy ceph-mds=18.0.0-6835-g00c1c13c-1jammy ceph-mgr=18.0.0-6835-g00c1c13c-1jammy ceph-common=18.0.0-6835-g00c1c13c-1jammy ceph-fuse=18.0.0-6835-g00c1c13c-1jammy ceph-test=18.0.0-6835-g00c1c13c-1jammy ceph-volume=18.0.0-6835-g00c1c13c-1jammy radosgw=18.0.0-6835-g00c1c13c-1jammy python3-rados=18.0.0-6835-g00c1c13c-1jammy python3-rgw=18.0.0-6835-g00c1c13c-1jammy python3-cephfs=18.0.0-6835-g00c1c13c-1jammy python3-rbd=18.0.0-6835-g00c1c13c-1jammy libcephfs2=18.0.0-6835-g00c1c13c-1jammy libcephfs-dev=18.0.0-6835-g00c1c13c-1jammy librados2=18.0.0-6835-g00c1c13c-1jammy librbd1=18.0.0-6835-g00c1c13c-1jammy rbd-fuse=18.0.0-6835-g00c1c13c-1jammy python3-cephfs=18.0.0-6835-g00c1c13c-1jammy cephfs-shell=18.0.0-6835-g00c1c13c-1jammy cephfs-top=18.0.0-6835-g00c1c13c-1jammy cephfs-mirror=18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:31:24.820 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2023-10-30T12:31:25.006 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2023-10-30T12:31:25.007 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2023-10-30T12:31:25.040 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2023-10-30T12:31:25.055 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6835-g00c1c13c-1jammy cephadm=18.0.0-6835-g00c1c13c-1jammy ceph-mds=18.0.0-6835-g00c1c13c-1jammy ceph-mgr=18.0.0-6835-g00c1c13c-1jammy ceph-common=18.0.0-6835-g00c1c13c-1jammy ceph-fuse=18.0.0-6835-g00c1c13c-1jammy ceph-test=18.0.0-6835-g00c1c13c-1jammy ceph-volume=18.0.0-6835-g00c1c13c-1jammy radosgw=18.0.0-6835-g00c1c13c-1jammy python3-rados=18.0.0-6835-g00c1c13c-1jammy python3-rgw=18.0.0-6835-g00c1c13c-1jammy python3-cephfs=18.0.0-6835-g00c1c13c-1jammy python3-rbd=18.0.0-6835-g00c1c13c-1jammy libcephfs2=18.0.0-6835-g00c1c13c-1jammy libcephfs-dev=18.0.0-6835-g00c1c13c-1jammy librados2=18.0.0-6835-g00c1c13c-1jammy librbd1=18.0.0-6835-g00c1c13c-1jammy rbd-fuse=18.0.0-6835-g00c1c13c-1jammy python3-cephfs=18.0.0-6835-g00c1c13c-1jammy cephfs-shell=18.0.0-6835-g00c1c13c-1jammy cephfs-top=18.0.0-6835-g00c1c13c-1jammy cephfs-mirror=18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:31:25.131 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2023-10-30T12:31:25.168 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages were automatically installed and are no longer required: 2023-10-30T12:31:25.168 INFO:teuthology.orchestra.run.smithi154.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-30T12:31:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-30T12:31:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-30T12:31:25.170 INFO:teuthology.orchestra.run.smithi154.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-10-30T12:31:25.170 INFO:teuthology.orchestra.run.smithi154.stdout: python2.7-minimal 2023-10-30T12:31:25.170 INFO:teuthology.orchestra.run.smithi154.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-30T12:31:25.170 INFO:teuthology.orchestra.run.smithi154.stdout:The following additional packages will be installed: 2023-10-30T12:31:25.171 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-30T12:31:25.171 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-10-30T12:31:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-30T12:31:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-10-30T12:31:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-10-30T12:31:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-10-30T12:31:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-10-30T12:31:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-10-30T12:31:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-10-30T12:31:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-10-30T12:31:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-10-30T12:31:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-10-30T12:31:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-10-30T12:31:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-30T12:31:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-10-30T12:31:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-10-30T12:31:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-10-30T12:31:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-10-30T12:31:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-10-30T12:31:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-10-30T12:31:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-10-30T12:31:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-10-30T12:31:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-30T12:31:25.176 INFO:teuthology.orchestra.run.smithi154.stdout:Suggested packages: 2023-10-30T12:31:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-10-30T12:31:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-10-30T12:31:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-10-30T12:31:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-10-30T12:31:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-10-30T12:31:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-10-30T12:31:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: python3-watchdog 2023-10-30T12:31:25.177 INFO:teuthology.orchestra.run.smithi154.stdout:Recommended packages: 2023-10-30T12:31:25.178 INFO:teuthology.orchestra.run.smithi154.stdout: btrfs-tools 2023-10-30T12:31:25.272 INFO:teuthology.orchestra.run.smithi154.stdout:The following NEW packages will be installed: 2023-10-30T12:31:25.272 INFO:teuthology.orchestra.run.smithi154.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-30T12:31:25.272 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-30T12:31:25.272 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-30T12:31:25.273 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-10-30T12:31:25.273 INFO:teuthology.orchestra.run.smithi154.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-30T12:31:25.273 INFO:teuthology.orchestra.run.smithi154.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-10-30T12:31:25.273 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-10-30T12:31:25.274 INFO:teuthology.orchestra.run.smithi154.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-10-30T12:31:25.274 INFO:teuthology.orchestra.run.smithi154.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-10-30T12:31:25.275 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-30T12:31:25.275 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-10-30T12:31:25.275 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-10-30T12:31:25.275 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-10-30T12:31:25.275 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-10-30T12:31:25.276 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-10-30T12:31:25.276 INFO:teuthology.orchestra.run.smithi154.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-10-30T12:31:25.276 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-10-30T12:31:25.276 INFO:teuthology.orchestra.run.smithi154.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-30T12:31:25.276 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-10-30T12:31:25.277 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-10-30T12:31:25.277 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-30T12:31:25.277 INFO:teuthology.orchestra.run.smithi154.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-30T12:31:25.277 INFO:teuthology.orchestra.run.smithi154.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-30T12:31:25.277 INFO:teuthology.orchestra.run.smithi154.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-30T12:31:25.278 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-30T12:31:25.278 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-10-30T12:31:25.278 INFO:teuthology.orchestra.run.smithi154.stdout: socat xmlstarlet 2023-10-30T12:31:25.302 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2023-10-30T12:31:25.302 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2023-10-30T12:31:25.417 INFO:teuthology.orchestra.run.smithi154.stdout:0 upgraded, 115 newly installed, 0 to remove and 260 not upgraded. 2023-10-30T12:31:25.417 INFO:teuthology.orchestra.run.smithi154.stdout:Need to get 184 MB of archives. 2023-10-30T12:31:25.418 INFO:teuthology.orchestra.run.smithi154.stdout:After this operation, 826 MB of additional disk space will be used. 2023-10-30T12:31:25.418 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-6835-g00c1c13c-1jammy [3,450 kB] 2023-10-30T12:31:25.459 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2023-10-30T12:31:25.459 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-30T12:31:25.459 INFO:teuthology.orchestra.run.smithi012.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-30T12:31:25.460 INFO:teuthology.orchestra.run.smithi012.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-30T12:31:25.460 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-30T12:31:25.460 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-10-30T12:31:25.462 INFO:teuthology.orchestra.run.smithi012.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-30T12:31:25.462 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-30T12:31:25.462 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2023-10-30T12:31:25.462 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-30T12:31:25.462 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-10-30T12:31:25.463 INFO:teuthology.orchestra.run.smithi012.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-30T12:31:25.463 INFO:teuthology.orchestra.run.smithi012.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-10-30T12:31:25.463 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-10-30T12:31:25.463 INFO:teuthology.orchestra.run.smithi012.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-10-30T12:31:25.463 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-10-30T12:31:25.464 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-10-30T12:31:25.464 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-10-30T12:31:25.464 INFO:teuthology.orchestra.run.smithi012.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-10-30T12:31:25.464 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-10-30T12:31:25.464 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-10-30T12:31:25.464 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-10-30T12:31:25.465 INFO:teuthology.orchestra.run.smithi012.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-30T12:31:25.465 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-10-30T12:31:25.465 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-10-30T12:31:25.465 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-10-30T12:31:25.465 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-10-30T12:31:25.466 INFO:teuthology.orchestra.run.smithi012.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-10-30T12:31:25.466 INFO:teuthology.orchestra.run.smithi012.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-10-30T12:31:25.466 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-10-30T12:31:25.466 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-10-30T12:31:25.466 INFO:teuthology.orchestra.run.smithi012.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-30T12:31:25.467 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2023-10-30T12:31:25.467 INFO:teuthology.orchestra.run.smithi012.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-10-30T12:31:25.467 INFO:teuthology.orchestra.run.smithi012.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-10-30T12:31:25.467 INFO:teuthology.orchestra.run.smithi012.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-10-30T12:31:25.468 INFO:teuthology.orchestra.run.smithi012.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-10-30T12:31:25.468 INFO:teuthology.orchestra.run.smithi012.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-10-30T12:31:25.468 INFO:teuthology.orchestra.run.smithi012.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-10-30T12:31:25.468 INFO:teuthology.orchestra.run.smithi012.stdout: python3-watchdog 2023-10-30T12:31:25.468 INFO:teuthology.orchestra.run.smithi012.stdout:Recommended packages: 2023-10-30T12:31:25.469 INFO:teuthology.orchestra.run.smithi012.stdout: btrfs-tools 2023-10-30T12:31:25.541 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-10-30T12:31:25.587 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-10-30T12:31:25.609 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2023-10-30T12:31:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-30T12:31:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-30T12:31:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-30T12:31:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-10-30T12:31:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-30T12:31:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-10-30T12:31:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-10-30T12:31:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-10-30T12:31:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-10-30T12:31:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-30T12:31:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-10-30T12:31:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-10-30T12:31:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-10-30T12:31:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-10-30T12:31:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-10-30T12:31:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-10-30T12:31:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-10-30T12:31:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-30T12:31:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-10-30T12:31:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-10-30T12:31:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-30T12:31:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-30T12:31:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-30T12:31:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-30T12:31:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-30T12:31:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-10-30T12:31:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: socat xmlstarlet 2023-10-30T12:31:25.644 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-6835-g00c1c13c-1jammy [3,131 kB] 2023-10-30T12:31:25.694 INFO:teuthology.orchestra.run.smithi012.stdout:0 upgraded, 115 newly installed, 0 to remove and 260 not upgraded. 2023-10-30T12:31:25.694 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 184 MB of archives. 2023-10-30T12:31:25.695 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 826 MB of additional disk space will be used. 2023-10-30T12:31:25.695 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-10-30T12:31:25.719 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-10-30T12:31:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-10-30T12:31:25.738 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-6835-g00c1c13c-1jammy [689 kB] 2023-10-30T12:31:25.760 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-6835-g00c1c13c-1jammy [356 kB] 2023-10-30T12:31:25.772 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-6835-g00c1c13c-1jammy [32.8 kB] 2023-10-30T12:31:25.773 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-6835-g00c1c13c-1jammy [180 kB] 2023-10-30T12:31:25.779 INFO:teuthology.orchestra.run.smithi154.stdout:Get:10 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-6835-g00c1c13c-1jammy [62.4 kB] 2023-10-30T12:31:25.780 INFO:teuthology.orchestra.run.smithi154.stdout:Get:11 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-6835-g00c1c13c-1jammy [330 kB] 2023-10-30T12:31:25.782 INFO:teuthology.orchestra.run.smithi154.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-10-30T12:31:25.792 INFO:teuthology.orchestra.run.smithi154.stdout:Get:13 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-6835-g00c1c13c-1jammy [6,226 kB] 2023-10-30T12:31:25.796 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-10-30T12:31:25.810 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-10-30T12:31:25.825 INFO:teuthology.orchestra.run.smithi154.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-10-30T12:31:25.833 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-10-30T12:31:25.836 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-10-30T12:31:25.853 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-10-30T12:31:25.900 INFO:teuthology.orchestra.run.smithi154.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-10-30T12:31:25.912 INFO:teuthology.orchestra.run.smithi154.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-10-30T12:31:25.920 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-10-30T12:31:25.923 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-10-30T12:31:25.938 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-10-30T12:31:25.941 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-10-30T12:31:25.942 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-10-30T12:31:25.953 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-10-30T12:31:25.953 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-10-30T12:31:25.954 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-10-30T12:31:25.955 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-10-30T12:31:25.957 INFO:teuthology.orchestra.run.smithi154.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-10-30T12:31:25.959 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-10-30T12:31:25.982 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-10-30T12:31:25.982 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-10-30T12:31:25.983 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-10-30T12:31:25.983 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-10-30T12:31:25.984 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-10-30T12:31:25.986 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-10-30T12:31:25.986 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-10-30T12:31:25.987 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-10-30T12:31:25.987 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-10-30T12:31:25.998 INFO:teuthology.orchestra.run.smithi154.stdout:Get:18 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-6835-g00c1c13c-1jammy [112 kB] 2023-10-30T12:31:26.001 INFO:teuthology.orchestra.run.smithi154.stdout:Get:19 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-6835-g00c1c13c-1jammy [439 kB] 2023-10-30T12:31:26.012 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-30T12:31:26.013 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-10-30T12:31:26.013 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-10-30T12:31:26.014 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-10-30T12:31:26.015 INFO:teuthology.orchestra.run.smithi154.stdout:Get:20 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-6835-g00c1c13c-1jammy [22.6 MB] 2023-10-30T12:31:26.015 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-10-30T12:31:26.016 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-10-30T12:31:26.038 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-10-30T12:31:26.038 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-10-30T12:31:26.039 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-10-30T12:31:26.039 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-10-30T12:31:26.040 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-6835-g00c1c13c-1jammy [3,450 kB] 2023-10-30T12:31:26.063 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-10-30T12:31:26.066 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-10-30T12:31:26.066 INFO:teuthology.orchestra.run.smithi012.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-10-30T12:31:26.068 INFO:teuthology.orchestra.run.smithi012.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-10-30T12:31:26.068 INFO:teuthology.orchestra.run.smithi012.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-10-30T12:31:26.071 INFO:teuthology.orchestra.run.smithi012.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-10-30T12:31:26.089 INFO:teuthology.orchestra.run.smithi012.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-10-30T12:31:26.091 INFO:teuthology.orchestra.run.smithi012.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-10-30T12:31:26.123 INFO:teuthology.orchestra.run.smithi154.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-10-30T12:31:26.132 INFO:teuthology.orchestra.run.smithi154.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-10-30T12:31:26.158 INFO:teuthology.orchestra.run.smithi154.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-10-30T12:31:26.172 INFO:teuthology.orchestra.run.smithi012.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-10-30T12:31:26.173 INFO:teuthology.orchestra.run.smithi012.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-10-30T12:31:26.174 INFO:teuthology.orchestra.run.smithi012.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-10-30T12:31:26.175 INFO:teuthology.orchestra.run.smithi012.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-30T12:31:26.176 INFO:teuthology.orchestra.run.smithi012.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-10-30T12:31:26.177 INFO:teuthology.orchestra.run.smithi012.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-10-30T12:31:26.205 INFO:teuthology.orchestra.run.smithi012.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-10-30T12:31:26.207 INFO:teuthology.orchestra.run.smithi012.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-10-30T12:31:26.207 INFO:teuthology.orchestra.run.smithi012.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-10-30T12:31:26.245 INFO:teuthology.orchestra.run.smithi012.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-10-30T12:31:26.245 INFO:teuthology.orchestra.run.smithi012.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-10-30T12:31:26.246 INFO:teuthology.orchestra.run.smithi012.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-10-30T12:31:26.246 INFO:teuthology.orchestra.run.smithi012.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-10-30T12:31:26.247 INFO:teuthology.orchestra.run.smithi012.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-10-30T12:31:26.248 INFO:teuthology.orchestra.run.smithi012.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-10-30T12:31:26.270 INFO:teuthology.orchestra.run.smithi012.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-10-30T12:31:26.270 INFO:teuthology.orchestra.run.smithi012.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-10-30T12:31:26.274 INFO:teuthology.orchestra.run.smithi012.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-10-30T12:31:26.296 INFO:teuthology.orchestra.run.smithi012.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-10-30T12:31:26.297 INFO:teuthology.orchestra.run.smithi012.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-10-30T12:31:26.298 INFO:teuthology.orchestra.run.smithi012.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-10-30T12:31:26.302 INFO:teuthology.orchestra.run.smithi012.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-10-30T12:31:26.305 INFO:teuthology.orchestra.run.smithi012.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-10-30T12:31:26.306 INFO:teuthology.orchestra.run.smithi012.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-10-30T12:31:26.313 INFO:teuthology.orchestra.run.smithi012.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-10-30T12:31:26.328 INFO:teuthology.orchestra.run.smithi012.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-10-30T12:31:26.328 INFO:teuthology.orchestra.run.smithi012.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-30T12:31:26.329 INFO:teuthology.orchestra.run.smithi012.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-10-30T12:31:26.330 INFO:teuthology.orchestra.run.smithi154.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-10-30T12:31:26.355 INFO:teuthology.orchestra.run.smithi012.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-10-30T12:31:26.358 INFO:teuthology.orchestra.run.smithi012.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-10-30T12:31:26.359 INFO:teuthology.orchestra.run.smithi012.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-10-30T12:31:26.370 INFO:teuthology.orchestra.run.smithi012.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-10-30T12:31:26.370 INFO:teuthology.orchestra.run.smithi012.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-10-30T12:31:26.371 INFO:teuthology.orchestra.run.smithi012.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-10-30T12:31:26.373 INFO:teuthology.orchestra.run.smithi012.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-10-30T12:31:26.388 INFO:teuthology.orchestra.run.smithi012.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-10-30T12:31:26.525 INFO:teuthology.orchestra.run.smithi154.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-10-30T12:31:26.859 INFO:teuthology.orchestra.run.smithi154.stdout:Get:26 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-6835-g00c1c13c-1jammy [6,858 kB] 2023-10-30T12:31:26.891 INFO:teuthology.orchestra.run.smithi154.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-10-30T12:31:26.899 INFO:teuthology.orchestra.run.smithi154.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-10-30T12:31:26.900 INFO:teuthology.orchestra.run.smithi154.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-10-30T12:31:26.912 INFO:teuthology.orchestra.run.smithi154.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-10-30T12:31:26.913 INFO:teuthology.orchestra.run.smithi154.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-10-30T12:31:26.915 INFO:teuthology.orchestra.run.smithi154.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-10-30T12:31:26.916 INFO:teuthology.orchestra.run.smithi154.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-10-30T12:31:26.919 INFO:teuthology.orchestra.run.smithi154.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-10-30T12:31:26.920 INFO:teuthology.orchestra.run.smithi154.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-10-30T12:31:27.045 INFO:teuthology.orchestra.run.smithi154.stdout:Get:36 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-6835-g00c1c13c-1jammy [238 kB] 2023-10-30T12:31:27.050 INFO:teuthology.orchestra.run.smithi154.stdout:Get:37 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-6835-g00c1c13c-1jammy [128 kB] 2023-10-30T12:31:27.053 INFO:teuthology.orchestra.run.smithi154.stdout:Get:38 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-6835-g00c1c13c-1jammy [1,496 kB] 2023-10-30T12:31:27.085 INFO:teuthology.orchestra.run.smithi154.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-10-30T12:31:27.100 INFO:teuthology.orchestra.run.smithi154.stdout:Get:40 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-6835-g00c1c13c-1jammy [9,892 kB] 2023-10-30T12:31:27.459 INFO:teuthology.orchestra.run.smithi154.stdout:Get:41 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-6835-g00c1c13c-1jammy [31.7 MB] 2023-10-30T12:31:27.521 INFO:teuthology.orchestra.run.smithi154.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-10-30T12:31:27.530 INFO:teuthology.orchestra.run.smithi154.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-10-30T12:31:27.536 INFO:teuthology.orchestra.run.smithi154.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-10-30T12:31:27.540 INFO:teuthology.orchestra.run.smithi154.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-10-30T12:31:27.557 INFO:teuthology.orchestra.run.smithi154.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-30T12:31:27.560 INFO:teuthology.orchestra.run.smithi154.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-10-30T12:31:27.563 INFO:teuthology.orchestra.run.smithi154.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-10-30T12:31:27.605 INFO:teuthology.orchestra.run.smithi154.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-10-30T12:31:27.618 INFO:teuthology.orchestra.run.smithi154.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-10-30T12:31:27.623 INFO:teuthology.orchestra.run.smithi154.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-10-30T12:31:27.773 INFO:teuthology.orchestra.run.smithi012.stdout:Get:83 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-6835-g00c1c13c-1jammy [3,131 kB] 2023-10-30T12:31:27.877 INFO:teuthology.orchestra.run.smithi154.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-10-30T12:31:27.960 INFO:teuthology.orchestra.run.smithi154.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-10-30T12:31:28.050 INFO:teuthology.orchestra.run.smithi154.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-10-30T12:31:28.107 INFO:teuthology.orchestra.run.smithi154.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-10-30T12:31:28.229 INFO:teuthology.orchestra.run.smithi154.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-10-30T12:31:28.317 INFO:teuthology.orchestra.run.smithi154.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-10-30T12:31:28.335 INFO:teuthology.orchestra.run.smithi154.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-10-30T12:31:28.364 INFO:teuthology.orchestra.run.smithi154.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-10-30T12:31:28.377 INFO:teuthology.orchestra.run.smithi154.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-10-30T12:31:28.414 INFO:teuthology.orchestra.run.smithi154.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-10-30T12:31:28.422 INFO:teuthology.orchestra.run.smithi154.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-10-30T12:31:28.604 INFO:teuthology.orchestra.run.smithi012.stdout:Get:84 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-6835-g00c1c13c-1jammy [689 kB] 2023-10-30T12:31:28.614 INFO:teuthology.orchestra.run.smithi154.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-10-30T12:31:28.745 INFO:teuthology.orchestra.run.smithi012.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-6835-g00c1c13c-1jammy [356 kB] 2023-10-30T12:31:28.783 INFO:teuthology.orchestra.run.smithi012.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-6835-g00c1c13c-1jammy [32.8 kB] 2023-10-30T12:31:28.784 INFO:teuthology.orchestra.run.smithi012.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-6835-g00c1c13c-1jammy [180 kB] 2023-10-30T12:31:28.792 INFO:teuthology.orchestra.run.smithi154.stdout:Get:64 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-6835-g00c1c13c-1jammy [14.1 kB] 2023-10-30T12:31:28.793 INFO:teuthology.orchestra.run.smithi154.stdout:Get:65 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-6835-g00c1c13c-1jammy [839 kB] 2023-10-30T12:31:28.809 INFO:teuthology.orchestra.run.smithi154.stdout:Get:66 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-6835-g00c1c13c-1jammy [2,148 kB] 2023-10-30T12:31:28.820 INFO:teuthology.orchestra.run.smithi012.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-6835-g00c1c13c-1jammy [62.4 kB] 2023-10-30T12:31:28.835 INFO:teuthology.orchestra.run.smithi012.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-6835-g00c1c13c-1jammy [330 kB] 2023-10-30T12:31:28.848 INFO:teuthology.orchestra.run.smithi154.stdout:Get:67 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-6835-g00c1c13c-1jammy [284 kB] 2023-10-30T12:31:28.852 INFO:teuthology.orchestra.run.smithi154.stdout:Get:68 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-6835-g00c1c13c-1jammy [138 kB] 2023-10-30T12:31:28.855 INFO:teuthology.orchestra.run.smithi154.stdout:Get:69 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-6835-g00c1c13c-1jammy [1,907 kB] 2023-10-30T12:31:28.876 INFO:teuthology.orchestra.run.smithi154.stdout:Get:70 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-6835-g00c1c13c-1jammy [8,625 kB] 2023-10-30T12:31:28.881 INFO:teuthology.orchestra.run.smithi012.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-6835-g00c1c13c-1jammy [6,226 kB] 2023-10-30T12:31:29.189 INFO:teuthology.orchestra.run.smithi154.stdout:Get:71 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-6835-g00c1c13c-1jammy [14.2 kB] 2023-10-30T12:31:29.190 INFO:teuthology.orchestra.run.smithi154.stdout:Get:72 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-6835-g00c1c13c-1jammy [47.8 MB] 2023-10-30T12:31:29.424 INFO:teuthology.orchestra.run.smithi154.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-10-30T12:31:29.426 INFO:teuthology.orchestra.run.smithi154.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-10-30T12:31:29.428 INFO:teuthology.orchestra.run.smithi154.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-10-30T12:31:29.430 INFO:teuthology.orchestra.run.smithi154.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-30T12:31:29.430 INFO:teuthology.orchestra.run.smithi154.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-10-30T12:31:29.433 INFO:teuthology.orchestra.run.smithi154.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-10-30T12:31:29.475 INFO:teuthology.orchestra.run.smithi154.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-10-30T12:31:29.478 INFO:teuthology.orchestra.run.smithi154.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-10-30T12:31:29.479 INFO:teuthology.orchestra.run.smithi154.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-10-30T12:31:29.540 INFO:teuthology.orchestra.run.smithi154.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-10-30T12:31:29.704 INFO:teuthology.orchestra.run.smithi154.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-10-30T12:31:29.865 INFO:teuthology.orchestra.run.smithi012.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-6835-g00c1c13c-1jammy [112 kB] 2023-10-30T12:31:29.876 INFO:teuthology.orchestra.run.smithi012.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-6835-g00c1c13c-1jammy [439 kB] 2023-10-30T12:31:29.938 INFO:teuthology.orchestra.run.smithi012.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-6835-g00c1c13c-1jammy [22.6 MB] 2023-10-30T12:31:29.962 INFO:teuthology.orchestra.run.smithi154.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-10-30T12:31:29.994 INFO:teuthology.orchestra.run.smithi154.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-10-30T12:31:30.020 INFO:teuthology.orchestra.run.smithi154.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-10-30T12:31:30.073 INFO:teuthology.orchestra.run.smithi154.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-10-30T12:31:30.092 INFO:teuthology.orchestra.run.smithi154.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-10-30T12:31:30.120 INFO:teuthology.orchestra.run.smithi154.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-10-30T12:31:30.248 INFO:teuthology.orchestra.run.smithi154.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-10-30T12:31:30.279 INFO:teuthology.orchestra.run.smithi154.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-10-30T12:31:30.299 INFO:teuthology.orchestra.run.smithi154.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-10-30T12:31:30.307 INFO:teuthology.orchestra.run.smithi154.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-10-30T12:31:30.511 INFO:teuthology.orchestra.run.smithi154.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-10-30T12:31:30.972 INFO:teuthology.orchestra.run.smithi154.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-10-30T12:31:30.976 INFO:teuthology.orchestra.run.smithi154.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-10-30T12:31:31.102 INFO:teuthology.orchestra.run.smithi154.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-10-30T12:31:31.150 INFO:teuthology.orchestra.run.smithi154.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-10-30T12:31:31.150 INFO:teuthology.orchestra.run.smithi154.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-30T12:31:31.156 INFO:teuthology.orchestra.run.smithi154.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-10-30T12:31:31.158 INFO:teuthology.orchestra.run.smithi154.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-10-30T12:31:31.173 INFO:teuthology.orchestra.run.smithi154.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-10-30T12:31:31.180 INFO:teuthology.orchestra.run.smithi154.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-10-30T12:31:31.241 INFO:teuthology.orchestra.run.smithi154.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-10-30T12:31:31.406 INFO:teuthology.orchestra.run.smithi154.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-10-30T12:31:31.474 INFO:teuthology.orchestra.run.smithi154.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-6835-g00c1c13c-1jammy [120 kB] 2023-10-30T12:31:31.476 INFO:teuthology.orchestra.run.smithi154.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-6835-g00c1c13c-1jammy [227 kB] 2023-10-30T12:31:31.481 INFO:teuthology.orchestra.run.smithi154.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-6835-g00c1c13c-1jammy [34.0 kB] 2023-10-30T12:31:31.482 INFO:teuthology.orchestra.run.smithi154.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-6835-g00c1c13c-1jammy [23.4 kB] 2023-10-30T12:31:31.482 INFO:teuthology.orchestra.run.smithi154.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-6835-g00c1c13c-1jammy [38.3 kB] 2023-10-30T12:31:31.483 INFO:teuthology.orchestra.run.smithi154.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-6835-g00c1c13c-1jammy [12.3 MB] 2023-10-30T12:31:31.572 INFO:teuthology.orchestra.run.smithi154.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-10-30T12:31:31.849 INFO:teuthology.orchestra.run.smithi154.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-10-30T12:31:31.874 INFO:teuthology.orchestra.run.smithi154.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-10-30T12:31:31.875 INFO:teuthology.orchestra.run.smithi154.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-6835-g00c1c13c-1jammy [91.6 kB] 2023-10-30T12:31:32.348 INFO:teuthology.orchestra.run.smithi012.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-6835-g00c1c13c-1jammy [6,858 kB] 2023-10-30T12:31:32.671 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 184 MB in 7s (27.0 MB/s) 2023-10-30T12:31:32.787 INFO:teuthology.orchestra.run.smithi012.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-6835-g00c1c13c-1jammy [238 kB] 2023-10-30T12:31:32.804 INFO:teuthology.orchestra.run.smithi012.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-6835-g00c1c13c-1jammy [128 kB] 2023-10-30T12:31:32.811 INFO:teuthology.orchestra.run.smithi012.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-6835-g00c1c13c-1jammy [1,496 kB] 2023-10-30T12:31:32.819 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-30T12:31:32.904 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 130506 files and directories currently installed.) 2023-10-30T12:31:32.908 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-10-30T12:31:32.929 INFO:teuthology.orchestra.run.smithi012.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-6835-g00c1c13c-1jammy [9,892 kB] 2023-10-30T12:31:32.959 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-10-30T12:31:33.228 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-30T12:31:33.249 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-10-30T12:31:33.286 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-30T12:31:33.515 INFO:teuthology.orchestra.run.smithi012.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-6835-g00c1c13c-1jammy [31.7 MB] 2023-10-30T12:31:33.606 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cmd2. 2023-10-30T12:31:33.626 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-10-30T12:31:33.664 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-10-30T12:31:33.994 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libfmt8:amd64. 2023-10-30T12:31:34.015 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-10-30T12:31:34.094 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-30T12:31:34.397 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-10-30T12:31:34.418 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-10-30T12:31:34.447 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-30T12:31:34.784 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-30T12:31:34.805 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-10-30T12:31:34.842 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-30T12:31:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-30T12:31:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-10-30T12:31:35.246 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-30T12:31:35.355 INFO:teuthology.orchestra.run.smithi012.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-6835-g00c1c13c-1jammy [14.1 kB] 2023-10-30T12:31:35.356 INFO:teuthology.orchestra.run.smithi012.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-6835-g00c1c13c-1jammy [839 kB] 2023-10-30T12:31:35.400 INFO:teuthology.orchestra.run.smithi012.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-6835-g00c1c13c-1jammy [2,148 kB] 2023-10-30T12:31:35.549 INFO:teuthology.orchestra.run.smithi012.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-6835-g00c1c13c-1jammy [284 kB] 2023-10-30T12:31:35.566 INFO:teuthology.orchestra.run.smithi012.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-6835-g00c1c13c-1jammy [138 kB] 2023-10-30T12:31:35.573 INFO:teuthology.orchestra.run.smithi012.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-6835-g00c1c13c-1jammy [1,907 kB] 2023-10-30T12:31:35.674 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-30T12:31:35.683 INFO:teuthology.orchestra.run.smithi012.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-6835-g00c1c13c-1jammy [8,625 kB] 2023-10-30T12:31:35.694 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-30T12:31:35.732 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:31:36.231 INFO:teuthology.orchestra.run.smithi012.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-6835-g00c1c13c-1jammy [14.2 kB] 2023-10-30T12:31:36.232 INFO:teuthology.orchestra.run.smithi012.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-6835-g00c1c13c-1jammy [47.8 MB] 2023-10-30T12:31:36.653 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-30T12:31:36.673 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-30T12:31:36.711 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:31:37.064 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-30T12:31:37.085 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-30T12:31:37.122 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:31:37.442 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-10-30T12:31:37.463 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-10-30T12:31:37.500 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-30T12:31:37.837 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librados2. 2023-10-30T12:31:37.857 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../011-librados2_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:37.895 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librados2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:38.348 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librbd1. 2023-10-30T12:31:38.368 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../012-librbd1_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:38.406 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librbd1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:38.825 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcephfs2. 2023-10-30T12:31:38.846 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:38.884 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcephfs2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:39.170 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rados. 2023-10-30T12:31:39.191 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../014-python3-rados_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:39.228 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rados (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:39.556 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-30T12:31:39.576 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:31:39.622 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-ceph-argparse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:39.743 INFO:teuthology.orchestra.run.smithi012.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-6835-g00c1c13c-1jammy [120 kB] 2023-10-30T12:31:39.744 INFO:teuthology.orchestra.run.smithi012.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-6835-g00c1c13c-1jammy [227 kB] 2023-10-30T12:31:39.746 INFO:teuthology.orchestra.run.smithi012.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-6835-g00c1c13c-1jammy [34.0 kB] 2023-10-30T12:31:39.747 INFO:teuthology.orchestra.run.smithi012.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-6835-g00c1c13c-1jammy [23.4 kB] 2023-10-30T12:31:39.748 INFO:teuthology.orchestra.run.smithi012.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-6835-g00c1c13c-1jammy [38.3 kB] 2023-10-30T12:31:39.750 INFO:teuthology.orchestra.run.smithi012.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-6835-g00c1c13c-1jammy [12.3 MB] 2023-10-30T12:31:39.866 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cephfs. 2023-10-30T12:31:39.878 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:39.916 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cephfs (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:40.211 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-30T12:31:40.232 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:31:40.269 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:40.431 INFO:teuthology.orchestra.run.smithi012.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-6835-g00c1c13c-1jammy [91.6 kB] 2023-10-30T12:31:40.606 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-prettytable. 2023-10-30T12:31:40.626 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-10-30T12:31:40.664 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-10-30T12:31:40.933 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rbd. 2023-10-30T12:31:40.945 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:40.983 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rbd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:41.020 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 184 MB in 15s (12.4 MB/s) 2023-10-30T12:31:41.128 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-30T12:31:41.199 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132932 files and directories currently installed.) 2023-10-30T12:31:41.203 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-10-30T12:31:41.269 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-10-30T12:31:41.378 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-30T12:31:41.399 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-10-30T12:31:41.436 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-30T12:31:41.564 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-30T12:31:41.578 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-10-30T12:31:41.622 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-30T12:31:41.933 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cmd2. 2023-10-30T12:31:41.947 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-10-30T12:31:41.991 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-10-30T12:31:42.014 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librgw2. 2023-10-30T12:31:42.034 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../021-librgw2_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:42.072 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librgw2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:42.311 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libfmt8:amd64. 2023-10-30T12:31:42.325 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-10-30T12:31:42.411 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-30T12:31:42.608 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rgw. 2023-10-30T12:31:42.620 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:42.658 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:42.731 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-10-30T12:31:42.746 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-10-30T12:31:42.781 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-30T12:31:42.986 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liboath0:amd64. 2023-10-30T12:31:43.006 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-10-30T12:31:43.044 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-10-30T12:31:43.110 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-30T12:31:43.125 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-10-30T12:31:43.168 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-30T12:31:43.330 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libradosstriper1. 2023-10-30T12:31:43.342 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:43.380 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libradosstriper1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:43.513 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-30T12:31:43.528 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-10-30T12:31:43.571 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-30T12:31:43.842 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-common. 2023-10-30T12:31:43.863 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../025-ceph-common_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:43.900 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:43.941 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-30T12:31:43.957 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-30T12:31:43.999 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:31:44.411 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-30T12:31:44.426 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-30T12:31:44.469 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:31:44.806 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-30T12:31:44.821 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-30T12:31:44.864 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:31:45.194 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-10-30T12:31:45.209 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-10-30T12:31:45.231 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-base. 2023-10-30T12:31:45.242 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-30T12:31:45.252 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../026-ceph-base_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:45.307 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-base (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:45.554 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librados2. 2023-10-30T12:31:45.569 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../011-librados2_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:45.612 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librados2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:45.978 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-10-30T12:31:45.998 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-10-30T12:31:46.027 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-10-30T12:31:46.248 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librbd1. 2023-10-30T12:31:46.264 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../012-librbd1_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:46.296 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cheroot. 2023-10-30T12:31:46.307 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librbd1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:46.317 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-10-30T12:31:46.355 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-30T12:31:46.651 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs2. 2023-10-30T12:31:46.665 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:46.666 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-10-30T12:31:46.687 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-10-30T12:31:46.701 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:46.716 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-10-30T12:31:46.937 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rados. 2023-10-30T12:31:46.951 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../014-python3-rados_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:46.986 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jaraco.text. 2023-10-30T12:31:46.995 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rados (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:47.010 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-10-30T12:31:47.219 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-10-30T12:31:47.315 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-30T12:31:47.330 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:31:47.373 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-argparse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:47.547 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-10-30T12:31:47.567 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-10-30T12:31:47.605 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-10-30T12:31:47.642 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cephfs. 2023-10-30T12:31:47.657 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:47.700 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cephfs (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:47.975 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-tempora. 2023-10-30T12:31:47.994 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-30T12:31:47.996 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-10-30T12:31:48.010 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:31:48.041 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-10-30T12:31:48.045 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:48.315 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-prettytable. 2023-10-30T12:31:48.329 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-10-30T12:31:48.364 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-10-30T12:31:48.394 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-portend. 2023-10-30T12:31:48.415 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-10-30T12:31:48.461 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-10-30T12:31:48.625 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rbd. 2023-10-30T12:31:48.640 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:48.717 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rbd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:48.831 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-10-30T12:31:48.851 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-10-30T12:31:48.897 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-10-30T12:31:49.045 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-30T12:31:49.060 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-10-30T12:31:49.112 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-30T12:31:49.217 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-30T12:31:49.237 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-10-30T12:31:49.283 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-10-30T12:31:49.406 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librgw2. 2023-10-30T12:31:49.421 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../021-librgw2_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:49.456 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librgw2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:49.628 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-natsort. 2023-10-30T12:31:49.641 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-10-30T12:31:49.678 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-10-30T12:31:49.883 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rgw. 2023-10-30T12:31:49.898 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:49.950 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:49.990 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-logutils. 2023-10-30T12:31:50.011 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-10-30T12:31:50.048 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-10-30T12:31:50.253 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liboath0:amd64. 2023-10-30T12:31:50.267 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-10-30T12:31:50.311 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-10-30T12:31:50.351 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-30T12:31:50.372 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-10-30T12:31:50.410 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-10-30T12:31:50.589 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libradosstriper1. 2023-10-30T12:31:50.604 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:50.648 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libradosstriper1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:50.671 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-mako. 2023-10-30T12:31:50.691 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-10-30T12:31:50.729 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-30T12:31:51.032 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-30T12:31:51.053 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-30T12:31:51.067 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-common. 2023-10-30T12:31:51.088 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../025-ceph-common_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:51.090 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-30T12:31:51.251 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:51.402 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-30T12:31:51.422 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-10-30T12:31:51.460 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-10-30T12:31:51.754 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-webob. 2023-10-30T12:31:51.775 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-10-30T12:31:51.813 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-10-30T12:31:52.174 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-waitress. 2023-10-30T12:31:52.195 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-10-30T12:31:52.235 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-30T12:31:52.273 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-base. 2023-10-30T12:31:52.289 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../026-ceph-base_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:52.333 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-base (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:52.644 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-tempita. 2023-10-30T12:31:52.656 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-10-30T12:31:52.694 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-10-30T12:31:52.986 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-10-30T12:31:52.996 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-paste. 2023-10-30T12:31:53.000 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-10-30T12:31:53.017 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-10-30T12:31:53.036 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-10-30T12:31:53.054 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-10-30T12:31:53.358 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-30T12:31:53.369 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-10-30T12:31:53.399 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-10-30T12:31:53.407 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cheroot. 2023-10-30T12:31:53.422 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-10-30T12:31:53.466 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-30T12:31:53.703 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-30T12:31:53.723 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-10-30T12:31:53.761 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-10-30T12:31:53.785 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-10-30T12:31:53.800 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-10-30T12:31:53.843 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-10-30T12:31:54.047 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-webtest. 2023-10-30T12:31:54.068 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-10-30T12:31:54.105 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-10-30T12:31:54.146 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.text. 2023-10-30T12:31:54.162 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-10-30T12:31:54.205 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-10-30T12:31:54.416 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pecan. 2023-10-30T12:31:54.437 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-10-30T12:31:54.475 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-10-30T12:31:54.476 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-10-30T12:31:54.490 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-10-30T12:31:54.524 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-10-30T12:31:54.810 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-tempora. 2023-10-30T12:31:54.826 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-10-30T12:31:54.838 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-certifi. 2023-10-30T12:31:54.859 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-10-30T12:31:54.868 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-10-30T12:31:54.896 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-10-30T12:31:55.188 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-portend. 2023-10-30T12:31:55.203 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-10-30T12:31:55.246 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-10-30T12:31:55.249 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-idna. 2023-10-30T12:31:55.270 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-10-30T12:31:55.307 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-idna (3.3-1) ... 2023-10-30T12:31:55.549 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-10-30T12:31:55.565 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-10-30T12:31:55.599 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-10-30T12:31:55.619 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-urllib3. 2023-10-30T12:31:55.640 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-10-30T12:31:55.677 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-10-30T12:31:55.877 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-30T12:31:55.893 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-10-30T12:31:55.935 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-10-30T12:31:56.014 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-requests. 2023-10-30T12:31:56.035 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-10-30T12:31:56.072 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-30T12:31:56.297 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-natsort. 2023-10-30T12:31:56.312 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-10-30T12:31:56.333 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-30T12:31:56.354 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-10-30T12:31:56.356 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-10-30T12:31:56.383 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-30T12:31:56.667 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-logutils. 2023-10-30T12:31:56.669 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-30T12:31:56.681 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:31:56.682 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-10-30T12:31:56.720 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-modules-core (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:56.725 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-10-30T12:31:57.028 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-30T12:31:57.044 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-10-30T12:31:57.078 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-10-30T12:31:57.125 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-30T12:31:57.145 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:57.183 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:57.323 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-mako. 2023-10-30T12:31:57.338 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-10-30T12:31:57.372 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-30T12:31:57.478 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-openssl. 2023-10-30T12:31:57.499 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-10-30T12:31:57.676 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-30T12:31:57.692 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-30T12:31:57.734 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-30T12:31:57.919 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-10-30T12:31:58.020 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-30T12:31:58.036 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-10-30T12:31:58.078 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-10-30T12:31:58.381 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webob. 2023-10-30T12:31:58.397 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-10-30T12:31:58.439 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-10-30T12:31:58.623 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr. 2023-10-30T12:31:58.643 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:58.681 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:59.084 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mon. 2023-10-30T12:31:59.105 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:31:59.142 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mon (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:31:59.409 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-waitress. 2023-10-30T12:31:59.424 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-10-30T12:31:59.462 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-30T12:31:59.703 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-tempita. 2023-10-30T12:31:59.719 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-10-30T12:31:59.753 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-10-30T12:32:00.048 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-paste. 2023-10-30T12:32:00.064 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-10-30T12:32:00.069 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-osd. 2023-10-30T12:32:00.090 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:00.106 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-10-30T12:32:00.129 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-osd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:00.443 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-30T12:32:00.458 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-10-30T12:32:00.501 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-10-30T12:32:00.745 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-30T12:32:00.761 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-10-30T12:32:00.803 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-10-30T12:32:01.232 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webtest. 2023-10-30T12:32:01.248 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-10-30T12:32:01.290 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-10-30T12:32:01.768 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pecan. 2023-10-30T12:32:01.783 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-10-30T12:32:01.826 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-10-30T12:32:02.008 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph. 2023-10-30T12:32:02.021 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../061-ceph_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:02.057 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:02.172 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-certifi. 2023-10-30T12:32:02.188 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-10-30T12:32:02.231 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-10-30T12:32:02.352 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-fuse. 2023-10-30T12:32:02.373 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:02.410 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:02.550 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-idna. 2023-10-30T12:32:02.565 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-10-30T12:32:02.600 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-idna (3.3-1) ... 2023-10-30T12:32:02.886 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-urllib3. 2023-10-30T12:32:02.902 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-10-30T12:32:02.944 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-10-30T12:32:03.014 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mds. 2023-10-30T12:32:03.034 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:03.071 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mds (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:03.248 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-requests. 2023-10-30T12:32:03.263 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-10-30T12:32:03.306 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-30T12:32:03.500 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephadm. 2023-10-30T12:32:03.520 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../064-cephadm_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:03.558 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:03.601 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-30T12:32:03.616 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-10-30T12:32:03.658 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-30T12:32:03.911 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-30T12:32:03.926 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:32:03.928 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-30T12:32:03.948 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-10-30T12:32:03.961 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-modules-core (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:03.994 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-10-30T12:32:04.364 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-babel-localedata. 2023-10-30T12:32:04.374 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-30T12:32:04.385 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-10-30T12:32:04.389 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:04.430 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-30T12:32:04.433 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:04.719 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-openssl. 2023-10-30T12:32:04.734 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-10-30T12:32:04.769 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-10-30T12:32:05.072 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr. 2023-10-30T12:32:05.087 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:05.138 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:05.528 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-babel. 2023-10-30T12:32:05.533 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mon. 2023-10-30T12:32:05.549 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-10-30T12:32:05.549 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:05.585 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-10-30T12:32:05.591 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mon (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:05.897 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jinja2. 2023-10-30T12:32:05.909 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-10-30T12:32:05.955 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-10-30T12:32:06.284 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-30T12:32:06.304 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:32:06.333 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:06.386 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-osd. 2023-10-30T12:32:06.402 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:06.444 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-osd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:06.637 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-30T12:32:06.658 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-10-30T12:32:06.695 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-10-30T12:32:06.982 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-30T12:32:07.002 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-10-30T12:32:07.040 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-30T12:32:07.326 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-routes. 2023-10-30T12:32:07.347 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-10-30T12:32:07.393 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-10-30T12:32:07.623 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph. 2023-10-30T12:32:07.639 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../061-ceph_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:07.673 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:07.730 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-30T12:32:07.750 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:32:07.788 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-dashboard (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:07.984 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-fuse. 2023-10-30T12:32:08.000 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:08.042 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:08.216 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-10-30T12:32:08.228 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-10-30T12:32:08.299 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-30T12:32:08.412 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mds. 2023-10-30T12:32:08.428 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:08.470 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mds (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:08.820 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-joblib. 2023-10-30T12:32:08.823 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephadm. 2023-10-30T12:32:08.839 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../064-cephadm_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:08.841 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-10-30T12:32:08.878 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-10-30T12:32:08.881 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:09.168 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-30T12:32:09.184 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-10-30T12:32:09.226 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-10-30T12:32:09.248 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-10-30T12:32:09.269 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-10-30T12:32:09.298 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-10-30T12:32:09.512 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-babel-localedata. 2023-10-30T12:32:09.527 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-10-30T12:32:09.562 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-30T12:32:09.584 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-sklearn. 2023-10-30T12:32:09.605 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-10-30T12:32:09.643 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-30T12:32:10.453 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-30T12:32:10.474 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:32:10.518 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:10.568 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-babel. 2023-10-30T12:32:10.582 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-10-30T12:32:10.617 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-10-30T12:32:10.887 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jinja2. 2023-10-30T12:32:10.901 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-10-30T12:32:10.945 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-10-30T12:32:11.249 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-30T12:32:11.263 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:32:11.307 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:11.530 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cachetools. 2023-10-30T12:32:11.542 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-10-30T12:32:11.579 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-10-30T12:32:11.635 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-30T12:32:11.650 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-10-30T12:32:11.685 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-10-30T12:32:11.849 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-30T12:32:11.861 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-10-30T12:32:11.891 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-10-30T12:32:12.113 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-30T12:32:12.128 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-10-30T12:32:12.180 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-30T12:32:12.211 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-30T12:32:12.232 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-10-30T12:32:12.269 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-10-30T12:32:12.466 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-routes. 2023-10-30T12:32:12.481 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-10-30T12:32:12.516 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-10-30T12:32:12.555 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rsa. 2023-10-30T12:32:12.567 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-10-30T12:32:12.596 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rsa (4.8-1) ... 2023-10-30T12:32:12.844 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-30T12:32:12.860 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:32:12.894 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-dashboard (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:12.996 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-google-auth. 2023-10-30T12:32:13.017 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-10-30T12:32:13.054 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-10-30T12:32:13.289 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-10-30T12:32:13.304 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-10-30T12:32:13.339 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-30T12:32:13.382 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-30T12:32:13.403 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-10-30T12:32:13.432 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-30T12:32:13.793 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-joblib. 2023-10-30T12:32:13.794 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-websocket. 2023-10-30T12:32:13.807 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-10-30T12:32:13.815 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-10-30T12:32:13.851 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-10-30T12:32:13.851 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-10-30T12:32:14.197 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-30T12:32:14.218 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-10-30T12:32:14.255 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-30T12:32:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-10-30T12:32:14.286 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-10-30T12:32:14.338 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-10-30T12:32:14.624 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn. 2023-10-30T12:32:14.640 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-10-30T12:32:14.683 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-30T12:32:14.885 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-30T12:32:14.898 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:32:14.935 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:15.272 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-30T12:32:15.294 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-10-30T12:32:15.338 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-30T12:32:15.449 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-30T12:32:15.464 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:32:15.499 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:15.608 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-30T12:32:15.620 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-10-30T12:32:15.658 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-30T12:32:15.935 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package jq. 2023-10-30T12:32:15.957 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-10-30T12:32:16.002 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-10-30T12:32:16.263 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package socat. 2023-10-30T12:32:16.285 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-10-30T12:32:16.330 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-10-30T12:32:16.452 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cachetools. 2023-10-30T12:32:16.468 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-10-30T12:32:16.510 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-10-30T12:32:16.674 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package xmlstarlet. 2023-10-30T12:32:16.693 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-10-30T12:32:16.732 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-10-30T12:32:16.805 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-30T12:32:16.849 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-10-30T12:32:16.863 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-10-30T12:32:17.062 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-test. 2023-10-30T12:32:17.074 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../093-ceph-test_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:17.120 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-test (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:17.192 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-30T12:32:17.208 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-10-30T12:32:17.250 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-10-30T12:32:17.569 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rsa. 2023-10-30T12:32:17.585 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-10-30T12:32:17.652 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rsa (4.8-1) ... 2023-10-30T12:32:18.066 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-google-auth. 2023-10-30T12:32:18.082 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-10-30T12:32:18.124 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-10-30T12:32:18.427 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-30T12:32:18.443 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-10-30T12:32:18.486 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-30T12:32:18.789 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-websocket. 2023-10-30T12:32:18.804 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-10-30T12:32:18.877 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-10-30T12:32:19.176 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-30T12:32:19.192 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-10-30T12:32:19.225 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-30T12:32:19.724 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-volume. 2023-10-30T12:32:19.746 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:32:19.790 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-volume (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:19.947 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-30T12:32:19.962 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:32:20.005 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:20.145 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-30T12:32:20.167 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:20.211 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephfs-mirror (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:20.300 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-30T12:32:20.314 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-10-30T12:32:20.350 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-30T12:32:20.565 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-colorama. 2023-10-30T12:32:20.587 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-10-30T12:32:20.631 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-10-30T12:32:20.678 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-30T12:32:20.693 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-10-30T12:32:20.736 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-30T12:32:20.884 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephfs-shell. 2023-10-30T12:32:20.896 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:32:20.934 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephfs-shell (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:21.005 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package jq. 2023-10-30T12:32:21.021 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-10-30T12:32:21.064 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-10-30T12:32:21.186 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephfs-top. 2023-10-30T12:32:21.208 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:32:21.253 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephfs-top (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:21.308 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package socat. 2023-10-30T12:32:21.323 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-10-30T12:32:21.383 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-10-30T12:32:21.514 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-30T12:32:21.527 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:21.564 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcephfs-dev (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:21.661 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package xmlstarlet. 2023-10-30T12:32:21.677 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-10-30T12:32:21.712 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-10-30T12:32:21.951 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package nvme-cli. 2023-10-30T12:32:21.972 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-10-30T12:32:22.009 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-10-30T12:32:22.282 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-test. 2023-10-30T12:32:22.297 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../093-ceph-test_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:22.332 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-test (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:22.347 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-30T12:32:22.362 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-10-30T12:32:22.388 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-10-30T12:32:22.834 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-iniconfig. 2023-10-30T12:32:22.855 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-10-30T12:32:22.900 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-10-30T12:32:23.203 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pastescript. 2023-10-30T12:32:23.225 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-10-30T12:32:23.269 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-30T12:32:23.573 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pluggy. 2023-10-30T12:32:23.595 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-10-30T12:32:23.631 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-10-30T12:32:23.934 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-psutil. 2023-10-30T12:32:23.956 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-10-30T12:32:24.000 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-10-30T12:32:24.227 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-volume. 2023-10-30T12:32:24.244 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:32:24.285 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-volume (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:24.345 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-py. 2023-10-30T12:32:24.367 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-10-30T12:32:24.403 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-py (1.10.0-1) ... 2023-10-30T12:32:24.673 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-30T12:32:24.689 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:24.731 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephfs-mirror (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:24.748 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pygments. 2023-10-30T12:32:24.770 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-10-30T12:32:24.815 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-10-30T12:32:25.026 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-colorama. 2023-10-30T12:32:25.043 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-10-30T12:32:25.076 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-10-30T12:32:25.294 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-30T12:32:25.303 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephfs-shell. 2023-10-30T12:32:25.316 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-10-30T12:32:25.319 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:32:25.361 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-10-30T12:32:25.370 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephfs-shell (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:25.623 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephfs-top. 2023-10-30T12:32:25.639 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:32:25.664 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-toml. 2023-10-30T12:32:25.681 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephfs-top (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:25.686 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-10-30T12:32:25.730 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-10-30T12:32:25.934 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-30T12:32:25.950 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:25.992 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs-dev (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:26.033 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pytest. 2023-10-30T12:32:26.055 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-10-30T12:32:26.100 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-10-30T12:32:26.312 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package nvme-cli. 2023-10-30T12:32:26.328 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-10-30T12:32:26.370 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-10-30T12:32:26.412 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-simplejson. 2023-10-30T12:32:26.433 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-10-30T12:32:26.478 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-10-30T12:32:26.717 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-30T12:32:26.732 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-10-30T12:32:26.748 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-30T12:32:26.761 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-10-30T12:32:26.766 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-10-30T12:32:26.798 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-10-30T12:32:27.194 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-iniconfig. 2023-10-30T12:32:27.210 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-10-30T12:32:27.252 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-10-30T12:32:27.556 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastescript. 2023-10-30T12:32:27.571 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-10-30T12:32:27.594 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package radosgw. 2023-10-30T12:32:27.614 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-30T12:32:27.616 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../113-radosgw_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:27.660 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking radosgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:27.967 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pluggy. 2023-10-30T12:32:27.983 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-10-30T12:32:28.025 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-10-30T12:32:28.320 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-psutil. 2023-10-30T12:32:28.336 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-10-30T12:32:28.378 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-10-30T12:32:28.421 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package rbd-fuse. 2023-10-30T12:32:28.434 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:28.471 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking rbd-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:28.723 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-py. 2023-10-30T12:32:28.739 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-10-30T12:32:28.765 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-10-30T12:32:28.781 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-py (1.10.0-1) ... 2023-10-30T12:32:29.034 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-30T12:32:29.109 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pygments. 2023-10-30T12:32:29.125 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-10-30T12:32:29.160 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-10-30T12:32:29.167 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-10-30T12:32:29.425 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-10-30T12:32:29.621 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-30T12:32:29.636 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-10-30T12:32:29.679 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-10-30T12:32:29.757 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-10-30T12:32:29.982 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-toml. 2023-10-30T12:32:29.997 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-10-30T12:32:30.040 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-10-30T12:32:30.184 INFO:teuthology.orchestra.run.smithi154.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-10-30T12:32:30.287 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:30.368 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pytest. 2023-10-30T12:32:30.384 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-10-30T12:32:30.426 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-10-30T12:32:30.738 INFO:teuthology.orchestra.run.smithi154.stdout:Adding system user cephadm....done 2023-10-30T12:32:30.746 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplejson. 2023-10-30T12:32:30.763 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-10-30T12:32:30.804 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-10-30T12:32:30.868 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-30T12:32:31.076 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-30T12:32:31.092 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-10-30T12:32:31.134 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-10-30T12:32:31.144 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-10-30T12:32:31.404 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-10-30T12:32:31.513 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-10-30T12:32:31.756 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-30T12:32:31.863 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package radosgw. 2023-10-30T12:32:31.878 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../113-radosgw_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:31.921 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking radosgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:32.007 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-10-30T12:32:32.133 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-py (1.10.0-1) ... 2023-10-30T12:32:32.435 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-10-30T12:32:32.757 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package rbd-fuse. 2023-10-30T12:32:32.774 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:32:32.803 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-10-30T12:32:32.832 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking rbd-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:33.080 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-10-30T12:32:33.102 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-10-30T12:32:33.331 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-10-30T12:32:33.412 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-30T12:32:33.538 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-10-30T12:32:33.607 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-10-30T12:32:33.811 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-10-30T12:32:33.833 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-10-30T12:32:34.093 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-10-30T12:32:34.209 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-10-30T12:32:34.352 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-ceph-argparse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:34.686 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-30T12:32:34.809 INFO:teuthology.orchestra.run.smithi012.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-10-30T12:32:34.865 INFO:teuthology.orchestra.run.smithi012.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-10-30T12:32:34.879 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-10-30T12:32:34.914 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:35.139 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-10-30T12:32:35.398 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-30T12:32:35.468 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user cephadm....done 2023-10-30T12:32:35.533 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-10-30T12:32:35.628 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-30T12:32:35.893 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-30T12:32:35.913 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-10-30T12:32:36.018 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-10-30T12:32:36.189 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-10-30T12:32:36.277 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-10-30T12:32:36.290 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-10-30T12:32:36.554 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-10-30T12:32:36.558 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-30T12:32:36.809 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-10-30T12:32:36.847 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-10-30T12:32:36.935 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-py (1.10.0-1) ... 2023-10-30T12:32:37.253 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-10-30T12:32:37.507 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-10-30T12:32:37.639 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-10-30T12:32:37.758 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-10-30T12:32:37.868 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-10-30T12:32:37.923 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-10-30T12:32:37.994 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-30T12:32:38.200 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-10-30T12:32:38.303 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-10-30T12:32:38.467 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-10-30T12:32:38.680 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-10-30T12:32:38.735 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-10-30T12:32:38.930 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-10-30T12:32:39.069 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-10-30T12:32:39.337 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-argparse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:39.349 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-30T12:32:39.613 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-30T12:32:39.634 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-10-30T12:32:39.739 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-10-30T12:32:39.935 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-30T12:32:39.999 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-10-30T12:32:40.242 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-30T12:32:40.278 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-10-30T12:32:40.342 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-10-30T12:32:40.529 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-10-30T12:32:40.655 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:40.702 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-30T12:32:40.828 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-10-30T12:32:40.940 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-idna (3.3-1) ... 2023-10-30T12:32:41.104 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-10-30T12:32:41.266 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-30T12:32:41.380 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-10-30T12:32:41.723 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-10-30T12:32:42.355 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:32:42.417 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-10-30T12:32:42.489 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-toml (0.10.2-1) ... 2023-10-30T12:32:42.701 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-10-30T12:32:42.741 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-30T12:32:42.827 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-10-30T12:32:42.850 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-10-30T12:32:42.953 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-30T12:32:42.968 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-10-30T12:32:43.277 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-10-30T12:32:43.288 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-10-30T12:32:43.537 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-10-30T12:32:43.697 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-10-30T12:32:43.772 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:32:43.898 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-30T12:32:43.957 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-10-30T12:32:44.024 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-10-30T12:32:44.342 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-10-30T12:32:44.343 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-30T12:32:44.616 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-10-30T12:32:44.652 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-10-30T12:32:44.877 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-10-30T12:32:44.978 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-30T12:32:45.145 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-30T12:32:45.346 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-10-30T12:32:45.388 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-10-30T12:32:45.614 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-10-30T12:32:45.647 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-30T12:32:45.740 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:45.774 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-10-30T12:32:46.025 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-idna (3.3-1) ... 2023-10-30T12:32:46.050 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-30T12:32:46.168 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-30T12:32:46.392 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-30T12:32:46.469 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-10-30T12:32:46.762 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-10-30T12:32:47.013 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-10-30T12:32:47.476 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-10-30T12:32:47.673 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:32:47.724 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-30T12:32:47.790 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-toml (0.10.2-1) ... 2023-10-30T12:32:47.850 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-10-30T12:32:48.058 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-30T12:32:48.168 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-10-30T12:32:48.235 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-10-30T12:32:48.294 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-10-30T12:32:48.495 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-portend (3.0.0-1) ... 2023-10-30T12:32:48.603 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-10-30T12:32:48.746 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:32:48.872 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-10-30T12:32:48.888 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-10-30T12:32:49.148 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:32:49.181 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-10-30T12:32:49.265 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-30T12:32:49.391 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-10-30T12:32:49.507 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-30T12:32:49.709 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-10-30T12:32:49.775 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-10-30T12:32:49.901 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rsa (4.8-1) ... 2023-10-30T12:32:49.977 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-10-30T12:32:50.169 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-10-30T12:32:50.253 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-10-30T12:32:50.445 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-30T12:32:50.529 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-30T12:32:50.721 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-10-30T12:32:50.797 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-10-30T12:32:51.065 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-30T12:32:51.131 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-30T12:32:51.166 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-10-30T12:32:51.424 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-10-30T12:32:51.442 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-30T12:32:51.568 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-30T12:32:51.776 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-30T12:32:51.869 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-10-30T12:32:51.902 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librados2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:52.037 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librgw2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:52.179 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-10-30T12:32:52.257 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libsqlite3-mod-ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:52.374 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcephfs2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:52.447 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-10-30T12:32:52.475 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libradosstriper1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:52.601 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-10-30T12:32:52.873 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-10-30T12:32:52.886 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librbd1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:52.996 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-modules-core (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:53.108 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-30T12:32:53.122 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:53.217 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-10-30T12:32:53.336 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-30T12:32:53.336 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-30T12:32:53.636 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-10-30T12:32:53.874 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcephfs-dev (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:53.912 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-portend (3.0.0-1) ... 2023-10-30T12:32:54.000 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rados (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:54.117 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephfs-top (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:54.205 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:32:54.243 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-30T12:32:54.306 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-10-30T12:32:54.624 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-10-30T12:32:54.925 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-30T12:32:55.242 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-10-30T12:32:55.369 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rsa (4.8-1) ... 2023-10-30T12:32:55.486 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rbd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:55.613 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up rbd-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:55.653 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-10-30T12:32:55.739 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:55.865 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cephfs (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:55.912 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-30T12:32:55.983 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephfs-shell (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:56.084 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:56.173 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-10-30T12:32:56.574 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-30T12:32:56.618 INFO:teuthology.orchestra.run.smithi154.stdout:Adding group ceph....done 2023-10-30T12:32:56.858 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-10-30T12:32:56.986 INFO:teuthology.orchestra.run.smithi154.stdout:Adding system user ceph....done 2023-10-30T12:32:57.164 INFO:teuthology.orchestra.run.smithi154.stdout:Setting system user ceph properties....done 2023-10-30T12:32:57.183 INFO:teuthology.orchestra.run.smithi154.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-30T12:32:57.211 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-30T12:32:57.320 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-30T12:32:57.337 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librados2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:57.492 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librgw2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:57.663 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-30T12:32:57.691 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libsqlite3-mod-ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:57.792 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:57.910 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libradosstriper1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:58.035 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-10-30T12:32:58.219 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephfs-mirror (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:58.419 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-30T12:32:58.420 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-30T12:32:58.437 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librbd1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:58.546 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-modules-core (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:58.647 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:58.882 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-30T12:32:58.883 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-30T12:32:58.964 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-test (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:59.090 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up radosgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:59.498 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs-dev (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:59.585 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-30T12:32:59.586 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-30T12:32:59.624 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rados (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:59.751 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephfs-top (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:32:59.877 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-30T12:33:00.125 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-base (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:00.482 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-30T12:33:01.019 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mds (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:01.239 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-30T12:33:01.239 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-30T12:33:01.244 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rbd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:01.348 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up rbd-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:01.449 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:01.567 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cephfs (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:01.693 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephfs-shell (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:01.762 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:01.819 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:01.979 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-30T12:33:01.980 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-30T12:33:02.399 INFO:teuthology.orchestra.run.smithi012.stdout:Adding group ceph....done 2023-10-30T12:33:02.514 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-osd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:02.774 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user ceph....done 2023-10-30T12:33:02.823 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-30T12:33:02.823 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-30T12:33:02.986 INFO:teuthology.orchestra.run.smithi012.stdout:Setting system user ceph properties....done 2023-10-30T12:33:03.391 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-k8sevents (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:03.391 INFO:teuthology.orchestra.run.smithi012.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-30T12:33:03.391 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-30T12:33:03.467 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:03.561 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-30T12:33:03.609 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mon (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:03.820 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-30T12:33:03.820 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-30T12:33:04.255 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephfs-mirror (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:04.327 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:04.469 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-30T12:33:04.470 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-30T12:33:04.470 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:04.596 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-dashboard (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:04.747 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-volume (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:05.097 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-test (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:05.159 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-30T12:33:05.198 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up radosgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:05.738 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-30T12:33:05.738 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-30T12:33:05.984 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-30T12:33:06.400 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-base (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:06.440 INFO:teuthology.orchestra.run.smithi154.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-30T12:33:06.442 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-30T12:33:06.549 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2023-10-30T12:33:06.737 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2023-10-30T12:33:06.738 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2023-10-30T12:33:06.764 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-30T12:33:06.901 INFO:teuthology.orchestra.run.smithi154.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-10-30T12:33:06.902 INFO:teuthology.orchestra.run.smithi154.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-10-30T12:33:06.902 INFO:teuthology.orchestra.run.smithi154.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-10-30T12:33:06.902 INFO:teuthology.orchestra.run.smithi154.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-10-30T12:33:06.903 INFO:teuthology.orchestra.run.smithi154.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-10-30T12:33:06.903 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages were automatically installed and are no longer required: 2023-10-30T12:33:06.903 INFO:teuthology.orchestra.run.smithi154.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-30T12:33:06.904 INFO:teuthology.orchestra.run.smithi154.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-30T12:33:06.905 INFO:teuthology.orchestra.run.smithi154.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-30T12:33:06.905 INFO:teuthology.orchestra.run.smithi154.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-10-30T12:33:06.906 INFO:teuthology.orchestra.run.smithi154.stdout: python2.7-minimal 2023-10-30T12:33:06.906 INFO:teuthology.orchestra.run.smithi154.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-30T12:33:06.906 INFO:teuthology.orchestra.run.smithi154.stdout:The following additional packages will be installed: 2023-10-30T12:33:06.906 INFO:teuthology.orchestra.run.smithi154.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-10-30T12:33:06.907 INFO:teuthology.orchestra.run.smithi154.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-10-30T12:33:06.907 INFO:teuthology.orchestra.run.smithi154.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-10-30T12:33:06.907 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-10-30T12:33:06.907 INFO:teuthology.orchestra.run.smithi154.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-30T12:33:06.907 INFO:teuthology.orchestra.run.smithi154.stdout:Suggested packages: 2023-10-30T12:33:06.908 INFO:teuthology.orchestra.run.smithi154.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-10-30T12:33:06.908 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-10-30T12:33:06.908 INFO:teuthology.orchestra.run.smithi154.stdout: libengine-pkcs11-openssl 2023-10-30T12:33:06.944 INFO:teuthology.orchestra.run.smithi154.stdout:The following NEW packages will be installed: 2023-10-30T12:33:06.944 INFO:teuthology.orchestra.run.smithi154.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-10-30T12:33:06.944 INFO:teuthology.orchestra.run.smithi154.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-10-30T12:33:06.944 INFO:teuthology.orchestra.run.smithi154.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-10-30T12:33:06.945 INFO:teuthology.orchestra.run.smithi154.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-10-30T12:33:06.945 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-10-30T12:33:06.945 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-10-30T12:33:06.945 INFO:teuthology.orchestra.run.smithi154.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-30T12:33:06.946 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages will be upgraded: 2023-10-30T12:33:06.946 INFO:teuthology.orchestra.run.smithi154.stdout: libssl3 2023-10-30T12:33:07.003 INFO:teuthology.orchestra.run.smithi154.stdout:1 upgraded, 39 newly installed, 0 to remove and 259 not upgraded. 2023-10-30T12:33:07.003 INFO:teuthology.orchestra.run.smithi154.stdout:Need to get 54.6 MB of archives. 2023-10-30T12:33:07.004 INFO:teuthology.orchestra.run.smithi154.stdout:After this operation, 202 MB of additional disk space will be used. 2023-10-30T12:33:07.004 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-10-30T12:33:07.131 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-10-30T12:33:07.249 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-10-30T12:33:07.270 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-10-30T12:33:07.272 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-10-30T12:33:07.276 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-10-30T12:33:07.278 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-10-30T12:33:07.279 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-10-30T12:33:07.282 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-10-30T12:33:07.284 INFO:teuthology.orchestra.run.smithi154.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-10-30T12:33:07.284 INFO:teuthology.orchestra.run.smithi154.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-10-30T12:33:07.300 INFO:teuthology.orchestra.run.smithi154.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-10-30T12:33:07.321 INFO:teuthology.orchestra.run.smithi154.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-10-30T12:33:07.485 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mds (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:07.724 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-30T12:33:07.724 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-30T12:33:07.845 INFO:teuthology.orchestra.run.smithi154.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-10-30T12:33:07.848 INFO:teuthology.orchestra.run.smithi154.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-10-30T12:33:07.848 INFO:teuthology.orchestra.run.smithi154.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-10-30T12:33:07.853 INFO:teuthology.orchestra.run.smithi154.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-10-30T12:33:07.853 INFO:teuthology.orchestra.run.smithi154.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-10-30T12:33:07.861 INFO:teuthology.orchestra.run.smithi154.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-10-30T12:33:07.864 INFO:teuthology.orchestra.run.smithi154.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-10-30T12:33:07.876 INFO:teuthology.orchestra.run.smithi154.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-10-30T12:33:07.877 INFO:teuthology.orchestra.run.smithi154.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-10-30T12:33:07.910 INFO:teuthology.orchestra.run.smithi154.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-10-30T12:33:07.910 INFO:teuthology.orchestra.run.smithi154.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-10-30T12:33:07.921 INFO:teuthology.orchestra.run.smithi154.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-10-30T12:33:07.950 INFO:teuthology.orchestra.run.smithi154.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-10-30T12:33:07.955 INFO:teuthology.orchestra.run.smithi154.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-10-30T12:33:07.956 INFO:teuthology.orchestra.run.smithi154.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-10-30T12:33:07.956 INFO:teuthology.orchestra.run.smithi154.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-10-30T12:33:07.957 INFO:teuthology.orchestra.run.smithi154.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-10-30T12:33:07.973 INFO:teuthology.orchestra.run.smithi154.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-10-30T12:33:07.975 INFO:teuthology.orchestra.run.smithi154.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-10-30T12:33:08.298 INFO:teuthology.orchestra.run.smithi154.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-10-30T12:33:08.299 INFO:teuthology.orchestra.run.smithi154.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-10-30T12:33:08.349 INFO:teuthology.orchestra.run.smithi154.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-10-30T12:33:08.350 INFO:teuthology.orchestra.run.smithi154.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-10-30T12:33:08.350 INFO:teuthology.orchestra.run.smithi154.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-10-30T12:33:08.351 INFO:teuthology.orchestra.run.smithi154.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-10-30T12:33:08.368 INFO:teuthology.orchestra.run.smithi154.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-10-30T12:33:08.369 INFO:teuthology.orchestra.run.smithi154.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-10-30T12:33:08.419 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-30T12:33:08.633 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-30T12:33:08.768 INFO:teuthology.orchestra.run.smithi154.stdout:Preconfiguring packages ... 2023-10-30T12:33:08.926 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 54.6 MB in 1s (38.5 MB/s) 2023-10-30T12:33:09.058 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package flex. 2023-10-30T12:33:09.142 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137425 files and directories currently installed.) 2023-10-30T12:33:09.147 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-10-30T12:33:09.181 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking flex (2.6.4-8build2) ... 2023-10-30T12:33:09.272 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-osd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:09.571 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-30T12:33:09.571 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-30T12:33:09.785 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-10-30T12:33:09.954 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-10-30T12:33:10.190 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-k8sevents (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:10.308 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:10.451 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mon (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:10.530 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-30T12:33:10.667 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-30T12:33:10.667 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-30T12:33:11.074 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package bison. 2023-10-30T12:33:11.163 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137498 files and directories currently installed.) 2023-10-30T12:33:11.167 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-10-30T12:33:11.217 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-10-30T12:33:11.293 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:11.445 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:11.571 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-dashboard (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:11.635 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-30T12:33:11.657 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-10-30T12:33:11.688 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-volume (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:33:11.701 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-10-30T12:33:11.996 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package dump. 2023-10-30T12:33:12.018 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-10-30T12:33:12.041 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-30T12:33:12.062 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking dump (0.4b47-3) ... 2023-10-30T12:33:12.333 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package indent. 2023-10-30T12:33:12.346 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:33:12.383 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-30T12:33:12.653 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-30T12:33:12.675 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-10-30T12:33:12.719 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-30T12:33:13.006 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-30T12:33:13.029 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-10-30T12:33:13.065 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-30T12:33:13.200 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-30T12:33:13.317 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-30T12:33:13.330 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-10-30T12:33:13.367 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-10-30T12:33:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-30T12:33:13.676 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-30T12:33:13.712 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libfl2:amd64. 2023-10-30T12:33:13.734 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-10-30T12:33:13.753 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2023-10-30T12:33:13.795 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-10-30T12:33:13.924 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2023-10-30T12:33:13.925 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2023-10-30T12:33:14.023 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-10-30T12:33:14.036 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-10-30T12:33:14.065 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-30T12:33:14.083 INFO:teuthology.orchestra.run.smithi012.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-10-30T12:33:14.084 INFO:teuthology.orchestra.run.smithi012.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-10-30T12:33:14.084 INFO:teuthology.orchestra.run.smithi012.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-10-30T12:33:14.084 INFO:teuthology.orchestra.run.smithi012.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-10-30T12:33:14.084 INFO:teuthology.orchestra.run.smithi012.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-10-30T12:33:14.085 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2023-10-30T12:33:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-30T12:33:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-30T12:33:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-30T12:33:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-30T12:33:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-30T12:33:14.087 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-30T12:33:14.087 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2023-10-30T12:33:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-10-30T12:33:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-10-30T12:33:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-10-30T12:33:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-10-30T12:33:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-30T12:33:14.088 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2023-10-30T12:33:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-10-30T12:33:14.089 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-10-30T12:33:14.089 INFO:teuthology.orchestra.run.smithi012.stdout: libengine-pkcs11-openssl 2023-10-30T12:33:14.139 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2023-10-30T12:33:14.140 INFO:teuthology.orchestra.run.smithi012.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-10-30T12:33:14.140 INFO:teuthology.orchestra.run.smithi012.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-10-30T12:33:14.140 INFO:teuthology.orchestra.run.smithi012.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-10-30T12:33:14.141 INFO:teuthology.orchestra.run.smithi012.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-10-30T12:33:14.141 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-10-30T12:33:14.141 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-10-30T12:33:14.141 INFO:teuthology.orchestra.run.smithi012.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-30T12:33:14.142 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages will be upgraded: 2023-10-30T12:33:14.142 INFO:teuthology.orchestra.run.smithi012.stdout: libssl3 2023-10-30T12:33:14.292 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjson-perl. 2023-10-30T12:33:14.305 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-10-30T12:33:14.326 INFO:teuthology.orchestra.run.smithi012.stdout:1 upgraded, 39 newly installed, 0 to remove and 259 not upgraded. 2023-10-30T12:33:14.326 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 54.6 MB of archives. 2023-10-30T12:33:14.327 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 202 MB of additional disk space will be used. 2023-10-30T12:33:14.327 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-10-30T12:33:14.342 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-10-30T12:33:14.662 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libllvm14:amd64. 2023-10-30T12:33:14.684 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-10-30T12:33:14.729 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-30T12:33:14.821 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-10-30T12:33:15.078 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-10-30T12:33:15.113 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-10-30T12:33:15.115 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-10-30T12:33:15.120 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-10-30T12:33:15.123 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-10-30T12:33:15.124 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-10-30T12:33:15.127 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-10-30T12:33:15.130 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-10-30T12:33:15.294 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-10-30T12:33:15.295 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-10-30T12:33:15.546 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-10-30T12:33:16.554 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-10-30T12:33:16.556 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-10-30T12:33:16.557 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-10-30T12:33:16.582 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-10-30T12:33:16.583 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-10-30T12:33:16.587 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-10-30T12:33:16.589 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-10-30T12:33:16.596 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-10-30T12:33:16.633 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-30T12:33:16.655 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-30T12:33:16.700 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:33:16.760 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-10-30T12:33:16.961 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-30T12:33:16.974 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-30T12:33:17.011 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:33:17.289 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-30T12:33:17.311 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-30T12:33:17.355 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:33:17.498 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-10-30T12:33:17.499 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-10-30T12:33:17.525 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-10-30T12:33:17.603 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-10-30T12:33:17.610 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-10-30T12:33:17.612 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-10-30T12:33:17.613 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-10-30T12:33:17.613 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-10-30T12:33:17.641 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-10-30T12:33:17.644 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-10-30T12:33:17.700 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-30T12:33:17.722 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-10-30T12:33:17.767 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-30T12:33:18.111 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-30T12:33:18.133 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-10-30T12:33:18.194 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-30T12:33:18.270 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-10-30T12:33:18.353 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-10-30T12:33:18.353 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-10-30T12:33:18.353 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-10-30T12:33:18.443 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-10-30T12:33:18.448 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-10-30T12:33:18.514 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-30T12:33:18.537 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:33:18.581 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:33:18.912 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-10-30T12:33:18.913 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-10-30T12:33:18.917 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-30T12:33:18.939 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:33:18.984 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:33:19.278 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-30T12:33:19.291 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:33:19.323 INFO:teuthology.orchestra.run.smithi012.stdout:Preconfiguring packages ... 2023-10-30T12:33:19.328 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:33:19.476 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 54.6 MB in 5s (11.4 MB/s) 2023-10-30T12:33:19.547 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-30T12:33:19.560 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-10-30T12:33:19.592 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package flex. 2023-10-30T12:33:19.597 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-30T12:33:19.668 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139851 files and directories currently installed.) 2023-10-30T12:33:19.671 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-10-30T12:33:19.725 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking flex (2.6.4-8build2) ... 2023-10-30T12:33:20.118 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-30T12:33:20.140 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-10-30T12:33:20.184 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-10-30T12:33:20.285 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-10-30T12:33:20.455 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-10-30T12:33:20.545 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package modemmanager. 2023-10-30T12:33:20.568 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-30T12:33:20.612 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:33:20.854 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-30T12:33:21.243 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package network-manager. 2023-10-30T12:33:21.266 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-10-30T12:33:21.318 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-10-30T12:33:21.405 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package bison. 2023-10-30T12:33:21.480 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139924 files and directories currently installed.) 2023-10-30T12:33:21.484 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-10-30T12:33:21.533 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-10-30T12:33:21.908 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-30T12:33:21.923 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-10-30T12:33:21.966 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-10-30T12:33:21.985 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ppp. 2023-10-30T12:33:22.008 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-10-30T12:33:22.303 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package dump. 2023-10-30T12:33:22.318 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-10-30T12:33:22.325 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-10-30T12:33:22.361 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking dump (0.4b47-3) ... 2023-10-30T12:33:22.648 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package indent. 2023-10-30T12:33:22.663 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:33:22.665 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package pptp-linux. 2023-10-30T12:33:22.687 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-10-30T12:33:22.706 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-30T12:33:22.731 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-10-30T12:33:22.917 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-30T12:33:22.933 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-10-30T12:33:22.967 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-30T12:33:22.984 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-30T12:33:23.006 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-10-30T12:33:23.050 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-10-30T12:33:23.254 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-30T12:33:23.269 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-10-30T12:33:23.304 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-30T12:33:23.354 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-30T12:33:23.376 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-10-30T12:33:23.420 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-client-common (238) ... 2023-10-30T12:33:23.557 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-30T12:33:23.573 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-10-30T12:33:23.615 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-10-30T12:33:23.716 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-client-14. 2023-10-30T12:33:23.738 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:33:23.782 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:33:24.019 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libfl2:amd64. 2023-10-30T12:33:24.035 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-10-30T12:33:24.077 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-10-30T12:33:24.330 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-10-30T12:33:24.346 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-10-30T12:33:24.388 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-30T12:33:24.396 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-common. 2023-10-30T12:33:24.418 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-10-30T12:33:24.481 INFO:teuthology.orchestra.run.smithi154.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-30T12:33:24.502 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-common (238) ... 2023-10-30T12:33:24.640 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjson-perl. 2023-10-30T12:33:24.656 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-10-30T12:33:24.724 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-10-30T12:33:24.911 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-14. 2023-10-30T12:33:24.934 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:33:24.973 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:33:25.127 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libllvm14:amd64. 2023-10-30T12:33:25.143 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-10-30T12:33:25.185 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-30T12:33:26.340 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql. 2023-10-30T12:33:26.355 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-10-30T12:33:26.398 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql (14+238) ... 2023-10-30T12:33:26.630 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-30T12:33:26.646 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-30T12:33:26.679 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:33:26.899 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-30T12:33:26.915 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-30T12:33:26.957 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:33:26.980 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-client. 2023-10-30T12:33:27.003 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-10-30T12:33:27.318 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-30T12:33:27.334 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-30T12:33:27.376 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:33:27.385 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-client (14+238) ... 2023-10-30T12:33:27.721 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-30T12:33:27.737 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-10-30T12:33:27.738 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-30T12:33:27.751 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-10-30T12:33:27.771 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-30T12:33:27.788 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-contrib (14+238) ... 2023-10-30T12:33:28.015 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-30T12:33:28.031 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-10-30T12:33:28.065 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-30T12:33:28.083 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package tcl8.6. 2023-10-30T12:33:28.105 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-10-30T12:33:28.150 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-30T12:33:28.368 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-30T12:33:28.384 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:33:28.418 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:33:28.427 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package tcl. 2023-10-30T12:33:28.442 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-10-30T12:33:28.491 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-10-30T12:33:28.795 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-30T12:33:28.812 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:33:28.822 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package wpasupplicant. 2023-10-30T12:33:28.844 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-10-30T12:33:28.854 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:33:28.888 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-30T12:33:29.243 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-30T12:33:29.256 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-10-30T12:33:29.293 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-10-30T12:33:29.449 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-30T12:33:29.465 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:33:29.499 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:33:29.655 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-30T12:33:29.677 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-10-30T12:33:29.729 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-30T12:33:29.785 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-30T12:33:29.801 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-10-30T12:33:29.843 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-30T12:33:30.005 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-client-common (238) ... 2023-10-30T12:33:30.235 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up flex (2.6.4-8build2) ... 2023-10-30T12:33:30.338 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-30T12:33:30.354 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-10-30T12:33:30.361 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-30T12:33:30.396 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-10-30T12:33:30.487 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:33:30.613 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-10-30T12:33:30.740 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-30T12:33:30.749 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package modemmanager. 2023-10-30T12:33:30.765 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-30T12:33:30.807 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:33:30.866 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:33:31.419 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-30T12:33:31.457 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package network-manager. 2023-10-30T12:33:31.473 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-10-30T12:33:31.512 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-30T12:33:31.524 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-10-30T12:33:31.896 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-30T12:33:31.896 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-30T12:33:32.098 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ppp. 2023-10-30T12:33:32.114 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-10-30T12:33:32.432 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-10-30T12:33:32.453 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-10-30T12:33:32.566 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-30T12:33:32.693 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-10-30T12:33:32.795 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package pptp-linux. 2023-10-30T12:33:32.809 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-10-30T12:33:32.818 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-10-30T12:33:32.853 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-10-30T12:33:32.945 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-10-30T12:33:33.071 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-30T12:33:33.106 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-30T12:33:33.121 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-10-30T12:33:33.164 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-10-30T12:33:33.197 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-10-30T12:33:33.467 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-30T12:33:33.482 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-10-30T12:33:33.517 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-client-common (238) ... 2023-10-30T12:33:33.813 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-client-14. 2023-10-30T12:33:33.819 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-30T12:33:33.829 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:33:33.871 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:33:33.945 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up dump (0.4b47-3) ... 2023-10-30T12:33:34.156 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-30T12:33:34.222 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:33:34.365 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-30T12:33:34.474 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up tcl (8.6.11+1build2) ... 2023-10-30T12:33:34.577 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-common. 2023-10-30T12:33:34.592 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjson-perl (4.04000-1) ... 2023-10-30T12:33:34.593 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-10-30T12:33:34.680 INFO:teuthology.orchestra.run.smithi012.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-30T12:33:34.710 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-common (238) ... 2023-10-30T12:33:34.718 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-10-30T12:33:34.808 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-10-30T12:33:34.878 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-30T12:33:35.004 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-30T12:33:35.169 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-14. 2023-10-30T12:33:35.180 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-30T12:33:35.185 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:33:35.230 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:33:35.315 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:33:35.441 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-10-30T12:33:35.609 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-client (14+238) ... 2023-10-30T12:33:35.735 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-10-30T12:33:36.230 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:33:36.230 INFO:teuthology.orchestra.run.smithi154.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-30T12:33:36.230 INFO:teuthology.orchestra.run.smithi154.stdout:which means they are currently configured by ifupdown: 2023-10-30T12:33:36.231 INFO:teuthology.orchestra.run.smithi154.stdout:- enp3s0f1 2023-10-30T12:33:36.231 INFO:teuthology.orchestra.run.smithi154.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-30T12:33:36.231 INFO:teuthology.orchestra.run.smithi154.stdout:remove their configuration from /etc/network/interfaces. 2023-10-30T12:33:36.231 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:33:36.346 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-30T12:33:36.472 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql. 2023-10-30T12:33:36.487 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-10-30T12:33:36.530 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql (14+238) ... 2023-10-30T12:33:36.699 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-30T12:33:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-client. 2023-10-30T12:33:36.822 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-10-30T12:33:36.866 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-client (14+238) ... 2023-10-30T12:33:37.086 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-30T12:33:37.109 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-30T12:33:37.125 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-10-30T12:33:37.184 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-contrib (14+238) ... 2023-10-30T12:33:37.479 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package tcl8.6. 2023-10-30T12:33:37.494 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-10-30T12:33:37.545 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-30T12:33:37.832 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package tcl. 2023-10-30T12:33:37.838 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:33:37.847 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-10-30T12:33:37.895 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-10-30T12:33:38.173 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-30T12:33:38.226 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package wpasupplicant. 2023-10-30T12:33:38.242 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-10-30T12:33:38.284 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-30T12:33:38.299 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-10-30T12:33:38.442 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-common (238) ... 2023-10-30T12:33:38.655 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-30T12:33:38.672 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-10-30T12:33:38.714 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-10-30T12:33:39.075 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-30T12:33:39.092 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-10-30T12:33:39.151 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-30T12:33:39.313 INFO:teuthology.orchestra.run.smithi154.stdout:Adding user postgres to group ssl-cert 2023-10-30T12:33:39.444 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-client-common (238) ... 2023-10-30T12:33:39.484 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:33:39.485 INFO:teuthology.orchestra.run.smithi154.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-30T12:33:39.683 INFO:teuthology.orchestra.run.smithi154.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-30T12:33:39.683 INFO:teuthology.orchestra.run.smithi154.stdout:Removing obsolete dictionary files: 2023-10-30T12:33:39.689 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up flex (2.6.4-8build2) ... 2023-10-30T12:33:39.815 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-30T12:33:39.942 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:33:40.067 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-10-30T12:33:40.169 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-30T12:33:40.270 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:33:40.287 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-30T12:33:40.719 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:33:41.049 INFO:teuthology.orchestra.run.smithi154.stdout:Creating new PostgreSQL cluster 14/main ... 2023-10-30T12:33:41.059 INFO:teuthology.orchestra.run.smithi154.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-10-30T12:33:41.131 INFO:teuthology.orchestra.run.smithi154.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-30T12:33:41.131 INFO:teuthology.orchestra.run.smithi154.stdout:This user must also own the server process. 2023-10-30T12:33:41.131 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:33:41.132 INFO:teuthology.orchestra.run.smithi154.stdout:The database cluster will be initialized with locale "en_US". 2023-10-30T12:33:41.132 INFO:teuthology.orchestra.run.smithi154.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-30T12:33:41.132 INFO:teuthology.orchestra.run.smithi154.stdout:The default text search configuration will be set to "english". 2023-10-30T12:33:41.132 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:33:41.132 INFO:teuthology.orchestra.run.smithi154.stdout:Data page checksums are disabled. 2023-10-30T12:33:41.133 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:33:41.133 INFO:teuthology.orchestra.run.smithi154.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-10-30T12:33:41.133 INFO:teuthology.orchestra.run.smithi154.stdout:creating subdirectories ... ok 2023-10-30T12:33:41.133 INFO:teuthology.orchestra.run.smithi154.stdout:selecting dynamic shared memory implementation ... posix 2023-10-30T12:33:41.147 INFO:teuthology.orchestra.run.smithi154.stdout:selecting default max_connections ... 100 2023-10-30T12:33:41.167 INFO:teuthology.orchestra.run.smithi154.stdout:selecting default shared_buffers ... 128MB 2023-10-30T12:33:41.174 INFO:teuthology.orchestra.run.smithi154.stdout:selecting default time zone ... Etc/UTC 2023-10-30T12:33:41.175 INFO:teuthology.orchestra.run.smithi154.stdout:creating configuration files ... ok 2023-10-30T12:33:41.287 INFO:teuthology.orchestra.run.smithi154.stdout:running bootstrap script ... ok 2023-10-30T12:33:41.288 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-30T12:33:41.383 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-30T12:33:41.743 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-30T12:33:41.744 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-30T12:33:41.953 INFO:teuthology.orchestra.run.smithi154.stdout:performing post-bootstrap initialization ... ok 2023-10-30T12:33:42.427 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-10-30T12:33:42.553 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-30T12:33:42.688 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-10-30T12:33:42.732 INFO:teuthology.orchestra.run.smithi154.stdout:syncing data to disk ... ok 2023-10-30T12:33:42.814 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-10-30T12:33:42.939 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-10-30T12:33:43.065 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-30T12:33:43.175 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-30T12:33:43.183 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-10-30T12:33:43.906 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-30T12:33:44.032 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up dump (0.4b47-3) ... 2023-10-30T12:33:44.127 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-30T12:33:44.191 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:33:44.685 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-30T12:33:44.811 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up tcl (8.6.11+1build2) ... 2023-10-30T12:33:44.937 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjson-perl (4.04000-1) ... 2023-10-30T12:33:45.063 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-10-30T12:33:45.153 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-10-30T12:33:45.223 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-30T12:33:45.349 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-30T12:33:45.542 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-30T12:33:45.651 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:33:45.761 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-10-30T12:33:45.962 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-client (14+238) ... 2023-10-30T12:33:45.992 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:33:46.088 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-10-30T12:33:46.118 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-contrib (14+238) ... 2023-10-30T12:33:46.244 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql (14+238) ... 2023-10-30T12:33:46.370 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:33:46.496 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:33:46.700 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:33:46.700 INFO:teuthology.orchestra.run.smithi012.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-30T12:33:46.700 INFO:teuthology.orchestra.run.smithi012.stdout:which means they are currently configured by ifupdown: 2023-10-30T12:33:46.701 INFO:teuthology.orchestra.run.smithi012.stdout:- ens1f0 2023-10-30T12:33:46.701 INFO:teuthology.orchestra.run.smithi012.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-30T12:33:46.701 INFO:teuthology.orchestra.run.smithi012.stdout:remove their configuration from /etc/network/interfaces. 2023-10-30T12:33:46.701 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:33:46.776 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-30T12:33:46.776 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-30T12:33:46.840 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-30T12:33:47.251 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-30T12:33:47.399 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-30T12:33:47.674 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-30T12:33:47.701 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-30T12:33:47.836 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-30T12:33:48.390 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:33:48.509 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-30T12:33:48.610 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-10-30T12:33:48.727 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-common (238) ... 2023-10-30T12:33:49.175 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-10-30T12:33:49.283 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-10-30T12:33:49.700 INFO:teuthology.orchestra.run.smithi012.stdout:Adding user postgres to group ssl-cert 2023-10-30T12:33:49.928 INFO:teuthology.orchestra.run.smithi154.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-30T12:33:49.931 DEBUG:teuthology.parallel:result is None 2023-10-30T12:33:49.956 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:33:49.956 INFO:teuthology.orchestra.run.smithi012.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-30T12:33:50.230 INFO:teuthology.orchestra.run.smithi012.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-30T12:33:50.231 INFO:teuthology.orchestra.run.smithi012.stdout:Removing obsolete dictionary files: 2023-10-30T12:33:50.954 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-30T12:33:51.505 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:33:51.858 INFO:teuthology.orchestra.run.smithi012.stdout:Creating new PostgreSQL cluster 14/main ... 2023-10-30T12:33:51.881 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-10-30T12:33:51.921 INFO:teuthology.orchestra.run.smithi012.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-30T12:33:51.922 INFO:teuthology.orchestra.run.smithi012.stdout:This user must also own the server process. 2023-10-30T12:33:51.922 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:33:51.922 INFO:teuthology.orchestra.run.smithi012.stdout:The database cluster will be initialized with locale "en_US". 2023-10-30T12:33:51.922 INFO:teuthology.orchestra.run.smithi012.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-30T12:33:51.922 INFO:teuthology.orchestra.run.smithi012.stdout:The default text search configuration will be set to "english". 2023-10-30T12:33:51.923 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:33:51.923 INFO:teuthology.orchestra.run.smithi012.stdout:Data page checksums are disabled. 2023-10-30T12:33:51.923 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:33:51.923 INFO:teuthology.orchestra.run.smithi012.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-10-30T12:33:51.924 INFO:teuthology.orchestra.run.smithi012.stdout:creating subdirectories ... ok 2023-10-30T12:33:51.924 INFO:teuthology.orchestra.run.smithi012.stdout:selecting dynamic shared memory implementation ... posix 2023-10-30T12:33:51.948 INFO:teuthology.orchestra.run.smithi012.stdout:selecting default max_connections ... 100 2023-10-30T12:33:51.985 INFO:teuthology.orchestra.run.smithi012.stdout:selecting default shared_buffers ... 128MB 2023-10-30T12:33:52.000 INFO:teuthology.orchestra.run.smithi012.stdout:selecting default time zone ... Etc/UTC 2023-10-30T12:33:52.000 INFO:teuthology.orchestra.run.smithi012.stdout:creating configuration files ... ok 2023-10-30T12:33:52.177 INFO:teuthology.orchestra.run.smithi012.stdout:running bootstrap script ... ok 2023-10-30T12:33:52.814 INFO:teuthology.orchestra.run.smithi012.stdout:performing post-bootstrap initialization ... ok 2023-10-30T12:33:53.399 INFO:teuthology.orchestra.run.smithi012.stdout:syncing data to disk ... ok 2023-10-30T12:33:53.934 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-30T12:33:56.964 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:33:57.089 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-contrib (14+238) ... 2023-10-30T12:33:57.215 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql (14+238) ... 2023-10-30T12:33:57.341 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:33:57.458 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:33:57.702 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-30T12:33:57.702 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-30T12:33:58.494 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-30T12:33:58.788 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-30T12:33:58.940 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-30T12:34:00.725 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-10-30T12:34:00.826 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-10-30T12:34:01.540 INFO:teuthology.orchestra.run.smithi012.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-30T12:34:01.544 DEBUG:teuthology.parallel:result is None 2023-10-30T12:34:01.545 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=00c1c13cea21a686b046584596fbeac5315a7d71 2023-10-30T12:34:01.772 DEBUG:teuthology.orchestra.run.smithi012:> dpkg-query -W -f '${Version}' ceph 2023-10-30T12:34:01.795 INFO:teuthology.orchestra.run.smithi012.stdout:18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:34:01.796 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:34:01.796 INFO:teuthology.task.install:The correct ceph version 18.0.0-6835-g00c1c13c-1jammy is installed. 2023-10-30T12:34:01.797 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=00c1c13cea21a686b046584596fbeac5315a7d71 2023-10-30T12:34:02.026 DEBUG:teuthology.orchestra.run.smithi154:> dpkg-query -W -f '${Version}' ceph 2023-10-30T12:34:02.046 INFO:teuthology.orchestra.run.smithi154.stdout:18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:34:02.046 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:34:02.047 INFO:teuthology.task.install:The correct ceph version 18.0.0-6835-g00c1c13c-1jammy is installed. 2023-10-30T12:34:02.047 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-30T12:34:02.048 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:02.048 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-30T12:34:02.070 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:02.071 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-30T12:34:02.102 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-30T12:34:02.103 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:02.103 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2023-10-30T12:34:02.131 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-30T12:34:02.199 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:02.199 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2023-10-30T12:34:02.213 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-30T12:34:02.267 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-30T12:34:02.268 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:02.268 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-30T12:34:02.290 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-30T12:34:02.351 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:02.351 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-30T12:34:02.364 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-30T12:34:02.420 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-30T12:34:02.420 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:02.421 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2023-10-30T12:34:02.441 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-30T12:34:02.507 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:02.508 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2023-10-30T12:34:02.521 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-30T12:34:02.576 INFO:teuthology.run_tasks:Running task ceph... 2023-10-30T12:34:02.642 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-30T12:34:02.642 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /var/log/ceph 2023-10-30T12:34:02.647 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /var/log/ceph 2023-10-30T12:34:02.663 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-30T12:34:02.664 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-30T12:34:02.710 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-30T12:34:02.727 INFO:tasks.ceph:Creating extra log directories... 2023-10-30T12:34:02.727 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-30T12:34:02.774 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-30T12:34:02.797 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-30T12:34:02.798 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-30T12:34:02.798 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-30T12:34:02.798 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:02.799 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-30T12:34:02.847 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-30T12:34:02.944 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:02.944 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-30T12:34:02.950 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-30T12:34:03.028 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-30T12:34:03.029 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-30T12:34:03.029 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026', 'branch': 'wip-lusov-balance-after-stop', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}}', 'email': None, 'first_in_suite': False, 'job_id': '7441026', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-lusov-balance-after-stop'}, 'ceph': {'cephfs': {'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds'], 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-balance-after-stop', 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 150, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 8397, 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71', 'sleep_before_teardown': 0, 'subset': '464/1024', 'suite': 'fs', 'suite_branch': 'wip-lusov-balance-after-stop', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '00c1c13cea21a686b046584596fbeac5315a7d71', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi012.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOIXUlZNg24Ie/pDwa4MO4k4bufpZa63YdkTdS0B3ACISQbEG5xzJxPppwdXu1r1e/5mohao6G9qZoXXF4DvJ6k=', 'smithi154.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFC+TrPsAKqCG2WeGXqVschBg9bQSR3yot22TUIQ+OSTc5On7+Srgg955m9m+XBsGGUGsXdNNBTJIM5PraPhF/4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds'], 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71', 'cluster': 'ceph'}}, {'kclient': None}, {'mds_thrash': None}, {'workunit': {'clients': {'all': ['fs/snaps']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '6899cd26fceddb2fec83dc1a1349394b28c8998e', 'timestamp': '2023-10-30_06:34:16', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.770726'} 2023-10-30T12:34:03.030 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-30T12:34:03.034 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-30T12:34:03.068 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/run/ceph 2023-10-30T12:34:03.094 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/run/ceph 2023-10-30T12:34:03.125 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:03.125 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2023-10-30T12:34:03.163 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-30T12:34:03.164 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2023-10-30T12:34:03.215 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-30T12:34:03.216 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:34:03.216 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 889 Links: 1 2023-10-30T12:34:03.216 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:34:03.216 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-10-30 12:27:35.474872376 +0000 2023-10-30T12:34:03.216 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-10-30 12:27:35.470872412 +0000 2023-10-30T12:34:03.217 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-10-30 12:27:35.470872412 +0000 2023-10-30T12:34:03.217 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2023-10-30 12:27:35.470872412 +0000 2023-10-30T12:34:03.217 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-30T12:34:03.277 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-10-30T12:34:03.278 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-10-30T12:34:03.278 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000367957 s, 1.4 MB/s 2023-10-30T12:34:03.279 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-30T12:34:03.333 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2023-10-30T12:34:03.387 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-30T12:34:03.388 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:34:03.388 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 903 Links: 1 2023-10-30T12:34:03.388 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:34:03.388 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-10-30 12:27:36.494863265 +0000 2023-10-30T12:34:03.388 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-10-30 12:27:36.130866516 +0000 2023-10-30T12:34:03.389 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-10-30 12:27:36.130866516 +0000 2023-10-30T12:34:03.389 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2023-10-30 12:27:36.130866516 +0000 2023-10-30T12:34:03.390 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-30T12:34:03.450 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-10-30T12:34:03.450 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-10-30T12:34:03.450 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000357747 s, 1.4 MB/s 2023-10-30T12:34:03.452 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-30T12:34:03.509 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2023-10-30T12:34:03.560 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-30T12:34:03.560 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:34:03.560 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 920 Links: 1 2023-10-30T12:34:03.561 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:34:03.561 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-10-30 12:27:37.170857227 +0000 2023-10-30T12:34:03.561 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-10-30 12:27:36.790860621 +0000 2023-10-30T12:34:03.561 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-10-30 12:27:36.790860621 +0000 2023-10-30T12:34:03.561 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2023-10-30 12:27:36.790860621 +0000 2023-10-30T12:34:03.562 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-30T12:34:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-10-30T12:34:03.623 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-10-30T12:34:03.623 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000365362 s, 1.4 MB/s 2023-10-30T12:34:03.625 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-30T12:34:03.677 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2023-10-30T12:34:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-30T12:34:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:34:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 939 Links: 1 2023-10-30T12:34:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:34:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-10-30 12:27:37.790851689 +0000 2023-10-30T12:34:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-10-30 12:27:37.414855047 +0000 2023-10-30T12:34:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-10-30 12:27:37.414855047 +0000 2023-10-30T12:34:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2023-10-30 12:27:37.414855047 +0000 2023-10-30T12:34:03.729 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-30T12:34:03.790 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-10-30T12:34:03.791 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-10-30T12:34:03.791 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000371524 s, 1.4 MB/s 2023-10-30T12:34:03.793 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-30T12:34:03.845 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-30T12:34:03.846 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:03.846 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2023-10-30T12:34:03.852 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-30T12:34:03.853 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2023-10-30T12:34:03.900 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-30T12:34:03.901 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:34:03.901 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 892 Links: 1 2023-10-30T12:34:03.901 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:34:03.901 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-10-30 12:27:02.680241887 +0000 2023-10-30T12:34:03.902 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-10-30 12:27:02.320246684 +0000 2023-10-30T12:34:03.902 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-10-30 12:27:02.320246684 +0000 2023-10-30T12:34:03.902 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2023-10-30T12:34:03.902 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-30T12:34:03.955 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-10-30T12:34:03.955 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-10-30T12:34:03.956 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000529611 s, 967 kB/s 2023-10-30T12:34:03.957 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-30T12:34:04.006 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2023-10-30T12:34:04.052 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-30T12:34:04.053 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:34:04.053 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 911 Links: 1 2023-10-30T12:34:04.053 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:34:04.053 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-10-30 12:27:03.280233895 +0000 2023-10-30T12:34:04.053 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-10-30 12:27:02.956238208 +0000 2023-10-30T12:34:04.054 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-10-30 12:27:02.956238208 +0000 2023-10-30T12:34:04.054 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2023-10-30T12:34:04.054 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-30T12:34:04.106 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-10-30T12:34:04.107 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-10-30T12:34:04.107 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000546212 s, 937 kB/s 2023-10-30T12:34:04.108 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-30T12:34:04.158 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2023-10-30T12:34:04.204 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-30T12:34:04.205 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:34:04.205 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 926 Links: 1 2023-10-30T12:34:04.205 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:34:04.205 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-10-30 12:27:03.864226122 +0000 2023-10-30T12:34:04.205 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-10-30 12:27:03.520230701 +0000 2023-10-30T12:34:04.206 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-10-30 12:27:03.520230701 +0000 2023-10-30T12:34:04.206 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2023-10-30T12:34:04.206 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-30T12:34:04.258 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-10-30T12:34:04.259 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-10-30T12:34:04.259 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.00047522 s, 1.1 MB/s 2023-10-30T12:34:04.260 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-30T12:34:04.309 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2023-10-30T12:34:04.357 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-30T12:34:04.357 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:34:04.357 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 945 Links: 1 2023-10-30T12:34:04.357 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:34:04.358 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-10-30 12:27:04.444218409 +0000 2023-10-30T12:34:04.358 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-10-30 12:27:04.096223035 +0000 2023-10-30T12:34:04.358 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-10-30 12:27:04.096223035 +0000 2023-10-30T12:34:04.358 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2023-10-30T12:34:04.359 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-30T12:34:04.411 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-10-30T12:34:04.411 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-10-30T12:34:04.411 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000506536 s, 1.0 MB/s 2023-10-30T12:34:04.412 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-30T12:34:04.461 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-30T12:34:04.462 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-30T12:34:04.462 INFO:tasks.ceph:Generating config... 2023-10-30T12:34:04.464 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-30T12:34:04.464 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2023-10-30T12:34:04.464 INFO:tasks.ceph:[client] debug client = 20 2023-10-30T12:34:04.464 INFO:tasks.ceph:[client] debug ms = 1 2023-10-30T12:34:04.465 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-30T12:34:04.465 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-30T12:34:04.465 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-30T12:34:04.466 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-30T12:34:04.466 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-10-30T12:34:04.466 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-30T12:34:04.466 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2023-10-30T12:34:04.467 INFO:tasks.ceph:[mds] mds bal merge size = 5 2023-10-30T12:34:04.467 INFO:tasks.ceph:[mds] mds bal split bits = 3 2023-10-30T12:34:04.467 INFO:tasks.ceph:[mds] mds bal split size = 100 2023-10-30T12:34:04.468 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-30T12:34:04.468 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-30T12:34:04.468 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2023-10-30T12:34:04.468 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-30T12:34:04.469 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-30T12:34:04.469 INFO:tasks.ceph:[mds] mds_dir_prefetch = False 2023-10-30T12:34:04.469 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2023-10-30T12:34:04.470 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2023-10-30T12:34:04.470 INFO:tasks.ceph:[mds] ms inject delay max = 1 2023-10-30T12:34:04.470 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2023-10-30T12:34:04.470 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2023-10-30T12:34:04.471 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2023-10-30T12:34:04.471 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-30T12:34:04.471 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-30T12:34:04.472 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-30T12:34:04.472 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-30T12:34:04.472 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-30T12:34:04.472 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-30T12:34:04.473 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-30T12:34:04.473 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-30T12:34:04.473 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-30T12:34:04.473 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-30T12:34:04.474 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-30T12:34:04.474 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-30T12:34:04.474 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-30T12:34:04.475 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-30T12:34:04.475 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-30T12:34:04.475 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-30T12:34:04.475 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-30T12:34:04.476 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-30T12:34:04.476 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-30T12:34:04.476 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2023-10-30T12:34:04.476 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-30T12:34:04.477 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-30T12:34:04.477 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-30T12:34:04.477 INFO:tasks.ceph:[osd] ms inject delay max = 1 2023-10-30T12:34:04.477 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2023-10-30T12:34:04.478 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2023-10-30T12:34:04.478 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2023-10-30T12:34:04.478 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-10-30T12:34:04.478 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-30T12:34:04.479 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-10-30T12:34:04.479 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-30T12:34:04.479 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-30T12:34:04.480 INFO:tasks.ceph:Setting up mon.a... 2023-10-30T12:34:04.480 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-30T12:34:04.529 INFO:teuthology.orchestra.run.smithi012.stdout:creating /etc/ceph/ceph.keyring 2023-10-30T12:34:04.543 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-30T12:34:04.593 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-30T12:34:04.615 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.12'), ('mon.b', '172.21.15.154'), ('mon.c', '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]')] 2023-10-30T12:34:04.616 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.12,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-30T12:34:04.617 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:04.617 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-30T12:34:04.667 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.12 --add b 172.21.15.154 --addv c '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:34:04.759 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:34:04.760 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: generated fsid 97b69bd8-a5ea-4668-9ffd-462651b43fe5 2023-10-30T12:34:04.760 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = pacific 2023-10-30T12:34:04.760 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 0 2023-10-30T12:34:04.760 INFO:teuthology.orchestra.run.smithi012.stdout:fsid 97b69bd8-a5ea-4668-9ffd-462651b43fe5 2023-10-30T12:34:04.761 INFO:teuthology.orchestra.run.smithi012.stdout:last_changed 2023-10-30T12:34:04.760610+0000 2023-10-30T12:34:04.761 INFO:teuthology.orchestra.run.smithi012.stdout:created 2023-10-30T12:34:04.760610+0000 2023-10-30T12:34:04.761 INFO:teuthology.orchestra.run.smithi012.stdout:min_mon_release 16 (pacific) 2023-10-30T12:34:04.762 INFO:teuthology.orchestra.run.smithi012.stdout:election_strategy: 1 2023-10-30T12:34:04.762 INFO:teuthology.orchestra.run.smithi012.stdout:0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2023-10-30T12:34:04.762 INFO:teuthology.orchestra.run.smithi012.stdout:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2023-10-30T12:34:04.763 INFO:teuthology.orchestra.run.smithi012.stdout:2: [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] mon.c 2023-10-30T12:34:04.763 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-30T12:34:04.765 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-30T12:34:04.774 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 97b69bd8-a5ea-4668-9ffd-462651b43fe5... 2023-10-30T12:34:04.776 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-30T12:34:04.826 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-30T12:34:04.855 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2023-10-30T12:34:04.856 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2023-10-30T12:34:04.856 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-30T12:34:04.857 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2023-10-30T12:34:04.857 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.858 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2023-10-30T12:34:04.858 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.859 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2023-10-30T12:34:04.859 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.859 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2023-10-30T12:34:04.859 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2023-10-30T12:34:04.860 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.860 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2023-10-30T12:34:04.860 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2023-10-30T12:34:04.860 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.860 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-30T12:34:04.861 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2023-10-30T12:34:04.861 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.861 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2023-10-30T12:34:04.861 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-30T12:34:04.861 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.862 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2023-10-30T12:34:04.862 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.862 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2023-10-30T12:34:04.862 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2023-10-30T12:34:04.863 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2023-10-30T12:34:04.863 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2023-10-30T12:34:04.863 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2023-10-30T12:34:04.863 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2023-10-30T12:34:04.863 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2023-10-30T12:34:04.864 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-30T12:34:04.864 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-30T12:34:04.864 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2023-10-30T12:34:04.865 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.865 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-30T12:34:04.865 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.865 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2023-10-30T12:34:04.866 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.866 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2023-10-30T12:34:04.866 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.867 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2023-10-30T12:34:04.867 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2023-10-30T12:34:04.868 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2023-10-30T12:34:04.868 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.12,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]" 2023-10-30T12:34:04.868 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on skipped message = False 2023-10-30T12:34:04.869 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 97b69bd8-a5ea-4668-9ffd-462651b43fe5 2023-10-30T12:34:04.871 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.871 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2023-10-30T12:34:04.871 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2023-10-30T12:34:04.872 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.872 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2023-10-30T12:34:04.872 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2023-10-30T12:34:04.873 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2023-10-30T12:34:04.873 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.874 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2023-10-30T12:34:04.874 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2023-10-30T12:34:04.874 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.875 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2023-10-30T12:34:04.875 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2023-10-30T12:34:04.875 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2023-10-30T12:34:04.876 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.877 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2023-10-30T12:34:04.877 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2023-10-30T12:34:04.877 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.877 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2023-10-30T12:34:04.877 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.878 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2023-10-30T12:34:04.878 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.878 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2023-10-30T12:34:04.878 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.878 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2023-10-30T12:34:04.879 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2023-10-30T12:34:04.879 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.879 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2023-10-30T12:34:04.879 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2023-10-30T12:34:04.880 INFO:teuthology.orchestra.run.smithi154.stdout: bdev async discard = True 2023-10-30T12:34:04.880 INFO:teuthology.orchestra.run.smithi154.stdout: bdev enable discard = True 2023-10-30T12:34:04.880 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore allocator = bitmap 2023-10-30T12:34:04.881 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2023-10-30T12:34:04.881 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2023-10-30T12:34:04.881 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 1/20 2023-10-30T12:34:04.882 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 1/20 2023-10-30T12:34:04.882 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-10-30T12:34:04.883 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2023-10-30T12:34:04.883 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 4/10 2023-10-30T12:34:04.883 INFO:teuthology.orchestra.run.smithi154.stdout: mon client directed command retry = 5 2023-10-30T12:34:04.884 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2023-10-30T12:34:04.884 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2023-10-30T12:34:04.884 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2023-10-30T12:34:04.885 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject delay max = 1 2023-10-30T12:34:04.885 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject delay probability = 0.005 2023-10-30T12:34:04.885 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject delay type = client mds 2023-10-30T12:34:04.886 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject socket failures = 2500 2023-10-30T12:34:04.886 INFO:teuthology.orchestra.run.smithi154.stdout: osd blocked scrub grace period = 3600 2023-10-30T12:34:04.886 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2023-10-30T12:34:04.887 INFO:teuthology.orchestra.run.smithi154.stdout: osd max markdown count = 1000 2023-10-30T12:34:04.887 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2023-10-30T12:34:04.888 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-10-30T12:34:04.888 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.888 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2023-10-30T12:34:04.889 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-10-30T12:34:04.889 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2023-10-30T12:34:04.889 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-10-30T12:34:04.889 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-10-30T12:34:04.890 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-10-30T12:34:04.890 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-10-30T12:34:04.890 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2023-10-30T12:34:04.890 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.891 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2023-10-30T12:34:04.891 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-10-30T12:34:04.891 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-10-30T12:34:04.891 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2023-10-30T12:34:04.891 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-10-30T12:34:04.892 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2023-10-30T12:34:04.892 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2023-10-30T12:34:04.892 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-10-30T12:34:04.892 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2023-10-30T12:34:04.892 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2023-10-30T12:34:04.893 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-10-30T12:34:04.893 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.893 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-30T12:34:04.893 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2023-10-30T12:34:04.893 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2023-10-30T12:34:04.893 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.894 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2023-10-30T12:34:04.894 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-30T12:34:04.894 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-30T12:34:04.894 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.894 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2023-10-30T12:34:04.895 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2023-10-30T12:34:04.895 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.895 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2023-10-30T12:34:04.895 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2023-10-30T12:34:04.895 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-30T12:34:04.896 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2023-10-30T12:34:04.896 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2023-10-30T12:34:04.896 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2023-10-30T12:34:04.896 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2023-10-30T12:34:04.896 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-30T12:34:04.897 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-30T12:34:04.898 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2023-10-30T12:34:04.899 INFO:teuthology.orchestra.run.smithi154.stdout: client_shutdown_timeout = 120 2023-10-30T12:34:04.899 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2023-10-30T12:34:04.899 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-10-30T12:34:04.899 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-10-30T12:34:04.899 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-10-30T12:34:04.900 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2023-10-30T12:34:04.900 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2023-10-30T12:34:04.900 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2023-10-30T12:34:04.900 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-10-30T12:34:04.900 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-10-30T12:34:04.901 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.e] 2023-10-30T12:34:04.901 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-10-30T12:34:04.901 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-10-30T12:34:04.901 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.f] 2023-10-30T12:34:04.901 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2023-10-30T12:34:04.902 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2023-10-30T12:34:04.902 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2023-10-30T12:34:04.902 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-10-30T12:34:04.902 INFO:teuthology.orchestra.run.smithi154.stdout: mds bal fragment size max = 10000 2023-10-30T12:34:04.902 INFO:teuthology.orchestra.run.smithi154.stdout: mds bal merge size = 5 2023-10-30T12:34:04.903 INFO:teuthology.orchestra.run.smithi154.stdout: mds bal split bits = 3 2023-10-30T12:34:04.903 INFO:teuthology.orchestra.run.smithi154.stdout: mds bal split size = 100 2023-10-30T12:34:04.903 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2023-10-30T12:34:04.903 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = True 2023-10-30T12:34:04.903 INFO:teuthology.orchestra.run.smithi154.stdout: mds oft prefetch dirfrags = True 2023-10-30T12:34:04.903 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2023-10-30T12:34:04.904 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = True 2023-10-30T12:34:04.904 INFO:teuthology.orchestra.run.smithi154.stdout: mds_dir_prefetch = False 2023-10-30T12:34:04.904 INFO:teuthology.orchestra.run.smithi154.stdout: mds_sleep_rank_change = 5000000.0 2023-10-30T12:34:04.904 INFO:teuthology.orchestra.run.smithi154.stdout: mon client directed command retry = 5 2023-10-30T12:34:04.904 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject delay max = 1 2023-10-30T12:34:04.905 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject delay probability = 0.005 2023-10-30T12:34:04.905 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject delay type = client mds osd 2023-10-30T12:34:04.906 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject socket failures = 2500 2023-10-30T12:34:04.906 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-10-30T12:34:04.906 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-10-30T12:34:04.907 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-10-30T12:34:04.910 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2023-10-30T12:34:04.910 INFO:teuthology.orchestra.run.smithi012.stdout: chdir = "" 2023-10-30T12:34:04.911 INFO:teuthology.orchestra.run.smithi012.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-30T12:34:04.911 INFO:teuthology.orchestra.run.smithi012.stdout: auth supported = cephx 2023-10-30T12:34:04.912 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.912 INFO:teuthology.orchestra.run.smithi012.stdout: filestore xattr use omap = true 2023-10-30T12:34:04.912 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.913 INFO:teuthology.orchestra.run.smithi012.stdout: mon clock drift allowed = 1.000 2023-10-30T12:34:04.913 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.913 INFO:teuthology.orchestra.run.smithi012.stdout: osd crush chooseleaf type = 0 2023-10-30T12:34:04.913 INFO:teuthology.orchestra.run.smithi012.stdout: auth debug = true 2023-10-30T12:34:04.913 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.914 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on old message = true 2023-10-30T12:34:04.914 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on bug = true 2023-10-30T12:34:04.914 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.914 INFO:teuthology.orchestra.run.smithi012.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-30T12:34:04.914 INFO:teuthology.orchestra.run.smithi012.stdout: mon pg warn max object skew = 0 2023-10-30T12:34:04.915 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.915 INFO:teuthology.orchestra.run.smithi012.stdout: # disable pg_autoscaler by default for new pools 2023-10-30T12:34:04.915 INFO:teuthology.orchestra.run.smithi012.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-30T12:34:04.915 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.915 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default size = 2 2023-10-30T12:34:04.916 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.916 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow primary affinity = true 2023-10-30T12:34:04.916 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow pg remap = true 2023-10-30T12:34:04.916 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on legacy crush tunables = false 2023-10-30T12:34:04.916 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on crush straw calc version zero = false 2023-10-30T12:34:04.917 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on no sortbitwise = false 2023-10-30T12:34:04.917 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on osd down out interval zero = false 2023-10-30T12:34:04.917 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on too few osds = false 2023-10-30T12:34:04.917 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-30T12:34:04.917 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-30T12:34:04.917 INFO:teuthology.orchestra.run.smithi012.stdout: mon_allow_pool_size_one = true 2023-10-30T12:34:04.918 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.918 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-30T12:34:04.918 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.918 INFO:teuthology.orchestra.run.smithi012.stdout: osd default data pool replay window = 5 2023-10-30T12:34:04.918 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.919 INFO:teuthology.orchestra.run.smithi012.stdout: mon allow pool delete = true 2023-10-30T12:34:04.919 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.919 INFO:teuthology.orchestra.run.smithi012.stdout: mon cluster log file level = debug 2023-10-30T12:34:04.919 INFO:teuthology.orchestra.run.smithi012.stdout: debug asserts on shutdown = true 2023-10-30T12:34:04.920 INFO:teuthology.orchestra.run.smithi012.stdout: mon health detail to clog = false 2023-10-30T12:34:04.920 INFO:teuthology.orchestra.run.smithi012.stdout: mon host = "172.21.15.12,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]" 2023-10-30T12:34:04.920 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on skipped message = False 2023-10-30T12:34:04.920 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = 97b69bd8-a5ea-4668-9ffd-462651b43fe5 2023-10-30T12:34:04.920 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.921 INFO:teuthology.orchestra.run.smithi012.stdout:[osd] 2023-10-30T12:34:04.921 INFO:teuthology.orchestra.run.smithi012.stdout: osd journal size = 100 2023-10-30T12:34:04.921 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.921 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub load threshold = 5.0 2023-10-30T12:34:04.921 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub max interval = 600 2023-10-30T12:34:04.922 INFO:teuthology.orchestra.run.smithi012.stdout: osd mclock profile = high_recovery_ops 2023-10-30T12:34:04.922 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.922 INFO:teuthology.orchestra.run.smithi012.stdout: osd recover clone overlap = true 2023-10-30T12:34:04.922 INFO:teuthology.orchestra.run.smithi012.stdout: osd recovery max chunk = 1048576 2023-10-30T12:34:04.922 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.923 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug shutdown = true 2023-10-30T12:34:04.923 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug op order = true 2023-10-30T12:34:04.923 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug verify stray on activate = true 2023-10-30T12:34:04.923 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.923 INFO:teuthology.orchestra.run.smithi012.stdout: osd open classes on start = true 2023-10-30T12:34:04.924 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug pg log writeout = true 2023-10-30T12:34:04.924 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.924 INFO:teuthology.orchestra.run.smithi012.stdout: osd deep scrub update digest min age = 30 2023-10-30T12:34:04.924 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.924 INFO:teuthology.orchestra.run.smithi012.stdout: osd map max advance = 10 2023-10-30T12:34:04.924 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.925 INFO:teuthology.orchestra.run.smithi012.stdout: journal zero on create = true 2023-10-30T12:34:04.925 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.925 INFO:teuthology.orchestra.run.smithi012.stdout: filestore ondisk finisher threads = 3 2023-10-30T12:34:04.925 INFO:teuthology.orchestra.run.smithi012.stdout: filestore apply finisher threads = 3 2023-10-30T12:34:04.925 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.926 INFO:teuthology.orchestra.run.smithi012.stdout: bdev debug aio = true 2023-10-30T12:34:04.926 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug misdirected ops = true 2023-10-30T12:34:04.926 INFO:teuthology.orchestra.run.smithi012.stdout: bdev async discard = True 2023-10-30T12:34:04.926 INFO:teuthology.orchestra.run.smithi012.stdout: bdev enable discard = True 2023-10-30T12:34:04.926 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore allocator = bitmap 2023-10-30T12:34:04.927 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore block size = 96636764160 2023-10-30T12:34:04.927 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore fsck on mount = True 2023-10-30T12:34:04.927 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluefs = 1/20 2023-10-30T12:34:04.927 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluestore = 1/20 2023-10-30T12:34:04.927 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-10-30T12:34:04.928 INFO:teuthology.orchestra.run.smithi012.stdout: debug osd = 20 2023-10-30T12:34:04.928 INFO:teuthology.orchestra.run.smithi012.stdout: debug rocksdb = 4/10 2023-10-30T12:34:04.928 INFO:teuthology.orchestra.run.smithi012.stdout: mon client directed command retry = 5 2023-10-30T12:34:04.928 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd backfillfull_ratio = 0.85 2023-10-30T12:34:04.928 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd full ratio = 0.9 2023-10-30T12:34:04.929 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd nearfull ratio = 0.8 2023-10-30T12:34:04.929 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject delay max = 1 2023-10-30T12:34:04.929 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject delay probability = 0.005 2023-10-30T12:34:04.929 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject delay type = client mds 2023-10-30T12:34:04.929 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject socket failures = 2500 2023-10-30T12:34:04.929 INFO:teuthology.orchestra.run.smithi012.stdout: osd blocked scrub grace period = 3600 2023-10-30T12:34:04.930 INFO:teuthology.orchestra.run.smithi012.stdout: osd failsafe full ratio = 0.95 2023-10-30T12:34:04.930 INFO:teuthology.orchestra.run.smithi012.stdout: osd max markdown count = 1000 2023-10-30T12:34:04.930 INFO:teuthology.orchestra.run.smithi012.stdout: osd objectstore = bluestore 2023-10-30T12:34:04.930 INFO:teuthology.orchestra.run.smithi012.stdout: osd op complaint time = 180 2023-10-30T12:34:04.930 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.931 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr] 2023-10-30T12:34:04.931 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-10-30T12:34:04.931 INFO:teuthology.orchestra.run.smithi012.stdout: debug mgr = 20 2023-10-30T12:34:04.931 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2023-10-30T12:34:04.931 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2023-10-30T12:34:04.932 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2023-10-30T12:34:04.932 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2023-10-30T12:34:04.932 INFO:teuthology.orchestra.run.smithi012.stdout: mgr/telemetry/nag = false 2023-10-30T12:34:04.932 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.932 INFO:teuthology.orchestra.run.smithi012.stdout:[mon] 2023-10-30T12:34:04.933 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-10-30T12:34:04.933 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2023-10-30T12:34:04.933 INFO:teuthology.orchestra.run.smithi012.stdout: debug paxos = 20 2023-10-30T12:34:04.933 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2023-10-30T12:34:04.933 INFO:teuthology.orchestra.run.smithi012.stdout: mon data avail warn = 5 2023-10-30T12:34:04.934 INFO:teuthology.orchestra.run.smithi012.stdout: mon mgr mkfs grace = 240 2023-10-30T12:34:04.934 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2023-10-30T12:34:04.934 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd reporter subtree level = osd 2023-10-30T12:34:04.934 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd prime pg temp = true 2023-10-30T12:34:04.934 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2023-10-30T12:34:04.935 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.935 INFO:teuthology.orchestra.run.smithi012.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-30T12:34:04.935 INFO:teuthology.orchestra.run.smithi012.stdout: auth mon ticket ttl = 660 # 11m 2023-10-30T12:34:04.935 INFO:teuthology.orchestra.run.smithi012.stdout: auth service ticket ttl = 240 # 4m 2023-10-30T12:34:04.935 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.936 INFO:teuthology.orchestra.run.smithi012.stdout: # don't complain about insecure global_id in the test suite 2023-10-30T12:34:04.936 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-30T12:34:04.936 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-30T12:34:04.936 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.936 INFO:teuthology.orchestra.run.smithi012.stdout: # 1m isn't quite enough 2023-10-30T12:34:04.937 INFO:teuthology.orchestra.run.smithi012.stdout: mon_down_mkfs_grace = 2m 2023-10-30T12:34:04.937 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.937 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_filestore_osds = false 2023-10-30T12:34:04.937 INFO:teuthology.orchestra.run.smithi012.stdout: mon op complaint time = 120 2023-10-30T12:34:04.937 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:04.938 INFO:teuthology.orchestra.run.smithi012.stdout:[client] 2023-10-30T12:34:04.938 INFO:teuthology.orchestra.run.smithi012.stdout: rgw cache enabled = true 2023-10-30T12:34:04.938 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable ops log = true 2023-10-30T12:34:04.938 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable usage log = true 2023-10-30T12:34:04.938 INFO:teuthology.orchestra.run.smithi012.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-30T12:34:04.939 INFO:teuthology.orchestra.run.smithi012.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-30T12:34:04.939 INFO:teuthology.orchestra.run.smithi012.stdout: client mount timeout = 600 2023-10-30T12:34:04.939 INFO:teuthology.orchestra.run.smithi012.stdout: client_shutdown_timeout = 120 2023-10-30T12:34:04.939 INFO:teuthology.orchestra.run.smithi012.stdout: debug client = 20 2023-10-30T12:34:04.939 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-10-30T12:34:04.939 INFO:teuthology.orchestra.run.smithi012.stdout: rados mon op timeout = 900 2023-10-30T12:34:04.940 INFO:teuthology.orchestra.run.smithi012.stdout: rados osd op timeout = 900 2023-10-30T12:34:04.940 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.a] 2023-10-30T12:34:04.940 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.b] 2023-10-30T12:34:04.940 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.c] 2023-10-30T12:34:04.940 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.a] 2023-10-30T12:34:04.941 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.c] 2023-10-30T12:34:04.941 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.e] 2023-10-30T12:34:04.941 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.b] 2023-10-30T12:34:04.941 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.d] 2023-10-30T12:34:04.941 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.f] 2023-10-30T12:34:04.941 INFO:teuthology.orchestra.run.smithi012.stdout:[mds] 2023-10-30T12:34:04.942 INFO:teuthology.orchestra.run.smithi012.stdout: debug mds = 20 2023-10-30T12:34:04.942 INFO:teuthology.orchestra.run.smithi012.stdout: debug mds balancer = 20 2023-10-30T12:34:04.942 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-10-30T12:34:04.942 INFO:teuthology.orchestra.run.smithi012.stdout: mds bal fragment size max = 10000 2023-10-30T12:34:04.942 INFO:teuthology.orchestra.run.smithi012.stdout: mds bal merge size = 5 2023-10-30T12:34:04.943 INFO:teuthology.orchestra.run.smithi012.stdout: mds bal split bits = 3 2023-10-30T12:34:04.943 INFO:teuthology.orchestra.run.smithi012.stdout: mds bal split size = 100 2023-10-30T12:34:04.943 INFO:teuthology.orchestra.run.smithi012.stdout: mds debug frag = True 2023-10-30T12:34:04.943 INFO:teuthology.orchestra.run.smithi012.stdout: mds debug scatterstat = True 2023-10-30T12:34:04.943 INFO:teuthology.orchestra.run.smithi012.stdout: mds oft prefetch dirfrags = True 2023-10-30T12:34:04.944 INFO:teuthology.orchestra.run.smithi012.stdout: mds op complaint time = 180 2023-10-30T12:34:04.944 INFO:teuthology.orchestra.run.smithi012.stdout: mds verify scatter = True 2023-10-30T12:34:04.944 INFO:teuthology.orchestra.run.smithi012.stdout: mds_dir_prefetch = False 2023-10-30T12:34:04.944 INFO:teuthology.orchestra.run.smithi012.stdout: mds_sleep_rank_change = 5000000.0 2023-10-30T12:34:04.944 INFO:teuthology.orchestra.run.smithi012.stdout: mon client directed command retry = 5 2023-10-30T12:34:04.945 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject delay max = 1 2023-10-30T12:34:04.945 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject delay probability = 0.005 2023-10-30T12:34:04.945 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject delay type = client mds osd 2023-10-30T12:34:04.945 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject socket failures = 2500 2023-10-30T12:34:04.945 INFO:teuthology.orchestra.run.smithi012.stdout: osd op complaint time = 180 2023-10-30T12:34:04.945 INFO:teuthology.orchestra.run.smithi012.stdout: rados mon op timeout = 900 2023-10-30T12:34:04.946 INFO:teuthology.orchestra.run.smithi012.stdout: rados osd op timeout = 900 2023-10-30T12:34:04.946 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-30T12:34:04.946 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-30T12:34:05.014 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-30T12:34:05.015 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:05.015 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-30T12:34:05.024 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:05.025 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-30T12:34:05.079 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi012.front.sepia.ceph.com 2023-10-30T12:34:05.079 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:05.080 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-30T12:34:05.080 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-30T12:34:05.160 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:05.161 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:34:05.211 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi154.front.sepia.ceph.com 2023-10-30T12:34:05.211 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:05.212 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-30T12:34:05.212 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-30T12:34:05.235 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:05.236 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:34:05.284 INFO:tasks.ceph:Setting up mon nodes... 2023-10-30T12:34:05.285 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-30T12:34:05.285 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-30T12:34:05.342 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-30T12:34:05.347 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-30T12:34:05.389 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-30T12:34:05.403 INFO:tasks.ceph:Setting up mds nodes... 2023-10-30T12:34:05.403 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-30T12:34:05.462 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-30T12:34:05.466 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-30T12:34:05.487 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-30T12:34:05.588 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-30T12:34:05.592 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-30T12:34:05.613 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2023-10-30T12:34:05.711 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2023-10-30T12:34:05.716 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2023-10-30T12:34:05.737 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-30T12:34:05.782 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-30T12:34:05.785 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-30T12:34:05.839 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-30T12:34:05.919 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-30T12:34:05.922 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-30T12:34:05.975 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2023-10-30T12:34:06.061 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2023-10-30T12:34:06.064 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2023-10-30T12:34:06.119 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-30T12:34:06.120 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-30T12:34:06.165 INFO:teuthology.orchestra.run.smithi012.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-30T12:34:06.183 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-30T12:34:06.183 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-30T12:34:06.184 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-30T12:34:06.251 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-30T12:34:06.252 INFO:tasks.ceph:role: osd.0 2023-10-30T12:34:06.252 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com 2023-10-30T12:34:06.252 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-30T12:34:06.322 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:34:06.323 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:34:06.323 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:34:06.323 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:34:06.323 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:34:06.323 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:34:06.324 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:34:06.324 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:34:06.324 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:34:06.324 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:34:06.786 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2023-10-30T12:34:06.789 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2023-10-30T12:34:06.789 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-30T12:34:06.893 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-30T12:34:06.912 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:34:06.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:34:06.914 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-30T12:34:06.979 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-30T12:34:06.980 INFO:tasks.ceph:role: osd.1 2023-10-30T12:34:06.980 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com 2023-10-30T12:34:06.980 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-30T12:34:07.053 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:34:07.053 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:34:07.053 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:34:07.053 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:34:07.054 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:34:07.054 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:34:07.054 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:34:07.054 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:34:07.055 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:34:07.055 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:34:07.535 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2023-10-30T12:34:07.538 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2023-10-30T12:34:07.538 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-30T12:34:07.695 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-30T12:34:07.713 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:34:07.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:34:07.715 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-30T12:34:07.785 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-30T12:34:07.786 INFO:tasks.ceph:role: osd.2 2023-10-30T12:34:07.786 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com 2023-10-30T12:34:07.786 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-30T12:34:07.856 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:34:07.857 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:34:07.857 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:34:07.857 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:34:07.857 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:34:07.857 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:34:07.858 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:34:07.858 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:34:07.858 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:34:07.858 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:34:08.345 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2023-10-30T12:34:08.348 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2023-10-30T12:34:08.348 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-30T12:34:08.459 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-30T12:34:08.477 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:34:08.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:34:08.479 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-30T12:34:08.545 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-30T12:34:08.546 INFO:tasks.ceph:role: osd.3 2023-10-30T12:34:08.546 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi012.front.sepia.ceph.com 2023-10-30T12:34:08.546 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-30T12:34:08.621 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:34:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:34:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:34:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:34:08.622 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:34:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:34:08.623 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:34:08.623 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:34:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:34:08.623 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:34:09.071 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2023-10-30T12:34:09.074 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2023-10-30T12:34:09.074 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-30T12:34:09.190 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-30T12:34:09.208 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:34:09.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:34:09.210 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:34:09.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:09.347+0000 7f0c6b3c6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-30T12:34:09.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:09.347+0000 7f0c6b3c6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-30T12:34:09.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:09.351+0000 7f0c6b3c6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-30T12:34:09.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:09.351+0000 7f0c6b3c6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-30T12:34:09.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:09.351+0000 7f0c6b3c6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-30T12:34:09.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:09.355+0000 7f0c6b3c6640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-30T12:34:12.369 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-30T12:34:12.392 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:34:12.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:12.491+0000 7f223450b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-30T12:34:12.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:12.491+0000 7f223450b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-30T12:34:12.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:12.491+0000 7f223450b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-30T12:34:12.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:12.491+0000 7f223450b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-30T12:34:12.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:12.491+0000 7f223450b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-30T12:34:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:12.495+0000 7f223450b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-30T12:34:15.487 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-30T12:34:15.510 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:34:15.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:15.611+0000 7feeaf182640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-30T12:34:15.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:15.611+0000 7feeaf182640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-30T12:34:15.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:15.611+0000 7feeaf182640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-30T12:34:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:15.611+0000 7feeaf182640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-30T12:34:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:15.611+0000 7feeaf182640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-30T12:34:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:15.611+0000 7feeaf182640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-30T12:34:18.623 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-30T12:34:18.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:34:18.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:18.747+0000 7ffada0a6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-30T12:34:18.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:18.747+0000 7ffada0a6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-30T12:34:18.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:18.747+0000 7ffada0a6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-30T12:34:18.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:18.747+0000 7ffada0a6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-30T12:34:18.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:18.747+0000 7ffada0a6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-30T12:34:18.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:18.747+0000 7ffada0a6640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-30T12:34:21.758 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-30T12:34:21.781 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-30T12:34:21.797 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-30T12:34:21.797 INFO:tasks.ceph:role: osd.4 2023-10-30T12:34:21.797 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-30T12:34:21.798 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-30T12:34:21.860 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:34:21.860 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:34:21.860 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:34:21.861 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:34:21.861 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:34:21.861 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:34:21.861 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:34:21.861 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:34:21.862 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:34:21.862 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:34:22.310 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2023-10-30T12:34:22.313 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-10-30T12:34:22.313 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-30T12:34:22.414 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-30T12:34:22.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:34:22.424 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:34:22.425 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-30T12:34:22.477 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-30T12:34:22.478 INFO:tasks.ceph:role: osd.5 2023-10-30T12:34:22.478 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-30T12:34:22.478 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-30T12:34:22.544 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:34:22.544 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:34:22.544 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:34:22.544 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:34:22.545 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:34:22.545 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:34:22.545 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:34:22.545 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:34:22.545 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:34:22.546 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:34:23.008 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2023-10-30T12:34:23.010 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-10-30T12:34:23.010 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-30T12:34:23.142 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-30T12:34:23.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:34:23.154 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:34:23.154 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-30T12:34:23.207 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-30T12:34:23.208 INFO:tasks.ceph:role: osd.6 2023-10-30T12:34:23.208 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-30T12:34:23.208 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-30T12:34:23.271 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:34:23.272 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:34:23.272 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:34:23.272 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:34:23.272 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:34:23.273 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:34:23.273 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:34:23.273 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:34:23.273 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:34:23.273 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:34:23.747 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2023-10-30T12:34:23.749 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-10-30T12:34:23.749 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-30T12:34:23.899 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-30T12:34:23.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:34:23.911 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:34:23.912 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-30T12:34:23.968 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-30T12:34:23.968 INFO:tasks.ceph:role: osd.7 2023-10-30T12:34:23.969 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-30T12:34:23.969 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-30T12:34:24.031 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:34:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:34:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:34:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:34:24.032 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:34:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:34:24.032 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:34:24.032 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:34:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:34:24.033 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:34:24.462 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2023-10-30T12:34:24.464 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-10-30T12:34:24.464 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-30T12:34:24.569 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-30T12:34:24.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:34:24.580 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:34:24.581 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:34:24.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:24.692+0000 7fdb4e982640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-30T12:34:24.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:24.696+0000 7fdb4e982640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-30T12:34:24.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:24.696+0000 7fdb4e982640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-30T12:34:24.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:24.696+0000 7fdb4e982640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-30T12:34:24.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:24.696+0000 7fdb4e982640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-30T12:34:24.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:24.696+0000 7fdb4e982640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-30T12:34:27.706 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-30T12:34:27.721 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:34:27.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:27.804+0000 7f129b0f3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-30T12:34:27.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:27.804+0000 7f129b0f3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-30T12:34:27.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:27.804+0000 7f129b0f3640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-30T12:34:27.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:27.804+0000 7f129b0f3640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-30T12:34:27.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:27.804+0000 7f129b0f3640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-30T12:34:27.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:27.804+0000 7f129b0f3640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-30T12:34:30.809 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-30T12:34:30.869 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:34:30.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:30.948+0000 7f0e4fa49640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-30T12:34:30.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:30.952+0000 7f0e4fa49640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-30T12:34:30.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:30.952+0000 7f0e4fa49640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-30T12:34:30.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:30.952+0000 7f0e4fa49640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-30T12:34:30.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:30.952+0000 7f0e4fa49640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-30T12:34:30.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:30.952+0000 7f0e4fa49640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-30T12:34:33.032 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:34:33.036 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:34:33.959 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-30T12:34:33.973 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:34:34.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:34.052+0000 7fc90550d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-30T12:34:34.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:34.056+0000 7fc90550d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-30T12:34:34.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:34.056+0000 7fc90550d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-30T12:34:34.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:34.056+0000 7fc90550d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-30T12:34:34.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:34.056+0000 7fc90550d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-30T12:34:34.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:34.056+0000 7fc90550d640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-30T12:34:37.059 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-30T12:34:37.116 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-30T12:34:37.117 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:37.117 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-30T12:34:37.140 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:37.141 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-30T12:34:37.205 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:37.205 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-30T12:34:37.269 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:37.269 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2023-10-30T12:34:37.336 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:37.337 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-30T12:34:37.403 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:37.403 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-30T12:34:37.469 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:37.469 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-30T12:34:37.532 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:37.533 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-30T12:34:37.597 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:37.597 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-30T12:34:37.611 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:37.611 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-30T12:34:37.667 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:37.667 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-30T12:34:37.726 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:37.727 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2023-10-30T12:34:37.779 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:37.780 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-30T12:34:37.839 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:37.839 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-30T12:34:37.894 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:37.894 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-30T12:34:37.946 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:37.947 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-30T12:34:38.003 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:38.004 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-30T12:34:38.014 INFO:tasks.ceph:Adding keys to all mons... 2023-10-30T12:34:38.014 DEBUG:teuthology.orchestra.run.smithi012:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-30T12:34:38.062 DEBUG:teuthology.orchestra.run.smithi154:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-30T12:34:38.074 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.y] 2023-10-30T12:34:38.074 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC9oj9lOSR1FBAAacEv/fyzDDTiYXhn9Qqf/Q== 2023-10-30T12:34:38.074 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-10-30T12:34:38.075 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC9oj9laJubGxAAHo2pznFGqT8M4Azq4YS24g== 2023-10-30T12:34:38.075 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-10-30T12:34:38.075 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC9oj9lt0chIxAA7mAjl70prJh3gLahaLzVDw== 2023-10-30T12:34:38.075 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.e] 2023-10-30T12:34:38.075 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC9oj9lzLF+KhAAbZE5/snvlMT1wGtpoSzdkw== 2023-10-30T12:34:38.076 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.0] 2023-10-30T12:34:38.076 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDBoj9lsu3FFBAAN/2EI2cPOWglDsadYVQrxQ== 2023-10-30T12:34:38.076 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.1] 2023-10-30T12:34:38.076 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDEoj9lTih/HRAAL9Ek54xIN10si3q8UIS4sw== 2023-10-30T12:34:38.076 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.2] 2023-10-30T12:34:38.077 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDHoj9lGReWJBAAKMF517uy8tXwyMuxjU20Yg== 2023-10-30T12:34:38.077 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.3] 2023-10-30T12:34:38.077 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDKoj9l4CqkLBAAf3HUCCHiMh/rohhYDN4F7w== 2023-10-30T12:34:38.077 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.x] 2023-10-30T12:34:38.078 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC9oj9lR1taFxAAZ1Zvw3Suv3AGI0hWXHWIjw== 2023-10-30T12:34:38.078 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-10-30T12:34:38.078 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC9oj9l59nJLhAA8TFyVAUaB8tPbDQd7eyYmg== 2023-10-30T12:34:38.078 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-10-30T12:34:38.078 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC9oj9leuvyNhAAgi8DHsEBlyBDnsixsqM1Hg== 2023-10-30T12:34:38.079 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.f] 2023-10-30T12:34:38.079 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC+oj9lsy3JAxAA1s1ZpAHUDqGg1cZfvKko8Q== 2023-10-30T12:34:38.079 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.4] 2023-10-30T12:34:38.079 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDQoj9lU36DKRAAmhAY8pBPeDZCZ2YW51ttJQ== 2023-10-30T12:34:38.079 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.5] 2023-10-30T12:34:38.080 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDToj9l0LQSMBAAXPvCnnivWPMj//Pw9Dp+kQ== 2023-10-30T12:34:38.080 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.6] 2023-10-30T12:34:38.080 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDWoj9lTWzdOBAABSwxGWRSvhmhguhT8NUlNw== 2023-10-30T12:34:38.080 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.7] 2023-10-30T12:34:38.081 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDaoj9lTvpyAxAAUNQRgeWMRRxTeb0cQzA1xg== 2023-10-30T12:34:38.081 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2023-10-30T12:34:38.081 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC+oj9laCruCRAA+9dPk/QebBxVskaegJqgWw== 2023-10-30T12:34:38.083 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr.y] 2023-10-30T12:34:38.083 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC9oj9lOSR1FBAAacEv/fyzDDTiYXhn9Qqf/Q== 2023-10-30T12:34:38.083 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.a] 2023-10-30T12:34:38.084 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC9oj9laJubGxAAHo2pznFGqT8M4Azq4YS24g== 2023-10-30T12:34:38.084 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.c] 2023-10-30T12:34:38.084 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC9oj9lt0chIxAA7mAjl70prJh3gLahaLzVDw== 2023-10-30T12:34:38.084 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.e] 2023-10-30T12:34:38.084 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC9oj9lzLF+KhAAbZE5/snvlMT1wGtpoSzdkw== 2023-10-30T12:34:38.085 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.0] 2023-10-30T12:34:38.085 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDBoj9lsu3FFBAAN/2EI2cPOWglDsadYVQrxQ== 2023-10-30T12:34:38.085 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.1] 2023-10-30T12:34:38.085 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDEoj9lTih/HRAAL9Ek54xIN10si3q8UIS4sw== 2023-10-30T12:34:38.086 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.2] 2023-10-30T12:34:38.086 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDHoj9lGReWJBAAKMF517uy8tXwyMuxjU20Yg== 2023-10-30T12:34:38.086 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.3] 2023-10-30T12:34:38.086 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDKoj9l4CqkLBAAf3HUCCHiMh/rohhYDN4F7w== 2023-10-30T12:34:38.086 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr.x] 2023-10-30T12:34:38.087 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC9oj9lR1taFxAAZ1Zvw3Suv3AGI0hWXHWIjw== 2023-10-30T12:34:38.087 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.b] 2023-10-30T12:34:38.087 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC9oj9l59nJLhAA8TFyVAUaB8tPbDQd7eyYmg== 2023-10-30T12:34:38.087 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.d] 2023-10-30T12:34:38.087 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC9oj9leuvyNhAAgi8DHsEBlyBDnsixsqM1Hg== 2023-10-30T12:34:38.088 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.f] 2023-10-30T12:34:38.088 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC+oj9lsy3JAxAA1s1ZpAHUDqGg1cZfvKko8Q== 2023-10-30T12:34:38.088 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.4] 2023-10-30T12:34:38.089 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDQoj9lU36DKRAAmhAY8pBPeDZCZ2YW51ttJQ== 2023-10-30T12:34:38.089 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.5] 2023-10-30T12:34:38.089 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDToj9l0LQSMBAAXPvCnnivWPMj//Pw9Dp+kQ== 2023-10-30T12:34:38.089 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.6] 2023-10-30T12:34:38.089 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDWoj9lTWzdOBAABSwxGWRSvhmhguhT8NUlNw== 2023-10-30T12:34:38.090 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.7] 2023-10-30T12:34:38.090 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDaoj9lTvpyAxAAUNQRgeWMRRxTeb0cQzA1xg== 2023-10-30T12:34:38.090 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2023-10-30T12:34:38.090 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC+oj9laCruCRAA+9dPk/QebBxVskaegJqgWw== 2023-10-30T12:34:38.091 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-30T12:34:38.130 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-30T12:34:38.187 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:34:38.191 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:34:38.249 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:34:38.253 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:34:38.325 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:34:38.330 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:34:38.385 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:38.389 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:38.459 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:38.463 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:38.546 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:38.550 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:38.630 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:38.634 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:38.714 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-30T12:34:38.718 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-30T12:34:38.799 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:34:38.803 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:34:38.894 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:34:38.898 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:34:38.983 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:34:38.987 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:34:39.066 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:39.070 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:39.148 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:39.152 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:39.227 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:39.231 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:39.312 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:39.316 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:34:39.402 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-30T12:34:39.406 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-30T12:34:39.489 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-30T12:34:39.490 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-30T12:34:39.511 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-30T12:34:39.627 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-30T12:34:39.649 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-30T12:34:39.663 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-30T12:34:39.762 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-30T12:34:39.773 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-30T12:34:39.824 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-30T12:34:39.921 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-30T12:34:39.931 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:34:39.935 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:34:39.978 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-30T12:34:39.979 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-30T12:34:39.979 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-30T12:34:39.986 INFO:tasks.ceph.mon.a:Started 2023-10-30T12:34:39.986 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-30T12:34:39.986 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-30T12:34:40.024 INFO:tasks.ceph.mon.b:Started 2023-10-30T12:34:40.025 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-30T12:34:40.025 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-30T12:34:40.027 INFO:tasks.ceph.mon.c:Started 2023-10-30T12:34:40.028 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-30T12:34:40.028 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-30T12:34:40.028 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-30T12:34:40.032 INFO:tasks.ceph.mgr.y:Started 2023-10-30T12:34:40.032 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-30T12:34:40.033 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-30T12:34:40.035 INFO:tasks.ceph.mgr.x:Started 2023-10-30T12:34:40.035 INFO:tasks.ceph:Setting crush tunables to default 2023-10-30T12:34:40.035 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd crush tunables default 2023-10-30T12:34:40.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.175+0000 7f682a19d640 1 Processor -- start 2023-10-30T12:34:40.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.175+0000 7f682a19d640 1 -- start start 2023-10-30T12:34:40.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.175+0000 7f682a19d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6824156930 0x7f6824156d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:40.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.175+0000 7f682a19d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68241572c0 con 0x7f68240722a0 2023-10-30T12:34:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.175+0000 7f682a19d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68240c0a30 con 0x7f68241508c0 2023-10-30T12:34:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.175+0000 7f682a19d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68240c85b0 con 0x7f6824156930 2023-10-30T12:34:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.179+0000 7f6823fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6824156930 0x7f6824156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.179+0000 7f6823fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6824156930 0x7f6824156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42492/0 (socket says 172.21.15.12:42492) 2023-10-30T12:34:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.179+0000 7f6823fff640 1 -- 172.21.15.12:0/611247365 learned_addr learned my addr 172.21.15.12:0/611247365 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:34:40.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.179+0000 7f6823fff640 1 -- 172.21.15.12:0/611247365 >> v2:172.21.15.12:3300/0 conn(0x7f6824156930 msgr2=0x7f6824156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-30T12:34:40.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.179+0000 7f6823fff640 1 -- 172.21.15.12:0/611247365 >> v2:172.21.15.12:3300/0 conn(0x7f6824156930 msgr2=0x7f6824156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:34:40.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.179+0000 7f6823fff640 1 --2- 172.21.15.12:0/611247365 >> v2:172.21.15.12:3300/0 conn(0x7f6824156930 0x7f6824156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:34:40.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.179+0000 7f6823fff640 1 --2- 172.21.15.12:0/611247365 >> v2:172.21.15.12:3300/0 conn(0x7f6824156930 0x7f6824156d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-30T12:34:40.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.383+0000 7f6823fff640 1 --2- 172.21.15.12:0/611247365 >> v2:172.21.15.12:3300/0 conn(0x7f6824156930 0x7f6824156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:40.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.383+0000 7f6823fff640 1 -- 172.21.15.12:0/611247365 >> v2:172.21.15.12:3300/0 conn(0x7f6824156930 msgr2=0x7f6824156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-30T12:34:40.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.383+0000 7f6823fff640 1 -- 172.21.15.12:0/611247365 >> v2:172.21.15.12:3300/0 conn(0x7f6824156930 msgr2=0x7f6824156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:34:40.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.383+0000 7f6823fff640 1 --2- 172.21.15.12:0/611247365 >> v2:172.21.15.12:3300/0 conn(0x7f6824156930 0x7f6824156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:34:40.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.383+0000 7f6823fff640 1 --2- 172.21.15.12:0/611247365 >> v2:172.21.15.12:3300/0 conn(0x7f6824156930 0x7f6824156d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-30T12:34:40.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68227fc640 1 -- 172.21.15.12:0/611247365 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3350160777 0 0) 0x7f68241572c0 con 0x7f68240722a0 2023-10-30T12:34:40.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68227fc640 1 -- 172.21.15.12:0/611247365 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f680c003530 con 0x7f68240722a0 2023-10-30T12:34:40.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68227fc640 1 -- 172.21.15.12:0/611247365 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3945918758 0 0) 0x7f680c003530 con 0x7f68240722a0 2023-10-30T12:34:40.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68227fc640 1 -- 172.21.15.12:0/611247365 >> v2:172.21.15.12:3300/0 conn(0x7f6824156930 msgr2=0x7f6824156d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-30T12:34:40.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68227fc640 1 --2- 172.21.15.12:0/611247365 >> v2:172.21.15.12:3300/0 conn(0x7f6824156930 0x7f6824156d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:40.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68227fc640 1 -- 172.21.15.12:0/611247365 >> v1:172.21.15.154:6789/0 conn(0x7f68241508c0 legacy=0x7f68240c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:40.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68227fc640 1 -- 172.21.15.12:0/611247365 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682406f650 con 0x7f68240722a0 2023-10-30T12:34:40.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68227fc640 1 -- 172.21.15.12:0/611247365 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f6818001680 con 0x7f68240722a0 2023-10-30T12:34:40.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68227fc640 1 -- 172.21.15.12:0/611247365 >> v1:172.21.15.12:6789/0 conn(0x7f68240722a0 legacy=0x7f68240bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:40.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68227fc640 1 --2- 172.21.15.12:0/611247365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f680c003980 0x7f680c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:40.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68227fc640 1 --2- 172.21.15.12:0/611247365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f680c0047d0 0x7f680c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:40.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68227fc640 1 --2- 172.21.15.12:0/611247365 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f680c0052b0 0x7f680c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:40.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68227fc640 1 -- 172.21.15.12:0/611247365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68240c85b0 con 0x7f680c003980 2023-10-30T12:34:40.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68227fc640 1 -- 172.21.15.12:0/611247365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c00c180 con 0x7f680c0047d0 2023-10-30T12:34:40.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68227fc640 1 -- 172.21.15.12:0/611247365 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680c00c340 con 0x7f680c0052b0 2023-10-30T12:34:40.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68237fe640 1 --2- 172.21.15.12:0/611247365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f680c003980 0x7f680c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:40.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f6822ffd640 1 --2- 172.21.15.12:0/611247365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f680c0047d0 0x7f680c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:40.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f6823fff640 1 --2- 172.21.15.12:0/611247365 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f680c0052b0 0x7f680c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:40.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68237fe640 1 -- 172.21.15.12:0/611247365 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f680c0052b0 msgr2=0x7f680c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:40.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68237fe640 1 --2- 172.21.15.12:0/611247365 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f680c0052b0 0x7f680c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:40.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68237fe640 1 -- 172.21.15.12:0/611247365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f680c0047d0 msgr2=0x7f680c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:40.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68237fe640 1 --2- 172.21.15.12:0/611247365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f680c0047d0 0x7f680c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:40.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f68237fe640 1 -- 172.21.15.12:0/611247365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c003530 con 0x7f680c003980 2023-10-30T12:34:40.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.407+0000 7f6823fff640 1 -- 172.21.15.12:0/611247365 reap_dead start 2023-10-30T12:34:40.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.411+0000 7f68237fe640 1 --2- 172.21.15.12:0/611247365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f680c003980 0x7f680c003da0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6814001780 tx=0x7f681401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3253fd6d06f6bf6e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:40.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.411+0000 7f68227fc640 1 -- 172.21.15.12:0/611247365 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68140052d0 con 0x7f680c003980 2023-10-30T12:34:40.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.411+0000 7f68227fc640 1 -- 172.21.15.12:0/611247365 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68140a7050 con 0x7f680c003980 2023-10-30T12:34:40.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.411+0000 7f68227fc640 1 -- 172.21.15.12:0/611247365 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814003650 con 0x7f680c003980 2023-10-30T12:34:40.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.411+0000 7f682a19d640 1 -- 172.21.15.12:0/611247365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f680c003980 msgr2=0x7f680c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:40.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.411+0000 7f682a19d640 1 --2- 172.21.15.12:0/611247365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f680c003980 0x7f680c003da0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6814001780 tx=0x7f681401c360 comp rx=0 tx=0).stop 2023-10-30T12:34:40.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.411+0000 7f682a19d640 1 -- 172.21.15.12:0/611247365 shutdown_connections 2023-10-30T12:34:40.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.411+0000 7f682a19d640 1 --2- 172.21.15.12:0/611247365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f680c003980 0x7f680c003da0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:40.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.411+0000 7f682a19d640 1 -- 172.21.15.12:0/611247365 >> 172.21.15.12:0/611247365 conn(0x7f68240c7910 msgr2=0x7f68240c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:40.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.411+0000 7f682a19d640 1 -- 172.21.15.12:0/611247365 shutdown_connections 2023-10-30T12:34:40.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.411+0000 7f682a19d640 1 -- 172.21.15.12:0/611247365 wait complete. 2023-10-30T12:34:40.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f682a19d640 1 Processor -- start 2023-10-30T12:34:40.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f682a19d640 1 -- start start 2023-10-30T12:34:40.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f682a19d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f680c003980 0x7f68241f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:40.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f682a19d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68241f4f30 0x7f6824145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:40.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f682a19d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68241558f0 0x7f6824146260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:40.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f682a19d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68241534b0 con 0x7f68241558f0 2023-10-30T12:34:40.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f682a19d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68241f3470 con 0x7f680c003980 2023-10-30T12:34:40.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f682a19d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6824157b40 con 0x7f68241f4f30 2023-10-30T12:34:40.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f68237fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f680c003980 0x7f68241f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:40.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f6822ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68241f4f30 0x7f6824145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:40.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f68237fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f680c003980 0x7f68241f49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:49094/0 (socket says 172.21.15.12:49094) 2023-10-30T12:34:40.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f68237fe640 1 -- 172.21.15.12:0/1322070086 learned_addr learned my addr 172.21.15.12:0/1322070086 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:34:40.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f6823fff640 1 --2- 172.21.15.12:0/1322070086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68241558f0 0x7f6824146260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:40.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f6823fff640 1 -- 172.21.15.12:0/1322070086 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68241f4f30 msgr2=0x7f6824145d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:40.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f6823fff640 1 --2- 172.21.15.12:0/1322070086 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68241f4f30 0x7f6824145d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:40.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f6823fff640 1 -- 172.21.15.12:0/1322070086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f680c003980 msgr2=0x7f68241f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:40.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f6823fff640 1 --2- 172.21.15.12:0/1322070086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f680c003980 0x7f68241f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:40.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f6823fff640 1 -- 172.21.15.12:0/1322070086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68241555c0 con 0x7f68241558f0 2023-10-30T12:34:40.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.415+0000 7f6823fff640 1 --2- 172.21.15.12:0/1322070086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68241558f0 0x7f6824146260 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6824156d10 tx=0x7f681c007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:40.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.419+0000 7f680bfff640 1 -- 172.21.15.12:0/1322070086 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c010bc0 con 0x7f68241558f0 2023-10-30T12:34:40.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.419+0000 7f680bfff640 1 -- 172.21.15.12:0/1322070086 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c00f040 con 0x7f68241558f0 2023-10-30T12:34:40.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.419+0000 7f680bfff640 1 -- 172.21.15.12:0/1322070086 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c015440 con 0x7f68241558f0 2023-10-30T12:34:40.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.419+0000 7f682a19d640 1 -- 172.21.15.12:0/1322070086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6824156410 con 0x7f68241558f0 2023-10-30T12:34:40.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.419+0000 7f682a19d640 1 -- 172.21.15.12:0/1322070086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68241552e0 con 0x7f68241558f0 2023-10-30T12:34:40.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.419+0000 7f680bfff640 1 -- 172.21.15.12:0/1322070086 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f681c0158a0 con 0x7f68241558f0 2023-10-30T12:34:40.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.419+0000 7f680bfff640 1 -- 172.21.15.12:0/1322070086 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f681c01e8c0 con 0x7f68241558f0 2023-10-30T12:34:40.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.419+0000 7f682a19d640 1 -- 172.21.15.12:0/1322070086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68240701b0 con 0x7f68241558f0 2023-10-30T12:34:40.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.423+0000 7f680bfff640 1 -- 172.21.15.12:0/1322070086 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f68240701b0 con 0x7f68241558f0 2023-10-30T12:34:40.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.487+0000 7f682a19d640 1 -- 172.21.15.12:0/1322070086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f68241f35e0 con 0x7f68241558f0 2023-10-30T12:34:40.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.491+0000 7f680bfff640 1 -- 172.21.15.12:0/1322070086 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f681c01e410 con 0x7f68241558f0 2023-10-30T12:34:40.492 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2023-10-30T12:34:40.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.491+0000 7f682a19d640 1 -- 172.21.15.12:0/1322070086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68241558f0 msgr2=0x7f6824146260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:40.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.491+0000 7f682a19d640 1 --2- 172.21.15.12:0/1322070086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68241558f0 0x7f6824146260 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6824156d10 tx=0x7f681c007e20 comp rx=0 tx=0).stop 2023-10-30T12:34:40.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.491+0000 7f682a19d640 1 -- 172.21.15.12:0/1322070086 shutdown_connections 2023-10-30T12:34:40.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.491+0000 7f682a19d640 1 --2- 172.21.15.12:0/1322070086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68241558f0 0x7f6824146260 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:40.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.491+0000 7f682a19d640 1 --2- 172.21.15.12:0/1322070086 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68241f4f30 0x7f6824145d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:40.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.491+0000 7f682a19d640 1 --2- 172.21.15.12:0/1322070086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f680c003980 0x7f68241f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:40.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.491+0000 7f682a19d640 1 -- 172.21.15.12:0/1322070086 >> 172.21.15.12:0/1322070086 conn(0x7f68240c7910 msgr2=0x7f68240c1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:40.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.491+0000 7f682a19d640 1 -- 172.21.15.12:0/1322070086 shutdown_connections 2023-10-30T12:34:40.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:40.491+0000 7f682a19d640 1 -- 172.21.15.12:0/1322070086 wait complete. 2023-10-30T12:34:40.513 INFO:tasks.ceph:check_enable_crimson: False 2023-10-30T12:34:40.513 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-30T12:34:40.513 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:40.514 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-30T12:34:40.536 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:40.537 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-30T12:34:40.601 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:40.600+0000 7f692bb33180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-30T12:34:40.605 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:40.605 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-30T12:34:40.623 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:40.619+0000 7f3d8f37e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-30T12:34:40.645 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:34:40.645 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-30T12:34:40.713 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:40.714 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-30T12:34:40.727 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:40.727 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-30T12:34:40.783 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:40.783 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-30T12:34:40.839 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-30T12:34:40.839 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-30T12:34:40.895 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new ce150b4b-3d29-49ec-9e06-bf801656e271 0 2023-10-30T12:34:41.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.060+0000 7f81ee9c0640 1 Processor -- start 2023-10-30T12:34:41.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.060+0000 7f81ee9c0640 1 -- start start 2023-10-30T12:34:41.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81ee9c0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f81e8153980 0x7f81e80caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81ee9c0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81e80cb1e0 0x7f81e815d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81ee9c0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e815a920 con 0x7f81e8159930 2023-10-30T12:34:41.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81ee9c0640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e80cb930 con 0x7f81e8153980 2023-10-30T12:34:41.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81ee9c0640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e8159dc0 con 0x7f81e80cb1e0 2023-10-30T12:34:41.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81e7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f81e8153980 0x7f81e80caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81ecf5d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81e80cb1e0 0x7f81e815d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81e7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f81e8153980 0x7f81e80caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60782/0 (socket says 172.21.15.154:60782) 2023-10-30T12:34:41.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81ecf5d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81e80cb1e0 0x7f81e815d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:40652/0 (socket says 172.21.15.154:40652) 2023-10-30T12:34:41.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81e7fff640 1 -- 172.21.15.154:0/333809587 learned_addr learned my addr 172.21.15.154:0/333809587 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-30T12:34:41.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81e7fff640 1 -- 172.21.15.154:0/333809587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81e80cb1e0 msgr2=0x7f81e815d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:41.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81e7fff640 1 --2- 172.21.15.154:0/333809587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81e80cb1e0 0x7f81e815d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81e7fff640 1 -- 172.21.15.154:0/333809587 >> v1:172.21.15.154:6789/0 conn(0x7f81e8159930 legacy=0x7f81e80cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81e7fff640 1 -- 172.21.15.154:0/333809587 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e815a080 con 0x7f81e8153980 2023-10-30T12:34:41.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81e77fe640 1 -- 172.21.15.154:0/333809587 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 427466996 0 0) 0x7f81e815a920 con 0x7f81e8159930 2023-10-30T12:34:41.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.064+0000 7f81ecf5d640 1 --2- 172.21.15.154:0/333809587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81e80cb1e0 0x7f81e815d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:34:41.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81e7fff640 1 --2- 172.21.15.154:0/333809587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81e8153980 0x7f81e80caca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009870 tx=0x7f81d801c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb848a84e77849 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:41.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81e77fe640 1 -- 172.21.15.154:0/333809587 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d80ab620 con 0x7f81e8153980 2023-10-30T12:34:41.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81e77fe640 1 -- 172.21.15.154:0/333809587 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d80ab780 con 0x7f81e8153980 2023-10-30T12:34:41.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81e77fe640 1 -- 172.21.15.154:0/333809587 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d80abb10 con 0x7f81e8153980 2023-10-30T12:34:41.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 -- 172.21.15.154:0/333809587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81e8153980 msgr2=0x7f81e80caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 --2- 172.21.15.154:0/333809587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81e8153980 0x7f81e80caca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009870 tx=0x7f81d801c510 comp rx=0 tx=0).stop 2023-10-30T12:34:41.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 -- 172.21.15.154:0/333809587 shutdown_connections 2023-10-30T12:34:41.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 --2- 172.21.15.154:0/333809587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81e80cb1e0 0x7f81e815d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 --2- 172.21.15.154:0/333809587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81e8153980 0x7f81e80caca0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 -- 172.21.15.154:0/333809587 >> 172.21.15.154:0/333809587 conn(0x7f81e806c0f0 msgr2=0x7f81e806c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:41.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 -- 172.21.15.154:0/333809587 shutdown_connections 2023-10-30T12:34:41.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 -- 172.21.15.154:0/333809587 wait complete. 2023-10-30T12:34:41.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 Processor -- start 2023-10-30T12:34:41.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 -- start start 2023-10-30T12:34:41.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81e80cb1e0 0x7f81e80c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81e8153980 0x7f81e8149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81e80cdc10 0x7f81e80ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e81f2b10 con 0x7f81e8153980 2023-10-30T12:34:41.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e80c8f40 con 0x7f81e80cb1e0 2023-10-30T12:34:41.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ee9c0640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e81f1ca0 con 0x7f81e80cdc10 2023-10-30T12:34:41.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81e7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81e80cb1e0 0x7f81e80c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ecf5d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81e80cdc10 0x7f81e80ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ecf5d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81e80cdc10 0x7f81e80ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:40656/0 (socket says 172.21.15.154:40656) 2023-10-30T12:34:41.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ecf5d640 1 -- 172.21.15.154:0/4205742505 learned_addr learned my addr 172.21.15.154:0/4205742505 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-30T12:34:41.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81dffff640 1 --2- 172.21.15.154:0/4205742505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81e8153980 0x7f81e8149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ecf5d640 1 -- 172.21.15.154:0/4205742505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81e80cb1e0 msgr2=0x7f81e80c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ecf5d640 1 --2- 172.21.15.154:0/4205742505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81e80cb1e0 0x7f81e80c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ecf5d640 1 -- 172.21.15.154:0/4205742505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81e8153980 msgr2=0x7f81e8149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ecf5d640 1 --2- 172.21.15.154:0/4205742505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81e8153980 0x7f81e8149d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ecf5d640 1 -- 172.21.15.154:0/4205742505 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e0009a70 con 0x7f81e80cdc10 2023-10-30T12:34:41.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81e7fff640 1 --2- 172.21.15.154:0/4205742505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81e80cb1e0 0x7f81e80c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:34:41.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.068+0000 7f81ecf5d640 1 --2- 172.21.15.154:0/4205742505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81e80cdc10 0x7f81e80ce0a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f81e000efc0 tx=0x7f81e0009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:41.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.072+0000 7f81e57fa640 1 -- 172.21.15.154:0/4205742505 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e0009d70 con 0x7f81e80cdc10 2023-10-30T12:34:41.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.072+0000 7f81e57fa640 1 -- 172.21.15.154:0/4205742505 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e000f040 con 0x7f81e80cdc10 2023-10-30T12:34:41.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.072+0000 7f81e57fa640 1 -- 172.21.15.154:0/4205742505 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e00039a0 con 0x7f81e80cdc10 2023-10-30T12:34:41.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.072+0000 7f81ee9c0640 1 -- 172.21.15.154:0/4205742505 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d80094d0 con 0x7f81e80cdc10 2023-10-30T12:34:41.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.072+0000 7f81ee9c0640 1 -- 172.21.15.154:0/4205742505 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e81f1e90 con 0x7f81e80cdc10 2023-10-30T12:34:41.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.072+0000 7f81e57fa640 1 -- 172.21.15.154:0/4205742505 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f81e00147a0 con 0x7f81e80cdc10 2023-10-30T12:34:41.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.072+0000 7f81e57fa640 1 -- 172.21.15.154:0/4205742505 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f81e00133e0 con 0x7f81e80cdc10 2023-10-30T12:34:41.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.072+0000 7f81ee9c0640 1 -- 172.21.15.154:0/4205742505 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e804e170 con 0x7f81e80cdc10 2023-10-30T12:34:41.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.072+0000 7f81e57fa640 1 -- 172.21.15.154:0/4205742505 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f81e0018030 con 0x7f81e80cdc10 2023-10-30T12:34:41.117 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:41.116+0000 7f692bb33180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-30T12:34:41.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.140+0000 7f81ee9c0640 1 -- 172.21.15.154:0/4205742505 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ce150b4b-3d29-49ec-9e06-bf801656e271", "id": 0} v 0) v1 -- 0x7f81e8157710 con 0x7f81e80cdc10 2023-10-30T12:34:41.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.148+0000 7f81e57fa640 1 -- 172.21.15.154:0/4205742505 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ce150b4b-3d29-49ec-9e06-bf801656e271", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f81e0014c80 con 0x7f81e80cdc10 2023-10-30T12:34:41.151 INFO:teuthology.orchestra.run.smithi154.stdout:0 2023-10-30T12:34:41.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.152+0000 7f81ee9c0640 1 -- 172.21.15.154:0/4205742505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81e80cdc10 msgr2=0x7f81e80ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.152+0000 7f81ee9c0640 1 --2- 172.21.15.154:0/4205742505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81e80cdc10 0x7f81e80ce0a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f81e000efc0 tx=0x7f81e0009f90 comp rx=0 tx=0).stop 2023-10-30T12:34:41.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.152+0000 7f81ee9c0640 1 -- 172.21.15.154:0/4205742505 shutdown_connections 2023-10-30T12:34:41.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.152+0000 7f81ee9c0640 1 --2- 172.21.15.154:0/4205742505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81e80cdc10 0x7f81e80ce0a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.152+0000 7f81ee9c0640 1 --2- 172.21.15.154:0/4205742505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81e8153980 0x7f81e8149d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.152+0000 7f81ee9c0640 1 --2- 172.21.15.154:0/4205742505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81e80cb1e0 0x7f81e80c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.152+0000 7f81ee9c0640 1 -- 172.21.15.154:0/4205742505 >> 172.21.15.154:0/4205742505 conn(0x7f81e806c0f0 msgr2=0x7f81e80c4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:41.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.152+0000 7f81ee9c0640 1 -- 172.21.15.154:0/4205742505 shutdown_connections 2023-10-30T12:34:41.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.152+0000 7f81ee9c0640 1 -- 172.21.15.154:0/4205742505 wait complete. 2023-10-30T12:34:41.167 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new b1ca990f-7ae6-42e1-9a39-94e0fd820e40 1 2023-10-30T12:34:41.211 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:41.211+0000 7f3d8f37e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-30T12:34:41.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.280+0000 7fcbc2c33640 1 Processor -- start 2023-10-30T12:34:41.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.280+0000 7fcbc2c33640 1 -- start start 2023-10-30T12:34:41.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.280+0000 7fcbc2c33640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbbc0c4080 0x7fcbbc06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.280+0000 7fcbc2c33640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcbbc0c7110 0x7fcbbc0c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.280+0000 7fcbc2c33640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbbc070580 con 0x7fcbbc150810 2023-10-30T12:34:41.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.280+0000 7fcbc2c33640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbbc0c37d0 con 0x7fcbbc0c7110 2023-10-30T12:34:41.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.280+0000 7fcbc2c33640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc150bf0 con 0x7fcbbc0c4080 2023-10-30T12:34:41.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.280+0000 7fcbc11d0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcbbc0c7110 0x7fcbbc0c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.280+0000 7fcbc11d0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcbbc0c7110 0x7fcbbc0c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60802/0 (socket says 172.21.15.154:60802) 2023-10-30T12:34:41.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.280+0000 7fcbc11d0640 1 -- 172.21.15.154:0/1543497293 learned_addr learned my addr 172.21.15.154:0/1543497293 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-30T12:34:41.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.280+0000 7fcbbbfff640 1 --2- 172.21.15.154:0/1543497293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbbc0c4080 0x7fcbbc06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbc11d0640 1 -- 172.21.15.154:0/1543497293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbbc0c4080 msgr2=0x7fcbbc06fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:41.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbc11d0640 1 --2- 172.21.15.154:0/1543497293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbbc0c4080 0x7fcbbc06fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbc11d0640 1 -- 172.21.15.154:0/1543497293 >> v1:172.21.15.12:6789/0 conn(0x7fcbbc150810 legacy=0x7fcbbc15a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:41.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbc11d0640 1 -- 172.21.15.154:0/1543497293 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc0c6070 con 0x7fcbbc0c7110 2023-10-30T12:34:41.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbc11d0640 1 --2- 172.21.15.154:0/1543497293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbbc0c7110 0x7fcbbc0c7fa0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009e50 tx=0x7fcbb401c9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ad44c0324448328 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:41.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbbb7fe640 1 -- 172.21.15.154:0/1543497293 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40a5650 con 0x7fcbbc0c7110 2023-10-30T12:34:41.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbbb7fe640 1 -- 172.21.15.154:0/1543497293 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb40a57b0 con 0x7fcbbc0c7110 2023-10-30T12:34:41.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbbb7fe640 1 -- 172.21.15.154:0/1543497293 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40a5b40 con 0x7fcbbc0c7110 2023-10-30T12:34:41.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbc2c33640 1 -- 172.21.15.154:0/1543497293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbbc0c7110 msgr2=0x7fcbbc0c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbc2c33640 1 --2- 172.21.15.154:0/1543497293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbbc0c7110 0x7fcbbc0c7fa0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009e50 tx=0x7fcbb401c9d0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbc2c33640 1 -- 172.21.15.154:0/1543497293 shutdown_connections 2023-10-30T12:34:41.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbc2c33640 1 --2- 172.21.15.154:0/1543497293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbbc0c4080 0x7fcbbc06fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbc2c33640 1 --2- 172.21.15.154:0/1543497293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbbc0c7110 0x7fcbbc0c7fa0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbc2c33640 1 -- 172.21.15.154:0/1543497293 >> 172.21.15.154:0/1543497293 conn(0x7fcbbc06c620 msgr2=0x7fcbbc069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:41.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbc2c33640 1 -- 172.21.15.154:0/1543497293 shutdown_connections 2023-10-30T12:34:41.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.284+0000 7fcbc2c33640 1 -- 172.21.15.154:0/1543497293 wait complete. 2023-10-30T12:34:41.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbc2c33640 1 Processor -- start 2023-10-30T12:34:41.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbc2c33640 1 -- start start 2023-10-30T12:34:41.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbc2c33640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcbbc0c4080 0x7fcbbc1f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbc2c33640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbbc0c7110 0x7fcbbc148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbc2c33640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbbc150810 0x7fcbbc13ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbc2c33640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc15f190 con 0x7fcbbc0c4080 2023-10-30T12:34:41.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbc2c33640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc14efc0 con 0x7fcbbc150810 2023-10-30T12:34:41.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbc2c33640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc14f2a0 con 0x7fcbbc0c7110 2023-10-30T12:34:41.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbbbfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbbc0c7110 0x7fcbbc148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbc11d0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbbc150810 0x7fcbbc13ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbbbfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbbc0c7110 0x7fcbbc148bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:40676/0 (socket says 172.21.15.154:40676) 2023-10-30T12:34:41.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbbbfff640 1 -- 172.21.15.154:0/3497867082 learned_addr learned my addr 172.21.15.154:0/3497867082 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-30T12:34:41.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbc09cf640 1 --2- 172.21.15.154:0/3497867082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcbbc0c4080 0x7fcbbc1f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbbbfff640 1 -- 172.21.15.154:0/3497867082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbbc150810 msgr2=0x7fcbbc13ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbbbfff640 1 --2- 172.21.15.154:0/3497867082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbbc150810 0x7fcbbc13ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbbbfff640 1 -- 172.21.15.154:0/3497867082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcbbc0c4080 msgr2=0x7fcbbc1f2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbbbfff640 1 --2- 172.21.15.154:0/3497867082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcbbc0c4080 0x7fcbbc1f2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbbbfff640 1 -- 172.21.15.154:0/3497867082 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb4009b30 con 0x7fcbbc0c7110 2023-10-30T12:34:41.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbc11d0640 1 --2- 172.21.15.154:0/3497867082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbbc150810 0x7fcbbc13ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:34:41.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbbbfff640 1 --2- 172.21.15.154:0/3497867082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbbc0c7110 0x7fcbbc148bf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcba800e930 tx=0x7fcba800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:41.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbb97fa640 1 -- 172.21.15.154:0/3497867082 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8009690 con 0x7fcbbc0c7110 2023-10-30T12:34:41.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbb97fa640 1 -- 172.21.15.154:0/3497867082 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba800ca00 con 0x7fcbbc0c7110 2023-10-30T12:34:41.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbc2c33640 1 -- 172.21.15.154:0/3497867082 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbbc14f580 con 0x7fcbbc0c7110 2023-10-30T12:34:41.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbb97fa640 1 -- 172.21.15.154:0/3497867082 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba800cd70 con 0x7fcbbc0c7110 2023-10-30T12:34:41.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbc2c33640 1 -- 172.21.15.154:0/3497867082 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbbc149130 con 0x7fcbbc0c7110 2023-10-30T12:34:41.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbb97fa640 1 -- 172.21.15.154:0/3497867082 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcba8005ca0 con 0x7fcbbc0c7110 2023-10-30T12:34:41.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbb97fa640 1 -- 172.21.15.154:0/3497867082 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fcba80138f0 con 0x7fcbbc0c7110 2023-10-30T12:34:41.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.288+0000 7fcbc2c33640 1 -- 172.21.15.154:0/3497867082 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbbc14f790 con 0x7fcbbc0c7110 2023-10-30T12:34:41.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.292+0000 7fcbb97fa640 1 -- 172.21.15.154:0/3497867082 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fcba8021060 con 0x7fcbbc0c7110 2023-10-30T12:34:41.302 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:41.300+0000 7f692bb33180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-30T12:34:41.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.360+0000 7fcbc2c33640 1 -- 172.21.15.154:0/3497867082 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b1ca990f-7ae6-42e1-9a39-94e0fd820e40", "id": 1} v 0) v1 -- 0x7fcbbc0c68e0 con 0x7fcbbc0c7110 2023-10-30T12:34:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.368+0000 7fcbb97fa640 1 -- 172.21.15.154:0/3497867082 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b1ca990f-7ae6-42e1-9a39-94e0fd820e40", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcba8017070 con 0x7fcbbc0c7110 2023-10-30T12:34:41.370 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-10-30T12:34:41.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.368+0000 7fcbc2c33640 1 -- 172.21.15.154:0/3497867082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbbc0c7110 msgr2=0x7fcbbc148bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.368+0000 7fcbc2c33640 1 --2- 172.21.15.154:0/3497867082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbbc0c7110 0x7fcbbc148bf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcba800e930 tx=0x7fcba800edf0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.368+0000 7fcbc2c33640 1 -- 172.21.15.154:0/3497867082 shutdown_connections 2023-10-30T12:34:41.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.368+0000 7fcbc2c33640 1 --2- 172.21.15.154:0/3497867082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbbc150810 0x7fcbbc13ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.368+0000 7fcbc2c33640 1 --2- 172.21.15.154:0/3497867082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbbc0c7110 0x7fcbbc148bf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.368+0000 7fcbc2c33640 1 --2- 172.21.15.154:0/3497867082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcbbc0c4080 0x7fcbbc1f2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.368+0000 7fcbc2c33640 1 -- 172.21.15.154:0/3497867082 >> 172.21.15.154:0/3497867082 conn(0x7fcbbc06c620 msgr2=0x7fcbbc14a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:41.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.368+0000 7fcbc2c33640 1 -- 172.21.15.154:0/3497867082 shutdown_connections 2023-10-30T12:34:41.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.368+0000 7fcbc2c33640 1 -- 172.21.15.154:0/3497867082 wait complete. 2023-10-30T12:34:41.386 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 2f0d49df-1f4f-47a5-9dc4-a253753a371f 2 2023-10-30T12:34:41.423 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:41.423+0000 7f3d8f37e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-30T12:34:41.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff64013a640 1 Processor -- start 2023-10-30T12:34:41.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff64013a640 1 -- start start 2023-10-30T12:34:41.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff64013a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6380c22e0 0x7ff6380c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff64013a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff6380c2830 0x7ff6380c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff64013a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff638070150 con 0x7ff63815d290 2023-10-30T12:34:41.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff64013a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63806fbd0 con 0x7ff6380c2830 2023-10-30T12:34:41.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff64013a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63806fe70 con 0x7ff6380c22e0 2023-10-30T12:34:41.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff63d6d5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6380c22e0 0x7ff6380c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff63ded6640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff63815d290 0x7ff63815d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55180/0 (socket says 172.21.15.154:55180) 2023-10-30T12:34:41.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff63ded6640 1 -- 172.21.15.154:0/224985761 learned_addr learned my addr 172.21.15.154:0/224985761 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-10-30T12:34:41.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff63e6d7640 1 --2- 172.21.15.154:0/224985761 >> v2:172.21.15.12:3300/0 conn(0x7ff6380c2830 0x7ff6380c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff63d6d5640 1 -- 172.21.15.154:0/224985761 >> v2:172.21.15.12:3300/0 conn(0x7ff6380c2830 msgr2=0x7ff6380c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:41.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff63d6d5640 1 --2- 172.21.15.154:0/224985761 >> v2:172.21.15.12:3300/0 conn(0x7ff6380c2830 0x7ff6380c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff63d6d5640 1 -- 172.21.15.154:0/224985761 >> v1:172.21.15.154:6789/0 conn(0x7ff63815d290 legacy=0x7ff63815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff63ced4640 1 -- 172.21.15.154:0/224985761 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782786002 0 0) 0x7ff638070150 con 0x7ff63815d290 2023-10-30T12:34:41.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff63d6d5640 1 -- 172.21.15.154:0/224985761 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6380c2e20 con 0x7ff6380c22e0 2023-10-30T12:34:41.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff63e6d7640 1 --2- 172.21.15.154:0/224985761 >> v2:172.21.15.12:3300/0 conn(0x7ff6380c2830 0x7ff6380c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:34:41.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff63d6d5640 1 --2- 172.21.15.154:0/224985761 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6380c22e0 0x7ff6380c3900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff6280097c0 tx=0x7ff62801ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6f9d01859b5cb86 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:41.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.504+0000 7ff63ced4640 1 -- 172.21.15.154:0/224985761 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280a8970 con 0x7ff6380c22e0 2023-10-30T12:34:41.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff63ced4640 1 -- 172.21.15.154:0/224985761 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6280a8ad0 con 0x7ff6380c22e0 2023-10-30T12:34:41.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff63ced4640 1 -- 172.21.15.154:0/224985761 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280a8e60 con 0x7ff6380c22e0 2023-10-30T12:34:41.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 -- 172.21.15.154:0/224985761 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6380c22e0 msgr2=0x7ff6380c3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 --2- 172.21.15.154:0/224985761 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6380c22e0 0x7ff6380c3900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff6280097c0 tx=0x7ff62801ec70 comp rx=0 tx=0).stop 2023-10-30T12:34:41.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 -- 172.21.15.154:0/224985761 shutdown_connections 2023-10-30T12:34:41.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 --2- 172.21.15.154:0/224985761 >> v2:172.21.15.12:3300/0 conn(0x7ff6380c2830 0x7ff6380c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 --2- 172.21.15.154:0/224985761 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6380c22e0 0x7ff6380c3900 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 -- 172.21.15.154:0/224985761 >> 172.21.15.154:0/224985761 conn(0x7ff6380ccfe0 msgr2=0x7ff6380cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:41.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 -- 172.21.15.154:0/224985761 shutdown_connections 2023-10-30T12:34:41.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 -- 172.21.15.154:0/224985761 wait complete. 2023-10-30T12:34:41.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 Processor -- start 2023-10-30T12:34:41.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 -- start start 2023-10-30T12:34:41.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6380c22e0 0x7ff6381f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6380c2830 0x7ff63815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff63815d290 0x7ff63806c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff638154d30 con 0x7ff6380c22e0 2023-10-30T12:34:41.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6381f1e20 con 0x7ff6380c2830 2023-10-30T12:34:41.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63806b0e0 con 0x7ff63815d290 2023-10-30T12:34:41.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff63d6d5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6380c2830 0x7ff63815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff63e6d7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff63815d290 0x7ff63806c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff63d6d5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6380c2830 0x7ff63815c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60820/0 (socket says 172.21.15.154:60820) 2023-10-30T12:34:41.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff63d6d5640 1 -- 172.21.15.154:0/4023625267 learned_addr learned my addr 172.21.15.154:0/4023625267 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-30T12:34:41.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff63ded6640 1 --2- 172.21.15.154:0/4023625267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6380c22e0 0x7ff6381f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff63d6d5640 1 -- 172.21.15.154:0/4023625267 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff63815d290 msgr2=0x7ff63806c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff63d6d5640 1 --2- 172.21.15.154:0/4023625267 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff63815d290 0x7ff63806c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff63d6d5640 1 -- 172.21.15.154:0/4023625267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6380c22e0 msgr2=0x7ff6381f18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff63d6d5640 1 --2- 172.21.15.154:0/4023625267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6380c22e0 0x7ff6381f18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff63d6d5640 1 -- 172.21.15.154:0/4023625267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff628009420 con 0x7ff6380c2830 2023-10-30T12:34:41.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff63e6d7640 1 --2- 172.21.15.154:0/4023625267 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff63815d290 0x7ff63806c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:34:41.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff63ded6640 1 --2- 172.21.15.154:0/4023625267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6380c22e0 0x7ff6381f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:34:41.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff63d6d5640 1 --2- 172.21.15.154:0/4023625267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6380c2830 0x7ff63815c800 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff62801c8a0 tx=0x7ff62801ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:41.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff62e7fc640 1 -- 172.21.15.154:0/4023625267 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280b0e40 con 0x7ff6380c2830 2023-10-30T12:34:41.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff62e7fc640 1 -- 172.21.15.154:0/4023625267 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6280b9410 con 0x7ff6380c2830 2023-10-30T12:34:41.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 -- 172.21.15.154:0/4023625267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63806b380 con 0x7ff6380c2830 2023-10-30T12:34:41.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff62e7fc640 1 -- 172.21.15.154:0/4023625267 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280b97c0 con 0x7ff6380c2830 2023-10-30T12:34:41.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff64013a640 1 -- 172.21.15.154:0/4023625267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63806d050 con 0x7ff6380c2830 2023-10-30T12:34:41.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff62e7fc640 1 -- 172.21.15.154:0/4023625267 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff6280b7040 con 0x7ff6380c2830 2023-10-30T12:34:41.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.508+0000 7ff62e7fc640 1 -- 172.21.15.154:0/4023625267 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7ff6280c5ce0 con 0x7ff6380c2830 2023-10-30T12:34:41.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.512+0000 7ff64013a640 1 -- 172.21.15.154:0/4023625267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63806ce90 con 0x7ff6380c2830 2023-10-30T12:34:41.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.512+0000 7ff62e7fc640 1 -- 172.21.15.154:0/4023625267 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7ff6280b5070 con 0x7ff6380c2830 2023-10-30T12:34:41.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.580+0000 7ff64013a640 1 -- 172.21.15.154:0/4023625267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2f0d49df-1f4f-47a5-9dc4-a253753a371f", "id": 2} v 0) v1 -- 0x7ff63806d240 con 0x7ff6380c2830 2023-10-30T12:34:41.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.588+0000 7ff62e7fc640 1 -- 172.21.15.154:0/4023625267 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2f0d49df-1f4f-47a5-9dc4-a253753a371f", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff6280c5460 con 0x7ff6380c2830 2023-10-30T12:34:41.591 INFO:teuthology.orchestra.run.smithi154.stdout:2 2023-10-30T12:34:41.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.592+0000 7ff64013a640 1 -- 172.21.15.154:0/4023625267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6380c2830 msgr2=0x7ff63815c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.592+0000 7ff64013a640 1 --2- 172.21.15.154:0/4023625267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6380c2830 0x7ff63815c800 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff62801c8a0 tx=0x7ff62801ba10 comp rx=0 tx=0).stop 2023-10-30T12:34:41.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.592+0000 7ff64013a640 1 -- 172.21.15.154:0/4023625267 shutdown_connections 2023-10-30T12:34:41.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.592+0000 7ff64013a640 1 --2- 172.21.15.154:0/4023625267 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff63815d290 0x7ff63806c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.592+0000 7ff64013a640 1 --2- 172.21.15.154:0/4023625267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6380c2830 0x7ff63815c800 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.592+0000 7ff64013a640 1 --2- 172.21.15.154:0/4023625267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6380c22e0 0x7ff6381f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.592+0000 7ff64013a640 1 -- 172.21.15.154:0/4023625267 >> 172.21.15.154:0/4023625267 conn(0x7ff6380ccfe0 msgr2=0x7ff6380c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:41.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.592+0000 7ff64013a640 1 -- 172.21.15.154:0/4023625267 shutdown_connections 2023-10-30T12:34:41.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.592+0000 7ff64013a640 1 -- 172.21.15.154:0/4023625267 wait complete. 2023-10-30T12:34:41.607 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 739f6118-6ac3-4a60-9da5-1a0ee98f9518 3 2023-10-30T12:34:41.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b74c7640 1 Processor -- start 2023-10-30T12:34:41.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b74c7640 1 -- start start 2023-10-30T12:34:41.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b74c7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b01539e0 0x7fc0b00c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b74c7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0b006d420 0x7fc0b01640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b74c7640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b015a670 con 0x7fc0b015a1a0 2023-10-30T12:34:41.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b74c7640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b00c0f70 con 0x7fc0b006d420 2023-10-30T12:34:41.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b74c7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b00c1250 con 0x7fc0b01539e0 2023-10-30T12:34:41.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b5263640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b01539e0 0x7fc0b00c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b5a64640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0b006d420 0x7fc0b01640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b5263640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b01539e0 0x7fc0b00c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:40694/0 (socket says 172.21.15.154:40694) 2023-10-30T12:34:41.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b5263640 1 -- 172.21.15.154:0/1473506538 learned_addr learned my addr 172.21.15.154:0/1473506538 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-30T12:34:41.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b5263640 1 -- 172.21.15.154:0/1473506538 >> v2:172.21.15.154:3300/0 conn(0x7fc0b006d420 msgr2=0x7fc0b01640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:41.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b5263640 1 --2- 172.21.15.154:0/1473506538 >> v2:172.21.15.154:3300/0 conn(0x7fc0b006d420 0x7fc0b01640a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b5263640 1 -- 172.21.15.154:0/1473506538 >> v1:172.21.15.12:6789/0 conn(0x7fc0b015a1a0 legacy=0x7fc0b00c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:41.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b5263640 1 -- 172.21.15.154:0/1473506538 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b00c1530 con 0x7fc0b01539e0 2023-10-30T12:34:41.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.720+0000 7fc0b5a64640 1 --2- 172.21.15.154:0/1473506538 >> v2:172.21.15.154:3300/0 conn(0x7fc0b006d420 0x7fc0b01640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:34:41.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.724+0000 7fc0b5263640 1 --2- 172.21.15.154:0/1473506538 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b01539e0 0x7fc0b00c9cb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009870 tx=0x7fc0ac01ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8576e8f66d97cf33 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:41.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.724+0000 7fc0a7fff640 1 -- 172.21.15.154:0/1473506538 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0a89b0 con 0x7fc0b01539e0 2023-10-30T12:34:41.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.724+0000 7fc0a7fff640 1 -- 172.21.15.154:0/1473506538 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ac0a8b10 con 0x7fc0b01539e0 2023-10-30T12:34:41.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.724+0000 7fc0a7fff640 1 -- 172.21.15.154:0/1473506538 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0a8ea0 con 0x7fc0b01539e0 2023-10-30T12:34:41.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.724+0000 7fc0b74c7640 1 -- 172.21.15.154:0/1473506538 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b01539e0 msgr2=0x7fc0b00c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.724+0000 7fc0b74c7640 1 --2- 172.21.15.154:0/1473506538 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b01539e0 0x7fc0b00c9cb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009870 tx=0x7fc0ac01ecf0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.724+0000 7fc0b74c7640 1 -- 172.21.15.154:0/1473506538 shutdown_connections 2023-10-30T12:34:41.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.724+0000 7fc0b74c7640 1 --2- 172.21.15.154:0/1473506538 >> v2:172.21.15.154:3300/0 conn(0x7fc0b006d420 0x7fc0b01640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.724+0000 7fc0b74c7640 1 --2- 172.21.15.154:0/1473506538 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b01539e0 0x7fc0b00c9cb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.724+0000 7fc0b74c7640 1 -- 172.21.15.154:0/1473506538 >> 172.21.15.154:0/1473506538 conn(0x7fc0b006c850 msgr2=0x7fc0b006cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:41.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.724+0000 7fc0b74c7640 1 -- 172.21.15.154:0/1473506538 shutdown_connections 2023-10-30T12:34:41.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.724+0000 7fc0b74c7640 1 -- 172.21.15.154:0/1473506538 wait complete. 2023-10-30T12:34:41.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b74c7640 1 Processor -- start 2023-10-30T12:34:41.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b74c7640 1 -- start start 2023-10-30T12:34:41.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b74c7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b006d420 0x7fc0b0155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b74c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0b01539e0 0x7fc0b015cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b74c7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0b015a1a0 0x7fc0b0150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b74c7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b00c58c0 con 0x7fc0b01539e0 2023-10-30T12:34:41.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b74c7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b00c5b30 con 0x7fc0b015a1a0 2023-10-30T12:34:41.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b74c7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b00c5e10 con 0x7fc0b006d420 2023-10-30T12:34:41.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b5a64640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0b015a1a0 0x7fc0b0150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b5263640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b006d420 0x7fc0b0155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b5a64640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0b015a1a0 0x7fc0b0150850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60840/0 (socket says 172.21.15.154:60840) 2023-10-30T12:34:41.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b5263640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b006d420 0x7fc0b0155ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:40706/0 (socket says 172.21.15.154:40706) 2023-10-30T12:34:41.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b5a64640 1 -- 172.21.15.154:0/1339925357 learned_addr learned my addr 172.21.15.154:0/1339925357 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-30T12:34:41.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b4a62640 1 --2- 172.21.15.154:0/1339925357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0b01539e0 0x7fc0b015cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b5263640 1 -- 172.21.15.154:0/1339925357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0b015a1a0 msgr2=0x7fc0b0150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b5263640 1 --2- 172.21.15.154:0/1339925357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0b015a1a0 0x7fc0b0150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b5263640 1 -- 172.21.15.154:0/1339925357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0b01539e0 msgr2=0x7fc0b015cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b5263640 1 --2- 172.21.15.154:0/1339925357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0b01539e0 0x7fc0b015cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b5263640 1 -- 172.21.15.154:0/1339925357 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a8009a70 con 0x7fc0b006d420 2023-10-30T12:34:41.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b5a64640 1 --2- 172.21.15.154:0/1339925357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0b015a1a0 0x7fc0b0150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:34:41.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b5263640 1 --2- 172.21.15.154:0/1339925357 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b006d420 0x7fc0b0155ef0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0af000 tx=0x7fc0ac01ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:41.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0a5ffb640 1 -- 172.21.15.154:0/1339925357 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0bf070 con 0x7fc0b006d420 2023-10-30T12:34:41.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0a5ffb640 1 -- 172.21.15.154:0/1339925357 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ac0b0b50 con 0x7fc0b006d420 2023-10-30T12:34:41.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b74c7640 1 -- 172.21.15.154:0/1339925357 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ac0094d0 con 0x7fc0b006d420 2023-10-30T12:34:41.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0a5ffb640 1 -- 172.21.15.154:0/1339925357 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0bb470 con 0x7fc0b006d420 2023-10-30T12:34:41.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b74c7640 1 -- 172.21.15.154:0/1339925357 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b00c63d0 con 0x7fc0b006d420 2023-10-30T12:34:41.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0a5ffb640 1 -- 172.21.15.154:0/1339925357 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc0ac0b7070 con 0x7fc0b006d420 2023-10-30T12:34:41.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0a5ffb640 1 -- 172.21.15.154:0/1339925357 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fc0ac0b9900 con 0x7fc0b006d420 2023-10-30T12:34:41.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.728+0000 7fc0b74c7640 1 -- 172.21.15.154:0/1339925357 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b004e170 con 0x7fc0b006d420 2023-10-30T12:34:41.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.732+0000 7fc0a5ffb640 1 -- 172.21.15.154:0/1339925357 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fc0ac0b5070 con 0x7fc0b006d420 2023-10-30T12:34:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.796+0000 7fc0b74c7640 1 -- 172.21.15.154:0/1339925357 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "739f6118-6ac3-4a60-9da5-1a0ee98f9518", "id": 3} v 0) v1 -- 0x7fc0b004e170 con 0x7fc0b006d420 2023-10-30T12:34:41.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.808+0000 7fc0a5ffb640 1 -- 172.21.15.154:0/1339925357 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "739f6118-6ac3-4a60-9da5-1a0ee98f9518", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc0ac0d20b0 con 0x7fc0b006d420 2023-10-30T12:34:41.809 INFO:teuthology.orchestra.run.smithi154.stdout:3 2023-10-30T12:34:41.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.808+0000 7fc0b74c7640 1 -- 172.21.15.154:0/1339925357 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b006d420 msgr2=0x7fc0b0155ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.808+0000 7fc0b74c7640 1 --2- 172.21.15.154:0/1339925357 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b006d420 0x7fc0b0155ef0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0af000 tx=0x7fc0ac01ba90 comp rx=0 tx=0).stop 2023-10-30T12:34:41.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.808+0000 7fc0b74c7640 1 -- 172.21.15.154:0/1339925357 shutdown_connections 2023-10-30T12:34:41.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.808+0000 7fc0b74c7640 1 --2- 172.21.15.154:0/1339925357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0b015a1a0 0x7fc0b0150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.808+0000 7fc0b74c7640 1 --2- 172.21.15.154:0/1339925357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0b01539e0 0x7fc0b015cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.808+0000 7fc0b74c7640 1 --2- 172.21.15.154:0/1339925357 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b006d420 0x7fc0b0155ef0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.808+0000 7fc0b74c7640 1 -- 172.21.15.154:0/1339925357 >> 172.21.15.154:0/1339925357 conn(0x7fc0b006c850 msgr2=0x7fc0b0070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:41.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.808+0000 7fc0b74c7640 1 -- 172.21.15.154:0/1339925357 shutdown_connections 2023-10-30T12:34:41.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.808+0000 7fc0b74c7640 1 -- 172.21.15.154:0/1339925357 wait complete. 2023-10-30T12:34:41.826 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 11ac0f0b-1ebc-41f9-afa0-90cbf86b73db 4 2023-10-30T12:34:41.882 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:41.880+0000 7f692bb33180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-30T12:34:41.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.936+0000 7fd0c38b3640 1 Processor -- start 2023-10-30T12:34:41.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.936+0000 7fd0c38b3640 1 -- start start 2023-10-30T12:34:41.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.936+0000 7fd0c38b3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd0bc15a1a0 0x7fd0bc0c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.936+0000 7fd0c38b3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0bc06d420 0x7fd0bc0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.936+0000 7fd0c38b3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0bc15a670 con 0x7fd0bc1539e0 2023-10-30T12:34:41.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.936+0000 7fd0c38b3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0bc0ca3c0 con 0x7fd0bc15a1a0 2023-10-30T12:34:41.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.936+0000 7fd0c38b3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc0c0f70 con 0x7fd0bc06d420 2023-10-30T12:34:41.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.936+0000 7fd0c1e50640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0bc06d420 0x7fd0bc0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.936+0000 7fd0c1e50640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0bc06d420 0x7fd0bc0c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:40712/0 (socket says 172.21.15.154:40712) 2023-10-30T12:34:41.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.936+0000 7fd0c1e50640 1 -- 172.21.15.154:0/1498171098 learned_addr learned my addr 172.21.15.154:0/1498171098 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-30T12:34:41.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.936+0000 7fd0c0e4e640 1 --2- 172.21.15.154:0/1498171098 >> v2:172.21.15.12:3300/0 conn(0x7fd0bc15a1a0 0x7fd0bc0c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.936+0000 7fd0c1e50640 1 -- 172.21.15.154:0/1498171098 >> v2:172.21.15.12:3300/0 conn(0x7fd0bc15a1a0 msgr2=0x7fd0bc0c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:41.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.936+0000 7fd0c1e50640 1 --2- 172.21.15.154:0/1498171098 >> v2:172.21.15.12:3300/0 conn(0x7fd0bc15a1a0 0x7fd0bc0c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.936+0000 7fd0c1e50640 1 -- 172.21.15.154:0/1498171098 >> v1:172.21.15.12:6789/0 conn(0x7fd0bc1539e0 legacy=0x7fd0bc0c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.936+0000 7fd0c1e50640 1 -- 172.21.15.154:0/1498171098 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc0c1250 con 0x7fd0bc06d420 2023-10-30T12:34:41.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.940+0000 7fd0c1e50640 1 --2- 172.21.15.154:0/1498171098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0bc06d420 0x7fd0bc0c7e30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009e50 tx=0x7fd0b401f1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2eea22735f88f2f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:41.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.940+0000 7fd0b3fff640 1 -- 172.21.15.154:0/1498171098 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b40af400 con 0x7fd0bc06d420 2023-10-30T12:34:41.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.940+0000 7fd0b3fff640 1 -- 172.21.15.154:0/1498171098 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b40ab050 con 0x7fd0bc06d420 2023-10-30T12:34:41.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.940+0000 7fd0b3fff640 1 -- 172.21.15.154:0/1498171098 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b40af780 con 0x7fd0bc06d420 2023-10-30T12:34:41.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.940+0000 7fd0c38b3640 1 -- 172.21.15.154:0/1498171098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0bc06d420 msgr2=0x7fd0bc0c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.940+0000 7fd0c38b3640 1 --2- 172.21.15.154:0/1498171098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0bc06d420 0x7fd0bc0c7e30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009e50 tx=0x7fd0b401f1f0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.940+0000 7fd0c38b3640 1 -- 172.21.15.154:0/1498171098 shutdown_connections 2023-10-30T12:34:41.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.940+0000 7fd0c38b3640 1 --2- 172.21.15.154:0/1498171098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0bc06d420 0x7fd0bc0c7e30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.940+0000 7fd0c38b3640 1 --2- 172.21.15.154:0/1498171098 >> v2:172.21.15.12:3300/0 conn(0x7fd0bc15a1a0 0x7fd0bc0c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.940+0000 7fd0c38b3640 1 -- 172.21.15.154:0/1498171098 >> 172.21.15.154:0/1498171098 conn(0x7fd0bc06c850 msgr2=0x7fd0bc06cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:41.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.940+0000 7fd0c38b3640 1 -- 172.21.15.154:0/1498171098 shutdown_connections 2023-10-30T12:34:41.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.940+0000 7fd0c38b3640 1 -- 172.21.15.154:0/1498171098 wait complete. 2023-10-30T12:34:41.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.940+0000 7fd0c38b3640 1 Processor -- start 2023-10-30T12:34:41.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.940+0000 7fd0c38b3640 1 -- start start 2023-10-30T12:34:41.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c38b3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0bc06d420 0x7fd0bc0c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c38b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0bc1539e0 0x7fd0bc15b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c38b3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0bc15a1a0 0x7fd0bc0ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:41.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c38b3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc1f21e0 con 0x7fd0bc06d420 2023-10-30T12:34:41.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c38b3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc15c2f0 con 0x7fd0bc1539e0 2023-10-30T12:34:41.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c38b3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc15bdf0 con 0x7fd0bc15a1a0 2023-10-30T12:34:41.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c0e4e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0bc1539e0 0x7fd0bc15b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c1e50640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0bc15a1a0 0x7fd0bc0ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c0e4e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0bc1539e0 0x7fd0bc15b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60854/0 (socket says 172.21.15.154:60854) 2023-10-30T12:34:41.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c1e50640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0bc15a1a0 0x7fd0bc0ca860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:40714/0 (socket says 172.21.15.154:40714) 2023-10-30T12:34:41.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c0e4e640 1 -- 172.21.15.154:0/1399136022 learned_addr learned my addr 172.21.15.154:0/1399136022 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-30T12:34:41.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c164f640 1 --2- 172.21.15.154:0/1399136022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0bc06d420 0x7fd0bc0c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:41.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c0e4e640 1 -- 172.21.15.154:0/1399136022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0bc15a1a0 msgr2=0x7fd0bc0ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c0e4e640 1 --2- 172.21.15.154:0/1399136022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0bc15a1a0 0x7fd0bc0ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c0e4e640 1 -- 172.21.15.154:0/1399136022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0bc06d420 msgr2=0x7fd0bc0c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:41.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c0e4e640 1 --2- 172.21.15.154:0/1399136022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0bc06d420 0x7fd0bc0c6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:41.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c0e4e640 1 -- 172.21.15.154:0/1399136022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b4009b30 con 0x7fd0bc1539e0 2023-10-30T12:34:41.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c1e50640 1 --2- 172.21.15.154:0/1399136022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0bc15a1a0 0x7fd0bc0ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:34:41.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c0e4e640 1 --2- 172.21.15.154:0/1399136022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0bc1539e0 0x7fd0bc15b8b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00d7d0 tx=0x7fd0ac00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:41.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0b1ffb640 1 -- 172.21.15.154:0/1399136022 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac00b6c0 con 0x7fd0bc1539e0 2023-10-30T12:34:41.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0b1ffb640 1 -- 172.21.15.154:0/1399136022 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac00b820 con 0x7fd0bc1539e0 2023-10-30T12:34:41.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0b1ffb640 1 -- 172.21.15.154:0/1399136022 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac00bbb0 con 0x7fd0bc1539e0 2023-10-30T12:34:41.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c38b3640 1 -- 172.21.15.154:0/1399136022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0bc1f2d00 con 0x7fd0bc1539e0 2023-10-30T12:34:41.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c38b3640 1 -- 172.21.15.154:0/1399136022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0bc15c010 con 0x7fd0bc1539e0 2023-10-30T12:34:41.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0b1ffb640 1 -- 172.21.15.154:0/1399136022 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd0ac01e760 con 0x7fd0bc1539e0 2023-10-30T12:34:41.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0b1ffb640 1 -- 172.21.15.154:0/1399136022 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fd0ac014a30 con 0x7fd0bc1539e0 2023-10-30T12:34:41.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.944+0000 7fd0c38b3640 1 -- 172.21.15.154:0/1399136022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0bc04e170 con 0x7fd0bc1539e0 2023-10-30T12:34:41.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:41.948+0000 7fd0b1ffb640 1 -- 172.21.15.154:0/1399136022 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fd0ac014c90 con 0x7fd0bc1539e0 2023-10-30T12:34:41.975 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:41.972+0000 7f692bb33180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-30T12:34:42.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.012+0000 7fd0c38b3640 1 -- 172.21.15.154:0/1399136022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "11ac0f0b-1ebc-41f9-afa0-90cbf86b73db", "id": 4} v 0) v1 -- 0x7fd0bc073e10 con 0x7fd0bc1539e0 2023-10-30T12:34:42.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.020+0000 7fd0b1ffb640 1 -- 172.21.15.154:0/1399136022 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "11ac0f0b-1ebc-41f9-afa0-90cbf86b73db", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd0ac02d0b0 con 0x7fd0bc1539e0 2023-10-30T12:34:42.023 INFO:teuthology.orchestra.run.smithi154.stdout:4 2023-10-30T12:34:42.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.024+0000 7fd0c38b3640 1 -- 172.21.15.154:0/1399136022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0bc1539e0 msgr2=0x7fd0bc15b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.024+0000 7fd0c38b3640 1 --2- 172.21.15.154:0/1399136022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0bc1539e0 0x7fd0bc15b8b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00d7d0 tx=0x7fd0ac00dc90 comp rx=0 tx=0).stop 2023-10-30T12:34:42.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.024+0000 7fd0c38b3640 1 -- 172.21.15.154:0/1399136022 shutdown_connections 2023-10-30T12:34:42.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.024+0000 7fd0c38b3640 1 --2- 172.21.15.154:0/1399136022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0bc15a1a0 0x7fd0bc0ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.024+0000 7fd0c38b3640 1 --2- 172.21.15.154:0/1399136022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0bc1539e0 0x7fd0bc15b8b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.024+0000 7fd0c38b3640 1 --2- 172.21.15.154:0/1399136022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0bc06d420 0x7fd0bc0c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.024+0000 7fd0c38b3640 1 -- 172.21.15.154:0/1399136022 >> 172.21.15.154:0/1399136022 conn(0x7fd0bc06c850 msgr2=0x7fd0bc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:42.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.024+0000 7fd0c38b3640 1 -- 172.21.15.154:0/1399136022 shutdown_connections 2023-10-30T12:34:42.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.024+0000 7fd0c38b3640 1 -- 172.21.15.154:0/1399136022 wait complete. 2023-10-30T12:34:42.029 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:42.028+0000 7f692bb33180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-30T12:34:42.040 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 310a87c6-f25b-4349-a5f3-8130e2a53210 5 2023-10-30T12:34:42.086 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:42.083+0000 7f3d8f37e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-30T12:34:42.110 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:42.108+0000 7f692bb33180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-30T12:34:42.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd3db11640 1 Processor -- start 2023-10-30T12:34:42.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd3db11640 1 -- start start 2023-10-30T12:34:42.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd3db11640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd380c2e20 0x7fcd380c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd3db11640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcd3815a070 0x7fcd3815a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd3db11640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd38159be0 con 0x7fcd38150800 2023-10-30T12:34:42.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd3db11640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd380c7020 con 0x7fcd3815a070 2023-10-30T12:34:42.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd3db11640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3815aa40 con 0x7fcd380c2e20 2023-10-30T12:34:42.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd2ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd380c2e20 0x7fcd380c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd2ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd380c2e20 0x7fcd380c6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60864/0 (socket says 172.21.15.154:60864) 2023-10-30T12:34:42.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd2ffff640 1 -- 172.21.15.154:0/951395169 learned_addr learned my addr 172.21.15.154:0/951395169 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-30T12:34:42.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd37fff640 1 --2- 172.21.15.154:0/951395169 >> v2:172.21.15.12:3300/0 conn(0x7fcd3815a070 0x7fcd3815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd2ffff640 1 -- 172.21.15.154:0/951395169 >> v2:172.21.15.12:3300/0 conn(0x7fcd3815a070 msgr2=0x7fcd3815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:42.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd2ffff640 1 --2- 172.21.15.154:0/951395169 >> v2:172.21.15.12:3300/0 conn(0x7fcd3815a070 0x7fcd3815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd2ffff640 1 -- 172.21.15.154:0/951395169 >> v1:172.21.15.154:6789/0 conn(0x7fcd38150800 legacy=0x7fcd380cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd2ffff640 1 -- 172.21.15.154:0/951395169 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd38156850 con 0x7fcd380c2e20 2023-10-30T12:34:42.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.148+0000 7fcd37fff640 1 --2- 172.21.15.154:0/951395169 >> v2:172.21.15.12:3300/0 conn(0x7fcd3815a070 0x7fcd3815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:34:42.154 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:42.147+0000 7f3d8f37e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-30T12:34:42.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.152+0000 7fcd2ffff640 1 --2- 172.21.15.154:0/951395169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd380c2e20 0x7fcd380c6ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcd380c9a70 tx=0x7fcd2001ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ccfeedbbb584db server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:42.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.152+0000 7fcd36ffd640 1 -- 172.21.15.154:0/951395169 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd200a89f0 con 0x7fcd380c2e20 2023-10-30T12:34:42.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.152+0000 7fcd36ffd640 1 -- 172.21.15.154:0/951395169 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd200a8b50 con 0x7fcd380c2e20 2023-10-30T12:34:42.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.152+0000 7fcd36ffd640 1 -- 172.21.15.154:0/951395169 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd200b0440 con 0x7fcd380c2e20 2023-10-30T12:34:42.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.152+0000 7fcd3db11640 1 -- 172.21.15.154:0/951395169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd380c2e20 msgr2=0x7fcd380c6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.152+0000 7fcd3db11640 1 --2- 172.21.15.154:0/951395169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd380c2e20 0x7fcd380c6ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcd380c9a70 tx=0x7fcd2001ecf0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.152+0000 7fcd3db11640 1 -- 172.21.15.154:0/951395169 shutdown_connections 2023-10-30T12:34:42.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.152+0000 7fcd3db11640 1 --2- 172.21.15.154:0/951395169 >> v2:172.21.15.12:3300/0 conn(0x7fcd3815a070 0x7fcd3815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.152+0000 7fcd3db11640 1 --2- 172.21.15.154:0/951395169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd380c2e20 0x7fcd380c6ae0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.152+0000 7fcd3db11640 1 -- 172.21.15.154:0/951395169 >> 172.21.15.154:0/951395169 conn(0x7fcd3806c850 msgr2=0x7fcd3806cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:42.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.152+0000 7fcd3db11640 1 -- 172.21.15.154:0/951395169 shutdown_connections 2023-10-30T12:34:42.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.152+0000 7fcd3db11640 1 -- 172.21.15.154:0/951395169 wait complete. 2023-10-30T12:34:42.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.152+0000 7fcd3db11640 1 Processor -- start 2023-10-30T12:34:42.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.152+0000 7fcd3db11640 1 -- start start 2023-10-30T12:34:42.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd3db11640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd380c2e20 0x7fcd38158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd3db11640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd38150800 0x7fcd381ed9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd3db11640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd3815a070 0x7fcd38145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd3db11640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd38147bd0 con 0x7fcd38150800 2023-10-30T12:34:42.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd3db11640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3814c190 con 0x7fcd3815a070 2023-10-30T12:34:42.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd3db11640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd381ed290 con 0x7fcd380c2e20 2023-10-30T12:34:42.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd37fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd3815a070 0x7fcd38145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd377fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd380c2e20 0x7fcd38158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd37fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd3815a070 0x7fcd38145eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60872/0 (socket says 172.21.15.154:60872) 2023-10-30T12:34:42.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd37fff640 1 -- 172.21.15.154:0/1349840748 learned_addr learned my addr 172.21.15.154:0/1349840748 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-30T12:34:42.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd377fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd380c2e20 0x7fcd38158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:40726/0 (socket says 172.21.15.154:40726) 2023-10-30T12:34:42.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd2ffff640 1 --2- 172.21.15.154:0/1349840748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd38150800 0x7fcd381ed9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd37fff640 1 -- 172.21.15.154:0/1349840748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd380c2e20 msgr2=0x7fcd38158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd37fff640 1 --2- 172.21.15.154:0/1349840748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd380c2e20 0x7fcd38158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd37fff640 1 -- 172.21.15.154:0/1349840748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd38150800 msgr2=0x7fcd381ed9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd37fff640 1 --2- 172.21.15.154:0/1349840748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd38150800 0x7fcd381ed9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd37fff640 1 -- 172.21.15.154:0/1349840748 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd200094d0 con 0x7fcd3815a070 2023-10-30T12:34:42.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd377fe640 1 --2- 172.21.15.154:0/1349840748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd380c2e20 0x7fcd38158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:34:42.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd37fff640 1 --2- 172.21.15.154:0/1349840748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd3815a070 0x7fcd38145eb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000cd50 tx=0x7fcd3000a650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:42.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd34ff9640 1 -- 172.21.15.154:0/1349840748 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3000f680 con 0x7fcd3815a070 2023-10-30T12:34:42.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd34ff9640 1 -- 172.21.15.154:0/1349840748 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3000f800 con 0x7fcd3815a070 2023-10-30T12:34:42.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd3db11640 1 -- 172.21.15.154:0/1349840748 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd380c59d0 con 0x7fcd3815a070 2023-10-30T12:34:42.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd34ff9640 1 -- 172.21.15.154:0/1349840748 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3000fbb0 con 0x7fcd3815a070 2023-10-30T12:34:42.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd3db11640 1 -- 172.21.15.154:0/1349840748 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3814c6a0 con 0x7fcd3815a070 2023-10-30T12:34:42.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd3db11640 1 -- 172.21.15.154:0/1349840748 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3804e170 con 0x7fcd3815a070 2023-10-30T12:34:42.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd34ff9640 1 -- 172.21.15.154:0/1349840748 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcd3000b040 con 0x7fcd3815a070 2023-10-30T12:34:42.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.156+0000 7fcd34ff9640 1 -- 172.21.15.154:0/1349840748 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fcd3000fd60 con 0x7fcd3815a070 2023-10-30T12:34:42.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.160+0000 7fcd34ff9640 1 -- 172.21.15.154:0/1349840748 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fcd300213f0 con 0x7fcd3815a070 2023-10-30T12:34:42.166 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:42.164+0000 7f692bb33180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-30T12:34:42.214 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:42.211+0000 7f3d8f37e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-30T12:34:42.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.220+0000 7fcd3db11640 1 -- 172.21.15.154:0/1349840748 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "310a87c6-f25b-4349-a5f3-8130e2a53210", "id": 5} v 0) v1 -- 0x7fcd380c60c0 con 0x7fcd3815a070 2023-10-30T12:34:42.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.228+0000 7fcd34ff9640 1 -- 172.21.15.154:0/1349840748 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "310a87c6-f25b-4349-a5f3-8130e2a53210", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcd30018a60 con 0x7fcd3815a070 2023-10-30T12:34:42.230 INFO:teuthology.orchestra.run.smithi154.stdout:5 2023-10-30T12:34:42.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.232+0000 7fcd3db11640 1 -- 172.21.15.154:0/1349840748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd3815a070 msgr2=0x7fcd38145eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.232+0000 7fcd3db11640 1 --2- 172.21.15.154:0/1349840748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd3815a070 0x7fcd38145eb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000cd50 tx=0x7fcd3000a650 comp rx=0 tx=0).stop 2023-10-30T12:34:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.232+0000 7fcd3db11640 1 -- 172.21.15.154:0/1349840748 shutdown_connections 2023-10-30T12:34:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.232+0000 7fcd3db11640 1 --2- 172.21.15.154:0/1349840748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd3815a070 0x7fcd38145eb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.232+0000 7fcd3db11640 1 --2- 172.21.15.154:0/1349840748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd38150800 0x7fcd381ed9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.232+0000 7fcd3db11640 1 --2- 172.21.15.154:0/1349840748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd380c2e20 0x7fcd38158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.232+0000 7fcd3db11640 1 -- 172.21.15.154:0/1349840748 >> 172.21.15.154:0/1349840748 conn(0x7fcd3806c850 msgr2=0x7fcd380c7170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:42.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.232+0000 7fcd3db11640 1 -- 172.21.15.154:0/1349840748 shutdown_connections 2023-10-30T12:34:42.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.232+0000 7fcd3db11640 1 -- 172.21.15.154:0/1349840748 wait complete. 2023-10-30T12:34:42.250 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new ff1a4b95-1e38-47f5-bcd7-fb53346c63dc 6 2023-10-30T12:34:42.251 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:42.252+0000 7f692bb33180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-30T12:34:42.306 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:42.303+0000 7f3d8f37e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-30T12:34:42.308 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:42.308+0000 7f692bb33180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-30T12:34:42.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.356+0000 7f20c5800640 1 Processor -- start 2023-10-30T12:34:42.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.356+0000 7f20c5800640 1 -- start start 2023-10-30T12:34:42.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.356+0000 7f20c5800640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f20c0153980 0x7f20c00caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.356+0000 7f20c5800640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c015a920 con 0x7f20c00cb1e0 2023-10-30T12:34:42.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.356+0000 7f20c5800640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c00cb930 con 0x7f20c0159930 2023-10-30T12:34:42.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.356+0000 7f20c5800640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c0159dc0 con 0x7f20c0153980 2023-10-30T12:34:42.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.356+0000 7f20beffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f20c0153980 0x7f20c00caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.356+0000 7f20be7fc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f20c0159930 0x7f20c00cf1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55200/0 (socket says 172.21.15.154:55200) 2023-10-30T12:34:42.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.356+0000 7f20be7fc640 1 -- 172.21.15.154:0/944733864 learned_addr learned my addr 172.21.15.154:0/944733864 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-10-30T12:34:42.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.356+0000 7f20beffd640 1 -- 172.21.15.154:0/944733864 >> v1:172.21.15.154:6789/0 conn(0x7f20c0159930 legacy=0x7f20c00cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.356+0000 7f20beffd640 1 -- 172.21.15.154:0/944733864 >> v1:172.21.15.12:6789/0 conn(0x7f20c00cb1e0 legacy=0x7f20c015d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.356+0000 7f20beffd640 1 -- 172.21.15.154:0/944733864 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c015a080 con 0x7f20c0153980 2023-10-30T12:34:42.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.360+0000 7f20beffd640 1 --2- 172.21.15.154:0/944733864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20c0153980 0x7f20c00caca0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f20b00097c0 tx=0x7f20b001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f588f7e410745d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:42.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.360+0000 7f20bdffb640 1 -- 172.21.15.154:0/944733864 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b00ad660 con 0x7f20c0153980 2023-10-30T12:34:42.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.360+0000 7f20bdffb640 1 -- 172.21.15.154:0/944733864 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b00ad7c0 con 0x7f20c0153980 2023-10-30T12:34:42.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.360+0000 7f20bdffb640 1 -- 172.21.15.154:0/944733864 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b00adb50 con 0x7f20c0153980 2023-10-30T12:34:42.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.360+0000 7f20c5800640 1 -- 172.21.15.154:0/944733864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20c0153980 msgr2=0x7f20c00caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.360+0000 7f20c5800640 1 --2- 172.21.15.154:0/944733864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20c0153980 0x7f20c00caca0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f20b00097c0 tx=0x7f20b001c450 comp rx=0 tx=0).stop 2023-10-30T12:34:42.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.360+0000 7f20c5800640 1 -- 172.21.15.154:0/944733864 shutdown_connections 2023-10-30T12:34:42.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.360+0000 7f20c5800640 1 --2- 172.21.15.154:0/944733864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20c0153980 0x7f20c00caca0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.360+0000 7f20c5800640 1 -- 172.21.15.154:0/944733864 >> 172.21.15.154:0/944733864 conn(0x7f20c006c0f0 msgr2=0x7f20c006c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:42.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.360+0000 7f20c5800640 1 -- 172.21.15.154:0/944733864 shutdown_connections 2023-10-30T12:34:42.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.360+0000 7f20c5800640 1 -- 172.21.15.154:0/944733864 wait complete. 2023-10-30T12:34:42.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20c5800640 1 Processor -- start 2023-10-30T12:34:42.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20c5800640 1 -- start start 2023-10-30T12:34:42.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20c5800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20c00cb1e0 0x7f20c0158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20c5800640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20c0153980 0x7f20c00ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20c5800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20c00b3100 0x7f20c00ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20c5800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00b4d50 con 0x7f20c00b3100 2023-10-30T12:34:42.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20c5800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00b5030 con 0x7f20c00cb1e0 2023-10-30T12:34:42.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20c5800640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c015b160 con 0x7f20c0153980 2023-10-30T12:34:42.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20beffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20c00cb1e0 0x7f20c0158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20beffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20c00cb1e0 0x7f20c0158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60890/0 (socket says 172.21.15.154:60890) 2023-10-30T12:34:42.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20be7fc640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20c0153980 0x7f20c00ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20beffd640 1 -- 172.21.15.154:0/3816475927 learned_addr learned my addr 172.21.15.154:0/3816475927 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-30T12:34:42.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20bf7fe640 1 --2- 172.21.15.154:0/3816475927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20c00b3100 0x7f20c00ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20beffd640 1 -- 172.21.15.154:0/3816475927 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20c0153980 msgr2=0x7f20c00ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20beffd640 1 --2- 172.21.15.154:0/3816475927 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20c0153980 0x7f20c00ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20beffd640 1 -- 172.21.15.154:0/3816475927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20c00b3100 msgr2=0x7f20c00ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20beffd640 1 --2- 172.21.15.154:0/3816475927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20c00b3100 0x7f20c00ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20beffd640 1 -- 172.21.15.154:0/3816475927 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b0009420 con 0x7f20c00cb1e0 2023-10-30T12:34:42.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20beffd640 1 --2- 172.21.15.154:0/3816475927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20c00cb1e0 0x7f20c0158d40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f20b00b0040 tx=0x7f20b001ac90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:42.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20a77fe640 1 -- 172.21.15.154:0/3816475927 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b00b9860 con 0x7f20c00cb1e0 2023-10-30T12:34:42.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20a77fe640 1 -- 172.21.15.154:0/3816475927 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b00b99e0 con 0x7f20c00cb1e0 2023-10-30T12:34:42.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20c5800640 1 -- 172.21.15.154:0/3816475927 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c015b440 con 0x7f20c00cb1e0 2023-10-30T12:34:42.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20a77fe640 1 -- 172.21.15.154:0/3816475927 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b00b9d90 con 0x7f20c00cb1e0 2023-10-30T12:34:42.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20c5800640 1 -- 172.21.15.154:0/3816475927 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c015cc80 con 0x7f20c00cb1e0 2023-10-30T12:34:42.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20a77fe640 1 -- 172.21.15.154:0/3816475927 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f20b00c4760 con 0x7f20c00cb1e0 2023-10-30T12:34:42.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20a77fe640 1 -- 172.21.15.154:0/3816475927 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f20b00b79f0 con 0x7f20c00cb1e0 2023-10-30T12:34:42.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.364+0000 7f20c5800640 1 -- 172.21.15.154:0/3816475927 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c004e170 con 0x7f20c00cb1e0 2023-10-30T12:34:42.372 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:42.367+0000 7f3d8f37e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-30T12:34:42.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.368+0000 7f20a77fe640 1 -- 172.21.15.154:0/3816475927 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f20b00b2070 con 0x7f20c00cb1e0 2023-10-30T12:34:42.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.436+0000 7f20c5800640 1 -- 172.21.15.154:0/3816475927 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ff1a4b95-1e38-47f5-bcd7-fb53346c63dc", "id": 6} v 0) v1 -- 0x7f20c0157580 con 0x7f20c00cb1e0 2023-10-30T12:34:42.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.444+0000 7f20a77fe640 1 -- 172.21.15.154:0/3816475927 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ff1a4b95-1e38-47f5-bcd7-fb53346c63dc", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f20b00b7c40 con 0x7f20c00cb1e0 2023-10-30T12:34:42.446 INFO:teuthology.orchestra.run.smithi154.stdout:6 2023-10-30T12:34:42.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.444+0000 7f20c5800640 1 -- 172.21.15.154:0/3816475927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20c00cb1e0 msgr2=0x7f20c0158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.444+0000 7f20c5800640 1 --2- 172.21.15.154:0/3816475927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20c00cb1e0 0x7f20c0158d40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f20b00b0040 tx=0x7f20b001ac90 comp rx=0 tx=0).stop 2023-10-30T12:34:42.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.444+0000 7f20c5800640 1 -- 172.21.15.154:0/3816475927 shutdown_connections 2023-10-30T12:34:42.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.444+0000 7f20c5800640 1 --2- 172.21.15.154:0/3816475927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20c00b3100 0x7f20c00ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.444+0000 7f20c5800640 1 --2- 172.21.15.154:0/3816475927 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20c0153980 0x7f20c00ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.444+0000 7f20c5800640 1 --2- 172.21.15.154:0/3816475927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20c00cb1e0 0x7f20c0158d40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.444+0000 7f20c5800640 1 -- 172.21.15.154:0/3816475927 >> 172.21.15.154:0/3816475927 conn(0x7f20c006c0f0 msgr2=0x7f20c0158100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:42.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.444+0000 7f20c5800640 1 -- 172.21.15.154:0/3816475927 shutdown_connections 2023-10-30T12:34:42.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.444+0000 7f20c5800640 1 -- 172.21.15.154:0/3816475927 wait complete. 2023-10-30T12:34:42.462 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new e6cbc67e-29b3-48a6-ba42-310011cafb14 7 2023-10-30T12:34:42.463 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:42.460+0000 7f692bb33180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-30T12:34:42.464 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:42.459+0000 7f3d8f37e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-30T12:34:42.524 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:42.524+0000 7f692bb33180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-30T12:34:42.525 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:42.523+0000 7f3d8f37e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-30T12:34:42.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2ad265640 1 Processor -- start 2023-10-30T12:34:42.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2ad265640 1 -- start start 2023-10-30T12:34:42.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2ad265640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc2a8150810 0x7fc2a806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2ad265640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc2a80c4080 0x7fc2a815a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2ad265640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a8070580 con 0x7fc2a80c7110 2023-10-30T12:34:42.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2ad265640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a80c37d0 con 0x7fc2a8150810 2023-10-30T12:34:42.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2ad265640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a8150bf0 con 0x7fc2a80c4080 2023-10-30T12:34:42.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2a67fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc2a80c4080 0x7fc2a815a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2a67fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc2a80c4080 0x7fc2a815a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60892/0 (socket says 172.21.15.154:60892) 2023-10-30T12:34:42.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2a67fc640 1 -- 172.21.15.154:0/1327410513 learned_addr learned my addr 172.21.15.154:0/1327410513 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-30T12:34:42.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2a6ffd640 1 --2- 172.21.15.154:0/1327410513 >> v2:172.21.15.12:3300/0 conn(0x7fc2a8150810 0x7fc2a806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2a67fc640 1 -- 172.21.15.154:0/1327410513 >> v2:172.21.15.12:3300/0 conn(0x7fc2a8150810 msgr2=0x7fc2a806fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:42.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2a67fc640 1 --2- 172.21.15.154:0/1327410513 >> v2:172.21.15.12:3300/0 conn(0x7fc2a8150810 0x7fc2a806fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2a67fc640 1 -- 172.21.15.154:0/1327410513 >> v1:172.21.15.12:6789/0 conn(0x7fc2a80c7110 legacy=0x7fc2a80c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:42.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2a67fc640 1 -- 172.21.15.154:0/1327410513 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a80c6070 con 0x7fc2a80c4080 2023-10-30T12:34:42.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2a6ffd640 1 --2- 172.21.15.154:0/1327410513 >> v2:172.21.15.12:3300/0 conn(0x7fc2a8150810 0x7fc2a806fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:34:42.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.580+0000 7fc2a67fc640 1 --2- 172.21.15.154:0/1327410513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a80c4080 0x7fc2a815a3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8066080 tx=0x7fc29c01c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a74f3f516a51bade server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:42.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2a5ffb640 1 -- 172.21.15.154:0/1327410513 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0ac410 con 0x7fc2a80c4080 2023-10-30T12:34:42.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2a5ffb640 1 -- 172.21.15.154:0/1327410513 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29c0ac570 con 0x7fc2a80c4080 2023-10-30T12:34:42.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2a5ffb640 1 -- 172.21.15.154:0/1327410513 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0ac900 con 0x7fc2a80c4080 2023-10-30T12:34:42.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 -- 172.21.15.154:0/1327410513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a80c4080 msgr2=0x7fc2a815a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 --2- 172.21.15.154:0/1327410513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a80c4080 0x7fc2a815a3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8066080 tx=0x7fc29c01c4c0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 -- 172.21.15.154:0/1327410513 shutdown_connections 2023-10-30T12:34:42.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 --2- 172.21.15.154:0/1327410513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a80c4080 0x7fc2a815a3c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 --2- 172.21.15.154:0/1327410513 >> v2:172.21.15.12:3300/0 conn(0x7fc2a8150810 0x7fc2a806fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 -- 172.21.15.154:0/1327410513 >> 172.21.15.154:0/1327410513 conn(0x7fc2a806c620 msgr2=0x7fc2a8069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:42.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 -- 172.21.15.154:0/1327410513 shutdown_connections 2023-10-30T12:34:42.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 -- 172.21.15.154:0/1327410513 wait complete. 2023-10-30T12:34:42.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 Processor -- start 2023-10-30T12:34:42.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 -- start start 2023-10-30T12:34:42.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2a80c4080 0x7fc2a8164d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2a80c7110 0x7fc2a8147720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a8150810 0x7fc2a8147c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81481a0 con 0x7fc2a80c7110 2023-10-30T12:34:42.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81f26b0 con 0x7fc2a8150810 2023-10-30T12:34:42.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81f2130 con 0x7fc2a80c4080 2023-10-30T12:34:42.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2a77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a8150810 0x7fc2a8147c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2a6ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2a80c4080 0x7fc2a8164d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2a77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a8150810 0x7fc2a8147c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60896/0 (socket says 172.21.15.154:60896) 2023-10-30T12:34:42.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2a77fe640 1 -- 172.21.15.154:0/1823704747 learned_addr learned my addr 172.21.15.154:0/1823704747 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-30T12:34:42.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2a67fc640 1 --2- 172.21.15.154:0/1823704747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2a80c7110 0x7fc2a8147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2a77fe640 1 -- 172.21.15.154:0/1823704747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2a80c4080 msgr2=0x7fc2a8164d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2a77fe640 1 --2- 172.21.15.154:0/1823704747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2a80c4080 0x7fc2a8164d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2a77fe640 1 -- 172.21.15.154:0/1823704747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2a80c7110 msgr2=0x7fc2a8147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2a77fe640 1 --2- 172.21.15.154:0/1823704747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2a80c7110 0x7fc2a8147720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2a77fe640 1 -- 172.21.15.154:0/1823704747 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c0094d0 con 0x7fc2a8150810 2023-10-30T12:34:42.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2a77fe640 1 --2- 172.21.15.154:0/1823704747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a8150810 0x7fc2a8147c60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c3c0 tx=0x7fc2a000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:42.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc28f7fe640 1 -- 172.21.15.154:0/1823704747 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0011790 con 0x7fc2a8150810 2023-10-30T12:34:42.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc28f7fe640 1 -- 172.21.15.154:0/1823704747 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a00118f0 con 0x7fc2a8150810 2023-10-30T12:34:42.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 -- 172.21.15.154:0/1823704747 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a814a6e0 con 0x7fc2a8150810 2023-10-30T12:34:42.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc28f7fe640 1 -- 172.21.15.154:0/1823704747 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0011ca0 con 0x7fc2a8150810 2023-10-30T12:34:42.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc2ad265640 1 -- 172.21.15.154:0/1823704747 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a8146f20 con 0x7fc2a8150810 2023-10-30T12:34:42.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc28f7fe640 1 -- 172.21.15.154:0/1823704747 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc2a001c760 con 0x7fc2a8150810 2023-10-30T12:34:42.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.584+0000 7fc28f7fe640 1 -- 172.21.15.154:0/1823704747 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fc2a0012d00 con 0x7fc2a8150810 2023-10-30T12:34:42.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.588+0000 7fc2ad265640 1 -- 172.21.15.154:0/1823704747 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a8146d80 con 0x7fc2a8150810 2023-10-30T12:34:42.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.592+0000 7fc28f7fe640 1 -- 172.21.15.154:0/1823704747 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fc2a8146d80 con 0x7fc2a8150810 2023-10-30T12:34:42.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.652+0000 7fc2ad265640 1 -- 172.21.15.154:0/1823704747 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e6cbc67e-29b3-48a6-ba42-310011cafb14", "id": 7} v 0) v1 -- 0x7fc2a80c68c0 con 0x7fc2a8150810 2023-10-30T12:34:42.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.660+0000 7fc28f7fe640 1 -- 172.21.15.154:0/1823704747 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e6cbc67e-29b3-48a6-ba42-310011cafb14", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc2a0011e00 con 0x7fc2a8150810 2023-10-30T12:34:42.660 INFO:teuthology.orchestra.run.smithi154.stdout:7 2023-10-30T12:34:42.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.660+0000 7fc2ad265640 1 -- 172.21.15.154:0/1823704747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a8150810 msgr2=0x7fc2a8147c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.660+0000 7fc2ad265640 1 --2- 172.21.15.154:0/1823704747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a8150810 0x7fc2a8147c60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c3c0 tx=0x7fc2a000c880 comp rx=0 tx=0).stop 2023-10-30T12:34:42.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.660+0000 7fc2ad265640 1 -- 172.21.15.154:0/1823704747 shutdown_connections 2023-10-30T12:34:42.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.660+0000 7fc2ad265640 1 --2- 172.21.15.154:0/1823704747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a8150810 0x7fc2a8147c60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.660+0000 7fc2ad265640 1 --2- 172.21.15.154:0/1823704747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2a80c7110 0x7fc2a8147720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.660+0000 7fc2ad265640 1 --2- 172.21.15.154:0/1823704747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2a80c4080 0x7fc2a8164d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.660+0000 7fc2ad265640 1 -- 172.21.15.154:0/1823704747 >> 172.21.15.154:0/1823704747 conn(0x7fc2a806c620 msgr2=0x7fc2a80c5110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.660+0000 7fc2ad265640 1 -- 172.21.15.154:0/1823704747 shutdown_connections 2023-10-30T12:34:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-30T12:34:42.660+0000 7fc2ad265640 1 -- 172.21.15.154:0/1823704747 wait complete. 2023-10-30T12:34:42.675 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-30T12:34:42.675 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-30T12:34:42.679 INFO:tasks.ceph.osd.0:Started 2023-10-30T12:34:42.679 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-30T12:34:42.680 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-30T12:34:42.683 INFO:tasks.ceph.osd.1:Started 2023-10-30T12:34:42.683 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-30T12:34:42.684 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-30T12:34:42.687 INFO:tasks.ceph.osd.2:Started 2023-10-30T12:34:42.688 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-30T12:34:42.688 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-30T12:34:42.689 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:42.687+0000 7f3d8f37e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-30T12:34:42.693 INFO:tasks.ceph.osd.3:Started 2023-10-30T12:34:42.694 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-30T12:34:42.694 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-30T12:34:42.696 INFO:tasks.ceph.osd.4:Started 2023-10-30T12:34:42.696 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-30T12:34:42.697 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-30T12:34:42.699 INFO:tasks.ceph.osd.5:Started 2023-10-30T12:34:42.701 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-30T12:34:42.702 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-30T12:34:42.704 INFO:tasks.ceph.osd.6:Started 2023-10-30T12:34:42.704 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-30T12:34:42.704 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-30T12:34:42.709 INFO:tasks.ceph.osd.7:Started 2023-10-30T12:34:42.710 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:34:42.784 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:42.783+0000 7f3d8f37e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-30T12:34:42.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f3434bb8640 1 Processor -- start 2023-10-30T12:34:42.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f3434bb8640 1 -- start start 2023-10-30T12:34:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f3434bb8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34301538a0 0x7f343015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f3434bb8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f34300ca0e0 0x7f343015a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f3434bb8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34300cb800 con 0x7f34300cf160 2023-10-30T12:34:42.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f3434bb8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343015dc30 con 0x7f34300ca0e0 2023-10-30T12:34:42.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f3434bb8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34300cb330 con 0x7f34301538a0 2023-10-30T12:34:42.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f342e59c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34301538a0 0x7f343015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f342dd9b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f34300ca0e0 0x7f343015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f342ed9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f34300cf160 0x7f34300cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58534/0 (socket says 172.21.15.12:58534) 2023-10-30T12:34:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f342ed9d640 1 -- 172.21.15.12:0/4093427650 learned_addr learned my addr 172.21.15.12:0/4093427650 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:34:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f342e59c640 1 -- 172.21.15.12:0/4093427650 >> v2:172.21.15.12:3300/0 conn(0x7f34300ca0e0 msgr2=0x7f343015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f342e59c640 1 --2- 172.21.15.12:0/4093427650 >> v2:172.21.15.12:3300/0 conn(0x7f34300ca0e0 0x7f343015a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f342e59c640 1 -- 172.21.15.12:0/4093427650 >> v1:172.21.15.12:6789/0 conn(0x7f34300cf160 legacy=0x7f34300cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f342e59c640 1 -- 172.21.15.12:0/4093427650 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34300cad70 con 0x7f34301538a0 2023-10-30T12:34:42.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.887+0000 7f342dd9b640 1 --2- 172.21.15.12:0/4093427650 >> v2:172.21.15.12:3300/0 conn(0x7f34300ca0e0 0x7f343015a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:34:42.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.891+0000 7f342e59c640 1 --2- 172.21.15.12:0/4093427650 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34301538a0 0x7f343015d6f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f34200097c0 tx=0x7f342001ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f0e5afb32b71bd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:42.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.891+0000 7f342d59a640 1 -- 172.21.15.12:0/4093427650 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34200a8970 con 0x7f34301538a0 2023-10-30T12:34:42.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.891+0000 7f342d59a640 1 -- 172.21.15.12:0/4093427650 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34200a8ad0 con 0x7f34301538a0 2023-10-30T12:34:42.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.891+0000 7f342d59a640 1 -- 172.21.15.12:0/4093427650 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34200a8e60 con 0x7f34301538a0 2023-10-30T12:34:42.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.891+0000 7f3434bb8640 1 -- 172.21.15.12:0/4093427650 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34301538a0 msgr2=0x7f343015d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.891+0000 7f3434bb8640 1 --2- 172.21.15.12:0/4093427650 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34301538a0 0x7f343015d6f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f34200097c0 tx=0x7f342001ec70 comp rx=0 tx=0).stop 2023-10-30T12:34:42.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.891+0000 7f3434bb8640 1 -- 172.21.15.12:0/4093427650 shutdown_connections 2023-10-30T12:34:42.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.891+0000 7f3434bb8640 1 --2- 172.21.15.12:0/4093427650 >> v2:172.21.15.12:3300/0 conn(0x7f34300ca0e0 0x7f343015a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.891+0000 7f3434bb8640 1 --2- 172.21.15.12:0/4093427650 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34301538a0 0x7f343015d6f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.891+0000 7f3434bb8640 1 -- 172.21.15.12:0/4093427650 >> 172.21.15.12:0/4093427650 conn(0x7f343006bfe0 msgr2=0x7f343006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:42.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.891+0000 7f3434bb8640 1 -- 172.21.15.12:0/4093427650 shutdown_connections 2023-10-30T12:34:42.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.891+0000 7f3434bb8640 1 -- 172.21.15.12:0/4093427650 wait complete. 2023-10-30T12:34:42.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f3434bb8640 1 Processor -- start 2023-10-30T12:34:42.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f3434bb8640 1 -- start start 2023-10-30T12:34:42.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f3434bb8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300ca0e0 0x7f34300ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f3434bb8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34300cf160 0x7f34300b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f3434bb8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34301538a0 0x7f34300b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:42.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f3434bb8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34301efcc0 con 0x7f34300cf160 2023-10-30T12:34:42.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f3434bb8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34301f01f0 con 0x7f34300ca0e0 2023-10-30T12:34:42.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f3434bb8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34301eff10 con 0x7f34301538a0 2023-10-30T12:34:42.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f342dd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34300cf160 0x7f34300b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f342dd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34300cf160 0x7f34300b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58878/0 (socket says 172.21.15.12:58878) 2023-10-30T12:34:42.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f342dd9b640 1 -- 172.21.15.12:0/2776790501 learned_addr learned my addr 172.21.15.12:0/2776790501 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:34:42.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f342ed9d640 1 --2- 172.21.15.12:0/2776790501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34301538a0 0x7f34300b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f342e59c640 1 --2- 172.21.15.12:0/2776790501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300ca0e0 0x7f34300ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:42.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f342dd9b640 1 -- 172.21.15.12:0/2776790501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34301538a0 msgr2=0x7f34300b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f342dd9b640 1 --2- 172.21.15.12:0/2776790501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34301538a0 0x7f34300b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f342dd9b640 1 -- 172.21.15.12:0/2776790501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300ca0e0 msgr2=0x7f34300ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f342dd9b640 1 --2- 172.21.15.12:0/2776790501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300ca0e0 0x7f34300ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f342dd9b640 1 -- 172.21.15.12:0/2776790501 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3424009420 con 0x7f34300cf160 2023-10-30T12:34:42.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f342dd9b640 1 --2- 172.21.15.12:0/2776790501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34300cf160 0x7f34300b3db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f342400ebe0 tx=0x7f342400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:42.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f341effd640 1 -- 172.21.15.12:0/2776790501 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342400eee0 con 0x7f34300cf160 2023-10-30T12:34:42.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f341effd640 1 -- 172.21.15.12:0/2776790501 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342400cc30 con 0x7f34300cf160 2023-10-30T12:34:42.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f341effd640 1 -- 172.21.15.12:0/2776790501 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424013470 con 0x7f34300cf160 2023-10-30T12:34:42.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f3434bb8640 1 -- 172.21.15.12:0/2776790501 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3420009420 con 0x7f34300cf160 2023-10-30T12:34:42.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.895+0000 7f3434bb8640 1 -- 172.21.15.12:0/2776790501 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34300b35b0 con 0x7f34300cf160 2023-10-30T12:34:42.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.899+0000 7f341effd640 1 -- 172.21.15.12:0/2776790501 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f34240138d0 con 0x7f34300cf160 2023-10-30T12:34:42.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.899+0000 7f341effd640 1 -- 172.21.15.12:0/2776790501 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f342401c450 con 0x7f34300cf160 2023-10-30T12:34:42.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.899+0000 7f3434bb8640 1 -- 172.21.15.12:0/2776790501 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34300b2a50 con 0x7f34300cf160 2023-10-30T12:34:42.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.899+0000 7f341effd640 1 -- 172.21.15.12:0/2776790501 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f3424021060 con 0x7f34300cf160 2023-10-30T12:34:42.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.971+0000 7f3434bb8640 1 -- 172.21.15.12:0/2776790501 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f343004e170 con 0x7f34300cf160 2023-10-30T12:34:42.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.975+0000 7f341effd640 1 -- 172.21.15.12:0/2776790501 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f3424017070 con 0x7f34300cf160 2023-10-30T12:34:42.976 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:42.976 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":10,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:34:42.656278+0000","last_up_change":"0.000000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:34:42.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.975+0000 7f3434bb8640 1 -- 172.21.15.12:0/2776790501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34300cf160 msgr2=0x7f34300b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:42.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.975+0000 7f3434bb8640 1 --2- 172.21.15.12:0/2776790501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34300cf160 0x7f34300b3db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f342400ebe0 tx=0x7f342400c520 comp rx=0 tx=0).stop 2023-10-30T12:34:42.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.975+0000 7f3434bb8640 1 -- 172.21.15.12:0/2776790501 shutdown_connections 2023-10-30T12:34:42.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.975+0000 7f3434bb8640 1 --2- 172.21.15.12:0/2776790501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34301538a0 0x7f34300b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.975+0000 7f3434bb8640 1 --2- 172.21.15.12:0/2776790501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34300cf160 0x7f34300b3db0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.975+0000 7f3434bb8640 1 --2- 172.21.15.12:0/2776790501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300ca0e0 0x7f34300ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:42.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.975+0000 7f3434bb8640 1 -- 172.21.15.12:0/2776790501 >> 172.21.15.12:0/2776790501 conn(0x7f343006bfe0 msgr2=0x7f3430152870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:42.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.975+0000 7f3434bb8640 1 -- 172.21.15.12:0/2776790501 shutdown_connections 2023-10-30T12:34:42.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:42.975+0000 7f3434bb8640 1 -- 172.21.15.12:0/2776790501 wait complete. 2023-10-30T12:34:42.997 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-30T12:34:42.997 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-30T12:34:43.114 INFO:tasks.ceph.mgr.x.smithi154.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-10-30T12:34:43.115 INFO:tasks.ceph.mgr.x.smithi154.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-10-30T12:34:43.115 INFO:tasks.ceph.mgr.x.smithi154.stderr: from numpy import show_config as show_numpy_config 2023-10-30T12:34:43.173 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:43.172+0000 7f692bb33180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-30T12:34:43.292 INFO:tasks.ceph.osd.5.smithi154.stderr:2023-10-30T12:34:43.292+0000 7f573492e640 -1 Falling back to public interface 2023-10-30T12:34:43.295 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:43.296+0000 7f692bb33180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-30T12:34:43.308 INFO:tasks.ceph.osd.4.smithi154.stderr:2023-10-30T12:34:43.308+0000 7f9eb94e5640 -1 Falling back to public interface 2023-10-30T12:34:43.309 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-30T12:34:43.308+0000 7f52dcf6a640 -1 Falling back to public interface 2023-10-30T12:34:43.310 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-30T12:34:43.308+0000 7f4d56126640 -1 Falling back to public interface 2023-10-30T12:34:43.348 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-10-30T12:34:43.347+0000 7fdf05d5a640 -1 Falling back to public interface 2023-10-30T12:34:43.356 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-10-30T12:34:43.355+0000 7f59b742b640 -1 Falling back to public interface 2023-10-30T12:34:43.368 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-10-30T12:34:43.367+0000 7f23259a1640 -1 Falling back to public interface 2023-10-30T12:34:43.376 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-10-30T12:34:43.375+0000 7ff6d9551640 -1 Falling back to public interface 2023-10-30T12:34:43.446 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:43.444+0000 7f692bb33180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-30T12:34:43.461 INFO:tasks.ceph.mgr.y.smithi012.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-10-30T12:34:43.461 INFO:tasks.ceph.mgr.y.smithi012.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-10-30T12:34:43.462 INFO:tasks.ceph.mgr.y.smithi012.stderr: from numpy import show_config as show_numpy_config 2023-10-30T12:34:43.506 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:43.503+0000 7f3d8f37e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-30T12:34:43.509 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:43.508+0000 7f692bb33180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-30T12:34:43.631 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:43.632+0000 7f692bb33180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-30T12:34:43.647 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:43.643+0000 7f3d8f37e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-30T12:34:43.801 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-30T12:34:43.828 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:43.827+0000 7f3d8f37e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-30T12:34:43.906 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:43.903+0000 7f3d8f37e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-30T12:34:43.934 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.931+0000 7f4b94cc9640 1 Processor -- start 2023-10-30T12:34:43.935 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.931+0000 7f4b94cc9640 1 -- start start 2023-10-30T12:34:43.935 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.935+0000 7f4b94cc9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b901539f0 0x7f4b900c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:43.939 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.935+0000 7f4b94cc9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b900c5390 con 0x7f4b900c9f90 2023-10-30T12:34:43.939 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.935+0000 7f4b94cc9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b9015a240 con 0x7f4b900cf080 2023-10-30T12:34:43.939 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.935+0000 7f4b94cc9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900ca980 con 0x7f4b901539f0 2023-10-30T12:34:43.939 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.935+0000 7f4b87fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4b900c9f90 0x7f4b90159740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58544/0 (socket says 172.21.15.12:58544) 2023-10-30T12:34:43.939 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.935+0000 7f4b87fff640 1 -- 172.21.15.12:0/2042742340 learned_addr learned my addr 172.21.15.12:0/2042742340 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:34:43.940 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.935+0000 7f4b8e59c640 1 --2- 172.21.15.12:0/2042742340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b901539f0 0x7f4b900c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:43.940 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.935+0000 7f4b8dd9b640 1 -- 172.21.15.12:0/2042742340 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499652335 0 0) 0x7f4b9015a240 con 0x7f4b900cf080 2023-10-30T12:34:43.940 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.935+0000 7f4b8dd9b640 1 -- 172.21.15.12:0/2042742340 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b7c003530 con 0x7f4b900cf080 2023-10-30T12:34:43.940 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.935+0000 7f4b8dd9b640 1 -- 172.21.15.12:0/2042742340 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865888980 0 0) 0x7f4b900c5390 con 0x7f4b900c9f90 2023-10-30T12:34:43.940 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.935+0000 7f4b8dd9b640 1 -- 172.21.15.12:0/2042742340 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b9015a240 con 0x7f4b900c9f90 2023-10-30T12:34:43.941 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.935+0000 7f4b8e59c640 1 -- 172.21.15.12:0/2042742340 >> v1:172.21.15.154:6789/0 conn(0x7f4b900cf080 legacy=0x7f4b901640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:43.941 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.935+0000 7f4b8dd9b640 1 -- 172.21.15.12:0/2042742340 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3447690527 0 0) 0x7f4b7c003530 con 0x7f4b900cf080 2023-10-30T12:34:43.941 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.935+0000 7f4b8e59c640 1 -- 172.21.15.12:0/2042742340 >> v1:172.21.15.12:6789/0 conn(0x7f4b900c9f90 legacy=0x7f4b90159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:43.941 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.935+0000 7f4b8e59c640 1 -- 172.21.15.12:0/2042742340 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9015a520 con 0x7f4b901539f0 2023-10-30T12:34:43.943 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.939+0000 7f4b8e59c640 1 --2- 172.21.15.12:0/2042742340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b901539f0 0x7f4b900c5820 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009770 tx=0x7f4b8001c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae6984f6e98d947e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:43.943 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.939+0000 7f4b8dd9b640 1 -- 172.21.15.12:0/2042742340 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b88001a00 con 0x7f4b901539f0 2023-10-30T12:34:43.943 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.939+0000 7f4b8dd9b640 1 -- 172.21.15.12:0/2042742340 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b800a7050 con 0x7f4b901539f0 2023-10-30T12:34:43.943 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.939+0000 7f4b8dd9b640 1 -- 172.21.15.12:0/2042742340 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b800ab620 con 0x7f4b901539f0 2023-10-30T12:34:43.943 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.939+0000 7f4b94cc9640 1 -- 172.21.15.12:0/2042742340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b901539f0 msgr2=0x7f4b900c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:43.944 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.939+0000 7f4b94cc9640 1 --2- 172.21.15.12:0/2042742340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b901539f0 0x7f4b900c5820 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009770 tx=0x7f4b8001c380 comp rx=0 tx=0).stop 2023-10-30T12:34:43.944 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.939+0000 7f4b94cc9640 1 -- 172.21.15.12:0/2042742340 shutdown_connections 2023-10-30T12:34:43.944 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.939+0000 7f4b94cc9640 1 --2- 172.21.15.12:0/2042742340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b901539f0 0x7f4b900c5820 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:43.944 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.939+0000 7f4b94cc9640 1 -- 172.21.15.12:0/2042742340 >> 172.21.15.12:0/2042742340 conn(0x7f4b90065bf0 msgr2=0x7f4b90066000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:43.944 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.939+0000 7f4b94cc9640 1 -- 172.21.15.12:0/2042742340 shutdown_connections 2023-10-30T12:34:43.945 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.939+0000 7f4b94cc9640 1 -- 172.21.15.12:0/2042742340 wait complete. 2023-10-30T12:34:43.945 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.939+0000 7f4b94cc9640 1 Processor -- start 2023-10-30T12:34:43.945 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.939+0000 7f4b94cc9640 1 -- start start 2023-10-30T12:34:43.947 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b94cc9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b900c9f90 0x7f4b900b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:43.948 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b94cc9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b900cf080 0x7f4b900b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:43.948 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b94cc9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b901539f0 0x7f4b90155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:43.948 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b94cc9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b90141950 con 0x7f4b900c9f90 2023-10-30T12:34:43.948 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b94cc9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b901554a0 con 0x7f4b900cf080 2023-10-30T12:34:43.949 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b94cc9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b90155af0 con 0x7f4b901539f0 2023-10-30T12:34:43.949 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b8e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b900c9f90 0x7f4b900b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:43.949 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b8e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b900c9f90 0x7f4b900b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58886/0 (socket says 172.21.15.12:58886) 2023-10-30T12:34:43.949 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b8e59c640 1 -- 172.21.15.12:0/952417971 learned_addr learned my addr 172.21.15.12:0/952417971 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:34:43.949 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b8ed9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b901539f0 0x7f4b90155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:43.950 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b87fff640 1 --2- 172.21.15.12:0/952417971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b900cf080 0x7f4b900b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:43.950 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b8e59c640 1 -- 172.21.15.12:0/952417971 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b901539f0 msgr2=0x7f4b90155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:43.950 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b8e59c640 1 --2- 172.21.15.12:0/952417971 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b901539f0 0x7f4b90155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:43.950 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b8e59c640 1 -- 172.21.15.12:0/952417971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b900cf080 msgr2=0x7f4b900b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:43.950 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b8e59c640 1 --2- 172.21.15.12:0/952417971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b900cf080 0x7f4b900b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:43.951 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b8e59c640 1 -- 172.21.15.12:0/952417971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b80009420 con 0x7f4b900c9f90 2023-10-30T12:34:43.951 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b8ed9d640 1 --2- 172.21.15.12:0/952417971 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b901539f0 0x7f4b90155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:34:43.951 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b8e59c640 1 --2- 172.21.15.12:0/952417971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b900c9f90 0x7f4b900b31f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4b8001c860 tx=0x7f4b8001abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:43.951 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b86ffd640 1 -- 172.21.15.12:0/952417971 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b800bc070 con 0x7f4b900c9f90 2023-10-30T12:34:43.951 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b86ffd640 1 -- 172.21.15.12:0/952417971 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b800abc70 con 0x7f4b900c9f90 2023-10-30T12:34:43.951 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b86ffd640 1 -- 172.21.15.12:0/952417971 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80004020 con 0x7f4b900c9f90 2023-10-30T12:34:43.952 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b94cc9640 1 -- 172.21.15.12:0/952417971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b90143a10 con 0x7f4b900c9f90 2023-10-30T12:34:43.952 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b94cc9640 1 -- 172.21.15.12:0/952417971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b90144690 con 0x7f4b900c9f90 2023-10-30T12:34:43.952 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b86ffd640 1 -- 172.21.15.12:0/952417971 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4b80003620 con 0x7f4b900c9f90 2023-10-30T12:34:43.952 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b86ffd640 1 -- 172.21.15.12:0/952417971 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4b800b7450 con 0x7f4b900c9f90 2023-10-30T12:34:43.952 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.943+0000 7f4b94cc9640 1 -- 172.21.15.12:0/952417971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9004e170 con 0x7f4b900c9f90 2023-10-30T12:34:43.953 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:43.947+0000 7f4b86ffd640 1 -- 172.21.15.12:0/952417971 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f4b800b0070 con 0x7f4b900c9f90 2023-10-30T12:34:44.008 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:44.007+0000 7f4b94cc9640 1 -- 172.21.15.12:0/952417971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b900caad0 con 0x7f4b900c9f90 2023-10-30T12:34:44.008 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:44.007+0000 7f4b86ffd640 1 -- 172.21.15.12:0/952417971 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4b800038d0 con 0x7f4b900c9f90 2023-10-30T12:34:44.009 INFO:teuthology.misc.health.smithi012.stdout: 2023-10-30T12:34:44.009 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":10,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:34:42.656278+0000","last_up_change":"0.000000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:34:44.010 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:44.007+0000 7f4b94cc9640 1 -- 172.21.15.12:0/952417971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b900c9f90 msgr2=0x7f4b900b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:44.010 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:44.007+0000 7f4b94cc9640 1 --2- 172.21.15.12:0/952417971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b900c9f90 0x7f4b900b31f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4b8001c860 tx=0x7f4b8001abc0 comp rx=0 tx=0).stop 2023-10-30T12:34:44.010 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:44.007+0000 7f4b94cc9640 1 -- 172.21.15.12:0/952417971 shutdown_connections 2023-10-30T12:34:44.011 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:44.007+0000 7f4b94cc9640 1 --2- 172.21.15.12:0/952417971 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b901539f0 0x7f4b90155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:44.011 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:44.007+0000 7f4b94cc9640 1 --2- 172.21.15.12:0/952417971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b900cf080 0x7f4b900b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:44.011 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:44.007+0000 7f4b94cc9640 1 --2- 172.21.15.12:0/952417971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b900c9f90 0x7f4b900b31f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:44.011 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:44.007+0000 7f4b94cc9640 1 -- 172.21.15.12:0/952417971 >> 172.21.15.12:0/952417971 conn(0x7f4b90065bf0 msgr2=0x7f4b90065fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:44.011 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:44.007+0000 7f4b94cc9640 1 -- 172.21.15.12:0/952417971 shutdown_connections 2023-10-30T12:34:44.012 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:44.007+0000 7f4b94cc9640 1 -- 172.21.15.12:0/952417971 wait complete. 2023-10-30T12:34:44.027 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-30T12:34:44.056 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:44.055+0000 7f3d8f37e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-30T12:34:44.232 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:44.232+0000 7f692bb33180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-30T12:34:44.307 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:44.308+0000 7f692bb33180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-30T12:34:44.477 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:44.476+0000 7f692bb33180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-30T12:34:44.550 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:44.548+0000 7f692bb33180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-30T12:34:44.773 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:44.771+0000 7f3d8f37e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-30T12:34:44.837 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:44.836+0000 7f692bb33180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-30T12:34:44.864 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:44.863+0000 7f3d8f37e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-30T12:34:44.896 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:44.896+0000 7f692bb33180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-30T12:34:45.065 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:45.063+0000 7f3d8f37e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-30T12:34:45.154 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:45.151+0000 7f3d8f37e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-30T12:34:45.498 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:45.495+0000 7f3d8f37e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-30T12:34:45.570 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-10-30T12:34:45.567+0000 7f3d8f37e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-30T12:34:45.945 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:45.944+0000 7f68d32a9640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-30T12:34:45.946 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-30T12:34:45.944+0000 7f68d32a9640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-30T12:34:47.127 INFO:tasks.ceph.osd.5.smithi154.stderr:2023-10-30T12:34:47.123+0000 7f573492e640 -1 osd.5 0 log_to_monitors true 2023-10-30T12:34:47.169 INFO:tasks.ceph.osd.4.smithi154.stderr:2023-10-30T12:34:47.167+0000 7f9eb94e5640 -1 osd.4 0 log_to_monitors true 2023-10-30T12:34:47.237 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-30T12:34:47.235+0000 7f4d56126640 -1 osd.7 0 log_to_monitors true 2023-10-30T12:34:47.238 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-30T12:34:47.235+0000 7f52dcf6a640 -1 osd.6 0 log_to_monitors true 2023-10-30T12:34:47.259 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-10-30T12:34:47.255+0000 7fdf05d5a640 -1 osd.2 0 log_to_monitors true 2023-10-30T12:34:47.262 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-10-30T12:34:47.259+0000 7f59b742b640 -1 osd.0 0 log_to_monitors true 2023-10-30T12:34:47.355 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-10-30T12:34:47.355+0000 7ff6d9551640 -1 osd.3 0 log_to_monitors true 2023-10-30T12:34:47.547 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-10-30T12:34:47.543+0000 7f23259a1640 -1 osd.1 0 log_to_monitors true 2023-10-30T12:34:50.419 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-10-30T12:34:50.415+0000 7fdf014df640 -1 osd.2 0 waiting for initial osdmap 2023-10-30T12:34:50.435 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-10-30T12:34:50.435+0000 7f59b2167640 -1 osd.0 0 waiting for initial osdmap 2023-10-30T12:34:50.462 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-10-30T12:34:50.459+0000 7f59aa7ae640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:34:50.462 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-10-30T12:34:50.459+0000 7fdef9b26640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:34:50.549 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-10-30T12:34:50.547+0000 7ff6d650e640 -1 osd.3 0 waiting for initial osdmap 2023-10-30T12:34:50.585 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-10-30T12:34:50.583+0000 7ff6cd31d640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:34:50.834 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-30T12:34:50.912 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-30T12:34:50.907+0000 7f52d86ef640 -1 osd.6 0 waiting for initial osdmap 2023-10-30T12:34:50.937 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-30T12:34:50.935+0000 7f52d0d36640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:34:50.974 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.971+0000 7f2eaff2a640 1 Processor -- start 2023-10-30T12:34:50.974 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.971+0000 7f2eaff2a640 1 -- start start 2023-10-30T12:34:50.975 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.971+0000 7f2eaff2a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2ea81539f0 0x7f2ea80c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:50.975 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.971+0000 7f2eaff2a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2ea80c9f90 0x7f2ea8159740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:50.976 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.971+0000 7f2eaff2a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea80c5390 con 0x7f2ea8159c80 2023-10-30T12:34:50.976 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.971+0000 7f2eaff2a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea80ca980 con 0x7f2ea80c9f90 2023-10-30T12:34:50.976 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.971+0000 7f2eaff2a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea815a240 con 0x7f2ea81539f0 2023-10-30T12:34:50.977 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.975+0000 7f2ead4c5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2ea80c9f90 0x7f2ea8159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:50.977 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.975+0000 7f2eadcc6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2ea81539f0 0x7f2ea80c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:50.977 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.975+0000 7f2eae4c7640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2ea8159c80 0x7f2ea80cf080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56688/0 (socket says 172.21.15.12:56688) 2023-10-30T12:34:50.977 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.975+0000 7f2eae4c7640 1 -- 172.21.15.12:0/2530562533 learned_addr learned my addr 172.21.15.12:0/2530562533 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:34:50.978 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.975+0000 7f2ead4c5640 1 -- 172.21.15.12:0/2530562533 >> v2:172.21.15.154:3300/0 conn(0x7f2ea81539f0 msgr2=0x7f2ea80c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:50.978 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.975+0000 7f2ead4c5640 1 --2- 172.21.15.12:0/2530562533 >> v2:172.21.15.154:3300/0 conn(0x7f2ea81539f0 0x7f2ea80c5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:50.978 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.975+0000 7f2ead4c5640 1 -- 172.21.15.12:0/2530562533 >> v1:172.21.15.12:6789/0 conn(0x7f2ea8159c80 legacy=0x7f2ea80cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:50.978 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.975+0000 7f2ead4c5640 1 -- 172.21.15.12:0/2530562533 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea815a520 con 0x7f2ea80c9f90 2023-10-30T12:34:50.979 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.975+0000 7f2eaccc4640 1 -- 172.21.15.12:0/2530562533 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501804613 0 0) 0x7f2ea80c5390 con 0x7f2ea8159c80 2023-10-30T12:34:50.979 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2ead4c5640 1 --2- 172.21.15.12:0/2530562533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea80c9f90 0x7f2ea8159740 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8066a30 tx=0x7f2e9801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e365184bf6aba55b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:50.980 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaccc4640 1 -- 172.21.15.12:0/2530562533 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e980ab600 con 0x7f2ea80c9f90 2023-10-30T12:34:50.980 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaccc4640 1 -- 172.21.15.12:0/2530562533 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e980a7050 con 0x7f2ea80c9f90 2023-10-30T12:34:50.980 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaccc4640 1 -- 172.21.15.12:0/2530562533 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e980ab980 con 0x7f2ea80c9f90 2023-10-30T12:34:50.981 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 -- 172.21.15.12:0/2530562533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea80c9f90 msgr2=0x7f2ea8159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:50.981 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 --2- 172.21.15.12:0/2530562533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea80c9f90 0x7f2ea8159740 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8066a30 tx=0x7f2e9801c3d0 comp rx=0 tx=0).stop 2023-10-30T12:34:50.982 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 -- 172.21.15.12:0/2530562533 shutdown_connections 2023-10-30T12:34:50.982 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 --2- 172.21.15.12:0/2530562533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea80c9f90 0x7f2ea8159740 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:50.982 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 --2- 172.21.15.12:0/2530562533 >> v2:172.21.15.154:3300/0 conn(0x7f2ea81539f0 0x7f2ea80c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:50.982 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 -- 172.21.15.12:0/2530562533 >> 172.21.15.12:0/2530562533 conn(0x7f2ea8065bf0 msgr2=0x7f2ea8066000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:50.983 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 -- 172.21.15.12:0/2530562533 shutdown_connections 2023-10-30T12:34:50.983 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 -- 172.21.15.12:0/2530562533 wait complete. 2023-10-30T12:34:50.983 INFO:tasks.ceph.osd.4.smithi154.stderr:2023-10-30T12:34:50.979+0000 7f9eb64a2640 -1 osd.4 0 waiting for initial osdmap 2023-10-30T12:34:50.986 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 Processor -- start 2023-10-30T12:34:50.986 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 -- start start 2023-10-30T12:34:50.986 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ea80c9f90 0x7f2ea81585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:50.986 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea81539f0 0x7f2ea8144830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:50.987 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ea8159c80 0x7f2ea8144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:50.987 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea8156830 con 0x7f2ea81539f0 2023-10-30T12:34:50.987 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea81551f0 con 0x7f2ea8159c80 2023-10-30T12:34:50.987 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.979+0000 7f2eaff2a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea81452b0 con 0x7f2ea80c9f90 2023-10-30T12:34:50.987 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2eadcc6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ea80c9f90 0x7f2ea81585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:50.988 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2eadcc6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ea80c9f90 0x7f2ea81585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:45524/0 (socket says 172.21.15.12:45524) 2023-10-30T12:34:50.988 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2eae4c7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ea8159c80 0x7f2ea8144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:50.988 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2eadcc6640 1 -- 172.21.15.12:0/1177921562 learned_addr learned my addr 172.21.15.12:0/1177921562 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:34:50.988 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2ead4c5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea81539f0 0x7f2ea8144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:50.988 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2eadcc6640 1 -- 172.21.15.12:0/1177921562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ea8159c80 msgr2=0x7f2ea8144d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:50.989 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2eadcc6640 1 --2- 172.21.15.12:0/1177921562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ea8159c80 0x7f2ea8144d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:50.989 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2eadcc6640 1 -- 172.21.15.12:0/1177921562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea81539f0 msgr2=0x7f2ea8144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:50.989 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2eadcc6640 1 --2- 172.21.15.12:0/1177921562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea81539f0 0x7f2ea8144830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:50.989 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2eadcc6640 1 -- 172.21.15.12:0/1177921562 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e98009420 con 0x7f2ea80c9f90 2023-10-30T12:34:50.989 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2ead4c5640 1 --2- 172.21.15.12:0/1177921562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea81539f0 0x7f2ea8144830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:34:50.990 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2eae4c7640 1 --2- 172.21.15.12:0/1177921562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ea8159c80 0x7f2ea8144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:34:50.990 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2eadcc6640 1 --2- 172.21.15.12:0/1177921562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ea80c9f90 0x7f2ea81585c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400e990 tx=0x7f2ea400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:50.990 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2e9e7fc640 1 -- 172.21.15.12:0/1177921562 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4009690 con 0x7f2ea80c9f90 2023-10-30T12:34:50.990 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2eaff2a640 1 -- 172.21.15.12:0/1177921562 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea815d610 con 0x7f2ea80c9f90 2023-10-30T12:34:50.991 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2e9e7fc640 1 -- 172.21.15.12:0/1177921562 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea400c980 con 0x7f2ea80c9f90 2023-10-30T12:34:50.991 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2e9e7fc640 1 -- 172.21.15.12:0/1177921562 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea400ccf0 con 0x7f2ea80c9f90 2023-10-30T12:34:50.991 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2eaff2a640 1 -- 172.21.15.12:0/1177921562 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea8154010 con 0x7f2ea80c9f90 2023-10-30T12:34:50.991 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2eaff2a640 1 -- 172.21.15.12:0/1177921562 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea804e170 con 0x7f2ea80c9f90 2023-10-30T12:34:50.992 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.983+0000 7f2e9e7fc640 1 -- 172.21.15.12:0/1177921562 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 4) v1 ==== 85794+0+0 (secure 0 0 0) 0x7f2ea40059a0 con 0x7f2ea80c9f90 2023-10-30T12:34:50.992 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.987+0000 7f2e9e7fc640 1 --2- 172.21.15.12:0/1177921562 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2e8c066440 0x7f2e8c068900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:50.992 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.987+0000 7f2ead4c5640 1 --2- 172.21.15.12:0/1177921562 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2e8c066440 0x7f2e8c068900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:50.993 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.987+0000 7f2ead4c5640 1 --2- 172.21.15.12:0/1177921562 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2e8c066440 0x7f2e8c068900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2e980ad000 tx=0x7f2e98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:50.993 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.987+0000 7f2e9e7fc640 1 -- 172.21.15.12:0/1177921562 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f2ea4055880 con 0x7f2ea80c9f90 2023-10-30T12:34:50.993 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-10-30T12:34:50.987+0000 7f232295e640 -1 osd.1 0 waiting for initial osdmap 2023-10-30T12:34:50.994 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:50.991+0000 7f2e9e7fc640 1 -- 172.21.15.12:0/1177921562 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2ea4051bc0 con 0x7f2ea80c9f90 2023-10-30T12:34:51.016 INFO:tasks.ceph.osd.4.smithi154.stderr:2023-10-30T12:34:51.015+0000 7f9ead2b1640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:34:51.025 INFO:tasks.ceph.osd.5.smithi154.stderr:2023-10-30T12:34:51.023+0000 7f57300b3640 -1 osd.5 0 waiting for initial osdmap 2023-10-30T12:34:51.031 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-10-30T12:34:51.031+0000 7f231976d640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:34:51.032 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-30T12:34:51.027+0000 7f4d518ab640 -1 osd.7 0 waiting for initial osdmap 2023-10-30T12:34:51.069 INFO:tasks.ceph.osd.5.smithi154.stderr:2023-10-30T12:34:51.067+0000 7f57286fa640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:34:51.069 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-30T12:34:51.067+0000 7f4d49ef2640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:34:51.154 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:51.151+0000 7f2eaff2a640 1 -- 172.21.15.12:0/1177921562 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ea8153dd0 con 0x7f2ea80c9f90 2023-10-30T12:34:51.156 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:51.155+0000 7f2e9e7fc640 1 -- 172.21.15.12:0/1177921562 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2ea4021020 con 0x7f2ea80c9f90 2023-10-30T12:34:51.157 INFO:teuthology.misc.health.smithi012.stdout: 2023-10-30T12:34:51.157 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":13,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:34:49.431707+0000","last_up_change":"0.000000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:34:51.158 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:51.155+0000 7f2eaff2a640 1 -- 172.21.15.12:0/1177921562 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2e8c066440 msgr2=0x7f2e8c068900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:51.158 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:51.155+0000 7f2eaff2a640 1 --2- 172.21.15.12:0/1177921562 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2e8c066440 0x7f2e8c068900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2e980ad000 tx=0x7f2e98008040 comp rx=0 tx=0).stop 2023-10-30T12:34:51.158 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:51.155+0000 7f2eaff2a640 1 -- 172.21.15.12:0/1177921562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ea80c9f90 msgr2=0x7f2ea81585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:51.159 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:51.155+0000 7f2eaff2a640 1 --2- 172.21.15.12:0/1177921562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ea80c9f90 0x7f2ea81585c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400e990 tx=0x7f2ea400ee50 comp rx=0 tx=0).stop 2023-10-30T12:34:51.159 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:51.155+0000 7f2eaff2a640 1 -- 172.21.15.12:0/1177921562 shutdown_connections 2023-10-30T12:34:51.159 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:51.155+0000 7f2eaff2a640 1 --2- 172.21.15.12:0/1177921562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ea8159c80 0x7f2ea8144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:51.159 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:51.155+0000 7f2eaff2a640 1 --2- 172.21.15.12:0/1177921562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea81539f0 0x7f2ea8144830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:51.159 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:51.155+0000 7f2eaff2a640 1 --2- 172.21.15.12:0/1177921562 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2e8c066440 0x7f2e8c068900 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:51.160 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:51.155+0000 7f2eaff2a640 1 --2- 172.21.15.12:0/1177921562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ea80c9f90 0x7f2ea81585c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:51.160 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:51.155+0000 7f2eaff2a640 1 -- 172.21.15.12:0/1177921562 >> 172.21.15.12:0/1177921562 conn(0x7f2ea8065bf0 msgr2=0x7f2ea8151c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:51.160 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:51.155+0000 7f2eaff2a640 1 -- 172.21.15.12:0/1177921562 shutdown_connections 2023-10-30T12:34:51.160 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:51.155+0000 7f2eaff2a640 1 -- 172.21.15.12:0/1177921562 wait complete. 2023-10-30T12:34:51.176 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-30T12:34:57.982 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-30T12:34:58.120 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bfabd3640 1 Processor -- start 2023-10-30T12:34:58.120 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bfabd3640 1 -- start start 2023-10-30T12:34:58.122 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bfabd3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bf41539f0 0x7f3bf40c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.122 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bfabd3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3bf40c9f90 0x7f3bf4159740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.123 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bfabd3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf40c5390 con 0x7f3bf4159c80 2023-10-30T12:34:58.123 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bfabd3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf40ca980 con 0x7f3bf40c9f90 2023-10-30T12:34:58.123 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bfabd3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf415a240 con 0x7f3bf41539f0 2023-10-30T12:34:58.123 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bf9170640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3bf4159c80 0x7f3bf40cf080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56692/0 (socket says 172.21.15.12:56692) 2023-10-30T12:34:58.123 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bf9170640 1 -- 172.21.15.12:0/1884292508 learned_addr learned my addr 172.21.15.12:0/1884292508 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:34:58.124 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bf896f640 1 --2- 172.21.15.12:0/1884292508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bf41539f0 0x7f3bf40c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.124 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bf3fff640 1 --2- 172.21.15.12:0/1884292508 >> v2:172.21.15.154:3300/0 conn(0x7f3bf40c9f90 0x7f3bf4159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.124 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bf37fe640 1 -- 172.21.15.12:0/1884292508 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436982348 0 0) 0x7f3bf40c5390 con 0x7f3bf4159c80 2023-10-30T12:34:58.125 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bf37fe640 1 -- 172.21.15.12:0/1884292508 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3be4003530 con 0x7f3bf4159c80 2023-10-30T12:34:58.125 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bf37fe640 1 -- 172.21.15.12:0/1884292508 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3006200004 0 0) 0x7f3be4003530 con 0x7f3bf4159c80 2023-10-30T12:34:58.125 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bf896f640 1 -- 172.21.15.12:0/1884292508 >> v2:172.21.15.154:3300/0 conn(0x7f3bf40c9f90 msgr2=0x7f3bf4159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:58.125 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bf896f640 1 --2- 172.21.15.12:0/1884292508 >> v2:172.21.15.154:3300/0 conn(0x7f3bf40c9f90 0x7f3bf4159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.125 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bf896f640 1 -- 172.21.15.12:0/1884292508 >> v1:172.21.15.12:6789/0 conn(0x7f3bf4159c80 legacy=0x7f3bf40cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.126 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.118+0000 7f3bf896f640 1 -- 172.21.15.12:0/1884292508 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf415a520 con 0x7f3bf41539f0 2023-10-30T12:34:58.127 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.122+0000 7f3bf896f640 1 --2- 172.21.15.12:0/1884292508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bf41539f0 0x7f3bf40c5820 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3be80097c0 tx=0x7f3be801c310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0561ed9eab1ef9e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:58.127 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.122+0000 7f3bf37fe640 1 -- 172.21.15.12:0/1884292508 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec001a00 con 0x7f3bf41539f0 2023-10-30T12:34:58.127 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.122+0000 7f3bf37fe640 1 -- 172.21.15.12:0/1884292508 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be80a7050 con 0x7f3bf41539f0 2023-10-30T12:34:58.128 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.122+0000 7f3bf37fe640 1 -- 172.21.15.12:0/1884292508 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be80ab600 con 0x7f3bf41539f0 2023-10-30T12:34:58.128 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.122+0000 7f3bfabd3640 1 -- 172.21.15.12:0/1884292508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bf41539f0 msgr2=0x7f3bf40c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.128 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.122+0000 7f3bfabd3640 1 --2- 172.21.15.12:0/1884292508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bf41539f0 0x7f3bf40c5820 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3be80097c0 tx=0x7f3be801c310 comp rx=0 tx=0).stop 2023-10-30T12:34:58.128 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.122+0000 7f3bfabd3640 1 -- 172.21.15.12:0/1884292508 shutdown_connections 2023-10-30T12:34:58.128 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.122+0000 7f3bfabd3640 1 --2- 172.21.15.12:0/1884292508 >> v2:172.21.15.154:3300/0 conn(0x7f3bf40c9f90 0x7f3bf4159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.129 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.122+0000 7f3bfabd3640 1 --2- 172.21.15.12:0/1884292508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bf41539f0 0x7f3bf40c5820 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.129 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.122+0000 7f3bfabd3640 1 -- 172.21.15.12:0/1884292508 >> 172.21.15.12:0/1884292508 conn(0x7f3bf4065bf0 msgr2=0x7f3bf4066000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:58.129 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.122+0000 7f3bfabd3640 1 -- 172.21.15.12:0/1884292508 shutdown_connections 2023-10-30T12:34:58.129 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.122+0000 7f3bfabd3640 1 -- 172.21.15.12:0/1884292508 wait complete. 2023-10-30T12:34:58.132 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bfabd3640 1 Processor -- start 2023-10-30T12:34:58.132 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bfabd3640 1 -- start start 2023-10-30T12:34:58.132 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bfabd3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bf40c9f90 0x7f3bf4146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.132 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bfabd3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bf41539f0 0x7f3bf4158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.132 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bfabd3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bf4159c80 0x7f3bf413ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.133 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bfabd3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4146890 con 0x7f3bf41539f0 2023-10-30T12:34:58.133 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bfabd3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4147260 con 0x7f3bf4159c80 2023-10-30T12:34:58.133 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bfabd3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf41459e0 con 0x7f3bf40c9f90 2023-10-30T12:34:58.133 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bf3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bf41539f0 0x7f3bf4158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.133 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bf3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bf41539f0 0x7f3bf4158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33368/0 (socket says 172.21.15.12:33368) 2023-10-30T12:34:58.134 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bf3fff640 1 -- 172.21.15.12:0/2230274977 learned_addr learned my addr 172.21.15.12:0/2230274977 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:34:58.134 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bf896f640 1 --2- 172.21.15.12:0/2230274977 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bf40c9f90 0x7f3bf4146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.134 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bf9170640 1 --2- 172.21.15.12:0/2230274977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bf4159c80 0x7f3bf413ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.135 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bf3fff640 1 -- 172.21.15.12:0/2230274977 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bf40c9f90 msgr2=0x7f3bf4146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.135 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bf3fff640 1 --2- 172.21.15.12:0/2230274977 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bf40c9f90 0x7f3bf4146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.135 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bf3fff640 1 -- 172.21.15.12:0/2230274977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bf4159c80 msgr2=0x7f3bf413ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.135 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bf3fff640 1 --2- 172.21.15.12:0/2230274977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bf4159c80 0x7f3bf413ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.135 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bf3fff640 1 -- 172.21.15.12:0/2230274977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be8009420 con 0x7f3bf41539f0 2023-10-30T12:34:58.136 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bf3fff640 1 --2- 172.21.15.12:0/2230274977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bf41539f0 0x7f3bf4158d90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3be000e930 tx=0x7f3be000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:58.136 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bf17fa640 1 -- 172.21.15.12:0/2230274977 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0009690 con 0x7f3bf41539f0 2023-10-30T12:34:58.136 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bf17fa640 1 -- 172.21.15.12:0/2230274977 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be000ca00 con 0x7f3bf41539f0 2023-10-30T12:34:58.136 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bf17fa640 1 -- 172.21.15.12:0/2230274977 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be000cd70 con 0x7f3bf41539f0 2023-10-30T12:34:58.137 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bfabd3640 1 -- 172.21.15.12:0/2230274977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf4145c80 con 0x7f3bf41539f0 2023-10-30T12:34:58.137 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.126+0000 7f3bfabd3640 1 -- 172.21.15.12:0/2230274977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf4146240 con 0x7f3bf41539f0 2023-10-30T12:34:58.137 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.130+0000 7f3bfabd3640 1 -- 172.21.15.12:0/2230274977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf404e170 con 0x7f3bf41539f0 2023-10-30T12:34:58.138 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.130+0000 7f3bf17fa640 1 -- 172.21.15.12:0/2230274977 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3be0013470 con 0x7f3bf41539f0 2023-10-30T12:34:58.138 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.130+0000 7f3bf17fa640 1 --2- 172.21.15.12:0/2230274977 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3bd80664a0 0x7f3bd8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.139 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.130+0000 7f3bf17fa640 1 -- 172.21.15.12:0/2230274977 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3be0083d70 con 0x7f3bf41539f0 2023-10-30T12:34:58.139 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.130+0000 7f3bf896f640 1 --2- 172.21.15.12:0/2230274977 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3bd80664a0 0x7f3bd8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.139 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.130+0000 7f3bf896f640 1 --2- 172.21.15.12:0/2230274977 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3bd80664a0 0x7f3bd8068960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3be80ad000 tx=0x7f3be80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:58.153 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.150+0000 7f3bf17fa640 1 -- 172.21.15.12:0/2230274977 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3be00515f0 con 0x7f3bf41539f0 2023-10-30T12:34:58.312 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.310+0000 7f3bfabd3640 1 -- 172.21.15.12:0/2230274977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bf4157160 con 0x7f3bf41539f0 2023-10-30T12:34:58.313 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.310+0000 7f3bf17fa640 1 -- 172.21.15.12:0/2230274977 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f3be0017090 con 0x7f3bf41539f0 2023-10-30T12:34:58.313 INFO:teuthology.misc.health.smithi012.stdout: 2023-10-30T12:34:58.314 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":18,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:34:56.098941+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:34:58.315 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.310+0000 7f3bfabd3640 1 -- 172.21.15.12:0/2230274977 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3bd80664a0 msgr2=0x7f3bd8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.315 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.310+0000 7f3bfabd3640 1 --2- 172.21.15.12:0/2230274977 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3bd80664a0 0x7f3bd8068960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3be80ad000 tx=0x7f3be80a6000 comp rx=0 tx=0).stop 2023-10-30T12:34:58.315 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.310+0000 7f3bfabd3640 1 -- 172.21.15.12:0/2230274977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bf41539f0 msgr2=0x7f3bf4158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.316 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.310+0000 7f3bfabd3640 1 --2- 172.21.15.12:0/2230274977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bf41539f0 0x7f3bf4158d90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3be000e930 tx=0x7f3be000edf0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.316 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.310+0000 7f3bfabd3640 1 -- 172.21.15.12:0/2230274977 shutdown_connections 2023-10-30T12:34:58.316 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.310+0000 7f3bfabd3640 1 --2- 172.21.15.12:0/2230274977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bf4159c80 0x7f3bf413ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.316 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.310+0000 7f3bfabd3640 1 --2- 172.21.15.12:0/2230274977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bf41539f0 0x7f3bf4158d90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.316 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.310+0000 7f3bfabd3640 1 --2- 172.21.15.12:0/2230274977 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3bd80664a0 0x7f3bd8068960 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.317 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.310+0000 7f3bfabd3640 1 --2- 172.21.15.12:0/2230274977 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bf40c9f90 0x7f3bf4146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.317 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.310+0000 7f3bfabd3640 1 -- 172.21.15.12:0/2230274977 >> 172.21.15.12:0/2230274977 conn(0x7f3bf4065bf0 msgr2=0x7f3bf41446a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:58.317 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.310+0000 7f3bfabd3640 1 -- 172.21.15.12:0/2230274977 shutdown_connections 2023-10-30T12:34:58.317 INFO:teuthology.misc.health.smithi012.stderr:2023-10-30T12:34:58.310+0000 7f3bfabd3640 1 -- 172.21.15.12:0/2230274977 wait complete. 2023-10-30T12:34:58.333 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-30T12:34:58.334 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-30T12:34:58.334 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-30T12:34:58.335 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-30T12:34:58.339 INFO:tasks.ceph.mds.a:Started 2023-10-30T12:34:58.339 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-30T12:34:58.339 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-30T12:34:58.343 INFO:tasks.ceph.mds.c:Started 2023-10-30T12:34:58.343 INFO:tasks.ceph.mds.e:Restarting daemon 2023-10-30T12:34:58.344 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-10-30T12:34:58.348 INFO:tasks.ceph.mds.e:Started 2023-10-30T12:34:58.348 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-30T12:34:58.349 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-30T12:34:58.353 INFO:tasks.ceph.mds.b:Started 2023-10-30T12:34:58.354 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-30T12:34:58.354 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-30T12:34:58.358 INFO:tasks.ceph.mds.d:Started 2023-10-30T12:34:58.359 INFO:tasks.ceph.mds.f:Restarting daemon 2023-10-30T12:34:58.359 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-10-30T12:34:58.362 INFO:tasks.ceph.mds.f:Started 2023-10-30T12:34:58.362 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-30T12:34:58.363 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:34:58.435 INFO:tasks.ceph.mds.f.smithi154.stdout:starting mds.f at 2023-10-30T12:34:58.435 INFO:tasks.ceph.mds.b.smithi154.stdout:starting mds.b at 2023-10-30T12:34:58.436 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2023-10-30T12:34:58.438 INFO:tasks.ceph.mds.c.smithi012.stdout:starting mds.c at 2023-10-30T12:34:58.438 INFO:tasks.ceph.mds.e.smithi012.stdout:starting mds.e at 2023-10-30T12:34:58.439 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2023-10-30T12:34:58.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.514+0000 7fdaf4e46640 1 Processor -- start 2023-10-30T12:34:58.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.514+0000 7fdaf4e46640 1 -- start start 2023-10-30T12:34:58.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.514+0000 7fdaf4e46640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdaf00c6540 0x7fdaf015e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.514+0000 7fdaf4e46640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf00c5810 con 0x7fdaf0070740 2023-10-30T12:34:58.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.514+0000 7fdaf4e46640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf00c69b0 con 0x7fdaf01580e0 2023-10-30T12:34:58.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.514+0000 7fdaf4e46640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf006acb0 con 0x7fdaf00c6540 2023-10-30T12:34:58.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.514+0000 7fdaedd9b640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdaf0070740 0x7fdaf006a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56710/0 (socket says 172.21.15.12:56710) 2023-10-30T12:34:58.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.514+0000 7fdaedd9b640 1 -- 172.21.15.12:0/2947475034 learned_addr learned my addr 172.21.15.12:0/2947475034 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:34:58.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.514+0000 7fdaeed9d640 1 --2- 172.21.15.12:0/2947475034 >> v2:172.21.15.12:3300/0 conn(0x7fdaf00c6540 0x7fdaf015e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.514+0000 7fdaeed9d640 1 -- 172.21.15.12:0/2947475034 >> v1:172.21.15.154:6789/0 conn(0x7fdaf01580e0 legacy=0x7fdaf0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.514+0000 7fdaeed9d640 1 -- 172.21.15.12:0/2947475034 >> v1:172.21.15.12:6789/0 conn(0x7fdaf0070740 legacy=0x7fdaf006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.518+0000 7fdaeed9d640 1 -- 172.21.15.12:0/2947475034 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf00c6bb0 con 0x7fdaf00c6540 2023-10-30T12:34:58.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.518+0000 7fdaed59a640 1 -- 172.21.15.12:0/2947475034 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 97932010 0 0) 0x7fdaf00c5810 con 0x7fdaf0070740 2023-10-30T12:34:58.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.518+0000 7fdaeed9d640 1 --2- 172.21.15.12:0/2947475034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaf00c6540 0x7fdaf015e700 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009d90 tx=0x7fdae801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a708e831da5334 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:58.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.518+0000 7fdaed59a640 1 -- 172.21.15.12:0/2947475034 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae80a5920 con 0x7fdaf00c6540 2023-10-30T12:34:58.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.518+0000 7fdaed59a640 1 -- 172.21.15.12:0/2947475034 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae80a5a80 con 0x7fdaf00c6540 2023-10-30T12:34:58.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.518+0000 7fdaed59a640 1 -- 172.21.15.12:0/2947475034 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae80a5e10 con 0x7fdaf00c6540 2023-10-30T12:34:58.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.518+0000 7fdaf4e46640 1 -- 172.21.15.12:0/2947475034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaf00c6540 msgr2=0x7fdaf015e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.518+0000 7fdaf4e46640 1 --2- 172.21.15.12:0/2947475034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaf00c6540 0x7fdaf015e700 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009d90 tx=0x7fdae801c950 comp rx=0 tx=0).stop 2023-10-30T12:34:58.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.518+0000 7fdaf4e46640 1 -- 172.21.15.12:0/2947475034 shutdown_connections 2023-10-30T12:34:58.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.518+0000 7fdaf4e46640 1 --2- 172.21.15.12:0/2947475034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaf00c6540 0x7fdaf015e700 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.518+0000 7fdaf4e46640 1 -- 172.21.15.12:0/2947475034 >> 172.21.15.12:0/2947475034 conn(0x7fdaf00c7910 msgr2=0x7fdaf00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:58.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.518+0000 7fdaf4e46640 1 -- 172.21.15.12:0/2947475034 shutdown_connections 2023-10-30T12:34:58.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.518+0000 7fdaf4e46640 1 -- 172.21.15.12:0/2947475034 wait complete. 2023-10-30T12:34:58.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaf4e46640 1 Processor -- start 2023-10-30T12:34:58.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaf4e46640 1 -- start start 2023-10-30T12:34:58.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaf4e46640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaf0070740 0x7fdaf0146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaf4e46640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaf00c6540 0x7fdaf01f6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaf4e46640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaf01580e0 0x7fdaf014cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaf4e46640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf01fa320 con 0x7fdaf01580e0 2023-10-30T12:34:58.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaf4e46640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf0149910 con 0x7fdaf0070740 2023-10-30T12:34:58.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaf4e46640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf01fa7b0 con 0x7fdaf00c6540 2023-10-30T12:34:58.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaeed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaf01580e0 0x7fdaf014cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaeed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaf01580e0 0x7fdaf014cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33442/0 (socket says 172.21.15.12:33442) 2023-10-30T12:34:58.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaeed9d640 1 -- 172.21.15.12:0/2563568447 learned_addr learned my addr 172.21.15.12:0/2563568447 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:34:58.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaee59c640 1 --2- 172.21.15.12:0/2563568447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaf0070740 0x7fdaf0146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaedd9b640 1 --2- 172.21.15.12:0/2563568447 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaf00c6540 0x7fdaf01f6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaeed9d640 1 -- 172.21.15.12:0/2563568447 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaf00c6540 msgr2=0x7fdaf01f6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaeed9d640 1 --2- 172.21.15.12:0/2563568447 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaf00c6540 0x7fdaf01f6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaeed9d640 1 -- 172.21.15.12:0/2563568447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaf0070740 msgr2=0x7fdaf0146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaeed9d640 1 --2- 172.21.15.12:0/2563568447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaf0070740 0x7fdaf0146c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaeed9d640 1 -- 172.21.15.12:0/2563568447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae8009a70 con 0x7fdaf01580e0 2023-10-30T12:34:58.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaeed9d640 1 --2- 172.21.15.12:0/2563568447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaf01580e0 0x7fdaf014cf40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdae80b0040 tx=0x7fdae80196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:58.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdadeffd640 1 -- 172.21.15.12:0/2563568447 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae80ade40 con 0x7fdaf01580e0 2023-10-30T12:34:58.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdadeffd640 1 -- 172.21.15.12:0/2563568447 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae80b9450 con 0x7fdaf01580e0 2023-10-30T12:34:58.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaf4e46640 1 -- 172.21.15.12:0/2563568447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf0149630 con 0x7fdaf01580e0 2023-10-30T12:34:58.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdadeffd640 1 -- 172.21.15.12:0/2563568447 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae80b97e0 con 0x7fdaf01580e0 2023-10-30T12:34:58.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.522+0000 7fdaf4e46640 1 -- 172.21.15.12:0/2563568447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf01f9cd0 con 0x7fdaf01580e0 2023-10-30T12:34:58.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.526+0000 7fdaf4e46640 1 -- 172.21.15.12:0/2563568447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf01fa4d0 con 0x7fdaf01580e0 2023-10-30T12:34:58.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.526+0000 7fdadeffd640 1 -- 172.21.15.12:0/2563568447 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdae80b9940 con 0x7fdaf01580e0 2023-10-30T12:34:58.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.526+0000 7fdadeffd640 1 --2- 172.21.15.12:0/2563568447 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdad00664d0 0x7fdad0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.526+0000 7fdadeffd640 1 -- 172.21.15.12:0/2563568447 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdae811ed20 con 0x7fdaf01580e0 2023-10-30T12:34:58.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.526+0000 7fdaee59c640 1 --2- 172.21.15.12:0/2563568447 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdad00664d0 0x7fdad0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.530+0000 7fdaee59c640 1 --2- 172.21.15.12:0/2563568447 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdad00664d0 0x7fdad0068990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdae0000f70 tx=0x7fdae0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:58.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.530+0000 7fdadeffd640 1 -- 172.21.15.12:0/2563568447 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdaf01fa4d0 con 0x7fdaf01580e0 2023-10-30T12:34:58.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.690+0000 7fdaf4e46640 1 -- 172.21.15.12:0/2563568447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdaf01fa4d0 con 0x7fdaf01580e0 2023-10-30T12:34:58.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.690+0000 7fdadeffd640 1 -- 172.21.15.12:0/2563568447 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12850 (secure 0 0 0) 0x7fdaf01fa4d0 con 0x7fdaf01580e0 2023-10-30T12:34:58.695 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:58.695 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":18,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:34:56.098941+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:34:58.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.690+0000 7fdaf4e46640 1 -- 172.21.15.12:0/2563568447 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdad00664d0 msgr2=0x7fdad0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.690+0000 7fdaf4e46640 1 --2- 172.21.15.12:0/2563568447 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdad00664d0 0x7fdad0068990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdae0000f70 tx=0x7fdae0009040 comp rx=0 tx=0).stop 2023-10-30T12:34:58.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.690+0000 7fdaf4e46640 1 -- 172.21.15.12:0/2563568447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaf01580e0 msgr2=0x7fdaf014cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.690+0000 7fdaf4e46640 1 --2- 172.21.15.12:0/2563568447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaf01580e0 0x7fdaf014cf40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdae80b0040 tx=0x7fdae80196f0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.690+0000 7fdaf4e46640 1 -- 172.21.15.12:0/2563568447 shutdown_connections 2023-10-30T12:34:58.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.690+0000 7fdaf4e46640 1 --2- 172.21.15.12:0/2563568447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaf01580e0 0x7fdaf014cf40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.690+0000 7fdaf4e46640 1 --2- 172.21.15.12:0/2563568447 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdad00664d0 0x7fdad0068990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.690+0000 7fdaf4e46640 1 --2- 172.21.15.12:0/2563568447 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaf00c6540 0x7fdaf01f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.690+0000 7fdaf4e46640 1 --2- 172.21.15.12:0/2563568447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaf0070740 0x7fdaf0146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.694+0000 7fdaf4e46640 1 -- 172.21.15.12:0/2563568447 >> 172.21.15.12:0/2563568447 conn(0x7fdaf00c7910 msgr2=0x7fdaf0153a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.694+0000 7fdaf4e46640 1 -- 172.21.15.12:0/2563568447 shutdown_connections 2023-10-30T12:34:58.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.694+0000 7fdaf4e46640 1 -- 172.21.15.12:0/2563568447 wait complete. 2023-10-30T12:34:58.716 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:34:53.017612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:34:58.717 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:34:58.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.858+0000 7f6bc74c9640 1 Processor -- start 2023-10-30T12:34:58.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.858+0000 7f6bc74c9640 1 -- start start 2023-10-30T12:34:58.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc74c9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6bc006f880 0x7f6bc006a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc74c9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc00c6cb0 con 0x7f6bc00c5810 2023-10-30T12:34:58.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc74c9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc00c6620 con 0x7f6bc006a7b0 2023-10-30T12:34:58.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc74c9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc0071d90 con 0x7f6bc006f880 2023-10-30T12:34:58.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc5265640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f6bc00c5810 0x7f6bc00c5bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56718/0 (socket says 172.21.15.12:56718) 2023-10-30T12:34:58.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc5265640 1 -- 172.21.15.12:0/3995992100 learned_addr learned my addr 172.21.15.12:0/3995992100 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:34:58.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc4a64640 1 --2- 172.21.15.12:0/3995992100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6bc006f880 0x7f6bc006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739955672 0 0) 0x7f6bc00c6cb0 con 0x7f6bc00c5810 2023-10-30T12:34:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bac003530 con 0x7f6bc00c5810 2023-10-30T12:34:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2647887751 0 0) 0x7f6bc00c6620 con 0x7f6bc006a7b0 2023-10-30T12:34:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bc00c6cb0 con 0x7f6bc006a7b0 2023-10-30T12:34:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3395414037 0 0) 0x7f6bac003530 con 0x7f6bc00c5810 2023-10-30T12:34:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6bc006f880 msgr2=0x7f6bc006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:58.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 --2- 172.21.15.12:0/3995992100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6bc006f880 0x7f6bc006a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 >> v1:172.21.15.154:6789/0 conn(0x7f6bc006a7b0 legacy=0x7f6bc0161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc01582f0 con 0x7f6bc00c5810 2023-10-30T12:34:58.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f6bbc001800 con 0x7f6bc00c5810 2023-10-30T12:34:58.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 >> v1:172.21.15.12:6789/0 conn(0x7f6bc00c5810 legacy=0x7f6bc00c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 --2- 172.21.15.12:0/3995992100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bac003b40 0x7f6bac003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 --2- 172.21.15.12:0/3995992100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bac0047d0 0x7f6bac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 --2- 172.21.15.12:0/3995992100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6bc006f880 0x7f6bac005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac00bb20 con 0x7f6bac0047d0 2023-10-30T12:34:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac00bc90 con 0x7f6bac003b40 2023-10-30T12:34:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac00be50 con 0x7f6bc006f880 2023-10-30T12:34:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc4a64640 1 --2- 172.21.15.12:0/3995992100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bac0047d0 0x7f6bac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc5265640 1 --2- 172.21.15.12:0/3995992100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bac003b40 0x7f6bac003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc5a66640 1 --2- 172.21.15.12:0/3995992100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6bc006f880 0x7f6bac005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc4a64640 1 -- 172.21.15.12:0/3995992100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6bc006f880 msgr2=0x7f6bac005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:58.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc4a64640 1 --2- 172.21.15.12:0/3995992100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6bc006f880 0x7f6bac005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc4a64640 1 -- 172.21.15.12:0/3995992100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bac003b40 msgr2=0x7f6bac003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:58.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc4a64640 1 --2- 172.21.15.12:0/3995992100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bac003b40 0x7f6bac003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc4a64640 1 -- 172.21.15.12:0/3995992100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac003530 con 0x7f6bac0047d0 2023-10-30T12:34:58.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.862+0000 7f6bc5265640 1 --2- 172.21.15.12:0/3995992100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bac003b40 0x7f6bac003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:34:58.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.866+0000 7f6bc4a64640 1 --2- 172.21.15.12:0/3995992100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bac0047d0 0x7f6bac004c50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009ae0 tx=0x7f6bb001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b65a8d650e5143 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:58.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.866+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00a6430 con 0x7f6bac0047d0 2023-10-30T12:34:58.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.866+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb00a6590 con 0x7f6bac0047d0 2023-10-30T12:34:58.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.866+0000 7f6bb7fff640 1 -- 172.21.15.12:0/3995992100 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00a6940 con 0x7f6bac0047d0 2023-10-30T12:34:58.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.866+0000 7f6bc74c9640 1 -- 172.21.15.12:0/3995992100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bac0047d0 msgr2=0x7f6bac004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.866+0000 7f6bc74c9640 1 --2- 172.21.15.12:0/3995992100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bac0047d0 0x7f6bac004c50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009ae0 tx=0x7f6bb001ca50 comp rx=0 tx=0).stop 2023-10-30T12:34:58.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.866+0000 7f6bc5a66640 1 -- 172.21.15.12:0/3995992100 reap_dead start 2023-10-30T12:34:58.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.866+0000 7f6bc74c9640 1 -- 172.21.15.12:0/3995992100 shutdown_connections 2023-10-30T12:34:58.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.866+0000 7f6bc74c9640 1 -- 172.21.15.12:0/3995992100 >> 172.21.15.12:0/3995992100 conn(0x7f6bc00c7910 msgr2=0x7f6bc00c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:58.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.866+0000 7f6bc74c9640 1 -- 172.21.15.12:0/3995992100 shutdown_connections 2023-10-30T12:34:58.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.866+0000 7f6bc74c9640 1 -- 172.21.15.12:0/3995992100 wait complete. 2023-10-30T12:34:58.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc74c9640 1 Processor -- start 2023-10-30T12:34:58.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc74c9640 1 -- start start 2023-10-30T12:34:58.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc74c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bac0047d0 0x7f6bc00ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc74c9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6bc00c25f0 0x7f6bc00c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc74c9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc01f3800 0x7f6bc01f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc74c9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc014f600 con 0x7f6bc01f3800 2023-10-30T12:34:58.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc74c9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc014e380 con 0x7f6bac0047d0 2023-10-30T12:34:58.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc74c9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc01f41c0 con 0x7f6bc00c25f0 2023-10-30T12:34:58.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc5a66640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc01f3800 0x7f6bc01f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc5a66640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc01f3800 0x7f6bc01f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33456/0 (socket says 172.21.15.12:33456) 2023-10-30T12:34:58.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc5a66640 1 -- 172.21.15.12:0/4075169976 learned_addr learned my addr 172.21.15.12:0/4075169976 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:34:58.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc5265640 1 --2- 172.21.15.12:0/4075169976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bac0047d0 0x7f6bc00ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc4a64640 1 --2- 172.21.15.12:0/4075169976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6bc00c25f0 0x7f6bc00c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc5a66640 1 -- 172.21.15.12:0/4075169976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6bc00c25f0 msgr2=0x7f6bc00c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc5a66640 1 --2- 172.21.15.12:0/4075169976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6bc00c25f0 0x7f6bc00c2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc5a66640 1 -- 172.21.15.12:0/4075169976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bac0047d0 msgr2=0x7f6bc00ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:58.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc5a66640 1 --2- 172.21.15.12:0/4075169976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bac0047d0 0x7f6bc00ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:58.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc5a66640 1 -- 172.21.15.12:0/4075169976 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc014d4d0 con 0x7f6bc01f3800 2023-10-30T12:34:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc5a66640 1 --2- 172.21.15.12:0/4075169976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc01f3800 0x7f6bc01f3c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6bc006b330 tx=0x7f6bb800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bb5ffb640 1 -- 172.21.15.12:0/4075169976 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8003980 con 0x7f6bc01f3800 2023-10-30T12:34:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bb5ffb640 1 -- 172.21.15.12:0/4075169976 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb8003ae0 con 0x7f6bc01f3800 2023-10-30T12:34:58.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bb5ffb640 1 -- 172.21.15.12:0/4075169976 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8003e90 con 0x7f6bc01f3800 2023-10-30T12:34:58.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc74c9640 1 -- 172.21.15.12:0/4075169976 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc014d770 con 0x7f6bc01f3800 2023-10-30T12:34:58.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc74c9640 1 -- 172.21.15.12:0/4075169976 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc014dd30 con 0x7f6bc01f3800 2023-10-30T12:34:58.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.870+0000 7f6bc74c9640 1 -- 172.21.15.12:0/4075169976 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc00c75d0 con 0x7f6bc01f3800 2023-10-30T12:34:58.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.874+0000 7f6bb5ffb640 1 -- 172.21.15.12:0/4075169976 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6bb80050d0 con 0x7f6bc01f3800 2023-10-30T12:34:58.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.874+0000 7f6bb5ffb640 1 --2- 172.21.15.12:0/4075169976 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6ba40664d0 0x7f6ba4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:58.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.874+0000 7f6bb5ffb640 1 -- 172.21.15.12:0/4075169976 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6bb80838b0 con 0x7f6bc01f3800 2023-10-30T12:34:58.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.874+0000 7f6bc5265640 1 --2- 172.21.15.12:0/4075169976 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6ba40664d0 0x7f6ba4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:58.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.874+0000 7f6bc5265640 1 --2- 172.21.15.12:0/4075169976 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6ba40664d0 0x7f6ba4068990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc000c10 tx=0x7f6bbc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:58.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:58.878+0000 7f6bb5ffb640 1 -- 172.21.15.12:0/4075169976 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6bb8051000 con 0x7f6bc01f3800 2023-10-30T12:34:59.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.034+0000 7f6bc74c9640 1 -- 172.21.15.12:0/4075169976 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6bc004e170 con 0x7f6bc01f3800 2023-10-30T12:34:59.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.034+0000 7f6bb5ffb640 1 -- 172.21.15.12:0/4075169976 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6bb8054620 con 0x7f6bc01f3800 2023-10-30T12:34:59.039 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-10-30T12:34:59.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.038+0000 7f6bc74c9640 1 -- 172.21.15.12:0/4075169976 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6ba40664d0 msgr2=0x7f6ba4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:59.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.038+0000 7f6bc74c9640 1 --2- 172.21.15.12:0/4075169976 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6ba40664d0 0x7f6ba4068990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc000c10 tx=0x7f6bbc009040 comp rx=0 tx=0).stop 2023-10-30T12:34:59.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.038+0000 7f6bc74c9640 1 -- 172.21.15.12:0/4075169976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc01f3800 msgr2=0x7f6bc01f3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:59.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.038+0000 7f6bc74c9640 1 --2- 172.21.15.12:0/4075169976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc01f3800 0x7f6bc01f3c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6bc006b330 tx=0x7f6bb800eb90 comp rx=0 tx=0).stop 2023-10-30T12:34:59.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.038+0000 7f6bc74c9640 1 -- 172.21.15.12:0/4075169976 shutdown_connections 2023-10-30T12:34:59.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.038+0000 7f6bc74c9640 1 --2- 172.21.15.12:0/4075169976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc01f3800 0x7f6bc01f3c80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:59.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.038+0000 7f6bc74c9640 1 --2- 172.21.15.12:0/4075169976 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6ba40664d0 0x7f6ba4068990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:59.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.038+0000 7f6bc74c9640 1 --2- 172.21.15.12:0/4075169976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6bc00c25f0 0x7f6bc00c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:59.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.038+0000 7f6bc74c9640 1 --2- 172.21.15.12:0/4075169976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6bac0047d0 0x7f6bc00ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:59.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.038+0000 7f6bc74c9640 1 -- 172.21.15.12:0/4075169976 >> 172.21.15.12:0/4075169976 conn(0x7f6bc00c7910 msgr2=0x7f6bc00c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:59.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.038+0000 7f6bc74c9640 1 -- 172.21.15.12:0/4075169976 shutdown_connections 2023-10-30T12:34:59.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.038+0000 7f6bc74c9640 1 -- 172.21.15.12:0/4075169976 wait complete. 2023-10-30T12:34:59.060 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:34:59.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb1e0ea640 1 Processor -- start 2023-10-30T12:34:59.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb1e0ea640 1 -- start start 2023-10-30T12:34:59.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb1e0ea640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdb180c6b80 0x7fdb180bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:59.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb1e0ea640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdb180c6f60 0x7fdb180c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:59.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb1e0ea640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb180c0a50 con 0x7fdb180c8430 2023-10-30T12:34:59.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb1e0ea640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb1806f6e0 con 0x7fdb180c6f60 2023-10-30T12:34:59.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb1e0ea640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb1806f9c0 con 0x7fdb180c6b80 2023-10-30T12:34:59.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb16ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdb180c6f60 0x7fdb180c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:59.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb17fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdb180c8430 0x7fdb1813ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56724/0 (socket says 172.21.15.12:56724) 2023-10-30T12:34:59.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb17fff640 1 -- 172.21.15.12:0/3798605593 learned_addr learned my addr 172.21.15.12:0/3798605593 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:34:59.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb177fe640 1 --2- 172.21.15.12:0/3798605593 >> v2:172.21.15.154:3300/0 conn(0x7fdb180c6b80 0x7fdb180bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:59.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb167fc640 1 -- 172.21.15.12:0/3798605593 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435258922 0 0) 0x7fdb180c0a50 con 0x7fdb180c8430 2023-10-30T12:34:59.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb167fc640 1 -- 172.21.15.12:0/3798605593 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb00003530 con 0x7fdb180c8430 2023-10-30T12:34:59.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb16ffd640 1 -- 172.21.15.12:0/3798605593 >> v2:172.21.15.154:3300/0 conn(0x7fdb180c6b80 msgr2=0x7fdb180bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:34:59.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb16ffd640 1 --2- 172.21.15.12:0/3798605593 >> v2:172.21.15.154:3300/0 conn(0x7fdb180c6b80 0x7fdb180bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:59.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb16ffd640 1 -- 172.21.15.12:0/3798605593 >> v1:172.21.15.12:6789/0 conn(0x7fdb180c8430 legacy=0x7fdb1813ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:59.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.202+0000 7fdb16ffd640 1 -- 172.21.15.12:0/3798605593 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1806fca0 con 0x7fdb180c6f60 2023-10-30T12:34:59.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.206+0000 7fdb16ffd640 1 --2- 172.21.15.12:0/3798605593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb180c6f60 0x7fdb180c0400 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdb1806f1b0 tx=0x7fdb0c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f9ceda2d100658 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:59.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.206+0000 7fdb167fc640 1 -- 172.21.15.12:0/3798605593 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c0ab600 con 0x7fdb180c6f60 2023-10-30T12:34:59.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.206+0000 7fdb167fc640 1 -- 172.21.15.12:0/3798605593 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c0a7050 con 0x7fdb180c6f60 2023-10-30T12:34:59.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.206+0000 7fdb167fc640 1 -- 172.21.15.12:0/3798605593 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c0ab980 con 0x7fdb180c6f60 2023-10-30T12:34:59.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.206+0000 7fdb1e0ea640 1 -- 172.21.15.12:0/3798605593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb180c6f60 msgr2=0x7fdb180c0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:59.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.206+0000 7fdb1e0ea640 1 --2- 172.21.15.12:0/3798605593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb180c6f60 0x7fdb180c0400 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdb1806f1b0 tx=0x7fdb0c01c460 comp rx=0 tx=0).stop 2023-10-30T12:34:59.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.206+0000 7fdb1e0ea640 1 -- 172.21.15.12:0/3798605593 shutdown_connections 2023-10-30T12:34:59.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.206+0000 7fdb1e0ea640 1 --2- 172.21.15.12:0/3798605593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb180c6f60 0x7fdb180c0400 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:59.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.206+0000 7fdb1e0ea640 1 --2- 172.21.15.12:0/3798605593 >> v2:172.21.15.154:3300/0 conn(0x7fdb180c6b80 0x7fdb180bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:59.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.206+0000 7fdb1e0ea640 1 -- 172.21.15.12:0/3798605593 >> 172.21.15.12:0/3798605593 conn(0x7fdb180c7910 msgr2=0x7fdb180c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:59.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.206+0000 7fdb1e0ea640 1 -- 172.21.15.12:0/3798605593 shutdown_connections 2023-10-30T12:34:59.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.206+0000 7fdb1e0ea640 1 -- 172.21.15.12:0/3798605593 wait complete. 2023-10-30T12:34:59.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb1e0ea640 1 Processor -- start 2023-10-30T12:34:59.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb1e0ea640 1 -- start start 2023-10-30T12:34:59.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb1e0ea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb180c6b80 0x7fdb1813afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:59.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb1e0ea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb180c6f60 0x7fdb181f0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:59.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb1e0ea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb180c8430 0x7fdb18135b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:59.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb1e0ea640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb180c8e80 con 0x7fdb180c6b80 2023-10-30T12:34:59.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb1e0ea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1813b670 con 0x7fdb180c6f60 2023-10-30T12:34:59.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb1e0ea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181efca0 con 0x7fdb180c8430 2023-10-30T12:34:59.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb180c6b80 0x7fdb1813afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:59.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb180c6b80 0x7fdb1813afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33482/0 (socket says 172.21.15.12:33482) 2023-10-30T12:34:59.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb177fe640 1 -- 172.21.15.12:0/1982892749 learned_addr learned my addr 172.21.15.12:0/1982892749 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:34:59.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb16ffd640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb180c6f60 0x7fdb181f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:59.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb17fff640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb180c8430 0x7fdb18135b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:59.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb177fe640 1 -- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb180c8430 msgr2=0x7fdb18135b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:59.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb177fe640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb180c8430 0x7fdb18135b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:59.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb177fe640 1 -- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb180c6f60 msgr2=0x7fdb181f0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:59.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb177fe640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb180c6f60 0x7fdb181f0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:59.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb177fe640 1 -- 172.21.15.12:0/1982892749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c009420 con 0x7fdb180c6b80 2023-10-30T12:34:59.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb16ffd640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb180c6f60 0x7fdb181f0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:34:59.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb17fff640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb180c8430 0x7fdb18135b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:34:59.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb177fe640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb180c6b80 0x7fdb1813afa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800e930 tx=0x7fdb0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:59.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdafffff640 1 -- 172.21.15.12:0/1982892749 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08009690 con 0x7fdb180c6b80 2023-10-30T12:34:59.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdafffff640 1 -- 172.21.15.12:0/1982892749 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0800ca00 con 0x7fdb180c6b80 2023-10-30T12:34:59.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdafffff640 1 -- 172.21.15.12:0/1982892749 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0800cd70 con 0x7fdb180c6b80 2023-10-30T12:34:59.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb1e0ea640 1 -- 172.21.15.12:0/1982892749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb181393f0 con 0x7fdb180c6b80 2023-10-30T12:34:59.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.210+0000 7fdb1e0ea640 1 -- 172.21.15.12:0/1982892749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb181eff40 con 0x7fdb180c6b80 2023-10-30T12:34:59.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.214+0000 7fdb1e0ea640 1 -- 172.21.15.12:0/1982892749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb181efb00 con 0x7fdb180c6b80 2023-10-30T12:34:59.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.214+0000 7fdafffff640 1 -- 172.21.15.12:0/1982892749 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdb08013470 con 0x7fdb180c6b80 2023-10-30T12:34:59.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.214+0000 7fdafffff640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdb040664a0 0x7fdb04068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:34:59.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.214+0000 7fdafffff640 1 -- 172.21.15.12:0/1982892749 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdb08083d90 con 0x7fdb180c6b80 2023-10-30T12:34:59.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.214+0000 7fdb16ffd640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdb040664a0 0x7fdb04068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:34:59.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.214+0000 7fdb16ffd640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdb040664a0 0x7fdb04068960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0ad000 tx=0x7fdb0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:34:59.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.218+0000 7fdafffff640 1 -- 172.21.15.12:0/1982892749 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdb181efb00 con 0x7fdb180c6b80 2023-10-30T12:34:59.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.406+0000 7fdb1e0ea640 1 -- 172.21.15.12:0/1982892749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb181efb00 con 0x7fdb180c6b80 2023-10-30T12:34:59.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.406+0000 7fdafffff640 1 -- 172.21.15.12:0/1982892749 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7fdb08021540 con 0x7fdb180c6b80 2023-10-30T12:34:59.411 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:34:59.411 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-30T12:34:59.412 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 2 2023-10-30T12:34:59.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.410+0000 7fdb1e0ea640 1 -- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdb040664a0 msgr2=0x7fdb04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:59.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.410+0000 7fdb1e0ea640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdb040664a0 0x7fdb04068960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0ad000 tx=0x7fdb0c008040 comp rx=0 tx=0).stop 2023-10-30T12:34:59.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.410+0000 7fdb1e0ea640 1 -- 172.21.15.12:0/1982892749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb180c6b80 msgr2=0x7fdb1813afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:34:59.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.410+0000 7fdb1e0ea640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb180c6b80 0x7fdb1813afa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800e930 tx=0x7fdb0800edf0 comp rx=0 tx=0).stop 2023-10-30T12:34:59.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.410+0000 7fdb1e0ea640 1 -- 172.21.15.12:0/1982892749 shutdown_connections 2023-10-30T12:34:59.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.410+0000 7fdb1e0ea640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdb040664a0 0x7fdb04068960 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:59.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.410+0000 7fdb1e0ea640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb180c8430 0x7fdb18135b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:59.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.410+0000 7fdb1e0ea640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb180c6f60 0x7fdb181f0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:59.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.410+0000 7fdb1e0ea640 1 --2- 172.21.15.12:0/1982892749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb180c6b80 0x7fdb1813afa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:34:59.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.410+0000 7fdb1e0ea640 1 -- 172.21.15.12:0/1982892749 >> 172.21.15.12:0/1982892749 conn(0x7fdb180c7910 msgr2=0x7fdb18137790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:34:59.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.410+0000 7fdb1e0ea640 1 -- 172.21.15.12:0/1982892749 shutdown_connections 2023-10-30T12:34:59.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:34:59.410+0000 7fdb1e0ea640 1 -- 172.21.15.12:0/1982892749 wait complete. 2023-10-30T12:35:01.433 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:35:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d8d52a640 1 Processor -- start 2023-10-30T12:35:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d8d52a640 1 -- start start 2023-10-30T12:35:01.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d8d52a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d881538a0 0x7f8d8815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:01.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d8d52a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8d880cf160 0x7f8d880cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:01.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d8d52a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d880cb800 con 0x7f8d880ca0e0 2023-10-30T12:35:01.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d8d52a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8815dc30 con 0x7f8d880cf160 2023-10-30T12:35:01.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d8d52a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d880cb330 con 0x7f8d881538a0 2023-10-30T12:35:01.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d7ffff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8d880ca0e0 0x7f8d8815a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54176/0 (socket says 172.21.15.12:54176) 2023-10-30T12:35:01.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d7ffff640 1 -- 172.21.15.12:0/3417956307 learned_addr learned my addr 172.21.15.12:0/3417956307 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:01.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d877fe640 1 --2- 172.21.15.12:0/3417956307 >> v2:172.21.15.12:3300/0 conn(0x7f8d880cf160 0x7f8d880cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:01.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d86ffd640 1 --2- 172.21.15.12:0/3417956307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d881538a0 0x7f8d8815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:01.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d877fe640 1 -- 172.21.15.12:0/3417956307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d881538a0 msgr2=0x7f8d8815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:01.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d877fe640 1 --2- 172.21.15.12:0/3417956307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d881538a0 0x7f8d8815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d877fe640 1 -- 172.21.15.12:0/3417956307 >> v1:172.21.15.12:6789/0 conn(0x7f8d880ca0e0 legacy=0x7f8d8815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:01.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d877fe640 1 -- 172.21.15.12:0/3417956307 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d880cad70 con 0x7f8d880cf160 2023-10-30T12:35:01.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.586+0000 7f8d867fc640 1 -- 172.21.15.12:0/3417956307 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440559534 0 0) 0x7f8d880cb800 con 0x7f8d880ca0e0 2023-10-30T12:35:01.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.590+0000 7f8d877fe640 1 --2- 172.21.15.12:0/3417956307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d880cf160 0x7f8d880cf590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009d90 tx=0x7f8d8001f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e368a5864bba27 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:01.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.590+0000 7f8d867fc640 1 -- 172.21.15.12:0/3417956307 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d800af600 con 0x7f8d880cf160 2023-10-30T12:35:01.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.590+0000 7f8d867fc640 1 -- 172.21.15.12:0/3417956307 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d800ab050 con 0x7f8d880cf160 2023-10-30T12:35:01.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.590+0000 7f8d867fc640 1 -- 172.21.15.12:0/3417956307 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d800af980 con 0x7f8d880cf160 2023-10-30T12:35:01.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.590+0000 7f8d8d52a640 1 -- 172.21.15.12:0/3417956307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d880cf160 msgr2=0x7f8d880cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:01.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.590+0000 7f8d8d52a640 1 --2- 172.21.15.12:0/3417956307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d880cf160 0x7f8d880cf590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009d90 tx=0x7f8d8001f130 comp rx=0 tx=0).stop 2023-10-30T12:35:01.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.590+0000 7f8d8d52a640 1 -- 172.21.15.12:0/3417956307 shutdown_connections 2023-10-30T12:35:01.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.590+0000 7f8d8d52a640 1 --2- 172.21.15.12:0/3417956307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d880cf160 0x7f8d880cf590 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.590+0000 7f8d8d52a640 1 --2- 172.21.15.12:0/3417956307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d881538a0 0x7f8d8815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.590+0000 7f8d8d52a640 1 -- 172.21.15.12:0/3417956307 >> 172.21.15.12:0/3417956307 conn(0x7f8d8806bfe0 msgr2=0x7f8d8806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:01.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.590+0000 7f8d8d52a640 1 -- 172.21.15.12:0/3417956307 shutdown_connections 2023-10-30T12:35:01.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.590+0000 7f8d8d52a640 1 -- 172.21.15.12:0/3417956307 wait complete. 2023-10-30T12:35:01.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d8d52a640 1 Processor -- start 2023-10-30T12:35:01.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d8d52a640 1 -- start start 2023-10-30T12:35:01.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d8d52a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d880ca0e0 0x7f8d8815c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:01.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d8d52a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d880cf160 0x7f8d88144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:01.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d8d52a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d881538a0 0x7f8d88158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:01.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d8d52a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8814dc60 con 0x7f8d880ca0e0 2023-10-30T12:35:01.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d8d52a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d881448d0 con 0x7f8d881538a0 2023-10-30T12:35:01.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d8d52a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8815ca10 con 0x7f8d880cf160 2023-10-30T12:35:01.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d86ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d880ca0e0 0x7f8d8815c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:01.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d86ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d880ca0e0 0x7f8d8815c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36732/0 (socket says 172.21.15.12:36732) 2023-10-30T12:35:01.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d86ffd640 1 -- 172.21.15.12:0/569514976 learned_addr learned my addr 172.21.15.12:0/569514976 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:01.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d7ffff640 1 --2- 172.21.15.12:0/569514976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d880cf160 0x7f8d88144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:01.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d877fe640 1 --2- 172.21.15.12:0/569514976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d881538a0 0x7f8d88158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:01.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d86ffd640 1 -- 172.21.15.12:0/569514976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d880cf160 msgr2=0x7f8d88144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:01.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d86ffd640 1 --2- 172.21.15.12:0/569514976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d880cf160 0x7f8d88144390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d86ffd640 1 -- 172.21.15.12:0/569514976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d881538a0 msgr2=0x7f8d88158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:01.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d86ffd640 1 --2- 172.21.15.12:0/569514976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d881538a0 0x7f8d88158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d86ffd640 1 -- 172.21.15.12:0/569514976 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d80009a70 con 0x7f8d880ca0e0 2023-10-30T12:35:01.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d877fe640 1 --2- 172.21.15.12:0/569514976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d881538a0 0x7f8d88158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:35:01.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d86ffd640 1 --2- 172.21.15.12:0/569514976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d880ca0e0 0x7f8d8815c420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800e870 tx=0x7f8d7800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:01.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d7f7fe640 1 -- 172.21.15.12:0/569514976 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d78009510 con 0x7f8d880ca0e0 2023-10-30T12:35:01.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d7f7fe640 1 -- 172.21.15.12:0/569514976 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7800eec0 con 0x7f8d880ca0e0 2023-10-30T12:35:01.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d7f7fe640 1 -- 172.21.15.12:0/569514976 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7800cb40 con 0x7f8d880ca0e0 2023-10-30T12:35:01.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d8d52a640 1 -- 172.21.15.12:0/569514976 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8815aa30 con 0x7f8d880ca0e0 2023-10-30T12:35:01.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d8d52a640 1 -- 172.21.15.12:0/569514976 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8815acf0 con 0x7f8d880ca0e0 2023-10-30T12:35:01.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.594+0000 7f8d8d52a640 1 -- 172.21.15.12:0/569514976 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d880b2a50 con 0x7f8d880ca0e0 2023-10-30T12:35:01.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.598+0000 7f8d7f7fe640 1 -- 172.21.15.12:0/569514976 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8d7800cca0 con 0x7f8d880ca0e0 2023-10-30T12:35:01.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.598+0000 7f8d7f7fe640 1 --2- 172.21.15.12:0/569514976 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8d680664a0 0x7f8d68068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.598+0000 7f8d7f7fe640 1 -- 172.21.15.12:0/569514976 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8d78083d00 con 0x7f8d880ca0e0 2023-10-30T12:35:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.598+0000 7f8d7ffff640 1 --2- 172.21.15.12:0/569514976 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8d680664a0 0x7f8d68068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.598+0000 7f8d7ffff640 1 --2- 172.21.15.12:0/569514976 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8d680664a0 0x7f8d68068960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8d880c5b90 tx=0x7f8d70007b60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:01.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.602+0000 7f8d7f7fe640 1 -- 172.21.15.12:0/569514976 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8d78051430 con 0x7f8d880ca0e0 2023-10-30T12:35:01.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.790+0000 7f8d8d52a640 1 -- 172.21.15.12:0/569514976 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d8804e170 con 0x7f8d880ca0e0 2023-10-30T12:35:01.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.790+0000 7f8d7f7fe640 1 -- 172.21.15.12:0/569514976 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+5277 (secure 0 0 0) 0x7f8d78054a50 con 0x7f8d880ca0e0 2023-10-30T12:35:01.796 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:01.796 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4285,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-30T12:35:01.796 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 3 2023-10-30T12:35:01.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.794+0000 7f8d8d52a640 1 -- 172.21.15.12:0/569514976 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8d680664a0 msgr2=0x7f8d68068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:01.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.794+0000 7f8d8d52a640 1 --2- 172.21.15.12:0/569514976 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8d680664a0 0x7f8d68068960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8d880c5b90 tx=0x7f8d70007b60 comp rx=0 tx=0).stop 2023-10-30T12:35:01.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.794+0000 7f8d8d52a640 1 -- 172.21.15.12:0/569514976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d880ca0e0 msgr2=0x7f8d8815c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:01.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.794+0000 7f8d8d52a640 1 --2- 172.21.15.12:0/569514976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d880ca0e0 0x7f8d8815c420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800e870 tx=0x7f8d7800ec70 comp rx=0 tx=0).stop 2023-10-30T12:35:01.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.794+0000 7f8d8d52a640 1 -- 172.21.15.12:0/569514976 shutdown_connections 2023-10-30T12:35:01.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.794+0000 7f8d8d52a640 1 --2- 172.21.15.12:0/569514976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d881538a0 0x7f8d88158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.794+0000 7f8d8d52a640 1 --2- 172.21.15.12:0/569514976 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8d680664a0 0x7f8d68068960 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.794+0000 7f8d8d52a640 1 --2- 172.21.15.12:0/569514976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d880cf160 0x7f8d88144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.794+0000 7f8d8d52a640 1 --2- 172.21.15.12:0/569514976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d880ca0e0 0x7f8d8815c420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.794+0000 7f8d8d52a640 1 -- 172.21.15.12:0/569514976 >> 172.21.15.12:0/569514976 conn(0x7f8d8806bfe0 msgr2=0x7f8d880c4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:01.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.794+0000 7f8d8d52a640 1 -- 172.21.15.12:0/569514976 shutdown_connections 2023-10-30T12:35:01.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.794+0000 7f8d8d52a640 1 -- 172.21.15.12:0/569514976 wait complete. 2023-10-30T12:35:01.817 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:35:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87bb7c5640 1 Processor -- start 2023-10-30T12:35:01.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87bb7c5640 1 -- start start 2023-10-30T12:35:01.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87bb7c5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f87b40cf160 0x7f87b40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:01.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87bb7c5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b40cb800 con 0x7f87b41538a0 2023-10-30T12:35:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87bb7c5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b40cb330 con 0x7f87b415d6f0 2023-10-30T12:35:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87bb7c5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b40cad70 con 0x7f87b40cf160 2023-10-30T12:35:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87b9561640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f87b41538a0 0x7f87b40ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54190/0 (socket says 172.21.15.12:54190) 2023-10-30T12:35:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87b9561640 1 -- 172.21.15.12:0/1941436335 learned_addr learned my addr 172.21.15.12:0/1941436335 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87b9d62640 1 --2- 172.21.15.12:0/1941436335 >> v2:172.21.15.154:3300/0 conn(0x7f87b40cf160 0x7f87b40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87abfff640 1 -- 172.21.15.12:0/1941436335 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716387415 0 0) 0x7f87b40cb800 con 0x7f87b41538a0 2023-10-30T12:35:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87abfff640 1 -- 172.21.15.12:0/1941436335 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a0003530 con 0x7f87b41538a0 2023-10-30T12:35:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87abfff640 1 -- 172.21.15.12:0/1941436335 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3315264966 0 0) 0x7f87a0003530 con 0x7f87b41538a0 2023-10-30T12:35:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87abfff640 1 -- 172.21.15.12:0/1941436335 >> v2:172.21.15.154:3300/0 conn(0x7f87b40cf160 msgr2=0x7f87b40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87abfff640 1 --2- 172.21.15.12:0/1941436335 >> v2:172.21.15.154:3300/0 conn(0x7f87b40cf160 0x7f87b40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87abfff640 1 -- 172.21.15.12:0/1941436335 >> v1:172.21.15.154:6789/0 conn(0x7f87b415d6f0 legacy=0x7f87b415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87abfff640 1 -- 172.21.15.12:0/1941436335 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b40cb050 con 0x7f87b41538a0 2023-10-30T12:35:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87abfff640 1 -- 172.21.15.12:0/1941436335 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f87b0001680 con 0x7f87b41538a0 2023-10-30T12:35:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87abfff640 1 -- 172.21.15.12:0/1941436335 >> v1:172.21.15.12:6789/0 conn(0x7f87b41538a0 legacy=0x7f87b40ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87abfff640 1 --2- 172.21.15.12:0/1941436335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87a0003980 0x7f87a0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:01.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87abfff640 1 --2- 172.21.15.12:0/1941436335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87a00047d0 0x7f87a0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:01.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87abfff640 1 --2- 172.21.15.12:0/1941436335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87a0005190 0x7f87a000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:01.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87abfff640 1 -- 172.21.15.12:0/1941436335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40cad70 con 0x7f87a00047d0 2023-10-30T12:35:01.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87abfff640 1 -- 172.21.15.12:0/1941436335 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a000c040 con 0x7f87a0005190 2023-10-30T12:35:01.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87abfff640 1 -- 172.21.15.12:0/1941436335 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a000c200 con 0x7f87a0003980 2023-10-30T12:35:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87b8d60640 1 --2- 172.21.15.12:0/1941436335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87a00047d0 0x7f87a0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87b9561640 1 --2- 172.21.15.12:0/1941436335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87a0003980 0x7f87a0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.958+0000 7f87b9d62640 1 --2- 172.21.15.12:0/1941436335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87a0005190 0x7f87a000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87b8d60640 1 -- 172.21.15.12:0/1941436335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87a0003980 msgr2=0x7f87a0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87b8d60640 1 --2- 172.21.15.12:0/1941436335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87a0003980 0x7f87a0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87b8d60640 1 -- 172.21.15.12:0/1941436335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87a0005190 msgr2=0x7f87a000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87b8d60640 1 --2- 172.21.15.12:0/1941436335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87a0005190 0x7f87a000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87b8d60640 1 -- 172.21.15.12:0/1941436335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a0003530 con 0x7f87a00047d0 2023-10-30T12:35:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87b9d62640 1 -- 172.21.15.12:0/1941436335 reap_dead start 2023-10-30T12:35:01.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87b8d60640 1 --2- 172.21.15.12:0/1941436335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87a00047d0 0x7f87a0004c50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f87a4001780 tx=0x7f87a401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ccace201a4527c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:01.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87abfff640 1 -- 172.21.15.12:0/1941436335 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a40052d0 con 0x7f87a00047d0 2023-10-30T12:35:01.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87abfff640 1 -- 172.21.15.12:0/1941436335 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a40a7050 con 0x7f87a00047d0 2023-10-30T12:35:01.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87abfff640 1 -- 172.21.15.12:0/1941436335 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4003600 con 0x7f87a00047d0 2023-10-30T12:35:01.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87bb7c5640 1 -- 172.21.15.12:0/1941436335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87a00047d0 msgr2=0x7f87a0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:01.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87bb7c5640 1 --2- 172.21.15.12:0/1941436335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87a00047d0 0x7f87a0004c50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f87a4001780 tx=0x7f87a401c3e0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87bb7c5640 1 -- 172.21.15.12:0/1941436335 shutdown_connections 2023-10-30T12:35:01.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87bb7c5640 1 --2- 172.21.15.12:0/1941436335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87a00047d0 0x7f87a0004c50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87bb7c5640 1 -- 172.21.15.12:0/1941436335 >> 172.21.15.12:0/1941436335 conn(0x7f87b406bfe0 msgr2=0x7f87b406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:01.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.962+0000 7f87bb7c5640 1 -- 172.21.15.12:0/1941436335 shutdown_connections 2023-10-30T12:35:01.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87bb7c5640 1 -- 172.21.15.12:0/1941436335 wait complete. 2023-10-30T12:35:01.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87bb7c5640 1 Processor -- start 2023-10-30T12:35:01.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87bb7c5640 1 -- start start 2023-10-30T12:35:01.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87bb7c5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87a00047d0 0x7f87b40b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:01.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87bb7c5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87b414e610 0x7f87b40ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:01.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87bb7c5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87b414f440 0x7f87b414c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:01.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87bb7c5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40b3060 con 0x7f87a00047d0 2023-10-30T12:35:01.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87bb7c5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b414e180 con 0x7f87b414f440 2023-10-30T12:35:01.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87bb7c5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b415bfc0 con 0x7f87b414e610 2023-10-30T12:35:01.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87b9561640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87a00047d0 0x7f87b40b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:01.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87b9561640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87a00047d0 0x7f87b40b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36760/0 (socket says 172.21.15.12:36760) 2023-10-30T12:35:01.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87b9561640 1 -- 172.21.15.12:0/1874251243 learned_addr learned my addr 172.21.15.12:0/1874251243 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:01.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87b9d62640 1 --2- 172.21.15.12:0/1874251243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87b414f440 0x7f87b414c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:01.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87b8d60640 1 --2- 172.21.15.12:0/1874251243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87b414e610 0x7f87b40ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:01.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87b9561640 1 -- 172.21.15.12:0/1874251243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87b414e610 msgr2=0x7f87b40ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:01.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87b9561640 1 --2- 172.21.15.12:0/1874251243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87b414e610 0x7f87b40ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87b9561640 1 -- 172.21.15.12:0/1874251243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87b414f440 msgr2=0x7f87b414c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:01.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87b9561640 1 --2- 172.21.15.12:0/1874251243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87b414f440 0x7f87b414c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:01.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87b9561640 1 -- 172.21.15.12:0/1874251243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b414db50 con 0x7f87a00047d0 2023-10-30T12:35:01.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87b8d60640 1 --2- 172.21.15.12:0/1874251243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87b414e610 0x7f87b40ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:01.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87b9561640 1 --2- 172.21.15.12:0/1874251243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87a00047d0 0x7f87b40b42b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f87b0007290 tx=0x7f87b0007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:01.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87a9ffb640 1 -- 172.21.15.12:0/1874251243 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b000f9c0 con 0x7f87a00047d0 2023-10-30T12:35:01.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87a9ffb640 1 -- 172.21.15.12:0/1874251243 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b000fb20 con 0x7f87a00047d0 2023-10-30T12:35:01.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87bb7c5640 1 -- 172.21.15.12:0/1874251243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b414ea30 con 0x7f87a00047d0 2023-10-30T12:35:01.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.966+0000 7f87a9ffb640 1 -- 172.21.15.12:0/1874251243 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b000fed0 con 0x7f87a00047d0 2023-10-30T12:35:01.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.970+0000 7f87bb7c5640 1 -- 172.21.15.12:0/1874251243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b414de10 con 0x7f87a00047d0 2023-10-30T12:35:01.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.970+0000 7f87bb7c5640 1 -- 172.21.15.12:0/1874251243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b414dfa0 con 0x7f87a00047d0 2023-10-30T12:35:01.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.970+0000 7f87a9ffb640 1 -- 172.21.15.12:0/1874251243 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f87b001d460 con 0x7f87a00047d0 2023-10-30T12:35:01.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.970+0000 7f87a9ffb640 1 --2- 172.21.15.12:0/1874251243 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f87980664d0 0x7f8798068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:01.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.970+0000 7f87a9ffb640 1 -- 172.21.15.12:0/1874251243 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f87b0083c10 con 0x7f87a00047d0 2023-10-30T12:35:01.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.970+0000 7f87b8d60640 1 --2- 172.21.15.12:0/1874251243 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f87980664d0 0x7f8798068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:01.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.974+0000 7f87b8d60640 1 --2- 172.21.15.12:0/1874251243 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f87980664d0 0x7f8798068990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f87a4001750 tx=0x7f87a40011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:01.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:01.974+0000 7f87a9ffb640 1 -- 172.21.15.12:0/1874251243 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f87b0051470 con 0x7f87a00047d0 2023-10-30T12:35:02.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.134+0000 7f87bb7c5640 1 -- 172.21.15.12:0/1874251243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87b404e170 con 0x7f87a00047d0 2023-10-30T12:35:02.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.134+0000 7f87a9ffb640 1 -- 172.21.15.12:0/1874251243 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f87b0054a90 con 0x7f87a00047d0 2023-10-30T12:35:02.139 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:02.140 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":18,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:34:56.098941+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:35:02.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.138+0000 7f87bb7c5640 1 -- 172.21.15.12:0/1874251243 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f87980664d0 msgr2=0x7f8798068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.138+0000 7f87bb7c5640 1 --2- 172.21.15.12:0/1874251243 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f87980664d0 0x7f8798068990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f87a4001750 tx=0x7f87a40011a0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.138+0000 7f87bb7c5640 1 -- 172.21.15.12:0/1874251243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87a00047d0 msgr2=0x7f87b40b42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.138+0000 7f87bb7c5640 1 --2- 172.21.15.12:0/1874251243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87a00047d0 0x7f87b40b42b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f87b0007290 tx=0x7f87b0007750 comp rx=0 tx=0).stop 2023-10-30T12:35:02.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.138+0000 7f87bb7c5640 1 -- 172.21.15.12:0/1874251243 shutdown_connections 2023-10-30T12:35:02.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.138+0000 7f87bb7c5640 1 --2- 172.21.15.12:0/1874251243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87b414f440 0x7f87b414c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.138+0000 7f87bb7c5640 1 --2- 172.21.15.12:0/1874251243 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f87980664d0 0x7f8798068990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.138+0000 7f87bb7c5640 1 --2- 172.21.15.12:0/1874251243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87b414e610 0x7f87b40ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.138+0000 7f87bb7c5640 1 --2- 172.21.15.12:0/1874251243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87a00047d0 0x7f87b40b42b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.138+0000 7f87bb7c5640 1 -- 172.21.15.12:0/1874251243 >> 172.21.15.12:0/1874251243 conn(0x7f87b406bfe0 msgr2=0x7f87b40c49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:02.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.138+0000 7f87bb7c5640 1 -- 172.21.15.12:0/1874251243 shutdown_connections 2023-10-30T12:35:02.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.138+0000 7f87bb7c5640 1 -- 172.21.15.12:0/1874251243 wait complete. 2023-10-30T12:35:02.160 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:34:53.017612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:35:02.161 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:35:02.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.302+0000 7f9bfa91d640 1 Processor -- start 2023-10-30T12:35:02.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.302+0000 7f9bfa91d640 1 -- start start 2023-10-30T12:35:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.302+0000 7f9bfa91d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9bf41508c0 0x7f9bf40c6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.306+0000 7f9bfa91d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf40c0a30 con 0x7f9bf40722a0 2023-10-30T12:35:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.306+0000 7f9bfa91d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf406f650 con 0x7f9bf40c6e60 2023-10-30T12:35:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.306+0000 7f9bfa91d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf40c85b0 con 0x7f9bf41508c0 2023-10-30T12:35:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.306+0000 7f9bf37fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9bf40722a0 0x7f9bf40bfec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54206/0 (socket says 172.21.15.12:54206) 2023-10-30T12:35:02.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.306+0000 7f9bf37fe640 1 -- 172.21.15.12:0/647437295 learned_addr learned my addr 172.21.15.12:0/647437295 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:02.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.306+0000 7f9bf3fff640 1 --2- 172.21.15.12:0/647437295 >> v2:172.21.15.154:3300/0 conn(0x7f9bf41508c0 0x7f9bf40c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:02.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.306+0000 7f9bf2ffd640 1 -- 172.21.15.12:0/647437295 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812058160 0 0) 0x7f9bf40c0a30 con 0x7f9bf40722a0 2023-10-30T12:35:02.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.306+0000 7f9bf2ffd640 1 -- 172.21.15.12:0/647437295 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bdc003530 con 0x7f9bf40722a0 2023-10-30T12:35:02.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.306+0000 7f9bf3fff640 1 -- 172.21.15.12:0/647437295 >> v1:172.21.15.154:6789/0 conn(0x7f9bf40c6e60 legacy=0x7f9bf4156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.306+0000 7f9bf3fff640 1 -- 172.21.15.12:0/647437295 >> v1:172.21.15.12:6789/0 conn(0x7f9bf40722a0 legacy=0x7f9bf40bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.306+0000 7f9bf3fff640 1 -- 172.21.15.12:0/647437295 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf406f930 con 0x7f9bf41508c0 2023-10-30T12:35:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.310+0000 7f9bf3fff640 1 --2- 172.21.15.12:0/647437295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bf41508c0 0x7f9bf40c6920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9be40097a0 tx=0x7f9be401c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d405a1982b6d02e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:02.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.310+0000 7f9bf2ffd640 1 -- 172.21.15.12:0/647437295 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be40ab600 con 0x7f9bf41508c0 2023-10-30T12:35:02.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.310+0000 7f9bf2ffd640 1 -- 172.21.15.12:0/647437295 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be40a7050 con 0x7f9bf41508c0 2023-10-30T12:35:02.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.310+0000 7f9bf2ffd640 1 -- 172.21.15.12:0/647437295 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be40ab980 con 0x7f9bf41508c0 2023-10-30T12:35:02.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.310+0000 7f9bfa91d640 1 -- 172.21.15.12:0/647437295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bf41508c0 msgr2=0x7f9bf40c6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.310+0000 7f9bfa91d640 1 --2- 172.21.15.12:0/647437295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bf41508c0 0x7f9bf40c6920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9be40097a0 tx=0x7f9be401c3d0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.310+0000 7f9bfa91d640 1 -- 172.21.15.12:0/647437295 shutdown_connections 2023-10-30T12:35:02.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.310+0000 7f9bfa91d640 1 --2- 172.21.15.12:0/647437295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bf41508c0 0x7f9bf40c6920 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.310+0000 7f9bfa91d640 1 -- 172.21.15.12:0/647437295 >> 172.21.15.12:0/647437295 conn(0x7f9bf40c7910 msgr2=0x7f9bf40c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:02.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.310+0000 7f9bfa91d640 1 -- 172.21.15.12:0/647437295 shutdown_connections 2023-10-30T12:35:02.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.310+0000 7f9bfa91d640 1 -- 172.21.15.12:0/647437295 wait complete. 2023-10-30T12:35:02.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.310+0000 7f9bfa91d640 1 Processor -- start 2023-10-30T12:35:02.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.310+0000 7f9bfa91d640 1 -- start start 2023-10-30T12:35:02.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bfa91d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bf40722a0 0x7f9bf414b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:02.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bfa91d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf40c6e60 0x7f9bf4142a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:02.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bfa91d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bf41508c0 0x7f9bf41f9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:02.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bfa91d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf4160970 con 0x7f9bf40c6e60 2023-10-30T12:35:02.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bfa91d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf414bfc0 con 0x7f9bf41508c0 2023-10-30T12:35:02.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bfa91d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf41602f0 con 0x7f9bf40722a0 2023-10-30T12:35:02.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bf37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf40c6e60 0x7f9bf4142a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:02.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bf37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf40c6e60 0x7f9bf4142a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36762/0 (socket says 172.21.15.12:36762) 2023-10-30T12:35:02.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bf37fe640 1 -- 172.21.15.12:0/2863674929 learned_addr learned my addr 172.21.15.12:0/2863674929 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:02.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bf3fff640 1 --2- 172.21.15.12:0/2863674929 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bf40722a0 0x7f9bf414b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:02.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bf8eba640 1 --2- 172.21.15.12:0/2863674929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bf41508c0 0x7f9bf41f9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:02.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bf37fe640 1 -- 172.21.15.12:0/2863674929 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bf40722a0 msgr2=0x7f9bf414b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bf37fe640 1 --2- 172.21.15.12:0/2863674929 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bf40722a0 0x7f9bf414b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bf37fe640 1 -- 172.21.15.12:0/2863674929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bf41508c0 msgr2=0x7f9bf41f9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bf37fe640 1 --2- 172.21.15.12:0/2863674929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bf41508c0 0x7f9bf41f9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bf37fe640 1 -- 172.21.15.12:0/2863674929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be40093b0 con 0x7f9bf40c6e60 2023-10-30T12:35:02.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bf37fe640 1 --2- 172.21.15.12:0/2863674929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf40c6e60 0x7f9bf4142a90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9be8001ec0 tx=0x7f9be800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:02.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bf0ff9640 1 -- 172.21.15.12:0/2863674929 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8005d50 con 0x7f9bf40c6e60 2023-10-30T12:35:02.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bf0ff9640 1 -- 172.21.15.12:0/2863674929 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be8007070 con 0x7f9bf40c6e60 2023-10-30T12:35:02.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bf0ff9640 1 -- 172.21.15.12:0/2863674929 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8007420 con 0x7f9bf40c6e60 2023-10-30T12:35:02.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bfa91d640 1 -- 172.21.15.12:0/2863674929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf406d6c0 con 0x7f9bf40c6e60 2023-10-30T12:35:02.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bfa91d640 1 -- 172.21.15.12:0/2863674929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf414bce0 con 0x7f9bf40c6e60 2023-10-30T12:35:02.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bfa91d640 1 -- 172.21.15.12:0/2863674929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf404e170 con 0x7f9bf40c6e60 2023-10-30T12:35:02.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.314+0000 7f9bf0ff9640 1 -- 172.21.15.12:0/2863674929 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9be8007580 con 0x7f9bf40c6e60 2023-10-30T12:35:02.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.318+0000 7f9bf0ff9640 1 --2- 172.21.15.12:0/2863674929 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9be00664a0 0x7f9be0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:02.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.318+0000 7f9bf0ff9640 1 -- 172.21.15.12:0/2863674929 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9be8082c90 con 0x7f9bf40c6e60 2023-10-30T12:35:02.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.318+0000 7f9bf3fff640 1 --2- 172.21.15.12:0/2863674929 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9be00664a0 0x7f9be0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:02.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.318+0000 7f9bf3fff640 1 --2- 172.21.15.12:0/2863674929 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9be00664a0 0x7f9be0068960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9be40ad000 tx=0x7f9be40a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:02.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.322+0000 7f9bf0ff9640 1 -- 172.21.15.12:0/2863674929 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9be80505d0 con 0x7f9bf40c6e60 2023-10-30T12:35:02.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.478+0000 7f9bfa91d640 1 -- 172.21.15.12:0/2863674929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9bf40b0bb0 con 0x7f9bf40c6e60 2023-10-30T12:35:02.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.478+0000 7f9bf0ff9640 1 -- 172.21.15.12:0/2863674929 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9be8017020 con 0x7f9bf40c6e60 2023-10-30T12:35:02.483 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-10-30T12:35:02.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.482+0000 7f9bfa91d640 1 -- 172.21.15.12:0/2863674929 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9be00664a0 msgr2=0x7f9be0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.482+0000 7f9bfa91d640 1 --2- 172.21.15.12:0/2863674929 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9be00664a0 0x7f9be0068960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9be40ad000 tx=0x7f9be40a6000 comp rx=0 tx=0).stop 2023-10-30T12:35:02.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.482+0000 7f9bfa91d640 1 -- 172.21.15.12:0/2863674929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf40c6e60 msgr2=0x7f9bf4142a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.482+0000 7f9bfa91d640 1 --2- 172.21.15.12:0/2863674929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf40c6e60 0x7f9bf4142a90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9be8001ec0 tx=0x7f9be800c960 comp rx=0 tx=0).stop 2023-10-30T12:35:02.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.482+0000 7f9bfa91d640 1 -- 172.21.15.12:0/2863674929 shutdown_connections 2023-10-30T12:35:02.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.482+0000 7f9bfa91d640 1 --2- 172.21.15.12:0/2863674929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bf41508c0 0x7f9bf41f9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.482+0000 7f9bfa91d640 1 --2- 172.21.15.12:0/2863674929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf40c6e60 0x7f9bf4142a90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.482+0000 7f9bfa91d640 1 --2- 172.21.15.12:0/2863674929 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9be00664a0 0x7f9be0068960 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.482+0000 7f9bfa91d640 1 --2- 172.21.15.12:0/2863674929 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9bf40722a0 0x7f9bf414b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.482+0000 7f9bfa91d640 1 -- 172.21.15.12:0/2863674929 >> 172.21.15.12:0/2863674929 conn(0x7f9bf40c7910 msgr2=0x7f9bf40c0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:02.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.482+0000 7f9bfa91d640 1 -- 172.21.15.12:0/2863674929 shutdown_connections 2023-10-30T12:35:02.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.482+0000 7f9bfa91d640 1 -- 172.21.15.12:0/2863674929 wait complete. 2023-10-30T12:35:02.504 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-30T12:35:02.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.642+0000 7f93434ea640 1 Processor -- start 2023-10-30T12:35:02.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.642+0000 7f93434ea640 1 -- start start 2023-10-30T12:35:02.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f93434ea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f933c0cf160 0x7f933c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:02.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f93434ea640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933c0cb800 con 0x7f933c15d6f0 2023-10-30T12:35:02.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f93434ea640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933c0cb330 con 0x7f933c1538a0 2023-10-30T12:35:02.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f93434ea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0cad70 con 0x7f933c0cf160 2023-10-30T12:35:02.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9340a85640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f933c15d6f0 0x7f933c15a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54208/0 (socket says 172.21.15.12:54208) 2023-10-30T12:35:02.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9340a85640 1 -- 172.21.15.12:0/205676874 learned_addr learned my addr 172.21.15.12:0/205676874 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:02.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9341a87640 1 --2- 172.21.15.12:0/205676874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f933c0cf160 0x7f933c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:02.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443153069 0 0) 0x7f933c0cb800 con 0x7f933c15d6f0 2023-10-30T12:35:02.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9328003530 con 0x7f933c15d6f0 2023-10-30T12:35:02.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20010007 0 0) 0x7f933c0cb330 con 0x7f933c1538a0 2023-10-30T12:35:02.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f933c0cb800 con 0x7f933c1538a0 2023-10-30T12:35:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2778511574 0 0) 0x7f9328003530 con 0x7f933c15d6f0 2023-10-30T12:35:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f933c0cf160 msgr2=0x7f933c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 --2- 172.21.15.12:0/205676874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f933c0cf160 0x7f933c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 >> v1:172.21.15.154:6789/0 conn(0x7f933c1538a0 legacy=0x7f933c0ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c0cb050 con 0x7f933c15d6f0 2023-10-30T12:35:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f932c001680 con 0x7f933c15d6f0 2023-10-30T12:35:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 >> v1:172.21.15.12:6789/0 conn(0x7f933c15d6f0 legacy=0x7f933c15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 --2- 172.21.15.12:0/205676874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f933c0cf160 0x7f9328003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 --2- 172.21.15.12:0/205676874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93280040a0 0x7f93280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 --2- 172.21.15.12:0/205676874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9328004d10 0x7f9328005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:02.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0cad70 con 0x7f9328004d10 2023-10-30T12:35:02.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932800ba70 con 0x7f93280040a0 2023-10-30T12:35:02.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932800bc30 con 0x7f933c0cf160 2023-10-30T12:35:02.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9341a87640 1 --2- 172.21.15.12:0/205676874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9328004d10 0x7f9328005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:02.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9341286640 1 --2- 172.21.15.12:0/205676874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f933c0cf160 0x7f9328003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:02.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9340a85640 1 --2- 172.21.15.12:0/205676874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93280040a0 0x7f93280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:02.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9341a87640 1 -- 172.21.15.12:0/205676874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f933c0cf160 msgr2=0x7f9328003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:02.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9341a87640 1 --2- 172.21.15.12:0/205676874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f933c0cf160 0x7f9328003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9341a87640 1 -- 172.21.15.12:0/205676874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93280040a0 msgr2=0x7f93280047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:02.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9341a87640 1 --2- 172.21.15.12:0/205676874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93280040a0 0x7f93280047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9341a87640 1 -- 172.21.15.12:0/205676874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9328003530 con 0x7f9328004d10 2023-10-30T12:35:02.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.646+0000 7f9340a85640 1 --2- 172.21.15.12:0/205676874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93280040a0 0x7f93280047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:02.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.650+0000 7f9341a87640 1 --2- 172.21.15.12:0/205676874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9328004d10 0x7f9328005190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9334009fc0 tx=0x7f933401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88df8ec1f6e166d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:02.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.650+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340a5c70 con 0x7f9328004d10 2023-10-30T12:35:02.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.650+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933400f040 con 0x7f9328004d10 2023-10-30T12:35:02.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.650+0000 7f9333fff640 1 -- 172.21.15.12:0/205676874 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340ad440 con 0x7f9328004d10 2023-10-30T12:35:02.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.650+0000 7f93434ea640 1 -- 172.21.15.12:0/205676874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9328004d10 msgr2=0x7f9328005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.650+0000 7f93434ea640 1 --2- 172.21.15.12:0/205676874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9328004d10 0x7f9328005190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9334009fc0 tx=0x7f933401cc60 comp rx=0 tx=0).stop 2023-10-30T12:35:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.650+0000 7f9341a87640 1 -- 172.21.15.12:0/205676874 reap_dead start 2023-10-30T12:35:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.650+0000 7f93434ea640 1 -- 172.21.15.12:0/205676874 shutdown_connections 2023-10-30T12:35:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.650+0000 7f93434ea640 1 -- 172.21.15.12:0/205676874 >> 172.21.15.12:0/205676874 conn(0x7f933c06bfe0 msgr2=0x7f933c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.650+0000 7f93434ea640 1 -- 172.21.15.12:0/205676874 shutdown_connections 2023-10-30T12:35:02.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.650+0000 7f93434ea640 1 -- 172.21.15.12:0/205676874 wait complete. 2023-10-30T12:35:02.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.650+0000 7f93434ea640 1 Processor -- start 2023-10-30T12:35:02.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.650+0000 7f93434ea640 1 -- start start 2023-10-30T12:35:02.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f93434ea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9328004d10 0x7f933c0b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:02.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f93434ea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c0c64d0 0x7f933c14c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f93434ea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f933c14c780 0x7f933c0c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f93434ea640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0c6c30 con 0x7f933c0c64d0 2023-10-30T12:35:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f93434ea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c14cc00 con 0x7f933c14c780 2023-10-30T12:35:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f93434ea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0c7e00 con 0x7f9328004d10 2023-10-30T12:35:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f9340a85640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c0c64d0 0x7f933c14c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:02.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f9340a85640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c0c64d0 0x7f933c14c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36780/0 (socket says 172.21.15.12:36780) 2023-10-30T12:35:02.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f9340a85640 1 -- 172.21.15.12:0/4061389655 learned_addr learned my addr 172.21.15.12:0/4061389655 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:02.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f9341286640 1 --2- 172.21.15.12:0/4061389655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9328004d10 0x7f933c0b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:02.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f9341a87640 1 --2- 172.21.15.12:0/4061389655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f933c14c780 0x7f933c0c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:02.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f9340a85640 1 -- 172.21.15.12:0/4061389655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9328004d10 msgr2=0x7f933c0b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f9340a85640 1 --2- 172.21.15.12:0/4061389655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9328004d10 0x7f933c0b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f9340a85640 1 -- 172.21.15.12:0/4061389655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f933c14c780 msgr2=0x7f933c0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f9340a85640 1 --2- 172.21.15.12:0/4061389655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f933c14c780 0x7f933c0c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f9340a85640 1 -- 172.21.15.12:0/4061389655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c154300 con 0x7f933c0c64d0 2023-10-30T12:35:02.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f9341a87640 1 --2- 172.21.15.12:0/4061389655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f933c14c780 0x7f933c0c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:35:02.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f9340a85640 1 --2- 172.21.15.12:0/4061389655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c0c64d0 0x7f933c14c240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f932c00e7f0 tx=0x7f932c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:02.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f9331ffb640 1 -- 172.21.15.12:0/4061389655 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c00c8d0 con 0x7f933c0c64d0 2023-10-30T12:35:02.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f9331ffb640 1 -- 172.21.15.12:0/4061389655 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932c00ca30 con 0x7f933c0c64d0 2023-10-30T12:35:02.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f93434ea640 1 -- 172.21.15.12:0/4061389655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c1f2d80 con 0x7f933c0c64d0 2023-10-30T12:35:02.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f9331ffb640 1 -- 172.21.15.12:0/4061389655 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c00cdc0 con 0x7f933c0c64d0 2023-10-30T12:35:02.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f93434ea640 1 -- 172.21.15.12:0/4061389655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933c154880 con 0x7f933c0c64d0 2023-10-30T12:35:02.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.654+0000 7f93434ea640 1 -- 172.21.15.12:0/4061389655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933c04e170 con 0x7f933c0c64d0 2023-10-30T12:35:02.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.658+0000 7f9331ffb640 1 -- 172.21.15.12:0/4061389655 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f932c01d460 con 0x7f933c0c64d0 2023-10-30T12:35:02.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.658+0000 7f9331ffb640 1 --2- 172.21.15.12:0/4061389655 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f93200664a0 0x7f9320068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:02.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.658+0000 7f9331ffb640 1 -- 172.21.15.12:0/4061389655 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f932c083ae0 con 0x7f933c0c64d0 2023-10-30T12:35:02.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.658+0000 7f9341286640 1 --2- 172.21.15.12:0/4061389655 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f93200664a0 0x7f9320068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:02.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.658+0000 7f9341286640 1 --2- 172.21.15.12:0/4061389655 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f93200664a0 0x7f9320068960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9338007af0 tx=0x7f9338009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:02.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.662+0000 7f9331ffb640 1 -- 172.21.15.12:0/4061389655 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f932c051340 con 0x7f933c0c64d0 2023-10-30T12:35:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.818+0000 7f93434ea640 1 -- 172.21.15.12:0/4061389655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f933c159830 con 0x7f933c0c64d0 2023-10-30T12:35:02.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.822+0000 7f9331ffb640 1 -- 172.21.15.12:0/4061389655 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7f932c001590 con 0x7f933c0c64d0 2023-10-30T12:35:02.824 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:02.824 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2023-10-30T12:35:02.825 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-30T12:35:02.825 INFO:teuthology.orchestra.run.smithi012.stdout: "poolnum": 1, 2023-10-30T12:35:02.825 INFO:teuthology.orchestra.run.smithi012.stdout: "poolname": ".mgr" 2023-10-30T12:35:02.825 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-30T12:35:02.825 INFO:teuthology.orchestra.run.smithi012.stdout:] 2023-10-30T12:35:02.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.822+0000 7f93434ea640 1 -- 172.21.15.12:0/4061389655 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f93200664a0 msgr2=0x7f9320068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.822+0000 7f93434ea640 1 --2- 172.21.15.12:0/4061389655 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f93200664a0 0x7f9320068960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9338007af0 tx=0x7f9338009040 comp rx=0 tx=0).stop 2023-10-30T12:35:02.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.822+0000 7f93434ea640 1 -- 172.21.15.12:0/4061389655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c0c64d0 msgr2=0x7f933c14c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.822+0000 7f93434ea640 1 --2- 172.21.15.12:0/4061389655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c0c64d0 0x7f933c14c240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f932c00e7f0 tx=0x7f932c00ecb0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.822+0000 7f93434ea640 1 -- 172.21.15.12:0/4061389655 shutdown_connections 2023-10-30T12:35:02.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.822+0000 7f93434ea640 1 --2- 172.21.15.12:0/4061389655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f933c14c780 0x7f933c0c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.822+0000 7f93434ea640 1 --2- 172.21.15.12:0/4061389655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c0c64d0 0x7f933c14c240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.822+0000 7f93434ea640 1 --2- 172.21.15.12:0/4061389655 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f93200664a0 0x7f9320068960 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.822+0000 7f93434ea640 1 --2- 172.21.15.12:0/4061389655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9328004d10 0x7f933c0b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.822+0000 7f93434ea640 1 -- 172.21.15.12:0/4061389655 >> 172.21.15.12:0/4061389655 conn(0x7f933c06bfe0 msgr2=0x7f933c15b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.822+0000 7f93434ea640 1 -- 172.21.15.12:0/4061389655 shutdown_connections 2023-10-30T12:35:02.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.822+0000 7f93434ea640 1 -- 172.21.15.12:0/4061389655 wait complete. 2023-10-30T12:35:02.845 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-30T12:35:02.845 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-30T12:35:02.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.990+0000 7f30f3809640 1 Processor -- start 2023-10-30T12:35:02.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.990+0000 7f30f3809640 1 -- start start 2023-10-30T12:35:02.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.990+0000 7f30f3809640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30ec14cf40 0x7f30ec069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:02.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.990+0000 7f30f3809640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f30ec15a700 0x7f30ec156490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:02.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.990+0000 7f30f3809640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ec15a270 con 0x7f30ec1569d0 2023-10-30T12:35:02.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.990+0000 7f30f3809640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ec06c7c0 con 0x7f30ec15a700 2023-10-30T12:35:02.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.990+0000 7f30f3809640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec06a3d0 con 0x7f30ec14cf40 2023-10-30T12:35:02.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.990+0000 7f30f0da4640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f30ec15a700 0x7f30ec156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:02.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.990+0000 7f30f0da4640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f30ec15a700 0x7f30ec156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36794/0 (socket says 172.21.15.12:36794) 2023-10-30T12:35:02.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.990+0000 7f30f0da4640 1 -- 172.21.15.12:0/938991731 learned_addr learned my addr 172.21.15.12:0/938991731 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:02.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.990+0000 7f30f15a5640 1 --2- 172.21.15.12:0/938991731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30ec14cf40 0x7f30ec069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:02.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.990+0000 7f30f0da4640 1 -- 172.21.15.12:0/938991731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30ec14cf40 msgr2=0x7f30ec069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:02.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.990+0000 7f30f0da4640 1 --2- 172.21.15.12:0/938991731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30ec14cf40 0x7f30ec069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:02.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.990+0000 7f30f0da4640 1 -- 172.21.15.12:0/938991731 >> v1:172.21.15.154:6789/0 conn(0x7f30ec1569d0 legacy=0x7f30ec0c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.990+0000 7f30f0da4640 1 -- 172.21.15.12:0/938991731 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ec156ed0 con 0x7f30ec15a700 2023-10-30T12:35:02.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.994+0000 7f30f0da4640 1 --2- 172.21.15.12:0/938991731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec15a700 0x7f30ec156490 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009870 tx=0x7f30dc01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd47ff540e7adb46 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:02.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.994+0000 7f30e3fff640 1 -- 172.21.15.12:0/938991731 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0a7070 con 0x7f30ec15a700 2023-10-30T12:35:02.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.994+0000 7f30e3fff640 1 -- 172.21.15.12:0/938991731 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc0ab430 con 0x7f30ec15a700 2023-10-30T12:35:02.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.994+0000 7f30e3fff640 1 -- 172.21.15.12:0/938991731 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0ab7c0 con 0x7f30ec15a700 2023-10-30T12:35:02.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.994+0000 7f30f3809640 1 -- 172.21.15.12:0/938991731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec15a700 msgr2=0x7f30ec156490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:02.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.994+0000 7f30f3809640 1 --2- 172.21.15.12:0/938991731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec15a700 0x7f30ec156490 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009870 tx=0x7f30dc01c540 comp rx=0 tx=0).stop 2023-10-30T12:35:03.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.994+0000 7f30f3809640 1 -- 172.21.15.12:0/938991731 shutdown_connections 2023-10-30T12:35:03.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.994+0000 7f30f3809640 1 --2- 172.21.15.12:0/938991731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec15a700 0x7f30ec156490 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:03.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.994+0000 7f30f3809640 1 --2- 172.21.15.12:0/938991731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30ec14cf40 0x7f30ec069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:03.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.994+0000 7f30f3809640 1 -- 172.21.15.12:0/938991731 >> 172.21.15.12:0/938991731 conn(0x7f30ec0c7910 msgr2=0x7f30ec0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:03.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.994+0000 7f30f3809640 1 -- 172.21.15.12:0/938991731 shutdown_connections 2023-10-30T12:35:03.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.994+0000 7f30f3809640 1 -- 172.21.15.12:0/938991731 wait complete. 2023-10-30T12:35:03.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.994+0000 7f30f3809640 1 Processor -- start 2023-10-30T12:35:03.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f3809640 1 -- start start 2023-10-30T12:35:03.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f3809640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec14cf40 0x7f30ec14ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:03.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f3809640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30ec1569d0 0x7f30ec149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:03.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f3809640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30ec15a700 0x7f30ec146c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:03.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f3809640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec141120 con 0x7f30ec14cf40 2023-10-30T12:35:03.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f3809640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec1418f0 con 0x7f30ec15a700 2023-10-30T12:35:03.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f3809640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec0c5230 con 0x7f30ec1569d0 2023-10-30T12:35:03.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f15a5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec14cf40 0x7f30ec14ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:03.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f15a5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec14cf40 0x7f30ec14ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36808/0 (socket says 172.21.15.12:36808) 2023-10-30T12:35:03.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f15a5640 1 -- 172.21.15.12:0/1899822417 learned_addr learned my addr 172.21.15.12:0/1899822417 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:03.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f0da4640 1 --2- 172.21.15.12:0/1899822417 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30ec1569d0 0x7f30ec149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:03.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f1da6640 1 --2- 172.21.15.12:0/1899822417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30ec15a700 0x7f30ec146c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:03.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f15a5640 1 -- 172.21.15.12:0/1899822417 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30ec1569d0 msgr2=0x7f30ec149ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:03.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f15a5640 1 --2- 172.21.15.12:0/1899822417 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30ec1569d0 0x7f30ec149ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:03.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f15a5640 1 -- 172.21.15.12:0/1899822417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30ec15a700 msgr2=0x7f30ec146c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:03.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f15a5640 1 --2- 172.21.15.12:0/1899822417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30ec15a700 0x7f30ec146c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:03.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f15a5640 1 -- 172.21.15.12:0/1899822417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc0094d0 con 0x7f30ec14cf40 2023-10-30T12:35:03.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f1da6640 1 --2- 172.21.15.12:0/1899822417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30ec15a700 0x7f30ec146c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:35:03.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f15a5640 1 --2- 172.21.15.12:0/1899822417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec14cf40 0x7f30ec14ab40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f30e800d830 tx=0x7f30e800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:03.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30e1ffb640 1 -- 172.21.15.12:0/1899822417 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e800b6c0 con 0x7f30ec14cf40 2023-10-30T12:35:03.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30e1ffb640 1 -- 172.21.15.12:0/1899822417 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e800b820 con 0x7f30ec14cf40 2023-10-30T12:35:03.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f3809640 1 -- 172.21.15.12:0/1899822417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ec146460 con 0x7f30ec14cf40 2023-10-30T12:35:03.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30e1ffb640 1 -- 172.21.15.12:0/1899822417 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e800bb90 con 0x7f30ec14cf40 2023-10-30T12:35:03.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f3809640 1 -- 172.21.15.12:0/1899822417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ec141610 con 0x7f30ec14cf40 2023-10-30T12:35:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:02.998+0000 7f30f3809640 1 -- 172.21.15.12:0/1899822417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30ec1462c0 con 0x7f30ec14cf40 2023-10-30T12:35:03.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:03.002+0000 7f30e1ffb640 1 -- 172.21.15.12:0/1899822417 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f30e801e460 con 0x7f30ec14cf40 2023-10-30T12:35:03.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:03.002+0000 7f30e1ffb640 1 --2- 172.21.15.12:0/1899822417 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f30d00664a0 0x7f30d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:03.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:03.002+0000 7f30e1ffb640 1 -- 172.21.15.12:0/1899822417 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f30e8084380 con 0x7f30ec14cf40 2023-10-30T12:35:03.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:03.002+0000 7f30f0da4640 1 --2- 172.21.15.12:0/1899822417 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f30d00664a0 0x7f30d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:03.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:03.002+0000 7f30f0da4640 1 --2- 172.21.15.12:0/1899822417 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f30d00664a0 0x7f30d0068960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0af000 tx=0x7f30dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:03.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:03.006+0000 7f30e1ffb640 1 -- 172.21.15.12:0/1899822417 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f30ec1462c0 con 0x7f30ec14cf40 2023-10-30T12:35:03.103 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:35:03.109 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:35:03.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:03.166+0000 7f30f3809640 1 -- 172.21.15.12:0/1899822417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f30ec1462c0 con 0x7f30ec14cf40 2023-10-30T12:35:04.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.138+0000 7f30e1ffb640 1 -- 172.21.15.12:0/1899822417 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f30ec1462c0 con 0x7f30ec14cf40 2023-10-30T12:35:04.142 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' created 2023-10-30T12:35:04.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.142+0000 7f30f3809640 1 -- 172.21.15.12:0/1899822417 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f30d00664a0 msgr2=0x7f30d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:04.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.142+0000 7f30f3809640 1 --2- 172.21.15.12:0/1899822417 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f30d00664a0 0x7f30d0068960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0af000 tx=0x7f30dc008040 comp rx=0 tx=0).stop 2023-10-30T12:35:04.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.142+0000 7f30f3809640 1 -- 172.21.15.12:0/1899822417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec14cf40 msgr2=0x7f30ec14ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:04.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.142+0000 7f30f3809640 1 --2- 172.21.15.12:0/1899822417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec14cf40 0x7f30ec14ab40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f30e800d830 tx=0x7f30e800dcf0 comp rx=0 tx=0).stop 2023-10-30T12:35:04.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.142+0000 7f30f3809640 1 -- 172.21.15.12:0/1899822417 shutdown_connections 2023-10-30T12:35:04.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.142+0000 7f30f3809640 1 --2- 172.21.15.12:0/1899822417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30ec15a700 0x7f30ec146c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:04.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.142+0000 7f30f3809640 1 --2- 172.21.15.12:0/1899822417 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f30d00664a0 0x7f30d0068960 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:04.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.142+0000 7f30f3809640 1 --2- 172.21.15.12:0/1899822417 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30ec1569d0 0x7f30ec149ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:04.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.142+0000 7f30f3809640 1 --2- 172.21.15.12:0/1899822417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec14cf40 0x7f30ec14ab40 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f30e800d830 tx=0x7f30e800dcf0 comp rx=0 tx=0).stop 2023-10-30T12:35:04.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.142+0000 7f30f3809640 1 -- 172.21.15.12:0/1899822417 >> 172.21.15.12:0/1899822417 conn(0x7f30ec0c7910 msgr2=0x7f30ec147880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:04.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.142+0000 7f30f3809640 1 -- 172.21.15.12:0/1899822417 shutdown_connections 2023-10-30T12:35:04.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.142+0000 7f30f3809640 1 -- 172.21.15.12:0/1899822417 wait complete. 2023-10-30T12:35:04.176 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-30T12:35:04.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05e1d0f640 1 Processor -- start 2023-10-30T12:35:04.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05e1d0f640 1 -- start start 2023-10-30T12:35:04.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05e1d0f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f05dc1539e0 0x7f05dc0c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:04.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05e1d0f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f05dc15a1a0 0x7f05dc0c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:04.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05e1d0f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05dc06d420 0x7f05dc0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:04.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05e1d0f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05dc15a670 con 0x7f05dc15a1a0 2023-10-30T12:35:04.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05e1d0f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05dc0c0f70 con 0x7f05dc1539e0 2023-10-30T12:35:04.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05e1d0f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc0c1250 con 0x7f05dc06d420 2023-10-30T12:35:04.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05daffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f05dc15a1a0 0x7f05dc0c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:04.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05daffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f05dc15a1a0 0x7f05dc0c78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36822/0 (socket says 172.21.15.12:36822) 2023-10-30T12:35:04.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05daffd640 1 -- 172.21.15.12:0/992840965 learned_addr learned my addr 172.21.15.12:0/992840965 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:04.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05dbfff640 1 --2- 172.21.15.12:0/992840965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05dc06d420 0x7f05dc0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:04.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05db7fe640 1 --2- 172.21.15.12:0/992840965 >> v2:172.21.15.154:3300/0 conn(0x7f05dc1539e0 0x7f05dc0c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:04.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05daffd640 1 -- 172.21.15.12:0/992840965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05dc06d420 msgr2=0x7f05dc0c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05daffd640 1 --2- 172.21.15.12:0/992840965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05dc06d420 0x7f05dc0c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05daffd640 1 -- 172.21.15.12:0/992840965 >> v2:172.21.15.154:3300/0 conn(0x7f05dc1539e0 msgr2=0x7f05dc0c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05daffd640 1 --2- 172.21.15.12:0/992840965 >> v2:172.21.15.154:3300/0 conn(0x7f05dc1539e0 0x7f05dc0c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05daffd640 1 -- 172.21.15.12:0/992840965 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc0c1530 con 0x7f05dc15a1a0 2023-10-30T12:35:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05db7fe640 1 --2- 172.21.15.12:0/992840965 >> v2:172.21.15.154:3300/0 conn(0x7f05dc1539e0 0x7f05dc0c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:04.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.374+0000 7f05dbfff640 1 --2- 172.21.15.12:0/992840965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05dc06d420 0x7f05dc0c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:04.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.378+0000 7f05daffd640 1 --2- 172.21.15.12:0/992840965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05dc15a1a0 0x7f05dc0c78f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0cbbe0 tx=0x7f05d001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e7100bf4038c2b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:04.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.378+0000 7f05da7fc640 1 -- 172.21.15.12:0/992840965 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d00a89b0 con 0x7f05dc15a1a0 2023-10-30T12:35:04.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.378+0000 7f05da7fc640 1 -- 172.21.15.12:0/992840965 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d00a8b10 con 0x7f05dc15a1a0 2023-10-30T12:35:04.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.378+0000 7f05da7fc640 1 -- 172.21.15.12:0/992840965 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d00a8ea0 con 0x7f05dc15a1a0 2023-10-30T12:35:04.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.378+0000 7f05e1d0f640 1 -- 172.21.15.12:0/992840965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05dc15a1a0 msgr2=0x7f05dc0c78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:04.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.378+0000 7f05e1d0f640 1 --2- 172.21.15.12:0/992840965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05dc15a1a0 0x7f05dc0c78f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0cbbe0 tx=0x7f05d001ecf0 comp rx=0 tx=0).stop 2023-10-30T12:35:04.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.378+0000 7f05e1d0f640 1 -- 172.21.15.12:0/992840965 shutdown_connections 2023-10-30T12:35:04.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.378+0000 7f05e1d0f640 1 --2- 172.21.15.12:0/992840965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05dc06d420 0x7f05dc0c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:04.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.378+0000 7f05e1d0f640 1 --2- 172.21.15.12:0/992840965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05dc15a1a0 0x7f05dc0c78f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:04.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.378+0000 7f05e1d0f640 1 --2- 172.21.15.12:0/992840965 >> v2:172.21.15.154:3300/0 conn(0x7f05dc1539e0 0x7f05dc0c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:04.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.378+0000 7f05e1d0f640 1 -- 172.21.15.12:0/992840965 >> 172.21.15.12:0/992840965 conn(0x7f05dc06c850 msgr2=0x7f05dc06cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:04.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.378+0000 7f05e1d0f640 1 -- 172.21.15.12:0/992840965 shutdown_connections 2023-10-30T12:35:04.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.378+0000 7f05e1d0f640 1 -- 172.21.15.12:0/992840965 wait complete. 2023-10-30T12:35:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05e1d0f640 1 Processor -- start 2023-10-30T12:35:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05e1d0f640 1 -- start start 2023-10-30T12:35:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05e1d0f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05dc06d420 0x7f05dc1f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05e1d0f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05dc1539e0 0x7f05dc1f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05e1d0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05dc15a1a0 0x7f05dc14e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05e1d0f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc1564e0 con 0x7f05dc15a1a0 2023-10-30T12:35:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05e1d0f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc1f0120 con 0x7f05dc1539e0 2023-10-30T12:35:04.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05e1d0f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc155560 con 0x7f05dc06d420 2023-10-30T12:35:04.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05dbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05dc15a1a0 0x7f05dc14e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:04.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05dbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05dc15a1a0 0x7f05dc14e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36832/0 (socket says 172.21.15.12:36832) 2023-10-30T12:35:04.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05dbfff640 1 -- 172.21.15.12:0/1007655382 learned_addr learned my addr 172.21.15.12:0/1007655382 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:04.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05db7fe640 1 --2- 172.21.15.12:0/1007655382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05dc06d420 0x7f05dc1f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:04.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05daffd640 1 --2- 172.21.15.12:0/1007655382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05dc1539e0 0x7f05dc1f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:04.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05dbfff640 1 -- 172.21.15.12:0/1007655382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05dc06d420 msgr2=0x7f05dc1f05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:04.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05dbfff640 1 --2- 172.21.15.12:0/1007655382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05dc06d420 0x7f05dc1f05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:04.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05dbfff640 1 -- 172.21.15.12:0/1007655382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05dc1539e0 msgr2=0x7f05dc1f0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:04.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05dbfff640 1 --2- 172.21.15.12:0/1007655382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05dc1539e0 0x7f05dc1f0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05dbfff640 1 -- 172.21.15.12:0/1007655382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d00094d0 con 0x7f05dc15a1a0 2023-10-30T12:35:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05dbfff640 1 --2- 172.21.15.12:0/1007655382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05dc15a1a0 0x7f05dc14e830 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f05d4007e10 tx=0x7f05d400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05c3fff640 1 -- 172.21.15.12:0/1007655382 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4003ba0 con 0x7f05dc15a1a0 2023-10-30T12:35:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05c3fff640 1 -- 172.21.15.12:0/1007655382 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d400f040 con 0x7f05dc15a1a0 2023-10-30T12:35:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05c3fff640 1 -- 172.21.15.12:0/1007655382 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d400de80 con 0x7f05dc15a1a0 2023-10-30T12:35:04.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.382+0000 7f05e1d0f640 1 -- 172.21.15.12:0/1007655382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05dc155840 con 0x7f05dc15a1a0 2023-10-30T12:35:04.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.386+0000 7f05e1d0f640 1 -- 172.21.15.12:0/1007655382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05dc1efdb0 con 0x7f05dc15a1a0 2023-10-30T12:35:04.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.386+0000 7f05e1d0f640 1 -- 172.21.15.12:0/1007655382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05dc1ef930 con 0x7f05dc15a1a0 2023-10-30T12:35:04.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.386+0000 7f05c3fff640 1 -- 172.21.15.12:0/1007655382 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f05d4004030 con 0x7f05dc15a1a0 2023-10-30T12:35:04.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.386+0000 7f05c3fff640 1 --2- 172.21.15.12:0/1007655382 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f05c80664a0 0x7f05c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:04.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.386+0000 7f05c3fff640 1 -- 172.21.15.12:0/1007655382 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f05d4083fe0 con 0x7f05dc15a1a0 2023-10-30T12:35:04.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.386+0000 7f05db7fe640 1 --2- 172.21.15.12:0/1007655382 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f05c80664a0 0x7f05c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:04.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.390+0000 7f05db7fe640 1 --2- 172.21.15.12:0/1007655382 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f05c80664a0 0x7f05c8068960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f05cc006e90 tx=0x7f05cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:04.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.390+0000 7f05c3fff640 1 -- 172.21.15.12:0/1007655382 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f05dc1ef930 con 0x7f05dc15a1a0 2023-10-30T12:35:04.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:04.550+0000 7f05e1d0f640 1 -- 172.21.15.12:0/1007655382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f05dc1ef930 con 0x7f05dc15a1a0 2023-10-30T12:35:05.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.146+0000 7f05c3fff640 1 -- 172.21.15.12:0/1007655382 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7f05dc1ef930 con 0x7f05dc15a1a0 2023-10-30T12:35:05.151 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' created 2023-10-30T12:35:05.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.150+0000 7f05e1d0f640 1 -- 172.21.15.12:0/1007655382 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f05c80664a0 msgr2=0x7f05c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:05.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.150+0000 7f05e1d0f640 1 --2- 172.21.15.12:0/1007655382 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f05c80664a0 0x7f05c8068960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f05cc006e90 tx=0x7f05cc008040 comp rx=0 tx=0).stop 2023-10-30T12:35:05.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.150+0000 7f05e1d0f640 1 -- 172.21.15.12:0/1007655382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05dc15a1a0 msgr2=0x7f05dc14e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:05.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.150+0000 7f05e1d0f640 1 --2- 172.21.15.12:0/1007655382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05dc15a1a0 0x7f05dc14e830 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f05d4007e10 tx=0x7f05d400d4a0 comp rx=0 tx=0).stop 2023-10-30T12:35:05.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.150+0000 7f05e1d0f640 1 -- 172.21.15.12:0/1007655382 shutdown_connections 2023-10-30T12:35:05.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.150+0000 7f05e1d0f640 1 --2- 172.21.15.12:0/1007655382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05dc15a1a0 0x7f05dc14e830 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:05.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.150+0000 7f05e1d0f640 1 --2- 172.21.15.12:0/1007655382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05dc1539e0 0x7f05dc1f0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:05.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.150+0000 7f05e1d0f640 1 --2- 172.21.15.12:0/1007655382 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f05c80664a0 0x7f05c8068960 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:05.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.150+0000 7f05e1d0f640 1 --2- 172.21.15.12:0/1007655382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05dc06d420 0x7f05dc1f05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:05.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.150+0000 7f05e1d0f640 1 -- 172.21.15.12:0/1007655382 >> 172.21.15.12:0/1007655382 conn(0x7f05dc06c850 msgr2=0x7f05dc070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:05.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.150+0000 7f05e1d0f640 1 -- 172.21.15.12:0/1007655382 shutdown_connections 2023-10-30T12:35:05.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.150+0000 7f05e1d0f640 1 -- 172.21.15.12:0/1007655382 wait complete. 2023-10-30T12:35:05.194 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-30T12:35:05.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.342+0000 7feb90c09640 1 Processor -- start 2023-10-30T12:35:05.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.342+0000 7feb90c09640 1 -- start start 2023-10-30T12:35:05.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.342+0000 7feb90c09640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feb8c153980 0x7feb8c0caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:05.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.342+0000 7feb90c09640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7feb8c159930 0x7feb8c0cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:05.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.342+0000 7feb90c09640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb8c15a920 con 0x7feb8c0cf1a0 2023-10-30T12:35:05.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.342+0000 7feb90c09640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8c0cb930 con 0x7feb8c153980 2023-10-30T12:35:05.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.342+0000 7feb90c09640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8c159dc0 con 0x7feb8c159930 2023-10-30T12:35:05.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.342+0000 7feb8a59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feb8c153980 0x7feb8c0caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:05.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.342+0000 7feb8a59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feb8c153980 0x7feb8c0caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36836/0 (socket says 172.21.15.12:36836) 2023-10-30T12:35:05.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.342+0000 7feb8a59c640 1 -- 172.21.15.12:0/1588060841 learned_addr learned my addr 172.21.15.12:0/1588060841 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:05.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.342+0000 7feb83fff640 1 --2- 172.21.15.12:0/1588060841 >> v2:172.21.15.154:3300/0 conn(0x7feb8c159930 0x7feb8c0cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:05.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.342+0000 7feb8a59c640 1 -- 172.21.15.12:0/1588060841 >> v2:172.21.15.154:3300/0 conn(0x7feb8c159930 msgr2=0x7feb8c0cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:05.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.342+0000 7feb8a59c640 1 --2- 172.21.15.12:0/1588060841 >> v2:172.21.15.154:3300/0 conn(0x7feb8c159930 0x7feb8c0cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:05.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.342+0000 7feb8a59c640 1 -- 172.21.15.12:0/1588060841 >> v1:172.21.15.154:6789/0 conn(0x7feb8c0cf1a0 legacy=0x7feb8c15d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:05.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.342+0000 7feb8a59c640 1 -- 172.21.15.12:0/1588060841 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8c15a080 con 0x7feb8c153980 2023-10-30T12:35:05.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.346+0000 7feb8a59c640 1 --2- 172.21.15.12:0/1588060841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb8c153980 0x7feb8c0caca0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0742c0 tx=0x7feb7c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f45ddadcaca3f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:05.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.346+0000 7feb89d9b640 1 -- 172.21.15.12:0/1588060841 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c0aa610 con 0x7feb8c153980 2023-10-30T12:35:05.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.346+0000 7feb89d9b640 1 -- 172.21.15.12:0/1588060841 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7c0a6040 con 0x7feb8c153980 2023-10-30T12:35:05.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.346+0000 7feb89d9b640 1 -- 172.21.15.12:0/1588060841 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c0aa990 con 0x7feb8c153980 2023-10-30T12:35:05.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.346+0000 7feb90c09640 1 -- 172.21.15.12:0/1588060841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb8c153980 msgr2=0x7feb8c0caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:05.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.346+0000 7feb90c09640 1 --2- 172.21.15.12:0/1588060841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb8c153980 0x7feb8c0caca0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0742c0 tx=0x7feb7c01c490 comp rx=0 tx=0).stop 2023-10-30T12:35:05.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.346+0000 7feb90c09640 1 -- 172.21.15.12:0/1588060841 shutdown_connections 2023-10-30T12:35:05.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.346+0000 7feb90c09640 1 --2- 172.21.15.12:0/1588060841 >> v2:172.21.15.154:3300/0 conn(0x7feb8c159930 0x7feb8c0cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:05.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.346+0000 7feb90c09640 1 --2- 172.21.15.12:0/1588060841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb8c153980 0x7feb8c0caca0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:05.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.346+0000 7feb90c09640 1 -- 172.21.15.12:0/1588060841 >> 172.21.15.12:0/1588060841 conn(0x7feb8c06c0f0 msgr2=0x7feb8c06c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:05.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.346+0000 7feb90c09640 1 -- 172.21.15.12:0/1588060841 shutdown_connections 2023-10-30T12:35:05.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.346+0000 7feb90c09640 1 -- 172.21.15.12:0/1588060841 wait complete. 2023-10-30T12:35:05.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb90c09640 1 Processor -- start 2023-10-30T12:35:05.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb90c09640 1 -- start start 2023-10-30T12:35:05.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb90c09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb8c0cf1a0 0x7feb8c1475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:05.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb90c09640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb8c153980 0x7feb8c15aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:05.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb90c09640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb8c0c42f0 0x7feb8c0b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:05.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb90c09640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c1f5db0 con 0x7feb8c0cf1a0 2023-10-30T12:35:05.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb90c09640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c1f18b0 con 0x7feb8c153980 2023-10-30T12:35:05.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb90c09640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c1f2880 con 0x7feb8c0c42f0 2023-10-30T12:35:05.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb8a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb8c0cf1a0 0x7feb8c1475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:05.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb8a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb8c0cf1a0 0x7feb8c1475e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36846/0 (socket says 172.21.15.12:36846) 2023-10-30T12:35:05.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb8a59c640 1 -- 172.21.15.12:0/2447880493 learned_addr learned my addr 172.21.15.12:0/2447880493 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:05.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb83fff640 1 --2- 172.21.15.12:0/2447880493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb8c153980 0x7feb8c15aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:05.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb8ad9d640 1 --2- 172.21.15.12:0/2447880493 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb8c0c42f0 0x7feb8c0b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:05.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb8a59c640 1 -- 172.21.15.12:0/2447880493 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb8c0c42f0 msgr2=0x7feb8c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:05.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb8a59c640 1 --2- 172.21.15.12:0/2447880493 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb8c0c42f0 0x7feb8c0b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:05.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb8a59c640 1 -- 172.21.15.12:0/2447880493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb8c153980 msgr2=0x7feb8c15aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:05.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb8a59c640 1 --2- 172.21.15.12:0/2447880493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb8c153980 0x7feb8c15aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:05.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb8a59c640 1 -- 172.21.15.12:0/2447880493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c0094d0 con 0x7feb8c0cf1a0 2023-10-30T12:35:05.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb83fff640 1 --2- 172.21.15.12:0/2447880493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb8c153980 0x7feb8c15aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:05.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb8a59c640 1 --2- 172.21.15.12:0/2447880493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb8c0cf1a0 0x7feb8c1475e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feb7c01a0c0 tx=0x7feb7c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:05.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb82ffd640 1 -- 172.21.15.12:0/2447880493 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c0b3800 con 0x7feb8c0cf1a0 2023-10-30T12:35:05.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb82ffd640 1 -- 172.21.15.12:0/2447880493 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7c0b3960 con 0x7feb8c0cf1a0 2023-10-30T12:35:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb90c09640 1 -- 172.21.15.12:0/2447880493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8c1f23f0 con 0x7feb8c0cf1a0 2023-10-30T12:35:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb82ffd640 1 -- 172.21.15.12:0/2447880493 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c0b3d10 con 0x7feb8c0cf1a0 2023-10-30T12:35:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.350+0000 7feb90c09640 1 -- 172.21.15.12:0/2447880493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8c1f5ad0 con 0x7feb8c0cf1a0 2023-10-30T12:35:05.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.354+0000 7feb90c09640 1 -- 172.21.15.12:0/2447880493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8c04e170 con 0x7feb8c0cf1a0 2023-10-30T12:35:05.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.354+0000 7feb82ffd640 1 -- 172.21.15.12:0/2447880493 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feb7c0b2380 con 0x7feb8c0cf1a0 2023-10-30T12:35:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.354+0000 7feb82ffd640 1 --2- 172.21.15.12:0/2447880493 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb6c0664a0 0x7feb6c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.354+0000 7feb82ffd640 1 -- 172.21.15.12:0/2447880493 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7feb7c11e5c0 con 0x7feb8c0cf1a0 2023-10-30T12:35:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.354+0000 7feb83fff640 1 --2- 172.21.15.12:0/2447880493 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb6c0664a0 0x7feb6c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.354+0000 7feb83fff640 1 --2- 172.21.15.12:0/2447880493 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb6c0664a0 0x7feb6c068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0b2590 tx=0x7feb74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:05.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.358+0000 7feb82ffd640 1 -- 172.21.15.12:0/2447880493 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feb7c0eba20 con 0x7feb8c0cf1a0 2023-10-30T12:35:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:05.550+0000 7feb90c09640 1 -- 172.21.15.12:0/2447880493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7feb8c0caa80 con 0x7feb8c0cf1a0 2023-10-30T12:35:06.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.170+0000 7feb82ffd640 1 -- 172.21.15.12:0/2447880493 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7feb7c0ef040 con 0x7feb8c0cf1a0 2023-10-30T12:35:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:new fs with metadata pool 2 and data pool 3 2023-10-30T12:35:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.174+0000 7feb90c09640 1 -- 172.21.15.12:0/2447880493 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb6c0664a0 msgr2=0x7feb6c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.174+0000 7feb90c09640 1 --2- 172.21.15.12:0/2447880493 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb6c0664a0 0x7feb6c068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0b2590 tx=0x7feb74008040 comp rx=0 tx=0).stop 2023-10-30T12:35:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.174+0000 7feb90c09640 1 -- 172.21.15.12:0/2447880493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb8c0cf1a0 msgr2=0x7feb8c1475e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.174+0000 7feb90c09640 1 --2- 172.21.15.12:0/2447880493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb8c0cf1a0 0x7feb8c1475e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feb7c01a0c0 tx=0x7feb7c01acd0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.174+0000 7feb90c09640 1 -- 172.21.15.12:0/2447880493 shutdown_connections 2023-10-30T12:35:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.174+0000 7feb90c09640 1 --2- 172.21.15.12:0/2447880493 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb6c0664a0 0x7feb6c068960 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.174+0000 7feb90c09640 1 --2- 172.21.15.12:0/2447880493 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb8c0c42f0 0x7feb8c0b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.174+0000 7feb90c09640 1 --2- 172.21.15.12:0/2447880493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb8c153980 0x7feb8c15aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.174+0000 7feb90c09640 1 --2- 172.21.15.12:0/2447880493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb8c0cf1a0 0x7feb8c1475e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.174+0000 7feb90c09640 1 -- 172.21.15.12:0/2447880493 >> 172.21.15.12:0/2447880493 conn(0x7feb8c06c0f0 msgr2=0x7feb8c1575e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.174+0000 7feb90c09640 1 -- 172.21.15.12:0/2447880493 shutdown_connections 2023-10-30T12:35:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.174+0000 7feb90c09640 1 -- 172.21.15.12:0/2447880493 wait complete. 2023-10-30T12:35:06.208 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:35:06.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8267609640 1 Processor -- start 2023-10-30T12:35:06.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8267609640 1 -- start start 2023-10-30T12:35:06.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8267609640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82601538a0 0x7f826015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:06.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8267609640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82600ca0e0 0x7f826015a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:06.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8267609640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82600cb800 con 0x7f82600cf160 2023-10-30T12:35:06.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8267609640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826015dc30 con 0x7f82601538a0 2023-10-30T12:35:06.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8267609640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82600cb330 con 0x7f82600ca0e0 2023-10-30T12:35:06.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8265ba6640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f82600cf160 0x7f82600cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54210/0 (socket says 172.21.15.12:54210) 2023-10-30T12:35:06.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8265ba6640 1 -- 172.21.15.12:0/2935285542 learned_addr learned my addr 172.21.15.12:0/2935285542 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:06.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8264ba4640 1 --2- 172.21.15.12:0/2935285542 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82600ca0e0 0x7f826015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:06.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f82653a5640 1 --2- 172.21.15.12:0/2935285542 >> v2:172.21.15.154:3300/0 conn(0x7f82601538a0 0x7f826015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:06.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8257fff640 1 -- 172.21.15.12:0/2935285542 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189878734 0 0) 0x7f82600cb800 con 0x7f82600cf160 2023-10-30T12:35:06.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8257fff640 1 -- 172.21.15.12:0/2935285542 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f824c003530 con 0x7f82600cf160 2023-10-30T12:35:06.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8264ba4640 1 -- 172.21.15.12:0/2935285542 >> v2:172.21.15.154:3300/0 conn(0x7f82601538a0 msgr2=0x7f826015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:06.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8264ba4640 1 --2- 172.21.15.12:0/2935285542 >> v2:172.21.15.154:3300/0 conn(0x7f82601538a0 0x7f826015d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8264ba4640 1 -- 172.21.15.12:0/2935285542 >> v1:172.21.15.12:6789/0 conn(0x7f82600cf160 legacy=0x7f82600cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:06.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.354+0000 7f8264ba4640 1 -- 172.21.15.12:0/2935285542 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82600cad70 con 0x7f82600ca0e0 2023-10-30T12:35:06.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.358+0000 7f8264ba4640 1 --2- 172.21.15.12:0/2935285542 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82600ca0e0 0x7f826015a090 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f82500097c0 tx=0x7f825001ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce97f0dcf8301b49 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:06.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.358+0000 7f8257fff640 1 -- 172.21.15.12:0/2935285542 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500a8970 con 0x7f82600ca0e0 2023-10-30T12:35:06.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.358+0000 7f8257fff640 1 -- 172.21.15.12:0/2935285542 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82500a8ad0 con 0x7f82600ca0e0 2023-10-30T12:35:06.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.358+0000 7f8257fff640 1 -- 172.21.15.12:0/2935285542 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500a8e40 con 0x7f82600ca0e0 2023-10-30T12:35:06.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.358+0000 7f8267609640 1 -- 172.21.15.12:0/2935285542 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82600ca0e0 msgr2=0x7f826015a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:06.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.358+0000 7f8267609640 1 --2- 172.21.15.12:0/2935285542 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82600ca0e0 0x7f826015a090 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f82500097c0 tx=0x7f825001ec70 comp rx=0 tx=0).stop 2023-10-30T12:35:06.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8267609640 1 -- 172.21.15.12:0/2935285542 shutdown_connections 2023-10-30T12:35:06.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8267609640 1 --2- 172.21.15.12:0/2935285542 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82600ca0e0 0x7f826015a090 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8267609640 1 --2- 172.21.15.12:0/2935285542 >> v2:172.21.15.154:3300/0 conn(0x7f82601538a0 0x7f826015d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8267609640 1 -- 172.21.15.12:0/2935285542 >> 172.21.15.12:0/2935285542 conn(0x7f826006bfe0 msgr2=0x7f826006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:06.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8267609640 1 -- 172.21.15.12:0/2935285542 shutdown_connections 2023-10-30T12:35:06.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8267609640 1 -- 172.21.15.12:0/2935285542 wait complete. 2023-10-30T12:35:06.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8267609640 1 Processor -- start 2023-10-30T12:35:06.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8267609640 1 -- start start 2023-10-30T12:35:06.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8267609640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82600ca0e0 0x7f82601f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:06.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8267609640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82600cf160 0x7f826015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:06.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8267609640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82601538a0 0x7f826014e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:06.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8267609640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8260150bf0 con 0x7f82600cf160 2023-10-30T12:35:06.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8267609640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826014e9f0 con 0x7f82600ca0e0 2023-10-30T12:35:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8267609640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826014e020 con 0x7f82601538a0 2023-10-30T12:35:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8264ba4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82600cf160 0x7f826015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8264ba4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82600cf160 0x7f826015bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36854/0 (socket says 172.21.15.12:36854) 2023-10-30T12:35:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8264ba4640 1 -- 172.21.15.12:0/1069127201 learned_addr learned my addr 172.21.15.12:0/1069127201 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f82653a5640 1 --2- 172.21.15.12:0/1069127201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82600ca0e0 0x7f82601f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:06.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8265ba6640 1 --2- 172.21.15.12:0/1069127201 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82601538a0 0x7f826014e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:06.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8264ba4640 1 -- 172.21.15.12:0/1069127201 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82601538a0 msgr2=0x7f826014e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:06.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8264ba4640 1 --2- 172.21.15.12:0/1069127201 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82601538a0 0x7f826014e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8264ba4640 1 -- 172.21.15.12:0/1069127201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82600ca0e0 msgr2=0x7f82601f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:06.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8264ba4640 1 --2- 172.21.15.12:0/1069127201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82600ca0e0 0x7f82601f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8264ba4640 1 -- 172.21.15.12:0/1069127201 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8250009420 con 0x7f82600cf160 2023-10-30T12:35:06.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.362+0000 7f8264ba4640 1 --2- 172.21.15.12:0/1069127201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82600cf160 0x7f826015bc20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8250000c10 tx=0x7f825001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:06.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.366+0000 7f8255ffb640 1 -- 172.21.15.12:0/1069127201 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250004450 con 0x7f82600cf160 2023-10-30T12:35:06.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.366+0000 7f8255ffb640 1 -- 172.21.15.12:0/1069127201 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8250003620 con 0x7f82600cf160 2023-10-30T12:35:06.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.366+0000 7f8255ffb640 1 -- 172.21.15.12:0/1069127201 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250003990 con 0x7f82600cf160 2023-10-30T12:35:06.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.366+0000 7f8267609640 1 -- 172.21.15.12:0/1069127201 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8260150350 con 0x7f82600cf160 2023-10-30T12:35:06.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.366+0000 7f8267609640 1 -- 172.21.15.12:0/1069127201 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8260150910 con 0x7f82600cf160 2023-10-30T12:35:06.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.366+0000 7f8267609640 1 -- 172.21.15.12:0/1069127201 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82600b2a50 con 0x7f82600cf160 2023-10-30T12:35:06.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.366+0000 7f8255ffb640 1 -- 172.21.15.12:0/1069127201 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f82500b7040 con 0x7f82600cf160 2023-10-30T12:35:06.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.366+0000 7f8255ffb640 1 --2- 172.21.15.12:0/1069127201 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82440664d0 0x7f8244068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:06.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.366+0000 7f8255ffb640 1 -- 172.21.15.12:0/1069127201 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8250121620 con 0x7f82600cf160 2023-10-30T12:35:06.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.366+0000 7f82653a5640 1 --2- 172.21.15.12:0/1069127201 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82440664d0 0x7f8244068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:06.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.370+0000 7f82653a5640 1 --2- 172.21.15.12:0/1069127201 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82440664d0 0x7f8244068990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f825c009af0 tx=0x7f825c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:06.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.370+0000 7f8255ffb640 1 -- 172.21.15.12:0/1069127201 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f82500eea90 con 0x7f82600cf160 2023-10-30T12:35:06.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.530+0000 7f8267609640 1 -- 172.21.15.12:0/1069127201 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82600cb020 con 0x7f82600cf160 2023-10-30T12:35:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.534+0000 7f8255ffb640 1 -- 172.21.15.12:0/1069127201 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16518 (secure 0 0 0) 0x7f82500b5090 con 0x7f82600cf160 2023-10-30T12:35:06.537 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:06.538 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:35:05.553392+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:35:03.216053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:35:04.597162+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:35:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.534+0000 7f8267609640 1 -- 172.21.15.12:0/1069127201 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82440664d0 msgr2=0x7f8244068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.534+0000 7f8267609640 1 --2- 172.21.15.12:0/1069127201 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82440664d0 0x7f8244068990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f825c009af0 tx=0x7f825c008040 comp rx=0 tx=0).stop 2023-10-30T12:35:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.534+0000 7f8267609640 1 -- 172.21.15.12:0/1069127201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82600cf160 msgr2=0x7f826015bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:06.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.534+0000 7f8267609640 1 --2- 172.21.15.12:0/1069127201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82600cf160 0x7f826015bc20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8250000c10 tx=0x7f825001ba10 comp rx=0 tx=0).stop 2023-10-30T12:35:06.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.534+0000 7f8267609640 1 -- 172.21.15.12:0/1069127201 shutdown_connections 2023-10-30T12:35:06.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.534+0000 7f8267609640 1 --2- 172.21.15.12:0/1069127201 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82440664d0 0x7f8244068990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.534+0000 7f8267609640 1 --2- 172.21.15.12:0/1069127201 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82601538a0 0x7f826014e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.534+0000 7f8267609640 1 --2- 172.21.15.12:0/1069127201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82600cf160 0x7f826015bc20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.534+0000 7f8267609640 1 --2- 172.21.15.12:0/1069127201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82600ca0e0 0x7f82601f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.534+0000 7f8267609640 1 -- 172.21.15.12:0/1069127201 >> 172.21.15.12:0/1069127201 conn(0x7f826006bfe0 msgr2=0x7f8260148b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:06.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.534+0000 7f8267609640 1 -- 172.21.15.12:0/1069127201 shutdown_connections 2023-10-30T12:35:06.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.534+0000 7f8267609640 1 -- 172.21.15.12:0/1069127201 wait complete. 2023-10-30T12:35:06.559 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:35:06.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e41f5640 1 Processor -- start 2023-10-30T12:35:06.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e41f5640 1 -- start start 2023-10-30T12:35:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e41f5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f42dc1538a0 0x7f42dc15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e41f5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42dc0cb800 con 0x7f42dc0cf160 2023-10-30T12:35:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e41f5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42dc15dc30 con 0x7f42dc0ca0e0 2023-10-30T12:35:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e41f5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42dc0cb330 con 0x7f42dc1538a0 2023-10-30T12:35:06.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e2792640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f42dc0cf160 0x7f42dc164070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54218/0 (socket says 172.21.15.12:54218) 2023-10-30T12:35:06.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e2792640 1 -- 172.21.15.12:0/2146245164 learned_addr learned my addr 172.21.15.12:0/2146245164 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:06.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e1f91640 1 --2- 172.21.15.12:0/2146245164 >> v2:172.21.15.154:3300/0 conn(0x7f42dc1538a0 0x7f42dc15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:06.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e0f8f640 1 -- 172.21.15.12:0/2146245164 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949404331 0 0) 0x7f42dc0cb800 con 0x7f42dc0cf160 2023-10-30T12:35:06.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e0f8f640 1 -- 172.21.15.12:0/2146245164 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42c8003530 con 0x7f42dc0cf160 2023-10-30T12:35:06.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e0f8f640 1 -- 172.21.15.12:0/2146245164 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1345592760 0 0) 0x7f42c8003530 con 0x7f42dc0cf160 2023-10-30T12:35:06.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e0f8f640 1 -- 172.21.15.12:0/2146245164 >> v2:172.21.15.154:3300/0 conn(0x7f42dc1538a0 msgr2=0x7f42dc15d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:06.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e0f8f640 1 --2- 172.21.15.12:0/2146245164 >> v2:172.21.15.154:3300/0 conn(0x7f42dc1538a0 0x7f42dc15d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e0f8f640 1 -- 172.21.15.12:0/2146245164 >> v1:172.21.15.154:6789/0 conn(0x7f42dc0ca0e0 legacy=0x7f42dc15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:06.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e0f8f640 1 -- 172.21.15.12:0/2146245164 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc0cad70 con 0x7f42dc0cf160 2023-10-30T12:35:06.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e0f8f640 1 -- 172.21.15.12:0/2146245164 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f42d4001a00 con 0x7f42dc0cf160 2023-10-30T12:35:06.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e0f8f640 1 -- 172.21.15.12:0/2146245164 >> v1:172.21.15.12:6789/0 conn(0x7f42dc0cf160 legacy=0x7f42dc164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:06.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e0f8f640 1 --2- 172.21.15.12:0/2146245164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42c8003980 0x7f42c8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:06.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e0f8f640 1 --2- 172.21.15.12:0/2146245164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c80047d0 0x7f42c8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:06.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e0f8f640 1 --2- 172.21.15.12:0/2146245164 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c8005190 0x7f42c800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:06.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e0f8f640 1 -- 172.21.15.12:0/2146245164 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0cb330 con 0x7f42c80047d0 2023-10-30T12:35:06.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e0f8f640 1 -- 172.21.15.12:0/2146245164 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c800c040 con 0x7f42c8003980 2023-10-30T12:35:06.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.698+0000 7f42e0f8f640 1 -- 172.21.15.12:0/2146245164 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c800c200 con 0x7f42c8005190 2023-10-30T12:35:06.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.702+0000 7f42e1790640 1 --2- 172.21.15.12:0/2146245164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c80047d0 0x7f42c8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:06.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.702+0000 7f42e1f91640 1 --2- 172.21.15.12:0/2146245164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42c8003980 0x7f42c8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:06.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.702+0000 7f42e2792640 1 --2- 172.21.15.12:0/2146245164 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c8005190 0x7f42c800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:06.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.702+0000 7f42e1790640 1 -- 172.21.15.12:0/2146245164 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c8005190 msgr2=0x7f42c800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:06.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.702+0000 7f42e1790640 1 --2- 172.21.15.12:0/2146245164 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c8005190 0x7f42c800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.702+0000 7f42e1790640 1 -- 172.21.15.12:0/2146245164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42c8003980 msgr2=0x7f42c8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:06.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.702+0000 7f42e1790640 1 --2- 172.21.15.12:0/2146245164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42c8003980 0x7f42c8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.702+0000 7f42e1790640 1 -- 172.21.15.12:0/2146245164 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c8003530 con 0x7f42c80047d0 2023-10-30T12:35:06.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.702+0000 7f42e2792640 1 -- 172.21.15.12:0/2146245164 reap_dead start 2023-10-30T12:35:06.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.702+0000 7f42e1790640 1 --2- 172.21.15.12:0/2146245164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c80047d0 0x7f42c8004c50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0015e0 tx=0x7f42cc01c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df8dd45632bdd2b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:06.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.702+0000 7f42e0f8f640 1 -- 172.21.15.12:0/2146245164 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc0052d0 con 0x7f42c80047d0 2023-10-30T12:35:06.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.702+0000 7f42e0f8f640 1 -- 172.21.15.12:0/2146245164 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42cc0a7050 con 0x7f42c80047d0 2023-10-30T12:35:06.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.702+0000 7f42e0f8f640 1 -- 172.21.15.12:0/2146245164 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc003600 con 0x7f42c80047d0 2023-10-30T12:35:06.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e41f5640 1 -- 172.21.15.12:0/2146245164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c80047d0 msgr2=0x7f42c8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:06.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e41f5640 1 --2- 172.21.15.12:0/2146245164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c80047d0 0x7f42c8004c50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0015e0 tx=0x7f42cc01c240 comp rx=0 tx=0).stop 2023-10-30T12:35:06.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e41f5640 1 -- 172.21.15.12:0/2146245164 shutdown_connections 2023-10-30T12:35:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e41f5640 1 --2- 172.21.15.12:0/2146245164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c80047d0 0x7f42c8004c50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e41f5640 1 -- 172.21.15.12:0/2146245164 >> 172.21.15.12:0/2146245164 conn(0x7f42dc06bfe0 msgr2=0x7f42dc06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e41f5640 1 -- 172.21.15.12:0/2146245164 shutdown_connections 2023-10-30T12:35:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e41f5640 1 -- 172.21.15.12:0/2146245164 wait complete. 2023-10-30T12:35:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e41f5640 1 Processor -- start 2023-10-30T12:35:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e41f5640 1 -- start start 2023-10-30T12:35:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e41f5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c80047d0 0x7f42dc14d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e41f5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42dc155380 0x7f42dc15acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e41f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42dc15cab0 0x7f42dc15b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:06.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e41f5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc15d3c0 con 0x7f42dc15cab0 2023-10-30T12:35:06.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e41f5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc153ea0 con 0x7f42dc155380 2023-10-30T12:35:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e41f5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc154740 con 0x7f42c80047d0 2023-10-30T12:35:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e2792640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42dc15cab0 0x7f42dc15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e2792640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42dc15cab0 0x7f42dc15b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36878/0 (socket says 172.21.15.12:36878) 2023-10-30T12:35:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e2792640 1 -- 172.21.15.12:0/209806380 learned_addr learned my addr 172.21.15.12:0/209806380 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e1f91640 1 --2- 172.21.15.12:0/209806380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c80047d0 0x7f42dc14d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e1790640 1 --2- 172.21.15.12:0/209806380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42dc155380 0x7f42dc15acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e2792640 1 -- 172.21.15.12:0/209806380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c80047d0 msgr2=0x7f42dc14d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e2792640 1 --2- 172.21.15.12:0/209806380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c80047d0 0x7f42dc14d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e2792640 1 -- 172.21.15.12:0/209806380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42dc155380 msgr2=0x7f42dc15acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e2792640 1 --2- 172.21.15.12:0/209806380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42dc155380 0x7f42dc15acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e2792640 1 -- 172.21.15.12:0/209806380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc15d5f0 con 0x7f42dc15cab0 2023-10-30T12:35:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.706+0000 7f42e1790640 1 --2- 172.21.15.12:0/209806380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42dc155380 0x7f42dc15acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.710+0000 7f42e2792640 1 --2- 172.21.15.12:0/209806380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42dc15cab0 0x7f42dc15b200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42dc072ce0 tx=0x7f42d4007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.710+0000 7f42d27fc640 1 -- 172.21.15.12:0/209806380 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d400e940 con 0x7f42dc15cab0 2023-10-30T12:35:06.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.710+0000 7f42d27fc640 1 -- 172.21.15.12:0/209806380 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d400eac0 con 0x7f42dc15cab0 2023-10-30T12:35:06.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.710+0000 7f42d27fc640 1 -- 172.21.15.12:0/209806380 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d400ee70 con 0x7f42dc15cab0 2023-10-30T12:35:06.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.710+0000 7f42e41f5640 1 -- 172.21.15.12:0/209806380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42dc15d8b0 con 0x7f42dc15cab0 2023-10-30T12:35:06.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.710+0000 7f42e41f5640 1 -- 172.21.15.12:0/209806380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42dc1543d0 con 0x7f42dc15cab0 2023-10-30T12:35:06.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.710+0000 7f42e41f5640 1 -- 172.21.15.12:0/209806380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42dc04e170 con 0x7f42dc15cab0 2023-10-30T12:35:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.710+0000 7f42d27fc640 1 -- 172.21.15.12:0/209806380 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f42d401d460 con 0x7f42dc15cab0 2023-10-30T12:35:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.710+0000 7f42d27fc640 1 --2- 172.21.15.12:0/209806380 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f42c00664d0 0x7f42c0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.710+0000 7f42d27fc640 1 -- 172.21.15.12:0/209806380 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42d4084180 con 0x7f42dc15cab0 2023-10-30T12:35:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.710+0000 7f42e1f91640 1 --2- 172.21.15.12:0/209806380 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f42c00664d0 0x7f42c0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.714+0000 7f42e1f91640 1 --2- 172.21.15.12:0/209806380 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f42c00664d0 0x7f42c0068990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009c60 tx=0x7f42d8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:06.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.714+0000 7f42d27fc640 1 -- 172.21.15.12:0/209806380 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f42d4051560 con 0x7f42dc15cab0 2023-10-30T12:35:06.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.874+0000 7f42e41f5640 1 -- 172.21.15.12:0/209806380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42dc157160 con 0x7f42dc15cab0 2023-10-30T12:35:06.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.874+0000 7f42d27fc640 1 -- 172.21.15.12:0/209806380 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16518 (secure 0 0 0) 0x7f42d4054b80 con 0x7f42dc15cab0 2023-10-30T12:35:06.880 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:06.880 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:35:05.553392+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:35:03.216053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:35:04.597162+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:35:06.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.878+0000 7f42e41f5640 1 -- 172.21.15.12:0/209806380 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f42c00664d0 msgr2=0x7f42c0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:06.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.878+0000 7f42e41f5640 1 --2- 172.21.15.12:0/209806380 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f42c00664d0 0x7f42c0068990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009c60 tx=0x7f42d8009190 comp rx=0 tx=0).stop 2023-10-30T12:35:06.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.878+0000 7f42e41f5640 1 -- 172.21.15.12:0/209806380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42dc15cab0 msgr2=0x7f42dc15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:06.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.878+0000 7f42e41f5640 1 --2- 172.21.15.12:0/209806380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42dc15cab0 0x7f42dc15b200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42dc072ce0 tx=0x7f42d4007b00 comp rx=0 tx=0).stop 2023-10-30T12:35:06.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.878+0000 7f42e41f5640 1 -- 172.21.15.12:0/209806380 shutdown_connections 2023-10-30T12:35:06.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.878+0000 7f42e41f5640 1 --2- 172.21.15.12:0/209806380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42dc15cab0 0x7f42dc15b200 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.878+0000 7f42e41f5640 1 --2- 172.21.15.12:0/209806380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42dc155380 0x7f42dc15acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.878+0000 7f42e41f5640 1 --2- 172.21.15.12:0/209806380 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f42c00664d0 0x7f42c0068990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.878+0000 7f42e41f5640 1 --2- 172.21.15.12:0/209806380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42c80047d0 0x7f42dc14d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:06.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.878+0000 7f42e41f5640 1 -- 172.21.15.12:0/209806380 >> 172.21.15.12:0/209806380 conn(0x7f42dc06bfe0 msgr2=0x7f42dc06c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:06.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.878+0000 7f42e41f5640 1 -- 172.21.15.12:0/209806380 shutdown_connections 2023-10-30T12:35:06.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:06.878+0000 7f42e41f5640 1 -- 172.21.15.12:0/209806380 wait complete. 2023-10-30T12:35:06.903 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-30T12:35:07.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.046+0000 7f7bfa9c4640 1 Processor -- start 2023-10-30T12:35:07.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.046+0000 7f7bfa9c4640 1 -- start start 2023-10-30T12:35:07.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.046+0000 7f7bfa9c4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7bf4153980 0x7f7bf40caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:07.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.046+0000 7f7bfa9c4640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7bf4159930 0x7f7bf40cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:07.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.046+0000 7f7bfa9c4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf40cf1a0 0x7f7bf40cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:07.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.046+0000 7f7bfa9c4640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf415a920 con 0x7f7bf4159930 2023-10-30T12:35:07.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.046+0000 7f7bfa9c4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf40cb930 con 0x7f7bf4153980 2023-10-30T12:35:07.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.046+0000 7f7bfa9c4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf4159dc0 con 0x7f7bf40cf1a0 2023-10-30T12:35:07.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.046+0000 7f7bf37fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7bf4159930 0x7f7bf40cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:07.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bf37fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7bf4159930 0x7f7bf40cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36882/0 (socket says 172.21.15.12:36882) 2023-10-30T12:35:07.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bf37fe640 1 -- 172.21.15.12:0/3179321962 learned_addr learned my addr 172.21.15.12:0/3179321962 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:07.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bf3fff640 1 --2- 172.21.15.12:0/3179321962 >> v2:172.21.15.154:3300/0 conn(0x7f7bf4153980 0x7f7bf40caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:07.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bf8f61640 1 --2- 172.21.15.12:0/3179321962 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf40cf1a0 0x7f7bf40cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:07.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bf37fe640 1 -- 172.21.15.12:0/3179321962 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf40cf1a0 msgr2=0x7f7bf40cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:07.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bf37fe640 1 --2- 172.21.15.12:0/3179321962 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf40cf1a0 0x7f7bf40cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:07.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bf37fe640 1 -- 172.21.15.12:0/3179321962 >> v2:172.21.15.154:3300/0 conn(0x7f7bf4153980 msgr2=0x7f7bf40caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:07.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bf37fe640 1 --2- 172.21.15.12:0/3179321962 >> v2:172.21.15.154:3300/0 conn(0x7f7bf4153980 0x7f7bf40caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:07.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bf37fe640 1 -- 172.21.15.12:0/3179321962 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf415a080 con 0x7f7bf4159930 2023-10-30T12:35:07.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bf8f61640 1 --2- 172.21.15.12:0/3179321962 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf40cf1a0 0x7f7bf40cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bf37fe640 1 --2- 172.21.15.12:0/3179321962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bf4159930 0x7f7bf40cb1e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40cfd90 tx=0x7f7be801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d624d282be82221 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bf2ffd640 1 -- 172.21.15.12:0/3179321962 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be80aa5f0 con 0x7f7bf4159930 2023-10-30T12:35:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bf2ffd640 1 -- 172.21.15.12:0/3179321962 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be80aa750 con 0x7f7bf4159930 2023-10-30T12:35:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bf2ffd640 1 -- 172.21.15.12:0/3179321962 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be80aaae0 con 0x7f7bf4159930 2023-10-30T12:35:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bfa9c4640 1 -- 172.21.15.12:0/3179321962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bf4159930 msgr2=0x7f7bf40cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.050+0000 7f7bfa9c4640 1 --2- 172.21.15.12:0/3179321962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bf4159930 0x7f7bf40cb1e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40cfd90 tx=0x7f7be801c4d0 comp rx=0 tx=0).stop 2023-10-30T12:35:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bfa9c4640 1 -- 172.21.15.12:0/3179321962 shutdown_connections 2023-10-30T12:35:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bfa9c4640 1 --2- 172.21.15.12:0/3179321962 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf40cf1a0 0x7f7bf40cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:07.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bfa9c4640 1 --2- 172.21.15.12:0/3179321962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bf4159930 0x7f7bf40cb1e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:07.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bfa9c4640 1 --2- 172.21.15.12:0/3179321962 >> v2:172.21.15.154:3300/0 conn(0x7f7bf4153980 0x7f7bf40caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:07.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bfa9c4640 1 -- 172.21.15.12:0/3179321962 >> 172.21.15.12:0/3179321962 conn(0x7f7bf406c0f0 msgr2=0x7f7bf406c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:07.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bfa9c4640 1 -- 172.21.15.12:0/3179321962 shutdown_connections 2023-10-30T12:35:07.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bfa9c4640 1 -- 172.21.15.12:0/3179321962 wait complete. 2023-10-30T12:35:07.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bfa9c4640 1 Processor -- start 2023-10-30T12:35:07.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bfa9c4640 1 -- start start 2023-10-30T12:35:07.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bfa9c4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf40cf1a0 0x7f7bf414dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:07.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bfa9c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bf4153980 0x7f7bf414c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:07.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bfa9c4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf4157ab0 0x7f7bf4145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:07.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bfa9c4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf4148990 con 0x7f7bf4153980 2023-10-30T12:35:07.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bfa9c4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf40cd110 con 0x7f7bf40cf1a0 2023-10-30T12:35:07.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bfa9c4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf40b3b10 con 0x7f7bf4157ab0 2023-10-30T12:35:07.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bf37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bf4153980 0x7f7bf414c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:07.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bf37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bf4153980 0x7f7bf414c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36888/0 (socket says 172.21.15.12:36888) 2023-10-30T12:35:07.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bf37fe640 1 -- 172.21.15.12:0/364036987 learned_addr learned my addr 172.21.15.12:0/364036987 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:07.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bf3fff640 1 --2- 172.21.15.12:0/364036987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf40cf1a0 0x7f7bf414dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:07.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bf8f61640 1 --2- 172.21.15.12:0/364036987 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf4157ab0 0x7f7bf4145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:07.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bf37fe640 1 -- 172.21.15.12:0/364036987 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf4157ab0 msgr2=0x7f7bf4145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:07.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bf37fe640 1 --2- 172.21.15.12:0/364036987 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf4157ab0 0x7f7bf4145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:07.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bf37fe640 1 -- 172.21.15.12:0/364036987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf40cf1a0 msgr2=0x7f7bf414dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:07.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bf37fe640 1 --2- 172.21.15.12:0/364036987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf40cf1a0 0x7f7bf414dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:07.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.054+0000 7f7bf37fe640 1 -- 172.21.15.12:0/364036987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be80094d0 con 0x7f7bf4153980 2023-10-30T12:35:07.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.058+0000 7f7bf37fe640 1 --2- 172.21.15.12:0/364036987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bf4153980 0x7f7bf414c300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7be80ac040 tx=0x7f7be8019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:07.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.058+0000 7f7bf0ff9640 1 -- 172.21.15.12:0/364036987 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be80be610 con 0x7f7bf4153980 2023-10-30T12:35:07.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.058+0000 7f7bf0ff9640 1 -- 172.21.15.12:0/364036987 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be80be770 con 0x7f7bf4153980 2023-10-30T12:35:07.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.058+0000 7f7bfa9c4640 1 -- 172.21.15.12:0/364036987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf4149240 con 0x7f7bf4153980 2023-10-30T12:35:07.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.058+0000 7f7bf0ff9640 1 -- 172.21.15.12:0/364036987 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be80beb20 con 0x7f7bf4153980 2023-10-30T12:35:07.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.058+0000 7f7bfa9c4640 1 -- 172.21.15.12:0/364036987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf41494b0 con 0x7f7bf4153980 2023-10-30T12:35:07.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.058+0000 7f7bfa9c4640 1 -- 172.21.15.12:0/364036987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf404e170 con 0x7f7bf4153980 2023-10-30T12:35:07.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.058+0000 7f7bf0ff9640 1 -- 172.21.15.12:0/364036987 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7bf415dd80 con 0x7f7bf4153980 2023-10-30T12:35:07.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.058+0000 7f7bf0ff9640 1 --2- 172.21.15.12:0/364036987 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7be00664a0 0x7f7be0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:07.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.058+0000 7f7bf0ff9640 1 -- 172.21.15.12:0/364036987 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7be80a7070 con 0x7f7bf4153980 2023-10-30T12:35:07.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.058+0000 7f7bf3fff640 1 --2- 172.21.15.12:0/364036987 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7be00664a0 0x7f7be0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:07.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.062+0000 7f7bf3fff640 1 --2- 172.21.15.12:0/364036987 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7be00664a0 0x7f7be0068960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009af0 tx=0x7f7be4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:07.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.062+0000 7f7bf0ff9640 1 -- 172.21.15.12:0/364036987 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7be80ebf10 con 0x7f7bf4153980 2023-10-30T12:35:07.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:07.258+0000 7f7bfa9c4640 1 -- 172.21.15.12:0/364036987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7bf4164090 con 0x7f7bf4153980 2023-10-30T12:35:08.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.170+0000 7f7bf0ff9640 1 -- 172.21.15.12:0/364036987 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7be80ef530 con 0x7f7bf4153980 2023-10-30T12:35:08.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.174+0000 7f7bfa9c4640 1 -- 172.21.15.12:0/364036987 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7be00664a0 msgr2=0x7f7be0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:08.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.174+0000 7f7bfa9c4640 1 --2- 172.21.15.12:0/364036987 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7be00664a0 0x7f7be0068960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009af0 tx=0x7f7be4008040 comp rx=0 tx=0).stop 2023-10-30T12:35:08.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.174+0000 7f7bfa9c4640 1 -- 172.21.15.12:0/364036987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bf4153980 msgr2=0x7f7bf414c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:08.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.174+0000 7f7bfa9c4640 1 --2- 172.21.15.12:0/364036987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bf4153980 0x7f7bf414c300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7be80ac040 tx=0x7f7be8019270 comp rx=0 tx=0).stop 2023-10-30T12:35:08.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.174+0000 7f7bfa9c4640 1 -- 172.21.15.12:0/364036987 shutdown_connections 2023-10-30T12:35:08.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.174+0000 7f7bfa9c4640 1 --2- 172.21.15.12:0/364036987 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7be00664a0 0x7f7be0068960 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:08.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.174+0000 7f7bfa9c4640 1 --2- 172.21.15.12:0/364036987 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bf4157ab0 0x7f7bf4145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:08.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.174+0000 7f7bfa9c4640 1 --2- 172.21.15.12:0/364036987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bf4153980 0x7f7bf414c300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:08.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.174+0000 7f7bfa9c4640 1 --2- 172.21.15.12:0/364036987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bf40cf1a0 0x7f7bf414dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:08.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.174+0000 7f7bfa9c4640 1 -- 172.21.15.12:0/364036987 >> 172.21.15.12:0/364036987 conn(0x7f7bf406c0f0 msgr2=0x7f7bf40c4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:08.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.174+0000 7f7bfa9c4640 1 -- 172.21.15.12:0/364036987 shutdown_connections 2023-10-30T12:35:08.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.174+0000 7f7bfa9c4640 1 -- 172.21.15.12:0/364036987 wait complete. 2023-10-30T12:35:08.201 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300} 2023-10-30T12:35:08.202 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-10-30T12:35:08.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.350+0000 7ffa8cbd3640 1 Processor -- start 2023-10-30T12:35:08.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.350+0000 7ffa8cbd3640 1 -- start start 2023-10-30T12:35:08.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.350+0000 7ffa8cbd3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffa88153980 0x7ffa880caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:08.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.350+0000 7ffa8cbd3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ffa880cb1e0 0x7ffa8815d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:08.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.350+0000 7ffa8cbd3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa8815a920 con 0x7ffa88159930 2023-10-30T12:35:08.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.350+0000 7ffa8cbd3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa880cb930 con 0x7ffa880cb1e0 2023-10-30T12:35:08.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.350+0000 7ffa8cbd3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa88159dc0 con 0x7ffa88153980 2023-10-30T12:35:08.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.350+0000 7ffa86d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ffa880cb1e0 0x7ffa8815d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:08.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.350+0000 7ffa86d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ffa880cb1e0 0x7ffa8815d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36904/0 (socket says 172.21.15.12:36904) 2023-10-30T12:35:08.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.350+0000 7ffa86d9d640 1 -- 172.21.15.12:0/780346039 learned_addr learned my addr 172.21.15.12:0/780346039 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:08.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.350+0000 7ffa8659c640 1 --2- 172.21.15.12:0/780346039 >> v2:172.21.15.154:3300/0 conn(0x7ffa88153980 0x7ffa880caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:08.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.350+0000 7ffa86d9d640 1 -- 172.21.15.12:0/780346039 >> v2:172.21.15.154:3300/0 conn(0x7ffa88153980 msgr2=0x7ffa880caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:08.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.350+0000 7ffa86d9d640 1 --2- 172.21.15.12:0/780346039 >> v2:172.21.15.154:3300/0 conn(0x7ffa88153980 0x7ffa880caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:08.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.350+0000 7ffa86d9d640 1 -- 172.21.15.12:0/780346039 >> v1:172.21.15.154:6789/0 conn(0x7ffa88159930 legacy=0x7ffa880cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:08.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.350+0000 7ffa86d9d640 1 -- 172.21.15.12:0/780346039 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8815a080 con 0x7ffa880cb1e0 2023-10-30T12:35:08.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.354+0000 7ffa86d9d640 1 --2- 172.21.15.12:0/780346039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa880cb1e0 0x7ffa8815d570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffa880cfd90 tx=0x7ffa8001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f000083364c279 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:08.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.354+0000 7ffa8559a640 1 -- 172.21.15.12:0/780346039 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa800a5770 con 0x7ffa880cb1e0 2023-10-30T12:35:08.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.354+0000 7ffa8559a640 1 -- 172.21.15.12:0/780346039 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa800a58d0 con 0x7ffa880cb1e0 2023-10-30T12:35:08.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.354+0000 7ffa8559a640 1 -- 172.21.15.12:0/780346039 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa800a5c60 con 0x7ffa880cb1e0 2023-10-30T12:35:08.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.354+0000 7ffa8cbd3640 1 -- 172.21.15.12:0/780346039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa880cb1e0 msgr2=0x7ffa8815d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:08.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.354+0000 7ffa8cbd3640 1 --2- 172.21.15.12:0/780346039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa880cb1e0 0x7ffa8815d570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffa880cfd90 tx=0x7ffa8001c9a0 comp rx=0 tx=0).stop 2023-10-30T12:35:08.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.354+0000 7ffa8cbd3640 1 -- 172.21.15.12:0/780346039 shutdown_connections 2023-10-30T12:35:08.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.354+0000 7ffa8cbd3640 1 --2- 172.21.15.12:0/780346039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa880cb1e0 0x7ffa8815d570 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:08.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.354+0000 7ffa8cbd3640 1 --2- 172.21.15.12:0/780346039 >> v2:172.21.15.154:3300/0 conn(0x7ffa88153980 0x7ffa880caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:08.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.354+0000 7ffa8cbd3640 1 -- 172.21.15.12:0/780346039 >> 172.21.15.12:0/780346039 conn(0x7ffa8806c0f0 msgr2=0x7ffa8806c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:08.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.354+0000 7ffa8cbd3640 1 -- 172.21.15.12:0/780346039 shutdown_connections 2023-10-30T12:35:08.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.354+0000 7ffa8cbd3640 1 -- 172.21.15.12:0/780346039 wait complete. 2023-10-30T12:35:08.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8cbd3640 1 Processor -- start 2023-10-30T12:35:08.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8cbd3640 1 -- start start 2023-10-30T12:35:08.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8cbd3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa880cb1e0 0x7ffa8814f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:08.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8cbd3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa88153980 0x7ffa88151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:08.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8cbd3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa88142e10 0x7ffa88143280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:08.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8cbd3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880b4fc0 con 0x7ffa880cb1e0 2023-10-30T12:35:08.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8cbd3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa881429f0 con 0x7ffa88153980 2023-10-30T12:35:08.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8cbd3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa88151730 con 0x7ffa88142e10 2023-10-30T12:35:08.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa880cb1e0 0x7ffa8814f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:08.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa880cb1e0 0x7ffa8814f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36908/0 (socket says 172.21.15.12:36908) 2023-10-30T12:35:08.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8659c640 1 -- 172.21.15.12:0/1382417812 learned_addr learned my addr 172.21.15.12:0/1382417812 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:08.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa85d9b640 1 --2- 172.21.15.12:0/1382417812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa88153980 0x7ffa88151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:08.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa86d9d640 1 --2- 172.21.15.12:0/1382417812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa88142e10 0x7ffa88143280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:08.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8659c640 1 -- 172.21.15.12:0/1382417812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa88142e10 msgr2=0x7ffa88143280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:08.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8659c640 1 --2- 172.21.15.12:0/1382417812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa88142e10 0x7ffa88143280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:08.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8659c640 1 -- 172.21.15.12:0/1382417812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa88153980 msgr2=0x7ffa88151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:08.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8659c640 1 --2- 172.21.15.12:0/1382417812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa88153980 0x7ffa88151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:08.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8659c640 1 -- 172.21.15.12:0/1382417812 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa80009b30 con 0x7ffa880cb1e0 2023-10-30T12:35:08.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa86d9d640 1 --2- 172.21.15.12:0/1382417812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa88142e10 0x7ffa88143280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:35:08.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8659c640 1 --2- 172.21.15.12:0/1382417812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa880cb1e0 0x7ffa8814f740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800e870 tx=0x7ffa7800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:08.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa76ffd640 1 -- 172.21.15.12:0/1382417812 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78009530 con 0x7ffa880cb1e0 2023-10-30T12:35:08.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa76ffd640 1 -- 172.21.15.12:0/1382417812 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7800c8e0 con 0x7ffa880cb1e0 2023-10-30T12:35:08.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8cbd3640 1 -- 172.21.15.12:0/1382417812 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa881437c0 con 0x7ffa880cb1e0 2023-10-30T12:35:08.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa76ffd640 1 -- 172.21.15.12:0/1382417812 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7800cc70 con 0x7ffa880cb1e0 2023-10-30T12:35:08.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.358+0000 7ffa8cbd3640 1 -- 172.21.15.12:0/1382417812 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8814eac0 con 0x7ffa880cb1e0 2023-10-30T12:35:08.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.362+0000 7ffa8cbd3640 1 -- 172.21.15.12:0/1382417812 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8804e170 con 0x7ffa880cb1e0 2023-10-30T12:35:08.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.362+0000 7ffa76ffd640 1 -- 172.21.15.12:0/1382417812 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ffa78013470 con 0x7ffa880cb1e0 2023-10-30T12:35:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.362+0000 7ffa76ffd640 1 --2- 172.21.15.12:0/1382417812 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa680664d0 0x7ffa68068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.362+0000 7ffa76ffd640 1 -- 172.21.15.12:0/1382417812 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa78084230 con 0x7ffa880cb1e0 2023-10-30T12:35:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.362+0000 7ffa85d9b640 1 --2- 172.21.15.12:0/1382417812 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa680664d0 0x7ffa68068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.362+0000 7ffa85d9b640 1 --2- 172.21.15.12:0/1382417812 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa680664d0 0x7ffa68068990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffa88144680 tx=0x7ffa7c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.366+0000 7ffa76ffd640 1 -- 172.21.15.12:0/1382417812 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ffa780516a0 con 0x7ffa880cb1e0 2023-10-30T12:35:08.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:08.558+0000 7ffa8cbd3640 1 -- 172.21.15.12:0/1382417812 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7ffa8815d050 con 0x7ffa880cb1e0 2023-10-30T12:35:09.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.182+0000 7ffa76ffd640 1 -- 172.21.15.12:0/1382417812 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7ffa78054cc0 con 0x7ffa880cb1e0 2023-10-30T12:35:09.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.182+0000 7ffa8cbd3640 1 -- 172.21.15.12:0/1382417812 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa680664d0 msgr2=0x7ffa68068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:09.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.182+0000 7ffa8cbd3640 1 --2- 172.21.15.12:0/1382417812 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa680664d0 0x7ffa68068990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffa88144680 tx=0x7ffa7c009040 comp rx=0 tx=0).stop 2023-10-30T12:35:09.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.182+0000 7ffa8cbd3640 1 -- 172.21.15.12:0/1382417812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa880cb1e0 msgr2=0x7ffa8814f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:09.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.182+0000 7ffa8cbd3640 1 --2- 172.21.15.12:0/1382417812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa880cb1e0 0x7ffa8814f740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800e870 tx=0x7ffa7800ed30 comp rx=0 tx=0).stop 2023-10-30T12:35:09.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.182+0000 7ffa8cbd3640 1 -- 172.21.15.12:0/1382417812 shutdown_connections 2023-10-30T12:35:09.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.182+0000 7ffa8cbd3640 1 --2- 172.21.15.12:0/1382417812 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa680664d0 0x7ffa68068990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:09.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.182+0000 7ffa8cbd3640 1 --2- 172.21.15.12:0/1382417812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa88142e10 0x7ffa88143280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:09.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.182+0000 7ffa8cbd3640 1 --2- 172.21.15.12:0/1382417812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa88153980 0x7ffa88151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:09.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.182+0000 7ffa8cbd3640 1 --2- 172.21.15.12:0/1382417812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa880cb1e0 0x7ffa8814f740 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:09.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.182+0000 7ffa8cbd3640 1 -- 172.21.15.12:0/1382417812 >> 172.21.15.12:0/1382417812 conn(0x7ffa8806c0f0 msgr2=0x7ffa88158800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:09.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.182+0000 7ffa8cbd3640 1 -- 172.21.15.12:0/1382417812 shutdown_connections 2023-10-30T12:35:09.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.186+0000 7ffa8cbd3640 1 -- 172.21.15.12:0/1382417812 wait complete. 2023-10-30T12:35:09.211 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-30T12:35:09.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.358+0000 7f15325d3640 1 Processor -- start 2023-10-30T12:35:09.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.362+0000 7f15325d3640 1 -- start start 2023-10-30T12:35:09.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.362+0000 7f15325d3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f152c0c5810 0x7f152c0c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:09.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.362+0000 7f15325d3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f152c0c6130 0x7f152c06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:09.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.362+0000 7f15325d3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152c0c6cb0 con 0x7f152c06f880 2023-10-30T12:35:09.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.362+0000 7f15325d3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152c071d90 con 0x7f152c0c6130 2023-10-30T12:35:09.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.362+0000 7f15325d3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152c1585d0 con 0x7f152c0c5810 2023-10-30T12:35:09.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.362+0000 7f152b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f152c0c6130 0x7f152c06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:09.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.362+0000 7f152b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f152c0c6130 0x7f152c06a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36918/0 (socket says 172.21.15.12:36918) 2023-10-30T12:35:09.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.362+0000 7f152b7fe640 1 -- 172.21.15.12:0/3454399366 learned_addr learned my addr 172.21.15.12:0/3454399366 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:09.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.362+0000 7f152bfff640 1 --2- 172.21.15.12:0/3454399366 >> v2:172.21.15.154:3300/0 conn(0x7f152c0c5810 0x7f152c0c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.362+0000 7f152b7fe640 1 -- 172.21.15.12:0/3454399366 >> v2:172.21.15.154:3300/0 conn(0x7f152c0c5810 msgr2=0x7f152c0c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.362+0000 7f152b7fe640 1 --2- 172.21.15.12:0/3454399366 >> v2:172.21.15.154:3300/0 conn(0x7f152c0c5810 0x7f152c0c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.362+0000 7f152b7fe640 1 -- 172.21.15.12:0/3454399366 >> v1:172.21.15.154:6789/0 conn(0x7f152c06f880 legacy=0x7f152c15ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.362+0000 7f152b7fe640 1 -- 172.21.15.12:0/3454399366 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c0c6620 con 0x7f152c0c6130 2023-10-30T12:35:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.366+0000 7f152b7fe640 1 --2- 172.21.15.12:0/3454399366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152c0c6130 0x7f152c06a270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f152c06b380 tx=0x7f152001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa2b90db7bf7b46 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.366+0000 7f152affd640 1 -- 172.21.15.12:0/3454399366 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15200ab600 con 0x7f152c0c6130 2023-10-30T12:35:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.366+0000 7f152affd640 1 -- 172.21.15.12:0/3454399366 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15200a7050 con 0x7f152c0c6130 2023-10-30T12:35:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.366+0000 7f152affd640 1 -- 172.21.15.12:0/3454399366 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15200ab980 con 0x7f152c0c6130 2023-10-30T12:35:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.366+0000 7f15325d3640 1 -- 172.21.15.12:0/3454399366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152c0c6130 msgr2=0x7f152c06a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:09.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.366+0000 7f15325d3640 1 --2- 172.21.15.12:0/3454399366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152c0c6130 0x7f152c06a270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f152c06b380 tx=0x7f152001c3e0 comp rx=0 tx=0).stop 2023-10-30T12:35:09.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.366+0000 7f15325d3640 1 -- 172.21.15.12:0/3454399366 shutdown_connections 2023-10-30T12:35:09.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.366+0000 7f15325d3640 1 --2- 172.21.15.12:0/3454399366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152c0c6130 0x7f152c06a270 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:09.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.366+0000 7f15325d3640 1 --2- 172.21.15.12:0/3454399366 >> v2:172.21.15.154:3300/0 conn(0x7f152c0c5810 0x7f152c0c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:09.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.366+0000 7f15325d3640 1 -- 172.21.15.12:0/3454399366 >> 172.21.15.12:0/3454399366 conn(0x7f152c0c7910 msgr2=0x7f152c0c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:09.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.366+0000 7f15325d3640 1 -- 172.21.15.12:0/3454399366 shutdown_connections 2023-10-30T12:35:09.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.366+0000 7f15325d3640 1 -- 172.21.15.12:0/3454399366 wait complete. 2023-10-30T12:35:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.366+0000 7f15325d3640 1 Processor -- start 2023-10-30T12:35:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.366+0000 7f15325d3640 1 -- start start 2023-10-30T12:35:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f15325d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f152c06f880 0x7f152c1f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f15325d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152c0c5810 0x7f152c1561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f15325d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152c0c6130 0x7f152c156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:09.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f15325d3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c0c1ac0 con 0x7f152c0c6130 2023-10-30T12:35:09.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f15325d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c1f5f50 con 0x7f152c06f880 2023-10-30T12:35:09.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f15325d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152c0c1d10 con 0x7f152c0c5810 2023-10-30T12:35:09.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f1530b70640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152c0c6130 0x7f152c156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:09.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f152bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f152c06f880 0x7f152c1f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:09.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f152bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f152c06f880 0x7f152c1f20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:51174/0 (socket says 172.21.15.12:51174) 2023-10-30T12:35:09.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f152bfff640 1 -- 172.21.15.12:0/3966538348 learned_addr learned my addr 172.21.15.12:0/3966538348 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:09.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f1530b70640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152c0c6130 0x7f152c156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36934/0 (socket says 172.21.15.12:36934) 2023-10-30T12:35:09.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f152b7fe640 1 --2- 172.21.15.12:0/3966538348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152c0c5810 0x7f152c1561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:09.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f1530b70640 1 -- 172.21.15.12:0/3966538348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152c0c5810 msgr2=0x7f152c1561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:09.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f1530b70640 1 --2- 172.21.15.12:0/3966538348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152c0c5810 0x7f152c1561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:09.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f1530b70640 1 -- 172.21.15.12:0/3966538348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f152c06f880 msgr2=0x7f152c1f20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:09.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f1530b70640 1 --2- 172.21.15.12:0/3966538348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f152c06f880 0x7f152c1f20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:09.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f1530b70640 1 -- 172.21.15.12:0/3966538348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15200094d0 con 0x7f152c0c6130 2023-10-30T12:35:09.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f1530b70640 1 --2- 172.21.15.12:0/3966538348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152c0c6130 0x7f152c156700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f152400c730 tx=0x7f152400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:09.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f1528ff9640 1 -- 172.21.15.12:0/3966538348 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524003b80 con 0x7f152c0c6130 2023-10-30T12:35:09.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f1528ff9640 1 -- 172.21.15.12:0/3966538348 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1524003ce0 con 0x7f152c0c6130 2023-10-30T12:35:09.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f1528ff9640 1 -- 172.21.15.12:0/3966538348 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524005310 con 0x7f152c0c6130 2023-10-30T12:35:09.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f15325d3640 1 -- 172.21.15.12:0/3966538348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152c0c1ff0 con 0x7f152c0c6130 2023-10-30T12:35:09.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f15325d3640 1 -- 172.21.15.12:0/3966538348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152c1f5900 con 0x7f152c0c6130 2023-10-30T12:35:09.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.370+0000 7f15325d3640 1 -- 172.21.15.12:0/3966538348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152c0c75d0 con 0x7f152c0c6130 2023-10-30T12:35:09.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.374+0000 7f1528ff9640 1 -- 172.21.15.12:0/3966538348 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1524005680 con 0x7f152c0c6130 2023-10-30T12:35:09.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.374+0000 7f1528ff9640 1 --2- 172.21.15.12:0/3966538348 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f15180664f0 0x7f15180689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:09.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.374+0000 7f1528ff9640 1 -- 172.21.15.12:0/3966538348 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1524083be0 con 0x7f152c0c6130 2023-10-30T12:35:09.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.374+0000 7f152bfff640 1 --2- 172.21.15.12:0/3966538348 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f15180664f0 0x7f15180689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:09.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.374+0000 7f152bfff640 1 --2- 172.21.15.12:0/3966538348 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f15180664f0 0x7f15180689b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f151c009b60 tx=0x7f151c006f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:09.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.378+0000 7f1528ff9640 1 -- 172.21.15.12:0/3966538348 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1524051160 con 0x7f152c0c6130 2023-10-30T12:35:09.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:09.566+0000 7f15325d3640 1 -- 172.21.15.12:0/3966538348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f152c04e170 con 0x7f152c0c6130 2023-10-30T12:35:10.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.182+0000 7f1528ff9640 1 -- 172.21.15.12:0/3966538348 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1524054780 con 0x7f152c0c6130 2023-10-30T12:35:10.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.186+0000 7f15325d3640 1 -- 172.21.15.12:0/3966538348 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f15180664f0 msgr2=0x7f15180689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:10.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.186+0000 7f15325d3640 1 --2- 172.21.15.12:0/3966538348 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f15180664f0 0x7f15180689b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f151c009b60 tx=0x7f151c006f90 comp rx=0 tx=0).stop 2023-10-30T12:35:10.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.186+0000 7f15325d3640 1 -- 172.21.15.12:0/3966538348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152c0c6130 msgr2=0x7f152c156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:10.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.186+0000 7f15325d3640 1 --2- 172.21.15.12:0/3966538348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152c0c6130 0x7f152c156700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f152400c730 tx=0x7f152400cbf0 comp rx=0 tx=0).stop 2023-10-30T12:35:10.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.186+0000 7f15325d3640 1 -- 172.21.15.12:0/3966538348 shutdown_connections 2023-10-30T12:35:10.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.186+0000 7f15325d3640 1 --2- 172.21.15.12:0/3966538348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152c0c6130 0x7f152c156700 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:10.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.186+0000 7f15325d3640 1 --2- 172.21.15.12:0/3966538348 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f15180664f0 0x7f15180689b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:10.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.186+0000 7f15325d3640 1 --2- 172.21.15.12:0/3966538348 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152c0c5810 0x7f152c1561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:10.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.186+0000 7f15325d3640 1 --2- 172.21.15.12:0/3966538348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f152c06f880 0x7f152c1f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:10.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.186+0000 7f15325d3640 1 -- 172.21.15.12:0/3966538348 >> 172.21.15.12:0/3966538348 conn(0x7f152c0c7910 msgr2=0x7f152c0c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:10.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.186+0000 7f15325d3640 1 -- 172.21.15.12:0/3966538348 shutdown_connections 2023-10-30T12:35:10.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.186+0000 7f15325d3640 1 -- 172.21.15.12:0/3966538348 wait complete. 2023-10-30T12:35:10.215 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-30T12:35:10.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f2984032640 1 Processor -- start 2023-10-30T12:35:10.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f2984032640 1 -- start start 2023-10-30T12:35:10.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f2984032640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f297c0722a0 0x7f297c0bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:10.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f2984032640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f297c0c0400 0x7f297c156930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:10.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f2984032640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297c0c0a30 con 0x7f297c1508c0 2023-10-30T12:35:10.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f2984032640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297c0c85b0 con 0x7f297c0722a0 2023-10-30T12:35:10.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f2984032640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297c0c0850 con 0x7f297c0c0400 2023-10-30T12:35:10.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f29815cd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f297c0722a0 0x7f297c0bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:10.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f29815cd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f297c0722a0 0x7f297c0bfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36942/0 (socket says 172.21.15.12:36942) 2023-10-30T12:35:10.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f29825cf640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f297c0c0400 0x7f297c156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:10.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f29815cd640 1 -- 172.21.15.12:0/4007893807 learned_addr learned my addr 172.21.15.12:0/4007893807 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:10.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f2981dce640 1 --1- 172.21.15.12:0/4007893807 >> v1:172.21.15.12:6789/0 conn(0x7f297c1508c0 0x7f297c0c6920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54220/0 (socket says 172.21.15.12:54220) 2023-10-30T12:35:10.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f29815cd640 1 -- 172.21.15.12:0/4007893807 >> v2:172.21.15.154:3300/0 conn(0x7f297c0c0400 msgr2=0x7f297c156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:10.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f29815cd640 1 --2- 172.21.15.12:0/4007893807 >> v2:172.21.15.154:3300/0 conn(0x7f297c0c0400 0x7f297c156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:10.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f29815cd640 1 -- 172.21.15.12:0/4007893807 >> v1:172.21.15.12:6789/0 conn(0x7f297c1508c0 legacy=0x7f297c0c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.362+0000 7f29815cd640 1 -- 172.21.15.12:0/4007893807 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c06f650 con 0x7f297c0722a0 2023-10-30T12:35:10.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.366+0000 7f29815cd640 1 --2- 172.21.15.12:0/4007893807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f297c0722a0 0x7f297c0bfec0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f297c0c2140 tx=0x7f296c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f70a41cfa9239f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:10.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.366+0000 7f2980dcc640 1 -- 172.21.15.12:0/4007893807 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0ad660 con 0x7f297c0722a0 2023-10-30T12:35:10.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.366+0000 7f2980dcc640 1 -- 172.21.15.12:0/4007893807 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c0ad7c0 con 0x7f297c0722a0 2023-10-30T12:35:10.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.366+0000 7f2980dcc640 1 -- 172.21.15.12:0/4007893807 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0adb50 con 0x7f297c0722a0 2023-10-30T12:35:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.366+0000 7f2984032640 1 -- 172.21.15.12:0/4007893807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f297c0722a0 msgr2=0x7f297c0bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.366+0000 7f2984032640 1 --2- 172.21.15.12:0/4007893807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f297c0722a0 0x7f297c0bfec0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f297c0c2140 tx=0x7f296c01c470 comp rx=0 tx=0).stop 2023-10-30T12:35:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.366+0000 7f2984032640 1 -- 172.21.15.12:0/4007893807 shutdown_connections 2023-10-30T12:35:10.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.366+0000 7f2984032640 1 --2- 172.21.15.12:0/4007893807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f297c0722a0 0x7f297c0bfec0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:10.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.366+0000 7f2984032640 1 --2- 172.21.15.12:0/4007893807 >> v2:172.21.15.154:3300/0 conn(0x7f297c0c0400 0x7f297c156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.366+0000 7f2984032640 1 -- 172.21.15.12:0/4007893807 >> 172.21.15.12:0/4007893807 conn(0x7f297c0c7910 msgr2=0x7f297c0c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.366+0000 7f2984032640 1 -- 172.21.15.12:0/4007893807 shutdown_connections 2023-10-30T12:35:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.366+0000 7f2984032640 1 -- 172.21.15.12:0/4007893807 wait complete. 2023-10-30T12:35:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f2984032640 1 Processor -- start 2023-10-30T12:35:10.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f2984032640 1 -- start start 2023-10-30T12:35:10.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f2984032640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c0722a0 0x7f297c0c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:10.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f2984032640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f297c0c0400 0x7f297c13d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:10.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f2984032640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f297c1508c0 0x7f297c1621b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:10.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f2984032640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c0cab40 con 0x7f297c0c0400 2023-10-30T12:35:10.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f2984032640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c163940 con 0x7f297c0722a0 2023-10-30T12:35:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f2984032640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297c0c9dd0 con 0x7f297c1508c0 2023-10-30T12:35:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f29815cd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f297c0c0400 0x7f297c13d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f29815cd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f297c0c0400 0x7f297c13d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36946/0 (socket says 172.21.15.12:36946) 2023-10-30T12:35:10.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f29815cd640 1 -- 172.21.15.12:0/203790515 learned_addr learned my addr 172.21.15.12:0/203790515 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:10.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f2981dce640 1 --2- 172.21.15.12:0/203790515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c0722a0 0x7f297c0c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:10.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f29825cf640 1 --2- 172.21.15.12:0/203790515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f297c1508c0 0x7f297c1621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:10.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f29815cd640 1 -- 172.21.15.12:0/203790515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f297c1508c0 msgr2=0x7f297c1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:10.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f29815cd640 1 --2- 172.21.15.12:0/203790515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f297c1508c0 0x7f297c1621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:10.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f29815cd640 1 -- 172.21.15.12:0/203790515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c0722a0 msgr2=0x7f297c0c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:10.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f29815cd640 1 --2- 172.21.15.12:0/203790515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c0722a0 0x7f297c0c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:10.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f29815cd640 1 -- 172.21.15.12:0/203790515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c0094d0 con 0x7f297c0c0400 2023-10-30T12:35:10.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f29825cf640 1 --2- 172.21.15.12:0/203790515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f297c1508c0 0x7f297c1621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:10.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f29815cd640 1 --2- 172.21.15.12:0/203790515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f297c0c0400 0x7f297c13d6f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f296c019080 tx=0x7f296c019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:10.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f29727fc640 1 -- 172.21.15.12:0/203790515 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0b6440 con 0x7f297c0c0400 2023-10-30T12:35:10.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f29727fc640 1 -- 172.21.15.12:0/203790515 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c0b65a0 con 0x7f297c0c0400 2023-10-30T12:35:10.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f29727fc640 1 -- 172.21.15.12:0/203790515 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0b6930 con 0x7f297c0c0400 2023-10-30T12:35:10.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f2984032640 1 -- 172.21.15.12:0/203790515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297c1636a0 con 0x7f297c0c0400 2023-10-30T12:35:10.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f2984032640 1 -- 172.21.15.12:0/203790515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297c1f3090 con 0x7f297c0c0400 2023-10-30T12:35:10.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.370+0000 7f2984032640 1 -- 172.21.15.12:0/203790515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297c161f70 con 0x7f297c0c0400 2023-10-30T12:35:10.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.374+0000 7f29727fc640 1 -- 172.21.15.12:0/203790515 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f296c0b4070 con 0x7f297c0c0400 2023-10-30T12:35:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.374+0000 7f29727fc640 1 --2- 172.21.15.12:0/203790515 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f29600664f0 0x7f29600689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.374+0000 7f29727fc640 1 -- 172.21.15.12:0/203790515 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f296c11de90 con 0x7f297c0c0400 2023-10-30T12:35:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.374+0000 7f2981dce640 1 --2- 172.21.15.12:0/203790515 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f29600664f0 0x7f29600689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.374+0000 7f2981dce640 1 --2- 172.21.15.12:0/203790515 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f29600664f0 0x7f29600689b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2978007fd0 tx=0x7f2978009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:10.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.378+0000 7f29727fc640 1 -- 172.21.15.12:0/203790515 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f296c0eb380 con 0x7f297c0c0400 2023-10-30T12:35:10.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:10.566+0000 7f2984032640 1 -- 172.21.15.12:0/203790515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f297c04e170 con 0x7f297c0c0400 2023-10-30T12:35:11.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.206+0000 7f29727fc640 1 -- 172.21.15.12:0/203790515 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) v1 ==== 115+0+0 (secure 0 0 0) 0x7f296c0ee9a0 con 0x7f297c0c0400 2023-10-30T12:35:11.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.210+0000 7f2984032640 1 -- 172.21.15.12:0/203790515 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f29600664f0 msgr2=0x7f29600689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:11.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.210+0000 7f2984032640 1 --2- 172.21.15.12:0/203790515 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f29600664f0 0x7f29600689b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2978007fd0 tx=0x7f2978009040 comp rx=0 tx=0).stop 2023-10-30T12:35:11.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.210+0000 7f2984032640 1 -- 172.21.15.12:0/203790515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f297c0c0400 msgr2=0x7f297c13d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:11.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.210+0000 7f2984032640 1 --2- 172.21.15.12:0/203790515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f297c0c0400 0x7f297c13d6f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f296c019080 tx=0x7f296c019210 comp rx=0 tx=0).stop 2023-10-30T12:35:11.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.210+0000 7f2984032640 1 -- 172.21.15.12:0/203790515 shutdown_connections 2023-10-30T12:35:11.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.210+0000 7f2984032640 1 --2- 172.21.15.12:0/203790515 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f29600664f0 0x7f29600689b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.210+0000 7f2984032640 1 --2- 172.21.15.12:0/203790515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f297c1508c0 0x7f297c1621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.210+0000 7f2984032640 1 --2- 172.21.15.12:0/203790515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f297c0c0400 0x7f297c13d6f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.210+0000 7f2984032640 1 --2- 172.21.15.12:0/203790515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c0722a0 0x7f297c0c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.210+0000 7f2984032640 1 -- 172.21.15.12:0/203790515 >> 172.21.15.12:0/203790515 conn(0x7f297c0c7910 msgr2=0x7f297c0c5820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:11.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.210+0000 7f2984032640 1 -- 172.21.15.12:0/203790515 shutdown_connections 2023-10-30T12:35:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.210+0000 7f2984032640 1 -- 172.21.15.12:0/203790515 wait complete. 2023-10-30T12:35:11.233 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-30T12:35:11.233 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:35:11.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.374+0000 7f7220846640 1 Processor -- start 2023-10-30T12:35:11.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.374+0000 7f7220846640 1 -- start start 2023-10-30T12:35:11.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.378+0000 7f7220846640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f721c1538a0 0x7f721c15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:11.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.378+0000 7f7220846640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f721c0cf160 0x7f721c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:11.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.378+0000 7f7220846640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721c0cb800 con 0x7f721c0ca0e0 2023-10-30T12:35:11.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.378+0000 7f7220846640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c15dc30 con 0x7f721c0cf160 2023-10-30T12:35:11.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.378+0000 7f7220846640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c0cb330 con 0x7f721c1538a0 2023-10-30T12:35:11.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.378+0000 7f721ad9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f721c0cf160 0x7f721c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:11.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.378+0000 7f721ad9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f721c0cf160 0x7f721c0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44148/0 (socket says 172.21.15.12:44148) 2023-10-30T12:35:11.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.378+0000 7f721ad9d640 1 -- 172.21.15.12:0/950904441 learned_addr learned my addr 172.21.15.12:0/950904441 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:11.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.378+0000 7f721a59c640 1 --2- 172.21.15.12:0/950904441 >> v2:172.21.15.154:3300/0 conn(0x7f721c1538a0 0x7f721c15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:11.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.378+0000 7f721ad9d640 1 -- 172.21.15.12:0/950904441 >> v2:172.21.15.154:3300/0 conn(0x7f721c1538a0 msgr2=0x7f721c15d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:11.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.378+0000 7f721ad9d640 1 --2- 172.21.15.12:0/950904441 >> v2:172.21.15.154:3300/0 conn(0x7f721c1538a0 0x7f721c15d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.378+0000 7f721ad9d640 1 -- 172.21.15.12:0/950904441 >> v1:172.21.15.154:6789/0 conn(0x7f721c0ca0e0 legacy=0x7f721c15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:11.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.378+0000 7f721ad9d640 1 -- 172.21.15.12:0/950904441 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c0cad70 con 0x7f721c0cf160 2023-10-30T12:35:11.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.378+0000 7f721ad9d640 1 --2- 172.21.15.12:0/950904441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f721c0cf160 0x7f721c0cf590 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f721c0cfd10 tx=0x7f721401f180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22f133cb876dca9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:11.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f721959a640 1 -- 172.21.15.12:0/950904441 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72140af400 con 0x7f721c0cf160 2023-10-30T12:35:11.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f721959a640 1 -- 172.21.15.12:0/950904441 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72140ab050 con 0x7f721c0cf160 2023-10-30T12:35:11.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f721959a640 1 -- 172.21.15.12:0/950904441 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72140af780 con 0x7f721c0cf160 2023-10-30T12:35:11.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 -- 172.21.15.12:0/950904441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f721c0cf160 msgr2=0x7f721c0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:11.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 --2- 172.21.15.12:0/950904441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f721c0cf160 0x7f721c0cf590 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f721c0cfd10 tx=0x7f721401f180 comp rx=0 tx=0).stop 2023-10-30T12:35:11.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 -- 172.21.15.12:0/950904441 shutdown_connections 2023-10-30T12:35:11.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 --2- 172.21.15.12:0/950904441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f721c0cf160 0x7f721c0cf590 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 --2- 172.21.15.12:0/950904441 >> v2:172.21.15.154:3300/0 conn(0x7f721c1538a0 0x7f721c15d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 -- 172.21.15.12:0/950904441 >> 172.21.15.12:0/950904441 conn(0x7f721c06bfe0 msgr2=0x7f721c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:11.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 -- 172.21.15.12:0/950904441 shutdown_connections 2023-10-30T12:35:11.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 -- 172.21.15.12:0/950904441 wait complete. 2023-10-30T12:35:11.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 Processor -- start 2023-10-30T12:35:11.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 -- start start 2023-10-30T12:35:11.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f721c0ca0e0 0x7f721c0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:11.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f721c0cf160 0x7f721c0b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:11.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f721c1538a0 0x7f721c1438b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:11.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0c9b80 con 0x7f721c0ca0e0 2023-10-30T12:35:11.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0b31a0 con 0x7f721c0cf160 2023-10-30T12:35:11.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.382+0000 7f7220846640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0c98c0 con 0x7f721c1538a0 2023-10-30T12:35:11.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f721a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f721c0ca0e0 0x7f721c0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:11.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f721a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f721c0ca0e0 0x7f721c0b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44150/0 (socket says 172.21.15.12:44150) 2023-10-30T12:35:11.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f721a59c640 1 -- 172.21.15.12:0/1638288750 learned_addr learned my addr 172.21.15.12:0/1638288750 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:11.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f721ad9d640 1 --2- 172.21.15.12:0/1638288750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f721c1538a0 0x7f721c1438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:11.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f7219d9b640 1 --2- 172.21.15.12:0/1638288750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f721c0cf160 0x7f721c0b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:11.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f721a59c640 1 -- 172.21.15.12:0/1638288750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f721c1538a0 msgr2=0x7f721c1438b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:11.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f721a59c640 1 --2- 172.21.15.12:0/1638288750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f721c1538a0 0x7f721c1438b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f721a59c640 1 -- 172.21.15.12:0/1638288750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f721c0cf160 msgr2=0x7f721c0b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:11.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f721a59c640 1 --2- 172.21.15.12:0/1638288750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f721c0cf160 0x7f721c0b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f721a59c640 1 -- 172.21.15.12:0/1638288750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7214009b30 con 0x7f721c0ca0e0 2023-10-30T12:35:11.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f721a59c640 1 --2- 172.21.15.12:0/1638288750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f721c0ca0e0 0x7f721c0b3630 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f720c00d830 tx=0x7f720c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:11.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f720affd640 1 -- 172.21.15.12:0/1638288750 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c00b6c0 con 0x7f721c0ca0e0 2023-10-30T12:35:11.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f720affd640 1 -- 172.21.15.12:0/1638288750 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c00b820 con 0x7f721c0ca0e0 2023-10-30T12:35:11.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f7220846640 1 -- 172.21.15.12:0/1638288750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721c1e3a70 con 0x7f721c0ca0e0 2023-10-30T12:35:11.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f720affd640 1 -- 172.21.15.12:0/1638288750 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c00bb90 con 0x7f721c0ca0e0 2023-10-30T12:35:11.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f7220846640 1 -- 172.21.15.12:0/1638288750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721c1e4030 con 0x7f721c0ca0e0 2023-10-30T12:35:11.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f7220846640 1 -- 172.21.15.12:0/1638288750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721c0b2a50 con 0x7f721c0ca0e0 2023-10-30T12:35:11.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f720affd640 1 -- 172.21.15.12:0/1638288750 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f720c01e460 con 0x7f721c0ca0e0 2023-10-30T12:35:11.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f720affd640 1 --2- 172.21.15.12:0/1638288750 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f71fc0664a0 0x7f71fc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:11.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.386+0000 7f720affd640 1 -- 172.21.15.12:0/1638288750 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f720c0846a0 con 0x7f721c0ca0e0 2023-10-30T12:35:11.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.390+0000 7f7219d9b640 1 --2- 172.21.15.12:0/1638288750 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f71fc0664a0 0x7f71fc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:11.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.390+0000 7f7219d9b640 1 --2- 172.21.15.12:0/1638288750 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f71fc0664a0 0x7f71fc068960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f721c146ce0 tx=0x7f7210008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:11.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.390+0000 7f720affd640 1 -- 172.21.15.12:0/1638288750 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f720c051a80 con 0x7f721c0ca0e0 2023-10-30T12:35:11.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.582+0000 7f7220846640 1 -- 172.21.15.12:0/1638288750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f721c1e4250 con 0x7f721c0ca0e0 2023-10-30T12:35:11.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.582+0000 7f720affd640 1 -- 172.21.15.12:0/1638288750 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6493 (secure 0 0 0) 0x7f720c0550a0 con 0x7f721c0ca0e0 2023-10-30T12:35:11.587 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:11.587 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:35:10.571618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:35:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2023-10-30T12:35:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.586+0000 7f7220846640 1 -- 172.21.15.12:0/1638288750 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f71fc0664a0 msgr2=0x7f71fc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.586+0000 7f7220846640 1 --2- 172.21.15.12:0/1638288750 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f71fc0664a0 0x7f71fc068960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f721c146ce0 tx=0x7f7210008040 comp rx=0 tx=0).stop 2023-10-30T12:35:11.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.586+0000 7f7220846640 1 -- 172.21.15.12:0/1638288750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f721c0ca0e0 msgr2=0x7f721c0b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:11.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.586+0000 7f7220846640 1 --2- 172.21.15.12:0/1638288750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f721c0ca0e0 0x7f721c0b3630 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f720c00d830 tx=0x7f720c00dcf0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.586+0000 7f7220846640 1 -- 172.21.15.12:0/1638288750 shutdown_connections 2023-10-30T12:35:11.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.586+0000 7f7220846640 1 --2- 172.21.15.12:0/1638288750 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f71fc0664a0 0x7f71fc068960 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.586+0000 7f7220846640 1 --2- 172.21.15.12:0/1638288750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f721c1538a0 0x7f721c1438b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.586+0000 7f7220846640 1 --2- 172.21.15.12:0/1638288750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f721c0cf160 0x7f721c0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.586+0000 7f7220846640 1 --2- 172.21.15.12:0/1638288750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f721c0ca0e0 0x7f721c0b3630 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.586+0000 7f7220846640 1 -- 172.21.15.12:0/1638288750 >> 172.21.15.12:0/1638288750 conn(0x7f721c06bfe0 msgr2=0x7f721c145d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:11.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.586+0000 7f7220846640 1 -- 172.21.15.12:0/1638288750 shutdown_connections 2023-10-30T12:35:11.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.586+0000 7f7220846640 1 -- 172.21.15.12:0/1638288750 wait complete. 2023-10-30T12:35:11.607 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:35:11.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.746+0000 7f79fc594640 1 Processor -- start 2023-10-30T12:35:11.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.746+0000 7f79fc594640 1 -- start start 2023-10-30T12:35:11.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.746+0000 7f79fc594640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f79f4070a00 0x7f79f4070e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:11.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.746+0000 7f79fc594640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f79f4069e90 0x7f79f406a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:11.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.746+0000 7f79fc594640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f40713b0 con 0x7f79f41539a0 2023-10-30T12:35:11.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.746+0000 7f79fc594640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f406fbb0 con 0x7f79f4070a00 2023-10-30T12:35:11.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.746+0000 7f79fc594640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f4153d80 con 0x7f79f4069e90 2023-10-30T12:35:11.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.746+0000 7f79f9b2f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f79f4070a00 0x7f79f4070e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:11.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.746+0000 7f79f9b2f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f79f4070a00 0x7f79f4070e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44154/0 (socket says 172.21.15.12:44154) 2023-10-30T12:35:11.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.746+0000 7f79f9b2f640 1 -- 172.21.15.12:0/1815634822 learned_addr learned my addr 172.21.15.12:0/1815634822 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:11.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.746+0000 7f79fab31640 1 --2- 172.21.15.12:0/1815634822 >> v2:172.21.15.154:3300/0 conn(0x7f79f4069e90 0x7f79f406a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:11.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.750+0000 7f79f9b2f640 1 -- 172.21.15.12:0/1815634822 >> v2:172.21.15.154:3300/0 conn(0x7f79f4069e90 msgr2=0x7f79f406a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:11.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.750+0000 7f79f9b2f640 1 --2- 172.21.15.12:0/1815634822 >> v2:172.21.15.154:3300/0 conn(0x7f79f4069e90 0x7f79f406a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.750+0000 7f79f9b2f640 1 -- 172.21.15.12:0/1815634822 >> v1:172.21.15.154:6789/0 conn(0x7f79f41539a0 legacy=0x7f79f415a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:11.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.750+0000 7f79f9b2f640 1 -- 172.21.15.12:0/1815634822 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f40c5c50 con 0x7f79f4070a00 2023-10-30T12:35:11.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.750+0000 7f79f9b2f640 1 --2- 172.21.15.12:0/1815634822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79f4070a00 0x7f79f4070e70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f79f40b1080 tx=0x7f79e401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d14a9997ac3f8f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:11.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.750+0000 7f79f932e640 1 -- 172.21.15.12:0/1815634822 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4004230 con 0x7f79f4070a00 2023-10-30T12:35:11.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.750+0000 7f79f932e640 1 -- 172.21.15.12:0/1815634822 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e40a7050 con 0x7f79f4070a00 2023-10-30T12:35:11.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.750+0000 7f79f932e640 1 -- 172.21.15.12:0/1815634822 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4003620 con 0x7f79f4070a00 2023-10-30T12:35:11.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.750+0000 7f79fc594640 1 -- 172.21.15.12:0/1815634822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79f4070a00 msgr2=0x7f79f4070e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.750+0000 7f79fc594640 1 --2- 172.21.15.12:0/1815634822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79f4070a00 0x7f79f4070e70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f79f40b1080 tx=0x7f79e401c440 comp rx=0 tx=0).stop 2023-10-30T12:35:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fc594640 1 -- 172.21.15.12:0/1815634822 shutdown_connections 2023-10-30T12:35:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fc594640 1 --2- 172.21.15.12:0/1815634822 >> v2:172.21.15.154:3300/0 conn(0x7f79f4069e90 0x7f79f406a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fc594640 1 --2- 172.21.15.12:0/1815634822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79f4070a00 0x7f79f4070e70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fc594640 1 -- 172.21.15.12:0/1815634822 >> 172.21.15.12:0/1815634822 conn(0x7f79f40c7910 msgr2=0x7f79f40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fc594640 1 -- 172.21.15.12:0/1815634822 shutdown_connections 2023-10-30T12:35:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fc594640 1 -- 172.21.15.12:0/1815634822 wait complete. 2023-10-30T12:35:11.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fc594640 1 Processor -- start 2023-10-30T12:35:11.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fc594640 1 -- start start 2023-10-30T12:35:11.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fc594640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79f4069e90 0x7f79f40c0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:11.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fc594640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79f4070a00 0x7f79f4145bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:11.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fc594640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79f41539a0 0x7f79f4146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:11.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fc594640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f41f6860 con 0x7f79f4070a00 2023-10-30T12:35:11.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fc594640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f41f5cb0 con 0x7f79f4069e90 2023-10-30T12:35:11.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fc594640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40c1ac0 con 0x7f79f41539a0 2023-10-30T12:35:11.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79f9b2f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79f4070a00 0x7f79f4145bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:11.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79f9b2f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79f4070a00 0x7f79f4145bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44166/0 (socket says 172.21.15.12:44166) 2023-10-30T12:35:11.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79f9b2f640 1 -- 172.21.15.12:0/2383231364 learned_addr learned my addr 172.21.15.12:0/2383231364 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:11.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fa330640 1 --2- 172.21.15.12:0/2383231364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79f4069e90 0x7f79f40c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:11.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79fab31640 1 --2- 172.21.15.12:0/2383231364 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79f41539a0 0x7f79f4146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:11.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79f9b2f640 1 -- 172.21.15.12:0/2383231364 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79f41539a0 msgr2=0x7f79f4146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:11.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79f9b2f640 1 --2- 172.21.15.12:0/2383231364 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79f41539a0 0x7f79f4146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79f9b2f640 1 -- 172.21.15.12:0/2383231364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79f4069e90 msgr2=0x7f79f40c0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:11.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79f9b2f640 1 --2- 172.21.15.12:0/2383231364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79f4069e90 0x7f79f40c0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79f9b2f640 1 -- 172.21.15.12:0/2383231364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e40094d0 con 0x7f79f4070a00 2023-10-30T12:35:11.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79f9b2f640 1 --2- 172.21.15.12:0/2383231364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79f4070a00 0x7f79f4145bc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79e4019050 tx=0x7f79e40191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.754+0000 7f79eaffd640 1 -- 172.21.15.12:0/2383231364 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4003c70 con 0x7f79f4070a00 2023-10-30T12:35:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.758+0000 7f79eaffd640 1 -- 172.21.15.12:0/2383231364 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e4003df0 con 0x7f79f4070a00 2023-10-30T12:35:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.758+0000 7f79eaffd640 1 -- 172.21.15.12:0/2383231364 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e40ab680 con 0x7f79f4070a00 2023-10-30T12:35:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.758+0000 7f79fc594640 1 -- 172.21.15.12:0/2383231364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f40c9f40 con 0x7f79f4070a00 2023-10-30T12:35:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.758+0000 7f79fc594640 1 -- 172.21.15.12:0/2383231364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f41f61a0 con 0x7f79f4070a00 2023-10-30T12:35:11.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.758+0000 7f79fc594640 1 -- 172.21.15.12:0/2383231364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f40c52f0 con 0x7f79f4070a00 2023-10-30T12:35:11.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.758+0000 7f79eaffd640 1 -- 172.21.15.12:0/2383231364 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f79e40b2040 con 0x7f79f4070a00 2023-10-30T12:35:11.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.758+0000 7f79eaffd640 1 --2- 172.21.15.12:0/2383231364 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f79d80664f0 0x7f79d80689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:11.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.758+0000 7f79eaffd640 1 -- 172.21.15.12:0/2383231364 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79e411e160 con 0x7f79f4070a00 2023-10-30T12:35:11.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.758+0000 7f79fa330640 1 --2- 172.21.15.12:0/2383231364 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f79d80664f0 0x7f79d80689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:11.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.762+0000 7f79fa330640 1 --2- 172.21.15.12:0/2383231364 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f79d80664f0 0x7f79d80689b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f79f0000f70 tx=0x7f79f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:11.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.762+0000 7f79eaffd640 1 -- 172.21.15.12:0/2383231364 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f79e40eb6c0 con 0x7f79f4070a00 2023-10-30T12:35:11.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.918+0000 7f79fc594640 1 -- 172.21.15.12:0/2383231364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79f41f69a0 con 0x7f79f4070a00 2023-10-30T12:35:11.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.922+0000 7f79eaffd640 1 -- 172.21.15.12:0/2383231364 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f79e40eece0 con 0x7f79f4070a00 2023-10-30T12:35:11.925 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:11.926 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":22,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:35:07.153665+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:35:03.216053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:35:04.597162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:35:11.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.922+0000 7f79fc594640 1 -- 172.21.15.12:0/2383231364 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f79d80664f0 msgr2=0x7f79d80689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:11.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.922+0000 7f79fc594640 1 --2- 172.21.15.12:0/2383231364 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f79d80664f0 0x7f79d80689b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f79f0000f70 tx=0x7f79f0009040 comp rx=0 tx=0).stop 2023-10-30T12:35:11.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.922+0000 7f79fc594640 1 -- 172.21.15.12:0/2383231364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79f4070a00 msgr2=0x7f79f4145bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:11.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.922+0000 7f79fc594640 1 --2- 172.21.15.12:0/2383231364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79f4070a00 0x7f79f4145bc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79e4019050 tx=0x7f79e40191e0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.922+0000 7f79fc594640 1 -- 172.21.15.12:0/2383231364 shutdown_connections 2023-10-30T12:35:11.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.922+0000 7f79fc594640 1 --2- 172.21.15.12:0/2383231364 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f79d80664f0 0x7f79d80689b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.922+0000 7f79fc594640 1 --2- 172.21.15.12:0/2383231364 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79f41539a0 0x7f79f4146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.922+0000 7f79fc594640 1 --2- 172.21.15.12:0/2383231364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79f4070a00 0x7f79f4145bc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.922+0000 7f79fc594640 1 --2- 172.21.15.12:0/2383231364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79f4069e90 0x7f79f40c0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.922+0000 7f79fc594640 1 -- 172.21.15.12:0/2383231364 >> 172.21.15.12:0/2383231364 conn(0x7f79f40c7910 msgr2=0x7f79f4152a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.922+0000 7f79fc594640 1 -- 172.21.15.12:0/2383231364 shutdown_connections 2023-10-30T12:35:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:11.922+0000 7f79fc594640 1 -- 172.21.15.12:0/2383231364 wait complete. 2023-10-30T12:35:11.947 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-30T12:35:11.948 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:35:12.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.086+0000 7f26ef33e640 1 Processor -- start 2023-10-30T12:35:12.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26ef33e640 1 -- start start 2023-10-30T12:35:12.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26ef33e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f26e80bfec0 0x7f26e80c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26ef33e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26e806a720 0x7f26e806abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26ef33e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e8071290 con 0x7f26e8159e60 2023-10-30T12:35:12.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26ef33e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e8071570 con 0x7f26e80bfec0 2023-10-30T12:35:12.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26ef33e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e8071850 con 0x7f26e806a720 2023-10-30T12:35:12.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26ed0da640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f26e8159e60 0x7f26e806f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:55324/0 (socket says 172.21.15.12:55324) 2023-10-30T12:35:12.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26ed0da640 1 -- 172.21.15.12:0/3236683821 learned_addr learned my addr 172.21.15.12:0/3236683821 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:12.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26ec8d9640 1 --2- 172.21.15.12:0/3236683821 >> v2:172.21.15.154:3300/0 conn(0x7f26e80bfec0 0x7f26e80c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26ed8db640 1 --2- 172.21.15.12:0/3236683821 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26e806a720 0x7f26e806abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26dffff640 1 -- 172.21.15.12:0/3236683821 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449934578 0 0) 0x7f26e8071290 con 0x7f26e8159e60 2023-10-30T12:35:12.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26dffff640 1 -- 172.21.15.12:0/3236683821 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26d4003530 con 0x7f26e8159e60 2023-10-30T12:35:12.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26ec8d9640 1 -- 172.21.15.12:0/3236683821 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26e806a720 msgr2=0x7f26e806abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:12.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26ec8d9640 1 --2- 172.21.15.12:0/3236683821 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26e806a720 0x7f26e806abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26ec8d9640 1 -- 172.21.15.12:0/3236683821 >> v1:172.21.15.12:6789/0 conn(0x7f26e8159e60 legacy=0x7f26e806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26dffff640 1 -- 172.21.15.12:0/3236683821 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2875739329 0 0) 0x7f26d4003530 con 0x7f26e8159e60 2023-10-30T12:35:12.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.090+0000 7f26ec8d9640 1 -- 172.21.15.12:0/3236683821 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e8071b30 con 0x7f26e80bfec0 2023-10-30T12:35:12.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.094+0000 7f26ec8d9640 1 --2- 172.21.15.12:0/3236683821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e80bfec0 0x7f26e80c1a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f26d80097c0 tx=0x7f26d801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=beb395a3857efe36 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:12.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.094+0000 7f26dffff640 1 -- 172.21.15.12:0/3236683821 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d80a8970 con 0x7f26e80bfec0 2023-10-30T12:35:12.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.094+0000 7f26dffff640 1 -- 172.21.15.12:0/3236683821 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d80a8ad0 con 0x7f26e80bfec0 2023-10-30T12:35:12.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.094+0000 7f26dffff640 1 -- 172.21.15.12:0/3236683821 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d80a8e40 con 0x7f26e80bfec0 2023-10-30T12:35:12.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.094+0000 7f26ef33e640 1 -- 172.21.15.12:0/3236683821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e80bfec0 msgr2=0x7f26e80c1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.094+0000 7f26ef33e640 1 --2- 172.21.15.12:0/3236683821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e80bfec0 0x7f26e80c1a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f26d80097c0 tx=0x7f26d801ec70 comp rx=0 tx=0).stop 2023-10-30T12:35:12.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.094+0000 7f26ef33e640 1 -- 172.21.15.12:0/3236683821 shutdown_connections 2023-10-30T12:35:12.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.094+0000 7f26ef33e640 1 --2- 172.21.15.12:0/3236683821 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26e806a720 0x7f26e806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.094+0000 7f26ef33e640 1 --2- 172.21.15.12:0/3236683821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e80bfec0 0x7f26e80c1a40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.094+0000 7f26ef33e640 1 -- 172.21.15.12:0/3236683821 >> 172.21.15.12:0/3236683821 conn(0x7f26e80c7910 msgr2=0x7f26e80c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:12.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.094+0000 7f26ef33e640 1 -- 172.21.15.12:0/3236683821 shutdown_connections 2023-10-30T12:35:12.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.094+0000 7f26ef33e640 1 -- 172.21.15.12:0/3236683821 wait complete. 2023-10-30T12:35:12.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.094+0000 7f26ef33e640 1 Processor -- start 2023-10-30T12:35:12.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ef33e640 1 -- start start 2023-10-30T12:35:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ef33e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26e806a720 0x7f26e81ee9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ef33e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e80bfec0 0x7f26e8151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ef33e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26e8159e60 0x7f26e8152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ef33e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80c52a0 con 0x7f26e806a720 2023-10-30T12:35:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ef33e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e814e850 con 0x7f26e80bfec0 2023-10-30T12:35:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ef33e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e814ea30 con 0x7f26e8159e60 2023-10-30T12:35:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ed0da640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26e806a720 0x7f26e81ee9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ed0da640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26e806a720 0x7f26e81ee9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44182/0 (socket says 172.21.15.12:44182) 2023-10-30T12:35:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ed0da640 1 -- 172.21.15.12:0/1202086358 learned_addr learned my addr 172.21.15.12:0/1202086358 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ed8db640 1 --2- 172.21.15.12:0/1202086358 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26e8159e60 0x7f26e8152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ec8d9640 1 --2- 172.21.15.12:0/1202086358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e80bfec0 0x7f26e8151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ed0da640 1 -- 172.21.15.12:0/1202086358 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26e8159e60 msgr2=0x7f26e8152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ed0da640 1 --2- 172.21.15.12:0/1202086358 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26e8159e60 0x7f26e8152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ed0da640 1 -- 172.21.15.12:0/1202086358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e80bfec0 msgr2=0x7f26e8151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ed0da640 1 --2- 172.21.15.12:0/1202086358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e80bfec0 0x7f26e8151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ed0da640 1 -- 172.21.15.12:0/1202086358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d8009420 con 0x7f26e806a720 2023-10-30T12:35:12.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ed0da640 1 --2- 172.21.15.12:0/1202086358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26e806a720 0x7f26e81ee9f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f26e400c410 tx=0x7f26e400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:12.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ddffb640 1 -- 172.21.15.12:0/1202086358 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4005cf0 con 0x7f26e806a720 2023-10-30T12:35:12.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ddffb640 1 -- 172.21.15.12:0/1202086358 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e4005e70 con 0x7f26e806a720 2023-10-30T12:35:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ddffb640 1 -- 172.21.15.12:0/1202086358 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e40072d0 con 0x7f26e806a720 2023-10-30T12:35:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ef33e640 1 -- 172.21.15.12:0/1202086358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e814ecd0 con 0x7f26e806a720 2023-10-30T12:35:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ef33e640 1 -- 172.21.15.12:0/1202086358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e814f290 con 0x7f26e806a720 2023-10-30T12:35:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.098+0000 7f26ef33e640 1 -- 172.21.15.12:0/1202086358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e804e170 con 0x7f26e806a720 2023-10-30T12:35:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.102+0000 7f26ddffb640 1 -- 172.21.15.12:0/1202086358 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f26e4007430 con 0x7f26e806a720 2023-10-30T12:35:12.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.102+0000 7f26ddffb640 1 --2- 172.21.15.12:0/1202086358 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f26cc0664a0 0x7f26cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.102+0000 7f26ddffb640 1 -- 172.21.15.12:0/1202086358 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26e4083610 con 0x7f26e806a720 2023-10-30T12:35:12.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.102+0000 7f26ec8d9640 1 --2- 172.21.15.12:0/1202086358 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f26cc0664a0 0x7f26cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.102+0000 7f26ec8d9640 1 --2- 172.21.15.12:0/1202086358 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f26cc0664a0 0x7f26cc068960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f26d80b2000 tx=0x7f26d80aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:12.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.106+0000 7f26ddffb640 1 -- 172.21.15.12:0/1202086358 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f26e4050970 con 0x7f26e806a720 2023-10-30T12:35:12.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.258+0000 7f26ef33e640 1 -- 172.21.15.12:0/1202086358 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26e8155e40 con 0x7f26cc0664a0 2023-10-30T12:35:12.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.282+0000 7f26ddffb640 1 -- 172.21.15.12:0/1202086358 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264564 (secure 0 0 0) 0x7f26e8155e40 con 0x7f26cc0664a0 2023-10-30T12:35:12.285 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:12.288 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-10-30T12:35:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.286+0000 7f26ef33e640 1 -- 172.21.15.12:0/1202086358 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f26cc0664a0 msgr2=0x7f26cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.286+0000 7f26ef33e640 1 --2- 172.21.15.12:0/1202086358 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f26cc0664a0 0x7f26cc068960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f26d80b2000 tx=0x7f26d80aa070 comp rx=0 tx=0).stop 2023-10-30T12:35:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.286+0000 7f26ef33e640 1 -- 172.21.15.12:0/1202086358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26e806a720 msgr2=0x7f26e81ee9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.286+0000 7f26ef33e640 1 --2- 172.21.15.12:0/1202086358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26e806a720 0x7f26e81ee9f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f26e400c410 tx=0x7f26e400c8d0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.286+0000 7f26ef33e640 1 -- 172.21.15.12:0/1202086358 shutdown_connections 2023-10-30T12:35:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.286+0000 7f26ef33e640 1 --2- 172.21.15.12:0/1202086358 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f26cc0664a0 0x7f26cc068960 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.286+0000 7f26ef33e640 1 --2- 172.21.15.12:0/1202086358 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f26e8159e60 0x7f26e8152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.286+0000 7f26ef33e640 1 --2- 172.21.15.12:0/1202086358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e80bfec0 0x7f26e8151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.286+0000 7f26ef33e640 1 --2- 172.21.15.12:0/1202086358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26e806a720 0x7f26e81ee9f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.286+0000 7f26ef33e640 1 -- 172.21.15.12:0/1202086358 >> 172.21.15.12:0/1202086358 conn(0x7f26e80c7910 msgr2=0x7f26e80c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.286+0000 7f26ef33e640 1 -- 172.21.15.12:0/1202086358 shutdown_connections 2023-10-30T12:35:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.286+0000 7f26ef33e640 1 -- 172.21.15.12:0/1202086358 wait complete. 2023-10-30T12:35:12.307 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-30T12:35:10.936456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219024,"kb_used_data":2836,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755696,"statfs":{"total":773094113280,"available":772869832704,"internally_reserved":0,"allocated":2904064,"data_stored":1767073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12729,"internal_metadata":221236807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002157"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250527+0000","last_change":"2023-10-30T12:35:06.223711+0000","last_active":"2023-10-30T12:35:07.250527+0000","last_peered":"2023-10-30T12:35:07.250527+0000","last_clean":"2023-10-30T12:35:07.250527+0000","last_became_active":"2023-10-30T12:35:06.222724+0000","last_became_peered":"2023-10-30T12:35:06.222724+0000","last_unstale":"2023-10-30T12:35:07.250527+0000","last_undegraded":"2023-10-30T12:35:07.250527+0000","last_fullsized":"2023-10-30T12:35:07.250527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:34:35.217274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.187239+0000","last_change":"2023-10-30T12:35:06.182748+0000","last_active":"2023-10-30T12:35:06.187239+0000","last_peered":"2023-10-30T12:35:06.187239+0000","last_clean":"2023-10-30T12:35:06.187239+0000","last_became_active":"2023-10-30T12:35:06.182441+0000","last_became_peered":"2023-10-30T12:35:06.182441+0000","last_unstale":"2023-10-30T12:35:06.187239+0000","last_undegraded":"2023-10-30T12:35:06.187239+0000","last_fullsized":"2023-10-30T12:35:06.187239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:04:30.261815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160588+0000","last_change":"2023-10-30T12:35:06.180426+0000","last_active":"2023-10-30T12:35:07.160588+0000","last_peered":"2023-10-30T12:35:07.160588+0000","last_clean":"2023-10-30T12:35:07.160588+0000","last_became_active":"2023-10-30T12:35:06.180132+0000","last_became_peered":"2023-10-30T12:35:06.180132+0000","last_unstale":"2023-10-30T12:35:07.160588+0000","last_undegraded":"2023-10-30T12:35:07.160588+0000","last_fullsized":"2023-10-30T12:35:07.160588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:33:22.216739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.228228+0000","last_change":"2023-10-30T12:35:06.217986+0000","last_active":"2023-10-30T12:35:06.228228+0000","last_peered":"2023-10-30T12:35:06.228228+0000","last_clean":"2023-10-30T12:35:06.228228+0000","last_became_active":"2023-10-30T12:35:06.213987+0000","last_became_peered":"2023-10-30T12:35:06.213987+0000","last_unstale":"2023-10-30T12:35:06.228228+0000","last_undegraded":"2023-10-30T12:35:06.228228+0000","last_fullsized":"2023-10-30T12:35:06.228228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:37:57.869908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249406+0000","last_change":"2023-10-30T12:35:06.216098+0000","last_active":"2023-10-30T12:35:07.249406+0000","last_peered":"2023-10-30T12:35:07.249406+0000","last_clean":"2023-10-30T12:35:07.249406+0000","last_became_active":"2023-10-30T12:35:06.214964+0000","last_became_peered":"2023-10-30T12:35:06.214964+0000","last_unstale":"2023-10-30T12:35:07.249406+0000","last_undegraded":"2023-10-30T12:35:07.249406+0000","last_fullsized":"2023-10-30T12:35:07.249406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:37:43.156964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.228169+0000","last_change":"2023-10-30T12:35:06.217608+0000","last_active":"2023-10-30T12:35:06.228169+0000","last_peered":"2023-10-30T12:35:06.228169+0000","last_clean":"2023-10-30T12:35:06.228169+0000","last_became_active":"2023-10-30T12:35:06.214219+0000","last_became_peered":"2023-10-30T12:35:06.214219+0000","last_unstale":"2023-10-30T12:35:06.228169+0000","last_undegraded":"2023-10-30T12:35:06.228169+0000","last_fullsized":"2023-10-30T12:35:06.228169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:37:19.767075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249000+0000","last_change":"2023-10-30T12:35:06.217120+0000","last_active":"2023-10-30T12:35:07.249000+0000","last_peered":"2023-10-30T12:35:07.249000+0000","last_clean":"2023-10-30T12:35:07.249000+0000","last_became_active":"2023-10-30T12:35:06.216468+0000","last_became_peered":"2023-10-30T12:35:06.216468+0000","last_unstale":"2023-10-30T12:35:07.249000+0000","last_undegraded":"2023-10-30T12:35:07.249000+0000","last_fullsized":"2023-10-30T12:35:07.249000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:26:07.313967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246707+0000","last_change":"2023-10-30T12:35:06.214767+0000","last_active":"2023-10-30T12:35:07.246707+0000","last_peered":"2023-10-30T12:35:07.246707+0000","last_clean":"2023-10-30T12:35:07.246707+0000","last_became_active":"2023-10-30T12:35:06.214384+0000","last_became_peered":"2023-10-30T12:35:06.214384+0000","last_unstale":"2023-10-30T12:35:07.246707+0000","last_undegraded":"2023-10-30T12:35:07.246707+0000","last_fullsized":"2023-10-30T12:35:07.246707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:29.866538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250843+0000","last_change":"2023-10-30T12:35:06.223503+0000","last_active":"2023-10-30T12:35:07.250843+0000","last_peered":"2023-10-30T12:35:07.250843+0000","last_clean":"2023-10-30T12:35:07.250843+0000","last_became_active":"2023-10-30T12:35:06.222692+0000","last_became_peered":"2023-10-30T12:35:06.222692+0000","last_unstale":"2023-10-30T12:35:07.250843+0000","last_undegraded":"2023-10-30T12:35:07.250843+0000","last_fullsized":"2023-10-30T12:35:07.250843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:31:36.926720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163304+0000","last_change":"2023-10-30T12:35:06.215938+0000","last_active":"2023-10-30T12:35:07.163304+0000","last_peered":"2023-10-30T12:35:07.163304+0000","last_clean":"2023-10-30T12:35:07.163304+0000","last_became_active":"2023-10-30T12:35:06.215384+0000","last_became_peered":"2023-10-30T12:35:06.215384+0000","last_unstale":"2023-10-30T12:35:07.163304+0000","last_undegraded":"2023-10-30T12:35:07.163304+0000","last_fullsized":"2023-10-30T12:35:07.163304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:46:21.253555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162425+0000","last_change":"2023-10-30T12:35:06.215965+0000","last_active":"2023-10-30T12:35:07.162425+0000","last_peered":"2023-10-30T12:35:07.162425+0000","last_clean":"2023-10-30T12:35:07.162425+0000","last_became_active":"2023-10-30T12:35:06.215355+0000","last_became_peered":"2023-10-30T12:35:06.215355+0000","last_unstale":"2023-10-30T12:35:07.162425+0000","last_undegraded":"2023-10-30T12:35:07.162425+0000","last_fullsized":"2023-10-30T12:35:07.162425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:07.929526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164185+0000","last_change":"2023-10-30T12:35:06.197278+0000","last_active":"2023-10-30T12:35:07.164185+0000","last_peered":"2023-10-30T12:35:07.164185+0000","last_clean":"2023-10-30T12:35:07.164185+0000","last_became_active":"2023-10-30T12:35:06.196503+0000","last_became_peered":"2023-10-30T12:35:06.196503+0000","last_unstale":"2023-10-30T12:35:07.164185+0000","last_undegraded":"2023-10-30T12:35:07.164185+0000","last_fullsized":"2023-10-30T12:35:07.164185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:06.928264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247199+0000","last_change":"2023-10-30T12:35:06.195721+0000","last_active":"2023-10-30T12:35:07.247199+0000","last_peered":"2023-10-30T12:35:07.247199+0000","last_clean":"2023-10-30T12:35:07.247199+0000","last_became_active":"2023-10-30T12:35:06.195408+0000","last_became_peered":"2023-10-30T12:35:06.195408+0000","last_unstale":"2023-10-30T12:35:07.247199+0000","last_undegraded":"2023-10-30T12:35:07.247199+0000","last_fullsized":"2023-10-30T12:35:07.247199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:59:02.872646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160258+0000","last_change":"2023-10-30T12:35:06.182939+0000","last_active":"2023-10-30T12:35:07.160258+0000","last_peered":"2023-10-30T12:35:07.160258+0000","last_clean":"2023-10-30T12:35:07.160258+0000","last_became_active":"2023-10-30T12:35:06.182573+0000","last_became_peered":"2023-10-30T12:35:06.182573+0000","last_unstale":"2023-10-30T12:35:07.160258+0000","last_undegraded":"2023-10-30T12:35:07.160258+0000","last_fullsized":"2023-10-30T12:35:07.160258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:50.345142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.190140+0000","last_change":"2023-10-30T12:35:06.185636+0000","last_active":"2023-10-30T12:35:06.190140+0000","last_peered":"2023-10-30T12:35:06.190140+0000","last_clean":"2023-10-30T12:35:06.190140+0000","last_became_active":"2023-10-30T12:35:06.185290+0000","last_became_peered":"2023-10-30T12:35:06.185290+0000","last_unstale":"2023-10-30T12:35:06.190140+0000","last_undegraded":"2023-10-30T12:35:06.190140+0000","last_fullsized":"2023-10-30T12:35:06.190140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:31:12.579628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.229715+0000","last_change":"2023-10-30T12:35:06.214488+0000","last_active":"2023-10-30T12:35:06.229715+0000","last_peered":"2023-10-30T12:35:06.229715+0000","last_clean":"2023-10-30T12:35:06.229715+0000","last_became_active":"2023-10-30T12:35:06.213627+0000","last_became_peered":"2023-10-30T12:35:06.213627+0000","last_unstale":"2023-10-30T12:35:06.229715+0000","last_undegraded":"2023-10-30T12:35:06.229715+0000","last_fullsized":"2023-10-30T12:35:06.229715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:51:08.645452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.229700+0000","last_change":"2023-10-30T12:35:06.224903+0000","last_active":"2023-10-30T12:35:06.229700+0000","last_peered":"2023-10-30T12:35:06.229700+0000","last_clean":"2023-10-30T12:35:06.229700+0000","last_became_active":"2023-10-30T12:35:06.221696+0000","last_became_peered":"2023-10-30T12:35:06.221696+0000","last_unstale":"2023-10-30T12:35:06.229700+0000","last_undegraded":"2023-10-30T12:35:06.229700+0000","last_fullsized":"2023-10-30T12:35:06.229700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:25:12.393720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246945+0000","last_change":"2023-10-30T12:35:06.198303+0000","last_active":"2023-10-30T12:35:07.246945+0000","last_peered":"2023-10-30T12:35:07.246945+0000","last_clean":"2023-10-30T12:35:07.246945+0000","last_became_active":"2023-10-30T12:35:06.198074+0000","last_became_peered":"2023-10-30T12:35:06.198074+0000","last_unstale":"2023-10-30T12:35:07.246945+0000","last_undegraded":"2023-10-30T12:35:07.246945+0000","last_fullsized":"2023-10-30T12:35:07.246945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:21:36.079586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250503+0000","last_change":"2023-10-30T12:35:06.218556+0000","last_active":"2023-10-30T12:35:07.250503+0000","last_peered":"2023-10-30T12:35:07.250503+0000","last_clean":"2023-10-30T12:35:07.250503+0000","last_became_active":"2023-10-30T12:35:06.215935+0000","last_became_peered":"2023-10-30T12:35:06.215935+0000","last_unstale":"2023-10-30T12:35:07.250503+0000","last_undegraded":"2023-10-30T12:35:07.250503+0000","last_fullsized":"2023-10-30T12:35:07.250503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:36:31.084542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162438+0000","last_change":"2023-10-30T12:35:06.201302+0000","last_active":"2023-10-30T12:35:07.162438+0000","last_peered":"2023-10-30T12:35:07.162438+0000","last_clean":"2023-10-30T12:35:07.162438+0000","last_became_active":"2023-10-30T12:35:06.199812+0000","last_became_peered":"2023-10-30T12:35:06.199812+0000","last_unstale":"2023-10-30T12:35:07.162438+0000","last_undegraded":"2023-10-30T12:35:07.162438+0000","last_fullsized":"2023-10-30T12:35:07.162438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:48.803738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160558+0000","last_change":"2023-10-30T12:35:06.217139+0000","last_active":"2023-10-30T12:35:07.160558+0000","last_peered":"2023-10-30T12:35:07.160558+0000","last_clean":"2023-10-30T12:35:07.160558+0000","last_became_active":"2023-10-30T12:35:06.216861+0000","last_became_peered":"2023-10-30T12:35:06.216861+0000","last_unstale":"2023-10-30T12:35:07.160558+0000","last_undegraded":"2023-10-30T12:35:07.160558+0000","last_fullsized":"2023-10-30T12:35:07.160558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:05:22.193180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.229733+0000","last_change":"2023-10-30T12:35:06.225140+0000","last_active":"2023-10-30T12:35:06.229733+0000","last_peered":"2023-10-30T12:35:06.229733+0000","last_clean":"2023-10-30T12:35:06.229733+0000","last_became_active":"2023-10-30T12:35:06.221841+0000","last_became_peered":"2023-10-30T12:35:06.221841+0000","last_unstale":"2023-10-30T12:35:06.229733+0000","last_undegraded":"2023-10-30T12:35:06.229733+0000","last_fullsized":"2023-10-30T12:35:06.229733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:05.853879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247032+0000","last_change":"2023-10-30T12:35:06.182662+0000","last_active":"2023-10-30T12:35:07.247032+0000","last_peered":"2023-10-30T12:35:07.247032+0000","last_clean":"2023-10-30T12:35:07.247032+0000","last_became_active":"2023-10-30T12:35:06.182314+0000","last_became_peered":"2023-10-30T12:35:06.182314+0000","last_unstale":"2023-10-30T12:35:07.247032+0000","last_undegraded":"2023-10-30T12:35:07.247032+0000","last_fullsized":"2023-10-30T12:35:07.247032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:41:49.587856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.227997+0000","last_change":"2023-10-30T12:35:06.214113+0000","last_active":"2023-10-30T12:35:06.227997+0000","last_peered":"2023-10-30T12:35:06.227997+0000","last_clean":"2023-10-30T12:35:06.227997+0000","last_became_active":"2023-10-30T12:35:06.213402+0000","last_became_peered":"2023-10-30T12:35:06.213402+0000","last_unstale":"2023-10-30T12:35:06.227997+0000","last_undegraded":"2023-10-30T12:35:06.227997+0000","last_fullsized":"2023-10-30T12:35:06.227997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:52:48.311491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247781+0000","last_change":"2023-10-30T12:35:06.188570+0000","last_active":"2023-10-30T12:35:07.247781+0000","last_peered":"2023-10-30T12:35:07.247781+0000","last_clean":"2023-10-30T12:35:07.247781+0000","last_became_active":"2023-10-30T12:35:06.188298+0000","last_became_peered":"2023-10-30T12:35:06.188298+0000","last_unstale":"2023-10-30T12:35:07.247781+0000","last_undegraded":"2023-10-30T12:35:07.247781+0000","last_fullsized":"2023-10-30T12:35:07.247781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:28.281254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163623+0000","last_change":"2023-10-30T12:35:06.208821+0000","last_active":"2023-10-30T12:35:07.163623+0000","last_peered":"2023-10-30T12:35:07.163623+0000","last_clean":"2023-10-30T12:35:07.163623+0000","last_became_active":"2023-10-30T12:35:06.202507+0000","last_became_peered":"2023-10-30T12:35:06.202507+0000","last_unstale":"2023-10-30T12:35:07.163623+0000","last_undegraded":"2023-10-30T12:35:07.163623+0000","last_fullsized":"2023-10-30T12:35:07.163623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:40:13.124316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.175568+0000","last_change":"2023-10-30T12:35:06.170313+0000","last_active":"2023-10-30T12:35:06.175568+0000","last_peered":"2023-10-30T12:35:06.175568+0000","last_clean":"2023-10-30T12:35:06.175568+0000","last_became_active":"2023-10-30T12:35:06.170030+0000","last_became_peered":"2023-10-30T12:35:06.170030+0000","last_unstale":"2023-10-30T12:35:06.175568+0000","last_undegraded":"2023-10-30T12:35:06.175568+0000","last_fullsized":"2023-10-30T12:35:06.175568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:03:31.913507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247487+0000","last_change":"2023-10-30T12:35:06.189448+0000","last_active":"2023-10-30T12:35:07.247487+0000","last_peered":"2023-10-30T12:35:07.247487+0000","last_clean":"2023-10-30T12:35:07.247487+0000","last_became_active":"2023-10-30T12:35:06.189171+0000","last_became_peered":"2023-10-30T12:35:06.189171+0000","last_unstale":"2023-10-30T12:35:07.247487+0000","last_undegraded":"2023-10-30T12:35:07.247487+0000","last_fullsized":"2023-10-30T12:35:07.247487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:18:51.463273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160658+0000","last_change":"2023-10-30T12:35:06.183680+0000","last_active":"2023-10-30T12:35:07.160658+0000","last_peered":"2023-10-30T12:35:07.160658+0000","last_clean":"2023-10-30T12:35:07.160658+0000","last_became_active":"2023-10-30T12:35:06.183348+0000","last_became_peered":"2023-10-30T12:35:06.183348+0000","last_unstale":"2023-10-30T12:35:07.160658+0000","last_undegraded":"2023-10-30T12:35:07.160658+0000","last_fullsized":"2023-10-30T12:35:07.160658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:54:25.429897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.220759+0000","last_change":"2023-10-30T12:35:06.206675+0000","last_active":"2023-10-30T12:35:06.220759+0000","last_peered":"2023-10-30T12:35:06.220759+0000","last_clean":"2023-10-30T12:35:06.220759+0000","last_became_active":"2023-10-30T12:35:06.196368+0000","last_became_peered":"2023-10-30T12:35:06.196368+0000","last_unstale":"2023-10-30T12:35:06.220759+0000","last_undegraded":"2023-10-30T12:35:06.220759+0000","last_fullsized":"2023-10-30T12:35:06.220759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:11:16.050410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246681+0000","last_change":"2023-10-30T12:35:06.214408+0000","last_active":"2023-10-30T12:35:07.246681+0000","last_peered":"2023-10-30T12:35:07.246681+0000","last_clean":"2023-10-30T12:35:07.246681+0000","last_became_active":"2023-10-30T12:35:06.213994+0000","last_became_peered":"2023-10-30T12:35:06.213994+0000","last_unstale":"2023-10-30T12:35:07.246681+0000","last_undegraded":"2023-10-30T12:35:07.246681+0000","last_fullsized":"2023-10-30T12:35:07.246681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:05:58.830698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162027+0000","last_change":"2023-10-30T12:35:06.198444+0000","last_active":"2023-10-30T12:35:07.162027+0000","last_peered":"2023-10-30T12:35:07.162027+0000","last_clean":"2023-10-30T12:35:07.162027+0000","last_became_active":"2023-10-30T12:35:06.197647+0000","last_became_peered":"2023-10-30T12:35:06.197647+0000","last_unstale":"2023-10-30T12:35:07.162027+0000","last_undegraded":"2023-10-30T12:35:07.162027+0000","last_fullsized":"2023-10-30T12:35:07.162027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:28:56.075420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163077+0000","last_change":"2023-10-30T12:35:06.168377+0000","last_active":"2023-10-30T12:35:07.163077+0000","last_peered":"2023-10-30T12:35:07.163077+0000","last_clean":"2023-10-30T12:35:07.163077+0000","last_became_active":"2023-10-30T12:35:06.167558+0000","last_became_peered":"2023-10-30T12:35:06.167558+0000","last_unstale":"2023-10-30T12:35:07.163077+0000","last_undegraded":"2023-10-30T12:35:07.163077+0000","last_fullsized":"2023-10-30T12:35:07.163077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:57:28.009164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247303+0000","last_change":"2023-10-30T12:35:05.200844+0000","last_active":"2023-10-30T12:35:07.247303+0000","last_peered":"2023-10-30T12:35:07.247303+0000","last_clean":"2023-10-30T12:35:07.247303+0000","last_became_active":"2023-10-30T12:35:05.200169+0000","last_became_peered":"2023-10-30T12:35:05.200169+0000","last_unstale":"2023-10-30T12:35:07.247303+0000","last_undegraded":"2023-10-30T12:35:07.247303+0000","last_fullsized":"2023-10-30T12:35:07.247303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:07:19.571357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160718+0000","last_change":"2023-10-30T12:35:06.194964+0000","last_active":"2023-10-30T12:35:07.160718+0000","last_peered":"2023-10-30T12:35:07.160718+0000","last_clean":"2023-10-30T12:35:07.160718+0000","last_became_active":"2023-10-30T12:35:06.194689+0000","last_became_peered":"2023-10-30T12:35:06.194689+0000","last_unstale":"2023-10-30T12:35:07.160718+0000","last_undegraded":"2023-10-30T12:35:07.160718+0000","last_fullsized":"2023-10-30T12:35:07.160718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:30:53.410341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162742+0000","last_change":"2023-10-30T12:35:05.183569+0000","last_active":"2023-10-30T12:35:07.162742+0000","last_peered":"2023-10-30T12:35:07.162742+0000","last_clean":"2023-10-30T12:35:07.162742+0000","last_became_active":"2023-10-30T12:35:05.182761+0000","last_became_peered":"2023-10-30T12:35:05.182761+0000","last_unstale":"2023-10-30T12:35:07.162742+0000","last_undegraded":"2023-10-30T12:35:07.162742+0000","last_fullsized":"2023-10-30T12:35:07.162742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:49:44.311814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247108+0000","last_change":"2023-10-30T12:35:05.181278+0000","last_active":"2023-10-30T12:35:07.247108+0000","last_peered":"2023-10-30T12:35:07.247108+0000","last_clean":"2023-10-30T12:35:07.247108+0000","last_became_active":"2023-10-30T12:35:05.180730+0000","last_became_peered":"2023-10-30T12:35:05.180730+0000","last_unstale":"2023-10-30T12:35:07.247108+0000","last_undegraded":"2023-10-30T12:35:07.247108+0000","last_fullsized":"2023-10-30T12:35:07.247108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:07:10.828707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246964+0000","last_change":"2023-10-30T12:35:06.180049+0000","last_active":"2023-10-30T12:35:07.246964+0000","last_peered":"2023-10-30T12:35:07.246964+0000","last_clean":"2023-10-30T12:35:07.246964+0000","last_became_active":"2023-10-30T12:35:06.179764+0000","last_became_peered":"2023-10-30T12:35:06.179764+0000","last_unstale":"2023-10-30T12:35:07.246964+0000","last_undegraded":"2023-10-30T12:35:07.246964+0000","last_fullsized":"2023-10-30T12:35:07.246964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:33.136727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.193600+0000","last_change":"2023-10-30T12:35:05.202539+0000","last_active":"2023-10-30T12:35:06.193600+0000","last_peered":"2023-10-30T12:35:06.193600+0000","last_clean":"2023-10-30T12:35:06.193600+0000","last_became_active":"2023-10-30T12:35:05.201853+0000","last_became_peered":"2023-10-30T12:35:05.201853+0000","last_unstale":"2023-10-30T12:35:06.193600+0000","last_undegraded":"2023-10-30T12:35:06.193600+0000","last_fullsized":"2023-10-30T12:35:06.193600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:33:33.151433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.175406+0000","last_change":"2023-10-30T12:35:06.173567+0000","last_active":"2023-10-30T12:35:06.175406+0000","last_peered":"2023-10-30T12:35:06.175406+0000","last_clean":"2023-10-30T12:35:06.175406+0000","last_became_active":"2023-10-30T12:35:06.170179+0000","last_became_peered":"2023-10-30T12:35:06.170179+0000","last_unstale":"2023-10-30T12:35:06.175406+0000","last_undegraded":"2023-10-30T12:35:06.175406+0000","last_fullsized":"2023-10-30T12:35:06.175406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:02:23.116259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163553+0000","last_change":"2023-10-30T12:35:05.198546+0000","last_active":"2023-10-30T12:35:07.163553+0000","last_peered":"2023-10-30T12:35:07.163553+0000","last_clean":"2023-10-30T12:35:07.163553+0000","last_became_active":"2023-10-30T12:35:05.197606+0000","last_became_peered":"2023-10-30T12:35:05.197606+0000","last_unstale":"2023-10-30T12:35:07.163553+0000","last_undegraded":"2023-10-30T12:35:07.163553+0000","last_fullsized":"2023-10-30T12:35:07.163553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:26:57.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.229829+0000","last_change":"2023-10-30T12:35:06.226044+0000","last_active":"2023-10-30T12:35:06.229829+0000","last_peered":"2023-10-30T12:35:06.229829+0000","last_clean":"2023-10-30T12:35:06.229829+0000","last_became_active":"2023-10-30T12:35:06.225392+0000","last_became_peered":"2023-10-30T12:35:06.225392+0000","last_unstale":"2023-10-30T12:35:06.229829+0000","last_undegraded":"2023-10-30T12:35:06.229829+0000","last_fullsized":"2023-10-30T12:35:06.229829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:47:38.246937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.175959+0000","last_change":"2023-10-30T12:35:05.197614+0000","last_active":"2023-10-30T12:35:06.175959+0000","last_peered":"2023-10-30T12:35:06.175959+0000","last_clean":"2023-10-30T12:35:06.175959+0000","last_became_active":"2023-10-30T12:35:05.196899+0000","last_became_peered":"2023-10-30T12:35:05.196899+0000","last_unstale":"2023-10-30T12:35:06.175959+0000","last_undegraded":"2023-10-30T12:35:06.175959+0000","last_fullsized":"2023-10-30T12:35:06.175959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:21:00.152426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160715+0000","last_change":"2023-10-30T12:35:06.222344+0000","last_active":"2023-10-30T12:35:07.160715+0000","last_peered":"2023-10-30T12:35:07.160715+0000","last_clean":"2023-10-30T12:35:07.160715+0000","last_became_active":"2023-10-30T12:35:06.222057+0000","last_became_peered":"2023-10-30T12:35:06.222057+0000","last_unstale":"2023-10-30T12:35:07.160715+0000","last_undegraded":"2023-10-30T12:35:07.160715+0000","last_fullsized":"2023-10-30T12:35:07.160715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:50:46.691223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.161741+0000","last_change":"2023-10-30T12:35:05.197501+0000","last_active":"2023-10-30T12:35:06.161741+0000","last_peered":"2023-10-30T12:35:06.161741+0000","last_clean":"2023-10-30T12:35:06.161741+0000","last_became_active":"2023-10-30T12:35:05.197190+0000","last_became_peered":"2023-10-30T12:35:05.197190+0000","last_unstale":"2023-10-30T12:35:06.161741+0000","last_undegraded":"2023-10-30T12:35:06.161741+0000","last_fullsized":"2023-10-30T12:35:06.161741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:53:31.121721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164505+0000","last_change":"2023-10-30T12:35:06.204885+0000","last_active":"2023-10-30T12:35:07.164505+0000","last_peered":"2023-10-30T12:35:07.164505+0000","last_clean":"2023-10-30T12:35:07.164505+0000","last_became_active":"2023-10-30T12:35:06.204122+0000","last_became_peered":"2023-10-30T12:35:06.204122+0000","last_unstale":"2023-10-30T12:35:07.164505+0000","last_undegraded":"2023-10-30T12:35:07.164505+0000","last_fullsized":"2023-10-30T12:35:07.164505+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:27:18.433575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162559+0000","last_change":"2023-10-30T12:35:05.180005+0000","last_active":"2023-10-30T12:35:07.162559+0000","last_peered":"2023-10-30T12:35:07.162559+0000","last_clean":"2023-10-30T12:35:07.162559+0000","last_became_active":"2023-10-30T12:35:05.179547+0000","last_became_peered":"2023-10-30T12:35:05.179547+0000","last_unstale":"2023-10-30T12:35:07.162559+0000","last_undegraded":"2023-10-30T12:35:07.162559+0000","last_fullsized":"2023-10-30T12:35:07.162559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:13:03.137433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249565+0000","last_change":"2023-10-30T12:35:06.216619+0000","last_active":"2023-10-30T12:35:07.249565+0000","last_peered":"2023-10-30T12:35:07.249565+0000","last_clean":"2023-10-30T12:35:07.249565+0000","last_became_active":"2023-10-30T12:35:06.215128+0000","last_became_peered":"2023-10-30T12:35:06.215128+0000","last_unstale":"2023-10-30T12:35:07.249565+0000","last_undegraded":"2023-10-30T12:35:07.249565+0000","last_fullsized":"2023-10-30T12:35:07.249565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:48:45.673159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160709+0000","last_change":"2023-10-30T12:35:05.190041+0000","last_active":"2023-10-30T12:35:07.160709+0000","last_peered":"2023-10-30T12:35:07.160709+0000","last_clean":"2023-10-30T12:35:07.160709+0000","last_became_active":"2023-10-30T12:35:05.189573+0000","last_became_peered":"2023-10-30T12:35:05.189573+0000","last_unstale":"2023-10-30T12:35:07.160709+0000","last_undegraded":"2023-10-30T12:35:07.160709+0000","last_fullsized":"2023-10-30T12:35:07.160709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:02:57.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160412+0000","last_change":"2023-10-30T12:35:06.217334+0000","last_active":"2023-10-30T12:35:07.160412+0000","last_peered":"2023-10-30T12:35:07.160412+0000","last_clean":"2023-10-30T12:35:07.160412+0000","last_became_active":"2023-10-30T12:35:06.216885+0000","last_became_peered":"2023-10-30T12:35:06.216885+0000","last_unstale":"2023-10-30T12:35:07.160412+0000","last_undegraded":"2023-10-30T12:35:07.160412+0000","last_fullsized":"2023-10-30T12:35:07.160412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:38:26.745898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.194047+0000","last_change":"2023-10-30T12:35:05.203198+0000","last_active":"2023-10-30T12:35:06.194047+0000","last_peered":"2023-10-30T12:35:06.194047+0000","last_clean":"2023-10-30T12:35:06.194047+0000","last_became_active":"2023-10-30T12:35:05.202612+0000","last_became_peered":"2023-10-30T12:35:05.202612+0000","last_unstale":"2023-10-30T12:35:06.194047+0000","last_undegraded":"2023-10-30T12:35:06.194047+0000","last_fullsized":"2023-10-30T12:35:06.194047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:55.986236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249574+0000","last_change":"2023-10-30T12:35:06.216307+0000","last_active":"2023-10-30T12:35:07.249574+0000","last_peered":"2023-10-30T12:35:07.249574+0000","last_clean":"2023-10-30T12:35:07.249574+0000","last_became_active":"2023-10-30T12:35:06.215553+0000","last_became_peered":"2023-10-30T12:35:06.215553+0000","last_unstale":"2023-10-30T12:35:07.249574+0000","last_undegraded":"2023-10-30T12:35:07.249574+0000","last_fullsized":"2023-10-30T12:35:07.249574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:46:25.396759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.161910+0000","last_change":"2023-10-30T12:35:05.181882+0000","last_active":"2023-10-30T12:35:07.161910+0000","last_peered":"2023-10-30T12:35:07.161910+0000","last_clean":"2023-10-30T12:35:07.161910+0000","last_became_active":"2023-10-30T12:35:05.181115+0000","last_became_peered":"2023-10-30T12:35:05.181115+0000","last_unstale":"2023-10-30T12:35:07.161910+0000","last_undegraded":"2023-10-30T12:35:07.161910+0000","last_fullsized":"2023-10-30T12:35:07.161910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:18:55.496940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162013+0000","last_change":"2023-10-30T12:35:06.198990+0000","last_active":"2023-10-30T12:35:07.162013+0000","last_peered":"2023-10-30T12:35:07.162013+0000","last_clean":"2023-10-30T12:35:07.162013+0000","last_became_active":"2023-10-30T12:35:06.197722+0000","last_became_peered":"2023-10-30T12:35:06.197722+0000","last_unstale":"2023-10-30T12:35:07.162013+0000","last_undegraded":"2023-10-30T12:35:07.162013+0000","last_fullsized":"2023-10-30T12:35:07.162013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:56:51.757976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246641+0000","last_change":"2023-10-30T12:35:05.169455+0000","last_active":"2023-10-30T12:35:07.246641+0000","last_peered":"2023-10-30T12:35:07.246641+0000","last_clean":"2023-10-30T12:35:07.246641+0000","last_became_active":"2023-10-30T12:35:05.169130+0000","last_became_peered":"2023-10-30T12:35:05.169130+0000","last_unstale":"2023-10-30T12:35:07.246641+0000","last_undegraded":"2023-10-30T12:35:07.246641+0000","last_fullsized":"2023-10-30T12:35:07.246641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:59:14.096315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249183+0000","last_change":"2023-10-30T12:35:06.215545+0000","last_active":"2023-10-30T12:35:07.249183+0000","last_peered":"2023-10-30T12:35:07.249183+0000","last_clean":"2023-10-30T12:35:07.249183+0000","last_became_active":"2023-10-30T12:35:06.214839+0000","last_became_peered":"2023-10-30T12:35:06.214839+0000","last_unstale":"2023-10-30T12:35:07.249183+0000","last_undegraded":"2023-10-30T12:35:07.249183+0000","last_fullsized":"2023-10-30T12:35:07.249183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:54:34.206981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164455+0000","last_change":"2023-10-30T12:35:05.189361+0000","last_active":"2023-10-30T12:35:07.164455+0000","last_peered":"2023-10-30T12:35:07.164455+0000","last_clean":"2023-10-30T12:35:07.164455+0000","last_became_active":"2023-10-30T12:35:05.188862+0000","last_became_peered":"2023-10-30T12:35:05.188862+0000","last_unstale":"2023-10-30T12:35:07.164455+0000","last_undegraded":"2023-10-30T12:35:07.164455+0000","last_fullsized":"2023-10-30T12:35:07.164455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:13.319732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250712+0000","last_change":"2023-10-30T12:35:06.223118+0000","last_active":"2023-10-30T12:35:07.250712+0000","last_peered":"2023-10-30T12:35:07.250712+0000","last_clean":"2023-10-30T12:35:07.250712+0000","last_became_active":"2023-10-30T12:35:06.216533+0000","last_became_peered":"2023-10-30T12:35:06.216533+0000","last_unstale":"2023-10-30T12:35:07.250712+0000","last_undegraded":"2023-10-30T12:35:07.250712+0000","last_fullsized":"2023-10-30T12:35:07.250712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:32:29.374526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247032+0000","last_change":"2023-10-30T12:35:05.177609+0000","last_active":"2023-10-30T12:35:07.247032+0000","last_peered":"2023-10-30T12:35:07.247032+0000","last_clean":"2023-10-30T12:35:07.247032+0000","last_became_active":"2023-10-30T12:35:05.177316+0000","last_became_peered":"2023-10-30T12:35:05.177316+0000","last_unstale":"2023-10-30T12:35:07.247032+0000","last_undegraded":"2023-10-30T12:35:07.247032+0000","last_fullsized":"2023-10-30T12:35:07.247032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:46.820925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251056+0000","last_change":"2023-10-30T12:35:06.222283+0000","last_active":"2023-10-30T12:35:07.251056+0000","last_peered":"2023-10-30T12:35:07.251056+0000","last_clean":"2023-10-30T12:35:07.251056+0000","last_became_active":"2023-10-30T12:35:06.221701+0000","last_became_peered":"2023-10-30T12:35:06.221701+0000","last_unstale":"2023-10-30T12:35:07.251056+0000","last_undegraded":"2023-10-30T12:35:07.251056+0000","last_fullsized":"2023-10-30T12:35:07.251056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:37:32.956627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.161885+0000","last_change":"2023-10-30T12:35:05.171787+0000","last_active":"2023-10-30T12:35:06.161885+0000","last_peered":"2023-10-30T12:35:06.161885+0000","last_clean":"2023-10-30T12:35:06.161885+0000","last_became_active":"2023-10-30T12:35:05.171491+0000","last_became_peered":"2023-10-30T12:35:05.171491+0000","last_unstale":"2023-10-30T12:35:06.161885+0000","last_undegraded":"2023-10-30T12:35:06.161885+0000","last_fullsized":"2023-10-30T12:35:06.161885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:43:15.331175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.228023+0000","last_change":"2023-10-30T12:35:06.214464+0000","last_active":"2023-10-30T12:35:06.228023+0000","last_peered":"2023-10-30T12:35:06.228023+0000","last_clean":"2023-10-30T12:35:06.228023+0000","last_became_active":"2023-10-30T12:35:06.213493+0000","last_became_peered":"2023-10-30T12:35:06.213493+0000","last_unstale":"2023-10-30T12:35:06.228023+0000","last_undegraded":"2023-10-30T12:35:06.228023+0000","last_fullsized":"2023-10-30T12:35:06.228023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:39:43.751284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246924+0000","last_change":"2023-10-30T12:35:05.197811+0000","last_active":"2023-10-30T12:35:07.246924+0000","last_peered":"2023-10-30T12:35:07.246924+0000","last_clean":"2023-10-30T12:35:07.246924+0000","last_became_active":"2023-10-30T12:35:05.197453+0000","last_became_peered":"2023-10-30T12:35:05.197453+0000","last_unstale":"2023-10-30T12:35:07.246924+0000","last_undegraded":"2023-10-30T12:35:07.246924+0000","last_fullsized":"2023-10-30T12:35:07.246924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:16:14.550887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249901+0000","last_change":"2023-10-30T12:35:06.220236+0000","last_active":"2023-10-30T12:35:07.249901+0000","last_peered":"2023-10-30T12:35:07.249901+0000","last_clean":"2023-10-30T12:35:07.249901+0000","last_became_active":"2023-10-30T12:35:06.219526+0000","last_became_peered":"2023-10-30T12:35:06.219526+0000","last_unstale":"2023-10-30T12:35:07.249901+0000","last_undegraded":"2023-10-30T12:35:07.249901+0000","last_fullsized":"2023-10-30T12:35:07.249901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:00:25.297590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.186752+0000","last_change":"2023-10-30T12:34:55.206976+0000","last_active":"2023-10-30T12:35:06.186752+0000","last_peered":"2023-10-30T12:35:06.186752+0000","last_clean":"2023-10-30T12:35:06.186752+0000","last_became_active":"2023-10-30T12:34:55.205923+0000","last_became_peered":"2023-10-30T12:34:55.205923+0000","last_unstale":"2023-10-30T12:35:06.186752+0000","last_undegraded":"2023-10-30T12:35:06.186752+0000","last_fullsized":"2023-10-30T12:35:06.186752+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_clean_scrub_stamp":"2023-10-30T12:34:53.444020+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:25:28.782111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.170843+0000","last_change":"2023-10-30T12:35:05.197974+0000","last_active":"2023-10-30T12:35:06.170843+0000","last_peered":"2023-10-30T12:35:06.170843+0000","last_clean":"2023-10-30T12:35:06.170843+0000","last_became_active":"2023-10-30T12:35:05.197201+0000","last_became_peered":"2023-10-30T12:35:05.197201+0000","last_unstale":"2023-10-30T12:35:06.170843+0000","last_undegraded":"2023-10-30T12:35:06.170843+0000","last_fullsized":"2023-10-30T12:35:06.170843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:39:47.119091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247224+0000","last_change":"2023-10-30T12:35:06.217973+0000","last_active":"2023-10-30T12:35:07.247224+0000","last_peered":"2023-10-30T12:35:07.247224+0000","last_clean":"2023-10-30T12:35:07.247224+0000","last_became_active":"2023-10-30T12:35:06.217687+0000","last_became_peered":"2023-10-30T12:35:06.217687+0000","last_unstale":"2023-10-30T12:35:07.247224+0000","last_undegraded":"2023-10-30T12:35:07.247224+0000","last_fullsized":"2023-10-30T12:35:07.247224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:28.392113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.248732+0000","last_change":"2023-10-30T12:35:05.208339+0000","last_active":"2023-10-30T12:35:07.248732+0000","last_peered":"2023-10-30T12:35:07.248732+0000","last_clean":"2023-10-30T12:35:07.248732+0000","last_became_active":"2023-10-30T12:35:05.207828+0000","last_became_peered":"2023-10-30T12:35:05.207828+0000","last_unstale":"2023-10-30T12:35:07.248732+0000","last_undegraded":"2023-10-30T12:35:07.248732+0000","last_fullsized":"2023-10-30T12:35:07.248732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:00.649048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160255+0000","last_change":"2023-10-30T12:35:06.195094+0000","last_active":"2023-10-30T12:35:07.160255+0000","last_peered":"2023-10-30T12:35:07.160255+0000","last_clean":"2023-10-30T12:35:07.160255+0000","last_became_active":"2023-10-30T12:35:06.194791+0000","last_became_peered":"2023-10-30T12:35:06.194791+0000","last_unstale":"2023-10-30T12:35:07.160255+0000","last_undegraded":"2023-10-30T12:35:07.160255+0000","last_fullsized":"2023-10-30T12:35:07.160255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:41:24.310349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246778+0000","last_change":"2023-10-30T12:35:05.176296+0000","last_active":"2023-10-30T12:35:07.246778+0000","last_peered":"2023-10-30T12:35:07.246778+0000","last_clean":"2023-10-30T12:35:07.246778+0000","last_became_active":"2023-10-30T12:35:05.175959+0000","last_became_peered":"2023-10-30T12:35:05.175959+0000","last_unstale":"2023-10-30T12:35:07.246778+0000","last_undegraded":"2023-10-30T12:35:07.246778+0000","last_fullsized":"2023-10-30T12:35:07.246778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:53.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.171746+0000","last_change":"2023-10-30T12:35:06.167250+0000","last_active":"2023-10-30T12:35:06.171746+0000","last_peered":"2023-10-30T12:35:06.171746+0000","last_clean":"2023-10-30T12:35:06.171746+0000","last_became_active":"2023-10-30T12:35:06.164711+0000","last_became_peered":"2023-10-30T12:35:06.164711+0000","last_unstale":"2023-10-30T12:35:06.171746+0000","last_undegraded":"2023-10-30T12:35:06.171746+0000","last_fullsized":"2023-10-30T12:35:06.171746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:07:29.447189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.194382+0000","last_change":"2023-10-30T12:35:05.198964+0000","last_active":"2023-10-30T12:35:06.194382+0000","last_peered":"2023-10-30T12:35:06.194382+0000","last_clean":"2023-10-30T12:35:06.194382+0000","last_became_active":"2023-10-30T12:35:05.196873+0000","last_became_peered":"2023-10-30T12:35:05.196873+0000","last_unstale":"2023-10-30T12:35:06.194382+0000","last_undegraded":"2023-10-30T12:35:06.194382+0000","last_fullsized":"2023-10-30T12:35:06.194382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:47.793585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163526+0000","last_change":"2023-10-30T12:35:06.184123+0000","last_active":"2023-10-30T12:35:07.163526+0000","last_peered":"2023-10-30T12:35:07.163526+0000","last_clean":"2023-10-30T12:35:07.163526+0000","last_became_active":"2023-10-30T12:35:06.183225+0000","last_became_peered":"2023-10-30T12:35:06.183225+0000","last_unstale":"2023-10-30T12:35:07.163526+0000","last_undegraded":"2023-10-30T12:35:07.163526+0000","last_fullsized":"2023-10-30T12:35:07.163526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:03:47.097151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.161554+0000","last_change":"2023-10-30T12:35:05.177571+0000","last_active":"2023-10-30T12:35:06.161554+0000","last_peered":"2023-10-30T12:35:06.161554+0000","last_clean":"2023-10-30T12:35:06.161554+0000","last_became_active":"2023-10-30T12:35:05.177266+0000","last_became_peered":"2023-10-30T12:35:05.177266+0000","last_unstale":"2023-10-30T12:35:06.161554+0000","last_undegraded":"2023-10-30T12:35:06.161554+0000","last_fullsized":"2023-10-30T12:35:06.161554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:14.601934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.220695+0000","last_change":"2023-10-30T12:35:06.206033+0000","last_active":"2023-10-30T12:35:06.220695+0000","last_peered":"2023-10-30T12:35:06.220695+0000","last_clean":"2023-10-30T12:35:06.220695+0000","last_became_active":"2023-10-30T12:35:06.196307+0000","last_became_peered":"2023-10-30T12:35:06.196307+0000","last_unstale":"2023-10-30T12:35:06.220695+0000","last_undegraded":"2023-10-30T12:35:06.220695+0000","last_fullsized":"2023-10-30T12:35:06.220695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:48:47.167585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250720+0000","last_change":"2023-10-30T12:35:05.207594+0000","last_active":"2023-10-30T12:35:07.250720+0000","last_peered":"2023-10-30T12:35:07.250720+0000","last_clean":"2023-10-30T12:35:07.250720+0000","last_became_active":"2023-10-30T12:35:05.206930+0000","last_became_peered":"2023-10-30T12:35:05.206930+0000","last_unstale":"2023-10-30T12:35:07.250720+0000","last_undegraded":"2023-10-30T12:35:07.250720+0000","last_fullsized":"2023-10-30T12:35:07.250720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:23:46.519077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.169595+0000","last_change":"2023-10-30T12:35:06.165126+0000","last_active":"2023-10-30T12:35:06.169595+0000","last_peered":"2023-10-30T12:35:06.169595+0000","last_clean":"2023-10-30T12:35:06.169595+0000","last_became_active":"2023-10-30T12:35:06.164733+0000","last_became_peered":"2023-10-30T12:35:06.164733+0000","last_unstale":"2023-10-30T12:35:06.169595+0000","last_undegraded":"2023-10-30T12:35:06.169595+0000","last_fullsized":"2023-10-30T12:35:06.169595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:21:00.186309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.160646+0000","last_change":"2023-10-30T12:35:05.185034+0000","last_active":"2023-10-30T12:35:06.160646+0000","last_peered":"2023-10-30T12:35:06.160646+0000","last_clean":"2023-10-30T12:35:06.160646+0000","last_became_active":"2023-10-30T12:35:05.184695+0000","last_became_peered":"2023-10-30T12:35:05.184695+0000","last_unstale":"2023-10-30T12:35:06.160646+0000","last_undegraded":"2023-10-30T12:35:06.160646+0000","last_fullsized":"2023-10-30T12:35:06.160646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:01:42.019128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247746+0000","last_change":"2023-10-30T12:35:06.204707+0000","last_active":"2023-10-30T12:35:07.247746+0000","last_peered":"2023-10-30T12:35:07.247746+0000","last_clean":"2023-10-30T12:35:07.247746+0000","last_became_active":"2023-10-30T12:35:06.204422+0000","last_became_peered":"2023-10-30T12:35:06.204422+0000","last_unstale":"2023-10-30T12:35:07.247746+0000","last_undegraded":"2023-10-30T12:35:07.247746+0000","last_fullsized":"2023-10-30T12:35:07.247746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:58:16.758575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163764+0000","last_change":"2023-10-30T12:35:05.200871+0000","last_active":"2023-10-30T12:35:07.163764+0000","last_peered":"2023-10-30T12:35:07.163764+0000","last_clean":"2023-10-30T12:35:07.163764+0000","last_became_active":"2023-10-30T12:35:05.200449+0000","last_became_peered":"2023-10-30T12:35:05.200449+0000","last_unstale":"2023-10-30T12:35:07.163764+0000","last_undegraded":"2023-10-30T12:35:07.163764+0000","last_fullsized":"2023-10-30T12:35:07.163764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:39:20.867587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160562+0000","last_change":"2023-10-30T12:35:06.180850+0000","last_active":"2023-10-30T12:35:07.160562+0000","last_peered":"2023-10-30T12:35:07.160562+0000","last_clean":"2023-10-30T12:35:07.160562+0000","last_became_active":"2023-10-30T12:35:06.179412+0000","last_became_peered":"2023-10-30T12:35:06.179412+0000","last_unstale":"2023-10-30T12:35:07.160562+0000","last_undegraded":"2023-10-30T12:35:07.160562+0000","last_fullsized":"2023-10-30T12:35:07.160562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:47:44.055205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.248731+0000","last_change":"2023-10-30T12:35:05.207392+0000","last_active":"2023-10-30T12:35:07.248731+0000","last_peered":"2023-10-30T12:35:07.248731+0000","last_clean":"2023-10-30T12:35:07.248731+0000","last_became_active":"2023-10-30T12:35:05.206607+0000","last_became_peered":"2023-10-30T12:35:05.206607+0000","last_unstale":"2023-10-30T12:35:07.248731+0000","last_undegraded":"2023-10-30T12:35:07.248731+0000","last_fullsized":"2023-10-30T12:35:07.248731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:36.758740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163445+0000","last_change":"2023-10-30T12:35:06.182042+0000","last_active":"2023-10-30T12:35:07.163445+0000","last_peered":"2023-10-30T12:35:07.163445+0000","last_clean":"2023-10-30T12:35:07.163445+0000","last_became_active":"2023-10-30T12:35:06.181256+0000","last_became_peered":"2023-10-30T12:35:06.181256+0000","last_unstale":"2023-10-30T12:35:07.163445+0000","last_undegraded":"2023-10-30T12:35:07.163445+0000","last_fullsized":"2023-10-30T12:35:07.163445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:13:08.109392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.161184+0000","last_change":"2023-10-30T12:35:05.201024+0000","last_active":"2023-10-30T12:35:06.161184+0000","last_peered":"2023-10-30T12:35:06.161184+0000","last_clean":"2023-10-30T12:35:06.161184+0000","last_became_active":"2023-10-30T12:35:05.200724+0000","last_became_peered":"2023-10-30T12:35:05.200724+0000","last_unstale":"2023-10-30T12:35:06.161184+0000","last_undegraded":"2023-10-30T12:35:06.161184+0000","last_fullsized":"2023-10-30T12:35:06.161184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:24.746469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247249+0000","last_change":"2023-10-30T12:35:06.174636+0000","last_active":"2023-10-30T12:35:07.247249+0000","last_peered":"2023-10-30T12:35:07.247249+0000","last_clean":"2023-10-30T12:35:07.247249+0000","last_became_active":"2023-10-30T12:35:06.173996+0000","last_became_peered":"2023-10-30T12:35:06.173996+0000","last_unstale":"2023-10-30T12:35:07.247249+0000","last_undegraded":"2023-10-30T12:35:07.247249+0000","last_fullsized":"2023-10-30T12:35:07.247249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:01:38.503494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.160152+0000","last_change":"2023-10-30T12:35:05.164079+0000","last_active":"2023-10-30T12:35:06.160152+0000","last_peered":"2023-10-30T12:35:06.160152+0000","last_clean":"2023-10-30T12:35:06.160152+0000","last_became_active":"2023-10-30T12:35:05.163794+0000","last_became_peered":"2023-10-30T12:35:05.163794+0000","last_unstale":"2023-10-30T12:35:06.160152+0000","last_undegraded":"2023-10-30T12:35:06.160152+0000","last_fullsized":"2023-10-30T12:35:06.160152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:19.795656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247344+0000","last_change":"2023-10-30T12:35:06.183531+0000","last_active":"2023-10-30T12:35:07.247344+0000","last_peered":"2023-10-30T12:35:07.247344+0000","last_clean":"2023-10-30T12:35:07.247344+0000","last_became_active":"2023-10-30T12:35:06.183226+0000","last_became_peered":"2023-10-30T12:35:06.183226+0000","last_unstale":"2023-10-30T12:35:07.247344+0000","last_undegraded":"2023-10-30T12:35:07.247344+0000","last_fullsized":"2023-10-30T12:35:07.247344+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:46.768356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249947+0000","last_change":"2023-10-30T12:35:05.206995+0000","last_active":"2023-10-30T12:35:07.249947+0000","last_peered":"2023-10-30T12:35:07.249947+0000","last_clean":"2023-10-30T12:35:07.249947+0000","last_became_active":"2023-10-30T12:35:05.206518+0000","last_became_peered":"2023-10-30T12:35:05.206518+0000","last_unstale":"2023-10-30T12:35:07.249947+0000","last_undegraded":"2023-10-30T12:35:07.249947+0000","last_fullsized":"2023-10-30T12:35:07.249947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:41:33.703261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162528+0000","last_change":"2023-10-30T12:35:06.199557+0000","last_active":"2023-10-30T12:35:07.162528+0000","last_peered":"2023-10-30T12:35:07.162528+0000","last_clean":"2023-10-30T12:35:07.162528+0000","last_became_active":"2023-10-30T12:35:06.197861+0000","last_became_peered":"2023-10-30T12:35:06.197861+0000","last_unstale":"2023-10-30T12:35:07.162528+0000","last_undegraded":"2023-10-30T12:35:07.162528+0000","last_fullsized":"2023-10-30T12:35:07.162528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:30:46.092876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.227958+0000","last_change":"2023-10-30T12:35:06.213279+0000","last_active":"2023-10-30T12:35:06.227958+0000","last_peered":"2023-10-30T12:35:06.227958+0000","last_clean":"2023-10-30T12:35:06.227958+0000","last_became_active":"2023-10-30T12:35:06.212548+0000","last_became_peered":"2023-10-30T12:35:06.212548+0000","last_unstale":"2023-10-30T12:35:06.227958+0000","last_undegraded":"2023-10-30T12:35:06.227958+0000","last_fullsized":"2023-10-30T12:35:06.227958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:18:23.179654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249642+0000","last_change":"2023-10-30T12:35:05.206980+0000","last_active":"2023-10-30T12:35:07.249642+0000","last_peered":"2023-10-30T12:35:07.249642+0000","last_clean":"2023-10-30T12:35:07.249642+0000","last_became_active":"2023-10-30T12:35:05.206559+0000","last_became_peered":"2023-10-30T12:35:05.206559+0000","last_unstale":"2023-10-30T12:35:07.249642+0000","last_undegraded":"2023-10-30T12:35:07.249642+0000","last_fullsized":"2023-10-30T12:35:07.249642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:40.259622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.227871+0000","last_change":"2023-10-30T12:35:06.213053+0000","last_active":"2023-10-30T12:35:06.227871+0000","last_peered":"2023-10-30T12:35:06.227871+0000","last_clean":"2023-10-30T12:35:06.227871+0000","last_became_active":"2023-10-30T12:35:06.212505+0000","last_became_peered":"2023-10-30T12:35:06.212505+0000","last_unstale":"2023-10-30T12:35:06.227871+0000","last_undegraded":"2023-10-30T12:35:06.227871+0000","last_fullsized":"2023-10-30T12:35:06.227871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:55:18.318463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250138+0000","last_change":"2023-10-30T12:35:05.207851+0000","last_active":"2023-10-30T12:35:07.250138+0000","last_peered":"2023-10-30T12:35:07.250138+0000","last_clean":"2023-10-30T12:35:07.250138+0000","last_became_active":"2023-10-30T12:35:05.206879+0000","last_became_peered":"2023-10-30T12:35:05.206879+0000","last_unstale":"2023-10-30T12:35:07.250138+0000","last_undegraded":"2023-10-30T12:35:07.250138+0000","last_fullsized":"2023-10-30T12:35:07.250138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:43:15.717507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160637+0000","last_change":"2023-10-30T12:35:06.222373+0000","last_active":"2023-10-30T12:35:07.160637+0000","last_peered":"2023-10-30T12:35:07.160637+0000","last_clean":"2023-10-30T12:35:07.160637+0000","last_became_active":"2023-10-30T12:35:06.222094+0000","last_became_peered":"2023-10-30T12:35:06.222094+0000","last_unstale":"2023-10-30T12:35:07.160637+0000","last_undegraded":"2023-10-30T12:35:07.160637+0000","last_fullsized":"2023-10-30T12:35:07.160637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:42:32.482862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247675+0000","last_change":"2023-10-30T12:35:05.174319+0000","last_active":"2023-10-30T12:35:07.247675+0000","last_peered":"2023-10-30T12:35:07.247675+0000","last_clean":"2023-10-30T12:35:07.247675+0000","last_became_active":"2023-10-30T12:35:05.173984+0000","last_became_peered":"2023-10-30T12:35:05.173984+0000","last_unstale":"2023-10-30T12:35:07.247675+0000","last_undegraded":"2023-10-30T12:35:07.247675+0000","last_fullsized":"2023-10-30T12:35:07.247675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:11.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247593+0000","last_change":"2023-10-30T12:35:06.198881+0000","last_active":"2023-10-30T12:35:07.247593+0000","last_peered":"2023-10-30T12:35:07.247593+0000","last_clean":"2023-10-30T12:35:07.247593+0000","last_became_active":"2023-10-30T12:35:06.198504+0000","last_became_peered":"2023-10-30T12:35:06.198504+0000","last_unstale":"2023-10-30T12:35:07.247593+0000","last_undegraded":"2023-10-30T12:35:07.247593+0000","last_fullsized":"2023-10-30T12:35:07.247593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:06:33.845147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160706+0000","last_change":"2023-10-30T12:35:05.200926+0000","last_active":"2023-10-30T12:35:07.160706+0000","last_peered":"2023-10-30T12:35:07.160706+0000","last_clean":"2023-10-30T12:35:07.160706+0000","last_became_active":"2023-10-30T12:35:05.200623+0000","last_became_peered":"2023-10-30T12:35:05.200623+0000","last_unstale":"2023-10-30T12:35:07.160706+0000","last_undegraded":"2023-10-30T12:35:07.160706+0000","last_fullsized":"2023-10-30T12:35:07.160706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:38.942922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":27654605},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":72564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":72564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":806912,"data_stored":662932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":200704,"data_stored":65789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":72564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":72564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:35:12.314 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:35:12.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.490+0000 7f602afaa640 1 Processor -- start 2023-10-30T12:35:12.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.490+0000 7f602afaa640 1 -- start start 2023-10-30T12:35:12.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.490+0000 7f602afaa640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f602406a720 0x7f602406abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.490+0000 7f602afaa640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6024071290 con 0x7f60240bfec0 2023-10-30T12:35:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.490+0000 7f602afaa640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6024071570 con 0x7f6024159e60 2023-10-30T12:35:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.490+0000 7f602afaa640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6024071850 con 0x7f602406a720 2023-10-30T12:35:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.490+0000 7f6023fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f60240bfec0 0x7f60240c1a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:55326/0 (socket says 172.21.15.12:55326) 2023-10-30T12:35:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.490+0000 7f6023fff640 1 -- 172.21.15.12:0/4248677817 learned_addr learned my addr 172.21.15.12:0/4248677817 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:12.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.490+0000 7f6029547640 1 --2- 172.21.15.12:0/4248677817 >> v2:172.21.15.154:3300/0 conn(0x7f602406a720 0x7f602406abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.494+0000 7f60237fe640 1 -- 172.21.15.12:0/4248677817 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3301025241 0 0) 0x7f6024071290 con 0x7f60240bfec0 2023-10-30T12:35:12.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.494+0000 7f60237fe640 1 -- 172.21.15.12:0/4248677817 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6014003450 con 0x7f60240bfec0 2023-10-30T12:35:12.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.494+0000 7f6029547640 1 -- 172.21.15.12:0/4248677817 >> v1:172.21.15.154:6789/0 conn(0x7f6024159e60 legacy=0x7f602406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.494+0000 7f6029547640 1 -- 172.21.15.12:0/4248677817 >> v1:172.21.15.12:6789/0 conn(0x7f60240bfec0 legacy=0x7f60240c1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.494+0000 7f60237fe640 1 -- 172.21.15.12:0/4248677817 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3166864131 0 0) 0x7f6014003450 con 0x7f60240bfec0 2023-10-30T12:35:12.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.494+0000 7f6029547640 1 -- 172.21.15.12:0/4248677817 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6024071b30 con 0x7f602406a720 2023-10-30T12:35:12.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.494+0000 7f6029547640 1 --2- 172.21.15.12:0/4248677817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f602406a720 0x7f602406abb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f601c009d90 tx=0x7f601c01f060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c120075c37098b2e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:12.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.494+0000 7f60237fe640 1 -- 172.21.15.12:0/4248677817 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6010001680 con 0x7f602406a720 2023-10-30T12:35:12.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.494+0000 7f60237fe640 1 -- 172.21.15.12:0/4248677817 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c0a8bf0 con 0x7f602406a720 2023-10-30T12:35:12.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.494+0000 7f60237fe640 1 -- 172.21.15.12:0/4248677817 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c0b0440 con 0x7f602406a720 2023-10-30T12:35:12.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f602afaa640 1 -- 172.21.15.12:0/4248677817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f602406a720 msgr2=0x7f602406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f602afaa640 1 --2- 172.21.15.12:0/4248677817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f602406a720 0x7f602406abb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f601c009d90 tx=0x7f601c01f060 comp rx=0 tx=0).stop 2023-10-30T12:35:12.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f602afaa640 1 -- 172.21.15.12:0/4248677817 shutdown_connections 2023-10-30T12:35:12.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f602afaa640 1 --2- 172.21.15.12:0/4248677817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f602406a720 0x7f602406abb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f602afaa640 1 -- 172.21.15.12:0/4248677817 >> 172.21.15.12:0/4248677817 conn(0x7f60240c7910 msgr2=0x7f60240c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f602afaa640 1 -- 172.21.15.12:0/4248677817 shutdown_connections 2023-10-30T12:35:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f602afaa640 1 -- 172.21.15.12:0/4248677817 wait complete. 2023-10-30T12:35:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f602afaa640 1 Processor -- start 2023-10-30T12:35:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f602afaa640 1 -- start start 2023-10-30T12:35:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f602afaa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f602406a720 0x7f60241ef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f602afaa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60240bfec0 0x7f60240c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f602afaa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6024159e60 0x7f60240c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f602afaa640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60240a9a10 con 0x7f6024159e60 2023-10-30T12:35:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f602afaa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60241ef930 con 0x7f60240bfec0 2023-10-30T12:35:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f602afaa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60240c4c30 con 0x7f602406a720 2023-10-30T12:35:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f6029547640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6024159e60 0x7f60240c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f6029547640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6024159e60 0x7f60240c50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44186/0 (socket says 172.21.15.12:44186) 2023-10-30T12:35:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f6029547640 1 -- 172.21.15.12:0/3709757100 learned_addr learned my addr 172.21.15.12:0/3709757100 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f6023fff640 1 --2- 172.21.15.12:0/3709757100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60240bfec0 0x7f60240c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f6028d46640 1 --2- 172.21.15.12:0/3709757100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f602406a720 0x7f60241ef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f6029547640 1 -- 172.21.15.12:0/3709757100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f602406a720 msgr2=0x7f60241ef3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f6029547640 1 --2- 172.21.15.12:0/3709757100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f602406a720 0x7f60241ef3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f6029547640 1 -- 172.21.15.12:0/3709757100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60240bfec0 msgr2=0x7f60240c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f6029547640 1 --2- 172.21.15.12:0/3709757100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60240bfec0 0x7f60240c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.498+0000 7f6029547640 1 -- 172.21.15.12:0/3709757100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c009a70 con 0x7f6024159e60 2023-10-30T12:35:12.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.502+0000 7f6029547640 1 --2- 172.21.15.12:0/3709757100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6024159e60 0x7f60240c50c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f601c01cc90 tx=0x7f601c01be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:12.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.502+0000 7f60217fa640 1 -- 172.21.15.12:0/3709757100 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c009450 con 0x7f6024159e60 2023-10-30T12:35:12.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.502+0000 7f60217fa640 1 -- 172.21.15.12:0/3709757100 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c0b0dc0 con 0x7f6024159e60 2023-10-30T12:35:12.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.502+0000 7f602afaa640 1 -- 172.21.15.12:0/3709757100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60241e0b80 con 0x7f6024159e60 2023-10-30T12:35:12.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.502+0000 7f60217fa640 1 -- 172.21.15.12:0/3709757100 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c0bb640 con 0x7f6024159e60 2023-10-30T12:35:12.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.502+0000 7f602afaa640 1 -- 172.21.15.12:0/3709757100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60240c4300 con 0x7f6024159e60 2023-10-30T12:35:12.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.502+0000 7f602afaa640 1 -- 172.21.15.12:0/3709757100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602404e170 con 0x7f6024159e60 2023-10-30T12:35:12.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.502+0000 7f60217fa640 1 -- 172.21.15.12:0/3709757100 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f601c0bb7a0 con 0x7f6024159e60 2023-10-30T12:35:12.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.502+0000 7f60217fa640 1 --2- 172.21.15.12:0/3709757100 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f60080664d0 0x7f6008068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.502+0000 7f60217fa640 1 -- 172.21.15.12:0/3709757100 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f601c1223b0 con 0x7f6024159e60 2023-10-30T12:35:12.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.502+0000 7f6028d46640 1 --2- 172.21.15.12:0/3709757100 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f60080664d0 0x7f6008068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.506+0000 7f6028d46640 1 --2- 172.21.15.12:0/3709757100 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f60080664d0 0x7f6008068990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6018000f70 tx=0x7f6018009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:12.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.506+0000 7f60217fa640 1 -- 172.21.15.12:0/3709757100 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f601c0ef830 con 0x7f6024159e60 2023-10-30T12:35:12.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.662+0000 7f602afaa640 1 -- 172.21.15.12:0/3709757100 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60240c1520 con 0x7f60080664d0 2023-10-30T12:35:12.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.682+0000 7f60217fa640 1 -- 172.21.15.12:0/3709757100 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264564 (secure 0 0 0) 0x7f6018001380 con 0x7f60080664d0 2023-10-30T12:35:12.689 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:12.690 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-10-30T12:35:12.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.686+0000 7f602afaa640 1 -- 172.21.15.12:0/3709757100 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f60080664d0 msgr2=0x7f6008068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.686+0000 7f602afaa640 1 --2- 172.21.15.12:0/3709757100 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f60080664d0 0x7f6008068990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6018000f70 tx=0x7f6018009040 comp rx=0 tx=0).stop 2023-10-30T12:35:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.686+0000 7f602afaa640 1 -- 172.21.15.12:0/3709757100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6024159e60 msgr2=0x7f60240c50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.686+0000 7f602afaa640 1 --2- 172.21.15.12:0/3709757100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6024159e60 0x7f60240c50c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f601c01cc90 tx=0x7f601c01be00 comp rx=0 tx=0).stop 2023-10-30T12:35:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.686+0000 7f602afaa640 1 -- 172.21.15.12:0/3709757100 shutdown_connections 2023-10-30T12:35:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.686+0000 7f602afaa640 1 --2- 172.21.15.12:0/3709757100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6024159e60 0x7f60240c50c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.686+0000 7f602afaa640 1 --2- 172.21.15.12:0/3709757100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60240bfec0 0x7f60240c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.686+0000 7f602afaa640 1 --2- 172.21.15.12:0/3709757100 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f60080664d0 0x7f6008068990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.686+0000 7f602afaa640 1 --2- 172.21.15.12:0/3709757100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f602406a720 0x7f60241ef3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.686+0000 7f602afaa640 1 -- 172.21.15.12:0/3709757100 >> 172.21.15.12:0/3709757100 conn(0x7f60240c7910 msgr2=0x7f60240ca0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:12.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.686+0000 7f602afaa640 1 -- 172.21.15.12:0/3709757100 shutdown_connections 2023-10-30T12:35:12.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.686+0000 7f602afaa640 1 -- 172.21.15.12:0/3709757100 wait complete. 2023-10-30T12:35:12.709 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-30T12:35:10.936456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219024,"kb_used_data":2836,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755696,"statfs":{"total":773094113280,"available":772869832704,"internally_reserved":0,"allocated":2904064,"data_stored":1767073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12729,"internal_metadata":221236807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002157"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250527+0000","last_change":"2023-10-30T12:35:06.223711+0000","last_active":"2023-10-30T12:35:07.250527+0000","last_peered":"2023-10-30T12:35:07.250527+0000","last_clean":"2023-10-30T12:35:07.250527+0000","last_became_active":"2023-10-30T12:35:06.222724+0000","last_became_peered":"2023-10-30T12:35:06.222724+0000","last_unstale":"2023-10-30T12:35:07.250527+0000","last_undegraded":"2023-10-30T12:35:07.250527+0000","last_fullsized":"2023-10-30T12:35:07.250527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:34:35.217274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.187239+0000","last_change":"2023-10-30T12:35:06.182748+0000","last_active":"2023-10-30T12:35:06.187239+0000","last_peered":"2023-10-30T12:35:06.187239+0000","last_clean":"2023-10-30T12:35:06.187239+0000","last_became_active":"2023-10-30T12:35:06.182441+0000","last_became_peered":"2023-10-30T12:35:06.182441+0000","last_unstale":"2023-10-30T12:35:06.187239+0000","last_undegraded":"2023-10-30T12:35:06.187239+0000","last_fullsized":"2023-10-30T12:35:06.187239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:04:30.261815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160588+0000","last_change":"2023-10-30T12:35:06.180426+0000","last_active":"2023-10-30T12:35:07.160588+0000","last_peered":"2023-10-30T12:35:07.160588+0000","last_clean":"2023-10-30T12:35:07.160588+0000","last_became_active":"2023-10-30T12:35:06.180132+0000","last_became_peered":"2023-10-30T12:35:06.180132+0000","last_unstale":"2023-10-30T12:35:07.160588+0000","last_undegraded":"2023-10-30T12:35:07.160588+0000","last_fullsized":"2023-10-30T12:35:07.160588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:33:22.216739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.228228+0000","last_change":"2023-10-30T12:35:06.217986+0000","last_active":"2023-10-30T12:35:06.228228+0000","last_peered":"2023-10-30T12:35:06.228228+0000","last_clean":"2023-10-30T12:35:06.228228+0000","last_became_active":"2023-10-30T12:35:06.213987+0000","last_became_peered":"2023-10-30T12:35:06.213987+0000","last_unstale":"2023-10-30T12:35:06.228228+0000","last_undegraded":"2023-10-30T12:35:06.228228+0000","last_fullsized":"2023-10-30T12:35:06.228228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:37:57.869908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249406+0000","last_change":"2023-10-30T12:35:06.216098+0000","last_active":"2023-10-30T12:35:07.249406+0000","last_peered":"2023-10-30T12:35:07.249406+0000","last_clean":"2023-10-30T12:35:07.249406+0000","last_became_active":"2023-10-30T12:35:06.214964+0000","last_became_peered":"2023-10-30T12:35:06.214964+0000","last_unstale":"2023-10-30T12:35:07.249406+0000","last_undegraded":"2023-10-30T12:35:07.249406+0000","last_fullsized":"2023-10-30T12:35:07.249406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:37:43.156964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.228169+0000","last_change":"2023-10-30T12:35:06.217608+0000","last_active":"2023-10-30T12:35:06.228169+0000","last_peered":"2023-10-30T12:35:06.228169+0000","last_clean":"2023-10-30T12:35:06.228169+0000","last_became_active":"2023-10-30T12:35:06.214219+0000","last_became_peered":"2023-10-30T12:35:06.214219+0000","last_unstale":"2023-10-30T12:35:06.228169+0000","last_undegraded":"2023-10-30T12:35:06.228169+0000","last_fullsized":"2023-10-30T12:35:06.228169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:37:19.767075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249000+0000","last_change":"2023-10-30T12:35:06.217120+0000","last_active":"2023-10-30T12:35:07.249000+0000","last_peered":"2023-10-30T12:35:07.249000+0000","last_clean":"2023-10-30T12:35:07.249000+0000","last_became_active":"2023-10-30T12:35:06.216468+0000","last_became_peered":"2023-10-30T12:35:06.216468+0000","last_unstale":"2023-10-30T12:35:07.249000+0000","last_undegraded":"2023-10-30T12:35:07.249000+0000","last_fullsized":"2023-10-30T12:35:07.249000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:26:07.313967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246707+0000","last_change":"2023-10-30T12:35:06.214767+0000","last_active":"2023-10-30T12:35:07.246707+0000","last_peered":"2023-10-30T12:35:07.246707+0000","last_clean":"2023-10-30T12:35:07.246707+0000","last_became_active":"2023-10-30T12:35:06.214384+0000","last_became_peered":"2023-10-30T12:35:06.214384+0000","last_unstale":"2023-10-30T12:35:07.246707+0000","last_undegraded":"2023-10-30T12:35:07.246707+0000","last_fullsized":"2023-10-30T12:35:07.246707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:29.866538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250843+0000","last_change":"2023-10-30T12:35:06.223503+0000","last_active":"2023-10-30T12:35:07.250843+0000","last_peered":"2023-10-30T12:35:07.250843+0000","last_clean":"2023-10-30T12:35:07.250843+0000","last_became_active":"2023-10-30T12:35:06.222692+0000","last_became_peered":"2023-10-30T12:35:06.222692+0000","last_unstale":"2023-10-30T12:35:07.250843+0000","last_undegraded":"2023-10-30T12:35:07.250843+0000","last_fullsized":"2023-10-30T12:35:07.250843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:31:36.926720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163304+0000","last_change":"2023-10-30T12:35:06.215938+0000","last_active":"2023-10-30T12:35:07.163304+0000","last_peered":"2023-10-30T12:35:07.163304+0000","last_clean":"2023-10-30T12:35:07.163304+0000","last_became_active":"2023-10-30T12:35:06.215384+0000","last_became_peered":"2023-10-30T12:35:06.215384+0000","last_unstale":"2023-10-30T12:35:07.163304+0000","last_undegraded":"2023-10-30T12:35:07.163304+0000","last_fullsized":"2023-10-30T12:35:07.163304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:46:21.253555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162425+0000","last_change":"2023-10-30T12:35:06.215965+0000","last_active":"2023-10-30T12:35:07.162425+0000","last_peered":"2023-10-30T12:35:07.162425+0000","last_clean":"2023-10-30T12:35:07.162425+0000","last_became_active":"2023-10-30T12:35:06.215355+0000","last_became_peered":"2023-10-30T12:35:06.215355+0000","last_unstale":"2023-10-30T12:35:07.162425+0000","last_undegraded":"2023-10-30T12:35:07.162425+0000","last_fullsized":"2023-10-30T12:35:07.162425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:07.929526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164185+0000","last_change":"2023-10-30T12:35:06.197278+0000","last_active":"2023-10-30T12:35:07.164185+0000","last_peered":"2023-10-30T12:35:07.164185+0000","last_clean":"2023-10-30T12:35:07.164185+0000","last_became_active":"2023-10-30T12:35:06.196503+0000","last_became_peered":"2023-10-30T12:35:06.196503+0000","last_unstale":"2023-10-30T12:35:07.164185+0000","last_undegraded":"2023-10-30T12:35:07.164185+0000","last_fullsized":"2023-10-30T12:35:07.164185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:06.928264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247199+0000","last_change":"2023-10-30T12:35:06.195721+0000","last_active":"2023-10-30T12:35:07.247199+0000","last_peered":"2023-10-30T12:35:07.247199+0000","last_clean":"2023-10-30T12:35:07.247199+0000","last_became_active":"2023-10-30T12:35:06.195408+0000","last_became_peered":"2023-10-30T12:35:06.195408+0000","last_unstale":"2023-10-30T12:35:07.247199+0000","last_undegraded":"2023-10-30T12:35:07.247199+0000","last_fullsized":"2023-10-30T12:35:07.247199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:59:02.872646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160258+0000","last_change":"2023-10-30T12:35:06.182939+0000","last_active":"2023-10-30T12:35:07.160258+0000","last_peered":"2023-10-30T12:35:07.160258+0000","last_clean":"2023-10-30T12:35:07.160258+0000","last_became_active":"2023-10-30T12:35:06.182573+0000","last_became_peered":"2023-10-30T12:35:06.182573+0000","last_unstale":"2023-10-30T12:35:07.160258+0000","last_undegraded":"2023-10-30T12:35:07.160258+0000","last_fullsized":"2023-10-30T12:35:07.160258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:50.345142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.190140+0000","last_change":"2023-10-30T12:35:06.185636+0000","last_active":"2023-10-30T12:35:06.190140+0000","last_peered":"2023-10-30T12:35:06.190140+0000","last_clean":"2023-10-30T12:35:06.190140+0000","last_became_active":"2023-10-30T12:35:06.185290+0000","last_became_peered":"2023-10-30T12:35:06.185290+0000","last_unstale":"2023-10-30T12:35:06.190140+0000","last_undegraded":"2023-10-30T12:35:06.190140+0000","last_fullsized":"2023-10-30T12:35:06.190140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:31:12.579628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.229715+0000","last_change":"2023-10-30T12:35:06.214488+0000","last_active":"2023-10-30T12:35:06.229715+0000","last_peered":"2023-10-30T12:35:06.229715+0000","last_clean":"2023-10-30T12:35:06.229715+0000","last_became_active":"2023-10-30T12:35:06.213627+0000","last_became_peered":"2023-10-30T12:35:06.213627+0000","last_unstale":"2023-10-30T12:35:06.229715+0000","last_undegraded":"2023-10-30T12:35:06.229715+0000","last_fullsized":"2023-10-30T12:35:06.229715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:51:08.645452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.229700+0000","last_change":"2023-10-30T12:35:06.224903+0000","last_active":"2023-10-30T12:35:06.229700+0000","last_peered":"2023-10-30T12:35:06.229700+0000","last_clean":"2023-10-30T12:35:06.229700+0000","last_became_active":"2023-10-30T12:35:06.221696+0000","last_became_peered":"2023-10-30T12:35:06.221696+0000","last_unstale":"2023-10-30T12:35:06.229700+0000","last_undegraded":"2023-10-30T12:35:06.229700+0000","last_fullsized":"2023-10-30T12:35:06.229700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:25:12.393720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246945+0000","last_change":"2023-10-30T12:35:06.198303+0000","last_active":"2023-10-30T12:35:07.246945+0000","last_peered":"2023-10-30T12:35:07.246945+0000","last_clean":"2023-10-30T12:35:07.246945+0000","last_became_active":"2023-10-30T12:35:06.198074+0000","last_became_peered":"2023-10-30T12:35:06.198074+0000","last_unstale":"2023-10-30T12:35:07.246945+0000","last_undegraded":"2023-10-30T12:35:07.246945+0000","last_fullsized":"2023-10-30T12:35:07.246945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:21:36.079586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250503+0000","last_change":"2023-10-30T12:35:06.218556+0000","last_active":"2023-10-30T12:35:07.250503+0000","last_peered":"2023-10-30T12:35:07.250503+0000","last_clean":"2023-10-30T12:35:07.250503+0000","last_became_active":"2023-10-30T12:35:06.215935+0000","last_became_peered":"2023-10-30T12:35:06.215935+0000","last_unstale":"2023-10-30T12:35:07.250503+0000","last_undegraded":"2023-10-30T12:35:07.250503+0000","last_fullsized":"2023-10-30T12:35:07.250503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:36:31.084542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162438+0000","last_change":"2023-10-30T12:35:06.201302+0000","last_active":"2023-10-30T12:35:07.162438+0000","last_peered":"2023-10-30T12:35:07.162438+0000","last_clean":"2023-10-30T12:35:07.162438+0000","last_became_active":"2023-10-30T12:35:06.199812+0000","last_became_peered":"2023-10-30T12:35:06.199812+0000","last_unstale":"2023-10-30T12:35:07.162438+0000","last_undegraded":"2023-10-30T12:35:07.162438+0000","last_fullsized":"2023-10-30T12:35:07.162438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:48.803738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160558+0000","last_change":"2023-10-30T12:35:06.217139+0000","last_active":"2023-10-30T12:35:07.160558+0000","last_peered":"2023-10-30T12:35:07.160558+0000","last_clean":"2023-10-30T12:35:07.160558+0000","last_became_active":"2023-10-30T12:35:06.216861+0000","last_became_peered":"2023-10-30T12:35:06.216861+0000","last_unstale":"2023-10-30T12:35:07.160558+0000","last_undegraded":"2023-10-30T12:35:07.160558+0000","last_fullsized":"2023-10-30T12:35:07.160558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:05:22.193180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.229733+0000","last_change":"2023-10-30T12:35:06.225140+0000","last_active":"2023-10-30T12:35:06.229733+0000","last_peered":"2023-10-30T12:35:06.229733+0000","last_clean":"2023-10-30T12:35:06.229733+0000","last_became_active":"2023-10-30T12:35:06.221841+0000","last_became_peered":"2023-10-30T12:35:06.221841+0000","last_unstale":"2023-10-30T12:35:06.229733+0000","last_undegraded":"2023-10-30T12:35:06.229733+0000","last_fullsized":"2023-10-30T12:35:06.229733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:05.853879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247032+0000","last_change":"2023-10-30T12:35:06.182662+0000","last_active":"2023-10-30T12:35:07.247032+0000","last_peered":"2023-10-30T12:35:07.247032+0000","last_clean":"2023-10-30T12:35:07.247032+0000","last_became_active":"2023-10-30T12:35:06.182314+0000","last_became_peered":"2023-10-30T12:35:06.182314+0000","last_unstale":"2023-10-30T12:35:07.247032+0000","last_undegraded":"2023-10-30T12:35:07.247032+0000","last_fullsized":"2023-10-30T12:35:07.247032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:41:49.587856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.227997+0000","last_change":"2023-10-30T12:35:06.214113+0000","last_active":"2023-10-30T12:35:06.227997+0000","last_peered":"2023-10-30T12:35:06.227997+0000","last_clean":"2023-10-30T12:35:06.227997+0000","last_became_active":"2023-10-30T12:35:06.213402+0000","last_became_peered":"2023-10-30T12:35:06.213402+0000","last_unstale":"2023-10-30T12:35:06.227997+0000","last_undegraded":"2023-10-30T12:35:06.227997+0000","last_fullsized":"2023-10-30T12:35:06.227997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:52:48.311491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247781+0000","last_change":"2023-10-30T12:35:06.188570+0000","last_active":"2023-10-30T12:35:07.247781+0000","last_peered":"2023-10-30T12:35:07.247781+0000","last_clean":"2023-10-30T12:35:07.247781+0000","last_became_active":"2023-10-30T12:35:06.188298+0000","last_became_peered":"2023-10-30T12:35:06.188298+0000","last_unstale":"2023-10-30T12:35:07.247781+0000","last_undegraded":"2023-10-30T12:35:07.247781+0000","last_fullsized":"2023-10-30T12:35:07.247781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:28.281254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163623+0000","last_change":"2023-10-30T12:35:06.208821+0000","last_active":"2023-10-30T12:35:07.163623+0000","last_peered":"2023-10-30T12:35:07.163623+0000","last_clean":"2023-10-30T12:35:07.163623+0000","last_became_active":"2023-10-30T12:35:06.202507+0000","last_became_peered":"2023-10-30T12:35:06.202507+0000","last_unstale":"2023-10-30T12:35:07.163623+0000","last_undegraded":"2023-10-30T12:35:07.163623+0000","last_fullsized":"2023-10-30T12:35:07.163623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:40:13.124316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.175568+0000","last_change":"2023-10-30T12:35:06.170313+0000","last_active":"2023-10-30T12:35:06.175568+0000","last_peered":"2023-10-30T12:35:06.175568+0000","last_clean":"2023-10-30T12:35:06.175568+0000","last_became_active":"2023-10-30T12:35:06.170030+0000","last_became_peered":"2023-10-30T12:35:06.170030+0000","last_unstale":"2023-10-30T12:35:06.175568+0000","last_undegraded":"2023-10-30T12:35:06.175568+0000","last_fullsized":"2023-10-30T12:35:06.175568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:03:31.913507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247487+0000","last_change":"2023-10-30T12:35:06.189448+0000","last_active":"2023-10-30T12:35:07.247487+0000","last_peered":"2023-10-30T12:35:07.247487+0000","last_clean":"2023-10-30T12:35:07.247487+0000","last_became_active":"2023-10-30T12:35:06.189171+0000","last_became_peered":"2023-10-30T12:35:06.189171+0000","last_unstale":"2023-10-30T12:35:07.247487+0000","last_undegraded":"2023-10-30T12:35:07.247487+0000","last_fullsized":"2023-10-30T12:35:07.247487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:18:51.463273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160658+0000","last_change":"2023-10-30T12:35:06.183680+0000","last_active":"2023-10-30T12:35:07.160658+0000","last_peered":"2023-10-30T12:35:07.160658+0000","last_clean":"2023-10-30T12:35:07.160658+0000","last_became_active":"2023-10-30T12:35:06.183348+0000","last_became_peered":"2023-10-30T12:35:06.183348+0000","last_unstale":"2023-10-30T12:35:07.160658+0000","last_undegraded":"2023-10-30T12:35:07.160658+0000","last_fullsized":"2023-10-30T12:35:07.160658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:54:25.429897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.220759+0000","last_change":"2023-10-30T12:35:06.206675+0000","last_active":"2023-10-30T12:35:06.220759+0000","last_peered":"2023-10-30T12:35:06.220759+0000","last_clean":"2023-10-30T12:35:06.220759+0000","last_became_active":"2023-10-30T12:35:06.196368+0000","last_became_peered":"2023-10-30T12:35:06.196368+0000","last_unstale":"2023-10-30T12:35:06.220759+0000","last_undegraded":"2023-10-30T12:35:06.220759+0000","last_fullsized":"2023-10-30T12:35:06.220759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:11:16.050410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246681+0000","last_change":"2023-10-30T12:35:06.214408+0000","last_active":"2023-10-30T12:35:07.246681+0000","last_peered":"2023-10-30T12:35:07.246681+0000","last_clean":"2023-10-30T12:35:07.246681+0000","last_became_active":"2023-10-30T12:35:06.213994+0000","last_became_peered":"2023-10-30T12:35:06.213994+0000","last_unstale":"2023-10-30T12:35:07.246681+0000","last_undegraded":"2023-10-30T12:35:07.246681+0000","last_fullsized":"2023-10-30T12:35:07.246681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:05:58.830698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162027+0000","last_change":"2023-10-30T12:35:06.198444+0000","last_active":"2023-10-30T12:35:07.162027+0000","last_peered":"2023-10-30T12:35:07.162027+0000","last_clean":"2023-10-30T12:35:07.162027+0000","last_became_active":"2023-10-30T12:35:06.197647+0000","last_became_peered":"2023-10-30T12:35:06.197647+0000","last_unstale":"2023-10-30T12:35:07.162027+0000","last_undegraded":"2023-10-30T12:35:07.162027+0000","last_fullsized":"2023-10-30T12:35:07.162027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:28:56.075420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163077+0000","last_change":"2023-10-30T12:35:06.168377+0000","last_active":"2023-10-30T12:35:07.163077+0000","last_peered":"2023-10-30T12:35:07.163077+0000","last_clean":"2023-10-30T12:35:07.163077+0000","last_became_active":"2023-10-30T12:35:06.167558+0000","last_became_peered":"2023-10-30T12:35:06.167558+0000","last_unstale":"2023-10-30T12:35:07.163077+0000","last_undegraded":"2023-10-30T12:35:07.163077+0000","last_fullsized":"2023-10-30T12:35:07.163077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:57:28.009164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247303+0000","last_change":"2023-10-30T12:35:05.200844+0000","last_active":"2023-10-30T12:35:07.247303+0000","last_peered":"2023-10-30T12:35:07.247303+0000","last_clean":"2023-10-30T12:35:07.247303+0000","last_became_active":"2023-10-30T12:35:05.200169+0000","last_became_peered":"2023-10-30T12:35:05.200169+0000","last_unstale":"2023-10-30T12:35:07.247303+0000","last_undegraded":"2023-10-30T12:35:07.247303+0000","last_fullsized":"2023-10-30T12:35:07.247303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:07:19.571357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160718+0000","last_change":"2023-10-30T12:35:06.194964+0000","last_active":"2023-10-30T12:35:07.160718+0000","last_peered":"2023-10-30T12:35:07.160718+0000","last_clean":"2023-10-30T12:35:07.160718+0000","last_became_active":"2023-10-30T12:35:06.194689+0000","last_became_peered":"2023-10-30T12:35:06.194689+0000","last_unstale":"2023-10-30T12:35:07.160718+0000","last_undegraded":"2023-10-30T12:35:07.160718+0000","last_fullsized":"2023-10-30T12:35:07.160718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:30:53.410341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162742+0000","last_change":"2023-10-30T12:35:05.183569+0000","last_active":"2023-10-30T12:35:07.162742+0000","last_peered":"2023-10-30T12:35:07.162742+0000","last_clean":"2023-10-30T12:35:07.162742+0000","last_became_active":"2023-10-30T12:35:05.182761+0000","last_became_peered":"2023-10-30T12:35:05.182761+0000","last_unstale":"2023-10-30T12:35:07.162742+0000","last_undegraded":"2023-10-30T12:35:07.162742+0000","last_fullsized":"2023-10-30T12:35:07.162742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:49:44.311814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247108+0000","last_change":"2023-10-30T12:35:05.181278+0000","last_active":"2023-10-30T12:35:07.247108+0000","last_peered":"2023-10-30T12:35:07.247108+0000","last_clean":"2023-10-30T12:35:07.247108+0000","last_became_active":"2023-10-30T12:35:05.180730+0000","last_became_peered":"2023-10-30T12:35:05.180730+0000","last_unstale":"2023-10-30T12:35:07.247108+0000","last_undegraded":"2023-10-30T12:35:07.247108+0000","last_fullsized":"2023-10-30T12:35:07.247108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:07:10.828707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246964+0000","last_change":"2023-10-30T12:35:06.180049+0000","last_active":"2023-10-30T12:35:07.246964+0000","last_peered":"2023-10-30T12:35:07.246964+0000","last_clean":"2023-10-30T12:35:07.246964+0000","last_became_active":"2023-10-30T12:35:06.179764+0000","last_became_peered":"2023-10-30T12:35:06.179764+0000","last_unstale":"2023-10-30T12:35:07.246964+0000","last_undegraded":"2023-10-30T12:35:07.246964+0000","last_fullsized":"2023-10-30T12:35:07.246964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:33.136727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.193600+0000","last_change":"2023-10-30T12:35:05.202539+0000","last_active":"2023-10-30T12:35:06.193600+0000","last_peered":"2023-10-30T12:35:06.193600+0000","last_clean":"2023-10-30T12:35:06.193600+0000","last_became_active":"2023-10-30T12:35:05.201853+0000","last_became_peered":"2023-10-30T12:35:05.201853+0000","last_unstale":"2023-10-30T12:35:06.193600+0000","last_undegraded":"2023-10-30T12:35:06.193600+0000","last_fullsized":"2023-10-30T12:35:06.193600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:33:33.151433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.175406+0000","last_change":"2023-10-30T12:35:06.173567+0000","last_active":"2023-10-30T12:35:06.175406+0000","last_peered":"2023-10-30T12:35:06.175406+0000","last_clean":"2023-10-30T12:35:06.175406+0000","last_became_active":"2023-10-30T12:35:06.170179+0000","last_became_peered":"2023-10-30T12:35:06.170179+0000","last_unstale":"2023-10-30T12:35:06.175406+0000","last_undegraded":"2023-10-30T12:35:06.175406+0000","last_fullsized":"2023-10-30T12:35:06.175406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:02:23.116259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163553+0000","last_change":"2023-10-30T12:35:05.198546+0000","last_active":"2023-10-30T12:35:07.163553+0000","last_peered":"2023-10-30T12:35:07.163553+0000","last_clean":"2023-10-30T12:35:07.163553+0000","last_became_active":"2023-10-30T12:35:05.197606+0000","last_became_peered":"2023-10-30T12:35:05.197606+0000","last_unstale":"2023-10-30T12:35:07.163553+0000","last_undegraded":"2023-10-30T12:35:07.163553+0000","last_fullsized":"2023-10-30T12:35:07.163553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:26:57.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.229829+0000","last_change":"2023-10-30T12:35:06.226044+0000","last_active":"2023-10-30T12:35:06.229829+0000","last_peered":"2023-10-30T12:35:06.229829+0000","last_clean":"2023-10-30T12:35:06.229829+0000","last_became_active":"2023-10-30T12:35:06.225392+0000","last_became_peered":"2023-10-30T12:35:06.225392+0000","last_unstale":"2023-10-30T12:35:06.229829+0000","last_undegraded":"2023-10-30T12:35:06.229829+0000","last_fullsized":"2023-10-30T12:35:06.229829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:47:38.246937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.175959+0000","last_change":"2023-10-30T12:35:05.197614+0000","last_active":"2023-10-30T12:35:06.175959+0000","last_peered":"2023-10-30T12:35:06.175959+0000","last_clean":"2023-10-30T12:35:06.175959+0000","last_became_active":"2023-10-30T12:35:05.196899+0000","last_became_peered":"2023-10-30T12:35:05.196899+0000","last_unstale":"2023-10-30T12:35:06.175959+0000","last_undegraded":"2023-10-30T12:35:06.175959+0000","last_fullsized":"2023-10-30T12:35:06.175959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:21:00.152426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160715+0000","last_change":"2023-10-30T12:35:06.222344+0000","last_active":"2023-10-30T12:35:07.160715+0000","last_peered":"2023-10-30T12:35:07.160715+0000","last_clean":"2023-10-30T12:35:07.160715+0000","last_became_active":"2023-10-30T12:35:06.222057+0000","last_became_peered":"2023-10-30T12:35:06.222057+0000","last_unstale":"2023-10-30T12:35:07.160715+0000","last_undegraded":"2023-10-30T12:35:07.160715+0000","last_fullsized":"2023-10-30T12:35:07.160715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:50:46.691223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.161741+0000","last_change":"2023-10-30T12:35:05.197501+0000","last_active":"2023-10-30T12:35:06.161741+0000","last_peered":"2023-10-30T12:35:06.161741+0000","last_clean":"2023-10-30T12:35:06.161741+0000","last_became_active":"2023-10-30T12:35:05.197190+0000","last_became_peered":"2023-10-30T12:35:05.197190+0000","last_unstale":"2023-10-30T12:35:06.161741+0000","last_undegraded":"2023-10-30T12:35:06.161741+0000","last_fullsized":"2023-10-30T12:35:06.161741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:53:31.121721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164505+0000","last_change":"2023-10-30T12:35:06.204885+0000","last_active":"2023-10-30T12:35:07.164505+0000","last_peered":"2023-10-30T12:35:07.164505+0000","last_clean":"2023-10-30T12:35:07.164505+0000","last_became_active":"2023-10-30T12:35:06.204122+0000","last_became_peered":"2023-10-30T12:35:06.204122+0000","last_unstale":"2023-10-30T12:35:07.164505+0000","last_undegraded":"2023-10-30T12:35:07.164505+0000","last_fullsized":"2023-10-30T12:35:07.164505+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:27:18.433575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162559+0000","last_change":"2023-10-30T12:35:05.180005+0000","last_active":"2023-10-30T12:35:07.162559+0000","last_peered":"2023-10-30T12:35:07.162559+0000","last_clean":"2023-10-30T12:35:07.162559+0000","last_became_active":"2023-10-30T12:35:05.179547+0000","last_became_peered":"2023-10-30T12:35:05.179547+0000","last_unstale":"2023-10-30T12:35:07.162559+0000","last_undegraded":"2023-10-30T12:35:07.162559+0000","last_fullsized":"2023-10-30T12:35:07.162559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:13:03.137433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249565+0000","last_change":"2023-10-30T12:35:06.216619+0000","last_active":"2023-10-30T12:35:07.249565+0000","last_peered":"2023-10-30T12:35:07.249565+0000","last_clean":"2023-10-30T12:35:07.249565+0000","last_became_active":"2023-10-30T12:35:06.215128+0000","last_became_peered":"2023-10-30T12:35:06.215128+0000","last_unstale":"2023-10-30T12:35:07.249565+0000","last_undegraded":"2023-10-30T12:35:07.249565+0000","last_fullsized":"2023-10-30T12:35:07.249565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:48:45.673159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160709+0000","last_change":"2023-10-30T12:35:05.190041+0000","last_active":"2023-10-30T12:35:07.160709+0000","last_peered":"2023-10-30T12:35:07.160709+0000","last_clean":"2023-10-30T12:35:07.160709+0000","last_became_active":"2023-10-30T12:35:05.189573+0000","last_became_peered":"2023-10-30T12:35:05.189573+0000","last_unstale":"2023-10-30T12:35:07.160709+0000","last_undegraded":"2023-10-30T12:35:07.160709+0000","last_fullsized":"2023-10-30T12:35:07.160709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:02:57.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160412+0000","last_change":"2023-10-30T12:35:06.217334+0000","last_active":"2023-10-30T12:35:07.160412+0000","last_peered":"2023-10-30T12:35:07.160412+0000","last_clean":"2023-10-30T12:35:07.160412+0000","last_became_active":"2023-10-30T12:35:06.216885+0000","last_became_peered":"2023-10-30T12:35:06.216885+0000","last_unstale":"2023-10-30T12:35:07.160412+0000","last_undegraded":"2023-10-30T12:35:07.160412+0000","last_fullsized":"2023-10-30T12:35:07.160412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:38:26.745898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.194047+0000","last_change":"2023-10-30T12:35:05.203198+0000","last_active":"2023-10-30T12:35:06.194047+0000","last_peered":"2023-10-30T12:35:06.194047+0000","last_clean":"2023-10-30T12:35:06.194047+0000","last_became_active":"2023-10-30T12:35:05.202612+0000","last_became_peered":"2023-10-30T12:35:05.202612+0000","last_unstale":"2023-10-30T12:35:06.194047+0000","last_undegraded":"2023-10-30T12:35:06.194047+0000","last_fullsized":"2023-10-30T12:35:06.194047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:55.986236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249574+0000","last_change":"2023-10-30T12:35:06.216307+0000","last_active":"2023-10-30T12:35:07.249574+0000","last_peered":"2023-10-30T12:35:07.249574+0000","last_clean":"2023-10-30T12:35:07.249574+0000","last_became_active":"2023-10-30T12:35:06.215553+0000","last_became_peered":"2023-10-30T12:35:06.215553+0000","last_unstale":"2023-10-30T12:35:07.249574+0000","last_undegraded":"2023-10-30T12:35:07.249574+0000","last_fullsized":"2023-10-30T12:35:07.249574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:46:25.396759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.161910+0000","last_change":"2023-10-30T12:35:05.181882+0000","last_active":"2023-10-30T12:35:07.161910+0000","last_peered":"2023-10-30T12:35:07.161910+0000","last_clean":"2023-10-30T12:35:07.161910+0000","last_became_active":"2023-10-30T12:35:05.181115+0000","last_became_peered":"2023-10-30T12:35:05.181115+0000","last_unstale":"2023-10-30T12:35:07.161910+0000","last_undegraded":"2023-10-30T12:35:07.161910+0000","last_fullsized":"2023-10-30T12:35:07.161910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:18:55.496940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162013+0000","last_change":"2023-10-30T12:35:06.198990+0000","last_active":"2023-10-30T12:35:07.162013+0000","last_peered":"2023-10-30T12:35:07.162013+0000","last_clean":"2023-10-30T12:35:07.162013+0000","last_became_active":"2023-10-30T12:35:06.197722+0000","last_became_peered":"2023-10-30T12:35:06.197722+0000","last_unstale":"2023-10-30T12:35:07.162013+0000","last_undegraded":"2023-10-30T12:35:07.162013+0000","last_fullsized":"2023-10-30T12:35:07.162013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:56:51.757976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246641+0000","last_change":"2023-10-30T12:35:05.169455+0000","last_active":"2023-10-30T12:35:07.246641+0000","last_peered":"2023-10-30T12:35:07.246641+0000","last_clean":"2023-10-30T12:35:07.246641+0000","last_became_active":"2023-10-30T12:35:05.169130+0000","last_became_peered":"2023-10-30T12:35:05.169130+0000","last_unstale":"2023-10-30T12:35:07.246641+0000","last_undegraded":"2023-10-30T12:35:07.246641+0000","last_fullsized":"2023-10-30T12:35:07.246641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:59:14.096315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249183+0000","last_change":"2023-10-30T12:35:06.215545+0000","last_active":"2023-10-30T12:35:07.249183+0000","last_peered":"2023-10-30T12:35:07.249183+0000","last_clean":"2023-10-30T12:35:07.249183+0000","last_became_active":"2023-10-30T12:35:06.214839+0000","last_became_peered":"2023-10-30T12:35:06.214839+0000","last_unstale":"2023-10-30T12:35:07.249183+0000","last_undegraded":"2023-10-30T12:35:07.249183+0000","last_fullsized":"2023-10-30T12:35:07.249183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:54:34.206981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164455+0000","last_change":"2023-10-30T12:35:05.189361+0000","last_active":"2023-10-30T12:35:07.164455+0000","last_peered":"2023-10-30T12:35:07.164455+0000","last_clean":"2023-10-30T12:35:07.164455+0000","last_became_active":"2023-10-30T12:35:05.188862+0000","last_became_peered":"2023-10-30T12:35:05.188862+0000","last_unstale":"2023-10-30T12:35:07.164455+0000","last_undegraded":"2023-10-30T12:35:07.164455+0000","last_fullsized":"2023-10-30T12:35:07.164455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:13.319732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250712+0000","last_change":"2023-10-30T12:35:06.223118+0000","last_active":"2023-10-30T12:35:07.250712+0000","last_peered":"2023-10-30T12:35:07.250712+0000","last_clean":"2023-10-30T12:35:07.250712+0000","last_became_active":"2023-10-30T12:35:06.216533+0000","last_became_peered":"2023-10-30T12:35:06.216533+0000","last_unstale":"2023-10-30T12:35:07.250712+0000","last_undegraded":"2023-10-30T12:35:07.250712+0000","last_fullsized":"2023-10-30T12:35:07.250712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:32:29.374526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247032+0000","last_change":"2023-10-30T12:35:05.177609+0000","last_active":"2023-10-30T12:35:07.247032+0000","last_peered":"2023-10-30T12:35:07.247032+0000","last_clean":"2023-10-30T12:35:07.247032+0000","last_became_active":"2023-10-30T12:35:05.177316+0000","last_became_peered":"2023-10-30T12:35:05.177316+0000","last_unstale":"2023-10-30T12:35:07.247032+0000","last_undegraded":"2023-10-30T12:35:07.247032+0000","last_fullsized":"2023-10-30T12:35:07.247032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:46.820925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251056+0000","last_change":"2023-10-30T12:35:06.222283+0000","last_active":"2023-10-30T12:35:07.251056+0000","last_peered":"2023-10-30T12:35:07.251056+0000","last_clean":"2023-10-30T12:35:07.251056+0000","last_became_active":"2023-10-30T12:35:06.221701+0000","last_became_peered":"2023-10-30T12:35:06.221701+0000","last_unstale":"2023-10-30T12:35:07.251056+0000","last_undegraded":"2023-10-30T12:35:07.251056+0000","last_fullsized":"2023-10-30T12:35:07.251056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:37:32.956627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.161885+0000","last_change":"2023-10-30T12:35:05.171787+0000","last_active":"2023-10-30T12:35:06.161885+0000","last_peered":"2023-10-30T12:35:06.161885+0000","last_clean":"2023-10-30T12:35:06.161885+0000","last_became_active":"2023-10-30T12:35:05.171491+0000","last_became_peered":"2023-10-30T12:35:05.171491+0000","last_unstale":"2023-10-30T12:35:06.161885+0000","last_undegraded":"2023-10-30T12:35:06.161885+0000","last_fullsized":"2023-10-30T12:35:06.161885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:43:15.331175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.228023+0000","last_change":"2023-10-30T12:35:06.214464+0000","last_active":"2023-10-30T12:35:06.228023+0000","last_peered":"2023-10-30T12:35:06.228023+0000","last_clean":"2023-10-30T12:35:06.228023+0000","last_became_active":"2023-10-30T12:35:06.213493+0000","last_became_peered":"2023-10-30T12:35:06.213493+0000","last_unstale":"2023-10-30T12:35:06.228023+0000","last_undegraded":"2023-10-30T12:35:06.228023+0000","last_fullsized":"2023-10-30T12:35:06.228023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:39:43.751284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246924+0000","last_change":"2023-10-30T12:35:05.197811+0000","last_active":"2023-10-30T12:35:07.246924+0000","last_peered":"2023-10-30T12:35:07.246924+0000","last_clean":"2023-10-30T12:35:07.246924+0000","last_became_active":"2023-10-30T12:35:05.197453+0000","last_became_peered":"2023-10-30T12:35:05.197453+0000","last_unstale":"2023-10-30T12:35:07.246924+0000","last_undegraded":"2023-10-30T12:35:07.246924+0000","last_fullsized":"2023-10-30T12:35:07.246924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:16:14.550887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249901+0000","last_change":"2023-10-30T12:35:06.220236+0000","last_active":"2023-10-30T12:35:07.249901+0000","last_peered":"2023-10-30T12:35:07.249901+0000","last_clean":"2023-10-30T12:35:07.249901+0000","last_became_active":"2023-10-30T12:35:06.219526+0000","last_became_peered":"2023-10-30T12:35:06.219526+0000","last_unstale":"2023-10-30T12:35:07.249901+0000","last_undegraded":"2023-10-30T12:35:07.249901+0000","last_fullsized":"2023-10-30T12:35:07.249901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:00:25.297590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.186752+0000","last_change":"2023-10-30T12:34:55.206976+0000","last_active":"2023-10-30T12:35:06.186752+0000","last_peered":"2023-10-30T12:35:06.186752+0000","last_clean":"2023-10-30T12:35:06.186752+0000","last_became_active":"2023-10-30T12:34:55.205923+0000","last_became_peered":"2023-10-30T12:34:55.205923+0000","last_unstale":"2023-10-30T12:35:06.186752+0000","last_undegraded":"2023-10-30T12:35:06.186752+0000","last_fullsized":"2023-10-30T12:35:06.186752+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_clean_scrub_stamp":"2023-10-30T12:34:53.444020+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:25:28.782111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.170843+0000","last_change":"2023-10-30T12:35:05.197974+0000","last_active":"2023-10-30T12:35:06.170843+0000","last_peered":"2023-10-30T12:35:06.170843+0000","last_clean":"2023-10-30T12:35:06.170843+0000","last_became_active":"2023-10-30T12:35:05.197201+0000","last_became_peered":"2023-10-30T12:35:05.197201+0000","last_unstale":"2023-10-30T12:35:06.170843+0000","last_undegraded":"2023-10-30T12:35:06.170843+0000","last_fullsized":"2023-10-30T12:35:06.170843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:39:47.119091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247224+0000","last_change":"2023-10-30T12:35:06.217973+0000","last_active":"2023-10-30T12:35:07.247224+0000","last_peered":"2023-10-30T12:35:07.247224+0000","last_clean":"2023-10-30T12:35:07.247224+0000","last_became_active":"2023-10-30T12:35:06.217687+0000","last_became_peered":"2023-10-30T12:35:06.217687+0000","last_unstale":"2023-10-30T12:35:07.247224+0000","last_undegraded":"2023-10-30T12:35:07.247224+0000","last_fullsized":"2023-10-30T12:35:07.247224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:28.392113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.248732+0000","last_change":"2023-10-30T12:35:05.208339+0000","last_active":"2023-10-30T12:35:07.248732+0000","last_peered":"2023-10-30T12:35:07.248732+0000","last_clean":"2023-10-30T12:35:07.248732+0000","last_became_active":"2023-10-30T12:35:05.207828+0000","last_became_peered":"2023-10-30T12:35:05.207828+0000","last_unstale":"2023-10-30T12:35:07.248732+0000","last_undegraded":"2023-10-30T12:35:07.248732+0000","last_fullsized":"2023-10-30T12:35:07.248732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:00.649048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160255+0000","last_change":"2023-10-30T12:35:06.195094+0000","last_active":"2023-10-30T12:35:07.160255+0000","last_peered":"2023-10-30T12:35:07.160255+0000","last_clean":"2023-10-30T12:35:07.160255+0000","last_became_active":"2023-10-30T12:35:06.194791+0000","last_became_peered":"2023-10-30T12:35:06.194791+0000","last_unstale":"2023-10-30T12:35:07.160255+0000","last_undegraded":"2023-10-30T12:35:07.160255+0000","last_fullsized":"2023-10-30T12:35:07.160255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:41:24.310349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246778+0000","last_change":"2023-10-30T12:35:05.176296+0000","last_active":"2023-10-30T12:35:07.246778+0000","last_peered":"2023-10-30T12:35:07.246778+0000","last_clean":"2023-10-30T12:35:07.246778+0000","last_became_active":"2023-10-30T12:35:05.175959+0000","last_became_peered":"2023-10-30T12:35:05.175959+0000","last_unstale":"2023-10-30T12:35:07.246778+0000","last_undegraded":"2023-10-30T12:35:07.246778+0000","last_fullsized":"2023-10-30T12:35:07.246778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:53.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.171746+0000","last_change":"2023-10-30T12:35:06.167250+0000","last_active":"2023-10-30T12:35:06.171746+0000","last_peered":"2023-10-30T12:35:06.171746+0000","last_clean":"2023-10-30T12:35:06.171746+0000","last_became_active":"2023-10-30T12:35:06.164711+0000","last_became_peered":"2023-10-30T12:35:06.164711+0000","last_unstale":"2023-10-30T12:35:06.171746+0000","last_undegraded":"2023-10-30T12:35:06.171746+0000","last_fullsized":"2023-10-30T12:35:06.171746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:07:29.447189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.194382+0000","last_change":"2023-10-30T12:35:05.198964+0000","last_active":"2023-10-30T12:35:06.194382+0000","last_peered":"2023-10-30T12:35:06.194382+0000","last_clean":"2023-10-30T12:35:06.194382+0000","last_became_active":"2023-10-30T12:35:05.196873+0000","last_became_peered":"2023-10-30T12:35:05.196873+0000","last_unstale":"2023-10-30T12:35:06.194382+0000","last_undegraded":"2023-10-30T12:35:06.194382+0000","last_fullsized":"2023-10-30T12:35:06.194382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:47.793585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163526+0000","last_change":"2023-10-30T12:35:06.184123+0000","last_active":"2023-10-30T12:35:07.163526+0000","last_peered":"2023-10-30T12:35:07.163526+0000","last_clean":"2023-10-30T12:35:07.163526+0000","last_became_active":"2023-10-30T12:35:06.183225+0000","last_became_peered":"2023-10-30T12:35:06.183225+0000","last_unstale":"2023-10-30T12:35:07.163526+0000","last_undegraded":"2023-10-30T12:35:07.163526+0000","last_fullsized":"2023-10-30T12:35:07.163526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:03:47.097151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.161554+0000","last_change":"2023-10-30T12:35:05.177571+0000","last_active":"2023-10-30T12:35:06.161554+0000","last_peered":"2023-10-30T12:35:06.161554+0000","last_clean":"2023-10-30T12:35:06.161554+0000","last_became_active":"2023-10-30T12:35:05.177266+0000","last_became_peered":"2023-10-30T12:35:05.177266+0000","last_unstale":"2023-10-30T12:35:06.161554+0000","last_undegraded":"2023-10-30T12:35:06.161554+0000","last_fullsized":"2023-10-30T12:35:06.161554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:14.601934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.220695+0000","last_change":"2023-10-30T12:35:06.206033+0000","last_active":"2023-10-30T12:35:06.220695+0000","last_peered":"2023-10-30T12:35:06.220695+0000","last_clean":"2023-10-30T12:35:06.220695+0000","last_became_active":"2023-10-30T12:35:06.196307+0000","last_became_peered":"2023-10-30T12:35:06.196307+0000","last_unstale":"2023-10-30T12:35:06.220695+0000","last_undegraded":"2023-10-30T12:35:06.220695+0000","last_fullsized":"2023-10-30T12:35:06.220695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:48:47.167585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250720+0000","last_change":"2023-10-30T12:35:05.207594+0000","last_active":"2023-10-30T12:35:07.250720+0000","last_peered":"2023-10-30T12:35:07.250720+0000","last_clean":"2023-10-30T12:35:07.250720+0000","last_became_active":"2023-10-30T12:35:05.206930+0000","last_became_peered":"2023-10-30T12:35:05.206930+0000","last_unstale":"2023-10-30T12:35:07.250720+0000","last_undegraded":"2023-10-30T12:35:07.250720+0000","last_fullsized":"2023-10-30T12:35:07.250720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:23:46.519077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.169595+0000","last_change":"2023-10-30T12:35:06.165126+0000","last_active":"2023-10-30T12:35:06.169595+0000","last_peered":"2023-10-30T12:35:06.169595+0000","last_clean":"2023-10-30T12:35:06.169595+0000","last_became_active":"2023-10-30T12:35:06.164733+0000","last_became_peered":"2023-10-30T12:35:06.164733+0000","last_unstale":"2023-10-30T12:35:06.169595+0000","last_undegraded":"2023-10-30T12:35:06.169595+0000","last_fullsized":"2023-10-30T12:35:06.169595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:21:00.186309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.160646+0000","last_change":"2023-10-30T12:35:05.185034+0000","last_active":"2023-10-30T12:35:06.160646+0000","last_peered":"2023-10-30T12:35:06.160646+0000","last_clean":"2023-10-30T12:35:06.160646+0000","last_became_active":"2023-10-30T12:35:05.184695+0000","last_became_peered":"2023-10-30T12:35:05.184695+0000","last_unstale":"2023-10-30T12:35:06.160646+0000","last_undegraded":"2023-10-30T12:35:06.160646+0000","last_fullsized":"2023-10-30T12:35:06.160646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:01:42.019128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247746+0000","last_change":"2023-10-30T12:35:06.204707+0000","last_active":"2023-10-30T12:35:07.247746+0000","last_peered":"2023-10-30T12:35:07.247746+0000","last_clean":"2023-10-30T12:35:07.247746+0000","last_became_active":"2023-10-30T12:35:06.204422+0000","last_became_peered":"2023-10-30T12:35:06.204422+0000","last_unstale":"2023-10-30T12:35:07.247746+0000","last_undegraded":"2023-10-30T12:35:07.247746+0000","last_fullsized":"2023-10-30T12:35:07.247746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:58:16.758575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163764+0000","last_change":"2023-10-30T12:35:05.200871+0000","last_active":"2023-10-30T12:35:07.163764+0000","last_peered":"2023-10-30T12:35:07.163764+0000","last_clean":"2023-10-30T12:35:07.163764+0000","last_became_active":"2023-10-30T12:35:05.200449+0000","last_became_peered":"2023-10-30T12:35:05.200449+0000","last_unstale":"2023-10-30T12:35:07.163764+0000","last_undegraded":"2023-10-30T12:35:07.163764+0000","last_fullsized":"2023-10-30T12:35:07.163764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:39:20.867587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160562+0000","last_change":"2023-10-30T12:35:06.180850+0000","last_active":"2023-10-30T12:35:07.160562+0000","last_peered":"2023-10-30T12:35:07.160562+0000","last_clean":"2023-10-30T12:35:07.160562+0000","last_became_active":"2023-10-30T12:35:06.179412+0000","last_became_peered":"2023-10-30T12:35:06.179412+0000","last_unstale":"2023-10-30T12:35:07.160562+0000","last_undegraded":"2023-10-30T12:35:07.160562+0000","last_fullsized":"2023-10-30T12:35:07.160562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:47:44.055205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.248731+0000","last_change":"2023-10-30T12:35:05.207392+0000","last_active":"2023-10-30T12:35:07.248731+0000","last_peered":"2023-10-30T12:35:07.248731+0000","last_clean":"2023-10-30T12:35:07.248731+0000","last_became_active":"2023-10-30T12:35:05.206607+0000","last_became_peered":"2023-10-30T12:35:05.206607+0000","last_unstale":"2023-10-30T12:35:07.248731+0000","last_undegraded":"2023-10-30T12:35:07.248731+0000","last_fullsized":"2023-10-30T12:35:07.248731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:36.758740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163445+0000","last_change":"2023-10-30T12:35:06.182042+0000","last_active":"2023-10-30T12:35:07.163445+0000","last_peered":"2023-10-30T12:35:07.163445+0000","last_clean":"2023-10-30T12:35:07.163445+0000","last_became_active":"2023-10-30T12:35:06.181256+0000","last_became_peered":"2023-10-30T12:35:06.181256+0000","last_unstale":"2023-10-30T12:35:07.163445+0000","last_undegraded":"2023-10-30T12:35:07.163445+0000","last_fullsized":"2023-10-30T12:35:07.163445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:13:08.109392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.161184+0000","last_change":"2023-10-30T12:35:05.201024+0000","last_active":"2023-10-30T12:35:06.161184+0000","last_peered":"2023-10-30T12:35:06.161184+0000","last_clean":"2023-10-30T12:35:06.161184+0000","last_became_active":"2023-10-30T12:35:05.200724+0000","last_became_peered":"2023-10-30T12:35:05.200724+0000","last_unstale":"2023-10-30T12:35:06.161184+0000","last_undegraded":"2023-10-30T12:35:06.161184+0000","last_fullsized":"2023-10-30T12:35:06.161184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:24.746469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247249+0000","last_change":"2023-10-30T12:35:06.174636+0000","last_active":"2023-10-30T12:35:07.247249+0000","last_peered":"2023-10-30T12:35:07.247249+0000","last_clean":"2023-10-30T12:35:07.247249+0000","last_became_active":"2023-10-30T12:35:06.173996+0000","last_became_peered":"2023-10-30T12:35:06.173996+0000","last_unstale":"2023-10-30T12:35:07.247249+0000","last_undegraded":"2023-10-30T12:35:07.247249+0000","last_fullsized":"2023-10-30T12:35:07.247249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:01:38.503494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.160152+0000","last_change":"2023-10-30T12:35:05.164079+0000","last_active":"2023-10-30T12:35:06.160152+0000","last_peered":"2023-10-30T12:35:06.160152+0000","last_clean":"2023-10-30T12:35:06.160152+0000","last_became_active":"2023-10-30T12:35:05.163794+0000","last_became_peered":"2023-10-30T12:35:05.163794+0000","last_unstale":"2023-10-30T12:35:06.160152+0000","last_undegraded":"2023-10-30T12:35:06.160152+0000","last_fullsized":"2023-10-30T12:35:06.160152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:19.795656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247344+0000","last_change":"2023-10-30T12:35:06.183531+0000","last_active":"2023-10-30T12:35:07.247344+0000","last_peered":"2023-10-30T12:35:07.247344+0000","last_clean":"2023-10-30T12:35:07.247344+0000","last_became_active":"2023-10-30T12:35:06.183226+0000","last_became_peered":"2023-10-30T12:35:06.183226+0000","last_unstale":"2023-10-30T12:35:07.247344+0000","last_undegraded":"2023-10-30T12:35:07.247344+0000","last_fullsized":"2023-10-30T12:35:07.247344+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:46.768356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249947+0000","last_change":"2023-10-30T12:35:05.206995+0000","last_active":"2023-10-30T12:35:07.249947+0000","last_peered":"2023-10-30T12:35:07.249947+0000","last_clean":"2023-10-30T12:35:07.249947+0000","last_became_active":"2023-10-30T12:35:05.206518+0000","last_became_peered":"2023-10-30T12:35:05.206518+0000","last_unstale":"2023-10-30T12:35:07.249947+0000","last_undegraded":"2023-10-30T12:35:07.249947+0000","last_fullsized":"2023-10-30T12:35:07.249947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:41:33.703261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162528+0000","last_change":"2023-10-30T12:35:06.199557+0000","last_active":"2023-10-30T12:35:07.162528+0000","last_peered":"2023-10-30T12:35:07.162528+0000","last_clean":"2023-10-30T12:35:07.162528+0000","last_became_active":"2023-10-30T12:35:06.197861+0000","last_became_peered":"2023-10-30T12:35:06.197861+0000","last_unstale":"2023-10-30T12:35:07.162528+0000","last_undegraded":"2023-10-30T12:35:07.162528+0000","last_fullsized":"2023-10-30T12:35:07.162528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:30:46.092876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.227958+0000","last_change":"2023-10-30T12:35:06.213279+0000","last_active":"2023-10-30T12:35:06.227958+0000","last_peered":"2023-10-30T12:35:06.227958+0000","last_clean":"2023-10-30T12:35:06.227958+0000","last_became_active":"2023-10-30T12:35:06.212548+0000","last_became_peered":"2023-10-30T12:35:06.212548+0000","last_unstale":"2023-10-30T12:35:06.227958+0000","last_undegraded":"2023-10-30T12:35:06.227958+0000","last_fullsized":"2023-10-30T12:35:06.227958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:18:23.179654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249642+0000","last_change":"2023-10-30T12:35:05.206980+0000","last_active":"2023-10-30T12:35:07.249642+0000","last_peered":"2023-10-30T12:35:07.249642+0000","last_clean":"2023-10-30T12:35:07.249642+0000","last_became_active":"2023-10-30T12:35:05.206559+0000","last_became_peered":"2023-10-30T12:35:05.206559+0000","last_unstale":"2023-10-30T12:35:07.249642+0000","last_undegraded":"2023-10-30T12:35:07.249642+0000","last_fullsized":"2023-10-30T12:35:07.249642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:40.259622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.227871+0000","last_change":"2023-10-30T12:35:06.213053+0000","last_active":"2023-10-30T12:35:06.227871+0000","last_peered":"2023-10-30T12:35:06.227871+0000","last_clean":"2023-10-30T12:35:06.227871+0000","last_became_active":"2023-10-30T12:35:06.212505+0000","last_became_peered":"2023-10-30T12:35:06.212505+0000","last_unstale":"2023-10-30T12:35:06.227871+0000","last_undegraded":"2023-10-30T12:35:06.227871+0000","last_fullsized":"2023-10-30T12:35:06.227871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:55:18.318463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250138+0000","last_change":"2023-10-30T12:35:05.207851+0000","last_active":"2023-10-30T12:35:07.250138+0000","last_peered":"2023-10-30T12:35:07.250138+0000","last_clean":"2023-10-30T12:35:07.250138+0000","last_became_active":"2023-10-30T12:35:05.206879+0000","last_became_peered":"2023-10-30T12:35:05.206879+0000","last_unstale":"2023-10-30T12:35:07.250138+0000","last_undegraded":"2023-10-30T12:35:07.250138+0000","last_fullsized":"2023-10-30T12:35:07.250138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:43:15.717507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160637+0000","last_change":"2023-10-30T12:35:06.222373+0000","last_active":"2023-10-30T12:35:07.160637+0000","last_peered":"2023-10-30T12:35:07.160637+0000","last_clean":"2023-10-30T12:35:07.160637+0000","last_became_active":"2023-10-30T12:35:06.222094+0000","last_became_peered":"2023-10-30T12:35:06.222094+0000","last_unstale":"2023-10-30T12:35:07.160637+0000","last_undegraded":"2023-10-30T12:35:07.160637+0000","last_fullsized":"2023-10-30T12:35:07.160637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:42:32.482862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247675+0000","last_change":"2023-10-30T12:35:05.174319+0000","last_active":"2023-10-30T12:35:07.247675+0000","last_peered":"2023-10-30T12:35:07.247675+0000","last_clean":"2023-10-30T12:35:07.247675+0000","last_became_active":"2023-10-30T12:35:05.173984+0000","last_became_peered":"2023-10-30T12:35:05.173984+0000","last_unstale":"2023-10-30T12:35:07.247675+0000","last_undegraded":"2023-10-30T12:35:07.247675+0000","last_fullsized":"2023-10-30T12:35:07.247675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:11.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247593+0000","last_change":"2023-10-30T12:35:06.198881+0000","last_active":"2023-10-30T12:35:07.247593+0000","last_peered":"2023-10-30T12:35:07.247593+0000","last_clean":"2023-10-30T12:35:07.247593+0000","last_became_active":"2023-10-30T12:35:06.198504+0000","last_became_peered":"2023-10-30T12:35:06.198504+0000","last_unstale":"2023-10-30T12:35:07.247593+0000","last_undegraded":"2023-10-30T12:35:07.247593+0000","last_fullsized":"2023-10-30T12:35:07.247593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:06:33.845147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160706+0000","last_change":"2023-10-30T12:35:05.200926+0000","last_active":"2023-10-30T12:35:07.160706+0000","last_peered":"2023-10-30T12:35:07.160706+0000","last_clean":"2023-10-30T12:35:07.160706+0000","last_became_active":"2023-10-30T12:35:05.200623+0000","last_became_peered":"2023-10-30T12:35:05.200623+0000","last_unstale":"2023-10-30T12:35:07.160706+0000","last_undegraded":"2023-10-30T12:35:07.160706+0000","last_fullsized":"2023-10-30T12:35:07.160706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:38.942922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":27654605},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":72564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":72564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":806912,"data_stored":662932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":200704,"data_stored":65789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":72564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":72564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:35:12.714 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-30T12:35:12.715 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-30T12:35:12.715 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-30T12:35:12.715 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-30T12:35:12.716 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-30T12:35:12.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.890+0000 7f92c95d9640 1 Processor -- start 2023-10-30T12:35:12.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.890+0000 7f92c95d9640 1 -- start start 2023-10-30T12:35:12.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.894+0000 7f92c95d9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f92c41538a0 0x7f92c415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.894+0000 7f92c95d9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f92c40cf160 0x7f92c40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.894+0000 7f92c95d9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c40cb800 con 0x7f92c40ca0e0 2023-10-30T12:35:12.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.894+0000 7f92c95d9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c415dc30 con 0x7f92c41538a0 2023-10-30T12:35:12.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.894+0000 7f92c95d9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c40cb330 con 0x7f92c40cf160 2023-10-30T12:35:12.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.894+0000 7f92c2ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f92c41538a0 0x7f92c415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.894+0000 7f92c2ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f92c41538a0 0x7f92c415d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44198/0 (socket says 172.21.15.12:44198) 2023-10-30T12:35:12.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.894+0000 7f92c2ffd640 1 -- 172.21.15.12:0/1389346764 learned_addr learned my addr 172.21.15.12:0/1389346764 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:12.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.894+0000 7f92bbfff640 1 --1- 172.21.15.12:0/1389346764 >> v1:172.21.15.12:6789/0 conn(0x7f92c40ca0e0 0x7f92c415a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:55340/0 (socket says 172.21.15.12:55340) 2023-10-30T12:35:12.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.894+0000 7f92c37fe640 1 --2- 172.21.15.12:0/1389346764 >> v2:172.21.15.154:3300/0 conn(0x7f92c40cf160 0x7f92c40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.894+0000 7f92c2ffd640 1 -- 172.21.15.12:0/1389346764 >> v2:172.21.15.154:3300/0 conn(0x7f92c40cf160 msgr2=0x7f92c40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:12.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.894+0000 7f92c2ffd640 1 --2- 172.21.15.12:0/1389346764 >> v2:172.21.15.154:3300/0 conn(0x7f92c40cf160 0x7f92c40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.894+0000 7f92c2ffd640 1 -- 172.21.15.12:0/1389346764 >> v1:172.21.15.12:6789/0 conn(0x7f92c40ca0e0 legacy=0x7f92c415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.894+0000 7f92c2ffd640 1 -- 172.21.15.12:0/1389346764 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c40cad70 con 0x7f92c41538a0 2023-10-30T12:35:12.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c2ffd640 1 --2- 172.21.15.12:0/1389346764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c41538a0 0x7f92c415d6f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f92c40b5330 tx=0x7f92b401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687bb286001de027 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:12.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c27fc640 1 -- 172.21.15.12:0/1389346764 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b40a89f0 con 0x7f92c41538a0 2023-10-30T12:35:12.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c27fc640 1 -- 172.21.15.12:0/1389346764 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b40a8b50 con 0x7f92c41538a0 2023-10-30T12:35:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c27fc640 1 -- 172.21.15.12:0/1389346764 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b40b0440 con 0x7f92c41538a0 2023-10-30T12:35:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 -- 172.21.15.12:0/1389346764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c41538a0 msgr2=0x7f92c415d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 --2- 172.21.15.12:0/1389346764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c41538a0 0x7f92c415d6f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f92c40b5330 tx=0x7f92b401ecf0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 -- 172.21.15.12:0/1389346764 shutdown_connections 2023-10-30T12:35:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 --2- 172.21.15.12:0/1389346764 >> v2:172.21.15.154:3300/0 conn(0x7f92c40cf160 0x7f92c40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 --2- 172.21.15.12:0/1389346764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c41538a0 0x7f92c415d6f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 -- 172.21.15.12:0/1389346764 >> 172.21.15.12:0/1389346764 conn(0x7f92c406bfe0 msgr2=0x7f92c406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 -- 172.21.15.12:0/1389346764 shutdown_connections 2023-10-30T12:35:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 -- 172.21.15.12:0/1389346764 wait complete. 2023-10-30T12:35:12.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 Processor -- start 2023-10-30T12:35:12.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 -- start start 2023-10-30T12:35:12.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c40ca0e0 0x7f92c4158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92c40cf160 0x7f92c40ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92c41538a0 0x7f92c4154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c41e3100 con 0x7f92c40ca0e0 2023-10-30T12:35:12.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c41596d0 con 0x7f92c41538a0 2023-10-30T12:35:12.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.898+0000 7f92c95d9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c41e33e0 con 0x7f92c40cf160 2023-10-30T12:35:12.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92c2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c40ca0e0 0x7f92c4158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92c2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c40ca0e0 0x7f92c4158840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44202/0 (socket says 172.21.15.12:44202) 2023-10-30T12:35:12.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92c2ffd640 1 -- 172.21.15.12:0/1448203449 learned_addr learned my addr 172.21.15.12:0/1448203449 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92c37fe640 1 --2- 172.21.15.12:0/1448203449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92c41538a0 0x7f92c4154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92bbfff640 1 --2- 172.21.15.12:0/1448203449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92c40cf160 0x7f92c40ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92c2ffd640 1 -- 172.21.15.12:0/1448203449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92c40cf160 msgr2=0x7f92c40ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92c2ffd640 1 --2- 172.21.15.12:0/1448203449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92c40cf160 0x7f92c40ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92c2ffd640 1 -- 172.21.15.12:0/1448203449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92c41538a0 msgr2=0x7f92c4154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:12.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92c2ffd640 1 --2- 172.21.15.12:0/1448203449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92c41538a0 0x7f92c4154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:12.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92c2ffd640 1 -- 172.21.15.12:0/1448203449 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b40094d0 con 0x7f92c40ca0e0 2023-10-30T12:35:12.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92c2ffd640 1 --2- 172.21.15.12:0/1448203449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c40ca0e0 0x7f92c4158840 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f92b401b900 tx=0x7f92b401ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:12.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92bb7fe640 1 -- 172.21.15.12:0/1448203449 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b40ba600 con 0x7f92c40ca0e0 2023-10-30T12:35:12.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92bb7fe640 1 -- 172.21.15.12:0/1448203449 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b40ba760 con 0x7f92c40ca0e0 2023-10-30T12:35:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92c95d9640 1 -- 172.21.15.12:0/1448203449 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c41e51b0 con 0x7f92c40ca0e0 2023-10-30T12:35:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92bb7fe640 1 -- 172.21.15.12:0/1448203449 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b40bab10 con 0x7f92c40ca0e0 2023-10-30T12:35:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92c95d9640 1 -- 172.21.15.12:0/1448203449 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c41e2d90 con 0x7f92c40ca0e0 2023-10-30T12:35:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92c95d9640 1 -- 172.21.15.12:0/1448203449 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c40b2a50 con 0x7f92c40ca0e0 2023-10-30T12:35:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.902+0000 7f92bb7fe640 1 -- 172.21.15.12:0/1448203449 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f92b40bac70 con 0x7f92c40ca0e0 2023-10-30T12:35:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.906+0000 7f92bb7fe640 1 --2- 172.21.15.12:0/1448203449 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f92a40664a0 0x7f92a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.906+0000 7f92bb7fe640 1 -- 172.21.15.12:0/1448203449 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92b4122560 con 0x7f92c40ca0e0 2023-10-30T12:35:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.906+0000 7f92bbfff640 1 --2- 172.21.15.12:0/1448203449 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f92a40664a0 0x7f92a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.906+0000 7f92bbfff640 1 --2- 172.21.15.12:0/1448203449 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f92a40664a0 0x7f92a4068960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92c40c97b0 tx=0x7f92ac007cb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:12.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:12.906+0000 7f92bb7fe640 1 -- 172.21.15.12:0/1448203449 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f92b40ef8b0 con 0x7f92c40ca0e0 2023-10-30T12:35:13.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.102+0000 7f92c95d9640 1 -- 172.21.15.12:0/1448203449 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f92c404e170 con 0x7f92c40ca0e0 2023-10-30T12:35:13.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.106+0000 7f92bb7fe640 1 -- 172.21.15.12:0/1448203449 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164437 (secure 0 0 0) 0x7f92b40f2ed0 con 0x7f92c40ca0e0 2023-10-30T12:35:13.111 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:13.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.110+0000 7f92c95d9640 1 -- 172.21.15.12:0/1448203449 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f92a40664a0 msgr2=0x7f92a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:13.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.110+0000 7f92c95d9640 1 --2- 172.21.15.12:0/1448203449 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f92a40664a0 0x7f92a4068960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92c40c97b0 tx=0x7f92ac007cb0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.110+0000 7f92c95d9640 1 -- 172.21.15.12:0/1448203449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c40ca0e0 msgr2=0x7f92c4158840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:13.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.110+0000 7f92c95d9640 1 --2- 172.21.15.12:0/1448203449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c40ca0e0 0x7f92c4158840 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f92b401b900 tx=0x7f92b401ba90 comp rx=0 tx=0).stop 2023-10-30T12:35:13.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.110+0000 7f92c95d9640 1 -- 172.21.15.12:0/1448203449 shutdown_connections 2023-10-30T12:35:13.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.110+0000 7f92c95d9640 1 --2- 172.21.15.12:0/1448203449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92c41538a0 0x7f92c4154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.110+0000 7f92c95d9640 1 --2- 172.21.15.12:0/1448203449 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f92a40664a0 0x7f92a4068960 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.110+0000 7f92c95d9640 1 --2- 172.21.15.12:0/1448203449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92c40cf160 0x7f92c40ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.110+0000 7f92c95d9640 1 --2- 172.21.15.12:0/1448203449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92c40ca0e0 0x7f92c4158840 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.110+0000 7f92c95d9640 1 -- 172.21.15.12:0/1448203449 >> 172.21.15.12:0/1448203449 conn(0x7f92c406bfe0 msgr2=0x7f92c4065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:13.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.110+0000 7f92c95d9640 1 -- 172.21.15.12:0/1448203449 shutdown_connections 2023-10-30T12:35:13.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.110+0000 7f92c95d9640 1 -- 172.21.15.12:0/1448203449 wait complete. 2023-10-30T12:35:13.131 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":5,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1474604059},{"type":"v1","addr":"172.21.15.154:6833","nonce":1474604059}]},"active_addr":"172.21.15.154:6833/1474604059","active_change":"2023-10-30T12:34:44.918731+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":1584118795}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":3641660150}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":3469077898}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":1225041088}]}]} 2023-10-30T12:35:13.136 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-30T12:35:13.136 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-30T12:35:13.136 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:35:13.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.278+0000 7fdc74cea640 1 Processor -- start 2023-10-30T12:35:13.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.278+0000 7fdc74cea640 1 -- start start 2023-10-30T12:35:13.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc74cea640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdc70069e90 0x7fdc7015a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc74cea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc7006cb80 0x7fdc7006d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc74cea640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7006d550 con 0x7fdc7006a270 2023-10-30T12:35:13.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc74cea640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc700c8570 con 0x7fdc70069e90 2023-10-30T12:35:13.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc74cea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc700c6c10 con 0x7fdc7006cb80 2023-10-30T12:35:13.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc67fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdc7006a270 0x7fdc700c9cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:55354/0 (socket says 172.21.15.12:55354) 2023-10-30T12:35:13.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc67fff640 1 -- 172.21.15.12:0/2225674458 learned_addr learned my addr 172.21.15.12:0/2225674458 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:13.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6e59c640 1 --2- 172.21.15.12:0/2225674458 >> v2:172.21.15.154:3300/0 conn(0x7fdc70069e90 0x7fdc7015a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6ed9d640 1 --2- 172.21.15.12:0/2225674458 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc7006cb80 0x7fdc7006d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 -- 172.21.15.12:0/2225674458 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2012832 0 0) 0x7fdc7006d550 con 0x7fdc7006a270 2023-10-30T12:35:13.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 -- 172.21.15.12:0/2225674458 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc5c003530 con 0x7fdc7006a270 2023-10-30T12:35:13.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 -- 172.21.15.12:0/2225674458 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2104208213 0 0) 0x7fdc5c003530 con 0x7fdc7006a270 2023-10-30T12:35:13.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 -- 172.21.15.12:0/2225674458 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc7006cb80 msgr2=0x7fdc7006d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:13.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 --2- 172.21.15.12:0/2225674458 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc7006cb80 0x7fdc7006d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 -- 172.21.15.12:0/2225674458 >> v2:172.21.15.154:3300/0 conn(0x7fdc70069e90 msgr2=0x7fdc7015a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:13.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 --2- 172.21.15.12:0/2225674458 >> v2:172.21.15.154:3300/0 conn(0x7fdc70069e90 0x7fdc7015a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 -- 172.21.15.12:0/2225674458 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc700c6ef0 con 0x7fdc7006a270 2023-10-30T12:35:13.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 -- 172.21.15.12:0/2225674458 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fdc58001680 con 0x7fdc7006a270 2023-10-30T12:35:13.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 -- 172.21.15.12:0/2225674458 >> v1:172.21.15.12:6789/0 conn(0x7fdc7006a270 legacy=0x7fdc700c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:13.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 --2- 172.21.15.12:0/2225674458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc5c003980 0x7fdc5c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 --2- 172.21.15.12:0/2225674458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc5c0047d0 0x7fdc5c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 --2- 172.21.15.12:0/2225674458 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc7006cb80 0x7fdc5c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 -- 172.21.15.12:0/2225674458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc700c8570 con 0x7fdc5c003980 2023-10-30T12:35:13.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 -- 172.21.15.12:0/2225674458 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc700c6c10 con 0x7fdc5c0047d0 2023-10-30T12:35:13.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6dd9b640 1 -- 172.21.15.12:0/2225674458 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c00b990 con 0x7fdc7006cb80 2023-10-30T12:35:13.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6e59c640 1 --2- 172.21.15.12:0/2225674458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc5c003980 0x7fdc5c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc67fff640 1 --2- 172.21.15.12:0/2225674458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc5c0047d0 0x7fdc5c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6ed9d640 1 --2- 172.21.15.12:0/2225674458 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc7006cb80 0x7fdc5c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6e59c640 1 -- 172.21.15.12:0/2225674458 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc7006cb80 msgr2=0x7fdc5c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:13.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6e59c640 1 --2- 172.21.15.12:0/2225674458 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc7006cb80 0x7fdc5c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6e59c640 1 -- 172.21.15.12:0/2225674458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc5c0047d0 msgr2=0x7fdc5c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:13.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6e59c640 1 --2- 172.21.15.12:0/2225674458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc5c0047d0 0x7fdc5c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.282+0000 7fdc6e59c640 1 -- 172.21.15.12:0/2225674458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5c003530 con 0x7fdc5c003980 2023-10-30T12:35:13.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.286+0000 7fdc6e59c640 1 --2- 172.21.15.12:0/2225674458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc5c003980 0x7fdc5c003da0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009ae0 tx=0x7fdc6001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8237d8ffb831c978 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:13.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.286+0000 7fdc6dd9b640 1 -- 172.21.15.12:0/2225674458 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc600b0400 con 0x7fdc5c003980 2023-10-30T12:35:13.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.286+0000 7fdc6dd9b640 1 -- 172.21.15.12:0/2225674458 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc600b0560 con 0x7fdc5c003980 2023-10-30T12:35:13.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.286+0000 7fdc6dd9b640 1 -- 172.21.15.12:0/2225674458 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc600b0910 con 0x7fdc5c003980 2023-10-30T12:35:13.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.286+0000 7fdc74cea640 1 -- 172.21.15.12:0/2225674458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc5c003980 msgr2=0x7fdc5c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:13.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.286+0000 7fdc74cea640 1 --2- 172.21.15.12:0/2225674458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc5c003980 0x7fdc5c003da0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009ae0 tx=0x7fdc6001ca90 comp rx=0 tx=0).stop 2023-10-30T12:35:13.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.286+0000 7fdc6ed9d640 1 -- 172.21.15.12:0/2225674458 reap_dead start 2023-10-30T12:35:13.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.286+0000 7fdc74cea640 1 -- 172.21.15.12:0/2225674458 shutdown_connections 2023-10-30T12:35:13.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.286+0000 7fdc74cea640 1 -- 172.21.15.12:0/2225674458 >> 172.21.15.12:0/2225674458 conn(0x7fdc7006dfd0 msgr2=0x7fdc7006e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:13.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.286+0000 7fdc74cea640 1 -- 172.21.15.12:0/2225674458 shutdown_connections 2023-10-30T12:35:13.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.286+0000 7fdc74cea640 1 -- 172.21.15.12:0/2225674458 wait complete. 2023-10-30T12:35:13.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.286+0000 7fdc74cea640 1 Processor -- start 2023-10-30T12:35:13.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.286+0000 7fdc74cea640 1 -- start start 2023-10-30T12:35:13.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc74cea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc5c003980 0x7fdc701f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc74cea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc7015ade0 0x7fdc701f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc74cea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc7014ef40 0x7fdc7014b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc74cea640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7015da00 con 0x7fdc5c003980 2023-10-30T12:35:13.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc74cea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc70151ae0 con 0x7fdc7014ef40 2023-10-30T12:35:13.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc74cea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc701f1a30 con 0x7fdc7015ade0 2023-10-30T12:35:13.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc6e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc5c003980 0x7fdc701f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc6e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc5c003980 0x7fdc701f59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44206/0 (socket says 172.21.15.12:44206) 2023-10-30T12:35:13.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc6e59c640 1 -- 172.21.15.12:0/3493284606 learned_addr learned my addr 172.21.15.12:0/3493284606 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:13.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc6ed9d640 1 --2- 172.21.15.12:0/3493284606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc7014ef40 0x7fdc7014b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc67fff640 1 --2- 172.21.15.12:0/3493284606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc7015ade0 0x7fdc701f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc6e59c640 1 -- 172.21.15.12:0/3493284606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc7015ade0 msgr2=0x7fdc701f2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:13.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc6e59c640 1 --2- 172.21.15.12:0/3493284606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc7015ade0 0x7fdc701f2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc6e59c640 1 -- 172.21.15.12:0/3493284606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc7014ef40 msgr2=0x7fdc7014b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:13.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc6e59c640 1 --2- 172.21.15.12:0/3493284606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc7014ef40 0x7fdc7014b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc6e59c640 1 -- 172.21.15.12:0/3493284606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc701f5ef0 con 0x7fdc5c003980 2023-10-30T12:35:13.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc6e59c640 1 --2- 172.21.15.12:0/3493284606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc5c003980 0x7fdc701f59b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdc600981a0 tx=0x7fdc60019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:13.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc66ffd640 1 -- 172.21.15.12:0/3493284606 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000f000 con 0x7fdc5c003980 2023-10-30T12:35:13.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc66ffd640 1 -- 172.21.15.12:0/3493284606 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc600b0c30 con 0x7fdc5c003980 2023-10-30T12:35:13.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc66ffd640 1 -- 172.21.15.12:0/3493284606 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc60004030 con 0x7fdc5c003980 2023-10-30T12:35:13.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc74cea640 1 -- 172.21.15.12:0/3493284606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc701f2980 con 0x7fdc5c003980 2023-10-30T12:35:13.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc74cea640 1 -- 172.21.15.12:0/3493284606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc70151520 con 0x7fdc5c003980 2023-10-30T12:35:13.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.290+0000 7fdc74cea640 1 -- 172.21.15.12:0/3493284606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7004e170 con 0x7fdc5c003980 2023-10-30T12:35:13.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.294+0000 7fdc66ffd640 1 -- 172.21.15.12:0/3493284606 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdc60004190 con 0x7fdc5c003980 2023-10-30T12:35:13.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.294+0000 7fdc66ffd640 1 --2- 172.21.15.12:0/3493284606 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc500664d0 0x7fdc50068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.294+0000 7fdc66ffd640 1 -- 172.21.15.12:0/3493284606 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc6011e0a0 con 0x7fdc5c003980 2023-10-30T12:35:13.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.294+0000 7fdc67fff640 1 --2- 172.21.15.12:0/3493284606 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc500664d0 0x7fdc50068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.294+0000 7fdc67fff640 1 --2- 172.21.15.12:0/3493284606 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc500664d0 0x7fdc50068990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdc70157c40 tx=0x7fdc58009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:13.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.298+0000 7fdc66ffd640 1 -- 172.21.15.12:0/3493284606 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdc600eb3f0 con 0x7fdc5c003980 2023-10-30T12:35:13.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.458+0000 7fdc74cea640 1 -- 172.21.15.12:0/3493284606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc701f5ef0 con 0x7fdc5c003980 2023-10-30T12:35:13.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.458+0000 7fdc66ffd640 1 -- 172.21.15.12:0/3493284606 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16505 (secure 0 0 0) 0x7fdc701f5ef0 con 0x7fdc5c003980 2023-10-30T12:35:13.462 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:13.462 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":22,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:35:07.153665+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:35:03.216053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:35:04.597162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:35:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.458+0000 7fdc74cea640 1 -- 172.21.15.12:0/3493284606 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc500664d0 msgr2=0x7fdc50068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.458+0000 7fdc74cea640 1 --2- 172.21.15.12:0/3493284606 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc500664d0 0x7fdc50068990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdc70157c40 tx=0x7fdc58009040 comp rx=0 tx=0).stop 2023-10-30T12:35:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.458+0000 7fdc74cea640 1 -- 172.21.15.12:0/3493284606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc5c003980 msgr2=0x7fdc701f59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.458+0000 7fdc74cea640 1 --2- 172.21.15.12:0/3493284606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc5c003980 0x7fdc701f59b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdc600981a0 tx=0x7fdc60019830 comp rx=0 tx=0).stop 2023-10-30T12:35:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.458+0000 7fdc74cea640 1 -- 172.21.15.12:0/3493284606 shutdown_connections 2023-10-30T12:35:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.458+0000 7fdc74cea640 1 --2- 172.21.15.12:0/3493284606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc7014ef40 0x7fdc7014b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.458+0000 7fdc74cea640 1 --2- 172.21.15.12:0/3493284606 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc500664d0 0x7fdc50068990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.458+0000 7fdc74cea640 1 --2- 172.21.15.12:0/3493284606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc7015ade0 0x7fdc701f2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.458+0000 7fdc74cea640 1 --2- 172.21.15.12:0/3493284606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc5c003980 0x7fdc701f59b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.458+0000 7fdc74cea640 1 -- 172.21.15.12:0/3493284606 >> 172.21.15.12:0/3493284606 conn(0x7fdc7006dfd0 msgr2=0x7fdc7006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:13.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.458+0000 7fdc74cea640 1 -- 172.21.15.12:0/3493284606 shutdown_connections 2023-10-30T12:35:13.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.458+0000 7fdc74cea640 1 -- 172.21.15.12:0/3493284606 wait complete. 2023-10-30T12:35:13.483 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-30T12:35:13.484 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:35:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.622+0000 7f5bc8b1d640 1 Processor -- start 2023-10-30T12:35:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.622+0000 7f5bc8b1d640 1 -- start start 2023-10-30T12:35:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bc8b1d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5bc40c6b80 0x7f5bc40bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bc8b1d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5bc40c6f60 0x7f5bc40c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bc8b1d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bc40c8430 0x7f5bc413ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bc8b1d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc40c0a50 con 0x7f5bc40c6f60 2023-10-30T12:35:13.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bc8b1d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc406f6e0 con 0x7f5bc40c6b80 2023-10-30T12:35:13.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bc8b1d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc406f9c0 con 0x7f5bc40c8430 2023-10-30T12:35:13.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bbbfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5bc40c6f60 0x7f5bc40c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bbbfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5bc40c6f60 0x7f5bc40c0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44222/0 (socket says 172.21.15.12:44222) 2023-10-30T12:35:13.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bbbfff640 1 -- 172.21.15.12:0/1438374062 learned_addr learned my addr 172.21.15.12:0/1438374062 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:13.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bc2d9d640 1 --2- 172.21.15.12:0/1438374062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bc40c8430 0x7f5bc413ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bc259c640 1 --2- 172.21.15.12:0/1438374062 >> v2:172.21.15.154:3300/0 conn(0x7f5bc40c6b80 0x7f5bc40bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bbbfff640 1 -- 172.21.15.12:0/1438374062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bc40c8430 msgr2=0x7f5bc413ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bbbfff640 1 --2- 172.21.15.12:0/1438374062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bc40c8430 0x7f5bc413ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bbbfff640 1 -- 172.21.15.12:0/1438374062 >> v2:172.21.15.154:3300/0 conn(0x7f5bc40c6b80 msgr2=0x7f5bc40bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:13.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bbbfff640 1 --2- 172.21.15.12:0/1438374062 >> v2:172.21.15.154:3300/0 conn(0x7f5bc40c6b80 0x7f5bc40bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bbbfff640 1 -- 172.21.15.12:0/1438374062 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc406fca0 con 0x7f5bc40c6f60 2023-10-30T12:35:13.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.626+0000 7f5bbbfff640 1 --2- 172.21.15.12:0/1438374062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bc40c6f60 0x7f5bc40c0400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5bc406f1b0 tx=0x7f5bac01e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7df642e56da3f46 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:13.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc1d9b640 1 -- 172.21.15.12:0/1438374062 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0aa610 con 0x7f5bc40c6f60 2023-10-30T12:35:13.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc1d9b640 1 -- 172.21.15.12:0/1438374062 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bac0aa770 con 0x7f5bc40c6f60 2023-10-30T12:35:13.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc1d9b640 1 -- 172.21.15.12:0/1438374062 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0aab00 con 0x7f5bc40c6f60 2023-10-30T12:35:13.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 -- 172.21.15.12:0/1438374062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bc40c6f60 msgr2=0x7f5bc40c0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:13.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 --2- 172.21.15.12:0/1438374062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bc40c6f60 0x7f5bc40c0400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5bc406f1b0 tx=0x7f5bac01e710 comp rx=0 tx=0).stop 2023-10-30T12:35:13.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 -- 172.21.15.12:0/1438374062 shutdown_connections 2023-10-30T12:35:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 --2- 172.21.15.12:0/1438374062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bc40c8430 0x7f5bc413ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 --2- 172.21.15.12:0/1438374062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bc40c6f60 0x7f5bc40c0400 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 --2- 172.21.15.12:0/1438374062 >> v2:172.21.15.154:3300/0 conn(0x7f5bc40c6b80 0x7f5bc40bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 -- 172.21.15.12:0/1438374062 >> 172.21.15.12:0/1438374062 conn(0x7f5bc40c7910 msgr2=0x7f5bc40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 -- 172.21.15.12:0/1438374062 shutdown_connections 2023-10-30T12:35:13.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 -- 172.21.15.12:0/1438374062 wait complete. 2023-10-30T12:35:13.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 Processor -- start 2023-10-30T12:35:13.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 -- start start 2023-10-30T12:35:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bc40c6b80 0x7f5bc40ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bc40c6f60 0x7f5bc40c2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bc40c8430 0x7f5bc40c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc40c4390 con 0x7f5bc40c6b80 2023-10-30T12:35:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc40c6170 con 0x7f5bc40c6f60 2023-10-30T12:35:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc8b1d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc4157790 con 0x7f5bc40c8430 2023-10-30T12:35:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bc40c6b80 0x7f5bc40ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bc40c6b80 0x7f5bc40ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44226/0 (socket says 172.21.15.12:44226) 2023-10-30T12:35:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.630+0000 7f5bc259c640 1 -- 172.21.15.12:0/569623143 learned_addr learned my addr 172.21.15.12:0/569623143 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bbbfff640 1 --2- 172.21.15.12:0/569623143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bc40c6f60 0x7f5bc40c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bc2d9d640 1 --2- 172.21.15.12:0/569623143 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bc40c8430 0x7f5bc40c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bc259c640 1 -- 172.21.15.12:0/569623143 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bc40c8430 msgr2=0x7f5bc40c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bc259c640 1 --2- 172.21.15.12:0/569623143 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bc40c8430 0x7f5bc40c2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bc259c640 1 -- 172.21.15.12:0/569623143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bc40c6f60 msgr2=0x7f5bc40c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bc259c640 1 --2- 172.21.15.12:0/569623143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bc40c6f60 0x7f5bc40c2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bc259c640 1 -- 172.21.15.12:0/569623143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac0094d0 con 0x7f5bc40c6b80 2023-10-30T12:35:13.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bc259c640 1 --2- 172.21.15.12:0/569623143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bc40c6b80 0x7f5bc40ca590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400d800 tx=0x7f5bb400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:13.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bbaffd640 1 -- 172.21.15.12:0/569623143 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb40096a0 con 0x7f5bc40c6b80 2023-10-30T12:35:13.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bbaffd640 1 -- 172.21.15.12:0/569623143 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb4009800 con 0x7f5bc40c6b80 2023-10-30T12:35:13.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bc8b1d640 1 -- 172.21.15.12:0/569623143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc40c5280 con 0x7f5bc40c6b80 2023-10-30T12:35:13.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bbaffd640 1 -- 172.21.15.12:0/569623143 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4009b70 con 0x7f5bc40c6b80 2023-10-30T12:35:13.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bc8b1d640 1 -- 172.21.15.12:0/569623143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc40c5840 con 0x7f5bc40c6b80 2023-10-30T12:35:13.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bc8b1d640 1 -- 172.21.15.12:0/569623143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc404e170 con 0x7f5bc40c6b80 2023-10-30T12:35:13.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bbaffd640 1 -- 172.21.15.12:0/569623143 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5bb4009cd0 con 0x7f5bc40c6b80 2023-10-30T12:35:13.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.634+0000 7f5bbaffd640 1 --2- 172.21.15.12:0/569623143 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ba40664a0 0x7f5ba4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:13.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.638+0000 7f5bbaffd640 1 -- 172.21.15.12:0/569623143 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bb40845d0 con 0x7f5bc40c6b80 2023-10-30T12:35:13.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.638+0000 7f5bbbfff640 1 --2- 172.21.15.12:0/569623143 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ba40664a0 0x7f5ba4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:13.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.638+0000 7f5bbbfff640 1 --2- 172.21.15.12:0/569623143 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ba40664a0 0x7f5ba4068960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0b1000 tx=0x7f5bac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:13.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.638+0000 7f5bbaffd640 1 -- 172.21.15.12:0/569623143 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5bb40519d0 con 0x7f5bc40c6b80 2023-10-30T12:35:13.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.802+0000 7f5bc8b1d640 1 -- 172.21.15.12:0/569623143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bc40c0dd0 con 0x7f5bc40c6b80 2023-10-30T12:35:13.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.802+0000 7f5bbaffd640 1 -- 172.21.15.12:0/569623143 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f5bb4054ff0 con 0x7f5bc40c6b80 2023-10-30T12:35:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:13.806 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":22,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:35:07.153665+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:35:03.216053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:35:04.597162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:35:13.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.802+0000 7f5bc8b1d640 1 -- 172.21.15.12:0/569623143 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ba40664a0 msgr2=0x7f5ba4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:13.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.802+0000 7f5bc8b1d640 1 --2- 172.21.15.12:0/569623143 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ba40664a0 0x7f5ba4068960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0b1000 tx=0x7f5bac008040 comp rx=0 tx=0).stop 2023-10-30T12:35:13.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.802+0000 7f5bc8b1d640 1 -- 172.21.15.12:0/569623143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bc40c6b80 msgr2=0x7f5bc40ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:13.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.802+0000 7f5bc8b1d640 1 --2- 172.21.15.12:0/569623143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bc40c6b80 0x7f5bc40ca590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400d800 tx=0x7f5bb400dcc0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.802+0000 7f5bc8b1d640 1 -- 172.21.15.12:0/569623143 shutdown_connections 2023-10-30T12:35:13.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.802+0000 7f5bc8b1d640 1 --2- 172.21.15.12:0/569623143 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ba40664a0 0x7f5ba4068960 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.802+0000 7f5bc8b1d640 1 --2- 172.21.15.12:0/569623143 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bc40c8430 0x7f5bc40c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.802+0000 7f5bc8b1d640 1 --2- 172.21.15.12:0/569623143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bc40c6f60 0x7f5bc40c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.802+0000 7f5bc8b1d640 1 --2- 172.21.15.12:0/569623143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bc40c6b80 0x7f5bc40ca590 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:13.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.802+0000 7f5bc8b1d640 1 -- 172.21.15.12:0/569623143 >> 172.21.15.12:0/569623143 conn(0x7f5bc40c7910 msgr2=0x7f5bc406a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:13.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.802+0000 7f5bc8b1d640 1 -- 172.21.15.12:0/569623143 shutdown_connections 2023-10-30T12:35:13.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:13.802+0000 7f5bc8b1d640 1 -- 172.21.15.12:0/569623143 wait complete. 2023-10-30T12:35:13.828 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-30T12:35:13.828 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-30T12:35:13.829 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-30T12:35:13.829 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-30T12:35:13.830 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-30T12:35:13.831 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-30T12:35:13.831 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-30T12:35:13.832 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-30T12:35:14.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.070+0000 7f7cd9f3e640 1 Processor -- start 2023-10-30T12:35:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.074+0000 7f7cd9f3e640 1 -- start start 2023-10-30T12:35:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.074+0000 7f7cd9f3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd4070840 0x7f7cd4070c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.074+0000 7f7cd9f3e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7cd4069e90 0x7f7cd406a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.074+0000 7f7cd9f3e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cd40c6880 con 0x7f7cd4153810 2023-10-30T12:35:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.074+0000 7f7cd9f3e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cd40711c0 con 0x7f7cd4069e90 2023-10-30T12:35:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.074+0000 7f7cd9f3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd40c5990 con 0x7f7cd4070840 2023-10-30T12:35:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.074+0000 7f7cd37fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7cd4153810 0x7f7cd415a2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:35996/0 (socket says 172.21.15.12:35996) 2023-10-30T12:35:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.074+0000 7f7cd37fe640 1 -- 172.21.15.12:0/3075739422 learned_addr learned my addr 172.21.15.12:0/3075739422 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.074+0000 7f7ccbfff640 1 --2- 172.21.15.12:0/3075739422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd4070840 0x7f7cd4070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.074+0000 7f7cd3fff640 1 --2- 172.21.15.12:0/3075739422 >> v2:172.21.15.154:3300/0 conn(0x7f7cd4069e90 0x7f7cd406a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.074+0000 7f7ccbfff640 1 -- 172.21.15.12:0/3075739422 >> v2:172.21.15.154:3300/0 conn(0x7f7cd4069e90 msgr2=0x7f7cd406a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.074+0000 7f7ccbfff640 1 --2- 172.21.15.12:0/3075739422 >> v2:172.21.15.154:3300/0 conn(0x7f7cd4069e90 0x7f7cd406a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.074+0000 7f7cd2ffd640 1 -- 172.21.15.12:0/3075739422 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697997662 0 0) 0x7f7cd40c6880 con 0x7f7cd4153810 2023-10-30T12:35:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.078+0000 7f7ccbfff640 1 -- 172.21.15.12:0/3075739422 >> v1:172.21.15.154:6789/0 conn(0x7f7cd4153810 legacy=0x7f7cd415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.078+0000 7f7ccbfff640 1 -- 172.21.15.12:0/3075739422 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd40c5c70 con 0x7f7cd4070840 2023-10-30T12:35:14.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.078+0000 7fd990794640 1 Processor -- start 2023-10-30T12:35:14.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.078+0000 7fd990794640 1 -- start start 2023-10-30T12:35:14.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.078+0000 7f82ca772640 1 Processor -- start 2023-10-30T12:35:14.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.078+0000 7f82ca772640 1 -- start start 2023-10-30T12:35:14.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7fd990794640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd9880c77b0 0x7fd98815e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7fd990794640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9880c52a0 0x7fd988071f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7fd990794640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9880c6060 con 0x7fd9880c69c0 2023-10-30T12:35:14.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7fd990794640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9880c5e80 con 0x7fd9880c77b0 2023-10-30T12:35:14.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7fd990794640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd988161380 con 0x7fd9880c52a0 2023-10-30T12:35:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f7ccbfff640 1 --2- 172.21.15.12:0/3075739422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd4070840 0x7f7cd4070c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0097c0 tx=0x7f7cbc01ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=631b2f298eff8c66 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7fd98ed31640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9880c52a0 0x7fd988071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7fd98e530640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd9880c77b0 0x7fd98815e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7fd98e530640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd9880c77b0 0x7fd98815e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44228/0 (socket says 172.21.15.12:44228) 2023-10-30T12:35:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f7cd2ffd640 1 -- 172.21.15.12:0/3075739422 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc0a8970 con 0x7f7cd4070840 2023-10-30T12:35:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f7cd2ffd640 1 -- 172.21.15.12:0/3075739422 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cbc0a8ad0 con 0x7f7cd4070840 2023-10-30T12:35:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f7cd2ffd640 1 -- 172.21.15.12:0/3075739422 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc0a8e60 con 0x7f7cd4070840 2023-10-30T12:35:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7fd98e530640 1 -- 172.21.15.12:0/416341146 learned_addr learned my addr 172.21.15.12:0/416341146 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7fd98ed31640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9880c52a0 0x7fd988071f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58008/0 (socket says 172.21.15.12:58008) 2023-10-30T12:35:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7fd98e530640 1 -- 172.21.15.12:0/416341146 >> v2:172.21.15.154:3300/0 conn(0x7fd9880c52a0 msgr2=0x7fd988071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7fd98e530640 1 --2- 172.21.15.12:0/416341146 >> v2:172.21.15.154:3300/0 conn(0x7fd9880c52a0 0x7fd988071f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7fd98e530640 1 -- 172.21.15.12:0/416341146 >> v1:172.21.15.154:6789/0 conn(0x7fd9880c69c0 legacy=0x7fd988161d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-30T12:35:14.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7fd98e530640 1 -- 172.21.15.12:0/416341146 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9880c6340 con 0x7fd9880c77b0 2023-10-30T12:35:14.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/3075739422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd4070840 msgr2=0x7f7cd4070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f7cd9f3e640 1 --2- 172.21.15.12:0/3075739422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd4070840 0x7f7cd4070c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0097c0 tx=0x7f7cbc01ec70 comp rx=0 tx=0).stop 2023-10-30T12:35:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/3075739422 shutdown_connections 2023-10-30T12:35:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f7cd9f3e640 1 --2- 172.21.15.12:0/3075739422 >> v2:172.21.15.154:3300/0 conn(0x7f7cd4069e90 0x7f7cd406a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f7cd9f3e640 1 --2- 172.21.15.12:0/3075739422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd4070840 0x7f7cd4070c80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/3075739422 >> 172.21.15.12:0/3075739422 conn(0x7f7cd40c7910 msgr2=0x7f7cd40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f82ca772640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82c4153810 0x7f82c415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f82ca772640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f82c4070840 0x7f82c4070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f82ca772640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c40c6880 con 0x7f82c4069e90 2023-10-30T12:35:14.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f82ca772640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c415a820 con 0x7f82c4070840 2023-10-30T12:35:14.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f82ca772640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c40711f0 con 0x7f82c4153810 2023-10-30T12:35:14.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f82c3fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82c4153810 0x7f82c415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f82c37fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f82c4070840 0x7f82c4070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f82c37fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f82c4070840 0x7f82c4070cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44230/0 (socket says 172.21.15.12:44230) 2023-10-30T12:35:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f82c37fe640 1 -- 172.21.15.12:0/295479914 learned_addr learned my addr 172.21.15.12:0/295479914 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f82c37fe640 1 -- 172.21.15.12:0/295479914 >> v2:172.21.15.154:3300/0 conn(0x7f82c4153810 msgr2=0x7f82c415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f82c37fe640 1 --2- 172.21.15.12:0/295479914 >> v2:172.21.15.154:3300/0 conn(0x7f82c4153810 0x7f82c415a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f82c37fe640 1 -- 172.21.15.12:0/295479914 >> v1:172.21.15.154:6789/0 conn(0x7f82c4069e90 legacy=0x7f82c406a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.082+0000 7f82c37fe640 1 -- 172.21.15.12:0/295479914 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c40c5990 con 0x7f82c4070840 2023-10-30T12:35:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.086+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/3075739422 shutdown_connections 2023-10-30T12:35:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.086+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/3075739422 wait complete. 2023-10-30T12:35:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.086+0000 7f7cd9f3e640 1 Processor -- start 2023-10-30T12:35:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.086+0000 7f7cd9f3e640 1 -- start start 2023-10-30T12:35:14.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.086+0000 7fd98e530640 1 --2- 172.21.15.12:0/416341146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9880c77b0 0x7fd98815e860 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd9880c8fe0 tx=0x7fd97801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51dae4332c0d6a4d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7fd98d52e640 1 -- 172.21.15.12:0/416341146 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9780aa640 con 0x7fd9880c77b0 2023-10-30T12:35:14.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7fd98d52e640 1 -- 172.21.15.12:0/416341146 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9780aa7a0 con 0x7fd9880c77b0 2023-10-30T12:35:14.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7fd98d52e640 1 -- 172.21.15.12:0/416341146 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9780aab30 con 0x7fd9880c77b0 2023-10-30T12:35:14.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7cd9f3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd4069e90 0x7f7cd41f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f82c37fe640 1 --2- 172.21.15.12:0/295479914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c4070840 0x7f82c4070cb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f82c4072640 tx=0x7f82b801ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed13b9139f6c3f74 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7cd9f3e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cd4070840 0x7f7cd4157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7fd990794640 1 -- 172.21.15.12:0/416341146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9880c77b0 msgr2=0x7fd98815e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7fd990794640 1 --2- 172.21.15.12:0/416341146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9880c77b0 0x7fd98815e860 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd9880c8fe0 tx=0x7fd97801c4e0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7fd990794640 1 -- 172.21.15.12:0/416341146 shutdown_connections 2023-10-30T12:35:14.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7fd990794640 1 --2- 172.21.15.12:0/416341146 >> v2:172.21.15.154:3300/0 conn(0x7fd9880c52a0 0x7fd988071f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7fd990794640 1 --2- 172.21.15.12:0/416341146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9880c77b0 0x7fd98815e860 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f82c2ffd640 1 -- 172.21.15.12:0/295479914 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80a89b0 con 0x7f82c4070840 2023-10-30T12:35:14.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7fd990794640 1 -- 172.21.15.12:0/416341146 >> 172.21.15.12:0/416341146 conn(0x7fd9880cb130 msgr2=0x7fd9880cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f82c2ffd640 1 -- 172.21.15.12:0/295479914 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b80a8b10 con 0x7f82c4070840 2023-10-30T12:35:14.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7cd9f3e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cd4153810 0x7f7cd4157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f8f48d65640 1 Processor -- start 2023-10-30T12:35:14.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7cd9f3e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd414d8c0 con 0x7f7cd4070840 2023-10-30T12:35:14.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7cd9f3e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd41fbc20 con 0x7f7cd4153810 2023-10-30T12:35:14.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7cd9f3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd41fbe70 con 0x7f7cd4069e90 2023-10-30T12:35:14.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7cd37fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd4069e90 0x7f7cd41f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7ccbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cd4070840 0x7f7cd4157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7cd37fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd4069e90 0x7f7cd41f5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40442/0 (socket says 172.21.15.12:40442) 2023-10-30T12:35:14.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f8f48d65640 1 -- start start 2023-10-30T12:35:14.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7cd37fe640 1 -- 172.21.15.12:0/4291107248 learned_addr learned my addr 172.21.15.12:0/4291107248 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7fd990794640 1 -- 172.21.15.12:0/416341146 shutdown_connections 2023-10-30T12:35:14.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7cd3fff640 1 --2- 172.21.15.12:0/4291107248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cd4153810 0x7f7cd4157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7fd990794640 1 -- 172.21.15.12:0/416341146 wait complete. 2023-10-30T12:35:14.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f82c2ffd640 1 -- 172.21.15.12:0/295479914 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80a89b0 con 0x7f82c4070840 2023-10-30T12:35:14.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7cd37fe640 1 -- 172.21.15.12:0/4291107248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cd4153810 msgr2=0x7f7cd4157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7cd37fe640 1 --2- 172.21.15.12:0/4291107248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cd4153810 0x7f7cd4157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7cd37fe640 1 -- 172.21.15.12:0/4291107248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cd4070840 msgr2=0x7f7cd4157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7cd37fe640 1 --2- 172.21.15.12:0/4291107248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cd4070840 0x7f7cd4157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.090+0000 7f7cd37fe640 1 -- 172.21.15.12:0/4291107248 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc009420 con 0x7f7cd4069e90 2023-10-30T12:35:14.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f82ca772640 1 -- 172.21.15.12:0/295479914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c4070840 msgr2=0x7f82c4070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f82ca772640 1 --2- 172.21.15.12:0/295479914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c4070840 0x7f82c4070cb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f82c4072640 tx=0x7f82b801ecb0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7fbddbd65640 1 Processor -- start 2023-10-30T12:35:14.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f7cd37fe640 1 --2- 172.21.15.12:0/4291107248 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd4069e90 0x7f7cd41f5aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400c4d0 tx=0x7f7cc400c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f7cd0ff9640 1 -- 172.21.15.12:0/4291107248 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc40075a0 con 0x7f7cd4069e90 2023-10-30T12:35:14.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f7cd0ff9640 1 -- 172.21.15.12:0/4291107248 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc40047f0 con 0x7f7cd4069e90 2023-10-30T12:35:14.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f7cd0ff9640 1 -- 172.21.15.12:0/4291107248 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4004ba0 con 0x7f7cd4069e90 2023-10-30T12:35:14.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f1062be4640 1 Processor -- start 2023-10-30T12:35:14.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7fbddbd65640 1 -- start start 2023-10-30T12:35:14.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/4291107248 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd41fc150 con 0x7f7cd4069e90 2023-10-30T12:35:14.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/4291107248 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd41fc710 con 0x7f7cd4069e90 2023-10-30T12:35:14.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f8f48d65640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44153810 0x7f8f4415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f8f48d65640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8f44070840 0x7f8f44070c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f1062be4640 1 -- start start 2023-10-30T12:35:14.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f8f48d65640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f440c6880 con 0x7f8f44069e90 2023-10-30T12:35:14.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f8f48d65640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f4415a820 con 0x7f8f44070840 2023-10-30T12:35:14.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f8f48d65640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440711c0 con 0x7f8f44153810 2023-10-30T12:35:14.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f8f4259c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44153810 0x7f8f4415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f8f4259c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44153810 0x7f8f4415a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40458/0 (socket says 172.21.15.12:40458) 2023-10-30T12:35:14.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f8f4259c640 1 -- 172.21.15.12:0/978451629 learned_addr learned my addr 172.21.15.12:0/978451629 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f82ca772640 1 -- 172.21.15.12:0/295479914 shutdown_connections 2023-10-30T12:35:14.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f82ca772640 1 --2- 172.21.15.12:0/295479914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c4070840 0x7f82c4070cb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f82ca772640 1 --2- 172.21.15.12:0/295479914 >> v2:172.21.15.154:3300/0 conn(0x7f82c4153810 0x7f82c415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f82ca772640 1 -- 172.21.15.12:0/295479914 >> 172.21.15.12:0/295479914 conn(0x7f82c40c7910 msgr2=0x7f82c40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f82ca772640 1 -- 172.21.15.12:0/295479914 shutdown_connections 2023-10-30T12:35:14.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f82ca772640 1 -- 172.21.15.12:0/295479914 wait complete. 2023-10-30T12:35:14.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f8f4259c640 1 -- 172.21.15.12:0/978451629 >> v2:172.21.15.154:3300/0 conn(0x7f8f44070840 msgr2=0x7f8f44070c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-30T12:35:14.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f8f4259c640 1 --2- 172.21.15.12:0/978451629 >> v2:172.21.15.154:3300/0 conn(0x7f8f44070840 0x7f8f44070c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f8f4259c640 1 -- 172.21.15.12:0/978451629 >> v1:172.21.15.12:6789/0 conn(0x7f8f44069e90 legacy=0x7f8f4406a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f8f4259c640 1 -- 172.21.15.12:0/978451629 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f440c5990 con 0x7f8f44153810 2023-10-30T12:35:14.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7fd990794640 1 Processor -- start 2023-10-30T12:35:14.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7fd990794640 1 -- start start 2023-10-30T12:35:14.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7fbddbd65640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd4158090 0x7fbdd4161be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f1062be4640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f105c070200 0x7f105c06ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7fbddbd65640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd40c72b0 con 0x7fbdd40c6b40 2023-10-30T12:35:14.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7fbddbd65640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd406acb0 con 0x7fbdd40c0ed0 2023-10-30T12:35:14.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7fbddbd65640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd4162120 con 0x7fbdd4158090 2023-10-30T12:35:14.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.094+0000 7f1062be4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f105c0c4030 0x7f105c06f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/4291107248 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7cd40c6230 con 0x7f7cd4069e90 2023-10-30T12:35:14.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f7cd0ff9640 1 -- 172.21.15.12:0/4291107248 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7cc4004d00 con 0x7f7cd4069e90 2023-10-30T12:35:14.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f1062be4640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105c156310 con 0x7f105c06f880 2023-10-30T12:35:14.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f1062be4640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105c06fdb0 con 0x7f105c070200 2023-10-30T12:35:14.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f1062be4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105c155720 con 0x7f105c0c4030 2023-10-30T12:35:14.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fd990794640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9880c52a0 0x7fd98814ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f1061be2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f105c070200 0x7f105c06ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f82ca772640 1 Processor -- start 2023-10-30T12:35:14.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f1061be2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f105c070200 0x7f105c06ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44256/0 (socket says 172.21.15.12:44256) 2023-10-30T12:35:14.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f1061be2640 1 -- 172.21.15.12:0/463151058 learned_addr learned my addr 172.21.15.12:0/463151058 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f82ca772640 1 -- start start 2023-10-30T12:35:14.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f1061be2640 1 -- 172.21.15.12:0/463151058 >> v2:172.21.15.154:3300/0 conn(0x7f105c0c4030 msgr2=0x7f105c06f340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-30T12:35:14.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f1061be2640 1 --2- 172.21.15.12:0/463151058 >> v2:172.21.15.154:3300/0 conn(0x7f105c0c4030 0x7f105c06f340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fbdd9b01640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd4158090 0x7fbdd4161be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fbdd9300640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbdd40c6b40 0x7fbdd4069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:55380/0 (socket says 172.21.15.12:55380) 2023-10-30T12:35:14.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fbdd9300640 1 -- 172.21.15.12:0/3969647935 learned_addr learned my addr 172.21.15.12:0/3969647935 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:14.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f1061be2640 1 -- 172.21.15.12:0/463151058 >> v1:172.21.15.12:6789/0 conn(0x7f105c06f880 legacy=0x7f105c0c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f1061be2640 1 -- 172.21.15.12:0/463151058 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c1559e0 con 0x7f105c070200 2023-10-30T12:35:14.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f7cd0ff9640 1 --2- 172.21.15.12:0/4291107248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cb40664a0 0x7f7cb4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f7cd0ff9640 1 -- 172.21.15.12:0/4291107248 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cc40542b0 con 0x7f7cd4069e90 2023-10-30T12:35:14.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f7cd0ff9640 1 --2- 172.21.15.12:0/4291107248 >> [v2:172.21.15.12:6816/2335476277,v1:172.21.15.12:6817/2335476277] conn(0x7f7cb406f250 0x7f7cb40716f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f7cd0ff9640 1 -- 172.21.15.12:0/4291107248 --> [v2:172.21.15.12:6816/2335476277,v1:172.21.15.12:6817/2335476277] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7cb4071dc0 con 0x7f7cb406f250 2023-10-30T12:35:14.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f8f4259c640 1 --2- 172.21.15.12:0/978451629 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44153810 0x7f8f4415a2e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009870 tx=0x7f8f3801ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40190326d3d8e4f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fbdd9b01640 1 -- 172.21.15.12:0/3969647935 >> v1:172.21.15.154:6789/0 conn(0x7fbdd40c0ed0 legacy=0x7fbdd406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fbdd9b01640 1 -- 172.21.15.12:0/3969647935 >> v1:172.21.15.12:6789/0 conn(0x7fbdd40c6b40 legacy=0x7fbdd4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fbdd9b01640 1 -- 172.21.15.12:0/3969647935 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd40c6fd0 con 0x7fbdd4158090 2023-10-30T12:35:14.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fbdd8aff640 1 -- 172.21.15.12:0/3969647935 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561441206 0 0) 0x7fbdd406acb0 con 0x7fbdd40c0ed0 2023-10-30T12:35:14.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f7ccbfff640 1 --2- 172.21.15.12:0/4291107248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cb40664a0 0x7f7cb4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fd990794640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9880c69c0 0x7fd98814bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fd990794640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9880c77b0 0x7fd98815d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fd990794640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9881f05b0 con 0x7fd9880c77b0 2023-10-30T12:35:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fd990794640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98815bc30 con 0x7fd9880c69c0 2023-10-30T12:35:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fd990794640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9881f64c0 con 0x7fd9880c52a0 2023-10-30T12:35:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fd98e530640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9880c52a0 0x7fd98814ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fd98e530640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9880c52a0 0x7fd98814ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40476/0 (socket says 172.21.15.12:40476) 2023-10-30T12:35:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fd98e530640 1 -- 172.21.15.12:0/4249511905 learned_addr learned my addr 172.21.15.12:0/4249511905 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fd98dd2f640 1 --2- 172.21.15.12:0/4249511905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9880c69c0 0x7fd98814bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7fd98ed31640 1 --2- 172.21.15.12:0/4249511905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9880c77b0 0x7fd98815d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f82ca772640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c4069e90 0x7f82c41e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f82ca772640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4070840 0x7f82c40c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.098+0000 7f7cd0ff9640 1 -- 172.21.15.12:0/4291107248 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cc4054080 con 0x7f7cd4069e90 2023-10-30T12:35:14.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f7cd3fff640 1 --2- 172.21.15.12:0/4291107248 >> [v2:172.21.15.12:6816/2335476277,v1:172.21.15.12:6817/2335476277] conn(0x7f7cb406f250 0x7f7cb40716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f8f4159a640 1 -- 172.21.15.12:0/978451629 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f380a89d0 con 0x7f8f44153810 2023-10-30T12:35:14.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f8f4159a640 1 -- 172.21.15.12:0/978451629 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f380a8b30 con 0x7f8f44153810 2023-10-30T12:35:14.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f8f4159a640 1 -- 172.21.15.12:0/978451629 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f380b0440 con 0x7f8f44153810 2023-10-30T12:35:14.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7fd98e530640 1 -- 172.21.15.12:0/4249511905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9880c69c0 msgr2=0x7fd98814bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7fd98e530640 1 --2- 172.21.15.12:0/4249511905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9880c69c0 0x7fd98814bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7fd98e530640 1 -- 172.21.15.12:0/4249511905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9880c77b0 msgr2=0x7fd98815d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7fd98e530640 1 --2- 172.21.15.12:0/4249511905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9880c77b0 0x7fd98815d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82ca772640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c4153810 0x7f82c414cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82c3fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c4069e90 0x7f82c41e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82ca772640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c41e5510 con 0x7f82c4153810 2023-10-30T12:35:14.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82ca772640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c41e5140 con 0x7f82c4070840 2023-10-30T12:35:14.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7fd98e530640 1 -- 172.21.15.12:0/4249511905 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9780094d0 con 0x7fd9880c52a0 2023-10-30T12:35:14.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82ca772640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c41e5760 con 0x7f82c4069e90 2023-10-30T12:35:14.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82c3fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c4069e90 0x7f82c41e0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40482/0 (socket says 172.21.15.12:40482) 2023-10-30T12:35:14.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82c3fff640 1 -- 172.21.15.12:0/3955888431 learned_addr learned my addr 172.21.15.12:0/3955888431 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82c37fe640 1 --2- 172.21.15.12:0/3955888431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4070840 0x7f82c40c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82c8d0f640 1 --2- 172.21.15.12:0/3955888431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c4153810 0x7f82c414cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7fd98e530640 1 --2- 172.21.15.12:0/4249511905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9880c52a0 0x7fd98814ba30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd97801ab90 tx=0x7fd97801ad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82c3fff640 1 -- 172.21.15.12:0/3955888431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4070840 msgr2=0x7f82c40c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82c3fff640 1 --2- 172.21.15.12:0/3955888431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4070840 0x7f82c40c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82c3fff640 1 -- 172.21.15.12:0/3955888431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c4153810 msgr2=0x7f82c414cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82c3fff640 1 --2- 172.21.15.12:0/3955888431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c4153810 0x7f82c414cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f8f48d65640 1 -- 172.21.15.12:0/978451629 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44153810 msgr2=0x7f8f4415a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f8f48d65640 1 --2- 172.21.15.12:0/978451629 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44153810 0x7f8f4415a2e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009870 tx=0x7f8f3801ed10 comp rx=0 tx=0).stop 2023-10-30T12:35:14.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82c3fff640 1 -- 172.21.15.12:0/3955888431 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b80094d0 con 0x7f82c4069e90 2023-10-30T12:35:14.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f7ccbfff640 1 --2- 172.21.15.12:0/4291107248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cb40664a0 0x7f7cb4068960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0b2000 tx=0x7f7cbc0aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f8f48d65640 1 -- 172.21.15.12:0/978451629 shutdown_connections 2023-10-30T12:35:14.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f8f48d65640 1 --2- 172.21.15.12:0/978451629 >> v2:172.21.15.154:3300/0 conn(0x7f8f44070840 0x7f8f44070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f8f48d65640 1 --2- 172.21.15.12:0/978451629 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44153810 0x7f8f4415a2e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f8f48d65640 1 -- 172.21.15.12:0/978451629 >> 172.21.15.12:0/978451629 conn(0x7f8f440c7910 msgr2=0x7f8f440c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82c3fff640 1 --2- 172.21.15.12:0/3955888431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c4069e90 0x7f82c41e0ce0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00ccd0 tx=0x7f82bc00a5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7fd97effd640 1 -- 172.21.15.12:0/4249511905 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9780bd5f0 con 0x7fd9880c52a0 2023-10-30T12:35:14.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f8f48d65640 1 -- 172.21.15.12:0/978451629 shutdown_connections 2023-10-30T12:35:14.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f8f48d65640 1 -- 172.21.15.12:0/978451629 wait complete. 2023-10-30T12:35:14.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82c0ff9640 1 -- 172.21.15.12:0/3955888431 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc0039a0 con 0x7f82c4069e90 2023-10-30T12:35:14.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f7cd3fff640 1 --2- 172.21.15.12:0/4291107248 >> [v2:172.21.15.12:6816/2335476277,v1:172.21.15.12:6817/2335476277] conn(0x7f7cb406f250 0x7f7cb40716f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82c0ff9640 1 -- 172.21.15.12:0/3955888431 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc003b00 con 0x7f82c4069e90 2023-10-30T12:35:14.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82c0ff9640 1 -- 172.21.15.12:0/3955888431 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc004020 con 0x7f82c4069e90 2023-10-30T12:35:14.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82ca772640 1 -- 172.21.15.12:0/3955888431 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c41df9e0 con 0x7f82c4069e90 2023-10-30T12:35:14.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7fedf6646640 1 Processor -- start 2023-10-30T12:35:14.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7fedf6646640 1 -- start start 2023-10-30T12:35:14.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f1061be2640 1 --2- 172.21.15.12:0/463151058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f105c070200 0x7f105c06ab20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f105c06b440 tx=0x7f105401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7459ac89857273b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f1060be0640 1 -- 172.21.15.12:0/463151058 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540a8a10 con 0x7f105c070200 2023-10-30T12:35:14.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f1060be0640 1 -- 172.21.15.12:0/463151058 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10540a8b70 con 0x7f105c070200 2023-10-30T12:35:14.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f1060be0640 1 -- 172.21.15.12:0/463151058 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540a8a10 con 0x7f105c070200 2023-10-30T12:35:14.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7fd97effd640 1 -- 172.21.15.12:0/4249511905 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9780bd770 con 0x7fd9880c52a0 2023-10-30T12:35:14.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.102+0000 7f82ca772640 1 -- 172.21.15.12:0/3955888431 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c40c3da0 con 0x7f82c4069e90 2023-10-30T12:35:14.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fbdd9b01640 1 --2- 172.21.15.12:0/3969647935 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd4158090 0x7fbdd4161be0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009dc0 tx=0x7fbdcc01f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14f6b427cd80e2b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fd990794640 1 -- 172.21.15.12:0/4249511905 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9881ef9a0 con 0x7fd9880c52a0 2023-10-30T12:35:14.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fd97effd640 1 -- 172.21.15.12:0/4249511905 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9780bdb20 con 0x7fd9880c52a0 2023-10-30T12:35:14.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fd990794640 1 -- 172.21.15.12:0/4249511905 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9881eff60 con 0x7fd9880c52a0 2023-10-30T12:35:14.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f8f48d65640 1 Processor -- start 2023-10-30T12:35:14.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fedf6646640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fedf0152ee0 0x7fedf01599d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fedf6646640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fedf00c0ce0 0x7fedf0156290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fedf6646640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf01567d0 con 0x7fedf00bfec0 2023-10-30T12:35:14.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fedf6646640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf01595b0 con 0x7fedf0152ee0 2023-10-30T12:35:14.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fedf6646640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf00c1130 con 0x7fedf00c0ce0 2023-10-30T12:35:14.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f8f48d65640 1 -- start start 2023-10-30T12:35:14.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f7cd0ff9640 1 -- 172.21.15.12:0/4291107248 <== osd.1 v2:172.21.15.12:6816/2335476277 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f7cb4071dc0 con 0x7f7cb406f250 2023-10-30T12:35:14.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f82ca772640 1 -- 172.21.15.12:0/3955888431 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f82c40c5f50 con 0x7f82c4069e90 2023-10-30T12:35:14.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f1062be4640 1 -- 172.21.15.12:0/463151058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f105c070200 msgr2=0x7f105c06ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f1062be4640 1 --2- 172.21.15.12:0/463151058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f105c070200 0x7f105c06ab20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f105c06b440 tx=0x7f105401ed10 comp rx=0 tx=0).stop 2023-10-30T12:35:14.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fbdd8aff640 1 -- 172.21.15.12:0/3969647935 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc0af600 con 0x7fbdd4158090 2023-10-30T12:35:14.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fbdd8aff640 1 -- 172.21.15.12:0/3969647935 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdcc0ab050 con 0x7fbdd4158090 2023-10-30T12:35:14.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fbdd8aff640 1 -- 172.21.15.12:0/3969647935 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc0af980 con 0x7fbdd4158090 2023-10-30T12:35:14.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fd97cff9640 1 -- 172.21.15.12:0/4249511905 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd964000fd0 con 0x7fd9880c52a0 2023-10-30T12:35:14.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fedeffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fedf0152ee0 0x7fedf01599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f8f48d65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 0x7f8f440c3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f8f48d65640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44070840 0x7f8f441f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fedeffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fedf0152ee0 0x7fedf01599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44294/0 (socket says 172.21.15.12:44294) 2023-10-30T12:35:14.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fedeffff640 1 -- 172.21.15.12:0/711823645 learned_addr learned my addr 172.21.15.12:0/711823645 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f8f48d65640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f44153810 0x7f8f44148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f8f48d65640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4415a110 con 0x7f8f44069e90 2023-10-30T12:35:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f1062be4640 1 -- 172.21.15.12:0/463151058 shutdown_connections 2023-10-30T12:35:14.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f1062be4640 1 --2- 172.21.15.12:0/463151058 >> v2:172.21.15.154:3300/0 conn(0x7f105c0c4030 0x7f105c06f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fedf4be3640 1 --2- 172.21.15.12:0/711823645 >> v2:172.21.15.154:3300/0 conn(0x7fedf00c0ce0 0x7fedf0156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f8f41d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44070840 0x7f8f441f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f8f41d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44070840 0x7f8f441f2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40496/0 (socket says 172.21.15.12:40496) 2023-10-30T12:35:14.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f8f41d9b640 1 -- 172.21.15.12:0/1727276088 learned_addr learned my addr 172.21.15.12:0/1727276088 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f1062be4640 1 --2- 172.21.15.12:0/463151058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f105c070200 0x7f105c06ab20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f1062be4640 1 -- 172.21.15.12:0/463151058 >> 172.21.15.12:0/463151058 conn(0x7f105c0c75d0 msgr2=0x7f105c0c79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f8f48d65640 1 -- 172.21.15.12:0/1727276088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f441f3770 con 0x7f8f44153810 2023-10-30T12:35:14.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f8f48d65640 1 -- 172.21.15.12:0/1727276088 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f441f3490 con 0x7f8f44070840 2023-10-30T12:35:14.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f8f42d9d640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f44153810 0x7f8f44148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f82c0ff9640 1 -- 172.21.15.12:0/3955888431 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f82bc004180 con 0x7f82c4069e90 2023-10-30T12:35:14.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f1062be4640 1 -- 172.21.15.12:0/463151058 shutdown_connections 2023-10-30T12:35:14.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7fedeeffd640 1 -- 172.21.15.12:0/711823645 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838923350 0 0) 0x7fedf01567d0 con 0x7fedf00bfec0 2023-10-30T12:35:14.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f1062be4640 1 -- 172.21.15.12:0/463151058 wait complete. 2023-10-30T12:35:14.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.106+0000 7f8f4259c640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 0x7f8f440c3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f8f42d9d640 1 -- 172.21.15.12:0/1727276088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44070840 msgr2=0x7f8f441f2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f8f42d9d640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44070840 0x7f8f441f2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f8f42d9d640 1 -- 172.21.15.12:0/1727276088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 msgr2=0x7f8f440c3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f8f42d9d640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 0x7f8f440c3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f8f42d9d640 1 -- 172.21.15.12:0/1727276088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f380094d0 con 0x7f8f44153810 2023-10-30T12:35:14.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f8f4259c640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 0x7f8f440c3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:35:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f8f41d9b640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44070840 0x7f8f441f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:35:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f82c0ff9640 1 --2- 172.21.15.12:0/3955888431 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82b00665a0 0x7f82b0068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f8f42d9d640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f44153810 0x7f8f44148fc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00be50 tx=0x7f8f3c00bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f82c0ff9640 1 -- 172.21.15.12:0/3955888431 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82bc0841a0 con 0x7f82c4069e90 2023-10-30T12:35:14.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f82c0ff9640 1 --2- 172.21.15.12:0/3955888431 >> [v2:172.21.15.12:6800/3034215139,v1:172.21.15.12:6801/3034215139] conn(0x7f82b006e7c0 0x7f82b0070be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f82c0ff9640 1 -- 172.21.15.12:0/3955888431 --> [v2:172.21.15.12:6800/3034215139,v1:172.21.15.12:6801/3034215139] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f82b0071290 con 0x7f82b006e7c0 2023-10-30T12:35:14.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f82c0ff9640 1 -- 172.21.15.12:0/3955888431 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82bc0867c0 con 0x7f82c4069e90 2023-10-30T12:35:14.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7fbddbd65640 1 -- 172.21.15.12:0/3969647935 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd4158090 msgr2=0x7fbdd4161be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7fd97effd640 1 -- 172.21.15.12:0/4249511905 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd9780a8050 con 0x7fd9880c52a0 2023-10-30T12:35:14.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f1062be4640 1 Processor -- start 2023-10-30T12:35:14.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7fedeeffd640 1 -- 172.21.15.12:0/711823645 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedd8003530 con 0x7fedf00bfec0 2023-10-30T12:35:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7fbddbd65640 1 --2- 172.21.15.12:0/3969647935 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd4158090 0x7fbdd4161be0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009dc0 tx=0x7fbdcc01f150 comp rx=0 tx=0).stop 2023-10-30T12:35:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f1062be4640 1 -- start start 2023-10-30T12:35:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7fedeffff640 1 -- 172.21.15.12:0/711823645 >> v2:172.21.15.154:3300/0 conn(0x7fedf00c0ce0 msgr2=0x7fedf0156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7fedeffff640 1 --2- 172.21.15.12:0/711823645 >> v2:172.21.15.154:3300/0 conn(0x7fedf00c0ce0 0x7fedf0156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7fedeffff640 1 -- 172.21.15.12:0/711823645 >> v1:172.21.15.12:6789/0 conn(0x7fedf00bfec0 legacy=0x7fedf00c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7fedeffff640 1 -- 172.21.15.12:0/711823645 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf00c0aa0 con 0x7fedf0152ee0 2023-10-30T12:35:14.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7fedf4be3640 1 --2- 172.21.15.12:0/711823645 >> v2:172.21.15.154:3300/0 conn(0x7fedf00c0ce0 0x7fedf0156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:14.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7fd97effd640 1 --2- 172.21.15.12:0/4249511905 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd96c066570 0x7fd96c068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7fd97effd640 1 -- 172.21.15.12:0/4249511905 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd97811ec90 con 0x7fd9880c52a0 2023-10-30T12:35:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f1062be4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105c06f880 0x7f105c153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f1062be4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f105c070200 0x7f105c0c3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7fd97effd640 1 --2- 172.21.15.12:0/4249511905 >> [v2:172.21.15.12:6808/2303962477,v1:172.21.15.12:6809/2303962477] conn(0x7fd96c06e790 0x7fd96c070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f10613e1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f105c070200 0x7f105c0c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f10613e1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f105c070200 0x7f105c0c3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44318/0 (socket says 172.21.15.12:44318) 2023-10-30T12:35:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f1061be2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105c06f880 0x7f105c153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f1061be2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105c06f880 0x7f105c153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40510/0 (socket says 172.21.15.12:40510) 2023-10-30T12:35:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f10613e1640 1 -- 172.21.15.12:0/3456694661 learned_addr learned my addr 172.21.15.12:0/3456694661 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f1062be4640 1 --2- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f105c0c4030 0x7f105c13b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f1062be4640 1 -- 172.21.15.12:0/3456694661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0c1410 con 0x7f105c070200 2023-10-30T12:35:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f1062be4640 1 -- 172.21.15.12:0/3456694661 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c13df00 con 0x7f105c0c4030 2023-10-30T12:35:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f1062be4640 1 -- 172.21.15.12:0/3456694661 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c1498c0 con 0x7f105c06f880 2023-10-30T12:35:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7fd97effd640 1 -- 172.21.15.12:0/4249511905 --> [v2:172.21.15.12:6808/2303962477,v1:172.21.15.12:6809/2303962477] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd96c071260 con 0x7fd96c06e790 2023-10-30T12:35:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7fd97effd640 1 -- 172.21.15.12:0/4249511905 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd97811f040 con 0x7fd9880c52a0 2023-10-30T12:35:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7fd98ed31640 1 --2- 172.21.15.12:0/4249511905 >> [v2:172.21.15.12:6808/2303962477,v1:172.21.15.12:6809/2303962477] conn(0x7fd96c06e790 0x7fd96c070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f8f32ffd640 1 -- 172.21.15.12:0/1727276088 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c014440 con 0x7f8f44153810 2023-10-30T12:35:14.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f8f32ffd640 1 -- 172.21.15.12:0/1727276088 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c0145a0 con 0x7f8f44153810 2023-10-30T12:35:14.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f8f32ffd640 1 -- 172.21.15.12:0/1727276088 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c014950 con 0x7f8f44153810 2023-10-30T12:35:14.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f8f48d65640 1 -- 172.21.15.12:0/1727276088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f44159770 con 0x7f8f44153810 2023-10-30T12:35:14.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f10623e3640 1 --2- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f105c0c4030 0x7f105c13b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.110+0000 7f82c37fe640 1 --2- 172.21.15.12:0/3955888431 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82b00665a0 0x7f82b0068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f10613e1640 1 -- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105c06f880 msgr2=0x7f105c153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f10613e1640 1 --2- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105c06f880 0x7f105c153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f10613e1640 1 -- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f105c0c4030 msgr2=0x7f105c13b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f10613e1640 1 --2- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f105c0c4030 0x7f105c13b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f10613e1640 1 -- 172.21.15.12:0/3456694661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1054009480 con 0x7f105c070200 2023-10-30T12:35:14.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f8f48d65640 1 -- 172.21.15.12:0/1727276088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f44159d30 con 0x7f8f44153810 2023-10-30T12:35:14.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7fd98dd2f640 1 --2- 172.21.15.12:0/4249511905 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd96c066570 0x7fd96c068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f82c8d0f640 1 --2- 172.21.15.12:0/3955888431 >> [v2:172.21.15.12:6800/3034215139,v1:172.21.15.12:6801/3034215139] conn(0x7f82b006e7c0 0x7f82b0070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f10613e1640 1 --2- 172.21.15.12:0/3456694661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f105c070200 0x7f105c0c3340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f105000ba50 tx=0x7f105000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7fd98ed31640 1 --2- 172.21.15.12:0/4249511905 >> [v2:172.21.15.12:6808/2303962477,v1:172.21.15.12:6809/2303962477] conn(0x7fd96c06e790 0x7fd96c070bb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f105a7fc640 1 -- 172.21.15.12:0/3456694661 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105000cbb0 con 0x7f105c070200 2023-10-30T12:35:14.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f105a7fc640 1 -- 172.21.15.12:0/3456694661 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105000cd10 con 0x7f105c070200 2023-10-30T12:35:14.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f105a7fc640 1 -- 172.21.15.12:0/3456694661 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050014680 con 0x7f105c070200 2023-10-30T12:35:14.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f8f48d65640 1 -- 172.21.15.12:0/1727276088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8f440c5f50 con 0x7f8f44153810 2023-10-30T12:35:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f1062be4640 1 -- 172.21.15.12:0/3456694661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c13cc70 con 0x7f105c070200 2023-10-30T12:35:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f82c37fe640 1 --2- 172.21.15.12:0/3955888431 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82b00665a0 0x7f82b0068a60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82b80b2000 tx=0x7f82b80aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f1062be4640 1 -- 172.21.15.12:0/3456694661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c13d8b0 con 0x7f105c070200 2023-10-30T12:35:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f1062be4640 1 -- 172.21.15.12:0/3456694661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f105c155fa0 con 0x7f105c070200 2023-10-30T12:35:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7fd97effd640 1 -- 172.21.15.12:0/4249511905 <== osd.0 v2:172.21.15.12:6808/2303962477 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fd96c071260 con 0x7fd96c06e790 2023-10-30T12:35:14.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f82c8d0f640 1 --2- 172.21.15.12:0/3955888431 >> [v2:172.21.15.12:6800/3034215139,v1:172.21.15.12:6801/3034215139] conn(0x7f82b006e7c0 0x7f82b0070be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7fbddbd65640 1 -- 172.21.15.12:0/3969647935 shutdown_connections 2023-10-30T12:35:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7fbddbd65640 1 --2- 172.21.15.12:0/3969647935 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd4158090 0x7fbdd4161be0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7fbddbd65640 1 -- 172.21.15.12:0/3969647935 >> 172.21.15.12:0/3969647935 conn(0x7fbdd40c7910 msgr2=0x7fbdd40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f6641809640 1 Processor -- start 2023-10-30T12:35:14.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f8f32ffd640 1 -- 172.21.15.12:0/1727276088 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8f3c01d3b0 con 0x7f8f44153810 2023-10-30T12:35:14.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f105a7fc640 1 -- 172.21.15.12:0/3456694661 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1050007370 con 0x7f105c070200 2023-10-30T12:35:14.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.114+0000 7f6641809640 1 -- start start 2023-10-30T12:35:14.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7fbddbd65640 1 -- 172.21.15.12:0/3969647935 shutdown_connections 2023-10-30T12:35:14.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7fedeffff640 1 --2- 172.21.15.12:0/711823645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedf0152ee0 0x7fedf01599d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fedf0072260 tx=0x7fede801ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06624bfc258a83a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f82c0ff9640 1 -- 172.21.15.12:0/3955888431 <== osd.2 v2:172.21.15.12:6800/3034215139 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f82b0071290 con 0x7f82b006e7c0 2023-10-30T12:35:14.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7fedeeffd640 1 -- 172.21.15.12:0/711823645 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede80a7d50 con 0x7fedf0152ee0 2023-10-30T12:35:14.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7fedeeffd640 1 -- 172.21.15.12:0/711823645 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede80ab070 con 0x7fedf0152ee0 2023-10-30T12:35:14.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7fedeeffd640 1 -- 172.21.15.12:0/711823645 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8003bc0 con 0x7fedf0152ee0 2023-10-30T12:35:14.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f105a7fc640 1 --2- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f10480664d0 0x7f1048068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f8f32ffd640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f24066570 0x7f8f24068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f105a7fc640 1 -- 172.21.15.12:0/3456694661 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1050083920 con 0x7f105c070200 2023-10-30T12:35:14.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f105a7fc640 1 --2- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:6808/3824067880,v1:172.21.15.154:6809/3824067880] conn(0x7f104806e670 0x7f1048070a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f105a7fc640 1 -- 172.21.15.12:0/3456694661 --> [v2:172.21.15.154:6808/3824067880,v1:172.21.15.154:6809/3824067880] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1048071140 con 0x7f104806e670 2023-10-30T12:35:14.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f105a7fc640 1 -- 172.21.15.12:0/3456694661 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10500882a0 con 0x7f105c070200 2023-10-30T12:35:14.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f1061be2640 1 --2- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f10480664d0 0x7f1048068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f10623e3640 1 --2- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:6808/3824067880,v1:172.21.15.154:6809/3824067880] conn(0x7f104806e670 0x7f1048070a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f8f32ffd640 1 -- 172.21.15.12:0/1727276088 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f3c084700 con 0x7f8f44153810 2023-10-30T12:35:14.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f8f32ffd640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.12:6824/3921928345,v1:172.21.15.12:6825/3921928345] conn(0x7f8f2406e790 0x7f8f24070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f8f32ffd640 1 -- 172.21.15.12:0/1727276088 --> [v2:172.21.15.12:6824/3921928345,v1:172.21.15.12:6825/3921928345] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f24071260 con 0x7f8f2406e790 2023-10-30T12:35:14.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f8f32ffd640 1 -- 172.21.15.12:0/1727276088 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f3c084ab0 con 0x7f8f44153810 2023-10-30T12:35:14.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f10623e3640 1 --2- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:6808/3824067880,v1:172.21.15.154:6809/3824067880] conn(0x7f104806e670 0x7f1048070a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f8f41d9b640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.12:6824/3921928345,v1:172.21.15.12:6825/3921928345] conn(0x7f8f2406e790 0x7f8f24070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f8f4259c640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f24066570 0x7f8f24068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f6641809640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f663c157f70 0x7f663c161b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f6641809640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f663c0cad80 0x7f663c0cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f8f41d9b640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.12:6824/3921928345,v1:172.21.15.12:6825/3921928345] conn(0x7f8f2406e790 0x7f8f24070bb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7fbddbd65640 1 -- 172.21.15.12:0/3969647935 wait complete. 2023-10-30T12:35:14.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7fbddbd65640 1 Processor -- start 2023-10-30T12:35:14.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7fbddbd65640 1 -- start start 2023-10-30T12:35:14.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f6641809640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f663c0cf160 0x7f663c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f8f4259c640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f24066570 0x7f8f24068a30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f380b2000 tx=0x7f8f380aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f6641809640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663c0ca8b0 con 0x7f663c0cad80 2023-10-30T12:35:14.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7f6641809640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663c1620d0 con 0x7f663c157f70 2023-10-30T12:35:14.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f6641809640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c158350 con 0x7f663c0cf160 2023-10-30T12:35:14.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f663b7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f663c0cf160 0x7f663c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f663a7fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f663c0cad80 0x7f663c0cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f663b7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f663c0cf160 0x7f663c0cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40512/0 (socket says 172.21.15.12:40512) 2023-10-30T12:35:14.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f663b7fe640 1 -- 172.21.15.12:0/333185700 learned_addr learned my addr 172.21.15.12:0/333185700 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f1061be2640 1 --2- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f10480664d0 0x7f1048068990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f10540b2000 tx=0x7f10540aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.118+0000 7fd98dd2f640 1 --2- 172.21.15.12:0/4249511905 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd96c066570 0x7fd96c068a30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd9880caba0 tx=0x7fd9840090e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f105a7fc640 1 -- 172.21.15.12:0/3456694661 <== osd.4 v2:172.21.15.154:6808/3824067880 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f1048071140 con 0x7f104806e670 2023-10-30T12:35:14.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fedf6646640 1 -- 172.21.15.12:0/711823645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedf0152ee0 msgr2=0x7fedf01599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fedf6646640 1 --2- 172.21.15.12:0/711823645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedf0152ee0 0x7fedf01599d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fedf0072260 tx=0x7fede801ed70 comp rx=0 tx=0).stop 2023-10-30T12:35:14.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbddbd65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdd40c0ed0 0x7fbdd40c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbddbd65640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdd4158090 0x7fbdd415efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f663affd640 1 --2- 172.21.15.12:0/333185700 >> v2:172.21.15.154:3300/0 conn(0x7f663c157f70 0x7f663c161b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f663a7fc640 1 -- 172.21.15.12:0/333185700 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f663c0cf160 msgr2=0x7f663c0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f663a7fc640 1 --2- 172.21.15.12:0/333185700 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f663c0cf160 0x7f663c0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f663a7fc640 1 -- 172.21.15.12:0/333185700 >> v2:172.21.15.154:3300/0 conn(0x7f663c157f70 msgr2=0x7f663c161b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f663a7fc640 1 --2- 172.21.15.12:0/333185700 >> v2:172.21.15.154:3300/0 conn(0x7f663c157f70 0x7f663c161b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f663a7fc640 1 -- 172.21.15.12:0/333185700 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c0ca2f0 con 0x7f663c0cad80 2023-10-30T12:35:14.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f663b7fe640 1 --2- 172.21.15.12:0/333185700 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f663c0cf160 0x7f663c0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:35:14.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f8f32ffd640 1 -- 172.21.15.12:0/1727276088 <== osd.3 v2:172.21.15.12:6824/3921928345 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f8f24071260 con 0x7f8f2406e790 2023-10-30T12:35:14.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7f663affd640 1 --2- 172.21.15.12:0/333185700 >> v2:172.21.15.154:3300/0 conn(0x7f663c157f70 0x7f663c161b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:35:14.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbddbd65640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd415cf20 0x7fbdd41608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbddbd65640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd415f8b0 con 0x7fbdd40c0ed0 2023-10-30T12:35:14.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbddbd65640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd41fc9d0 con 0x7fbdd4158090 2023-10-30T12:35:14.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbddbd65640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd415fb00 con 0x7fbdd415cf20 2023-10-30T12:35:14.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbdd9300640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdd4158090 0x7fbdd415efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbdd9300640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdd4158090 0x7fbdd415efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58094/0 (socket says 172.21.15.12:58094) 2023-10-30T12:35:14.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbdd9300640 1 -- 172.21.15.12:0/213678126 learned_addr learned my addr 172.21.15.12:0/213678126 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbdd9b01640 1 --2- 172.21.15.12:0/213678126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdd40c0ed0 0x7fbdd40c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbdda302640 1 --2- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd415cf20 0x7fbdd41608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fedf6646640 1 -- 172.21.15.12:0/711823645 shutdown_connections 2023-10-30T12:35:14.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fedf6646640 1 --2- 172.21.15.12:0/711823645 >> v2:172.21.15.154:3300/0 conn(0x7fedf00c0ce0 0x7fedf0156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fedf6646640 1 --2- 172.21.15.12:0/711823645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedf0152ee0 0x7fedf01599d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fedf6646640 1 -- 172.21.15.12:0/711823645 >> 172.21.15.12:0/711823645 conn(0x7fedf00c7910 msgr2=0x7fedf00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbdd9300640 1 -- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd415cf20 msgr2=0x7fbdd41608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbdd9300640 1 --2- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd415cf20 0x7fbdd41608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbdd9300640 1 -- 172.21.15.12:0/213678126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdd40c0ed0 msgr2=0x7fbdd40c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbdd9300640 1 --2- 172.21.15.12:0/213678126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdd40c0ed0 0x7fbdd40c63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbdd9300640 1 -- 172.21.15.12:0/213678126 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc009a70 con 0x7fbdd4158090 2023-10-30T12:35:14.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbdd9300640 1 --2- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdd4158090 0x7fbdd415efe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000c500 tx=0x7fbdd000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbdca7fc640 1 -- 172.21.15.12:0/213678126 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0005cf0 con 0x7fbdd4158090 2023-10-30T12:35:14.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbddbd65640 1 -- 172.21.15.12:0/213678126 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd415fde0 con 0x7fbdd4158090 2023-10-30T12:35:14.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbddbd65640 1 -- 172.21.15.12:0/213678126 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd41603a0 con 0x7fbdd4158090 2023-10-30T12:35:14.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbdca7fc640 1 -- 172.21.15.12:0/213678126 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd0005e50 con 0x7fbdd4158090 2023-10-30T12:35:14.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.122+0000 7fbdca7fc640 1 -- 172.21.15.12:0/213678126 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0007290 con 0x7fbdd4158090 2023-10-30T12:35:14.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.126+0000 7fedf6646640 1 -- 172.21.15.12:0/711823645 shutdown_connections 2023-10-30T12:35:14.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.126+0000 7fedf6646640 1 -- 172.21.15.12:0/711823645 wait complete. 2023-10-30T12:35:14.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.126+0000 7fbddbd65640 1 -- 172.21.15.12:0/213678126 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbdbc000fd0 con 0x7fbdd4158090 2023-10-30T12:35:14.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.126+0000 7f663a7fc640 1 --2- 172.21.15.12:0/333185700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f663c0cad80 0x7f663c0cc400 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f663c072ce0 tx=0x7f662c01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ed3d825110fd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedf6646640 1 Processor -- start 2023-10-30T12:35:14.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fbdca7fc640 1 -- 172.21.15.12:0/213678126 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbdd00073f0 con 0x7fbdd4158090 2023-10-30T12:35:14.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7f6639ffb640 1 -- 172.21.15.12:0/333185700 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c0a8910 con 0x7f663c0cad80 2023-10-30T12:35:14.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7f6639ffb640 1 -- 172.21.15.12:0/333185700 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c0a8a70 con 0x7f663c0cad80 2023-10-30T12:35:14.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7f6639ffb640 1 -- 172.21.15.12:0/333185700 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c0a8e00 con 0x7f663c0cad80 2023-10-30T12:35:14.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7f6641809640 1 -- 172.21.15.12:0/333185700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f663c0cad80 msgr2=0x7f663c0cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7f6641809640 1 --2- 172.21.15.12:0/333185700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f663c0cad80 0x7f663c0cc400 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f663c072ce0 tx=0x7f662c01ec50 comp rx=0 tx=0).stop 2023-10-30T12:35:14.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedf6646640 1 -- start start 2023-10-30T12:35:14.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedf6646640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedf00bfec0 0x7fedf01f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7f6641809640 1 -- 172.21.15.12:0/333185700 shutdown_connections 2023-10-30T12:35:14.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7f6641809640 1 --2- 172.21.15.12:0/333185700 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f663c0cf160 0x7f663c0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7f6641809640 1 --2- 172.21.15.12:0/333185700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f663c0cad80 0x7f663c0cc400 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7f6641809640 1 --2- 172.21.15.12:0/333185700 >> v2:172.21.15.154:3300/0 conn(0x7f663c157f70 0x7f663c161b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7f6641809640 1 -- 172.21.15.12:0/333185700 >> 172.21.15.12:0/333185700 conn(0x7f663c06bfe0 msgr2=0x7f663c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedf6646640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fedf00c0ce0 0x7fedf00c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fbdca7fc640 1 --2- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbdb80664d0 0x7fbdb8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedf6646640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedf0152ee0 0x7fedf014f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedf6646640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf01f66d0 con 0x7fedf0152ee0 2023-10-30T12:35:14.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedf6646640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf00c5c00 con 0x7fedf00bfec0 2023-10-30T12:35:14.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedf6646640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf00c53e0 con 0x7fedf00c0ce0 2023-10-30T12:35:14.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedeffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedf00bfec0 0x7fedf01f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedef7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fedf00c0ce0 0x7fedf00c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedef7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fedf00c0ce0 0x7fedf00c4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40530/0 (socket says 172.21.15.12:40530) 2023-10-30T12:35:14.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedef7fe640 1 -- 172.21.15.12:0/2030094848 learned_addr learned my addr 172.21.15.12:0/2030094848 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7f6641809640 1 -- 172.21.15.12:0/333185700 shutdown_connections 2023-10-30T12:35:14.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7f6641809640 1 -- 172.21.15.12:0/333185700 wait complete. 2023-10-30T12:35:14.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedf4be3640 1 --2- 172.21.15.12:0/2030094848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedf0152ee0 0x7fedf014f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fbdca7fc640 1 -- 172.21.15.12:0/213678126 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdd0083990 con 0x7fbdd4158090 2023-10-30T12:35:14.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedef7fe640 1 -- 172.21.15.12:0/2030094848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedf00bfec0 msgr2=0x7fedf01f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedef7fe640 1 --2- 172.21.15.12:0/2030094848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedf00bfec0 0x7fedf01f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedef7fe640 1 -- 172.21.15.12:0/2030094848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedf0152ee0 msgr2=0x7fedf014f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedef7fe640 1 --2- 172.21.15.12:0/2030094848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedf0152ee0 0x7fedf014f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fedef7fe640 1 -- 172.21.15.12:0/2030094848 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede800d050 con 0x7fedf00c0ce0 2023-10-30T12:35:14.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.130+0000 7fbdd9b01640 1 --2- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbdb80664d0 0x7fbdb8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.134+0000 7fbdca7fc640 1 --2- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:6800/2056450204,v1:172.21.15.154:6801/2056450204] conn(0x7fbdb806e6f0 0x7fbdb8070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.134+0000 7fbdca7fc640 1 -- 172.21.15.12:0/213678126 --> [v2:172.21.15.154:6800/2056450204,v1:172.21.15.154:6801/2056450204] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbdb80711c0 con 0x7fbdb806e6f0 2023-10-30T12:35:14.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.134+0000 7fbdca7fc640 1 -- 172.21.15.12:0/213678126 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdd00882a0 con 0x7fbdd4158090 2023-10-30T12:35:14.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.134+0000 7fedef7fe640 1 --2- 172.21.15.12:0/2030094848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fedf00c0ce0 0x7fedf00c4ea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fede400c430 tx=0x7fede400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.134+0000 7fedecff9640 1 -- 172.21.15.12:0/2030094848 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede40052d0 con 0x7fedf00c0ce0 2023-10-30T12:35:14.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.134+0000 7fedf6646640 1 -- 172.21.15.12:0/2030094848 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf00c2370 con 0x7fedf00c0ce0 2023-10-30T12:35:14.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.134+0000 7fedf6646640 1 -- 172.21.15.12:0/2030094848 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf00c2930 con 0x7fedf00c0ce0 2023-10-30T12:35:14.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.134+0000 7fedecff9640 1 -- 172.21.15.12:0/2030094848 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede4005450 con 0x7fedf00c0ce0 2023-10-30T12:35:14.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.134+0000 7fedecff9640 1 -- 172.21.15.12:0/2030094848 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede4003840 con 0x7fedf00c0ce0 2023-10-30T12:35:14.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.134+0000 7fedf6646640 1 -- 172.21.15.12:0/2030094848 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fedd0000fd0 con 0x7fedf00c0ce0 2023-10-30T12:35:14.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.134+0000 7fbdda302640 1 --2- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:6800/2056450204,v1:172.21.15.154:6801/2056450204] conn(0x7fbdb806e6f0 0x7fbdb8070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.134+0000 7fbdda302640 1 --2- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:6800/2056450204,v1:172.21.15.154:6801/2056450204] conn(0x7fbdb806e6f0 0x7fbdb8070b10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.134+0000 7f6641809640 1 Processor -- start 2023-10-30T12:35:14.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.134+0000 7fbdd9b01640 1 --2- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbdb80664d0 0x7fbdb8068990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc0b1000 tx=0x7fbdcc0aa000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f6641809640 1 -- start start 2023-10-30T12:35:14.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7fbdca7fc640 1 -- 172.21.15.12:0/213678126 <== osd.5 v2:172.21.15.154:6800/2056450204 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fbdb80711c0 con 0x7fbdb806e6f0 2023-10-30T12:35:14.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7fedecff9640 1 -- 172.21.15.12:0/2030094848 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fede40039a0 con 0x7fedf00c0ce0 2023-10-30T12:35:14.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f6641809640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f663c0cad80 0x7f663c0b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f6641809640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f663c0cf160 0x7f663c153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f6641809640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f663c157f70 0x7f663c0b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f6641809640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0b4a10 con 0x7f663c0cf160 2023-10-30T12:35:14.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f6641809640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0b33d0 con 0x7f663c157f70 2023-10-30T12:35:14.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f6641809640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c1f3c50 con 0x7f663c0cad80 2023-10-30T12:35:14.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f663affd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f663c0cad80 0x7f663c0b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f663affd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f663c0cad80 0x7f663c0b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40532/0 (socket says 172.21.15.12:40532) 2023-10-30T12:35:14.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f663affd640 1 -- 172.21.15.12:0/2955814739 learned_addr learned my addr 172.21.15.12:0/2955814739 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/4291107248 --> [v2:172.21.15.12:6816/2335476277,v1:172.21.15.12:6817/2335476277] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7cd40c5f50 con 0x7f7cb406f250 2023-10-30T12:35:14.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7fedecff9640 1 --2- 172.21.15.12:0/2030094848 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feddc0664a0 0x7feddc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7fedecff9640 1 -- 172.21.15.12:0/2030094848 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fede4083900 con 0x7fedf00c0ce0 2023-10-30T12:35:14.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f663a7fc640 1 --2- 172.21.15.12:0/2955814739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f663c0cf160 0x7f663c153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f663affd640 1 -- 172.21.15.12:0/2955814739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f663c157f70 msgr2=0x7f663c0b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f663affd640 1 --2- 172.21.15.12:0/2955814739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f663c157f70 0x7f663c0b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f663affd640 1 -- 172.21.15.12:0/2955814739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f663c0cf160 msgr2=0x7f663c153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f663affd640 1 --2- 172.21.15.12:0/2955814739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f663c0cf160 0x7f663c153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7f663affd640 1 -- 172.21.15.12:0/2955814739 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66300094d0 con 0x7f663c0cad80 2023-10-30T12:35:14.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7fedecff9640 1 --2- 172.21.15.12:0/2030094848 >> [v2:172.21.15.154:6815/219299884,v1:172.21.15.154:6818/219299884] conn(0x7feddc06e6c0 0x7feddc070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7fedeffff640 1 --2- 172.21.15.12:0/2030094848 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feddc0664a0 0x7feddc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7fedecff9640 1 -- 172.21.15.12:0/2030094848 --> [v2:172.21.15.154:6815/219299884,v1:172.21.15.154:6818/219299884] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feddc071190 con 0x7feddc06e6c0 2023-10-30T12:35:14.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.138+0000 7fedecff9640 1 -- 172.21.15.12:0/2030094848 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fede4083cb0 con 0x7fedf00c0ce0 2023-10-30T12:35:14.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f7cd0ff9640 1 -- 172.21.15.12:0/4291107248 <== osd.1 v2:172.21.15.12:6816/2335476277 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7ccc009cb0 con 0x7f7cb406f250 2023-10-30T12:35:14.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f663affd640 1 --2- 172.21.15.12:0/2955814739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f663c0cad80 0x7f663c0b44d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f663400ce30 tx=0x7f663400a650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7fedf4be3640 1 --2- 172.21.15.12:0/2030094848 >> [v2:172.21.15.154:6815/219299884,v1:172.21.15.154:6818/219299884] conn(0x7feddc06e6c0 0x7feddc070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f6623fff640 1 -- 172.21.15.12:0/2955814739 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663400f680 con 0x7f663c0cad80 2023-10-30T12:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f6623fff640 1 -- 172.21.15.12:0/2955814739 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663400f7e0 con 0x7f663c0cad80 2023-10-30T12:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f6623fff640 1 -- 172.21.15.12:0/2955814739 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663400fb70 con 0x7f663c0cad80 2023-10-30T12:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/4291107248 >> [v2:172.21.15.12:6816/2335476277,v1:172.21.15.12:6817/2335476277] conn(0x7f7cb406f250 msgr2=0x7f7cb40716f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f7cd9f3e640 1 --2- 172.21.15.12:0/4291107248 >> [v2:172.21.15.12:6816/2335476277,v1:172.21.15.12:6817/2335476277] conn(0x7f7cb406f250 0x7f7cb40716f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/4291107248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cb40664a0 msgr2=0x7f7cb4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f7cd9f3e640 1 --2- 172.21.15.12:0/4291107248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cb40664a0 0x7f7cb4068960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0b2000 tx=0x7f7cbc0aa070 comp rx=0 tx=0).stop 2023-10-30T12:35:14.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/4291107248 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd4069e90 msgr2=0x7f7cd41f5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f7cd9f3e640 1 --2- 172.21.15.12:0/4291107248 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd4069e90 0x7f7cd41f5aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400c4d0 tx=0x7f7cc400c990 comp rx=0 tx=0).stop 2023-10-30T12:35:14.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f7cd3fff640 1 -- 172.21.15.12:0/4291107248 reap_dead start 2023-10-30T12:35:14.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/4291107248 shutdown_connections 2023-10-30T12:35:14.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/4291107248 >> 172.21.15.12:0/4291107248 conn(0x7f7cd40c7910 msgr2=0x7f7cd40bfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/4291107248 shutdown_connections 2023-10-30T12:35:14.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f6641809640 1 -- 172.21.15.12:0/2955814739 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662c009420 con 0x7f663c0cad80 2023-10-30T12:35:14.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f6641809640 1 -- 172.21.15.12:0/2955814739 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c1f41f0 con 0x7f663c0cad80 2023-10-30T12:35:14.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f7cd9f3e640 1 -- 172.21.15.12:0/4291107248 wait complete. 2023-10-30T12:35:14.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7fedf4be3640 1 --2- 172.21.15.12:0/2030094848 >> [v2:172.21.15.154:6815/219299884,v1:172.21.15.154:6818/219299884] conn(0x7feddc06e6c0 0x7feddc070ae0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.146+0000 7fedeffff640 1 --2- 172.21.15.12:0/2030094848 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feddc0664a0 0x7feddc068960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fede80b1000 tx=0x7fede80a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.146+0000 7fedecff9640 1 -- 172.21.15.12:0/2030094848 <== osd.7 v2:172.21.15.154:6815/219299884 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7feddc071190 con 0x7feddc06e6c0 2023-10-30T12:35:14.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.142+0000 7f6641809640 1 -- 172.21.15.12:0/2955814739 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f663c0cb920 con 0x7f663c0cad80 2023-10-30T12:35:14.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.146+0000 7f6623fff640 1 -- 172.21.15.12:0/2955814739 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f663400b040 con 0x7f663c0cad80 2023-10-30T12:35:14.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.146+0000 7f82ca772640 1 -- 172.21.15.12:0/3955888431 --> [v2:172.21.15.12:6800/3034215139,v1:172.21.15.12:6801/3034215139] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f82c40c5c70 con 0x7f82b006e7c0 2023-10-30T12:35:14.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.146+0000 7fd97cff9640 1 -- 172.21.15.12:0/4249511905 --> [v2:172.21.15.12:6808/2303962477,v1:172.21.15.12:6809/2303962477] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd964002cc0 con 0x7fd96c06e790 2023-10-30T12:35:14.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.146+0000 7f8f48d65640 1 -- 172.21.15.12:0/1727276088 --> [v2:172.21.15.12:6824/3921928345,v1:172.21.15.12:6825/3921928345] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8f440c5c70 con 0x7f8f2406e790 2023-10-30T12:35:14.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.146+0000 7fd97effd640 1 -- 172.21.15.12:0/4249511905 <== osd.0 v2:172.21.15.12:6808/2303962477 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd964002cc0 con 0x7fd96c06e790 2023-10-30T12:35:14.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.146+0000 7f82c0ff9640 1 -- 172.21.15.12:0/3955888431 <== osd.2 v2:172.21.15.12:6800/3034215139 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f82b400a2f0 con 0x7f82b006e7c0 2023-10-30T12:35:14.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.146+0000 7f8f32ffd640 1 -- 172.21.15.12:0/1727276088 <== osd.3 v2:172.21.15.12:6824/3921928345 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8f34009a40 con 0x7f8f2406e790 2023-10-30T12:35:14.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f8f48d65640 1 -- 172.21.15.12:0/1727276088 >> [v2:172.21.15.12:6824/3921928345,v1:172.21.15.12:6825/3921928345] conn(0x7f8f2406e790 msgr2=0x7f8f24070bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f8f48d65640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.12:6824/3921928345,v1:172.21.15.12:6825/3921928345] conn(0x7f8f2406e790 0x7f8f24070bb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.146+0000 7f6623fff640 1 --2- 172.21.15.12:0/2955814739 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66280664a0 0x7f6628068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f6623fff640 1 -- 172.21.15.12:0/2955814739 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6634084a00 con 0x7f663c0cad80 2023-10-30T12:35:14.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7fd990794640 1 -- 172.21.15.12:0/4249511905 >> [v2:172.21.15.12:6808/2303962477,v1:172.21.15.12:6809/2303962477] conn(0x7fd96c06e790 msgr2=0x7fd96c070bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7fd990794640 1 --2- 172.21.15.12:0/4249511905 >> [v2:172.21.15.12:6808/2303962477,v1:172.21.15.12:6809/2303962477] conn(0x7fd96c06e790 0x7fd96c070bb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f8f48d65640 1 -- 172.21.15.12:0/1727276088 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f24066570 msgr2=0x7f8f24068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f8f48d65640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f24066570 0x7f8f24068a30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f380b2000 tx=0x7f8f380aa070 comp rx=0 tx=0).stop 2023-10-30T12:35:14.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f8f48d65640 1 -- 172.21.15.12:0/1727276088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f44153810 msgr2=0x7f8f44148fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f8f48d65640 1 --2- 172.21.15.12:0/1727276088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f44153810 0x7f8f44148fc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00be50 tx=0x7f8f3c00bf40 comp rx=0 tx=0).stop 2023-10-30T12:35:14.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f8f42d9d640 1 -- 172.21.15.12:0/1727276088 reap_dead start 2023-10-30T12:35:14.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f6623fff640 1 --2- 172.21.15.12:0/2955814739 >> [v2:172.21.15.154:6817/1102176766,v1:172.21.15.154:6819/1102176766] conn(0x7f662806e6c0 0x7f6628070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f6623fff640 1 -- 172.21.15.12:0/2955814739 --> [v2:172.21.15.154:6817/1102176766,v1:172.21.15.154:6819/1102176766] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6628071190 con 0x7f662806e6c0 2023-10-30T12:35:14.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f6623fff640 1 -- 172.21.15.12:0/2955814739 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6634084db0 con 0x7f663c0cad80 2023-10-30T12:35:14.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f82ca772640 1 -- 172.21.15.12:0/3955888431 >> [v2:172.21.15.12:6800/3034215139,v1:172.21.15.12:6801/3034215139] conn(0x7f82b006e7c0 msgr2=0x7f82b0070be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f82ca772640 1 --2- 172.21.15.12:0/3955888431 >> [v2:172.21.15.12:6800/3034215139,v1:172.21.15.12:6801/3034215139] conn(0x7f82b006e7c0 0x7f82b0070be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f8f48d65640 1 -- 172.21.15.12:0/1727276088 shutdown_connections 2023-10-30T12:35:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f8f48d65640 1 -- 172.21.15.12:0/1727276088 >> 172.21.15.12:0/1727276088 conn(0x7f8f440c7910 msgr2=0x7f8f4406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7fd990794640 1 -- 172.21.15.12:0/4249511905 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd96c066570 msgr2=0x7fd96c068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7fd990794640 1 --2- 172.21.15.12:0/4249511905 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd96c066570 0x7fd96c068a30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd9880caba0 tx=0x7fd9840090e0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7fd990794640 1 -- 172.21.15.12:0/4249511905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9880c52a0 msgr2=0x7fd98814ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7fd990794640 1 --2- 172.21.15.12:0/4249511905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9880c52a0 0x7fd98814ba30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd97801ab90 tx=0x7fd97801ad20 comp rx=0 tx=0).stop 2023-10-30T12:35:14.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7fd98ed31640 1 -- 172.21.15.12:0/4249511905 reap_dead start 2023-10-30T12:35:14.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7fd990794640 1 -- 172.21.15.12:0/4249511905 shutdown_connections 2023-10-30T12:35:14.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7fd990794640 1 -- 172.21.15.12:0/4249511905 >> 172.21.15.12:0/4249511905 conn(0x7fd9880cb130 msgr2=0x7fd988069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f82ca772640 1 -- 172.21.15.12:0/3955888431 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82b00665a0 msgr2=0x7f82b0068a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f82ca772640 1 --2- 172.21.15.12:0/3955888431 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82b00665a0 0x7f82b0068a60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82b80b2000 tx=0x7f82b80aa070 comp rx=0 tx=0).stop 2023-10-30T12:35:14.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f8f48d65640 1 -- 172.21.15.12:0/1727276088 shutdown_connections 2023-10-30T12:35:14.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f663b7fe640 1 --2- 172.21.15.12:0/2955814739 >> [v2:172.21.15.154:6817/1102176766,v1:172.21.15.154:6819/1102176766] conn(0x7f662806e6c0 0x7f6628070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f8f48d65640 1 -- 172.21.15.12:0/1727276088 wait complete. 2023-10-30T12:35:14.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f82ca772640 1 -- 172.21.15.12:0/3955888431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c4069e90 msgr2=0x7f82c41e0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f82ca772640 1 --2- 172.21.15.12:0/3955888431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c4069e90 0x7f82c41e0ce0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00ccd0 tx=0x7f82bc00a5c0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f82c8d0f640 1 -- 172.21.15.12:0/3955888431 reap_dead start 2023-10-30T12:35:14.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f82ca772640 1 -- 172.21.15.12:0/3955888431 shutdown_connections 2023-10-30T12:35:14.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f82ca772640 1 -- 172.21.15.12:0/3955888431 >> 172.21.15.12:0/3955888431 conn(0x7f82c40c7910 msgr2=0x7f82c414dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7fd990794640 1 -- 172.21.15.12:0/4249511905 shutdown_connections 2023-10-30T12:35:14.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f82ca772640 1 -- 172.21.15.12:0/3955888431 shutdown_connections 2023-10-30T12:35:14.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f82ca772640 1 -- 172.21.15.12:0/3955888431 wait complete. 2023-10-30T12:35:14.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7fd990794640 1 -- 172.21.15.12:0/4249511905 wait complete. 2023-10-30T12:35:14.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f663a7fc640 1 --2- 172.21.15.12:0/2955814739 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66280664a0 0x7f6628068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.150+0000 7f663b7fe640 1 --2- 172.21.15.12:0/2955814739 >> [v2:172.21.15.154:6817/1102176766,v1:172.21.15.154:6819/1102176766] conn(0x7f662806e6c0 0x7f6628070ae0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.154+0000 7f663a7fc640 1 --2- 172.21.15.12:0/2955814739 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66280664a0 0x7f6628068960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f662c0b2000 tx=0x7f662c0aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.154+0000 7f6623fff640 1 -- 172.21.15.12:0/2955814739 <== osd.6 v2:172.21.15.154:6817/1102176766 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f6628071190 con 0x7f662806e6c0 2023-10-30T12:35:14.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.154+0000 7f1062be4640 1 -- 172.21.15.12:0/3456694661 --> [v2:172.21.15.154:6808/3824067880,v1:172.21.15.154:6809/3824067880] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f105c155cc0 con 0x7f104806e670 2023-10-30T12:35:14.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.162+0000 7f105a7fc640 1 -- 172.21.15.12:0/3456694661 <== osd.4 v2:172.21.15.154:6808/3824067880 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f104c0012f0 con 0x7f104806e670 2023-10-30T12:35:14.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.166+0000 7f1062be4640 1 -- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:6808/3824067880,v1:172.21.15.154:6809/3824067880] conn(0x7f104806e670 msgr2=0x7f1048070a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.166+0000 7f1062be4640 1 --2- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:6808/3824067880,v1:172.21.15.154:6809/3824067880] conn(0x7f104806e670 0x7f1048070a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.166+0000 7f1062be4640 1 -- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f10480664d0 msgr2=0x7f1048068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.166+0000 7f1062be4640 1 --2- 172.21.15.12:0/3456694661 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f10480664d0 0x7f1048068990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f10540b2000 tx=0x7f10540aa070 comp rx=0 tx=0).stop 2023-10-30T12:35:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.166+0000 7f1062be4640 1 -- 172.21.15.12:0/3456694661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f105c070200 msgr2=0x7f105c0c3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.166+0000 7f1062be4640 1 --2- 172.21.15.12:0/3456694661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f105c070200 0x7f105c0c3340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f105000ba50 tx=0x7f105000bf10 comp rx=0 tx=0).stop 2023-10-30T12:35:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.166+0000 7f10623e3640 1 -- 172.21.15.12:0/3456694661 reap_dead start 2023-10-30T12:35:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.166+0000 7f1062be4640 1 -- 172.21.15.12:0/3456694661 shutdown_connections 2023-10-30T12:35:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.166+0000 7f1062be4640 1 -- 172.21.15.12:0/3456694661 >> 172.21.15.12:0/3456694661 conn(0x7f105c0c75d0 msgr2=0x7f105c147c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.166+0000 7fbddbd65640 1 -- 172.21.15.12:0/213678126 --> [v2:172.21.15.154:6800/2056450204,v1:172.21.15.154:6801/2056450204] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbdbc002cc0 con 0x7fbdb806e6f0 2023-10-30T12:35:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.170+0000 7fbdca7fc640 1 -- 172.21.15.12:0/213678126 <== osd.5 v2:172.21.15.154:6800/2056450204 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbdbc002cc0 con 0x7fbdb806e6f0 2023-10-30T12:35:14.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.170+0000 7fbddbd65640 1 -- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:6800/2056450204,v1:172.21.15.154:6801/2056450204] conn(0x7fbdb806e6f0 msgr2=0x7fbdb8070b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.170+0000 7fbddbd65640 1 --2- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:6800/2056450204,v1:172.21.15.154:6801/2056450204] conn(0x7fbdb806e6f0 0x7fbdb8070b10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.170+0000 7fbddbd65640 1 -- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbdb80664d0 msgr2=0x7fbdb8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.170+0000 7fbddbd65640 1 --2- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbdb80664d0 0x7fbdb8068990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc0b1000 tx=0x7fbdcc0aa000 comp rx=0 tx=0).stop 2023-10-30T12:35:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.170+0000 7fbddbd65640 1 -- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdd4158090 msgr2=0x7fbdd415efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.170+0000 7fbddbd65640 1 --2- 172.21.15.12:0/213678126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdd4158090 0x7fbdd415efe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000c500 tx=0x7fbdd000c9c0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.170+0000 7fbdda302640 1 -- 172.21.15.12:0/213678126 reap_dead start 2023-10-30T12:35:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.170+0000 7fbddbd65640 1 -- 172.21.15.12:0/213678126 shutdown_connections 2023-10-30T12:35:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.170+0000 7fbddbd65640 1 -- 172.21.15.12:0/213678126 >> 172.21.15.12:0/213678126 conn(0x7fbdd40c7910 msgr2=0x7fbdd415c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.170+0000 7fbddbd65640 1 -- 172.21.15.12:0/213678126 shutdown_connections 2023-10-30T12:35:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.170+0000 7fbddbd65640 1 -- 172.21.15.12:0/213678126 wait complete. 2023-10-30T12:35:14.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.170+0000 7f1062be4640 1 -- 172.21.15.12:0/3456694661 shutdown_connections 2023-10-30T12:35:14.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.170+0000 7f1062be4640 1 -- 172.21.15.12:0/3456694661 wait complete. 2023-10-30T12:35:14.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.174+0000 7fedf6646640 1 -- 172.21.15.12:0/2030094848 --> [v2:172.21.15.154:6815/219299884,v1:172.21.15.154:6818/219299884] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fedd0002cc0 con 0x7feddc06e6c0 2023-10-30T12:35:14.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.174+0000 7fedecff9640 1 -- 172.21.15.12:0/2030094848 <== osd.7 v2:172.21.15.154:6815/219299884 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fedd0002cc0 con 0x7feddc06e6c0 2023-10-30T12:35:14.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.178+0000 7fedf6646640 1 -- 172.21.15.12:0/2030094848 >> [v2:172.21.15.154:6815/219299884,v1:172.21.15.154:6818/219299884] conn(0x7feddc06e6c0 msgr2=0x7feddc070ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.178+0000 7fedf6646640 1 --2- 172.21.15.12:0/2030094848 >> [v2:172.21.15.154:6815/219299884,v1:172.21.15.154:6818/219299884] conn(0x7feddc06e6c0 0x7feddc070ae0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.178+0000 7fedf6646640 1 -- 172.21.15.12:0/2030094848 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feddc0664a0 msgr2=0x7feddc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.178+0000 7fedf6646640 1 --2- 172.21.15.12:0/2030094848 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feddc0664a0 0x7feddc068960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fede80b1000 tx=0x7fede80a9070 comp rx=0 tx=0).stop 2023-10-30T12:35:14.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.178+0000 7fedf6646640 1 -- 172.21.15.12:0/2030094848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fedf00c0ce0 msgr2=0x7fedf00c4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.178+0000 7fedf6646640 1 --2- 172.21.15.12:0/2030094848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fedf00c0ce0 0x7fedf00c4ea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fede400c430 tx=0x7fede400c8f0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.182+0000 7fedf4be3640 1 -- 172.21.15.12:0/2030094848 reap_dead start 2023-10-30T12:35:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:60129542151 2023-10-30T12:35:14.271 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-30T12:35:14.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.186+0000 7f6641809640 1 -- 172.21.15.12:0/2955814739 --> [v2:172.21.15.154:6817/1102176766,v1:172.21.15.154:6819/1102176766] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f663c0ca5d0 con 0x7f662806e6c0 2023-10-30T12:35:14.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.186+0000 7fedf6646640 1 -- 172.21.15.12:0/2030094848 shutdown_connections 2023-10-30T12:35:14.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.186+0000 7fedf6646640 1 -- 172.21.15.12:0/2030094848 >> 172.21.15.12:0/2030094848 conn(0x7fedf00c7910 msgr2=0x7fedf0151430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.186+0000 7fedf6646640 1 -- 172.21.15.12:0/2030094848 shutdown_connections 2023-10-30T12:35:14.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.186+0000 7fedf6646640 1 -- 172.21.15.12:0/2030094848 wait complete. 2023-10-30T12:35:14.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.186+0000 7f6623fff640 1 -- 172.21.15.12:0/2955814739 <== osd.6 v2:172.21.15.154:6817/1102176766 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6630009bd0 con 0x7f662806e6c0 2023-10-30T12:35:14.275 INFO:teuthology.orchestra.run.smithi012.stdout:60129542153 2023-10-30T12:35:14.275 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-30T12:35:14.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.186+0000 7f6641809640 1 -- 172.21.15.12:0/2955814739 >> [v2:172.21.15.154:6817/1102176766,v1:172.21.15.154:6819/1102176766] conn(0x7f662806e6c0 msgr2=0x7f6628070ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.186+0000 7f6641809640 1 --2- 172.21.15.12:0/2955814739 >> [v2:172.21.15.154:6817/1102176766,v1:172.21.15.154:6819/1102176766] conn(0x7f662806e6c0 0x7f6628070ae0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.186+0000 7f6641809640 1 -- 172.21.15.12:0/2955814739 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66280664a0 msgr2=0x7f6628068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.186+0000 7f6641809640 1 --2- 172.21.15.12:0/2955814739 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66280664a0 0x7f6628068960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f662c0b2000 tx=0x7f662c0aa070 comp rx=0 tx=0).stop 2023-10-30T12:35:14.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.186+0000 7f6641809640 1 -- 172.21.15.12:0/2955814739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f663c0cad80 msgr2=0x7f663c0b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.186+0000 7f6641809640 1 --2- 172.21.15.12:0/2955814739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f663c0cad80 0x7f663c0b44d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f663400ce30 tx=0x7f663400a650 comp rx=0 tx=0).stop 2023-10-30T12:35:14.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.186+0000 7f663b7fe640 1 -- 172.21.15.12:0/2955814739 reap_dead start 2023-10-30T12:35:14.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.186+0000 7f6641809640 1 -- 172.21.15.12:0/2955814739 shutdown_connections 2023-10-30T12:35:14.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.186+0000 7f6641809640 1 -- 172.21.15.12:0/2955814739 >> 172.21.15.12:0/2955814739 conn(0x7f663c06bfe0 msgr2=0x7f663c155670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.279 INFO:teuthology.orchestra.run.smithi012.stdout:60129542151 2023-10-30T12:35:14.279 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-30T12:35:14.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.190+0000 7f6641809640 1 -- 172.21.15.12:0/2955814739 shutdown_connections 2023-10-30T12:35:14.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.190+0000 7f6641809640 1 -- 172.21.15.12:0/2955814739 wait complete. 2023-10-30T12:35:14.281 INFO:teuthology.orchestra.run.smithi012.stdout:60129542153 2023-10-30T12:35:14.281 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-30T12:35:14.282 INFO:teuthology.orchestra.run.smithi012.stdout:60129542151 2023-10-30T12:35:14.283 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-30T12:35:14.283 INFO:teuthology.orchestra.run.smithi012.stdout:60129542151 2023-10-30T12:35:14.284 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-30T12:35:14.285 INFO:teuthology.orchestra.run.smithi012.stdout:60129542152 2023-10-30T12:35:14.285 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-30T12:35:14.286 INFO:teuthology.orchestra.run.smithi012.stdout:60129542153 2023-10-30T12:35:14.287 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-30T12:35:14.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.550+0000 7f64ac66b640 1 Processor -- start 2023-10-30T12:35:14.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.550+0000 7f64ac66b640 1 -- start start 2023-10-30T12:35:14.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.554+0000 7f64ac66b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f64a414c140 0x7f64a4156020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.554+0000 7f64ac66b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64a40c2700 0x7f64a415ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.554+0000 7f64a9c06640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f64a40c2290 0x7f64a406d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:55414/0 (socket says 172.21.15.12:55414) 2023-10-30T12:35:14.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.554+0000 7f64a9c06640 1 -- 172.21.15.12:0/3102991805 learned_addr learned my addr 172.21.15.12:0/3102991805 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:14.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.554+0000 7f64ac66b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a40c2b10 con 0x7f64a40c2290 2023-10-30T12:35:14.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.554+0000 7f64ac66b640 1 -- 172.21.15.12:0/3102991805 --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a40c6370 con 0x7f64a414c140 2023-10-30T12:35:14.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.554+0000 7f64ac66b640 1 -- 172.21.15.12:0/3102991805 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a4159af0 con 0x7f64a40c2700 2023-10-30T12:35:14.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.554+0000 7f64aac08640 1 --2- 172.21.15.12:0/3102991805 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64a40c2700 0x7f64a415ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.554+0000 7f64a9405640 1 -- 172.21.15.12:0/3102991805 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469537274 0 0) 0x7f64a40c2b10 con 0x7f64a40c2290 2023-10-30T12:35:14.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.554+0000 7f64a9405640 1 -- 172.21.15.12:0/3102991805 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6490003530 con 0x7f64a40c2290 2023-10-30T12:35:14.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.554+0000 7f64aac08640 1 -- 172.21.15.12:0/3102991805 >> v2:172.21.15.12:3300/0 conn(0x7f64a414c140 msgr2=0x7f64a4156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.554+0000 7f64aac08640 1 --2- 172.21.15.12:0/3102991805 >> v2:172.21.15.12:3300/0 conn(0x7f64a414c140 0x7f64a4156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.554+0000 7f64aac08640 1 -- 172.21.15.12:0/3102991805 >> v1:172.21.15.12:6789/0 conn(0x7f64a40c2290 legacy=0x7f64a406d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.554+0000 7f64aac08640 1 -- 172.21.15.12:0/3102991805 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a40c6090 con 0x7f64a40c2700 2023-10-30T12:35:14.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.562+0000 7f5d2d703640 1 Processor -- start 2023-10-30T12:35:14.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.562+0000 7f5d2d703640 1 -- start start 2023-10-30T12:35:14.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.562+0000 7f64aac08640 1 --2- 172.21.15.12:0/3102991805 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64a40c2700 0x7f64a415ff20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6494013db0 tx=0x7f64940273b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9799a8eb73ddbd9f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.562+0000 7f64a9405640 1 -- 172.21.15.12:0/3102991805 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64940b7600 con 0x7f64a40c2700 2023-10-30T12:35:14.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.562+0000 7f5d2d703640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d280c5550 0x7f5d280cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.562+0000 7f64a9405640 1 -- 172.21.15.12:0/3102991805 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64940b3050 con 0x7f64a40c2700 2023-10-30T12:35:14.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.562+0000 7f64ac66b640 1 -- 172.21.15.12:0/3102991805 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64a40c2700 msgr2=0x7f64a415ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.562+0000 7f64a9405640 1 -- 172.21.15.12:0/3102991805 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64940b7960 con 0x7f64a40c2700 2023-10-30T12:35:14.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.562+0000 7f5d2d703640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5d280cb050 0x7f5d280cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.562+0000 7f5d2d703640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5d280cab00 0x7f5d280cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.562+0000 7f64ac66b640 1 --2- 172.21.15.12:0/3102991805 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64a40c2700 0x7f64a415ff20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6494013db0 tx=0x7f64940273b0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.562+0000 7f5d2d703640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d280cba30 con 0x7f5d280cb050 2023-10-30T12:35:14.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.562+0000 7f5d2d703640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d280ca290 con 0x7f5d280cab00 2023-10-30T12:35:14.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.562+0000 7f5d2d703640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d280c9cf0 con 0x7f5d280c5550 2023-10-30T12:35:14.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f5d2cf02640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5d280cab00 0x7f5d280cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f64ac66b640 1 -- 172.21.15.12:0/3102991805 shutdown_connections 2023-10-30T12:35:14.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f64ac66b640 1 --2- 172.21.15.12:0/3102991805 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64a40c2700 0x7f64a415ff20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f64ac66b640 1 --2- 172.21.15.12:0/3102991805 >> v2:172.21.15.12:3300/0 conn(0x7f64a414c140 0x7f64a4156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f64ac66b640 1 -- 172.21.15.12:0/3102991805 >> 172.21.15.12:0/3102991805 conn(0x7f64a40cac50 msgr2=0x7f64a40cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f64ac66b640 1 -- 172.21.15.12:0/3102991805 shutdown_connections 2023-10-30T12:35:14.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f64ac66b640 1 -- 172.21.15.12:0/3102991805 wait complete. 2023-10-30T12:35:14.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f5d277fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5d280cb050 0x7f5d280cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f5d277fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5d280cb050 0x7f5d280cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44376/0 (socket says 172.21.15.12:44376) 2023-10-30T12:35:14.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f5d277fe640 1 -- 172.21.15.12:0/1234921316 learned_addr learned my addr 172.21.15.12:0/1234921316 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f5d27fff640 1 --2- 172.21.15.12:0/1234921316 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d280c5550 0x7f5d280cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f5d277fe640 1 -- 172.21.15.12:0/1234921316 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d280c5550 msgr2=0x7f5d280cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f5d277fe640 1 --2- 172.21.15.12:0/1234921316 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d280c5550 0x7f5d280cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f5d277fe640 1 -- 172.21.15.12:0/1234921316 >> v2:172.21.15.154:3300/0 conn(0x7f5d280cab00 msgr2=0x7f5d280cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f5d277fe640 1 --2- 172.21.15.12:0/1234921316 >> v2:172.21.15.154:3300/0 conn(0x7f5d280cab00 0x7f5d280cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f5d277fe640 1 -- 172.21.15.12:0/1234921316 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2815d990 con 0x7f5d280cb050 2023-10-30T12:35:14.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f5d27fff640 1 --2- 172.21.15.12:0/1234921316 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d280c5550 0x7f5d280cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:14.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f5d2cf02640 1 --2- 172.21.15.12:0/1234921316 >> v2:172.21.15.154:3300/0 conn(0x7f5d280cab00 0x7f5d280cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:35:14.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7ff7e5e09640 1 Processor -- start 2023-10-30T12:35:14.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f64ac66b640 1 Processor -- start 2023-10-30T12:35:14.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7f64ac66b640 1 -- start start 2023-10-30T12:35:14.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.566+0000 7ff7e5e09640 1 -- start start 2023-10-30T12:35:14.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7ff7e5e09640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff7d80d9230 0x7ff7d8049450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7ff7e5e09640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7d800ed70 0x7ff7d8007180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7ff7e5e09640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d804a0e0 con 0x7ff7d804eed0 2023-10-30T12:35:14.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7ff7e5e09640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d80d9610 con 0x7ff7d80d9230 2023-10-30T12:35:14.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7ff7e5e09640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d80df9e0 con 0x7ff7d800ed70 2023-10-30T12:35:14.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7f64ac66b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64a40c2290 0x7f64a41ea510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7ff7deffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff7d804eed0 0x7ff7d804ae60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:36038/0 (socket says 172.21.15.12:36038) 2023-10-30T12:35:14.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7ff7deffd640 1 -- 172.21.15.12:0/856289464 learned_addr learned my addr 172.21.15.12:0/856289464 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:14.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7ff7dffff640 1 --2- 172.21.15.12:0/856289464 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7d800ed70 0x7ff7d8007180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7f64ac66b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64a40c2700 0x7f64a414b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7ff7df7fe640 1 --2- 172.21.15.12:0/856289464 >> v2:172.21.15.12:3300/0 conn(0x7ff7d80d9230 0x7ff7d8049450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7ff7de7fc640 1 -- 172.21.15.12:0/856289464 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156004870 0 0) 0x7ff7d804a0e0 con 0x7ff7d804eed0 2023-10-30T12:35:14.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7ff7de7fc640 1 -- 172.21.15.12:0/856289464 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7d0003530 con 0x7ff7d804eed0 2023-10-30T12:35:14.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7ff7dffff640 1 -- 172.21.15.12:0/856289464 >> v2:172.21.15.12:3300/0 conn(0x7ff7d80d9230 msgr2=0x7ff7d8049450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7ff7dffff640 1 --2- 172.21.15.12:0/856289464 >> v2:172.21.15.12:3300/0 conn(0x7ff7d80d9230 0x7ff7d8049450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7ff7dffff640 1 -- 172.21.15.12:0/856289464 >> v1:172.21.15.154:6789/0 conn(0x7ff7d804eed0 legacy=0x7ff7d804ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7ff7dffff640 1 -- 172.21.15.12:0/856289464 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d804a380 con 0x7ff7d800ed70 2023-10-30T12:35:14.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7f5d277fe640 1 --2- 172.21.15.12:0/1234921316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d280cb050 0x7f5d280cf440 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5d280cce20 tx=0x7f5d1c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67ac9d814ccf7b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7f64aa407640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64a40c2290 0x7f64a41ea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7f64aa407640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64a40c2290 0x7f64a41ea510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40582/0 (socket says 172.21.15.12:40582) 2023-10-30T12:35:14.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7f64ac66b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64a414c140 0x7f64a414b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7f64aa407640 1 -- 172.21.15.12:0/2307174985 learned_addr learned my addr 172.21.15.12:0/2307174985 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7f64ac66b640 1 -- 172.21.15.12:0/2307174985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a4154ab0 con 0x7f64a414c140 2023-10-30T12:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7f64aac08640 1 --2- 172.21.15.12:0/2307174985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64a414c140 0x7f64a414b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7f64a9c06640 1 --2- 172.21.15.12:0/2307174985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64a40c2700 0x7f64a414b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7f64ac66b640 1 -- 172.21.15.12:0/2307174985 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a4153100 con 0x7f64a40c2700 2023-10-30T12:35:14.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.570+0000 7f64ac66b640 1 -- 172.21.15.12:0/2307174985 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a4154d00 con 0x7f64a40c2290 2023-10-30T12:35:14.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f64aac08640 1 -- 172.21.15.12:0/2307174985 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64a40c2290 msgr2=0x7f64a41ea510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f64aac08640 1 --2- 172.21.15.12:0/2307174985 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64a40c2290 0x7f64a41ea510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f64aac08640 1 -- 172.21.15.12:0/2307174985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64a40c2700 msgr2=0x7f64a414b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f64aac08640 1 --2- 172.21.15.12:0/2307174985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64a40c2700 0x7f64a414b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f5d26ffd640 1 -- 172.21.15.12:0/1234921316 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0ab600 con 0x7f5d280cb050 2023-10-30T12:35:14.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f5d26ffd640 1 -- 172.21.15.12:0/1234921316 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c0a7050 con 0x7f5d280cb050 2023-10-30T12:35:14.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f5d26ffd640 1 -- 172.21.15.12:0/1234921316 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0ab980 con 0x7f5d280cb050 2023-10-30T12:35:14.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f64aac08640 1 -- 172.21.15.12:0/2307174985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6494013a10 con 0x7f64a414c140 2023-10-30T12:35:14.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f64a9c06640 1 --2- 172.21.15.12:0/2307174985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64a40c2700 0x7f64a414b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:35:14.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f5d2d703640 1 -- 172.21.15.12:0/1234921316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d280cb050 msgr2=0x7f5d280cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f5d2d703640 1 --2- 172.21.15.12:0/1234921316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d280cb050 0x7f5d280cf440 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5d280cce20 tx=0x7f5d1c01c500 comp rx=0 tx=0).stop 2023-10-30T12:35:14.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f5d2d703640 1 -- 172.21.15.12:0/1234921316 shutdown_connections 2023-10-30T12:35:14.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f5d2d703640 1 --2- 172.21.15.12:0/1234921316 >> v2:172.21.15.154:3300/0 conn(0x7f5d280cab00 0x7f5d280cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f5d2d703640 1 --2- 172.21.15.12:0/1234921316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d280cb050 0x7f5d280cf440 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f5d2d703640 1 --2- 172.21.15.12:0/1234921316 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d280c5550 0x7f5d280cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f5d2d703640 1 -- 172.21.15.12:0/1234921316 >> 172.21.15.12:0/1234921316 conn(0x7f5d2806bed0 msgr2=0x7f5d2806c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f64aac08640 1 --2- 172.21.15.12:0/2307174985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64a414c140 0x7f64a414b8f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f64a406dd70 tx=0x7f6494024150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f649affd640 1 -- 172.21.15.12:0/2307174985 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494003b30 con 0x7f64a414c140 2023-10-30T12:35:14.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7ff7dffff640 1 --2- 172.21.15.12:0/856289464 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7d800ed70 0x7ff7d8007180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff7e002cb50 tx=0x7ff7e0099d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e107722fdbbb662 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f649affd640 1 -- 172.21.15.12:0/2307174985 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64940b0e90 con 0x7f64a414c140 2023-10-30T12:35:14.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f5d2d703640 1 -- 172.21.15.12:0/1234921316 shutdown_connections 2023-10-30T12:35:14.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f5d2d703640 1 -- 172.21.15.12:0/1234921316 wait complete. 2023-10-30T12:35:14.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f64ac66b640 1 -- 172.21.15.12:0/2307174985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a4154fe0 con 0x7f64a414c140 2023-10-30T12:35:14.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f64ac66b640 1 -- 172.21.15.12:0/2307174985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a41555a0 con 0x7f64a414c140 2023-10-30T12:35:14.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7f649affd640 1 -- 172.21.15.12:0/2307174985 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494003eb0 con 0x7f64a414c140 2023-10-30T12:35:14.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7ff7de7fc640 1 -- 172.21.15.12:0/856289464 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0072040 con 0x7ff7d800ed70 2023-10-30T12:35:14.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7ff7de7fc640 1 -- 172.21.15.12:0/856289464 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e0123ab0 con 0x7ff7d800ed70 2023-10-30T12:35:14.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.574+0000 7ff7de7fc640 1 -- 172.21.15.12:0/856289464 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e012b440 con 0x7ff7d800ed70 2023-10-30T12:35:14.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7fbeb9a9e640 1 Processor -- start 2023-10-30T12:35:14.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7f64ac66b640 1 -- 172.21.15.12:0/2307174985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648c005260 con 0x7f64a414c140 2023-10-30T12:35:14.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7f5d2d703640 1 Processor -- start 2023-10-30T12:35:14.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7f5d2d703640 1 -- start start 2023-10-30T12:35:14.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7ff7e5e09640 1 -- 172.21.15.12:0/856289464 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7d800ed70 msgr2=0x7ff7d8007180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7ff7e5e09640 1 --2- 172.21.15.12:0/856289464 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7d800ed70 0x7ff7d8007180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff7e002cb50 tx=0x7ff7e0099d40 comp rx=0 tx=0).stop 2023-10-30T12:35:14.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7ff7e5e09640 1 -- 172.21.15.12:0/856289464 shutdown_connections 2023-10-30T12:35:14.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7ff7e5e09640 1 --2- 172.21.15.12:0/856289464 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7d800ed70 0x7ff7d8007180 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7ff7e5e09640 1 --2- 172.21.15.12:0/856289464 >> v2:172.21.15.12:3300/0 conn(0x7ff7d80d9230 0x7ff7d8049450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7ff7e5e09640 1 -- 172.21.15.12:0/856289464 >> 172.21.15.12:0/856289464 conn(0x7ff7d800e170 msgr2=0x7ff7d800e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7f5d2d703640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d280c5550 0x7f5d2806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7f5d2d703640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d280cab00 0x7f5d280cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7f5d2d703640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d280cb050 0x7f5d280cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7f5d2d703640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d280cf0b0 con 0x7f5d280c5550 2023-10-30T12:35:14.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7f5d2d703640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d280c7370 con 0x7f5d280cb050 2023-10-30T12:35:14.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7f5d2d703640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d280ccf10 con 0x7f5d280cab00 2023-10-30T12:35:14.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7ff7e5e09640 1 -- 172.21.15.12:0/856289464 shutdown_connections 2023-10-30T12:35:14.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7ff7e5e09640 1 -- 172.21.15.12:0/856289464 wait complete. 2023-10-30T12:35:14.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7fbeb9a9e640 1 -- start start 2023-10-30T12:35:14.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7fbeb9a9e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbeb4153c60 0x7fbeb40ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7fbeb9a9e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbeb40c4430 0x7fbeb40cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7fbeb9a9e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbeb4159830 0x7fbeb4159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7fbeb9a9e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb4154040 con 0x7fbeb4159830 2023-10-30T12:35:14.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7fbeb9a9e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb40cb780 con 0x7fbeb4153c60 2023-10-30T12:35:14.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7fbeb9a9e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40cc9f0 con 0x7fbeb40c4430 2023-10-30T12:35:14.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.578+0000 7f649affd640 1 -- 172.21.15.12:0/2307174985 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f64940be040 con 0x7f64a414c140 2023-10-30T12:35:14.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f5d27fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d280c5550 0x7f5d2806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fe537394640 1 Processor -- start 2023-10-30T12:35:14.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f5d27fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d280c5550 0x7f5d2806f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44402/0 (socket says 172.21.15.12:44402) 2023-10-30T12:35:14.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f5d27fff640 1 -- 172.21.15.12:0/748223545 learned_addr learned my addr 172.21.15.12:0/748223545 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f5d277fe640 1 --2- 172.21.15.12:0/748223545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d280cab00 0x7f5d280cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fe537394640 1 -- start start 2023-10-30T12:35:14.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f5d2cf02640 1 --2- 172.21.15.12:0/748223545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d280cb050 0x7f5d280cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f5d27fff640 1 -- 172.21.15.12:0/748223545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d280cab00 msgr2=0x7f5d280cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f5d27fff640 1 --2- 172.21.15.12:0/748223545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d280cab00 0x7f5d280cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f5d27fff640 1 -- 172.21.15.12:0/748223545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d280cb050 msgr2=0x7f5d280cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f5d27fff640 1 --2- 172.21.15.12:0/748223545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d280cb050 0x7f5d280cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f5d27fff640 1 -- 172.21.15.12:0/748223545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d20009a70 con 0x7f5d280c5550 2023-10-30T12:35:14.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f5d2cf02640 1 --2- 172.21.15.12:0/748223545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d280cb050 0x7f5d280cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:14.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fbeb2ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbeb40c4430 0x7fbeb40cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f649affd640 1 --2- 172.21.15.12:0/2307174985 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f64880664f0 0x7f64880689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f649affd640 1 -- 172.21.15.12:0/2307174985 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f649412a380 con 0x7f64a414c140 2023-10-30T12:35:14.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fbeb2ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbeb40c4430 0x7fbeb40cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40602/0 (socket says 172.21.15.12:40602) 2023-10-30T12:35:14.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fbeb2ffd640 1 -- 172.21.15.12:0/1481779557 learned_addr learned my addr 172.21.15.12:0/1481779557 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f77c0ac9640 1 Processor -- start 2023-10-30T12:35:14.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f64aa407640 1 --2- 172.21.15.12:0/2307174985 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f64880664f0 0x7f64880689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fbeb3fff640 1 --2- 172.21.15.12:0/1481779557 >> v2:172.21.15.12:3300/0 conn(0x7fbeb4159830 0x7fbeb4159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7ff7e5e09640 1 Processor -- start 2023-10-30T12:35:14.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f5d27fff640 1 --2- 172.21.15.12:0/748223545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d280c5550 0x7f5d2806f2e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800d800 tx=0x7f5d1800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7ff7e5e09640 1 -- start start 2023-10-30T12:35:14.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f64aa407640 1 --2- 172.21.15.12:0/2307174985 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f64880664f0 0x7f64880689b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009a50 tx=0x7f64a0009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f77efb1d640 1 Processor -- start 2023-10-30T12:35:14.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f77efb1d640 1 -- start start 2023-10-30T12:35:14.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fbeb2ffd640 1 -- 172.21.15.12:0/1481779557 >> v2:172.21.15.154:3300/0 conn(0x7fbeb4153c60 msgr2=0x7fbeb40ca360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-30T12:35:14.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fbeb2ffd640 1 --2- 172.21.15.12:0/1481779557 >> v2:172.21.15.154:3300/0 conn(0x7fbeb4153c60 0x7fbeb40ca360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fbeb2ffd640 1 -- 172.21.15.12:0/1481779557 >> v2:172.21.15.12:3300/0 conn(0x7fbeb4159830 msgr2=0x7fbeb4159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fbeb2ffd640 1 --2- 172.21.15.12:0/1481779557 >> v2:172.21.15.12:3300/0 conn(0x7fbeb4159830 0x7fbeb4159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fbeb2ffd640 1 -- 172.21.15.12:0/1481779557 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb415a2e0 con 0x7fbeb40c4430 2023-10-30T12:35:14.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7fbeb3fff640 1 --2- 172.21.15.12:0/1481779557 >> v2:172.21.15.12:3300/0 conn(0x7fbeb4159830 0x7fbeb4159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:35:14.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fe537394640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe530071920 0x7fe53013f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fe537394640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe530071d80 0x7fe53013fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fe537394640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe530070010 con 0x7fe5300c08d0 2023-10-30T12:35:14.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fe537394640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe530072260 con 0x7fe530071d80 2023-10-30T12:35:14.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fe537394640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53006f6e0 con 0x7fe530071920 2023-10-30T12:35:14.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fe535130640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe5300c08d0 0x7fe5300c6b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:55430/0 (socket says 172.21.15.12:55430) 2023-10-30T12:35:14.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fe535130640 1 -- 172.21.15.12:0/599517547 learned_addr learned my addr 172.21.15.12:0/599517547 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:14.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fe53492f640 1 --2- 172.21.15.12:0/599517547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe530071920 0x7fe53013f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fe535931640 1 --2- 172.21.15.12:0/599517547 >> v2:172.21.15.12:3300/0 conn(0x7fe530071d80 0x7fe53013fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7f77c0ac9640 1 -- start start 2023-10-30T12:35:14.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77c0ac9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f77bc161be0 0x7f77bc06a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77c0ac9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77bc0c72b0 con 0x7f77bc0c6b40 2023-10-30T12:35:14.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77c0ac9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77bc0c6fd0 con 0x7f77bc158090 2023-10-30T12:35:14.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77c0ac9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77bc15dd30 con 0x7f77bc161be0 2023-10-30T12:35:14.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fe53492f640 1 -- 172.21.15.12:0/599517547 >> v2:172.21.15.12:3300/0 conn(0x7fe530071d80 msgr2=0x7fe53013fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fe53492f640 1 --2- 172.21.15.12:0/599517547 >> v2:172.21.15.12:3300/0 conn(0x7fe530071d80 0x7fe53013fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fe53492f640 1 -- 172.21.15.12:0/599517547 >> v1:172.21.15.12:6789/0 conn(0x7fe5300c08d0 legacy=0x7fe5300c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.582+0000 7fe53492f640 1 -- 172.21.15.12:0/599517547 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53006f9c0 con 0x7fe530071920 2023-10-30T12:35:14.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77bad9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f77bc161be0 0x7f77bc06a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b9d9b640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f77bc0c6b40 0x7f77bc0c0ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:55440/0 (socket says 172.21.15.12:55440) 2023-10-30T12:35:14.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b9d9b640 1 -- 172.21.15.12:0/2740319937 learned_addr learned my addr 172.21.15.12:0/2740319937 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:14.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77efb1d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f77e8158090 0x7f77e8161be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77efb1d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77e80c6b40 0x7f77e8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7e5e09640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7d800ed70 0x7ff7d8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7e5e09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7d804eed0 0x7ff7d81740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7e5e09640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7d80d9230 0x7ff7d80d0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7df7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7d800ed70 0x7ff7d8006c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7e5e09640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d8175030 con 0x7ff7d804eed0 2023-10-30T12:35:14.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7df7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7d800ed70 0x7ff7d8006c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58152/0 (socket says 172.21.15.12:58152) 2023-10-30T12:35:14.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7df7fe640 1 -- 172.21.15.12:0/1202388338 learned_addr learned my addr 172.21.15.12:0/1202388338 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7e5e09640 1 -- 172.21.15.12:0/1202388338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d8174a70 con 0x7ff7d800ed70 2023-10-30T12:35:14.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77efb1d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f77e806a3d0 0x7f77e806a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77efb1d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e8162120 con 0x7f77e806a3d0 2023-10-30T12:35:14.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77efb1d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e806ad60 con 0x7f77e8158090 2023-10-30T12:35:14.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77efb1d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e80c72b0 con 0x7f77e80c6b40 2023-10-30T12:35:14.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77ed8b9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f77e8158090 0x7f77e8161be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77ed8b9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f77e8158090 0x7f77e8161be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58166/0 (socket says 172.21.15.12:58166) 2023-10-30T12:35:14.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77ed8b9640 1 -- 172.21.15.12:0/2633044376 learned_addr learned my addr 172.21.15.12:0/2633044376 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7e5e09640 1 -- 172.21.15.12:0/1202388338 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d8174d50 con 0x7ff7d80d9230 2023-10-30T12:35:14.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7dffff640 1 --2- 172.21.15.12:0/1202388338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7d80d9230 0x7ff7d80d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7deffd640 1 --2- 172.21.15.12:0/1202388338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7d804eed0 0x7ff7d81740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77ed0b8640 1 --2- 172.21.15.12:0/2633044376 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77e80c6b40 0x7f77e8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b959a640 1 -- 172.21.15.12:0/2740319937 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899744175 0 0) 0x7f77bc0c6fd0 con 0x7f77bc158090 2023-10-30T12:35:14.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b959a640 1 -- 172.21.15.12:0/2740319937 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77a4003530 con 0x7f77bc158090 2023-10-30T12:35:14.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77ee0ba640 1 --2- 172.21.15.12:0/2633044376 >> v2:172.21.15.12:3300/0 conn(0x7f77e806a3d0 0x7f77e806a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77ed8b9640 1 -- 172.21.15.12:0/2633044376 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77e80c6b40 msgr2=0x7f77e8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77ed8b9640 1 --2- 172.21.15.12:0/2633044376 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77e80c6b40 0x7f77e8069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77ed8b9640 1 -- 172.21.15.12:0/2633044376 >> v2:172.21.15.12:3300/0 conn(0x7f77e806a3d0 msgr2=0x7f77e806a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77ed8b9640 1 --2- 172.21.15.12:0/2633044376 >> v2:172.21.15.12:3300/0 conn(0x7f77e806a3d0 0x7f77e806a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77ed8b9640 1 -- 172.21.15.12:0/2633044376 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e80c6fd0 con 0x7f77e8158090 2023-10-30T12:35:14.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7dffff640 1 -- 172.21.15.12:0/1202388338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7d800ed70 msgr2=0x7ff7d8006c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7dffff640 1 --2- 172.21.15.12:0/1202388338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7d800ed70 0x7ff7d8006c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7dffff640 1 -- 172.21.15.12:0/1202388338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7d804eed0 msgr2=0x7ff7d81740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7dffff640 1 --2- 172.21.15.12:0/1202388338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7d804eed0 0x7ff7d81740e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7fb0533d9640 1 Processor -- start 2023-10-30T12:35:14.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7fb0533d9640 1 -- start start 2023-10-30T12:35:14.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7ff7dffff640 1 -- 172.21.15.12:0/1202388338 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d80cf6b0 con 0x7ff7d80d9230 2023-10-30T12:35:14.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b959a640 1 -- 172.21.15.12:0/2740319937 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4047710857 0 0) 0x7f77a4003530 con 0x7f77bc158090 2023-10-30T12:35:14.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b959a640 1 -- 172.21.15.12:0/2740319937 >> v2:172.21.15.12:3300/0 conn(0x7f77bc161be0 msgr2=0x7f77bc06a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b959a640 1 --2- 172.21.15.12:0/2740319937 >> v2:172.21.15.12:3300/0 conn(0x7f77bc161be0 0x7f77bc06a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b959a640 1 -- 172.21.15.12:0/2740319937 >> v1:172.21.15.12:6789/0 conn(0x7f77bc0c6b40 legacy=0x7f77bc0c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b959a640 1 -- 172.21.15.12:0/2740319937 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc15e010 con 0x7f77bc158090 2023-10-30T12:35:14.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b959a640 1 -- 172.21.15.12:0/2740319937 <== mon.1 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f77ac001a00 con 0x7f77bc158090 2023-10-30T12:35:14.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b959a640 1 -- 172.21.15.12:0/2740319937 >> v1:172.21.15.154:6789/0 conn(0x7f77bc158090 legacy=0x7f77bc069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b959a640 1 --2- 172.21.15.12:0/2740319937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77a4003980 0x7f77a4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b959a640 1 --2- 172.21.15.12:0/2740319937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77a40047d0 0x7f77a4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b959a640 1 --2- 172.21.15.12:0/2740319937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77a4005190 0x7f77a400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b959a640 1 -- 172.21.15.12:0/2740319937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc15dd30 con 0x7f77a4005190 2023-10-30T12:35:14.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b959a640 1 -- 172.21.15.12:0/2740319937 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a400c040 con 0x7f77a40047d0 2023-10-30T12:35:14.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.586+0000 7f77b959a640 1 -- 172.21.15.12:0/2740319937 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a400c200 con 0x7f77a4003980 2023-10-30T12:35:14.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fbeb2ffd640 1 --2- 172.21.15.12:0/1481779557 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbeb40c4430 0x7fbeb40cc4b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4072ce0 tx=0x7fbea801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fab66d23d2b8d9d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7ff7dffff640 1 --2- 172.21.15.12:0/1202388338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7d80d9230 0x7ff7d80d0190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00682d0 tx=0x7ff7e0068c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f77b9d9b640 1 --2- 172.21.15.12:0/2740319937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77a40047d0 0x7f77a4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f77bad9d640 1 --2- 172.21.15.12:0/2740319937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77a4005190 0x7f77a400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f77ba59c640 1 --2- 172.21.15.12:0/2740319937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77a4003980 0x7f77a4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7ff7cbfff640 1 -- 172.21.15.12:0/1202388338 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e013b070 con 0x7ff7d80d9230 2023-10-30T12:35:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fbeb27fc640 1 -- 172.21.15.12:0/1481779557 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea80ac630 con 0x7fbeb40c4430 2023-10-30T12:35:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f77b9d9b640 1 -- 172.21.15.12:0/2740319937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77a4003980 msgr2=0x7f77a4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f77b9d9b640 1 --2- 172.21.15.12:0/2740319937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77a4003980 0x7f77a4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f77b9d9b640 1 -- 172.21.15.12:0/2740319937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77a4005190 msgr2=0x7f77a400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7ff7cbfff640 1 -- 172.21.15.12:0/1202388338 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e012baf0 con 0x7ff7d80d9230 2023-10-30T12:35:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7ff7cbfff640 1 -- 172.21.15.12:0/1202388338 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0134410 con 0x7ff7d80d9230 2023-10-30T12:35:14.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7ff7e5e09640 1 -- 172.21.15.12:0/1202388338 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d80cf990 con 0x7ff7d80d9230 2023-10-30T12:35:14.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7ff7e5e09640 1 -- 172.21.15.12:0/1202388338 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d80077c0 con 0x7ff7d80d9230 2023-10-30T12:35:14.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f77b9d9b640 1 --2- 172.21.15.12:0/2740319937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77a4005190 0x7f77a400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fbeb27fc640 1 -- 172.21.15.12:0/1481779557 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea80ac790 con 0x7fbeb40c4430 2023-10-30T12:35:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fb0533d9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb04c06aa00 0x7fb04c16b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fb0533d9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb04c0c5810 0x7fb04c070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f77b9d9b640 1 -- 172.21.15.12:0/2740319937 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc15e010 con 0x7f77a40047d0 2023-10-30T12:35:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fb0533d9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04c16b8f0 con 0x7fb04c072040 2023-10-30T12:35:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fb0533d9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04c0707d0 con 0x7fb04c06aa00 2023-10-30T12:35:14.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fb0533d9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04c072420 con 0x7fb04c0c5810 2023-10-30T12:35:14.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fbeb27fc640 1 -- 172.21.15.12:0/1481779557 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea80acb20 con 0x7fbeb40c4430 2023-10-30T12:35:14.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fbeb9a9e640 1 -- 172.21.15.12:0/1481779557 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbeb40c4430 msgr2=0x7fbeb40cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fbeb9a9e640 1 --2- 172.21.15.12:0/1481779557 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbeb40c4430 0x7fbeb40cc4b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4072ce0 tx=0x7fbea801c540 comp rx=0 tx=0).stop 2023-10-30T12:35:14.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f77bad9d640 1 --2- 172.21.15.12:0/2740319937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77a4005190 0x7f77a400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:35:14.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f77bad9d640 1 -- 172.21.15.12:0/2740319937 reap_dead start 2023-10-30T12:35:14.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fbeb9a9e640 1 -- 172.21.15.12:0/1481779557 shutdown_connections 2023-10-30T12:35:14.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fbeb9a9e640 1 --2- 172.21.15.12:0/1481779557 >> v2:172.21.15.12:3300/0 conn(0x7fbeb4159830 0x7fbeb4159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fbeb9a9e640 1 --2- 172.21.15.12:0/1481779557 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbeb40c4430 0x7fbeb40cc4b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fbeb9a9e640 1 --2- 172.21.15.12:0/1481779557 >> v2:172.21.15.154:3300/0 conn(0x7fbeb4153c60 0x7fbeb40ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fbeb9a9e640 1 -- 172.21.15.12:0/1481779557 >> 172.21.15.12:0/1481779557 conn(0x7fbeb406bfe0 msgr2=0x7fbeb406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fb050974640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb04c06aa00 0x7fb04c16b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fb050974640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb04c06aa00 0x7fb04c16b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44464/0 (socket says 172.21.15.12:44464) 2023-10-30T12:35:14.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fb050974640 1 -- 172.21.15.12:0/1144940224 learned_addr learned my addr 172.21.15.12:0/1144940224 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fbeb9a9e640 1 -- 172.21.15.12:0/1481779557 shutdown_connections 2023-10-30T12:35:14.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f5d24ff9640 1 -- 172.21.15.12:0/748223545 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d18004030 con 0x7f5d280c5550 2023-10-30T12:35:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f5d24ff9640 1 -- 172.21.15.12:0/748223545 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d18004190 con 0x7f5d280c5550 2023-10-30T12:35:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f5d24ff9640 1 -- 172.21.15.12:0/748223545 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d18004520 con 0x7f5d280c5550 2023-10-30T12:35:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f5d2d703640 1 -- 172.21.15.12:0/748223545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1c009480 con 0x7f5d280c5550 2023-10-30T12:35:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fb051976640 1 --2- 172.21.15.12:0/1144940224 >> v2:172.21.15.154:3300/0 conn(0x7fb04c0c5810 0x7fb04c070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fbeb9a9e640 1 -- 172.21.15.12:0/1481779557 wait complete. 2023-10-30T12:35:14.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f5d2d703640 1 -- 172.21.15.12:0/748223545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d281fa6d0 con 0x7f5d280c5550 2023-10-30T12:35:14.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fe53492f640 1 --2- 172.21.15.12:0/599517547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe530071920 0x7fe53013f600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0097c0 tx=0x7fe52c01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=885f8edf51bdc0e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7f649affd640 1 -- 172.21.15.12:0/2307174985 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f64940f7b00 con 0x7f64a414c140 2023-10-30T12:35:14.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7fb050974640 1 -- 172.21.15.12:0/1144940224 >> v2:172.21.15.154:3300/0 conn(0x7fb04c0c5810 msgr2=0x7fb04c070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7fb050974640 1 --2- 172.21.15.12:0/1144940224 >> v2:172.21.15.154:3300/0 conn(0x7fb04c0c5810 0x7fb04c070bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7fb050974640 1 -- 172.21.15.12:0/1144940224 >> v1:172.21.15.12:6789/0 conn(0x7fb04c072040 legacy=0x7fb04c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:14.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7f77ed8b9640 1 --2- 172.21.15.12:0/2633044376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77e8158090 0x7f77e8161be0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f77e8072640 tx=0x7f77e401ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=264d783136e67bc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7fb050974640 1 -- 172.21.15.12:0/1144940224 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c0c5de0 con 0x7fb04c06aa00 2023-10-30T12:35:14.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7fe527fff640 1 -- 172.21.15.12:0/599517547 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c0ad640 con 0x7fe530071920 2023-10-30T12:35:14.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7fe527fff640 1 -- 172.21.15.12:0/599517547 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52c0ad7a0 con 0x7fe530071920 2023-10-30T12:35:14.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7fe527fff640 1 -- 172.21.15.12:0/599517547 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c0adb30 con 0x7fe530071920 2023-10-30T12:35:14.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7f77ec8b7640 1 -- 172.21.15.12:0/2633044376 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e40a89b0 con 0x7f77e8158090 2023-10-30T12:35:14.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7f77ec8b7640 1 -- 172.21.15.12:0/2633044376 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e40a8b10 con 0x7f77e8158090 2023-10-30T12:35:14.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7f77ec8b7640 1 -- 172.21.15.12:0/2633044376 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e40a8ea0 con 0x7f77e8158090 2023-10-30T12:35:14.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7fe537394640 1 -- 172.21.15.12:0/599517547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe530071920 msgr2=0x7fe53013f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7fe537394640 1 --2- 172.21.15.12:0/599517547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe530071920 0x7fe53013f600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0097c0 tx=0x7fe52c01c480 comp rx=0 tx=0).stop 2023-10-30T12:35:14.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7f77efb1d640 1 -- 172.21.15.12:0/2633044376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77e8158090 msgr2=0x7f77e8161be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7f77efb1d640 1 --2- 172.21.15.12:0/2633044376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77e8158090 0x7f77e8161be0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f77e8072640 tx=0x7f77e401ecf0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.590+0000 7ff7c9ffb640 1 -- 172.21.15.12:0/1202388338 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b8005260 con 0x7ff7d80d9230 2023-10-30T12:35:14.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7fbeb9a9e640 1 Processor -- start 2023-10-30T12:35:14.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7fe537394640 1 -- 172.21.15.12:0/599517547 shutdown_connections 2023-10-30T12:35:14.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7fe537394640 1 --2- 172.21.15.12:0/599517547 >> v2:172.21.15.12:3300/0 conn(0x7fe530071d80 0x7fe53013fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7fe537394640 1 --2- 172.21.15.12:0/599517547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe530071920 0x7fe53013f600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7fe537394640 1 -- 172.21.15.12:0/599517547 >> 172.21.15.12:0/599517547 conn(0x7fe5300c7910 msgr2=0x7fe5300c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7f77efb1d640 1 -- 172.21.15.12:0/2633044376 shutdown_connections 2023-10-30T12:35:14.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7f77efb1d640 1 --2- 172.21.15.12:0/2633044376 >> v2:172.21.15.12:3300/0 conn(0x7f77e806a3d0 0x7f77e806a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7f77efb1d640 1 --2- 172.21.15.12:0/2633044376 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77e80c6b40 0x7f77e8069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7f77efb1d640 1 --2- 172.21.15.12:0/2633044376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77e8158090 0x7f77e8161be0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7f77efb1d640 1 -- 172.21.15.12:0/2633044376 >> 172.21.15.12:0/2633044376 conn(0x7f77e80c7910 msgr2=0x7f77e80c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7f5d2d703640 1 -- 172.21.15.12:0/748223545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d08005260 con 0x7f5d280c5550 2023-10-30T12:35:14.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7fbeb9a9e640 1 -- start start 2023-10-30T12:35:14.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7f77efb1d640 1 -- 172.21.15.12:0/2633044376 shutdown_connections 2023-10-30T12:35:14.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7f77efb1d640 1 -- 172.21.15.12:0/2633044376 wait complete. 2023-10-30T12:35:14.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7ff7cbfff640 1 -- 172.21.15.12:0/1202388338 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff7e0132040 con 0x7ff7d80d9230 2023-10-30T12:35:14.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7ff7cbfff640 1 --2- 172.21.15.12:0/1202388338 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff7c40664d0 0x7ff7c4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb9a9e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbeb40c4430 0x7fbeb4141540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb9a9e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbeb4153c60 0x7fbeb4158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.594+0000 7f5d24ff9640 1 -- 172.21.15.12:0/748223545 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5d180096a0 con 0x7f5d280c5550 2023-10-30T12:35:14.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb9a9e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbeb4159830 0x7fbeb4158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb9a9e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40cd920 con 0x7fbeb4159830 2023-10-30T12:35:14.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb9a9e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb4147730 con 0x7fbeb4153c60 2023-10-30T12:35:14.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb9a9e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb4145be0 con 0x7fbeb40c4430 2023-10-30T12:35:14.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb37fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbeb40c4430 0x7fbeb4141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb37fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbeb40c4430 0x7fbeb4141540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40644/0 (socket says 172.21.15.12:40644) 2023-10-30T12:35:14.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb37fe640 1 -- 172.21.15.12:0/118899028 learned_addr learned my addr 172.21.15.12:0/118899028 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fb050974640 1 --2- 172.21.15.12:0/1144940224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04c06aa00 0x7fb04c16b3b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb04c06b550 tx=0x7fb03c01ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0af8326c2db4ea4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7ff7df7fe640 1 --2- 172.21.15.12:0/1202388338 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff7c40664d0 0x7ff7c4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f5d24ff9640 1 --2- 172.21.15.12:0/748223545 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5d140664d0 0x7f5d14068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7ff7cbfff640 1 -- 172.21.15.12:0/1202388338 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7e016d590 con 0x7ff7d80d9230 2023-10-30T12:35:14.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f5d24ff9640 1 -- 172.21.15.12:0/748223545 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d18084a40 con 0x7f5d280c5550 2023-10-30T12:35:14.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f77efb1d640 1 Processor -- start 2023-10-30T12:35:14.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb3fff640 1 --2- 172.21.15.12:0/118899028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbeb4159830 0x7fbeb4158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb2ffd640 1 --2- 172.21.15.12:0/118899028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbeb4153c60 0x7fbeb4158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f77efb1d640 1 -- start start 2023-10-30T12:35:14.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fe537394640 1 -- 172.21.15.12:0/599517547 shutdown_connections 2023-10-30T12:35:14.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb37fe640 1 -- 172.21.15.12:0/118899028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbeb4153c60 msgr2=0x7fbeb4158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb37fe640 1 --2- 172.21.15.12:0/118899028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbeb4153c60 0x7fbeb4158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb37fe640 1 -- 172.21.15.12:0/118899028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbeb4159830 msgr2=0x7fbeb4158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb37fe640 1 --2- 172.21.15.12:0/118899028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbeb4159830 0x7fbeb4158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb37fe640 1 -- 172.21.15.12:0/118899028 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea80094d0 con 0x7fbeb40c4430 2023-10-30T12:35:14.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fe537394640 1 -- 172.21.15.12:0/599517547 wait complete. 2023-10-30T12:35:14.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f77b9d9b640 1 --2- 172.21.15.12:0/2740319937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77a40047d0 0x7f77a4004c50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f77b0001850 tx=0x7f77b001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2ea404db6a19753 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f5d277fe640 1 --2- 172.21.15.12:0/748223545 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5d140664d0 0x7f5d14068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7ff7df7fe640 1 --2- 172.21.15.12:0/1202388338 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff7c40664d0 0x7ff7c4068990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4007a70 tx=0x7ff7d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f77b959a640 1 -- 172.21.15.12:0/2740319937 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0004ab0 con 0x7f77a40047d0 2023-10-30T12:35:14.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7fbeb37fe640 1 --2- 172.21.15.12:0/118899028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbeb40c4430 0x7fbeb4141540 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00ba80 tx=0x7fbeac00bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f77b959a640 1 -- 172.21.15.12:0/2740319937 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b0004c10 con 0x7f77a40047d0 2023-10-30T12:35:14.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f77b959a640 1 -- 172.21.15.12:0/2740319937 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0004ab0 con 0x7f77a40047d0 2023-10-30T12:35:14.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f77efb1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77e806a3d0 0x7f77e8154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f77efb1d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77e8158090 0x7f77e80c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f77efb1d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77e81533b0 0x7f77e81f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f5d277fe640 1 --2- 172.21.15.12:0/748223545 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5d140664d0 0x7f5d14068990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0ad000 tx=0x7f5d1c004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f77efb1d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e80c1360 con 0x7f77e806a3d0 2023-10-30T12:35:14.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f77efb1d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e80c1600 con 0x7f77e81533b0 2023-10-30T12:35:14.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.598+0000 7f77efb1d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e8153ce0 con 0x7f77e8158090 2023-10-30T12:35:14.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77ed8b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77e806a3d0 0x7f77e8154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77ed8b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77e806a3d0 0x7f77e8154d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44494/0 (socket says 172.21.15.12:44494) 2023-10-30T12:35:14.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77ed8b9640 1 -- 172.21.15.12:0/3227293625 learned_addr learned my addr 172.21.15.12:0/3227293625 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77ed0b8640 1 --2- 172.21.15.12:0/3227293625 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77e8158090 0x7f77e80c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77ee0ba640 1 --2- 172.21.15.12:0/3227293625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77e81533b0 0x7f77e81f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe537394640 1 Processor -- start 2023-10-30T12:35:14.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77c0ac9640 1 -- 172.21.15.12:0/2740319937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77a40047d0 msgr2=0x7f77a4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77c0ac9640 1 --2- 172.21.15.12:0/2740319937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77a40047d0 0x7f77a4004c50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f77b0001850 tx=0x7f77b001c4a0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe537394640 1 -- start start 2023-10-30T12:35:14.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77ed8b9640 1 -- 172.21.15.12:0/3227293625 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77e8158090 msgr2=0x7f77e80c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77ed8b9640 1 --2- 172.21.15.12:0/3227293625 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77e8158090 0x7f77e80c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77ed8b9640 1 -- 172.21.15.12:0/3227293625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77e81533b0 msgr2=0x7f77e81f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77ed8b9640 1 --2- 172.21.15.12:0/3227293625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77e81533b0 0x7f77e81f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77ed8b9640 1 -- 172.21.15.12:0/3227293625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e40094d0 con 0x7f77e806a3d0 2023-10-30T12:35:14.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77c0ac9640 1 -- 172.21.15.12:0/2740319937 shutdown_connections 2023-10-30T12:35:14.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77c0ac9640 1 --2- 172.21.15.12:0/2740319937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77a40047d0 0x7f77a4004c50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77c0ac9640 1 -- 172.21.15.12:0/2740319937 >> 172.21.15.12:0/2740319937 conn(0x7f77bc0c7910 msgr2=0x7f77bc0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77c0ac9640 1 -- 172.21.15.12:0/2740319937 shutdown_connections 2023-10-30T12:35:14.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77c0ac9640 1 -- 172.21.15.12:0/2740319937 wait complete. 2023-10-30T12:35:14.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe537394640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe530071920 0x7fe5301587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe537394640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe530071d80 0x7fe530158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe537394640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5300c08d0 0x7fe5300c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe537394640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300c3660 con 0x7fe5300c08d0 2023-10-30T12:35:14.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe537394640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300b0840 con 0x7fe530071d80 2023-10-30T12:35:14.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe537394640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe530155850 con 0x7fe530071920 2023-10-30T12:35:14.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe535931640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5300c08d0 0x7fe5300c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe535931640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5300c08d0 0x7fe5300c3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44510/0 (socket says 172.21.15.12:44510) 2023-10-30T12:35:14.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe535931640 1 -- 172.21.15.12:0/1468957130 learned_addr learned my addr 172.21.15.12:0/1468957130 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77ed8b9640 1 --2- 172.21.15.12:0/3227293625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77e806a3d0 0x7f77e8154d80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f77e401f590 tx=0x7f77e401ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe53492f640 1 --2- 172.21.15.12:0/1468957130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe530071d80 0x7fe530158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe535931640 1 -- 172.21.15.12:0/1468957130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe530071920 msgr2=0x7fe5301587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe535931640 1 --2- 172.21.15.12:0/1468957130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe530071920 0x7fe5301587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe535931640 1 -- 172.21.15.12:0/1468957130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe530071d80 msgr2=0x7fe530158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe535931640 1 --2- 172.21.15.12:0/1468957130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe530071d80 0x7fe530158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fb043fff640 1 -- 172.21.15.12:0/1144940224 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0a89b0 con 0x7fb04c06aa00 2023-10-30T12:35:14.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe535931640 1 -- 172.21.15.12:0/1468957130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52c009420 con 0x7fe5300c08d0 2023-10-30T12:35:14.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe53492f640 1 --2- 172.21.15.12:0/1468957130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe530071d80 0x7fe530158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:35:14.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f5d24ff9640 1 -- 172.21.15.12:0/748223545 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5d18051fa0 con 0x7f5d280c5550 2023-10-30T12:35:14.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fb043fff640 1 -- 172.21.15.12:0/1144940224 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c0a8b10 con 0x7fb04c06aa00 2023-10-30T12:35:14.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77de7fc640 1 -- 172.21.15.12:0/3227293625 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e40b1db0 con 0x7f77e806a3d0 2023-10-30T12:35:14.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fb043fff640 1 -- 172.21.15.12:0/1144940224 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0a8ea0 con 0x7fb04c06aa00 2023-10-30T12:35:14.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77de7fc640 1 -- 172.21.15.12:0/3227293625 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e40b9440 con 0x7f77e806a3d0 2023-10-30T12:35:14.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77de7fc640 1 -- 172.21.15.12:0/3227293625 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e40b97d0 con 0x7f77e806a3d0 2023-10-30T12:35:14.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77efb1d640 1 -- 172.21.15.12:0/3227293625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e8153fc0 con 0x7f77e806a3d0 2023-10-30T12:35:14.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7f77efb1d640 1 -- 172.21.15.12:0/3227293625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e8154580 con 0x7f77e806a3d0 2023-10-30T12:35:14.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.602+0000 7fe535931640 1 --2- 172.21.15.12:0/1468957130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5300c08d0 0x7fe5300c3a80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe528007de0 tx=0x7fe52800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7f77c0ac9640 1 Processor -- start 2023-10-30T12:35:14.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7fb0533d9640 1 -- 172.21.15.12:0/1144940224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04c06aa00 msgr2=0x7fb04c16b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7fb0533d9640 1 --2- 172.21.15.12:0/1144940224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04c06aa00 0x7fb04c16b3b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb04c06b550 tx=0x7fb03c01ecb0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7fe525ffb640 1 -- 172.21.15.12:0/1468957130 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe528014620 con 0x7fe5300c08d0 2023-10-30T12:35:14.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7fe525ffb640 1 -- 172.21.15.12:0/1468957130 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52800ee90 con 0x7fe5300c08d0 2023-10-30T12:35:14.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7fe525ffb640 1 -- 172.21.15.12:0/1468957130 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5280149c0 con 0x7fe5300c08d0 2023-10-30T12:35:14.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7f77c0ac9640 1 -- start start 2023-10-30T12:35:14.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7fb0533d9640 1 -- 172.21.15.12:0/1144940224 shutdown_connections 2023-10-30T12:35:14.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7fb0533d9640 1 --2- 172.21.15.12:0/1144940224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04c06aa00 0x7fb04c16b3b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7fb0533d9640 1 --2- 172.21.15.12:0/1144940224 >> v2:172.21.15.154:3300/0 conn(0x7fb04c0c5810 0x7fb04c070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7fb0533d9640 1 -- 172.21.15.12:0/1144940224 >> 172.21.15.12:0/1144940224 conn(0x7fb04c0c7910 msgr2=0x7fb04c0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7f77efb1d640 1 -- 172.21.15.12:0/3227293625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e80b0eb0 con 0x7f77e806a3d0 2023-10-30T12:35:14.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7fe537394640 1 -- 172.21.15.12:0/1468957130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe530155b10 con 0x7fe5300c08d0 2023-10-30T12:35:14.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7fe537394640 1 -- 172.21.15.12:0/1468957130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5300c2cc0 con 0x7fe5300c08d0 2023-10-30T12:35:14.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7fb0533d9640 1 -- 172.21.15.12:0/1144940224 shutdown_connections 2023-10-30T12:35:14.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7f77c0ac9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77a40047d0 0x7f77bc1f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7f77c0ac9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77bc1525b0 0x7f77bc155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7f77de7fc640 1 -- 172.21.15.12:0/3227293625 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f77e40b7070 con 0x7f77e806a3d0 2023-10-30T12:35:14.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7f77c0ac9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77bc156430 0x7f77bc160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7fe537394640 1 -- 172.21.15.12:0/1468957130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5300c37a0 con 0x7fe5300c08d0 2023-10-30T12:35:14.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7f77c0ac9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc1f3cf0 con 0x7f77a40047d0 2023-10-30T12:35:14.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7f77c0ac9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc156c10 con 0x7f77bc156430 2023-10-30T12:35:14.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7f77c0ac9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc0c63d0 con 0x7f77bc1525b0 2023-10-30T12:35:14.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7f77b9d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77bc1525b0 0x7f77bc155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7f77b9d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77bc1525b0 0x7f77bc155ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40666/0 (socket says 172.21.15.12:40666) 2023-10-30T12:35:14.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7f77b9d9b640 1 -- 172.21.15.12:0/721304642 learned_addr learned my addr 172.21.15.12:0/721304642 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7fb0533d9640 1 -- 172.21.15.12:0/1144940224 wait complete. 2023-10-30T12:35:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77de7fc640 1 --2- 172.21.15.12:0/3227293625 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f77cc0665a0 0x7f77cc068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77de7fc640 1 -- 172.21.15.12:0/3227293625 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77e4121640 con 0x7f77e806a3d0 2023-10-30T12:35:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.606+0000 7f77bad9d640 1 --2- 172.21.15.12:0/721304642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77bc156430 0x7f77bc160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77bad9d640 1 -- 172.21.15.12:0/721304642 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77bc1525b0 msgr2=0x7f77bc155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77bad9d640 1 --2- 172.21.15.12:0/721304642 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77bc1525b0 0x7f77bc155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77bad9d640 1 -- 172.21.15.12:0/721304642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77a40047d0 msgr2=0x7f77bc1f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77bad9d640 1 --2- 172.21.15.12:0/721304642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77a40047d0 0x7f77bc1f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77bad9d640 1 -- 172.21.15.12:0/721304642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc156930 con 0x7f77bc156430 2023-10-30T12:35:14.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77b9d9b640 1 --2- 172.21.15.12:0/721304642 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77bc1525b0 0x7f77bc155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:35:14.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77bad9d640 1 --2- 172.21.15.12:0/721304642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77bc156430 0x7f77bc160a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f77bc070680 tx=0x7f77b4011e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7fb0533d9640 1 Processor -- start 2023-10-30T12:35:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77ed0b8640 1 --2- 172.21.15.12:0/3227293625 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f77cc0665a0 0x7f77cc068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77aaffd640 1 -- 172.21.15.12:0/721304642 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b401b070 con 0x7f77bc156430 2023-10-30T12:35:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77aaffd640 1 -- 172.21.15.12:0/721304642 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b4004020 con 0x7f77bc156430 2023-10-30T12:35:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77aaffd640 1 -- 172.21.15.12:0/721304642 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40041d0 con 0x7f77bc156430 2023-10-30T12:35:14.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7fe525ffb640 1 -- 172.21.15.12:0/1468957130 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe528014b20 con 0x7fe5300c08d0 2023-10-30T12:35:14.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77c0ac9640 1 -- 172.21.15.12:0/721304642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77bc0c5610 con 0x7f77bc156430 2023-10-30T12:35:14.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77c0ac9640 1 -- 172.21.15.12:0/721304642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77bc0c5bd0 con 0x7f77bc156430 2023-10-30T12:35:14.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7fb0533d9640 1 -- start start 2023-10-30T12:35:14.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77ed0b8640 1 --2- 172.21.15.12:0/3227293625 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f77cc0665a0 0x7f77cc068a60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f77e81f7a10 tx=0x7f77d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7f77c0ac9640 1 -- 172.21.15.12:0/721304642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77bc04e170 con 0x7f77bc156430 2023-10-30T12:35:14.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7fe525ffb640 1 --2- 172.21.15.12:0/1468957130 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe5140664f0 0x7fe5140689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7fe525ffb640 1 -- 172.21.15.12:0/1468957130 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe528085630 con 0x7fe5300c08d0 2023-10-30T12:35:14.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.610+0000 7fe535130640 1 --2- 172.21.15.12:0/1468957130 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe5140664f0 0x7fe5140689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb0533d9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb04c06aa00 0x7fb04c1f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb0533d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04c072040 0x7fb04c1f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb051175640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb04c06aa00 0x7fb04c1f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb0533d9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb04c0c5810 0x7fb04c0c3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb0533d9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0c4900 con 0x7fb04c072040 2023-10-30T12:35:14.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb0533d9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0c3cf0 con 0x7fb04c06aa00 2023-10-30T12:35:14.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb0533d9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0c3fd0 con 0x7fb04c0c5810 2023-10-30T12:35:14.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb051175640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb04c06aa00 0x7fb04c1f16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58238/0 (socket says 172.21.15.12:58238) 2023-10-30T12:35:14.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb051175640 1 -- 172.21.15.12:0/57347114 learned_addr learned my addr 172.21.15.12:0/57347114 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:14.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb051976640 1 --2- 172.21.15.12:0/57347114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb04c0c5810 0x7fb04c0c3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb050974640 1 --2- 172.21.15.12:0/57347114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04c072040 0x7fb04c1f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fbe9bfff640 1 -- 172.21.15.12:0/118899028 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac00cc90 con 0x7fbeb40c4430 2023-10-30T12:35:14.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fbe9bfff640 1 -- 172.21.15.12:0/118899028 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac00cdf0 con 0x7fbeb40c4430 2023-10-30T12:35:14.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fbe9bfff640 1 -- 172.21.15.12:0/118899028 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac014680 con 0x7fbeb40c4430 2023-10-30T12:35:14.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb051175640 1 -- 172.21.15.12:0/57347114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb04c0c5810 msgr2=0x7fb04c0c3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb051175640 1 --2- 172.21.15.12:0/57347114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb04c0c5810 0x7fb04c0c3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb051175640 1 -- 172.21.15.12:0/57347114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04c072040 msgr2=0x7fb04c1f1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb051175640 1 --2- 172.21.15.12:0/57347114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04c072040 0x7fb04c1f1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb051175640 1 -- 172.21.15.12:0/57347114 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c0094d0 con 0x7fb04c06aa00 2023-10-30T12:35:14.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7ff7cbfff640 1 -- 172.21.15.12:0/1202388338 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff7e0169960 con 0x7ff7d80d9230 2023-10-30T12:35:14.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fb051175640 1 --2- 172.21.15.12:0/57347114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb04c06aa00 0x7fb04c1f16d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb04800c490 tx=0x7fb048001e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fbeb9a9e640 1 -- 172.21.15.12:0/118899028 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb4147960 con 0x7fbeb40c4430 2023-10-30T12:35:14.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fe535130640 1 --2- 172.21.15.12:0/1468957130 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe5140664f0 0x7fe5140689b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe520000f70 tx=0x7fe520007a90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7fbeb9a9e640 1 -- 172.21.15.12:0/118899028 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb40cd390 con 0x7fbeb40c4430 2023-10-30T12:35:14.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7f77aaffd640 1 -- 172.21.15.12:0/721304642 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f77b4004330 con 0x7f77bc156430 2023-10-30T12:35:14.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.614+0000 7f77aaffd640 1 --2- 172.21.15.12:0/721304642 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f779c0664f0 0x7f779c0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.618+0000 7fe525ffb640 1 -- 172.21.15.12:0/1468957130 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe528052a90 con 0x7fe5300c08d0 2023-10-30T12:35:14.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.618+0000 7fbeb9a9e640 1 -- 172.21.15.12:0/118899028 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb404e170 con 0x7fbeb40c4430 2023-10-30T12:35:14.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.618+0000 7f77aaffd640 1 -- 172.21.15.12:0/721304642 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77b40844a0 con 0x7f77bc156430 2023-10-30T12:35:14.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.618+0000 7f77ba59c640 1 --2- 172.21.15.12:0/721304642 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f779c0664f0 0x7f779c0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.618+0000 7fbe9bfff640 1 -- 172.21.15.12:0/118899028 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbeac007370 con 0x7fbeb40c4430 2023-10-30T12:35:14.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.618+0000 7f77ba59c640 1 --2- 172.21.15.12:0/721304642 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f779c0664f0 0x7f779c0689b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f77ac007e70 tx=0x7f77ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.622+0000 7fbe9bfff640 1 --2- 172.21.15.12:0/118899028 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbea0066520 0x7fbea00689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.622+0000 7f77de7fc640 1 -- 172.21.15.12:0/3227293625 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f77e40eeb50 con 0x7f77e806a3d0 2023-10-30T12:35:14.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.622+0000 7fbe9bfff640 1 -- 172.21.15.12:0/118899028 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbeac0838d0 con 0x7fbeb40c4430 2023-10-30T12:35:14.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.622+0000 7fbeb2ffd640 1 --2- 172.21.15.12:0/118899028 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbea0066520 0x7fbea00689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.622+0000 7fbeb2ffd640 1 --2- 172.21.15.12:0/118899028 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbea0066520 0x7fbea00689e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbea80ae000 tx=0x7fbea8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.626+0000 7f77aaffd640 1 -- 172.21.15.12:0/721304642 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f77b4051970 con 0x7f77bc156430 2023-10-30T12:35:14.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.626+0000 7fb041ffb640 1 -- 172.21.15.12:0/57347114 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048003600 con 0x7fb04c06aa00 2023-10-30T12:35:14.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.626+0000 7fb041ffb640 1 -- 172.21.15.12:0/57347114 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb048003760 con 0x7fb04c06aa00 2023-10-30T12:35:14.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.626+0000 7fb041ffb640 1 -- 172.21.15.12:0/57347114 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048003b50 con 0x7fb04c06aa00 2023-10-30T12:35:14.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.626+0000 7fb0533d9640 1 -- 172.21.15.12:0/57347114 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04c0c42b0 con 0x7fb04c06aa00 2023-10-30T12:35:14.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.630+0000 7fb0533d9640 1 -- 172.21.15.12:0/57347114 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c1f2380 con 0x7fb04c06aa00 2023-10-30T12:35:14.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.630+0000 7fb0533d9640 1 -- 172.21.15.12:0/57347114 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04c066cf0 con 0x7fb04c06aa00 2023-10-30T12:35:14.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.634+0000 7fb041ffb640 1 -- 172.21.15.12:0/57347114 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb048003cb0 con 0x7fb04c06aa00 2023-10-30T12:35:14.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.634+0000 7fbe9bfff640 1 -- 172.21.15.12:0/118899028 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbeac050e70 con 0x7fbeb40c4430 2023-10-30T12:35:14.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.634+0000 7fb041ffb640 1 --2- 172.21.15.12:0/57347114 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb030066520 0x7fb0300689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:14.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.634+0000 7fb041ffb640 1 -- 172.21.15.12:0/57347114 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0480834d0 con 0x7fb04c06aa00 2023-10-30T12:35:14.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.634+0000 7fb050974640 1 --2- 172.21.15.12:0/57347114 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb030066520 0x7fb0300689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:14.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.638+0000 7fb050974640 1 --2- 172.21.15.12:0/57347114 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb030066520 0x7fb0300689e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0b2000 tx=0x7fb03c0aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:14.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.646+0000 7fb041ffb640 1 -- 172.21.15.12:0/57347114 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb048050930 con 0x7fb04c06aa00 2023-10-30T12:35:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.918+0000 7f5d2d703640 1 -- 172.21.15.12:0/748223545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5d080057e0 con 0x7f5d280c5550 2023-10-30T12:35:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.926+0000 7f64ac66b640 1 -- 172.21.15.12:0/2307174985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f648c0050d0 con 0x7f64a414c140 2023-10-30T12:35:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.926+0000 7f649affd640 1 -- 172.21.15.12:0/2307174985 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64940bc090 con 0x7f64a414c140 2023-10-30T12:35:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.926+0000 7f5d24ff9640 1 -- 172.21.15.12:0/748223545 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5d18018090 con 0x7f5d280c5550 2023-10-30T12:35:14.931 INFO:teuthology.orchestra.run.smithi012.stdout:60129542150 2023-10-30T12:35:14.932 INFO:teuthology.orchestra.run.smithi012.stdout:60129542152 2023-10-30T12:35:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f64ac66b640 1 -- 172.21.15.12:0/2307174985 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f64880664f0 msgr2=0x7f64880689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f64ac66b640 1 --2- 172.21.15.12:0/2307174985 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f64880664f0 0x7f64880689b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009a50 tx=0x7f64a0009210 comp rx=0 tx=0).stop 2023-10-30T12:35:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f64ac66b640 1 -- 172.21.15.12:0/2307174985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64a414c140 msgr2=0x7f64a414b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f64ac66b640 1 --2- 172.21.15.12:0/2307174985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64a414c140 0x7f64a414b8f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f64a406dd70 tx=0x7f6494024150 comp rx=0 tx=0).stop 2023-10-30T12:35:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f5d2d703640 1 -- 172.21.15.12:0/748223545 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5d140664d0 msgr2=0x7f5d14068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f5d2d703640 1 --2- 172.21.15.12:0/748223545 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5d140664d0 0x7f5d14068990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0ad000 tx=0x7f5d1c004740 comp rx=0 tx=0).stop 2023-10-30T12:35:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f5d2d703640 1 -- 172.21.15.12:0/748223545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d280c5550 msgr2=0x7f5d2806f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f5d2d703640 1 --2- 172.21.15.12:0/748223545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d280c5550 0x7f5d2806f2e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800d800 tx=0x7f5d1800dcc0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f64ac66b640 1 -- 172.21.15.12:0/2307174985 shutdown_connections 2023-10-30T12:35:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f64ac66b640 1 --2- 172.21.15.12:0/2307174985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64a414c140 0x7f64a414b8f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f64ac66b640 1 --2- 172.21.15.12:0/2307174985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64a40c2700 0x7f64a414b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f64ac66b640 1 --2- 172.21.15.12:0/2307174985 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f64880664f0 0x7f64880689b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f64ac66b640 1 --2- 172.21.15.12:0/2307174985 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64a40c2290 0x7f64a41ea510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f64ac66b640 1 -- 172.21.15.12:0/2307174985 >> 172.21.15.12:0/2307174985 conn(0x7f64a40cac50 msgr2=0x7f64a41df2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f64ac66b640 1 -- 172.21.15.12:0/2307174985 shutdown_connections 2023-10-30T12:35:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f64ac66b640 1 -- 172.21.15.12:0/2307174985 wait complete. 2023-10-30T12:35:14.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f5d2d703640 1 -- 172.21.15.12:0/748223545 shutdown_connections 2023-10-30T12:35:14.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f5d2d703640 1 --2- 172.21.15.12:0/748223545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d280cb050 0x7f5d280cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f5d2d703640 1 --2- 172.21.15.12:0/748223545 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5d140664d0 0x7f5d14068990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f5d2d703640 1 --2- 172.21.15.12:0/748223545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d280cab00 0x7f5d280cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f5d2d703640 1 --2- 172.21.15.12:0/748223545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d280c5550 0x7f5d2806f2e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f5d2d703640 1 -- 172.21.15.12:0/748223545 >> 172.21.15.12:0/748223545 conn(0x7f5d2806bed0 msgr2=0x7f5d2814e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.930+0000 7f5d2d703640 1 -- 172.21.15.12:0/748223545 shutdown_connections 2023-10-30T12:35:14.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.934+0000 7f5d2d703640 1 -- 172.21.15.12:0/748223545 wait complete. 2023-10-30T12:35:14.955 INFO:teuthology.orchestra.run.smithi012.stdout:60129542150 2023-10-30T12:35:14.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.950+0000 7ff7c9ffb640 1 -- 172.21.15.12:0/1202388338 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff7b80057e0 con 0x7ff7d80d9230 2023-10-30T12:35:14.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.950+0000 7ff7cbfff640 1 -- 172.21.15.12:0/1202388338 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff7e0130070 con 0x7ff7d80d9230 2023-10-30T12:35:14.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.954+0000 7fe537394640 1 -- 172.21.15.12:0/1468957130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe5300c37a0 con 0x7fe5300c08d0 2023-10-30T12:35:14.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.954+0000 7ff7c9ffb640 1 -- 172.21.15.12:0/1202388338 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff7c40664d0 msgr2=0x7ff7c4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.954+0000 7ff7c9ffb640 1 --2- 172.21.15.12:0/1202388338 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff7c40664d0 0x7ff7c4068990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4007a70 tx=0x7ff7d4008040 comp rx=0 tx=0).stop 2023-10-30T12:35:14.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.954+0000 7ff7c9ffb640 1 -- 172.21.15.12:0/1202388338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7d80d9230 msgr2=0x7ff7d80d0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.954+0000 7ff7c9ffb640 1 --2- 172.21.15.12:0/1202388338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7d80d9230 0x7ff7d80d0190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00682d0 tx=0x7ff7e0068c90 comp rx=0 tx=0).stop 2023-10-30T12:35:14.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.954+0000 7ff7c9ffb640 1 -- 172.21.15.12:0/1202388338 shutdown_connections 2023-10-30T12:35:14.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.954+0000 7ff7c9ffb640 1 --2- 172.21.15.12:0/1202388338 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff7c40664d0 0x7ff7c4068990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.954+0000 7ff7c9ffb640 1 --2- 172.21.15.12:0/1202388338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7d80d9230 0x7ff7d80d0190 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.954+0000 7ff7c9ffb640 1 --2- 172.21.15.12:0/1202388338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7d804eed0 0x7ff7d81740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.954+0000 7ff7c9ffb640 1 --2- 172.21.15.12:0/1202388338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7d800ed70 0x7ff7d8006c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.954+0000 7ff7c9ffb640 1 -- 172.21.15.12:0/1202388338 >> 172.21.15.12:0/1202388338 conn(0x7ff7d800e170 msgr2=0x7ff7d80537e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.954+0000 7fe525ffb640 1 -- 172.21.15.12:0/1468957130 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe5300c37a0 con 0x7fe5300c08d0 2023-10-30T12:35:14.962 INFO:teuthology.orchestra.run.smithi012.stdout:60129542150 2023-10-30T12:35:14.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.954+0000 7ff7c9ffb640 1 -- 172.21.15.12:0/1202388338 shutdown_connections 2023-10-30T12:35:14.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.954+0000 7ff7c9ffb640 1 -- 172.21.15.12:0/1202388338 wait complete. 2023-10-30T12:35:14.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.958+0000 7f77efb1d640 1 -- 172.21.15.12:0/3227293625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f77e80b0860 con 0x7f77e806a3d0 2023-10-30T12:35:14.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.958+0000 7f77de7fc640 1 -- 172.21.15.12:0/3227293625 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f77e40f2170 con 0x7f77e806a3d0 2023-10-30T12:35:14.964 INFO:teuthology.orchestra.run.smithi012.stdout:60129542152 2023-10-30T12:35:14.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.958+0000 7f77c0ac9640 1 -- 172.21.15.12:0/721304642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f77bc15e5a0 con 0x7f77bc156430 2023-10-30T12:35:14.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.958+0000 7fe537394640 1 -- 172.21.15.12:0/1468957130 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe5140664f0 msgr2=0x7fe5140689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.958+0000 7fe537394640 1 --2- 172.21.15.12:0/1468957130 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe5140664f0 0x7fe5140689b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe520000f70 tx=0x7fe520007a90 comp rx=0 tx=0).stop 2023-10-30T12:35:14.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.958+0000 7fe537394640 1 -- 172.21.15.12:0/1468957130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5300c08d0 msgr2=0x7fe5300c3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.958+0000 7fe537394640 1 --2- 172.21.15.12:0/1468957130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5300c08d0 0x7fe5300c3a80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe528007de0 tx=0x7fe52800e4b0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.958+0000 7fe537394640 1 -- 172.21.15.12:0/1468957130 shutdown_connections 2023-10-30T12:35:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.958+0000 7fe537394640 1 --2- 172.21.15.12:0/1468957130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5300c08d0 0x7fe5300c3a80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.958+0000 7fe537394640 1 --2- 172.21.15.12:0/1468957130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe530071d80 0x7fe530158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.958+0000 7fe537394640 1 --2- 172.21.15.12:0/1468957130 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe5140664f0 0x7fe5140689b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.958+0000 7fe537394640 1 --2- 172.21.15.12:0/1468957130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe530071920 0x7fe5301587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.958+0000 7fe537394640 1 -- 172.21.15.12:0/1468957130 >> 172.21.15.12:0/1468957130 conn(0x7fe5300c7910 msgr2=0x7fe530069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77aaffd640 1 -- 172.21.15.12:0/721304642 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f77b4054f90 con 0x7f77bc156430 2023-10-30T12:35:14.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7fe537394640 1 -- 172.21.15.12:0/1468957130 shutdown_connections 2023-10-30T12:35:14.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7fe537394640 1 -- 172.21.15.12:0/1468957130 wait complete. 2023-10-30T12:35:14.969 INFO:teuthology.orchestra.run.smithi012.stdout:60129542151 2023-10-30T12:35:14.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77efb1d640 1 -- 172.21.15.12:0/3227293625 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f77cc0665a0 msgr2=0x7f77cc068a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77efb1d640 1 --2- 172.21.15.12:0/3227293625 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f77cc0665a0 0x7f77cc068a60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f77e81f7a10 tx=0x7f77d8008040 comp rx=0 tx=0).stop 2023-10-30T12:35:14.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77efb1d640 1 -- 172.21.15.12:0/3227293625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77e806a3d0 msgr2=0x7f77e8154d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77efb1d640 1 --2- 172.21.15.12:0/3227293625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77e806a3d0 0x7f77e8154d80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f77e401f590 tx=0x7f77e401ba90 comp rx=0 tx=0).stop 2023-10-30T12:35:14.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77efb1d640 1 -- 172.21.15.12:0/3227293625 shutdown_connections 2023-10-30T12:35:14.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77efb1d640 1 --2- 172.21.15.12:0/3227293625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77e81533b0 0x7f77e81f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77efb1d640 1 --2- 172.21.15.12:0/3227293625 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f77cc0665a0 0x7f77cc068a60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77efb1d640 1 --2- 172.21.15.12:0/3227293625 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77e8158090 0x7f77e80c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77efb1d640 1 --2- 172.21.15.12:0/3227293625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77e806a3d0 0x7f77e8154d80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77efb1d640 1 -- 172.21.15.12:0/3227293625 >> 172.21.15.12:0/3227293625 conn(0x7f77e80c7910 msgr2=0x7f77e8155e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77efb1d640 1 -- 172.21.15.12:0/3227293625 shutdown_connections 2023-10-30T12:35:14.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77efb1d640 1 -- 172.21.15.12:0/3227293625 wait complete. 2023-10-30T12:35:14.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77c0ac9640 1 -- 172.21.15.12:0/721304642 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f779c0664f0 msgr2=0x7f779c0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77c0ac9640 1 --2- 172.21.15.12:0/721304642 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f779c0664f0 0x7f779c0689b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f77ac007e70 tx=0x7f77ac009040 comp rx=0 tx=0).stop 2023-10-30T12:35:14.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77c0ac9640 1 -- 172.21.15.12:0/721304642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77bc156430 msgr2=0x7f77bc160a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.962+0000 7f77c0ac9640 1 --2- 172.21.15.12:0/721304642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77bc156430 0x7f77bc160a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f77bc070680 tx=0x7f77b4011e30 comp rx=0 tx=0).stop 2023-10-30T12:35:14.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.966+0000 7f77c0ac9640 1 -- 172.21.15.12:0/721304642 shutdown_connections 2023-10-30T12:35:14.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.966+0000 7f77c0ac9640 1 --2- 172.21.15.12:0/721304642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77bc156430 0x7f77bc160a60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.966+0000 7f77c0ac9640 1 --2- 172.21.15.12:0/721304642 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f779c0664f0 0x7f779c0689b0 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f77ac007e70 tx=0x7f77ac009040 comp rx=0 tx=0).stop 2023-10-30T12:35:14.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.966+0000 7f77c0ac9640 1 --2- 172.21.15.12:0/721304642 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77bc1525b0 0x7f77bc155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.966+0000 7f77c0ac9640 1 --2- 172.21.15.12:0/721304642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77a40047d0 0x7f77bc1f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.966+0000 7f77c0ac9640 1 -- 172.21.15.12:0/721304642 >> 172.21.15.12:0/721304642 conn(0x7f77bc0c7910 msgr2=0x7f77bc0706b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.966+0000 7fbeb9a9e640 1 -- 172.21.15.12:0/118899028 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbeb40caa20 con 0x7fbeb40c4430 2023-10-30T12:35:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.966+0000 7f77c0ac9640 1 -- 172.21.15.12:0/721304642 shutdown_connections 2023-10-30T12:35:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.966+0000 7f77c0ac9640 1 -- 172.21.15.12:0/721304642 wait complete. 2023-10-30T12:35:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.966+0000 7fbe9bfff640 1 -- 172.21.15.12:0/118899028 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbeac054490 con 0x7fbeb40c4430 2023-10-30T12:35:14.976 INFO:teuthology.orchestra.run.smithi012.stdout:60129542150 2023-10-30T12:35:14.977 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.1 2023-10-30T12:35:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.970+0000 7fbeb9a9e640 1 -- 172.21.15.12:0/118899028 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbea0066520 msgr2=0x7fbea00689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.970+0000 7fbeb9a9e640 1 --2- 172.21.15.12:0/118899028 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbea0066520 0x7fbea00689e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbea80ae000 tx=0x7fbea8008040 comp rx=0 tx=0).stop 2023-10-30T12:35:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.970+0000 7fbeb9a9e640 1 -- 172.21.15.12:0/118899028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbeb40c4430 msgr2=0x7fbeb4141540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.970+0000 7fbeb9a9e640 1 --2- 172.21.15.12:0/118899028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbeb40c4430 0x7fbeb4141540 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00ba80 tx=0x7fbeac00bf40 comp rx=0 tx=0).stop 2023-10-30T12:35:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.970+0000 7fbeb9a9e640 1 -- 172.21.15.12:0/118899028 shutdown_connections 2023-10-30T12:35:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.970+0000 7fbeb9a9e640 1 --2- 172.21.15.12:0/118899028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbeb4159830 0x7fbeb4158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.970+0000 7fbeb9a9e640 1 --2- 172.21.15.12:0/118899028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbeb4153c60 0x7fbeb4158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.970+0000 7fbeb9a9e640 1 --2- 172.21.15.12:0/118899028 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbea0066520 0x7fbea00689e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.970+0000 7fbeb9a9e640 1 --2- 172.21.15.12:0/118899028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbeb40c4430 0x7fbeb4141540 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.970+0000 7fbeb9a9e640 1 -- 172.21.15.12:0/118899028 >> 172.21.15.12:0/118899028 conn(0x7fbeb406bfe0 msgr2=0x7fbeb406c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.970+0000 7fbeb9a9e640 1 -- 172.21.15.12:0/118899028 shutdown_connections 2023-10-30T12:35:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.970+0000 7fbeb9a9e640 1 -- 172.21.15.12:0/118899028 wait complete. 2023-10-30T12:35:14.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.0 2023-10-30T12:35:14.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.970+0000 7fb0533d9640 1 -- 172.21.15.12:0/57347114 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb04c06aed0 con 0x7fb04c06aa00 2023-10-30T12:35:14.982 INFO:teuthology.orchestra.run.smithi012.stdout:60129542152 2023-10-30T12:35:14.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.974+0000 7fb041ffb640 1 -- 172.21.15.12:0/57347114 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb048053f50 con 0x7fb04c06aa00 2023-10-30T12:35:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.974+0000 7fb0533d9640 1 -- 172.21.15.12:0/57347114 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb030066520 msgr2=0x7fb0300689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.974+0000 7fb0533d9640 1 --2- 172.21.15.12:0/57347114 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb030066520 0x7fb0300689e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0b2000 tx=0x7fb03c0aa070 comp rx=0 tx=0).stop 2023-10-30T12:35:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.974+0000 7fb0533d9640 1 -- 172.21.15.12:0/57347114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb04c06aa00 msgr2=0x7fb04c1f16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.974+0000 7fb0533d9640 1 --2- 172.21.15.12:0/57347114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb04c06aa00 0x7fb04c1f16d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb04800c490 tx=0x7fb048001e80 comp rx=0 tx=0).stop 2023-10-30T12:35:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.974+0000 7fb0533d9640 1 -- 172.21.15.12:0/57347114 shutdown_connections 2023-10-30T12:35:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.974+0000 7fb0533d9640 1 --2- 172.21.15.12:0/57347114 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb030066520 0x7fb0300689e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.974+0000 7fb0533d9640 1 --2- 172.21.15.12:0/57347114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb04c0c5810 0x7fb04c0c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.974+0000 7fb0533d9640 1 --2- 172.21.15.12:0/57347114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04c072040 0x7fb04c1f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.974+0000 7fb0533d9640 1 --2- 172.21.15.12:0/57347114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb04c06aa00 0x7fb04c1f16d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.974+0000 7fb0533d9640 1 -- 172.21.15.12:0/57347114 >> 172.21.15.12:0/57347114 conn(0x7fb04c0c7910 msgr2=0x7fb04c0c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:14.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.974+0000 7fb0533d9640 1 -- 172.21.15.12:0/57347114 shutdown_connections 2023-10-30T12:35:14.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:14.974+0000 7fb0533d9640 1 -- 172.21.15.12:0/57347114 wait complete. 2023-10-30T12:35:14.999 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.2 2023-10-30T12:35:15.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.5 2023-10-30T12:35:15.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.4 2023-10-30T12:35:15.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.7 2023-10-30T12:35:15.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.3 2023-10-30T12:35:15.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.6 2023-10-30T12:35:15.978 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-30T12:35:15.982 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-30T12:35:16.000 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-30T12:35:16.004 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-30T12:35:16.004 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-30T12:35:16.009 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-30T12:35:16.011 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-30T12:35:16.013 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-30T12:35:16.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.202+0000 7fa806796640 1 Processor -- start 2023-10-30T12:35:16.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.206+0000 7f5824bf2640 1 Processor -- start 2023-10-30T12:35:16.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.206+0000 7fa806796640 1 -- start start 2023-10-30T12:35:16.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.206+0000 7f5824bf2640 1 -- start start 2023-10-30T12:35:16.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.206+0000 7f5824bf2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f581804b740 0x7f58180de390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.206+0000 7fa806796640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa7f8034ff0 0x7fa7f804b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.206+0000 7f5824bf2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f58180de8d0 0x7f58180e5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.206+0000 7fa806796640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa7f804c8c0 0x7fa7f80de270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.206+0000 7f5824bf2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58180452f0 con 0x7f58180d4e20 2023-10-30T12:35:16.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.206+0000 7f581f7fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f58180d4e20 0x7f5818034880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:36058/0 (socket says 172.21.15.12:36058) 2023-10-30T12:35:16.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.206+0000 7f581f7fe640 1 -- 172.21.15.12:0/2298675238 learned_addr learned my addr 172.21.15.12:0/2298675238 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:16.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f5824bf2640 1 -- 172.21.15.12:0/2298675238 --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5818007940 con 0x7f58180de8d0 2023-10-30T12:35:16.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f5824bf2640 1 -- 172.21.15.12:0/2298675238 --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58180df0e0 con 0x7f581804b740 2023-10-30T12:35:16.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7fa805794640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa7f80d4e80 0x7fa7f80348e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:55460/0 (socket says 172.21.15.12:55460) 2023-10-30T12:35:16.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581effd640 1 --2- 172.21.15.12:0/2298675238 >> v2:172.21.15.154:3300/0 conn(0x7f581804b740 0x7f58180de390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7fa805f95640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa7f804c8c0 0x7fa7f80de270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7fa805f95640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa7f804c8c0 0x7fa7f80de270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58256/0 (socket says 172.21.15.12:58256) 2023-10-30T12:35:16.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 -- 172.21.15.12:0/2298675238 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567951431 0 0) 0x7f58180452f0 con 0x7f58180d4e20 2023-10-30T12:35:16.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581ffff640 1 --2- 172.21.15.12:0/2298675238 >> v2:172.21.15.12:3300/0 conn(0x7f58180de8d0 0x7f58180e5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7fa805f95640 1 -- 172.21.15.12:0/3072946409 learned_addr learned my addr 172.21.15.12:0/3072946409 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:16.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 -- 172.21.15.12:0/2298675238 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5810003530 con 0x7f58180d4e20 2023-10-30T12:35:16.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7fa804f93640 1 --2- 172.21.15.12:0/3072946409 >> v2:172.21.15.12:3300/0 conn(0x7fa7f8034ff0 0x7fa7f804b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7fa806796640 1 -- 172.21.15.12:0/3072946409 --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f804bb80 con 0x7fa7f80d4e80 2023-10-30T12:35:16.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7fa806796640 1 -- 172.21.15.12:0/3072946409 --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f804ae80 con 0x7fa7f8034ff0 2023-10-30T12:35:16.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 -- 172.21.15.12:0/2298675238 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1453761907 0 0) 0x7f5810003530 con 0x7f58180d4e20 2023-10-30T12:35:16.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 -- 172.21.15.12:0/2298675238 >> v2:172.21.15.154:3300/0 conn(0x7f581804b740 msgr2=0x7f58180de390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 --2- 172.21.15.12:0/2298675238 >> v2:172.21.15.154:3300/0 conn(0x7f581804b740 0x7f58180de390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 -- 172.21.15.12:0/2298675238 >> v2:172.21.15.12:3300/0 conn(0x7f58180de8d0 msgr2=0x7f58180e5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7fa806796640 1 -- 172.21.15.12:0/3072946409 --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f8006b20 con 0x7fa7f804c8c0 2023-10-30T12:35:16.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7fa805f95640 1 -- 172.21.15.12:0/3072946409 >> v2:172.21.15.12:3300/0 conn(0x7fa7f8034ff0 msgr2=0x7fa7f804b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 --2- 172.21.15.12:0/2298675238 >> v2:172.21.15.12:3300/0 conn(0x7f58180de8d0 0x7f58180e5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 -- 172.21.15.12:0/2298675238 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5818006b80 con 0x7f58180d4e20 2023-10-30T12:35:16.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581effd640 1 --2- 172.21.15.12:0/2298675238 >> v2:172.21.15.154:3300/0 conn(0x7f581804b740 0x7f58180de390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:35:16.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 -- 172.21.15.12:0/2298675238 <== mon.1 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f582004e9c0 con 0x7f58180d4e20 2023-10-30T12:35:16.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 -- 172.21.15.12:0/2298675238 >> v1:172.21.15.154:6789/0 conn(0x7f58180d4e20 legacy=0x7f5818034880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 --2- 172.21.15.12:0/2298675238 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5810003980 0x7f5810003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 --2- 172.21.15.12:0/2298675238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5810004320 0x7f58100047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 --2- 172.21.15.12:0/2298675238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5810004e60 0x7f581000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 -- 172.21.15.12:0/2298675238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58180df0e0 con 0x7f5810004e60 2023-10-30T12:35:16.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 -- 172.21.15.12:0/2298675238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58100056a0 con 0x7f5810004320 2023-10-30T12:35:16.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581e7fc640 1 -- 172.21.15.12:0/2298675238 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581000e150 con 0x7f5810003980 2023-10-30T12:35:16.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581ffff640 1 --2- 172.21.15.12:0/2298675238 >> v2:172.21.15.12:3300/0 conn(0x7f58180de8d0 0x7f58180e5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:35:16.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7f581f7fe640 1 --2- 172.21.15.12:0/2298675238 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5810003980 0x7f5810003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7fa805f95640 1 --2- 172.21.15.12:0/3072946409 >> v2:172.21.15.12:3300/0 conn(0x7fa7f8034ff0 0x7fa7f804b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7fa805f95640 1 -- 172.21.15.12:0/3072946409 >> v1:172.21.15.12:6789/0 conn(0x7fa7f80d4e80 legacy=0x7fa7f80348e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.210+0000 7fa805f95640 1 -- 172.21.15.12:0/3072946409 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f8006e00 con 0x7fa7f804c8c0 2023-10-30T12:35:16.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.214+0000 7f581ffff640 1 --2- 172.21.15.12:0/2298675238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5810004e60 0x7f581000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.214+0000 7f581ffff640 1 -- 172.21.15.12:0/2298675238 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5810003980 msgr2=0x7f5810003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.214+0000 7f581ffff640 1 --2- 172.21.15.12:0/2298675238 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5810003980 0x7f5810003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.214+0000 7f581ffff640 1 -- 172.21.15.12:0/2298675238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5810004320 msgr2=0x7f58100047f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-30T12:35:16.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.214+0000 7f581ffff640 1 --2- 172.21.15.12:0/2298675238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5810004320 0x7f58100047f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.214+0000 7f581ffff640 1 -- 172.21.15.12:0/2298675238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5810003530 con 0x7f5810004e60 2023-10-30T12:35:16.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7fa805f95640 1 --2- 172.21.15.12:0/3072946409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f804c8c0 0x7fa7f80de270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80d8870 tx=0x7fa7f001e6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1edc998bce8a6fe server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7fa7f7fff640 1 -- 172.21.15.12:0/3072946409 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00a8680 con 0x7fa7f804c8c0 2023-10-30T12:35:16.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7fa7f7fff640 1 -- 172.21.15.12:0/3072946409 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f00a87e0 con 0x7fa7f804c8c0 2023-10-30T12:35:16.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7fa7f7fff640 1 -- 172.21.15.12:0/3072946409 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00a8680 con 0x7fa7f804c8c0 2023-10-30T12:35:16.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7f581ffff640 1 -- 172.21.15.12:0/2298675238 reap_dead start 2023-10-30T12:35:16.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7f581ffff640 1 --2- 172.21.15.12:0/2298675238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5810004e60 0x7f581000db90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f580c009c90 tx=0x7f580c01cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d5634ceb80ccfc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7f581e7fc640 1 -- 172.21.15.12:0/2298675238 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c004020 con 0x7f5810004e60 2023-10-30T12:35:16.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7f581e7fc640 1 -- 172.21.15.12:0/2298675238 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c004180 con 0x7f5810004e60 2023-10-30T12:35:16.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7f581e7fc640 1 -- 172.21.15.12:0/2298675238 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c0a5850 con 0x7f5810004e60 2023-10-30T12:35:16.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7fa806796640 1 -- 172.21.15.12:0/3072946409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f804c8c0 msgr2=0x7fa7f80de270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7fa806796640 1 --2- 172.21.15.12:0/3072946409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f804c8c0 0x7fa7f80de270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80d8870 tx=0x7fa7f001e6b0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7fa806796640 1 -- 172.21.15.12:0/3072946409 shutdown_connections 2023-10-30T12:35:16.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7fa806796640 1 --2- 172.21.15.12:0/3072946409 >> v2:172.21.15.12:3300/0 conn(0x7fa7f8034ff0 0x7fa7f804b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7fa806796640 1 --2- 172.21.15.12:0/3072946409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f804c8c0 0x7fa7f80de270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7fa806796640 1 -- 172.21.15.12:0/3072946409 >> 172.21.15.12:0/3072946409 conn(0x7fa7f80502d0 msgr2=0x7fa7f80506e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7f5824bf2640 1 -- 172.21.15.12:0/2298675238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5810004e60 msgr2=0x7f581000db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.218+0000 7f5824bf2640 1 --2- 172.21.15.12:0/2298675238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5810004e60 0x7f581000db90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f580c009c90 tx=0x7f580c01cc70 comp rx=0 tx=0).stop 2023-10-30T12:35:16.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.222+0000 7f5824bf2640 1 -- 172.21.15.12:0/2298675238 shutdown_connections 2023-10-30T12:35:16.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.222+0000 7f5824bf2640 1 --2- 172.21.15.12:0/2298675238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5810004e60 0x7f581000db90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.222+0000 7f5824bf2640 1 -- 172.21.15.12:0/2298675238 >> 172.21.15.12:0/2298675238 conn(0x7f58180502e0 msgr2=0x7f58180506f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.222+0000 7fa806796640 1 -- 172.21.15.12:0/3072946409 shutdown_connections 2023-10-30T12:35:16.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.222+0000 7f5824bf2640 1 -- 172.21.15.12:0/2298675238 shutdown_connections 2023-10-30T12:35:16.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.222+0000 7f5824bf2640 1 -- 172.21.15.12:0/2298675238 wait complete. 2023-10-30T12:35:16.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.222+0000 7fa806796640 1 -- 172.21.15.12:0/3072946409 wait complete. 2023-10-30T12:35:16.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.222+0000 7fa806796640 1 Processor -- start 2023-10-30T12:35:16.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.222+0000 7f5824bf2640 1 Processor -- start 2023-10-30T12:35:16.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.222+0000 7fa806796640 1 -- start start 2023-10-30T12:35:16.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.222+0000 7f5824bf2640 1 -- start start 2023-10-30T12:35:16.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7fa806796640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f8034ff0 0x7fa7f80dd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7fa806796640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f804c8c0 0x7fa7f80e3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7fa806796640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7f80d4e80 0x7fa7f80e43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7fa806796640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f80e3a00 con 0x7fa7f8034ff0 2023-10-30T12:35:16.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7fa806796640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f80cbdd0 con 0x7fa7f804c8c0 2023-10-30T12:35:16.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7fa806796640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f8177d50 con 0x7fa7f80d4e80 2023-10-30T12:35:16.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7f5824bf2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5810004e60 0x7f58180d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7f5824bf2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58180dbf60 0x7f58180dc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7f5824bf2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5818183fb0 0x7f5818184430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7fa805f95640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7f80d4e80 0x7fa7f80e43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7fa805f95640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7f80d4e80 0x7fa7f80e43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40696/0 (socket says 172.21.15.12:40696) 2023-10-30T12:35:16.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7fa805f95640 1 -- 172.21.15.12:0/1385919398 learned_addr learned my addr 172.21.15.12:0/1385919398 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:16.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7f581f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5810004e60 0x7f58180d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7f581f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5810004e60 0x7f58180d24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58292/0 (socket says 172.21.15.12:58292) 2023-10-30T12:35:16.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7f581f7fe640 1 -- 172.21.15.12:0/129744871 learned_addr learned my addr 172.21.15.12:0/129744871 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:16.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7fa805794640 1 --2- 172.21.15.12:0/1385919398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f8034ff0 0x7fa7f80dd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7f581ffff640 1 --2- 172.21.15.12:0/129744871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5818183fb0 0x7f5818184430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7f5824bf2640 1 -- 172.21.15.12:0/129744871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58180da710 con 0x7f58180dbf60 2023-10-30T12:35:16.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7f581effd640 1 --2- 172.21.15.12:0/129744871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58180dbf60 0x7f58180dc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7f5824bf2640 1 -- 172.21.15.12:0/129744871 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5818184970 con 0x7f5810004e60 2023-10-30T12:35:16.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7f5824bf2640 1 -- 172.21.15.12:0/129744871 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58180d8db0 con 0x7f5818183fb0 2023-10-30T12:35:16.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7fa805f95640 1 -- 172.21.15.12:0/1385919398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f804c8c0 msgr2=0x7fa7f80e3e90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-30T12:35:16.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7fa805f95640 1 --2- 172.21.15.12:0/1385919398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f804c8c0 0x7fa7f80e3e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7fa805f95640 1 -- 172.21.15.12:0/1385919398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f8034ff0 msgr2=0x7fa7f80dd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7fa805f95640 1 --2- 172.21.15.12:0/1385919398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f8034ff0 0x7fa7f80dd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.226+0000 7fa805f95640 1 -- 172.21.15.12:0/1385919398 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f00094d0 con 0x7fa7f80d4e80 2023-10-30T12:35:16.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7f581ffff640 1 -- 172.21.15.12:0/129744871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5810004e60 msgr2=0x7f58180d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7f581ffff640 1 --2- 172.21.15.12:0/129744871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5810004e60 0x7f58180d24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7f581ffff640 1 -- 172.21.15.12:0/129744871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58180dbf60 msgr2=0x7f58180dc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7f581ffff640 1 --2- 172.21.15.12:0/129744871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58180dbf60 0x7f58180dc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7f581ffff640 1 -- 172.21.15.12:0/129744871 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58180d9090 con 0x7f5818183fb0 2023-10-30T12:35:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7f581effd640 1 --2- 172.21.15.12:0/129744871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58180dbf60 0x7f58180dc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:35:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7fa805f95640 1 --2- 172.21.15.12:0/1385919398 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7f80d4e80 0x7fa7f80e43d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa7f001ef50 tx=0x7fa7f001b450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7f581ffff640 1 --2- 172.21.15.12:0/129744871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5818183fb0 0x7f5818184430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f581804e1b0 tx=0x7f580c019a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7f580bfff640 1 -- 172.21.15.12:0/129744871 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c0044f0 con 0x7f5818183fb0 2023-10-30T12:35:16.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7f580bfff640 1 -- 172.21.15.12:0/129744871 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c0b2040 con 0x7f5818183fb0 2023-10-30T12:35:16.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7f580bfff640 1 -- 172.21.15.12:0/129744871 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c003840 con 0x7f5818183fb0 2023-10-30T12:35:16.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7fa7f5ffb640 1 -- 172.21.15.12:0/1385919398 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00a8680 con 0x7fa7f80d4e80 2023-10-30T12:35:16.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7f5824bf2640 1 -- 172.21.15.12:0/129744871 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58180d9370 con 0x7f5818183fb0 2023-10-30T12:35:16.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7f5824bf2640 1 -- 172.21.15.12:0/129744871 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58180d9930 con 0x7f5818183fb0 2023-10-30T12:35:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7fa7f5ffb640 1 -- 172.21.15.12:0/1385919398 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f00a97f0 con 0x7fa7f80d4e80 2023-10-30T12:35:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7fa7f5ffb640 1 -- 172.21.15.12:0/1385919398 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00a8680 con 0x7fa7f80d4e80 2023-10-30T12:35:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7fa806796640 1 -- 172.21.15.12:0/1385919398 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f8178030 con 0x7fa7f80d4e80 2023-10-30T12:35:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.230+0000 7fa806796640 1 -- 172.21.15.12:0/1385919398 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f81785f0 con 0x7fa7f80d4e80 2023-10-30T12:35:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.234+0000 7f5809ffb640 1 -- 172.21.15.12:0/129744871 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581804e5c0 con 0x7f5818183fb0 2023-10-30T12:35:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.234+0000 7fa806796640 1 -- 172.21.15.12:0/1385919398 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e0005260 con 0x7fa7f80d4e80 2023-10-30T12:35:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.234+0000 7fa7f5ffb640 1 -- 172.21.15.12:0/1385919398 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa7f00a9960 con 0x7fa7f80d4e80 2023-10-30T12:35:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.238+0000 7f580bfff640 1 -- 172.21.15.12:0/129744871 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f580c0039a0 con 0x7f5818183fb0 2023-10-30T12:35:16.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.238+0000 7fa7f5ffb640 1 --2- 172.21.15.12:0/1385919398 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7e4066530 0x7fa7e40689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.238+0000 7fa7f5ffb640 1 -- 172.21.15.12:0/1385919398 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7f011e250 con 0x7fa7f80d4e80 2023-10-30T12:35:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.238+0000 7f580bfff640 1 --2- 172.21.15.12:0/129744871 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f58040664a0 0x7f5804068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.238+0000 7f580bfff640 1 -- 172.21.15.12:0/129744871 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f580c11dd00 con 0x7f5818183fb0 2023-10-30T12:35:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.238+0000 7f581f7fe640 1 --2- 172.21.15.12:0/129744871 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f58040664a0 0x7f5804068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.238+0000 7fa805794640 1 --2- 172.21.15.12:0/1385919398 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7e4066530 0x7fa7e40689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.238+0000 7fa805794640 1 --2- 172.21.15.12:0/1385919398 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7e4066530 0x7fa7e40689f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa800063ef0 tx=0x7fa80006d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.242+0000 7f581f7fe640 1 --2- 172.21.15.12:0/129744871 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f58040664a0 0x7f5804068960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5820075640 tx=0x7f582006d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.254+0000 7fa7f5ffb640 1 -- 172.21.15.12:0/1385919398 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa7f00eb520 con 0x7fa7f80d4e80 2023-10-30T12:35:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.270+0000 7f580bfff640 1 -- 172.21.15.12:0/129744871 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f581804e5c0 con 0x7f5818183fb0 2023-10-30T12:35:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.270+0000 7f7d1c6ea640 1 Processor -- start 2023-10-30T12:35:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.270+0000 7f7d1c6ea640 1 -- start start 2023-10-30T12:35:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.274+0000 7f7d1c6ea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7d14153c60 0x7f7d140ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.274+0000 7f7d1c6ea640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7d14159830 0x7f7d14159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.274+0000 7f7d1c6ea640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d140cb780 con 0x7f7d140c4430 2023-10-30T12:35:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.274+0000 7f7d1c6ea640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d14154040 con 0x7f7d14159830 2023-10-30T12:35:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.274+0000 7f7d1c6ea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1415a2e0 con 0x7f7d14153c60 2023-10-30T12:35:16.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.274+0000 7f7d1ac87640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7d14159830 0x7f7d14159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.274+0000 7f7d1ac87640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7d14159830 0x7f7d14159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58304/0 (socket says 172.21.15.12:58304) 2023-10-30T12:35:16.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.274+0000 7f7d1ac87640 1 -- 172.21.15.12:0/2721738647 learned_addr learned my addr 172.21.15.12:0/2721738647 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.274+0000 7f7d1a486640 1 --2- 172.21.15.12:0/2721738647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7d14153c60 0x7f7d140ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.274+0000 7f7d1ac87640 1 -- 172.21.15.12:0/2721738647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7d14153c60 msgr2=0x7f7d140ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.274+0000 7f7d1ac87640 1 --2- 172.21.15.12:0/2721738647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7d14153c60 0x7f7d140ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.274+0000 7f7d1ac87640 1 -- 172.21.15.12:0/2721738647 >> v1:172.21.15.12:6789/0 conn(0x7f7d140c4430 legacy=0x7f7d140cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.274+0000 7f7d1ac87640 1 -- 172.21.15.12:0/2721738647 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1415a5c0 con 0x7f7d14159830 2023-10-30T12:35:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.278+0000 7f7d1ac87640 1 --2- 172.21.15.12:0/2721738647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d14159830 0x7f7d14159ce0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009e50 tx=0x7f7d0c01c9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe7ff0ad092831d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.278+0000 7f7d19484640 1 -- 172.21.15.12:0/2721738647 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0a5740 con 0x7f7d14159830 2023-10-30T12:35:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.278+0000 7f7d19484640 1 -- 172.21.15.12:0/2721738647 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c0a58a0 con 0x7f7d14159830 2023-10-30T12:35:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.278+0000 7f7d19484640 1 -- 172.21.15.12:0/2721738647 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0a5c30 con 0x7f7d14159830 2023-10-30T12:35:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.278+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2721738647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d14159830 msgr2=0x7f7d14159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.278+0000 7f7d1c6ea640 1 --2- 172.21.15.12:0/2721738647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d14159830 0x7f7d14159ce0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009e50 tx=0x7f7d0c01c9d0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.282+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2721738647 shutdown_connections 2023-10-30T12:35:16.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.282+0000 7f7d1c6ea640 1 --2- 172.21.15.12:0/2721738647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d14159830 0x7f7d14159ce0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.282+0000 7f7d1c6ea640 1 --2- 172.21.15.12:0/2721738647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7d14153c60 0x7f7d140ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.282+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2721738647 >> 172.21.15.12:0/2721738647 conn(0x7f7d1406bfe0 msgr2=0x7f7d1406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.282+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2721738647 shutdown_connections 2023-10-30T12:35:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.282+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2721738647 wait complete. 2023-10-30T12:35:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.286+0000 7f7f951a7640 1 Processor -- start 2023-10-30T12:35:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.286+0000 7f7f951a7640 1 -- start start 2023-10-30T12:35:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.286+0000 7f7d1c6ea640 1 Processor -- start 2023-10-30T12:35:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.286+0000 7f7f951a7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f9015a250 0x7f7f9015aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.286+0000 7f7f951a7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f9015cbc0 0x7f7f9006cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.286+0000 7f7f951a7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9015db30 con 0x7f7f90153960 2023-10-30T12:35:16.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.286+0000 7f7f951a7640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f90153d40 con 0x7f7f9015a250 2023-10-30T12:35:16.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.286+0000 7f7f951a7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9015d570 con 0x7f7f9015cbc0 2023-10-30T12:35:16.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.286+0000 7f7d1c6ea640 1 -- start start 2023-10-30T12:35:16.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.286+0000 7f7f8ffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7f90153960 0x7f7f900c6dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:36068/0 (socket says 172.21.15.12:36068) 2023-10-30T12:35:16.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.286+0000 7f7f8f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f9015a250 0x7f7f9015aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.286+0000 7f7f8ffff640 1 -- 172.21.15.12:0/1586982187 learned_addr learned my addr 172.21.15.12:0/1586982187 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:16.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.286+0000 7f7f949a6640 1 --2- 172.21.15.12:0/1586982187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f9015cbc0 0x7f7f9006cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7f949a6640 1 -- 172.21.15.12:0/1586982187 >> v2:172.21.15.154:3300/0 conn(0x7f7f9015a250 msgr2=0x7f7f9015aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7f949a6640 1 --2- 172.21.15.12:0/1586982187 >> v2:172.21.15.154:3300/0 conn(0x7f7f9015a250 0x7f7f9015aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7f949a6640 1 -- 172.21.15.12:0/1586982187 >> v1:172.21.15.154:6789/0 conn(0x7f7f90153960 legacy=0x7f7f900c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7f949a6640 1 -- 172.21.15.12:0/1586982187 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9015d850 con 0x7f7f9015cbc0 2023-10-30T12:35:16.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7d1c6ea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d140c4430 0x7f7d14148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7d1c6ea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7d14153c60 0x7f7d14148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7d1c6ea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d14159830 0x7f7d141feb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7d19c85640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7d14153c60 0x7f7d14148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7d19c85640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7d14153c60 0x7f7d14148b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40744/0 (socket says 172.21.15.12:40744) 2023-10-30T12:35:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7fd08efa1640 1 Processor -- start 2023-10-30T12:35:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd08efa1640 1 -- start start 2023-10-30T12:35:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7d1a486640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d140c4430 0x7f7d14148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7d1ac87640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d14159830 0x7f7d141feb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7d19c85640 1 -- 172.21.15.12:0/2737370610 learned_addr learned my addr 172.21.15.12:0/2737370610 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2737370610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140ce0e0 con 0x7f7d14159830 2023-10-30T12:35:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2737370610 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140ce690 con 0x7f7d140c4430 2023-10-30T12:35:16.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2737370610 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d141492d0 con 0x7f7d14153c60 2023-10-30T12:35:16.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.290+0000 7f7d19c85640 1 -- 172.21.15.12:0/2737370610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d140c4430 msgr2=0x7f7d14148600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7d19c85640 1 --2- 172.21.15.12:0/2737370610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d140c4430 0x7f7d14148600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7d19c85640 1 -- 172.21.15.12:0/2737370610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d14159830 msgr2=0x7f7d141feb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7d19c85640 1 --2- 172.21.15.12:0/2737370610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d14159830 0x7f7d141feb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7d19c85640 1 -- 172.21.15.12:0/2737370610 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c009b30 con 0x7f7d14153c60 2023-10-30T12:35:16.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7d1ac87640 1 --2- 172.21.15.12:0/2737370610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d14159830 0x7f7d141feb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:35:16.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7d19c85640 1 --2- 172.21.15.12:0/2737370610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7d14153c60 0x7f7d14148b40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400b410 tx=0x7f7d0400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7f949a6640 1 --2- 172.21.15.12:0/1586982187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f9015cbc0 0x7f7f9006cec0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009d90 tx=0x7f7f8801c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aaf80c5c5a14abb server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7d0affd640 1 -- 172.21.15.12:0/2737370610 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04004020 con 0x7f7d14153c60 2023-10-30T12:35:16.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7f8effd640 1 -- 172.21.15.12:0/1586982187 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880a5920 con 0x7f7f9015cbc0 2023-10-30T12:35:16.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7f8effd640 1 -- 172.21.15.12:0/1586982187 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f880a5a80 con 0x7f7f9015cbc0 2023-10-30T12:35:16.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7f8effd640 1 -- 172.21.15.12:0/1586982187 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880a5e10 con 0x7f7f9015cbc0 2023-10-30T12:35:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd08efa1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd08806ad90 0x7fd088152de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd08efa1640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd0880c8430 0x7fd088156280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd08efa1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd0880c6670 0x7fd088069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd08efa1640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd088070820 con 0x7fd0880c8430 2023-10-30T12:35:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd08efa1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0881567c0 con 0x7fd0880c6670 2023-10-30T12:35:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd08efa1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0880c7400 con 0x7fd08806ad90 2023-10-30T12:35:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd08cd3d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd08806ad90 0x7fd088152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f112f59e640 1 Processor -- start 2023-10-30T12:35:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7f951a7640 1 -- 172.21.15.12:0/1586982187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f9015cbc0 msgr2=0x7f7f9006cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7f951a7640 1 --2- 172.21.15.12:0/1586982187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f9015cbc0 0x7f7f9006cec0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009d90 tx=0x7f7f8801c950 comp rx=0 tx=0).stop 2023-10-30T12:35:16.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7f951a7640 1 -- 172.21.15.12:0/1586982187 shutdown_connections 2023-10-30T12:35:16.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7f951a7640 1 --2- 172.21.15.12:0/1586982187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f9015cbc0 0x7f7f9006cec0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7f951a7640 1 --2- 172.21.15.12:0/1586982187 >> v2:172.21.15.154:3300/0 conn(0x7f7f9015a250 0x7f7f9015aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7f951a7640 1 -- 172.21.15.12:0/1586982187 >> 172.21.15.12:0/1586982187 conn(0x7f7f900cae10 msgr2=0x7f7f900cb220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd08cd3d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd08806ad90 0x7fd088152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40760/0 (socket says 172.21.15.12:40760) 2023-10-30T12:35:16.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd08cd3d640 1 -- 172.21.15.12:0/3580036327 learned_addr learned my addr 172.21.15.12:0/3580036327 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:16.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7d0affd640 1 -- 172.21.15.12:0/2737370610 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d04004180 con 0x7f7d14153c60 2023-10-30T12:35:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2737370610 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d14149ae0 con 0x7f7d14153c60 2023-10-30T12:35:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7d0affd640 1 -- 172.21.15.12:0/2737370610 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04011630 con 0x7f7d14153c60 2023-10-30T12:35:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2737370610 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d14149520 con 0x7f7d14153c60 2023-10-30T12:35:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7d1a486640 1 --2- 172.21.15.12:0/2737370610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d140c4430 0x7f7d14148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:35:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd087fff640 1 --2- 172.21.15.12:0/3580036327 >> v2:172.21.15.12:3300/0 conn(0x7fd0880c8430 0x7fd088156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2737370610 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc005260 con 0x7f7d14153c60 2023-10-30T12:35:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f7f951a7640 1 -- 172.21.15.12:0/1586982187 shutdown_connections 2023-10-30T12:35:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd08cd3d640 1 -- 172.21.15.12:0/3580036327 >> v2:172.21.15.154:3300/0 conn(0x7fd0880c6670 msgr2=0x7fd088069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd08cd3d640 1 --2- 172.21.15.12:0/3580036327 >> v2:172.21.15.154:3300/0 conn(0x7fd0880c6670 0x7fd088069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd08cd3d640 1 -- 172.21.15.12:0/3580036327 >> v2:172.21.15.12:3300/0 conn(0x7fd0880c8430 msgr2=0x7fd088156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd08cd3d640 1 --2- 172.21.15.12:0/3580036327 >> v2:172.21.15.12:3300/0 conn(0x7fd0880c8430 0x7fd088156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7fd08cd3d640 1 -- 172.21.15.12:0/3580036327 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0880c76c0 con 0x7fd08806ad90 2023-10-30T12:35:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.294+0000 7f112f59e640 1 -- start start 2023-10-30T12:35:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7fad79c46640 1 Processor -- start 2023-10-30T12:35:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7fad79c46640 1 -- start start 2023-10-30T12:35:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7fad79c46640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad74158090 0x7fad74161be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7fad79c46640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fad740c0ed0 0x7fad7406a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7fad73fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fad740c0ed0 0x7fad7406a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7fad73fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fad740c0ed0 0x7fad7406a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58344/0 (socket says 172.21.15.12:58344) 2023-10-30T12:35:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7fad73fff640 1 -- 172.21.15.12:0/3805076033 learned_addr learned my addr 172.21.15.12:0/3805076033 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7fad737fe640 1 --2- 172.21.15.12:0/3805076033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad74158090 0x7fad74161be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7fad79c46640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad740c72b0 con 0x7fad740c6b40 2023-10-30T12:35:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7fad79c46640 1 -- 172.21.15.12:0/3805076033 --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad74162120 con 0x7fad740c0ed0 2023-10-30T12:35:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7fad79c46640 1 -- 172.21.15.12:0/3805076033 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad740c6fd0 con 0x7fad74158090 2023-10-30T12:35:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.302+0000 7f7d0affd640 1 -- 172.21.15.12:0/2737370610 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7d04011790 con 0x7f7d14153c60 2023-10-30T12:35:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f7f951a7640 1 -- 172.21.15.12:0/1586982187 wait complete. 2023-10-30T12:35:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f112f59e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1130153850 0x7f11300c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f112f59e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f113015a2c0 0x7f113015aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f112f59e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f113015cbc0 0x7f113015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f112f59e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f113015dba0 con 0x7f113015cbc0 2023-10-30T12:35:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f112f59e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11301595d0 con 0x7f1130153850 2023-10-30T12:35:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f112f59e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11300c7380 con 0x7f113015a2c0 2023-10-30T12:35:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f112dd9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f113015a2c0 0x7f113015aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f112dd9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f113015a2c0 0x7f113015aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40768/0 (socket says 172.21.15.12:40768) 2023-10-30T12:35:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f112e59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1130153850 0x7f11300c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f112dd9b640 1 -- 172.21.15.12:0/2169569506 learned_addr learned my addr 172.21.15.12:0/2169569506 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:16.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f112ed9d640 1 --2- 172.21.15.12:0/2169569506 >> v2:172.21.15.12:3300/0 conn(0x7f113015cbc0 0x7f113015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f112dd9b640 1 -- 172.21.15.12:0/2169569506 >> v2:172.21.15.154:3300/0 conn(0x7f1130153850 msgr2=0x7f11300c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f112dd9b640 1 --2- 172.21.15.12:0/2169569506 >> v2:172.21.15.154:3300/0 conn(0x7f1130153850 0x7f11300c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f112dd9b640 1 -- 172.21.15.12:0/2169569506 >> v2:172.21.15.12:3300/0 conn(0x7f113015cbc0 msgr2=0x7f113015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7f112dd9b640 1 --2- 172.21.15.12:0/2169569506 >> v2:172.21.15.12:3300/0 conn(0x7f113015cbc0 0x7f113015cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7fad737fe640 1 -- 172.21.15.12:0/3805076033 >> v2:172.21.15.154:3300/0 conn(0x7fad740c0ed0 msgr2=0x7fad7406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7fad737fe640 1 --2- 172.21.15.12:0/3805076033 >> v2:172.21.15.154:3300/0 conn(0x7fad740c0ed0 0x7fad7406a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7fad737fe640 1 -- 172.21.15.12:0/3805076033 >> v1:172.21.15.154:6789/0 conn(0x7fad740c6b40 legacy=0x7fad74069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.298+0000 7fad737fe640 1 -- 172.21.15.12:0/3805076033 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7415dd30 con 0x7fad74158090 2023-10-30T12:35:16.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.302+0000 7f112dd9b640 1 -- 172.21.15.12:0/2169569506 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113015b000 con 0x7f113015a2c0 2023-10-30T12:35:16.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.302+0000 7f112ed9d640 1 --2- 172.21.15.12:0/2169569506 >> v2:172.21.15.12:3300/0 conn(0x7f113015cbc0 0x7f113015cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:35:16.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.302+0000 7fd08cd3d640 1 --2- 172.21.15.12:0/3580036327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd08806ad90 0x7fd088152de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd08806f4d0 tx=0x7fd07c01c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=667cd2c37ff0a862 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.302+0000 7f7d0affd640 1 --2- 172.21.15.12:0/2737370610 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cf8066570 0x7f7cf8068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.302+0000 7f7d0affd640 1 -- 172.21.15.12:0/2737370610 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d040839e0 con 0x7f7d14153c60 2023-10-30T12:35:16.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.302+0000 7f7d1a486640 1 --2- 172.21.15.12:0/2737370610 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cf8066570 0x7f7cf8068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.302+0000 7f7f951a7640 1 Processor -- start 2023-10-30T12:35:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.302+0000 7f7f951a7640 1 -- start start 2023-10-30T12:35:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.302+0000 7fad737fe640 1 --2- 172.21.15.12:0/3805076033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad74158090 0x7fad74161be0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad640097c0 tx=0x7fad6401ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a10375dd516a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.302+0000 7f7d1a486640 1 --2- 172.21.15.12:0/2737370610 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cf8066570 0x7f7cf8068a30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009ae0 tx=0x7f7d10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7faa3d0dc640 1 Processor -- start 2023-10-30T12:35:16.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f951a7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f90153960 0x7f7f900c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f112dd9b640 1 --2- 172.21.15.12:0/2169569506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f113015a2c0 0x7f113015aac0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f11300c5ca0 tx=0x7f112401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a26d89f2014369c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f951a7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f9015a250 0x7f7f90141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f951a7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f9015cbc0 0x7f7f90142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f951a7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f901574d0 con 0x7f7f90153960 2023-10-30T12:35:16.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f951a7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f901f9ee0 con 0x7f7f9015cbc0 2023-10-30T12:35:16.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f951a7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f90157210 con 0x7f7f9015a250 2023-10-30T12:35:16.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7faa3d0dc640 1 -- start start 2023-10-30T12:35:16.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f8f7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f9015a250 0x7f7f90141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f8f7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f9015a250 0x7f7f90141fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40782/0 (socket says 172.21.15.12:40782) 2023-10-30T12:35:16.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f8f7fe640 1 -- 172.21.15.12:0/3531264465 learned_addr learned my addr 172.21.15.12:0/3531264465 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:16.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f949a6640 1 --2- 172.21.15.12:0/3531264465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f9015cbc0 0x7f7f90142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f8ffff640 1 --2- 172.21.15.12:0/3531264465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f90153960 0x7f7f900c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f949a6640 1 -- 172.21.15.12:0/3531264465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f9015a250 msgr2=0x7f7f90141fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f949a6640 1 --2- 172.21.15.12:0/3531264465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f9015a250 0x7f7f90141fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f949a6640 1 -- 172.21.15.12:0/3531264465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f90153960 msgr2=0x7f7f900c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f949a6640 1 --2- 172.21.15.12:0/3531264465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f90153960 0x7f7f900c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f949a6640 1 -- 172.21.15.12:0/3531264465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f88009a70 con 0x7f7f9015cbc0 2023-10-30T12:35:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f8f7fe640 1 --2- 172.21.15.12:0/3531264465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f9015a250 0x7f7f90141fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:35:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f949a6640 1 --2- 172.21.15.12:0/3531264465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f9015cbc0 0x7f7f90142520 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7f880ac000 tx=0x7f7f880196f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fd0877fe640 1 -- 172.21.15.12:0/3580036327 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0ac630 con 0x7fd08806ad90 2023-10-30T12:35:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fad727fc640 1 -- 172.21.15.12:0/3805076033 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad640a8970 con 0x7fad74158090 2023-10-30T12:35:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fad727fc640 1 -- 172.21.15.12:0/3805076033 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad640a8ad0 con 0x7fad74158090 2023-10-30T12:35:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fad727fc640 1 -- 172.21.15.12:0/3805076033 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad640a8e60 con 0x7fad74158090 2023-10-30T12:35:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f112d59a640 1 -- 172.21.15.12:0/2169569506 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11240ab600 con 0x7f113015a2c0 2023-10-30T12:35:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f8cff9640 1 -- 172.21.15.12:0/3531264465 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880bf070 con 0x7f7f9015cbc0 2023-10-30T12:35:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f8cff9640 1 -- 172.21.15.12:0/3531264465 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f880aeb60 con 0x7f7f9015cbc0 2023-10-30T12:35:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f8cff9640 1 -- 172.21.15.12:0/3531264465 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880b7410 con 0x7f7f9015cbc0 2023-10-30T12:35:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f951a7640 1 -- 172.21.15.12:0/3531264465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f90158c00 con 0x7f7f9015cbc0 2023-10-30T12:35:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f951a7640 1 -- 172.21.15.12:0/3531264465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f901591c0 con 0x7f7f9015cbc0 2023-10-30T12:35:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fd0877fe640 1 -- 172.21.15.12:0/3580036327 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c0ac790 con 0x7fd08806ad90 2023-10-30T12:35:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fd0877fe640 1 -- 172.21.15.12:0/3580036327 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0acb20 con 0x7fd08806ad90 2023-10-30T12:35:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f112d59a640 1 -- 172.21.15.12:0/2169569506 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11240a7050 con 0x7f113015a2c0 2023-10-30T12:35:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f112d59a640 1 -- 172.21.15.12:0/2169569506 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11240ab980 con 0x7f113015a2c0 2023-10-30T12:35:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fad79c46640 1 -- 172.21.15.12:0/3805076033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad74158090 msgr2=0x7fad74161be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fad79c46640 1 --2- 172.21.15.12:0/3805076033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad74158090 0x7fad74161be0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad640097c0 tx=0x7fad6401ec70 comp rx=0 tx=0).stop 2023-10-30T12:35:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fad79c46640 1 -- 172.21.15.12:0/3805076033 shutdown_connections 2023-10-30T12:35:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fad79c46640 1 --2- 172.21.15.12:0/3805076033 >> v2:172.21.15.154:3300/0 conn(0x7fad740c0ed0 0x7fad7406a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fad79c46640 1 --2- 172.21.15.12:0/3805076033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad74158090 0x7fad74161be0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7faa3d0dc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faa3000a0c0 0x7faa300e0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7faa3d0dc640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa3000a4a0 0x7faa30050270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7faa3d0dc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faa300d68e0 0x7faa300d6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7faa3d0dc640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa300d72d0 con 0x7faa300d68e0 2023-10-30T12:35:16.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7faa3d0dc640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa3004fe10 con 0x7faa3000a0c0 2023-10-30T12:35:16.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7faa3d0dc640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa300475d0 con 0x7faa3000a4a0 2023-10-30T12:35:16.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fad79c46640 1 -- 172.21.15.12:0/3805076033 >> 172.21.15.12:0/3805076033 conn(0x7fad740c7910 msgr2=0x7fad740c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7d0affd640 1 -- 172.21.15.12:0/2737370610 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7d04050ed0 con 0x7f7d14153c60 2023-10-30T12:35:16.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fd08efa1640 1 -- 172.21.15.12:0/3580036327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd08806ad90 msgr2=0x7fd088152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fd08efa1640 1 --2- 172.21.15.12:0/3580036327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd08806ad90 0x7fd088152de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd08806f4d0 tx=0x7fd07c01c500 comp rx=0 tx=0).stop 2023-10-30T12:35:16.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fad79c46640 1 -- 172.21.15.12:0/3805076033 shutdown_connections 2023-10-30T12:35:16.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7fad79c46640 1 -- 172.21.15.12:0/3805076033 wait complete. 2023-10-30T12:35:16.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7f7f951a7640 1 -- 172.21.15.12:0/3531264465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9004e170 con 0x7f7f9015cbc0 2023-10-30T12:35:16.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.306+0000 7faa377fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa3000a4a0 0x7faa30050270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7fd08efa1640 1 -- 172.21.15.12:0/3580036327 shutdown_connections 2023-10-30T12:35:16.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7fd08efa1640 1 --2- 172.21.15.12:0/3580036327 >> v2:172.21.15.154:3300/0 conn(0x7fd0880c6670 0x7fd088069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7fd08efa1640 1 --2- 172.21.15.12:0/3580036327 >> v2:172.21.15.12:3300/0 conn(0x7fd0880c8430 0x7fd088156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7fd08efa1640 1 --2- 172.21.15.12:0/3580036327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd08806ad90 0x7fd088152de0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7faa377fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa3000a4a0 0x7faa30050270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40788/0 (socket says 172.21.15.12:40788) 2023-10-30T12:35:16.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7faa377fe640 1 -- 172.21.15.12:0/1729299089 learned_addr learned my addr 172.21.15.12:0/1729299089 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:16.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7faa37fff640 1 --2- 172.21.15.12:0/1729299089 >> v2:172.21.15.154:3300/0 conn(0x7faa3000a0c0 0x7faa300e0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7faa3c8db640 1 --2- 172.21.15.12:0/1729299089 >> v2:172.21.15.12:3300/0 conn(0x7faa300d68e0 0x7faa300d6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7faa377fe640 1 -- 172.21.15.12:0/1729299089 >> v2:172.21.15.154:3300/0 conn(0x7faa3000a0c0 msgr2=0x7faa300e0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7faa377fe640 1 --2- 172.21.15.12:0/1729299089 >> v2:172.21.15.154:3300/0 conn(0x7faa3000a0c0 0x7faa300e0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7faa377fe640 1 -- 172.21.15.12:0/1729299089 >> v2:172.21.15.12:3300/0 conn(0x7faa300d68e0 msgr2=0x7faa300d6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7faa377fe640 1 --2- 172.21.15.12:0/1729299089 >> v2:172.21.15.12:3300/0 conn(0x7faa300d68e0 0x7faa300d6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7faa377fe640 1 -- 172.21.15.12:0/1729299089 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa300478b0 con 0x7faa3000a4a0 2023-10-30T12:35:16.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7faa37fff640 1 --2- 172.21.15.12:0/1729299089 >> v2:172.21.15.154:3300/0 conn(0x7faa3000a0c0 0x7faa300e0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:16.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7fd08efa1640 1 -- 172.21.15.12:0/3580036327 >> 172.21.15.12:0/3580036327 conn(0x7fd0880c7910 msgr2=0x7fd0880c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7f7f8cff9640 1 -- 172.21.15.12:0/3531264465 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7f880b4050 con 0x7f7f9015cbc0 2023-10-30T12:35:16.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7f7f8cff9640 1 --2- 172.21.15.12:0/3531264465 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7f7c066520 0x7f7f7c0689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7f7f8cff9640 1 -- 172.21.15.12:0/3531264465 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f8811e4c0 con 0x7f7f9015cbc0 2023-10-30T12:35:16.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7fd08efa1640 1 -- 172.21.15.12:0/3580036327 shutdown_connections 2023-10-30T12:35:16.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7fd08efa1640 1 -- 172.21.15.12:0/3580036327 wait complete. 2023-10-30T12:35:16.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7fd08efa1640 1 Processor -- start 2023-10-30T12:35:16.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7f112f59e640 1 -- 172.21.15.12:0/2169569506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f113015a2c0 msgr2=0x7f113015aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7f112f59e640 1 --2- 172.21.15.12:0/2169569506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f113015a2c0 0x7f113015aac0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f11300c5ca0 tx=0x7f112401c470 comp rx=0 tx=0).stop 2023-10-30T12:35:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7f112f59e640 1 -- 172.21.15.12:0/2169569506 shutdown_connections 2023-10-30T12:35:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7f112f59e640 1 --2- 172.21.15.12:0/2169569506 >> v2:172.21.15.12:3300/0 conn(0x7f113015cbc0 0x7f113015cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7f112f59e640 1 --2- 172.21.15.12:0/2169569506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f113015a2c0 0x7f113015aac0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7f112f59e640 1 --2- 172.21.15.12:0/2169569506 >> v2:172.21.15.154:3300/0 conn(0x7f1130153850 0x7f11300c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7f112f59e640 1 -- 172.21.15.12:0/2169569506 >> 172.21.15.12:0/2169569506 conn(0x7f11300cae10 msgr2=0x7f11300cb220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7f112f59e640 1 -- 172.21.15.12:0/2169569506 shutdown_connections 2023-10-30T12:35:16.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.310+0000 7f112f59e640 1 -- 172.21.15.12:0/2169569506 wait complete. 2023-10-30T12:35:16.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fd08efa1640 1 -- start start 2023-10-30T12:35:16.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fad79c46640 1 Processor -- start 2023-10-30T12:35:16.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7f112f59e640 1 Processor -- start 2023-10-30T12:35:16.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7faa377fe640 1 --2- 172.21.15.12:0/1729299089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa3000a4a0 0x7faa30050270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa3004f0e0 tx=0x7faa2401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1efc8f6432701c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7f7f8ffff640 1 --2- 172.21.15.12:0/3531264465 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7f7c066520 0x7f7f7c0689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7faa36ffd640 1 -- 172.21.15.12:0/1729299089 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa240ac600 con 0x7faa3000a4a0 2023-10-30T12:35:16.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7faa36ffd640 1 -- 172.21.15.12:0/1729299089 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa240a8050 con 0x7faa3000a4a0 2023-10-30T12:35:16.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7faa36ffd640 1 -- 172.21.15.12:0/1729299089 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa240ac980 con 0x7faa3000a4a0 2023-10-30T12:35:16.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7f7f8ffff640 1 --2- 172.21.15.12:0/3531264465 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7f7c066520 0x7f7f7c0689e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7f80005890 tx=0x7f7f80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7faa3d0dc640 1 -- 172.21.15.12:0/1729299089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa3000a4a0 msgr2=0x7faa30050270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7faa3d0dc640 1 --2- 172.21.15.12:0/1729299089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa3000a4a0 0x7faa30050270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa3004f0e0 tx=0x7faa2401c4d0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7faa3d0dc640 1 -- 172.21.15.12:0/1729299089 shutdown_connections 2023-10-30T12:35:16.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7faa3d0dc640 1 --2- 172.21.15.12:0/1729299089 >> v2:172.21.15.12:3300/0 conn(0x7faa300d68e0 0x7faa300d6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7faa3d0dc640 1 --2- 172.21.15.12:0/1729299089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa3000a4a0 0x7faa30050270 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7faa3d0dc640 1 --2- 172.21.15.12:0/1729299089 >> v2:172.21.15.154:3300/0 conn(0x7faa3000a0c0 0x7faa300e0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7faa3d0dc640 1 -- 172.21.15.12:0/1729299089 >> 172.21.15.12:0/1729299089 conn(0x7faa3000d9e0 msgr2=0x7faa3000c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7faa3d0dc640 1 -- 172.21.15.12:0/1729299089 shutdown_connections 2023-10-30T12:35:16.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fd08efa1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd08806ad90 0x7fd08814f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7faa3d0dc640 1 -- 172.21.15.12:0/1729299089 wait complete. 2023-10-30T12:35:16.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fd08efa1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0880c6670 0x7fd0880ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fad79c46640 1 -- start start 2023-10-30T12:35:16.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fad79c46640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad740c0ed0 0x7fad7414a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fad79c46640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad74158090 0x7fad740c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fad79c46640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad7414f680 0x7fad740c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fad79c46640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7415ad30 con 0x7fad74158090 2023-10-30T12:35:16.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fad79c46640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7414f3c0 con 0x7fad7414f680 2023-10-30T12:35:16.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fad79c46640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7414ee00 con 0x7fad740c0ed0 2023-10-30T12:35:16.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fd08cd3d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd08806ad90 0x7fd08814f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fd08cd3d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd08806ad90 0x7fd08814f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40802/0 (socket says 172.21.15.12:40802) 2023-10-30T12:35:16.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fd08cd3d640 1 -- 172.21.15.12:0/1673943586 learned_addr learned my addr 172.21.15.12:0/1673943586 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:16.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fd087fff640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0880c6670 0x7fd0880ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fd08efa1640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0880c8430 0x7fd088143e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7f112f59e640 1 -- start start 2023-10-30T12:35:16.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7f112f59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1130153850 0x7f1130202570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7f112f59e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f113015a2c0 0x7f11300c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7f112f59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f113015cbc0 0x7f1130152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7f112f59e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1130200780 con 0x7f113015cbc0 2023-10-30T12:35:16.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7f112f59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113015e0c0 con 0x7f1130153850 2023-10-30T12:35:16.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7f112f59e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1130202ab0 con 0x7f113015a2c0 2023-10-30T12:35:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.314+0000 7fad73fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad7414f680 0x7fad740c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd08efa1640 1 -- 172.21.15.12:0/1673943586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd088144cb0 con 0x7fd0880c6670 2023-10-30T12:35:16.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd08efa1640 1 -- 172.21.15.12:0/1673943586 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0880cac30 con 0x7fd0880c8430 2023-10-30T12:35:16.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd08efa1640 1 -- 172.21.15.12:0/1673943586 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0881439b0 con 0x7fd08806ad90 2023-10-30T12:35:16.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f112e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1130153850 0x7f1130202570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f112e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1130153850 0x7f1130202570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58384/0 (socket says 172.21.15.12:58384) 2023-10-30T12:35:16.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f112e59c640 1 -- 172.21.15.12:0/2302139290 learned_addr learned my addr 172.21.15.12:0/2302139290 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:16.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd08d53e640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0880c8430 0x7fd088143e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f112dd9b640 1 --2- 172.21.15.12:0/2302139290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f113015a2c0 0x7f11300c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fad73fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad7414f680 0x7fad740c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58378/0 (socket says 172.21.15.12:58378) 2023-10-30T12:35:16.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fad73fff640 1 -- 172.21.15.12:0/3440970248 learned_addr learned my addr 172.21.15.12:0/3440970248 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:16.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fad72ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad74158090 0x7fad740c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fad737fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad740c0ed0 0x7fad7414a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd08d53e640 1 -- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd08806ad90 msgr2=0x7fd08814f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd08d53e640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd08806ad90 0x7fd08814f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd08d53e640 1 -- 172.21.15.12:0/1673943586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0880c6670 msgr2=0x7fd0880ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd08d53e640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0880c6670 0x7fd0880ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd08d53e640 1 -- 172.21.15.12:0/1673943586 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07c0094d0 con 0x7fd0880c8430 2023-10-30T12:35:16.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd08cd3d640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd08806ad90 0x7fd08814f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd08d53e640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0880c8430 0x7fd088143e70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd07800b3e0 tx=0x7fd078009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd0857fa640 1 -- 172.21.15.12:0/1673943586 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078011830 con 0x7fd0880c8430 2023-10-30T12:35:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd0857fa640 1 -- 172.21.15.12:0/1673943586 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd078011990 con 0x7fd0880c8430 2023-10-30T12:35:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd0857fa640 1 -- 172.21.15.12:0/1673943586 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078011d00 con 0x7fd0880c8430 2023-10-30T12:35:16.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f112ed9d640 1 --2- 172.21.15.12:0/2302139290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f113015cbc0 0x7f1130152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f112e59c640 1 -- 172.21.15.12:0/2302139290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f113015a2c0 msgr2=0x7f11300c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f112e59c640 1 --2- 172.21.15.12:0/2302139290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f113015a2c0 0x7f11300c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f112e59c640 1 -- 172.21.15.12:0/2302139290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f113015cbc0 msgr2=0x7f1130152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f112e59c640 1 --2- 172.21.15.12:0/2302139290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f113015cbc0 0x7f1130152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f112e59c640 1 -- 172.21.15.12:0/2302139290 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11240094d0 con 0x7f1130153850 2023-10-30T12:35:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fad72ffd640 1 -- 172.21.15.12:0/3440970248 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad740c0ed0 msgr2=0x7fad7414a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fad72ffd640 1 --2- 172.21.15.12:0/3440970248 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad740c0ed0 0x7fad7414a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fad72ffd640 1 -- 172.21.15.12:0/3440970248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad7414f680 msgr2=0x7fad740c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fad72ffd640 1 --2- 172.21.15.12:0/3440970248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad7414f680 0x7fad740c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fad72ffd640 1 -- 172.21.15.12:0/3440970248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad64009420 con 0x7fad74158090 2023-10-30T12:35:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f112dd9b640 1 --2- 172.21.15.12:0/2302139290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f113015a2c0 0x7f11300c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fad72ffd640 1 --2- 172.21.15.12:0/3440970248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad74158090 0x7fad740c5e40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fad68000f70 tx=0x7fad6800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd08efa1640 1 -- 172.21.15.12:0/1673943586 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08813d3d0 con 0x7fd0880c8430 2023-10-30T12:35:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd08efa1640 1 -- 172.21.15.12:0/1673943586 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08813d990 con 0x7fd0880c8430 2023-10-30T12:35:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7faa3d0dc640 1 Processor -- start 2023-10-30T12:35:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f112e59c640 1 --2- 172.21.15.12:0/2302139290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1130153850 0x7f1130202570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f112000d800 tx=0x7f112000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fad5bfff640 1 -- 172.21.15.12:0/3440970248 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad680052d0 con 0x7fad74158090 2023-10-30T12:35:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fad5bfff640 1 -- 172.21.15.12:0/3440970248 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad68005430 con 0x7fad74158090 2023-10-30T12:35:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fad5bfff640 1 -- 172.21.15.12:0/3440970248 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68003840 con 0x7fad74158090 2023-10-30T12:35:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7faa3d0dc640 1 -- start start 2023-10-30T12:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f7f8cff9640 1 -- 172.21.15.12:0/3531264465 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7f880eb890 con 0x7f7f9015cbc0 2023-10-30T12:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f111effd640 1 -- 172.21.15.12:0/2302139290 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112000b660 con 0x7f1130153850 2023-10-30T12:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f111effd640 1 -- 172.21.15.12:0/2302139290 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112000b7c0 con 0x7f1130153850 2023-10-30T12:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f111effd640 1 -- 172.21.15.12:0/2302139290 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112000bb30 con 0x7f1130153850 2023-10-30T12:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7f112f59e640 1 -- 172.21.15.12:0/2302139290 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11300c57c0 con 0x7f1130153850 2023-10-30T12:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7f112f59e640 1 -- 172.21.15.12:0/2302139290 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11300c42f0 con 0x7f1130153850 2023-10-30T12:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.318+0000 7fd08efa1640 1 -- 172.21.15.12:0/1673943586 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0880c6130 con 0x7fd0880c8430 2023-10-30T12:35:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7fad79c46640 1 -- 172.21.15.12:0/3440970248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7414f0e0 con 0x7fad74158090 2023-10-30T12:35:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7fad79c46640 1 -- 172.21.15.12:0/3440970248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad741f4cd0 con 0x7fad74158090 2023-10-30T12:35:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7faa3d0dc640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa3000a0c0 0x7faa300d7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7faa3d0dc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa3000a4a0 0x7faa300d5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7faa3d0dc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa300d68e0 0x7faa30049810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7faa3d0dc640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa300d8890 con 0x7faa3000a4a0 2023-10-30T12:35:16.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7faa3d0dc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa300d9830 con 0x7faa300d68e0 2023-10-30T12:35:16.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7faa3d0dc640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3004b880 con 0x7faa3000a0c0 2023-10-30T12:35:16.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7faa377fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa3000a4a0 0x7faa300d5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7faa377fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa3000a4a0 0x7faa300d5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44698/0 (socket says 172.21.15.12:44698) 2023-10-30T12:35:16.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7faa377fe640 1 -- 172.21.15.12:0/1630732992 learned_addr learned my addr 172.21.15.12:0/1630732992 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:16.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7faa377fe640 1 -- 172.21.15.12:0/1630732992 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa3000a0c0 msgr2=0x7faa300d7c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-30T12:35:16.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7faa377fe640 1 --2- 172.21.15.12:0/1630732992 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa3000a0c0 0x7faa300d7c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7faa377fe640 1 -- 172.21.15.12:0/1630732992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa300d68e0 msgr2=0x7faa30049810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-30T12:35:16.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7faa377fe640 1 --2- 172.21.15.12:0/1630732992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa300d68e0 0x7faa30049810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7faa377fe640 1 -- 172.21.15.12:0/1630732992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa240094d0 con 0x7faa3000a4a0 2023-10-30T12:35:16.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7faa377fe640 1 --2- 172.21.15.12:0/1630732992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa3000a4a0 0x7faa300d5200 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa2401ab80 tx=0x7faa2401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7f112f59e640 1 -- 172.21.15.12:0/2302139290 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113004e170 con 0x7f1130153850 2023-10-30T12:35:16.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7f111effd640 1 -- 172.21.15.12:0/2302139290 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f112000bc90 con 0x7f1130153850 2023-10-30T12:35:16.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7fd0857fa640 1 -- 172.21.15.12:0/1673943586 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd07803a450 con 0x7fd0880c8430 2023-10-30T12:35:16.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7f111effd640 1 --2- 172.21.15.12:0/2302139290 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1110066520 0x7f11100689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7fd0857fa640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd0740664d0 0x7fd074068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7f111effd640 1 -- 172.21.15.12:0/2302139290 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1120084300 con 0x7f1130153850 2023-10-30T12:35:16.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7fd0857fa640 1 -- 172.21.15.12:0/1673943586 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0780844b0 con 0x7fd0880c8430 2023-10-30T12:35:16.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.322+0000 7fad79c46640 1 -- 172.21.15.12:0/3440970248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad7406aed0 con 0x7fad74158090 2023-10-30T12:35:16.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7fad5bfff640 1 -- 172.21.15.12:0/3440970248 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fad680039a0 con 0x7fad74158090 2023-10-30T12:35:16.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7fd08cd3d640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd0740664d0 0x7fd074068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7faa34ff9640 1 -- 172.21.15.12:0/1630732992 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa240bc070 con 0x7faa3000a4a0 2023-10-30T12:35:16.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7faa3d0dc640 1 -- 172.21.15.12:0/1630732992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa300d82d0 con 0x7faa3000a4a0 2023-10-30T12:35:16.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7faa3d0dc640 1 -- 172.21.15.12:0/1630732992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa300d9a30 con 0x7faa3000a4a0 2023-10-30T12:35:16.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7f112dd9b640 1 --2- 172.21.15.12:0/2302139290 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1110066520 0x7f11100689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7fad5bfff640 1 --2- 172.21.15.12:0/3440970248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fad60066570 0x7fad60068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7fad5bfff640 1 -- 172.21.15.12:0/3440970248 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad680837e0 con 0x7fad74158090 2023-10-30T12:35:16.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7faa3d0dc640 1 -- 172.21.15.12:0/1630732992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa300057a0 con 0x7faa3000a4a0 2023-10-30T12:35:16.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7fd0857fa640 1 -- 172.21.15.12:0/1673943586 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd078051910 con 0x7fd0880c8430 2023-10-30T12:35:16.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7fd08cd3d640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd0740664d0 0x7fd074068990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd07c000c10 tx=0x7fd07c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7fad737fe640 1 --2- 172.21.15.12:0/3440970248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fad60066570 0x7fad60068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.326+0000 7fad737fe640 1 --2- 172.21.15.12:0/3440970248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fad60066570 0x7fad60068a30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fad640b2000 tx=0x7fad640aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.330+0000 7f112dd9b640 1 --2- 172.21.15.12:0/2302139290 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1110066520 0x7f11100689e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f11300c5ca0 tx=0x7f1124008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.330+0000 7f111effd640 1 -- 172.21.15.12:0/2302139290 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1120051760 con 0x7f1130153850 2023-10-30T12:35:16.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.338+0000 7fad5bfff640 1 -- 172.21.15.12:0/3440970248 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fad68050d80 con 0x7fad74158090 2023-10-30T12:35:16.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.338+0000 7faa34ff9640 1 -- 172.21.15.12:0/1630732992 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa240ace70 con 0x7faa3000a4a0 2023-10-30T12:35:16.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.338+0000 7faa34ff9640 1 -- 172.21.15.12:0/1630732992 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa240b7650 con 0x7faa3000a4a0 2023-10-30T12:35:16.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.338+0000 7faa34ff9640 1 -- 172.21.15.12:0/1630732992 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faa240b3070 con 0x7faa3000a4a0 2023-10-30T12:35:16.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.342+0000 7faa34ff9640 1 --2- 172.21.15.12:0/1630732992 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faa1c066520 0x7faa1c0689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.342+0000 7faa34ff9640 1 -- 172.21.15.12:0/1630732992 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa2411e7c0 con 0x7faa3000a4a0 2023-10-30T12:35:16.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.350+0000 7faa37fff640 1 --2- 172.21.15.12:0/1630732992 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faa1c066520 0x7faa1c0689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.350+0000 7faa37fff640 1 --2- 172.21.15.12:0/1630732992 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faa1c066520 0x7faa1c0689e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00a890 tx=0x7faa2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.350+0000 7faa34ff9640 1 -- 172.21.15.12:0/1630732992 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faa240ebcb0 con 0x7faa3000a4a0 2023-10-30T12:35:16.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.594+0000 7fa806796640 1 -- 172.21.15.12:0/1385919398 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa7e0005510 con 0x7fa7f80d4e80 2023-10-30T12:35:16.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.598+0000 7fa7f5ffb640 1 -- 172.21.15.12:0/1385919398 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa7f00eeb40 con 0x7fa7f80d4e80 2023-10-30T12:35:16.601 INFO:teuthology.orchestra.run.smithi012.stdout:60129542153 2023-10-30T12:35:16.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.602+0000 7f5809ffb640 1 -- 172.21.15.12:0/129744871 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5818183cd0 con 0x7f5818183fb0 2023-10-30T12:35:16.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.602+0000 7f580bfff640 1 -- 172.21.15.12:0/129744871 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5818183cd0 con 0x7f5818183fb0 2023-10-30T12:35:16.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.602+0000 7fa806796640 1 -- 172.21.15.12:0/1385919398 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7e4066530 msgr2=0x7fa7e40689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.602+0000 7fa806796640 1 --2- 172.21.15.12:0/1385919398 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7e4066530 0x7fa7e40689f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa800063ef0 tx=0x7fa80006d040 comp rx=0 tx=0).stop 2023-10-30T12:35:16.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.602+0000 7fa806796640 1 -- 172.21.15.12:0/1385919398 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7f80d4e80 msgr2=0x7fa7f80e43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.602+0000 7fa806796640 1 --2- 172.21.15.12:0/1385919398 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7f80d4e80 0x7fa7f80e43d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa7f001ef50 tx=0x7fa7f001b450 comp rx=0 tx=0).stop 2023-10-30T12:35:16.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.602+0000 7fa806796640 1 -- 172.21.15.12:0/1385919398 shutdown_connections 2023-10-30T12:35:16.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.602+0000 7fa806796640 1 --2- 172.21.15.12:0/1385919398 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7e4066530 0x7fa7e40689f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.602+0000 7fa806796640 1 --2- 172.21.15.12:0/1385919398 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7f80d4e80 0x7fa7f80e43d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.602+0000 7fa806796640 1 --2- 172.21.15.12:0/1385919398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f804c8c0 0x7fa7f80e3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.602+0000 7fa806796640 1 --2- 172.21.15.12:0/1385919398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f8034ff0 0x7fa7f80dd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.602+0000 7fa806796640 1 -- 172.21.15.12:0/1385919398 >> 172.21.15.12:0/1385919398 conn(0x7fa7f80502d0 msgr2=0x7fa7f8005610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.608 INFO:teuthology.orchestra.run.smithi012.stdout:60129542151 2023-10-30T12:35:16.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.602+0000 7fa806796640 1 -- 172.21.15.12:0/1385919398 shutdown_connections 2023-10-30T12:35:16.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.606+0000 7fa806796640 1 -- 172.21.15.12:0/1385919398 wait complete. 2023-10-30T12:35:16.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.606+0000 7f5809ffb640 1 -- 172.21.15.12:0/129744871 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f58040664a0 msgr2=0x7f5804068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.606+0000 7f5809ffb640 1 --2- 172.21.15.12:0/129744871 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f58040664a0 0x7f5804068960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5820075640 tx=0x7f582006d040 comp rx=0 tx=0).stop 2023-10-30T12:35:16.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.606+0000 7f5809ffb640 1 -- 172.21.15.12:0/129744871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5818183fb0 msgr2=0x7f5818184430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.606+0000 7f5809ffb640 1 --2- 172.21.15.12:0/129744871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5818183fb0 0x7f5818184430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f581804e1b0 tx=0x7f580c019a10 comp rx=0 tx=0).stop 2023-10-30T12:35:16.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.606+0000 7f5809ffb640 1 -- 172.21.15.12:0/129744871 shutdown_connections 2023-10-30T12:35:16.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.606+0000 7f5809ffb640 1 --2- 172.21.15.12:0/129744871 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f58040664a0 0x7f5804068960 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.606+0000 7f5809ffb640 1 --2- 172.21.15.12:0/129744871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5818183fb0 0x7f5818184430 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.606+0000 7f5809ffb640 1 --2- 172.21.15.12:0/129744871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58180dbf60 0x7f58180dc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.606+0000 7f5809ffb640 1 --2- 172.21.15.12:0/129744871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5810004e60 0x7f58180d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.606+0000 7f5809ffb640 1 -- 172.21.15.12:0/129744871 >> 172.21.15.12:0/129744871 conn(0x7f58180502e0 msgr2=0x7f58180506c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.606+0000 7f5809ffb640 1 -- 172.21.15.12:0/129744871 shutdown_connections 2023-10-30T12:35:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.606+0000 7f5809ffb640 1 -- 172.21.15.12:0/129744871 wait complete. 2023-10-30T12:35:16.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.0 2023-10-30T12:35:16.644 DEBUG:teuthology.parallel:result is None 2023-10-30T12:35:16.646 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.1 2023-10-30T12:35:16.647 DEBUG:teuthology.parallel:result is None 2023-10-30T12:35:16.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.642+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2737370610 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7cfc005510 con 0x7f7d14153c60 2023-10-30T12:35:16.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.646+0000 7f7d0affd640 1 -- 172.21.15.12:0/2737370610 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7d04015090 con 0x7f7d14153c60 2023-10-30T12:35:16.649 INFO:teuthology.orchestra.run.smithi012.stdout:60129542151 2023-10-30T12:35:16.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.646+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2737370610 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cf8066570 msgr2=0x7f7cf8068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.646+0000 7f7d1c6ea640 1 --2- 172.21.15.12:0/2737370610 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cf8066570 0x7f7cf8068a30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009ae0 tx=0x7f7d10008040 comp rx=0 tx=0).stop 2023-10-30T12:35:16.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.646+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2737370610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7d14153c60 msgr2=0x7f7d14148b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.646+0000 7f7d1c6ea640 1 --2- 172.21.15.12:0/2737370610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7d14153c60 0x7f7d14148b40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400b410 tx=0x7f7d0400b8d0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.646+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2737370610 shutdown_connections 2023-10-30T12:35:16.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.646+0000 7f7d1c6ea640 1 --2- 172.21.15.12:0/2737370610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d14159830 0x7f7d141feb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.646+0000 7f7d1c6ea640 1 --2- 172.21.15.12:0/2737370610 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cf8066570 0x7f7cf8068a30 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009ae0 tx=0x7f7d10008040 comp rx=0 tx=0).stop 2023-10-30T12:35:16.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.646+0000 7f7d1c6ea640 1 --2- 172.21.15.12:0/2737370610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7d14153c60 0x7f7d14148b40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.646+0000 7f7d1c6ea640 1 --2- 172.21.15.12:0/2737370610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d140c4430 0x7f7d14148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.646+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2737370610 >> 172.21.15.12:0/2737370610 conn(0x7f7d1406bfe0 msgr2=0x7f7d14146c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.646+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2737370610 shutdown_connections 2023-10-30T12:35:16.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.646+0000 7f7d1c6ea640 1 -- 172.21.15.12:0/2737370610 wait complete. 2023-10-30T12:35:16.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.654+0000 7fad79c46640 1 -- 172.21.15.12:0/3440970248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fad7404e170 con 0x7fad74158090 2023-10-30T12:35:16.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.654+0000 7fad5bfff640 1 -- 172.21.15.12:0/3440970248 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fad68017020 con 0x7fad74158090 2023-10-30T12:35:16.658 INFO:teuthology.orchestra.run.smithi012.stdout:60129542153 2023-10-30T12:35:16.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.654+0000 7fad79c46640 1 -- 172.21.15.12:0/3440970248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fad60066570 msgr2=0x7fad60068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.654+0000 7fad79c46640 1 --2- 172.21.15.12:0/3440970248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fad60066570 0x7fad60068a30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fad640b2000 tx=0x7fad640aa070 comp rx=0 tx=0).stop 2023-10-30T12:35:16.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.654+0000 7fad79c46640 1 -- 172.21.15.12:0/3440970248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad74158090 msgr2=0x7fad740c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.654+0000 7fad79c46640 1 --2- 172.21.15.12:0/3440970248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad74158090 0x7fad740c5e40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fad68000f70 tx=0x7fad6800c890 comp rx=0 tx=0).stop 2023-10-30T12:35:16.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.654+0000 7fad79c46640 1 -- 172.21.15.12:0/3440970248 shutdown_connections 2023-10-30T12:35:16.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.654+0000 7fad79c46640 1 --2- 172.21.15.12:0/3440970248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad7414f680 0x7fad740c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.654+0000 7fad79c46640 1 --2- 172.21.15.12:0/3440970248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad74158090 0x7fad740c5e40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.654+0000 7fad79c46640 1 --2- 172.21.15.12:0/3440970248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fad60066570 0x7fad60068a30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.654+0000 7fad79c46640 1 --2- 172.21.15.12:0/3440970248 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad740c0ed0 0x7fad7414a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.654+0000 7fad79c46640 1 -- 172.21.15.12:0/3440970248 >> 172.21.15.12:0/3440970248 conn(0x7fad740c7910 msgr2=0x7fad7415d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.658+0000 7fad79c46640 1 -- 172.21.15.12:0/3440970248 shutdown_connections 2023-10-30T12:35:16.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.658+0000 7fad79c46640 1 -- 172.21.15.12:0/3440970248 wait complete. 2023-10-30T12:35:16.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.662+0000 7f7f951a7640 1 -- 172.21.15.12:0/3531264465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7f901fa020 con 0x7f7f9015cbc0 2023-10-30T12:35:16.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.662+0000 7f7f8cff9640 1 -- 172.21.15.12:0/3531264465 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7f880b2090 con 0x7f7f9015cbc0 2023-10-30T12:35:16.666 INFO:teuthology.orchestra.run.smithi012.stdout:60129542151 2023-10-30T12:35:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.666+0000 7f7f951a7640 1 -- 172.21.15.12:0/3531264465 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7f7c066520 msgr2=0x7f7f7c0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.666+0000 7f7f951a7640 1 --2- 172.21.15.12:0/3531264465 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7f7c066520 0x7f7f7c0689e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7f80005890 tx=0x7f7f80009040 comp rx=0 tx=0).stop 2023-10-30T12:35:16.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.666+0000 7fd08efa1640 1 -- 172.21.15.12:0/1673943586 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd0880c6130 con 0x7fd0880c8430 2023-10-30T12:35:16.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.666+0000 7f7f951a7640 1 -- 172.21.15.12:0/3531264465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f9015cbc0 msgr2=0x7f7f90142520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.666+0000 7f7f951a7640 1 --2- 172.21.15.12:0/3531264465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f9015cbc0 0x7f7f90142520 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7f880ac000 tx=0x7f7f880196f0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.666+0000 7f7f951a7640 1 -- 172.21.15.12:0/3531264465 shutdown_connections 2023-10-30T12:35:16.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.666+0000 7f7f951a7640 1 --2- 172.21.15.12:0/3531264465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f9015cbc0 0x7f7f90142520 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.666+0000 7f7f951a7640 1 --2- 172.21.15.12:0/3531264465 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7f7c066520 0x7f7f7c0689e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.666+0000 7f7f951a7640 1 --2- 172.21.15.12:0/3531264465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f9015a250 0x7f7f90141fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.666+0000 7f7f951a7640 1 --2- 172.21.15.12:0/3531264465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f90153960 0x7f7f900c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.666+0000 7f7f951a7640 1 -- 172.21.15.12:0/3531264465 >> 172.21.15.12:0/3531264465 conn(0x7f7f900cae10 msgr2=0x7f7f900c5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.666+0000 7f7f951a7640 1 -- 172.21.15.12:0/3531264465 shutdown_connections 2023-10-30T12:35:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.666+0000 7f7f951a7640 1 -- 172.21.15.12:0/3531264465 wait complete. 2023-10-30T12:35:16.674 INFO:teuthology.orchestra.run.smithi012.stdout:60129542151 2023-10-30T12:35:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.666+0000 7fd0857fa640 1 -- 172.21.15.12:0/1673943586 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd0880c6130 con 0x7fd0880c8430 2023-10-30T12:35:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.670+0000 7fd08efa1640 1 -- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd0740664d0 msgr2=0x7fd074068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.670+0000 7fd08efa1640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd0740664d0 0x7fd074068990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd07c000c10 tx=0x7fd07c0a7000 comp rx=0 tx=0).stop 2023-10-30T12:35:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.670+0000 7fd08efa1640 1 -- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0880c8430 msgr2=0x7fd088143e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.670+0000 7fd08efa1640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0880c8430 0x7fd088143e70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd07800b3e0 tx=0x7fd078009f90 comp rx=0 tx=0).stop 2023-10-30T12:35:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.670+0000 7f112f59e640 1 -- 172.21.15.12:0/2302139290 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f11300c68e0 con 0x7f1130153850 2023-10-30T12:35:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.670+0000 7fd08efa1640 1 -- 172.21.15.12:0/1673943586 shutdown_connections 2023-10-30T12:35:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.670+0000 7fd08efa1640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0880c8430 0x7fd088143e70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.670+0000 7fd08efa1640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0880c6670 0x7fd0880ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.670+0000 7fd08efa1640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd0740664d0 0x7fd074068990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.670+0000 7fd08efa1640 1 --2- 172.21.15.12:0/1673943586 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd08806ad90 0x7fd08814f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.670+0000 7fd08efa1640 1 -- 172.21.15.12:0/1673943586 >> 172.21.15.12:0/1673943586 conn(0x7fd0880c7910 msgr2=0x7fd088071450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.670+0000 7fd08efa1640 1 -- 172.21.15.12:0/1673943586 shutdown_connections 2023-10-30T12:35:16.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.670+0000 7fd08efa1640 1 -- 172.21.15.12:0/1673943586 wait complete. 2023-10-30T12:35:16.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.670+0000 7f111effd640 1 -- 172.21.15.12:0/2302139290 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1120054d80 con 0x7f1130153850 2023-10-30T12:35:16.680 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.2 2023-10-30T12:35:16.681 INFO:teuthology.orchestra.run.smithi012.stdout:60129542153 2023-10-30T12:35:16.681 DEBUG:teuthology.parallel:result is None 2023-10-30T12:35:16.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.674+0000 7f112f59e640 1 -- 172.21.15.12:0/2302139290 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1110066520 msgr2=0x7f11100689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.674+0000 7f112f59e640 1 --2- 172.21.15.12:0/2302139290 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1110066520 0x7f11100689e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f11300c5ca0 tx=0x7f1124008040 comp rx=0 tx=0).stop 2023-10-30T12:35:16.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.674+0000 7f112f59e640 1 -- 172.21.15.12:0/2302139290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1130153850 msgr2=0x7f1130202570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.674+0000 7f112f59e640 1 --2- 172.21.15.12:0/2302139290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1130153850 0x7f1130202570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f112000d800 tx=0x7f112000dcc0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.674+0000 7f112f59e640 1 -- 172.21.15.12:0/2302139290 shutdown_connections 2023-10-30T12:35:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.674+0000 7f112f59e640 1 --2- 172.21.15.12:0/2302139290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f113015cbc0 0x7f1130152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.674+0000 7f112f59e640 1 --2- 172.21.15.12:0/2302139290 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1110066520 0x7f11100689e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.674+0000 7f112f59e640 1 --2- 172.21.15.12:0/2302139290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f113015a2c0 0x7f11300c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.674+0000 7f112f59e640 1 --2- 172.21.15.12:0/2302139290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1130153850 0x7f1130202570 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f112000d800 tx=0x7f112000dcc0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.674+0000 7f112f59e640 1 -- 172.21.15.12:0/2302139290 >> 172.21.15.12:0/2302139290 conn(0x7f11300cae10 msgr2=0x7f113006f440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.678+0000 7f112f59e640 1 -- 172.21.15.12:0/2302139290 shutdown_connections 2023-10-30T12:35:16.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.678+0000 7f112f59e640 1 -- 172.21.15.12:0/2302139290 wait complete. 2023-10-30T12:35:16.686 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.3 2023-10-30T12:35:16.687 DEBUG:teuthology.parallel:result is None 2023-10-30T12:35:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.686+0000 7faa3d0dc640 1 -- 172.21.15.12:0/1630732992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7faa30048ef0 con 0x7faa3000a4a0 2023-10-30T12:35:16.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.686+0000 7faa34ff9640 1 -- 172.21.15.12:0/1630732992 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faa240ebcb0 con 0x7faa3000a4a0 2023-10-30T12:35:16.691 INFO:teuthology.orchestra.run.smithi012.stdout:60129542152 2023-10-30T12:35:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.690+0000 7faa3d0dc640 1 -- 172.21.15.12:0/1630732992 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faa1c066520 msgr2=0x7faa1c0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.690+0000 7faa3d0dc640 1 --2- 172.21.15.12:0/1630732992 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faa1c066520 0x7faa1c0689e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00a890 tx=0x7faa2c008040 comp rx=0 tx=0).stop 2023-10-30T12:35:16.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.690+0000 7faa3d0dc640 1 -- 172.21.15.12:0/1630732992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa3000a4a0 msgr2=0x7faa300d5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.690+0000 7faa3d0dc640 1 --2- 172.21.15.12:0/1630732992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa3000a4a0 0x7faa300d5200 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa2401ab80 tx=0x7faa2401ad10 comp rx=0 tx=0).stop 2023-10-30T12:35:16.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.690+0000 7faa3d0dc640 1 -- 172.21.15.12:0/1630732992 shutdown_connections 2023-10-30T12:35:16.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.690+0000 7faa3d0dc640 1 --2- 172.21.15.12:0/1630732992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa300d68e0 0x7faa30049810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.690+0000 7faa3d0dc640 1 --2- 172.21.15.12:0/1630732992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa3000a4a0 0x7faa300d5200 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.690+0000 7faa3d0dc640 1 --2- 172.21.15.12:0/1630732992 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faa1c066520 0x7faa1c0689e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.690+0000 7faa3d0dc640 1 --2- 172.21.15.12:0/1630732992 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa3000a0c0 0x7faa300d7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.690+0000 7faa3d0dc640 1 -- 172.21.15.12:0/1630732992 >> 172.21.15.12:0/1630732992 conn(0x7faa3000d9e0 msgr2=0x7faa300d5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.690+0000 7faa3d0dc640 1 -- 172.21.15.12:0/1630732992 shutdown_connections 2023-10-30T12:35:16.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.690+0000 7faa3d0dc640 1 -- 172.21.15.12:0/1630732992 wait complete. 2023-10-30T12:35:16.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.4 2023-10-30T12:35:16.701 DEBUG:teuthology.parallel:result is None 2023-10-30T12:35:16.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.5 2023-10-30T12:35:16.704 DEBUG:teuthology.parallel:result is None 2023-10-30T12:35:16.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.7 2023-10-30T12:35:16.705 DEBUG:teuthology.parallel:result is None 2023-10-30T12:35:16.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.6 2023-10-30T12:35:16.716 DEBUG:teuthology.parallel:result is None 2023-10-30T12:35:16.716 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-30T12:35:16.717 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:35:16.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda3d817640 1 Processor -- start 2023-10-30T12:35:16.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda3d817640 1 -- start start 2023-10-30T12:35:16.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda3d817640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fda381538f0 0x7fda38159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda3d817640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fda380cefa0 0x7fda380cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda3d817640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda380ca6a0 con 0x7fda3815cd50 2023-10-30T12:35:16.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda3d817640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda380cf8f0 con 0x7fda381538f0 2023-10-30T12:35:16.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda3d817640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda3815d300 con 0x7fda380cefa0 2023-10-30T12:35:16.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda3c815640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fda381538f0 0x7fda38159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda37fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fda3815cd50 0x7fda380c9f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:55490/0 (socket says 172.21.15.12:55490) 2023-10-30T12:35:16.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda37fff640 1 -- 172.21.15.12:0/1662049133 learned_addr learned my addr 172.21.15.12:0/1662049133 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:16.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda3d016640 1 --2- 172.21.15.12:0/1662049133 >> v2:172.21.15.154:3300/0 conn(0x7fda380cefa0 0x7fda380cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda3c815640 1 -- 172.21.15.12:0/1662049133 >> v2:172.21.15.154:3300/0 conn(0x7fda380cefa0 msgr2=0x7fda380cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:16.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda3c815640 1 --2- 172.21.15.12:0/1662049133 >> v2:172.21.15.154:3300/0 conn(0x7fda380cefa0 0x7fda380cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda3c815640 1 -- 172.21.15.12:0/1662049133 >> v1:172.21.15.12:6789/0 conn(0x7fda3815cd50 legacy=0x7fda380c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda3c815640 1 -- 172.21.15.12:0/1662049133 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3815d5e0 con 0x7fda381538f0 2023-10-30T12:35:16.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.902+0000 7fda377fe640 1 -- 172.21.15.12:0/1662049133 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111729602 0 0) 0x7fda380ca6a0 con 0x7fda3815cd50 2023-10-30T12:35:16.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.906+0000 7fda3c815640 1 --2- 172.21.15.12:0/1662049133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda381538f0 0x7fda38159fb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fda38066a40 tx=0x7fda2c01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3d72b605790980 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.906+0000 7fda377fe640 1 -- 172.21.15.12:0/1662049133 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0a8950 con 0x7fda381538f0 2023-10-30T12:35:16.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.906+0000 7fda377fe640 1 -- 172.21.15.12:0/1662049133 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c0a8ab0 con 0x7fda381538f0 2023-10-30T12:35:16.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.906+0000 7fda377fe640 1 -- 172.21.15.12:0/1662049133 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0a8e40 con 0x7fda381538f0 2023-10-30T12:35:16.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.906+0000 7fda3d817640 1 -- 172.21.15.12:0/1662049133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda381538f0 msgr2=0x7fda38159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.906+0000 7fda3d817640 1 --2- 172.21.15.12:0/1662049133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda381538f0 0x7fda38159fb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fda38066a40 tx=0x7fda2c01ec50 comp rx=0 tx=0).stop 2023-10-30T12:35:16.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.906+0000 7fda3d817640 1 -- 172.21.15.12:0/1662049133 shutdown_connections 2023-10-30T12:35:16.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.906+0000 7fda3d817640 1 --2- 172.21.15.12:0/1662049133 >> v2:172.21.15.154:3300/0 conn(0x7fda380cefa0 0x7fda380cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.906+0000 7fda3d817640 1 --2- 172.21.15.12:0/1662049133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda381538f0 0x7fda38159fb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.906+0000 7fda3d817640 1 -- 172.21.15.12:0/1662049133 >> 172.21.15.12:0/1662049133 conn(0x7fda3806a640 msgr2=0x7fda38065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:16.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.906+0000 7fda3d817640 1 -- 172.21.15.12:0/1662049133 shutdown_connections 2023-10-30T12:35:16.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.906+0000 7fda3d817640 1 -- 172.21.15.12:0/1662049133 wait complete. 2023-10-30T12:35:16.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3d817640 1 Processor -- start 2023-10-30T12:35:16.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3d817640 1 -- start start 2023-10-30T12:35:16.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3d817640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda380cefa0 0x7fda3815bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3d817640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda381538f0 0x7fda381f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3d817640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda3815cd50 0x7fda381f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3d817640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda381597c0 con 0x7fda380cefa0 2023-10-30T12:35:16.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3d817640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda380b3df0 con 0x7fda381538f0 2023-10-30T12:35:16.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3d817640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda381432b0 con 0x7fda3815cd50 2023-10-30T12:35:16.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3c815640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda380cefa0 0x7fda3815bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3c815640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda380cefa0 0x7fda3815bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44712/0 (socket says 172.21.15.12:44712) 2023-10-30T12:35:16.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3c815640 1 -- 172.21.15.12:0/2862127742 learned_addr learned my addr 172.21.15.12:0/2862127742 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:16.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda37fff640 1 --2- 172.21.15.12:0/2862127742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda381538f0 0x7fda381f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3d016640 1 --2- 172.21.15.12:0/2862127742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda3815cd50 0x7fda381f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3c815640 1 -- 172.21.15.12:0/2862127742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda3815cd50 msgr2=0x7fda381f2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3c815640 1 --2- 172.21.15.12:0/2862127742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda3815cd50 0x7fda381f2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3c815640 1 -- 172.21.15.12:0/2862127742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda381538f0 msgr2=0x7fda381f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:16.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3c815640 1 --2- 172.21.15.12:0/2862127742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda381538f0 0x7fda381f2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:16.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3c815640 1 -- 172.21.15.12:0/2862127742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c009420 con 0x7fda380cefa0 2023-10-30T12:35:16.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3c815640 1 --2- 172.21.15.12:0/2862127742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda380cefa0 0x7fda3815bbd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0b3040 tx=0x7fda2c01b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda357fa640 1 -- 172.21.15.12:0/2862127742 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0b0dd0 con 0x7fda380cefa0 2023-10-30T12:35:16.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda357fa640 1 -- 172.21.15.12:0/2862127742 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c0b9430 con 0x7fda380cefa0 2023-10-30T12:35:16.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda357fa640 1 -- 172.21.15.12:0/2862127742 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0b97e0 con 0x7fda380cefa0 2023-10-30T12:35:16.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3d817640 1 -- 172.21.15.12:0/2862127742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda380b3830 con 0x7fda380cefa0 2023-10-30T12:35:16.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.910+0000 7fda3d817640 1 -- 172.21.15.12:0/2862127742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda38159460 con 0x7fda380cefa0 2023-10-30T12:35:16.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.914+0000 7fda3d817640 1 -- 172.21.15.12:0/2862127742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda380b3690 con 0x7fda380cefa0 2023-10-30T12:35:16.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.914+0000 7fda357fa640 1 -- 172.21.15.12:0/2862127742 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fda2c0b7040 con 0x7fda380cefa0 2023-10-30T12:35:16.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.914+0000 7fda357fa640 1 --2- 172.21.15.12:0/2862127742 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda1c0664a0 0x7fda1c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:16.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.914+0000 7fda357fa640 1 -- 172.21.15.12:0/2862127742 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda2c121a90 con 0x7fda380cefa0 2023-10-30T12:35:16.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.914+0000 7fda37fff640 1 --2- 172.21.15.12:0/2862127742 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda1c0664a0 0x7fda1c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:16.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.914+0000 7fda37fff640 1 --2- 172.21.15.12:0/2862127742 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda1c0664a0 0x7fda1c068960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fda3814d790 tx=0x7fda24007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:16.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:16.918+0000 7fda357fa640 1 -- 172.21.15.12:0/2862127742 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fda380b3690 con 0x7fda380cefa0 2023-10-30T12:35:17.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.074+0000 7fda3d817640 1 -- 172.21.15.12:0/2862127742 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fda3815d8c0 con 0x7fda1c0664a0 2023-10-30T12:35:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.094+0000 7fda357fa640 1 -- 172.21.15.12:0/2862127742 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264515 (secure 0 0 0) 0x7fda24007da0 con 0x7fda1c0664a0 2023-10-30T12:35:17.100 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:17.101 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-10-30T12:35:17.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.098+0000 7fda3d817640 1 -- 172.21.15.12:0/2862127742 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda1c0664a0 msgr2=0x7fda1c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.098+0000 7fda3d817640 1 --2- 172.21.15.12:0/2862127742 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda1c0664a0 0x7fda1c068960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fda3814d790 tx=0x7fda24007c10 comp rx=0 tx=0).stop 2023-10-30T12:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.098+0000 7fda3d817640 1 -- 172.21.15.12:0/2862127742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda380cefa0 msgr2=0x7fda3815bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.098+0000 7fda3d817640 1 --2- 172.21.15.12:0/2862127742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda380cefa0 0x7fda3815bbd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0b3040 tx=0x7fda2c01b9f0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.098+0000 7fda3d817640 1 -- 172.21.15.12:0/2862127742 shutdown_connections 2023-10-30T12:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.098+0000 7fda3d817640 1 --2- 172.21.15.12:0/2862127742 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda1c0664a0 0x7fda1c068960 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.098+0000 7fda3d817640 1 --2- 172.21.15.12:0/2862127742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda3815cd50 0x7fda381f2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.098+0000 7fda3d817640 1 --2- 172.21.15.12:0/2862127742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda381538f0 0x7fda381f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.098+0000 7fda3d817640 1 --2- 172.21.15.12:0/2862127742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda380cefa0 0x7fda3815bbd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.098+0000 7fda3d817640 1 -- 172.21.15.12:0/2862127742 >> 172.21.15.12:0/2862127742 conn(0x7fda3806a640 msgr2=0x7fda381492c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:17.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.098+0000 7fda3d817640 1 -- 172.21.15.12:0/2862127742 shutdown_connections 2023-10-30T12:35:17.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.098+0000 7fda3d817640 1 -- 172.21.15.12:0/2862127742 wait complete. 2023-10-30T12:35:17.120 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-30T12:35:16.938867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219112,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755608,"statfs":{"total":773094113280,"available":772869742592,"internally_reserved":0,"allocated":3055616,"data_stored":1816468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12729,"internal_metadata":221236807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003921"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250527+0000","last_change":"2023-10-30T12:35:06.223711+0000","last_active":"2023-10-30T12:35:07.250527+0000","last_peered":"2023-10-30T12:35:07.250527+0000","last_clean":"2023-10-30T12:35:07.250527+0000","last_became_active":"2023-10-30T12:35:06.222724+0000","last_became_peered":"2023-10-30T12:35:06.222724+0000","last_unstale":"2023-10-30T12:35:07.250527+0000","last_undegraded":"2023-10-30T12:35:07.250527+0000","last_fullsized":"2023-10-30T12:35:07.250527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:34:35.217274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159376+0000","last_change":"2023-10-30T12:35:06.182748+0000","last_active":"2023-10-30T12:35:07.159376+0000","last_peered":"2023-10-30T12:35:07.159376+0000","last_clean":"2023-10-30T12:35:07.159376+0000","last_became_active":"2023-10-30T12:35:06.182441+0000","last_became_peered":"2023-10-30T12:35:06.182441+0000","last_unstale":"2023-10-30T12:35:07.159376+0000","last_undegraded":"2023-10-30T12:35:07.159376+0000","last_fullsized":"2023-10-30T12:35:07.159376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:26:28.943768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160588+0000","last_change":"2023-10-30T12:35:06.180426+0000","last_active":"2023-10-30T12:35:07.160588+0000","last_peered":"2023-10-30T12:35:07.160588+0000","last_clean":"2023-10-30T12:35:07.160588+0000","last_became_active":"2023-10-30T12:35:06.180132+0000","last_became_peered":"2023-10-30T12:35:06.180132+0000","last_unstale":"2023-10-30T12:35:07.160588+0000","last_undegraded":"2023-10-30T12:35:07.160588+0000","last_fullsized":"2023-10-30T12:35:07.160588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:33:22.216739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250158+0000","last_change":"2023-10-30T12:35:06.217986+0000","last_active":"2023-10-30T12:35:07.250158+0000","last_peered":"2023-10-30T12:35:07.250158+0000","last_clean":"2023-10-30T12:35:07.250158+0000","last_became_active":"2023-10-30T12:35:06.213987+0000","last_became_peered":"2023-10-30T12:35:06.213987+0000","last_unstale":"2023-10-30T12:35:07.250158+0000","last_undegraded":"2023-10-30T12:35:07.250158+0000","last_fullsized":"2023-10-30T12:35:07.250158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:51:20.515999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249406+0000","last_change":"2023-10-30T12:35:06.216098+0000","last_active":"2023-10-30T12:35:07.249406+0000","last_peered":"2023-10-30T12:35:07.249406+0000","last_clean":"2023-10-30T12:35:07.249406+0000","last_became_active":"2023-10-30T12:35:06.214964+0000","last_became_peered":"2023-10-30T12:35:06.214964+0000","last_unstale":"2023-10-30T12:35:07.249406+0000","last_undegraded":"2023-10-30T12:35:07.249406+0000","last_fullsized":"2023-10-30T12:35:07.249406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:37:43.156964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249861+0000","last_change":"2023-10-30T12:35:06.217608+0000","last_active":"2023-10-30T12:35:07.249861+0000","last_peered":"2023-10-30T12:35:07.249861+0000","last_clean":"2023-10-30T12:35:07.249861+0000","last_became_active":"2023-10-30T12:35:06.214219+0000","last_became_peered":"2023-10-30T12:35:06.214219+0000","last_unstale":"2023-10-30T12:35:07.249861+0000","last_undegraded":"2023-10-30T12:35:07.249861+0000","last_fullsized":"2023-10-30T12:35:07.249861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:35:27.431813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249000+0000","last_change":"2023-10-30T12:35:06.217120+0000","last_active":"2023-10-30T12:35:07.249000+0000","last_peered":"2023-10-30T12:35:07.249000+0000","last_clean":"2023-10-30T12:35:07.249000+0000","last_became_active":"2023-10-30T12:35:06.216468+0000","last_became_peered":"2023-10-30T12:35:06.216468+0000","last_unstale":"2023-10-30T12:35:07.249000+0000","last_undegraded":"2023-10-30T12:35:07.249000+0000","last_fullsized":"2023-10-30T12:35:07.249000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:26:07.313967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246707+0000","last_change":"2023-10-30T12:35:06.214767+0000","last_active":"2023-10-30T12:35:07.246707+0000","last_peered":"2023-10-30T12:35:07.246707+0000","last_clean":"2023-10-30T12:35:07.246707+0000","last_became_active":"2023-10-30T12:35:06.214384+0000","last_became_peered":"2023-10-30T12:35:06.214384+0000","last_unstale":"2023-10-30T12:35:07.246707+0000","last_undegraded":"2023-10-30T12:35:07.246707+0000","last_fullsized":"2023-10-30T12:35:07.246707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:29.866538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250843+0000","last_change":"2023-10-30T12:35:06.223503+0000","last_active":"2023-10-30T12:35:07.250843+0000","last_peered":"2023-10-30T12:35:07.250843+0000","last_clean":"2023-10-30T12:35:07.250843+0000","last_became_active":"2023-10-30T12:35:06.222692+0000","last_became_peered":"2023-10-30T12:35:06.222692+0000","last_unstale":"2023-10-30T12:35:07.250843+0000","last_undegraded":"2023-10-30T12:35:07.250843+0000","last_fullsized":"2023-10-30T12:35:07.250843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:31:36.926720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163304+0000","last_change":"2023-10-30T12:35:06.215938+0000","last_active":"2023-10-30T12:35:07.163304+0000","last_peered":"2023-10-30T12:35:07.163304+0000","last_clean":"2023-10-30T12:35:07.163304+0000","last_became_active":"2023-10-30T12:35:06.215384+0000","last_became_peered":"2023-10-30T12:35:06.215384+0000","last_unstale":"2023-10-30T12:35:07.163304+0000","last_undegraded":"2023-10-30T12:35:07.163304+0000","last_fullsized":"2023-10-30T12:35:07.163304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:46:21.253555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162425+0000","last_change":"2023-10-30T12:35:06.215965+0000","last_active":"2023-10-30T12:35:07.162425+0000","last_peered":"2023-10-30T12:35:07.162425+0000","last_clean":"2023-10-30T12:35:07.162425+0000","last_became_active":"2023-10-30T12:35:06.215355+0000","last_became_peered":"2023-10-30T12:35:06.215355+0000","last_unstale":"2023-10-30T12:35:07.162425+0000","last_undegraded":"2023-10-30T12:35:07.162425+0000","last_fullsized":"2023-10-30T12:35:07.162425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:07.929526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164185+0000","last_change":"2023-10-30T12:35:06.197278+0000","last_active":"2023-10-30T12:35:07.164185+0000","last_peered":"2023-10-30T12:35:07.164185+0000","last_clean":"2023-10-30T12:35:07.164185+0000","last_became_active":"2023-10-30T12:35:06.196503+0000","last_became_peered":"2023-10-30T12:35:06.196503+0000","last_unstale":"2023-10-30T12:35:07.164185+0000","last_undegraded":"2023-10-30T12:35:07.164185+0000","last_fullsized":"2023-10-30T12:35:07.164185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:06.928264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247199+0000","last_change":"2023-10-30T12:35:06.195721+0000","last_active":"2023-10-30T12:35:07.247199+0000","last_peered":"2023-10-30T12:35:07.247199+0000","last_clean":"2023-10-30T12:35:07.247199+0000","last_became_active":"2023-10-30T12:35:06.195408+0000","last_became_peered":"2023-10-30T12:35:06.195408+0000","last_unstale":"2023-10-30T12:35:07.247199+0000","last_undegraded":"2023-10-30T12:35:07.247199+0000","last_fullsized":"2023-10-30T12:35:07.247199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:59:02.872646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160258+0000","last_change":"2023-10-30T12:35:06.182939+0000","last_active":"2023-10-30T12:35:07.160258+0000","last_peered":"2023-10-30T12:35:07.160258+0000","last_clean":"2023-10-30T12:35:07.160258+0000","last_became_active":"2023-10-30T12:35:06.182573+0000","last_became_peered":"2023-10-30T12:35:06.182573+0000","last_unstale":"2023-10-30T12:35:07.160258+0000","last_undegraded":"2023-10-30T12:35:07.160258+0000","last_fullsized":"2023-10-30T12:35:07.160258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:50.345142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.158834+0000","last_change":"2023-10-30T12:35:06.185636+0000","last_active":"2023-10-30T12:35:07.158834+0000","last_peered":"2023-10-30T12:35:07.158834+0000","last_clean":"2023-10-30T12:35:07.158834+0000","last_became_active":"2023-10-30T12:35:06.185290+0000","last_became_peered":"2023-10-30T12:35:06.185290+0000","last_unstale":"2023-10-30T12:35:07.158834+0000","last_undegraded":"2023-10-30T12:35:07.158834+0000","last_fullsized":"2023-10-30T12:35:07.158834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:26:44.783714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.248972+0000","last_change":"2023-10-30T12:35:06.214488+0000","last_active":"2023-10-30T12:35:07.248972+0000","last_peered":"2023-10-30T12:35:07.248972+0000","last_clean":"2023-10-30T12:35:07.248972+0000","last_became_active":"2023-10-30T12:35:06.213627+0000","last_became_peered":"2023-10-30T12:35:06.213627+0000","last_unstale":"2023-10-30T12:35:07.248972+0000","last_undegraded":"2023-10-30T12:35:07.248972+0000","last_fullsized":"2023-10-30T12:35:07.248972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:22:47.148407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251739+0000","last_change":"2023-10-30T12:35:06.224903+0000","last_active":"2023-10-30T12:35:07.251739+0000","last_peered":"2023-10-30T12:35:07.251739+0000","last_clean":"2023-10-30T12:35:07.251739+0000","last_became_active":"2023-10-30T12:35:06.221696+0000","last_became_peered":"2023-10-30T12:35:06.221696+0000","last_unstale":"2023-10-30T12:35:07.251739+0000","last_undegraded":"2023-10-30T12:35:07.251739+0000","last_fullsized":"2023-10-30T12:35:07.251739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:31:10.439162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246945+0000","last_change":"2023-10-30T12:35:06.198303+0000","last_active":"2023-10-30T12:35:07.246945+0000","last_peered":"2023-10-30T12:35:07.246945+0000","last_clean":"2023-10-30T12:35:07.246945+0000","last_became_active":"2023-10-30T12:35:06.198074+0000","last_became_peered":"2023-10-30T12:35:06.198074+0000","last_unstale":"2023-10-30T12:35:07.246945+0000","last_undegraded":"2023-10-30T12:35:07.246945+0000","last_fullsized":"2023-10-30T12:35:07.246945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:21:36.079586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250503+0000","last_change":"2023-10-30T12:35:06.218556+0000","last_active":"2023-10-30T12:35:07.250503+0000","last_peered":"2023-10-30T12:35:07.250503+0000","last_clean":"2023-10-30T12:35:07.250503+0000","last_became_active":"2023-10-30T12:35:06.215935+0000","last_became_peered":"2023-10-30T12:35:06.215935+0000","last_unstale":"2023-10-30T12:35:07.250503+0000","last_undegraded":"2023-10-30T12:35:07.250503+0000","last_fullsized":"2023-10-30T12:35:07.250503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:36:31.084542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162438+0000","last_change":"2023-10-30T12:35:06.201302+0000","last_active":"2023-10-30T12:35:07.162438+0000","last_peered":"2023-10-30T12:35:07.162438+0000","last_clean":"2023-10-30T12:35:07.162438+0000","last_became_active":"2023-10-30T12:35:06.199812+0000","last_became_peered":"2023-10-30T12:35:06.199812+0000","last_unstale":"2023-10-30T12:35:07.162438+0000","last_undegraded":"2023-10-30T12:35:07.162438+0000","last_fullsized":"2023-10-30T12:35:07.162438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:48.803738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160558+0000","last_change":"2023-10-30T12:35:06.217139+0000","last_active":"2023-10-30T12:35:07.160558+0000","last_peered":"2023-10-30T12:35:07.160558+0000","last_clean":"2023-10-30T12:35:07.160558+0000","last_became_active":"2023-10-30T12:35:06.216861+0000","last_became_peered":"2023-10-30T12:35:06.216861+0000","last_unstale":"2023-10-30T12:35:07.160558+0000","last_undegraded":"2023-10-30T12:35:07.160558+0000","last_fullsized":"2023-10-30T12:35:07.160558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:05:22.193180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249973+0000","last_change":"2023-10-30T12:35:06.225140+0000","last_active":"2023-10-30T12:35:07.249973+0000","last_peered":"2023-10-30T12:35:07.249973+0000","last_clean":"2023-10-30T12:35:07.249973+0000","last_became_active":"2023-10-30T12:35:06.221841+0000","last_became_peered":"2023-10-30T12:35:06.221841+0000","last_unstale":"2023-10-30T12:35:07.249973+0000","last_undegraded":"2023-10-30T12:35:07.249973+0000","last_fullsized":"2023-10-30T12:35:07.249973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:51:57.131025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247032+0000","last_change":"2023-10-30T12:35:06.182662+0000","last_active":"2023-10-30T12:35:07.247032+0000","last_peered":"2023-10-30T12:35:07.247032+0000","last_clean":"2023-10-30T12:35:07.247032+0000","last_became_active":"2023-10-30T12:35:06.182314+0000","last_became_peered":"2023-10-30T12:35:06.182314+0000","last_unstale":"2023-10-30T12:35:07.247032+0000","last_undegraded":"2023-10-30T12:35:07.247032+0000","last_fullsized":"2023-10-30T12:35:07.247032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:41:49.587856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249533+0000","last_change":"2023-10-30T12:35:06.214113+0000","last_active":"2023-10-30T12:35:07.249533+0000","last_peered":"2023-10-30T12:35:07.249533+0000","last_clean":"2023-10-30T12:35:07.249533+0000","last_became_active":"2023-10-30T12:35:06.213402+0000","last_became_peered":"2023-10-30T12:35:06.213402+0000","last_unstale":"2023-10-30T12:35:07.249533+0000","last_undegraded":"2023-10-30T12:35:07.249533+0000","last_fullsized":"2023-10-30T12:35:07.249533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:14:58.409296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247781+0000","last_change":"2023-10-30T12:35:06.188570+0000","last_active":"2023-10-30T12:35:07.247781+0000","last_peered":"2023-10-30T12:35:07.247781+0000","last_clean":"2023-10-30T12:35:07.247781+0000","last_became_active":"2023-10-30T12:35:06.188298+0000","last_became_peered":"2023-10-30T12:35:06.188298+0000","last_unstale":"2023-10-30T12:35:07.247781+0000","last_undegraded":"2023-10-30T12:35:07.247781+0000","last_fullsized":"2023-10-30T12:35:07.247781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:28.281254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163623+0000","last_change":"2023-10-30T12:35:06.208821+0000","last_active":"2023-10-30T12:35:07.163623+0000","last_peered":"2023-10-30T12:35:07.163623+0000","last_clean":"2023-10-30T12:35:07.163623+0000","last_became_active":"2023-10-30T12:35:06.202507+0000","last_became_peered":"2023-10-30T12:35:06.202507+0000","last_unstale":"2023-10-30T12:35:07.163623+0000","last_undegraded":"2023-10-30T12:35:07.163623+0000","last_fullsized":"2023-10-30T12:35:07.163623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:40:13.124316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159171+0000","last_change":"2023-10-30T12:35:06.170313+0000","last_active":"2023-10-30T12:35:07.159171+0000","last_peered":"2023-10-30T12:35:07.159171+0000","last_clean":"2023-10-30T12:35:07.159171+0000","last_became_active":"2023-10-30T12:35:06.170030+0000","last_became_peered":"2023-10-30T12:35:06.170030+0000","last_unstale":"2023-10-30T12:35:07.159171+0000","last_undegraded":"2023-10-30T12:35:07.159171+0000","last_fullsized":"2023-10-30T12:35:07.159171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:49:59.706208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247487+0000","last_change":"2023-10-30T12:35:06.189448+0000","last_active":"2023-10-30T12:35:07.247487+0000","last_peered":"2023-10-30T12:35:07.247487+0000","last_clean":"2023-10-30T12:35:07.247487+0000","last_became_active":"2023-10-30T12:35:06.189171+0000","last_became_peered":"2023-10-30T12:35:06.189171+0000","last_unstale":"2023-10-30T12:35:07.247487+0000","last_undegraded":"2023-10-30T12:35:07.247487+0000","last_fullsized":"2023-10-30T12:35:07.247487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:18:51.463273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160658+0000","last_change":"2023-10-30T12:35:06.183680+0000","last_active":"2023-10-30T12:35:07.160658+0000","last_peered":"2023-10-30T12:35:07.160658+0000","last_clean":"2023-10-30T12:35:07.160658+0000","last_became_active":"2023-10-30T12:35:06.183348+0000","last_became_peered":"2023-10-30T12:35:06.183348+0000","last_unstale":"2023-10-30T12:35:07.160658+0000","last_undegraded":"2023-10-30T12:35:07.160658+0000","last_fullsized":"2023-10-30T12:35:07.160658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:54:25.429897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250238+0000","last_change":"2023-10-30T12:35:06.206675+0000","last_active":"2023-10-30T12:35:07.250238+0000","last_peered":"2023-10-30T12:35:07.250238+0000","last_clean":"2023-10-30T12:35:07.250238+0000","last_became_active":"2023-10-30T12:35:06.196368+0000","last_became_peered":"2023-10-30T12:35:06.196368+0000","last_unstale":"2023-10-30T12:35:07.250238+0000","last_undegraded":"2023-10-30T12:35:07.250238+0000","last_fullsized":"2023-10-30T12:35:07.250238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:35.929311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246681+0000","last_change":"2023-10-30T12:35:06.214408+0000","last_active":"2023-10-30T12:35:07.246681+0000","last_peered":"2023-10-30T12:35:07.246681+0000","last_clean":"2023-10-30T12:35:07.246681+0000","last_became_active":"2023-10-30T12:35:06.213994+0000","last_became_peered":"2023-10-30T12:35:06.213994+0000","last_unstale":"2023-10-30T12:35:07.246681+0000","last_undegraded":"2023-10-30T12:35:07.246681+0000","last_fullsized":"2023-10-30T12:35:07.246681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:05:58.830698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162027+0000","last_change":"2023-10-30T12:35:06.198444+0000","last_active":"2023-10-30T12:35:07.162027+0000","last_peered":"2023-10-30T12:35:07.162027+0000","last_clean":"2023-10-30T12:35:07.162027+0000","last_became_active":"2023-10-30T12:35:06.197647+0000","last_became_peered":"2023-10-30T12:35:06.197647+0000","last_unstale":"2023-10-30T12:35:07.162027+0000","last_undegraded":"2023-10-30T12:35:07.162027+0000","last_fullsized":"2023-10-30T12:35:07.162027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:28:56.075420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163077+0000","last_change":"2023-10-30T12:35:06.168377+0000","last_active":"2023-10-30T12:35:07.163077+0000","last_peered":"2023-10-30T12:35:07.163077+0000","last_clean":"2023-10-30T12:35:07.163077+0000","last_became_active":"2023-10-30T12:35:06.167558+0000","last_became_peered":"2023-10-30T12:35:06.167558+0000","last_unstale":"2023-10-30T12:35:07.163077+0000","last_undegraded":"2023-10-30T12:35:07.163077+0000","last_fullsized":"2023-10-30T12:35:07.163077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:57:28.009164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.444498+0000","last_change":"2023-10-30T12:35:05.200844+0000","last_active":"2023-10-30T12:35:07.444498+0000","last_peered":"2023-10-30T12:35:07.444498+0000","last_clean":"2023-10-30T12:35:07.444498+0000","last_became_active":"2023-10-30T12:35:05.200169+0000","last_became_peered":"2023-10-30T12:35:05.200169+0000","last_unstale":"2023-10-30T12:35:07.444498+0000","last_undegraded":"2023-10-30T12:35:07.444498+0000","last_fullsized":"2023-10-30T12:35:07.444498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:07:19.571357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160718+0000","last_change":"2023-10-30T12:35:06.194964+0000","last_active":"2023-10-30T12:35:07.160718+0000","last_peered":"2023-10-30T12:35:07.160718+0000","last_clean":"2023-10-30T12:35:07.160718+0000","last_became_active":"2023-10-30T12:35:06.194689+0000","last_became_peered":"2023-10-30T12:35:06.194689+0000","last_unstale":"2023-10-30T12:35:07.160718+0000","last_undegraded":"2023-10-30T12:35:07.160718+0000","last_fullsized":"2023-10-30T12:35:07.160718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:30:53.410341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.461020+0000","last_change":"2023-10-30T12:35:05.183569+0000","last_active":"2023-10-30T12:35:07.461020+0000","last_peered":"2023-10-30T12:35:07.461020+0000","last_clean":"2023-10-30T12:35:07.461020+0000","last_became_active":"2023-10-30T12:35:05.182761+0000","last_became_peered":"2023-10-30T12:35:05.182761+0000","last_unstale":"2023-10-30T12:35:07.461020+0000","last_undegraded":"2023-10-30T12:35:07.461020+0000","last_fullsized":"2023-10-30T12:35:07.461020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:49:44.311814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247108+0000","last_change":"2023-10-30T12:35:05.181278+0000","last_active":"2023-10-30T12:35:07.247108+0000","last_peered":"2023-10-30T12:35:07.247108+0000","last_clean":"2023-10-30T12:35:07.247108+0000","last_became_active":"2023-10-30T12:35:05.180730+0000","last_became_peered":"2023-10-30T12:35:05.180730+0000","last_unstale":"2023-10-30T12:35:07.247108+0000","last_undegraded":"2023-10-30T12:35:07.247108+0000","last_fullsized":"2023-10-30T12:35:07.247108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:07:10.828707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246964+0000","last_change":"2023-10-30T12:35:06.180049+0000","last_active":"2023-10-30T12:35:07.246964+0000","last_peered":"2023-10-30T12:35:07.246964+0000","last_clean":"2023-10-30T12:35:07.246964+0000","last_became_active":"2023-10-30T12:35:06.179764+0000","last_became_peered":"2023-10-30T12:35:06.179764+0000","last_unstale":"2023-10-30T12:35:07.246964+0000","last_undegraded":"2023-10-30T12:35:07.246964+0000","last_fullsized":"2023-10-30T12:35:07.246964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:33.136727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.469327+0000","last_change":"2023-10-30T12:35:05.202539+0000","last_active":"2023-10-30T12:35:06.469327+0000","last_peered":"2023-10-30T12:35:06.469327+0000","last_clean":"2023-10-30T12:35:06.469327+0000","last_became_active":"2023-10-30T12:35:05.201853+0000","last_became_peered":"2023-10-30T12:35:05.201853+0000","last_unstale":"2023-10-30T12:35:06.469327+0000","last_undegraded":"2023-10-30T12:35:06.469327+0000","last_fullsized":"2023-10-30T12:35:06.469327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:33:33.151433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.158869+0000","last_change":"2023-10-30T12:35:06.173567+0000","last_active":"2023-10-30T12:35:07.158869+0000","last_peered":"2023-10-30T12:35:07.158869+0000","last_clean":"2023-10-30T12:35:07.158869+0000","last_became_active":"2023-10-30T12:35:06.170179+0000","last_became_peered":"2023-10-30T12:35:06.170179+0000","last_unstale":"2023-10-30T12:35:07.158869+0000","last_undegraded":"2023-10-30T12:35:07.158869+0000","last_fullsized":"2023-10-30T12:35:07.158869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:39:07.880510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163553+0000","last_change":"2023-10-30T12:35:05.198546+0000","last_active":"2023-10-30T12:35:07.163553+0000","last_peered":"2023-10-30T12:35:07.163553+0000","last_clean":"2023-10-30T12:35:07.163553+0000","last_became_active":"2023-10-30T12:35:05.197606+0000","last_became_peered":"2023-10-30T12:35:05.197606+0000","last_unstale":"2023-10-30T12:35:07.163553+0000","last_undegraded":"2023-10-30T12:35:07.163553+0000","last_fullsized":"2023-10-30T12:35:07.163553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:26:57.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251322+0000","last_change":"2023-10-30T12:35:06.226044+0000","last_active":"2023-10-30T12:35:07.251322+0000","last_peered":"2023-10-30T12:35:07.251322+0000","last_clean":"2023-10-30T12:35:07.251322+0000","last_became_active":"2023-10-30T12:35:06.225392+0000","last_became_peered":"2023-10-30T12:35:06.225392+0000","last_unstale":"2023-10-30T12:35:07.251322+0000","last_undegraded":"2023-10-30T12:35:07.251322+0000","last_fullsized":"2023-10-30T12:35:07.251322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:13:10.796549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.469631+0000","last_change":"2023-10-30T12:35:05.197614+0000","last_active":"2023-10-30T12:35:06.469631+0000","last_peered":"2023-10-30T12:35:06.469631+0000","last_clean":"2023-10-30T12:35:06.469631+0000","last_became_active":"2023-10-30T12:35:05.196899+0000","last_became_peered":"2023-10-30T12:35:05.196899+0000","last_unstale":"2023-10-30T12:35:06.469631+0000","last_undegraded":"2023-10-30T12:35:06.469631+0000","last_fullsized":"2023-10-30T12:35:06.469631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:21:00.152426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160715+0000","last_change":"2023-10-30T12:35:06.222344+0000","last_active":"2023-10-30T12:35:07.160715+0000","last_peered":"2023-10-30T12:35:07.160715+0000","last_clean":"2023-10-30T12:35:07.160715+0000","last_became_active":"2023-10-30T12:35:06.222057+0000","last_became_peered":"2023-10-30T12:35:06.222057+0000","last_unstale":"2023-10-30T12:35:07.160715+0000","last_undegraded":"2023-10-30T12:35:07.160715+0000","last_fullsized":"2023-10-30T12:35:07.160715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:50:46.691223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.198046+0000","last_change":"2023-10-30T12:35:05.197501+0000","last_active":"2023-10-30T12:35:11.198046+0000","last_peered":"2023-10-30T12:35:11.198046+0000","last_clean":"2023-10-30T12:35:11.198046+0000","last_became_active":"2023-10-30T12:35:05.197190+0000","last_became_peered":"2023-10-30T12:35:05.197190+0000","last_unstale":"2023-10-30T12:35:11.198046+0000","last_undegraded":"2023-10-30T12:35:11.198046+0000","last_fullsized":"2023-10-30T12:35:11.198046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:53:31.121721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164505+0000","last_change":"2023-10-30T12:35:06.204885+0000","last_active":"2023-10-30T12:35:07.164505+0000","last_peered":"2023-10-30T12:35:07.164505+0000","last_clean":"2023-10-30T12:35:07.164505+0000","last_became_active":"2023-10-30T12:35:06.204122+0000","last_became_peered":"2023-10-30T12:35:06.204122+0000","last_unstale":"2023-10-30T12:35:07.164505+0000","last_undegraded":"2023-10-30T12:35:07.164505+0000","last_fullsized":"2023-10-30T12:35:07.164505+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:27:18.433575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.201072+0000","last_change":"2023-10-30T12:35:05.180005+0000","last_active":"2023-10-30T12:35:11.201072+0000","last_peered":"2023-10-30T12:35:11.201072+0000","last_clean":"2023-10-30T12:35:11.201072+0000","last_became_active":"2023-10-30T12:35:05.179547+0000","last_became_peered":"2023-10-30T12:35:05.179547+0000","last_unstale":"2023-10-30T12:35:11.201072+0000","last_undegraded":"2023-10-30T12:35:11.201072+0000","last_fullsized":"2023-10-30T12:35:11.201072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:13:03.137433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249565+0000","last_change":"2023-10-30T12:35:06.216619+0000","last_active":"2023-10-30T12:35:07.249565+0000","last_peered":"2023-10-30T12:35:07.249565+0000","last_clean":"2023-10-30T12:35:07.249565+0000","last_became_active":"2023-10-30T12:35:06.215128+0000","last_became_peered":"2023-10-30T12:35:06.215128+0000","last_unstale":"2023-10-30T12:35:07.249565+0000","last_undegraded":"2023-10-30T12:35:07.249565+0000","last_fullsized":"2023-10-30T12:35:07.249565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:48:45.673159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.196935+0000","last_change":"2023-10-30T12:35:05.190041+0000","last_active":"2023-10-30T12:35:11.196935+0000","last_peered":"2023-10-30T12:35:11.196935+0000","last_clean":"2023-10-30T12:35:11.196935+0000","last_became_active":"2023-10-30T12:35:05.189573+0000","last_became_peered":"2023-10-30T12:35:05.189573+0000","last_unstale":"2023-10-30T12:35:11.196935+0000","last_undegraded":"2023-10-30T12:35:11.196935+0000","last_fullsized":"2023-10-30T12:35:11.196935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:02:57.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160412+0000","last_change":"2023-10-30T12:35:06.217334+0000","last_active":"2023-10-30T12:35:07.160412+0000","last_peered":"2023-10-30T12:35:07.160412+0000","last_clean":"2023-10-30T12:35:07.160412+0000","last_became_active":"2023-10-30T12:35:06.216885+0000","last_became_peered":"2023-10-30T12:35:06.216885+0000","last_unstale":"2023-10-30T12:35:07.160412+0000","last_undegraded":"2023-10-30T12:35:07.160412+0000","last_fullsized":"2023-10-30T12:35:07.160412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:38:26.745898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.469429+0000","last_change":"2023-10-30T12:35:05.203198+0000","last_active":"2023-10-30T12:35:06.469429+0000","last_peered":"2023-10-30T12:35:06.469429+0000","last_clean":"2023-10-30T12:35:06.469429+0000","last_became_active":"2023-10-30T12:35:05.202612+0000","last_became_peered":"2023-10-30T12:35:05.202612+0000","last_unstale":"2023-10-30T12:35:06.469429+0000","last_undegraded":"2023-10-30T12:35:06.469429+0000","last_fullsized":"2023-10-30T12:35:06.469429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:55.986236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249574+0000","last_change":"2023-10-30T12:35:06.216307+0000","last_active":"2023-10-30T12:35:07.249574+0000","last_peered":"2023-10-30T12:35:07.249574+0000","last_clean":"2023-10-30T12:35:07.249574+0000","last_became_active":"2023-10-30T12:35:06.215553+0000","last_became_peered":"2023-10-30T12:35:06.215553+0000","last_unstale":"2023-10-30T12:35:07.249574+0000","last_undegraded":"2023-10-30T12:35:07.249574+0000","last_fullsized":"2023-10-30T12:35:07.249574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:46:25.396759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.161910+0000","last_change":"2023-10-30T12:35:05.181882+0000","last_active":"2023-10-30T12:35:07.161910+0000","last_peered":"2023-10-30T12:35:07.161910+0000","last_clean":"2023-10-30T12:35:07.161910+0000","last_became_active":"2023-10-30T12:35:05.181115+0000","last_became_peered":"2023-10-30T12:35:05.181115+0000","last_unstale":"2023-10-30T12:35:07.161910+0000","last_undegraded":"2023-10-30T12:35:07.161910+0000","last_fullsized":"2023-10-30T12:35:07.161910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:18:55.496940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162013+0000","last_change":"2023-10-30T12:35:06.198990+0000","last_active":"2023-10-30T12:35:07.162013+0000","last_peered":"2023-10-30T12:35:07.162013+0000","last_clean":"2023-10-30T12:35:07.162013+0000","last_became_active":"2023-10-30T12:35:06.197722+0000","last_became_peered":"2023-10-30T12:35:06.197722+0000","last_unstale":"2023-10-30T12:35:07.162013+0000","last_undegraded":"2023-10-30T12:35:07.162013+0000","last_fullsized":"2023-10-30T12:35:07.162013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:56:51.757976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.197517+0000","last_change":"2023-10-30T12:35:05.169455+0000","last_active":"2023-10-30T12:35:11.197517+0000","last_peered":"2023-10-30T12:35:11.197517+0000","last_clean":"2023-10-30T12:35:11.197517+0000","last_became_active":"2023-10-30T12:35:05.169130+0000","last_became_peered":"2023-10-30T12:35:05.169130+0000","last_unstale":"2023-10-30T12:35:11.197517+0000","last_undegraded":"2023-10-30T12:35:11.197517+0000","last_fullsized":"2023-10-30T12:35:11.197517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:59:14.096315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249183+0000","last_change":"2023-10-30T12:35:06.215545+0000","last_active":"2023-10-30T12:35:07.249183+0000","last_peered":"2023-10-30T12:35:07.249183+0000","last_clean":"2023-10-30T12:35:07.249183+0000","last_became_active":"2023-10-30T12:35:06.214839+0000","last_became_peered":"2023-10-30T12:35:06.214839+0000","last_unstale":"2023-10-30T12:35:07.249183+0000","last_undegraded":"2023-10-30T12:35:07.249183+0000","last_fullsized":"2023-10-30T12:35:07.249183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:54:34.206981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.452194+0000","last_change":"2023-10-30T12:35:05.189361+0000","last_active":"2023-10-30T12:35:07.452194+0000","last_peered":"2023-10-30T12:35:07.452194+0000","last_clean":"2023-10-30T12:35:07.452194+0000","last_became_active":"2023-10-30T12:35:05.188862+0000","last_became_peered":"2023-10-30T12:35:05.188862+0000","last_unstale":"2023-10-30T12:35:07.452194+0000","last_undegraded":"2023-10-30T12:35:07.452194+0000","last_fullsized":"2023-10-30T12:35:07.452194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:13.319732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250712+0000","last_change":"2023-10-30T12:35:06.223118+0000","last_active":"2023-10-30T12:35:07.250712+0000","last_peered":"2023-10-30T12:35:07.250712+0000","last_clean":"2023-10-30T12:35:07.250712+0000","last_became_active":"2023-10-30T12:35:06.216533+0000","last_became_peered":"2023-10-30T12:35:06.216533+0000","last_unstale":"2023-10-30T12:35:07.250712+0000","last_undegraded":"2023-10-30T12:35:07.250712+0000","last_fullsized":"2023-10-30T12:35:07.250712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:32:29.374526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.195596+0000","last_change":"2023-10-30T12:35:05.177609+0000","last_active":"2023-10-30T12:35:11.195596+0000","last_peered":"2023-10-30T12:35:11.195596+0000","last_clean":"2023-10-30T12:35:11.195596+0000","last_became_active":"2023-10-30T12:35:05.177316+0000","last_became_peered":"2023-10-30T12:35:05.177316+0000","last_unstale":"2023-10-30T12:35:11.195596+0000","last_undegraded":"2023-10-30T12:35:11.195596+0000","last_fullsized":"2023-10-30T12:35:11.195596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:46.820925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251056+0000","last_change":"2023-10-30T12:35:06.222283+0000","last_active":"2023-10-30T12:35:07.251056+0000","last_peered":"2023-10-30T12:35:07.251056+0000","last_clean":"2023-10-30T12:35:07.251056+0000","last_became_active":"2023-10-30T12:35:06.221701+0000","last_became_peered":"2023-10-30T12:35:06.221701+0000","last_unstale":"2023-10-30T12:35:07.251056+0000","last_undegraded":"2023-10-30T12:35:07.251056+0000","last_fullsized":"2023-10-30T12:35:07.251056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:37:32.956627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159413+0000","last_change":"2023-10-30T12:35:05.171787+0000","last_active":"2023-10-30T12:35:07.159413+0000","last_peered":"2023-10-30T12:35:07.159413+0000","last_clean":"2023-10-30T12:35:07.159413+0000","last_became_active":"2023-10-30T12:35:05.171491+0000","last_became_peered":"2023-10-30T12:35:05.171491+0000","last_unstale":"2023-10-30T12:35:07.159413+0000","last_undegraded":"2023-10-30T12:35:07.159413+0000","last_fullsized":"2023-10-30T12:35:07.159413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:43:15.331175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250517+0000","last_change":"2023-10-30T12:35:06.214464+0000","last_active":"2023-10-30T12:35:07.250517+0000","last_peered":"2023-10-30T12:35:07.250517+0000","last_clean":"2023-10-30T12:35:07.250517+0000","last_became_active":"2023-10-30T12:35:06.213493+0000","last_became_peered":"2023-10-30T12:35:06.213493+0000","last_unstale":"2023-10-30T12:35:07.250517+0000","last_undegraded":"2023-10-30T12:35:07.250517+0000","last_fullsized":"2023-10-30T12:35:07.250517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:00:18.624720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246924+0000","last_change":"2023-10-30T12:35:05.197811+0000","last_active":"2023-10-30T12:35:07.246924+0000","last_peered":"2023-10-30T12:35:07.246924+0000","last_clean":"2023-10-30T12:35:07.246924+0000","last_became_active":"2023-10-30T12:35:05.197453+0000","last_became_peered":"2023-10-30T12:35:05.197453+0000","last_unstale":"2023-10-30T12:35:07.246924+0000","last_undegraded":"2023-10-30T12:35:07.246924+0000","last_fullsized":"2023-10-30T12:35:07.246924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:16:14.550887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249901+0000","last_change":"2023-10-30T12:35:06.220236+0000","last_active":"2023-10-30T12:35:07.249901+0000","last_peered":"2023-10-30T12:35:07.249901+0000","last_clean":"2023-10-30T12:35:07.249901+0000","last_became_active":"2023-10-30T12:35:06.219526+0000","last_became_peered":"2023-10-30T12:35:06.219526+0000","last_unstale":"2023-10-30T12:35:07.249901+0000","last_undegraded":"2023-10-30T12:35:07.249901+0000","last_fullsized":"2023-10-30T12:35:07.249901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:00:25.297590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.186752+0000","last_change":"2023-10-30T12:34:55.206976+0000","last_active":"2023-10-30T12:35:06.186752+0000","last_peered":"2023-10-30T12:35:06.186752+0000","last_clean":"2023-10-30T12:35:06.186752+0000","last_became_active":"2023-10-30T12:34:55.205923+0000","last_became_peered":"2023-10-30T12:34:55.205923+0000","last_unstale":"2023-10-30T12:35:06.186752+0000","last_undegraded":"2023-10-30T12:35:06.186752+0000","last_fullsized":"2023-10-30T12:35:06.186752+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_clean_scrub_stamp":"2023-10-30T12:34:53.444020+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:25:28.782111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.469475+0000","last_change":"2023-10-30T12:35:05.197974+0000","last_active":"2023-10-30T12:35:06.469475+0000","last_peered":"2023-10-30T12:35:06.469475+0000","last_clean":"2023-10-30T12:35:06.469475+0000","last_became_active":"2023-10-30T12:35:05.197201+0000","last_became_peered":"2023-10-30T12:35:05.197201+0000","last_unstale":"2023-10-30T12:35:06.469475+0000","last_undegraded":"2023-10-30T12:35:06.469475+0000","last_fullsized":"2023-10-30T12:35:06.469475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:39:47.119091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247224+0000","last_change":"2023-10-30T12:35:06.217973+0000","last_active":"2023-10-30T12:35:07.247224+0000","last_peered":"2023-10-30T12:35:07.247224+0000","last_clean":"2023-10-30T12:35:07.247224+0000","last_became_active":"2023-10-30T12:35:06.217687+0000","last_became_peered":"2023-10-30T12:35:06.217687+0000","last_unstale":"2023-10-30T12:35:07.247224+0000","last_undegraded":"2023-10-30T12:35:07.247224+0000","last_fullsized":"2023-10-30T12:35:07.247224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:28.392113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.248732+0000","last_change":"2023-10-30T12:35:05.208339+0000","last_active":"2023-10-30T12:35:07.248732+0000","last_peered":"2023-10-30T12:35:07.248732+0000","last_clean":"2023-10-30T12:35:07.248732+0000","last_became_active":"2023-10-30T12:35:05.207828+0000","last_became_peered":"2023-10-30T12:35:05.207828+0000","last_unstale":"2023-10-30T12:35:07.248732+0000","last_undegraded":"2023-10-30T12:35:07.248732+0000","last_fullsized":"2023-10-30T12:35:07.248732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:00.649048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160255+0000","last_change":"2023-10-30T12:35:06.195094+0000","last_active":"2023-10-30T12:35:07.160255+0000","last_peered":"2023-10-30T12:35:07.160255+0000","last_clean":"2023-10-30T12:35:07.160255+0000","last_became_active":"2023-10-30T12:35:06.194791+0000","last_became_peered":"2023-10-30T12:35:06.194791+0000","last_unstale":"2023-10-30T12:35:07.160255+0000","last_undegraded":"2023-10-30T12:35:07.160255+0000","last_fullsized":"2023-10-30T12:35:07.160255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:41:24.310349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246778+0000","last_change":"2023-10-30T12:35:05.176296+0000","last_active":"2023-10-30T12:35:07.246778+0000","last_peered":"2023-10-30T12:35:07.246778+0000","last_clean":"2023-10-30T12:35:07.246778+0000","last_became_active":"2023-10-30T12:35:05.175959+0000","last_became_peered":"2023-10-30T12:35:05.175959+0000","last_unstale":"2023-10-30T12:35:07.246778+0000","last_undegraded":"2023-10-30T12:35:07.246778+0000","last_fullsized":"2023-10-30T12:35:07.246778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:53.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159052+0000","last_change":"2023-10-30T12:35:06.167250+0000","last_active":"2023-10-30T12:35:07.159052+0000","last_peered":"2023-10-30T12:35:07.159052+0000","last_clean":"2023-10-30T12:35:07.159052+0000","last_became_active":"2023-10-30T12:35:06.164711+0000","last_became_peered":"2023-10-30T12:35:06.164711+0000","last_unstale":"2023-10-30T12:35:07.159052+0000","last_undegraded":"2023-10-30T12:35:07.159052+0000","last_fullsized":"2023-10-30T12:35:07.159052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:48:18.710410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.469452+0000","last_change":"2023-10-30T12:35:05.198964+0000","last_active":"2023-10-30T12:35:06.469452+0000","last_peered":"2023-10-30T12:35:06.469452+0000","last_clean":"2023-10-30T12:35:06.469452+0000","last_became_active":"2023-10-30T12:35:05.196873+0000","last_became_peered":"2023-10-30T12:35:05.196873+0000","last_unstale":"2023-10-30T12:35:06.469452+0000","last_undegraded":"2023-10-30T12:35:06.469452+0000","last_fullsized":"2023-10-30T12:35:06.469452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:47.793585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163526+0000","last_change":"2023-10-30T12:35:06.184123+0000","last_active":"2023-10-30T12:35:07.163526+0000","last_peered":"2023-10-30T12:35:07.163526+0000","last_clean":"2023-10-30T12:35:07.163526+0000","last_became_active":"2023-10-30T12:35:06.183225+0000","last_became_peered":"2023-10-30T12:35:06.183225+0000","last_unstale":"2023-10-30T12:35:07.163526+0000","last_undegraded":"2023-10-30T12:35:07.163526+0000","last_fullsized":"2023-10-30T12:35:07.163526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:03:47.097151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.543947+0000","last_change":"2023-10-30T12:35:05.177571+0000","last_active":"2023-10-30T12:35:06.543947+0000","last_peered":"2023-10-30T12:35:06.543947+0000","last_clean":"2023-10-30T12:35:06.543947+0000","last_became_active":"2023-10-30T12:35:05.177266+0000","last_became_peered":"2023-10-30T12:35:05.177266+0000","last_unstale":"2023-10-30T12:35:06.543947+0000","last_undegraded":"2023-10-30T12:35:06.543947+0000","last_fullsized":"2023-10-30T12:35:06.543947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:14.601934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250949+0000","last_change":"2023-10-30T12:35:06.206033+0000","last_active":"2023-10-30T12:35:07.250949+0000","last_peered":"2023-10-30T12:35:07.250949+0000","last_clean":"2023-10-30T12:35:07.250949+0000","last_became_active":"2023-10-30T12:35:06.196307+0000","last_became_peered":"2023-10-30T12:35:06.196307+0000","last_unstale":"2023-10-30T12:35:07.250949+0000","last_undegraded":"2023-10-30T12:35:07.250949+0000","last_fullsized":"2023-10-30T12:35:07.250949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:18:28.848259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250720+0000","last_change":"2023-10-30T12:35:05.207594+0000","last_active":"2023-10-30T12:35:07.250720+0000","last_peered":"2023-10-30T12:35:07.250720+0000","last_clean":"2023-10-30T12:35:07.250720+0000","last_became_active":"2023-10-30T12:35:05.206930+0000","last_became_peered":"2023-10-30T12:35:05.206930+0000","last_unstale":"2023-10-30T12:35:07.250720+0000","last_undegraded":"2023-10-30T12:35:07.250720+0000","last_fullsized":"2023-10-30T12:35:07.250720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:23:46.519077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159110+0000","last_change":"2023-10-30T12:35:06.165126+0000","last_active":"2023-10-30T12:35:07.159110+0000","last_peered":"2023-10-30T12:35:07.159110+0000","last_clean":"2023-10-30T12:35:07.159110+0000","last_became_active":"2023-10-30T12:35:06.164733+0000","last_became_peered":"2023-10-30T12:35:06.164733+0000","last_unstale":"2023-10-30T12:35:07.159110+0000","last_undegraded":"2023-10-30T12:35:07.159110+0000","last_fullsized":"2023-10-30T12:35:07.159110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:29.336555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.544006+0000","last_change":"2023-10-30T12:35:05.185034+0000","last_active":"2023-10-30T12:35:06.544006+0000","last_peered":"2023-10-30T12:35:06.544006+0000","last_clean":"2023-10-30T12:35:06.544006+0000","last_became_active":"2023-10-30T12:35:05.184695+0000","last_became_peered":"2023-10-30T12:35:05.184695+0000","last_unstale":"2023-10-30T12:35:06.544006+0000","last_undegraded":"2023-10-30T12:35:06.544006+0000","last_fullsized":"2023-10-30T12:35:06.544006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:01:42.019128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247746+0000","last_change":"2023-10-30T12:35:06.204707+0000","last_active":"2023-10-30T12:35:07.247746+0000","last_peered":"2023-10-30T12:35:07.247746+0000","last_clean":"2023-10-30T12:35:07.247746+0000","last_became_active":"2023-10-30T12:35:06.204422+0000","last_became_peered":"2023-10-30T12:35:06.204422+0000","last_unstale":"2023-10-30T12:35:07.247746+0000","last_undegraded":"2023-10-30T12:35:07.247746+0000","last_fullsized":"2023-10-30T12:35:07.247746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:58:16.758575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.201581+0000","last_change":"2023-10-30T12:35:05.200871+0000","last_active":"2023-10-30T12:35:11.201581+0000","last_peered":"2023-10-30T12:35:11.201581+0000","last_clean":"2023-10-30T12:35:11.201581+0000","last_became_active":"2023-10-30T12:35:05.200449+0000","last_became_peered":"2023-10-30T12:35:05.200449+0000","last_unstale":"2023-10-30T12:35:11.201581+0000","last_undegraded":"2023-10-30T12:35:11.201581+0000","last_fullsized":"2023-10-30T12:35:11.201581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:39:20.867587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160562+0000","last_change":"2023-10-30T12:35:06.180850+0000","last_active":"2023-10-30T12:35:07.160562+0000","last_peered":"2023-10-30T12:35:07.160562+0000","last_clean":"2023-10-30T12:35:07.160562+0000","last_became_active":"2023-10-30T12:35:06.179412+0000","last_became_peered":"2023-10-30T12:35:06.179412+0000","last_unstale":"2023-10-30T12:35:07.160562+0000","last_undegraded":"2023-10-30T12:35:07.160562+0000","last_fullsized":"2023-10-30T12:35:07.160562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:47:44.055205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.248731+0000","last_change":"2023-10-30T12:35:05.207392+0000","last_active":"2023-10-30T12:35:07.248731+0000","last_peered":"2023-10-30T12:35:07.248731+0000","last_clean":"2023-10-30T12:35:07.248731+0000","last_became_active":"2023-10-30T12:35:05.206607+0000","last_became_peered":"2023-10-30T12:35:05.206607+0000","last_unstale":"2023-10-30T12:35:07.248731+0000","last_undegraded":"2023-10-30T12:35:07.248731+0000","last_fullsized":"2023-10-30T12:35:07.248731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:36.758740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163445+0000","last_change":"2023-10-30T12:35:06.182042+0000","last_active":"2023-10-30T12:35:07.163445+0000","last_peered":"2023-10-30T12:35:07.163445+0000","last_clean":"2023-10-30T12:35:07.163445+0000","last_became_active":"2023-10-30T12:35:06.181256+0000","last_became_peered":"2023-10-30T12:35:06.181256+0000","last_unstale":"2023-10-30T12:35:07.163445+0000","last_undegraded":"2023-10-30T12:35:07.163445+0000","last_fullsized":"2023-10-30T12:35:07.163445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:13:08.109392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.543919+0000","last_change":"2023-10-30T12:35:05.201024+0000","last_active":"2023-10-30T12:35:06.543919+0000","last_peered":"2023-10-30T12:35:06.543919+0000","last_clean":"2023-10-30T12:35:06.543919+0000","last_became_active":"2023-10-30T12:35:05.200724+0000","last_became_peered":"2023-10-30T12:35:05.200724+0000","last_unstale":"2023-10-30T12:35:06.543919+0000","last_undegraded":"2023-10-30T12:35:06.543919+0000","last_fullsized":"2023-10-30T12:35:06.543919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:24.746469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247249+0000","last_change":"2023-10-30T12:35:06.174636+0000","last_active":"2023-10-30T12:35:07.247249+0000","last_peered":"2023-10-30T12:35:07.247249+0000","last_clean":"2023-10-30T12:35:07.247249+0000","last_became_active":"2023-10-30T12:35:06.173996+0000","last_became_peered":"2023-10-30T12:35:06.173996+0000","last_unstale":"2023-10-30T12:35:07.247249+0000","last_undegraded":"2023-10-30T12:35:07.247249+0000","last_fullsized":"2023-10-30T12:35:07.247249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:01:38.503494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.543930+0000","last_change":"2023-10-30T12:35:05.164079+0000","last_active":"2023-10-30T12:35:06.543930+0000","last_peered":"2023-10-30T12:35:06.543930+0000","last_clean":"2023-10-30T12:35:06.543930+0000","last_became_active":"2023-10-30T12:35:05.163794+0000","last_became_peered":"2023-10-30T12:35:05.163794+0000","last_unstale":"2023-10-30T12:35:06.543930+0000","last_undegraded":"2023-10-30T12:35:06.543930+0000","last_fullsized":"2023-10-30T12:35:06.543930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:19.795656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247344+0000","last_change":"2023-10-30T12:35:06.183531+0000","last_active":"2023-10-30T12:35:07.247344+0000","last_peered":"2023-10-30T12:35:07.247344+0000","last_clean":"2023-10-30T12:35:07.247344+0000","last_became_active":"2023-10-30T12:35:06.183226+0000","last_became_peered":"2023-10-30T12:35:06.183226+0000","last_unstale":"2023-10-30T12:35:07.247344+0000","last_undegraded":"2023-10-30T12:35:07.247344+0000","last_fullsized":"2023-10-30T12:35:07.247344+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:46.768356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.200408+0000","last_change":"2023-10-30T12:35:05.206995+0000","last_active":"2023-10-30T12:35:11.200408+0000","last_peered":"2023-10-30T12:35:11.200408+0000","last_clean":"2023-10-30T12:35:11.200408+0000","last_became_active":"2023-10-30T12:35:05.206518+0000","last_became_peered":"2023-10-30T12:35:05.206518+0000","last_unstale":"2023-10-30T12:35:11.200408+0000","last_undegraded":"2023-10-30T12:35:11.200408+0000","last_fullsized":"2023-10-30T12:35:11.200408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:41:33.703261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162528+0000","last_change":"2023-10-30T12:35:06.199557+0000","last_active":"2023-10-30T12:35:07.162528+0000","last_peered":"2023-10-30T12:35:07.162528+0000","last_clean":"2023-10-30T12:35:07.162528+0000","last_became_active":"2023-10-30T12:35:06.197861+0000","last_became_peered":"2023-10-30T12:35:06.197861+0000","last_unstale":"2023-10-30T12:35:07.162528+0000","last_undegraded":"2023-10-30T12:35:07.162528+0000","last_fullsized":"2023-10-30T12:35:07.162528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:30:46.092876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250766+0000","last_change":"2023-10-30T12:35:06.213279+0000","last_active":"2023-10-30T12:35:07.250766+0000","last_peered":"2023-10-30T12:35:07.250766+0000","last_clean":"2023-10-30T12:35:07.250766+0000","last_became_active":"2023-10-30T12:35:06.212548+0000","last_became_peered":"2023-10-30T12:35:06.212548+0000","last_unstale":"2023-10-30T12:35:07.250766+0000","last_undegraded":"2023-10-30T12:35:07.250766+0000","last_fullsized":"2023-10-30T12:35:07.250766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:01:57.127208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.200195+0000","last_change":"2023-10-30T12:35:05.206980+0000","last_active":"2023-10-30T12:35:11.200195+0000","last_peered":"2023-10-30T12:35:11.200195+0000","last_clean":"2023-10-30T12:35:11.200195+0000","last_became_active":"2023-10-30T12:35:05.206559+0000","last_became_peered":"2023-10-30T12:35:05.206559+0000","last_unstale":"2023-10-30T12:35:11.200195+0000","last_undegraded":"2023-10-30T12:35:11.200195+0000","last_fullsized":"2023-10-30T12:35:11.200195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:40.259622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250210+0000","last_change":"2023-10-30T12:35:06.213053+0000","last_active":"2023-10-30T12:35:07.250210+0000","last_peered":"2023-10-30T12:35:07.250210+0000","last_clean":"2023-10-30T12:35:07.250210+0000","last_became_active":"2023-10-30T12:35:06.212505+0000","last_became_peered":"2023-10-30T12:35:06.212505+0000","last_unstale":"2023-10-30T12:35:07.250210+0000","last_undegraded":"2023-10-30T12:35:07.250210+0000","last_fullsized":"2023-10-30T12:35:07.250210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:44:02.073639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.200471+0000","last_change":"2023-10-30T12:35:05.207851+0000","last_active":"2023-10-30T12:35:11.200471+0000","last_peered":"2023-10-30T12:35:11.200471+0000","last_clean":"2023-10-30T12:35:11.200471+0000","last_became_active":"2023-10-30T12:35:05.206879+0000","last_became_peered":"2023-10-30T12:35:05.206879+0000","last_unstale":"2023-10-30T12:35:11.200471+0000","last_undegraded":"2023-10-30T12:35:11.200471+0000","last_fullsized":"2023-10-30T12:35:11.200471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:43:15.717507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160637+0000","last_change":"2023-10-30T12:35:06.222373+0000","last_active":"2023-10-30T12:35:07.160637+0000","last_peered":"2023-10-30T12:35:07.160637+0000","last_clean":"2023-10-30T12:35:07.160637+0000","last_became_active":"2023-10-30T12:35:06.222094+0000","last_became_peered":"2023-10-30T12:35:06.222094+0000","last_unstale":"2023-10-30T12:35:07.160637+0000","last_undegraded":"2023-10-30T12:35:07.160637+0000","last_fullsized":"2023-10-30T12:35:07.160637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:42:32.482862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.444544+0000","last_change":"2023-10-30T12:35:05.174319+0000","last_active":"2023-10-30T12:35:07.444544+0000","last_peered":"2023-10-30T12:35:07.444544+0000","last_clean":"2023-10-30T12:35:07.444544+0000","last_became_active":"2023-10-30T12:35:05.173984+0000","last_became_peered":"2023-10-30T12:35:05.173984+0000","last_unstale":"2023-10-30T12:35:07.444544+0000","last_undegraded":"2023-10-30T12:35:07.444544+0000","last_fullsized":"2023-10-30T12:35:07.444544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:11.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247593+0000","last_change":"2023-10-30T12:35:06.198881+0000","last_active":"2023-10-30T12:35:07.247593+0000","last_peered":"2023-10-30T12:35:07.247593+0000","last_clean":"2023-10-30T12:35:07.247593+0000","last_became_active":"2023-10-30T12:35:06.198504+0000","last_became_peered":"2023-10-30T12:35:06.198504+0000","last_unstale":"2023-10-30T12:35:07.247593+0000","last_undegraded":"2023-10-30T12:35:07.247593+0000","last_fullsized":"2023-10-30T12:35:07.247593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:06:33.845147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.199384+0000","last_change":"2023-10-30T12:35:05.200926+0000","last_active":"2023-10-30T12:35:11.199384+0000","last_peered":"2023-10-30T12:35:11.199384+0000","last_clean":"2023-10-30T12:35:11.199384+0000","last_became_active":"2023-10-30T12:35:05.200623+0000","last_became_peered":"2023-10-30T12:35:05.200623+0000","last_unstale":"2023-10-30T12:35:11.199384+0000","last_undegraded":"2023-10-30T12:35:11.199384+0000","last_fullsized":"2023-10-30T12:35:11.199384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:38.942922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":823296,"data_stored":669814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":27654605},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4890000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5609999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5289999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":78990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":831488,"data_stored":669870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":78954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8010000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.5410000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.847}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.2939999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.3040000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3280000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8799999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":79024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8220000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6919999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2309999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0960000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:35:17.126 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:35:17.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd766b640 1 Processor -- start 2023-10-30T12:35:17.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd766b640 1 -- start start 2023-10-30T12:35:17.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd766b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7facd01580e0 0x7facd0070740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:17.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd766b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facd00c6540 0x7facd015e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:17.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd766b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7facd0069e90 0x7facd006a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:17.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd766b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd00c5810 con 0x7facd0069e90 2023-10-30T12:35:17.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd766b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd00c6960 con 0x7facd01580e0 2023-10-30T12:35:17.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd766b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd00c6bb0 con 0x7facd00c6540 2023-10-30T12:35:17.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd5c08640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7facd0069e90 0x7facd006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:17.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd5c08640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7facd0069e90 0x7facd006a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44728/0 (socket says 172.21.15.12:44728) 2023-10-30T12:35:17.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd5c08640 1 -- 172.21.15.12:0/3946028320 learned_addr learned my addr 172.21.15.12:0/3946028320 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:17.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd5407640 1 --2- 172.21.15.12:0/3946028320 >> v2:172.21.15.154:3300/0 conn(0x7facd01580e0 0x7facd0070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:17.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd4c06640 1 --2- 172.21.15.12:0/3946028320 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facd00c6540 0x7facd015e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:17.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd5c08640 1 -- 172.21.15.12:0/3946028320 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facd00c6540 msgr2=0x7facd015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:17.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd5c08640 1 --2- 172.21.15.12:0/3946028320 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facd00c6540 0x7facd015e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd5c08640 1 -- 172.21.15.12:0/3946028320 >> v2:172.21.15.154:3300/0 conn(0x7facd01580e0 msgr2=0x7facd0070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:17.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd5c08640 1 --2- 172.21.15.12:0/3946028320 >> v2:172.21.15.154:3300/0 conn(0x7facd01580e0 0x7facd0070740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.306+0000 7facd5c08640 1 -- 172.21.15.12:0/3946028320 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd00c6e70 con 0x7facd0069e90 2023-10-30T12:35:17.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.310+0000 7facd5c08640 1 --2- 172.21.15.12:0/3946028320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facd0069e90 0x7facd006a320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7facd006eea0 tx=0x7facc801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a4545ae4ad66b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:17.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.310+0000 7facc7fff640 1 -- 172.21.15.12:0/3946028320 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc80a5760 con 0x7facd0069e90 2023-10-30T12:35:17.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.310+0000 7facc7fff640 1 -- 172.21.15.12:0/3946028320 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc80a58c0 con 0x7facd0069e90 2023-10-30T12:35:17.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.310+0000 7facc7fff640 1 -- 172.21.15.12:0/3946028320 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc80a5c50 con 0x7facd0069e90 2023-10-30T12:35:17.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.310+0000 7facd766b640 1 -- 172.21.15.12:0/3946028320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facd0069e90 msgr2=0x7facd006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:17.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.310+0000 7facd766b640 1 --2- 172.21.15.12:0/3946028320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facd0069e90 0x7facd006a320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7facd006eea0 tx=0x7facc801c9d0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.310+0000 7facd766b640 1 -- 172.21.15.12:0/3946028320 shutdown_connections 2023-10-30T12:35:17.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.310+0000 7facd766b640 1 --2- 172.21.15.12:0/3946028320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facd0069e90 0x7facd006a320 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.310+0000 7facd766b640 1 --2- 172.21.15.12:0/3946028320 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facd00c6540 0x7facd015e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.310+0000 7facd766b640 1 --2- 172.21.15.12:0/3946028320 >> v2:172.21.15.154:3300/0 conn(0x7facd01580e0 0x7facd0070740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.310+0000 7facd766b640 1 -- 172.21.15.12:0/3946028320 >> 172.21.15.12:0/3946028320 conn(0x7facd00c7910 msgr2=0x7facd00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:17.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.310+0000 7facd766b640 1 -- 172.21.15.12:0/3946028320 shutdown_connections 2023-10-30T12:35:17.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.310+0000 7facd766b640 1 -- 172.21.15.12:0/3946028320 wait complete. 2023-10-30T12:35:17.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd766b640 1 Processor -- start 2023-10-30T12:35:17.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd766b640 1 -- start start 2023-10-30T12:35:17.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd766b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facd0069e90 0x7facd01f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:17.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd766b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facd00c6540 0x7facd00c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:17.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd766b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facd01580e0 0x7facd0160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:17.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd766b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd00c3ec0 con 0x7facd0069e90 2023-10-30T12:35:17.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd766b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd01f74c0 con 0x7facd01580e0 2023-10-30T12:35:17.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd766b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd00c9c60 con 0x7facd00c6540 2023-10-30T12:35:17.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd5407640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facd0069e90 0x7facd01f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:17.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd5407640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facd0069e90 0x7facd01f6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44732/0 (socket says 172.21.15.12:44732) 2023-10-30T12:35:17.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd5407640 1 -- 172.21.15.12:0/74389957 learned_addr learned my addr 172.21.15.12:0/74389957 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:17.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd5c08640 1 --2- 172.21.15.12:0/74389957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facd01580e0 0x7facd0160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:17.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd4c06640 1 --2- 172.21.15.12:0/74389957 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facd00c6540 0x7facd00c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:17.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd5407640 1 -- 172.21.15.12:0/74389957 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facd00c6540 msgr2=0x7facd00c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:17.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd5407640 1 --2- 172.21.15.12:0/74389957 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facd00c6540 0x7facd00c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd5407640 1 -- 172.21.15.12:0/74389957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facd01580e0 msgr2=0x7facd0160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:17.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd5407640 1 --2- 172.21.15.12:0/74389957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facd01580e0 0x7facd0160930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd5407640 1 -- 172.21.15.12:0/74389957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc8009b30 con 0x7facd0069e90 2023-10-30T12:35:17.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd5407640 1 --2- 172.21.15.12:0/74389957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facd0069e90 0x7facd01f6f80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faccc00d5f0 tx=0x7faccc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:17.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facc5ffb640 1 -- 172.21.15.12:0/74389957 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc0125f0 con 0x7facd0069e90 2023-10-30T12:35:17.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facc5ffb640 1 -- 172.21.15.12:0/74389957 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faccc012750 con 0x7facd0069e90 2023-10-30T12:35:17.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd766b640 1 -- 172.21.15.12:0/74389957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd00c9f00 con 0x7facd0069e90 2023-10-30T12:35:17.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facc5ffb640 1 -- 172.21.15.12:0/74389957 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc012ae0 con 0x7facd0069e90 2023-10-30T12:35:17.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.314+0000 7facd766b640 1 -- 172.21.15.12:0/74389957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd00ca4a0 con 0x7facd0069e90 2023-10-30T12:35:17.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.318+0000 7facd766b640 1 -- 172.21.15.12:0/74389957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd004e170 con 0x7facd0069e90 2023-10-30T12:35:17.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.318+0000 7facc5ffb640 1 -- 172.21.15.12:0/74389957 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faccc012c40 con 0x7facd0069e90 2023-10-30T12:35:17.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.318+0000 7facc5ffb640 1 --2- 172.21.15.12:0/74389957 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7facb40664f0 0x7facb40689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:17.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.318+0000 7facc5ffb640 1 -- 172.21.15.12:0/74389957 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faccc084120 con 0x7facd0069e90 2023-10-30T12:35:17.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.318+0000 7facd4c06640 1 --2- 172.21.15.12:0/74389957 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7facb40664f0 0x7facb40689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:17.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.318+0000 7facd4c06640 1 --2- 172.21.15.12:0/74389957 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7facb40664f0 0x7facb40689b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7facd0152cc0 tx=0x7facc0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:17.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.322+0000 7facc5ffb640 1 -- 172.21.15.12:0/74389957 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faccc0516a0 con 0x7facd0069e90 2023-10-30T12:35:17.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.474+0000 7facd766b640 1 -- 172.21.15.12:0/74389957 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7facd00c6020 con 0x7facb40664f0 2023-10-30T12:35:17.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.494+0000 7facc5ffb640 1 -- 172.21.15.12:0/74389957 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264515 (secure 0 0 0) 0x7facc000b720 con 0x7facb40664f0 2023-10-30T12:35:17.501 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:17.503 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-10-30T12:35:17.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.498+0000 7facd766b640 1 -- 172.21.15.12:0/74389957 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7facb40664f0 msgr2=0x7facb40689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:17.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.498+0000 7facd766b640 1 --2- 172.21.15.12:0/74389957 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7facb40664f0 0x7facb40689b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7facd0152cc0 tx=0x7facc0008040 comp rx=0 tx=0).stop 2023-10-30T12:35:17.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.498+0000 7facd766b640 1 -- 172.21.15.12:0/74389957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facd0069e90 msgr2=0x7facd01f6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:17.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.498+0000 7facd766b640 1 --2- 172.21.15.12:0/74389957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facd0069e90 0x7facd01f6f80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faccc00d5f0 tx=0x7faccc00dab0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.498+0000 7facd766b640 1 -- 172.21.15.12:0/74389957 shutdown_connections 2023-10-30T12:35:17.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.498+0000 7facd766b640 1 --2- 172.21.15.12:0/74389957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facd01580e0 0x7facd0160930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.498+0000 7facd766b640 1 --2- 172.21.15.12:0/74389957 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7facb40664f0 0x7facb40689b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.498+0000 7facd766b640 1 --2- 172.21.15.12:0/74389957 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7facd00c6540 0x7facd00c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.498+0000 7facd766b640 1 --2- 172.21.15.12:0/74389957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facd0069e90 0x7facd01f6f80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.498+0000 7facd766b640 1 -- 172.21.15.12:0/74389957 >> 172.21.15.12:0/74389957 conn(0x7facd00c7910 msgr2=0x7facd00c0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:17.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.498+0000 7facd766b640 1 -- 172.21.15.12:0/74389957 shutdown_connections 2023-10-30T12:35:17.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.498+0000 7facd766b640 1 -- 172.21.15.12:0/74389957 wait complete. 2023-10-30T12:35:17.522 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-30T12:35:16.938867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219112,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755608,"statfs":{"total":773094113280,"available":772869742592,"internally_reserved":0,"allocated":3055616,"data_stored":1816468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12729,"internal_metadata":221236807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003921"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250527+0000","last_change":"2023-10-30T12:35:06.223711+0000","last_active":"2023-10-30T12:35:07.250527+0000","last_peered":"2023-10-30T12:35:07.250527+0000","last_clean":"2023-10-30T12:35:07.250527+0000","last_became_active":"2023-10-30T12:35:06.222724+0000","last_became_peered":"2023-10-30T12:35:06.222724+0000","last_unstale":"2023-10-30T12:35:07.250527+0000","last_undegraded":"2023-10-30T12:35:07.250527+0000","last_fullsized":"2023-10-30T12:35:07.250527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:34:35.217274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159376+0000","last_change":"2023-10-30T12:35:06.182748+0000","last_active":"2023-10-30T12:35:07.159376+0000","last_peered":"2023-10-30T12:35:07.159376+0000","last_clean":"2023-10-30T12:35:07.159376+0000","last_became_active":"2023-10-30T12:35:06.182441+0000","last_became_peered":"2023-10-30T12:35:06.182441+0000","last_unstale":"2023-10-30T12:35:07.159376+0000","last_undegraded":"2023-10-30T12:35:07.159376+0000","last_fullsized":"2023-10-30T12:35:07.159376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:26:28.943768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160588+0000","last_change":"2023-10-30T12:35:06.180426+0000","last_active":"2023-10-30T12:35:07.160588+0000","last_peered":"2023-10-30T12:35:07.160588+0000","last_clean":"2023-10-30T12:35:07.160588+0000","last_became_active":"2023-10-30T12:35:06.180132+0000","last_became_peered":"2023-10-30T12:35:06.180132+0000","last_unstale":"2023-10-30T12:35:07.160588+0000","last_undegraded":"2023-10-30T12:35:07.160588+0000","last_fullsized":"2023-10-30T12:35:07.160588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:33:22.216739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250158+0000","last_change":"2023-10-30T12:35:06.217986+0000","last_active":"2023-10-30T12:35:07.250158+0000","last_peered":"2023-10-30T12:35:07.250158+0000","last_clean":"2023-10-30T12:35:07.250158+0000","last_became_active":"2023-10-30T12:35:06.213987+0000","last_became_peered":"2023-10-30T12:35:06.213987+0000","last_unstale":"2023-10-30T12:35:07.250158+0000","last_undegraded":"2023-10-30T12:35:07.250158+0000","last_fullsized":"2023-10-30T12:35:07.250158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:51:20.515999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249406+0000","last_change":"2023-10-30T12:35:06.216098+0000","last_active":"2023-10-30T12:35:07.249406+0000","last_peered":"2023-10-30T12:35:07.249406+0000","last_clean":"2023-10-30T12:35:07.249406+0000","last_became_active":"2023-10-30T12:35:06.214964+0000","last_became_peered":"2023-10-30T12:35:06.214964+0000","last_unstale":"2023-10-30T12:35:07.249406+0000","last_undegraded":"2023-10-30T12:35:07.249406+0000","last_fullsized":"2023-10-30T12:35:07.249406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:37:43.156964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249861+0000","last_change":"2023-10-30T12:35:06.217608+0000","last_active":"2023-10-30T12:35:07.249861+0000","last_peered":"2023-10-30T12:35:07.249861+0000","last_clean":"2023-10-30T12:35:07.249861+0000","last_became_active":"2023-10-30T12:35:06.214219+0000","last_became_peered":"2023-10-30T12:35:06.214219+0000","last_unstale":"2023-10-30T12:35:07.249861+0000","last_undegraded":"2023-10-30T12:35:07.249861+0000","last_fullsized":"2023-10-30T12:35:07.249861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:35:27.431813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249000+0000","last_change":"2023-10-30T12:35:06.217120+0000","last_active":"2023-10-30T12:35:07.249000+0000","last_peered":"2023-10-30T12:35:07.249000+0000","last_clean":"2023-10-30T12:35:07.249000+0000","last_became_active":"2023-10-30T12:35:06.216468+0000","last_became_peered":"2023-10-30T12:35:06.216468+0000","last_unstale":"2023-10-30T12:35:07.249000+0000","last_undegraded":"2023-10-30T12:35:07.249000+0000","last_fullsized":"2023-10-30T12:35:07.249000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:26:07.313967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246707+0000","last_change":"2023-10-30T12:35:06.214767+0000","last_active":"2023-10-30T12:35:07.246707+0000","last_peered":"2023-10-30T12:35:07.246707+0000","last_clean":"2023-10-30T12:35:07.246707+0000","last_became_active":"2023-10-30T12:35:06.214384+0000","last_became_peered":"2023-10-30T12:35:06.214384+0000","last_unstale":"2023-10-30T12:35:07.246707+0000","last_undegraded":"2023-10-30T12:35:07.246707+0000","last_fullsized":"2023-10-30T12:35:07.246707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:29.866538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250843+0000","last_change":"2023-10-30T12:35:06.223503+0000","last_active":"2023-10-30T12:35:07.250843+0000","last_peered":"2023-10-30T12:35:07.250843+0000","last_clean":"2023-10-30T12:35:07.250843+0000","last_became_active":"2023-10-30T12:35:06.222692+0000","last_became_peered":"2023-10-30T12:35:06.222692+0000","last_unstale":"2023-10-30T12:35:07.250843+0000","last_undegraded":"2023-10-30T12:35:07.250843+0000","last_fullsized":"2023-10-30T12:35:07.250843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:31:36.926720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163304+0000","last_change":"2023-10-30T12:35:06.215938+0000","last_active":"2023-10-30T12:35:07.163304+0000","last_peered":"2023-10-30T12:35:07.163304+0000","last_clean":"2023-10-30T12:35:07.163304+0000","last_became_active":"2023-10-30T12:35:06.215384+0000","last_became_peered":"2023-10-30T12:35:06.215384+0000","last_unstale":"2023-10-30T12:35:07.163304+0000","last_undegraded":"2023-10-30T12:35:07.163304+0000","last_fullsized":"2023-10-30T12:35:07.163304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:46:21.253555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162425+0000","last_change":"2023-10-30T12:35:06.215965+0000","last_active":"2023-10-30T12:35:07.162425+0000","last_peered":"2023-10-30T12:35:07.162425+0000","last_clean":"2023-10-30T12:35:07.162425+0000","last_became_active":"2023-10-30T12:35:06.215355+0000","last_became_peered":"2023-10-30T12:35:06.215355+0000","last_unstale":"2023-10-30T12:35:07.162425+0000","last_undegraded":"2023-10-30T12:35:07.162425+0000","last_fullsized":"2023-10-30T12:35:07.162425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:07.929526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164185+0000","last_change":"2023-10-30T12:35:06.197278+0000","last_active":"2023-10-30T12:35:07.164185+0000","last_peered":"2023-10-30T12:35:07.164185+0000","last_clean":"2023-10-30T12:35:07.164185+0000","last_became_active":"2023-10-30T12:35:06.196503+0000","last_became_peered":"2023-10-30T12:35:06.196503+0000","last_unstale":"2023-10-30T12:35:07.164185+0000","last_undegraded":"2023-10-30T12:35:07.164185+0000","last_fullsized":"2023-10-30T12:35:07.164185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:06.928264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247199+0000","last_change":"2023-10-30T12:35:06.195721+0000","last_active":"2023-10-30T12:35:07.247199+0000","last_peered":"2023-10-30T12:35:07.247199+0000","last_clean":"2023-10-30T12:35:07.247199+0000","last_became_active":"2023-10-30T12:35:06.195408+0000","last_became_peered":"2023-10-30T12:35:06.195408+0000","last_unstale":"2023-10-30T12:35:07.247199+0000","last_undegraded":"2023-10-30T12:35:07.247199+0000","last_fullsized":"2023-10-30T12:35:07.247199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:59:02.872646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160258+0000","last_change":"2023-10-30T12:35:06.182939+0000","last_active":"2023-10-30T12:35:07.160258+0000","last_peered":"2023-10-30T12:35:07.160258+0000","last_clean":"2023-10-30T12:35:07.160258+0000","last_became_active":"2023-10-30T12:35:06.182573+0000","last_became_peered":"2023-10-30T12:35:06.182573+0000","last_unstale":"2023-10-30T12:35:07.160258+0000","last_undegraded":"2023-10-30T12:35:07.160258+0000","last_fullsized":"2023-10-30T12:35:07.160258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:50.345142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.158834+0000","last_change":"2023-10-30T12:35:06.185636+0000","last_active":"2023-10-30T12:35:07.158834+0000","last_peered":"2023-10-30T12:35:07.158834+0000","last_clean":"2023-10-30T12:35:07.158834+0000","last_became_active":"2023-10-30T12:35:06.185290+0000","last_became_peered":"2023-10-30T12:35:06.185290+0000","last_unstale":"2023-10-30T12:35:07.158834+0000","last_undegraded":"2023-10-30T12:35:07.158834+0000","last_fullsized":"2023-10-30T12:35:07.158834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:26:44.783714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.248972+0000","last_change":"2023-10-30T12:35:06.214488+0000","last_active":"2023-10-30T12:35:07.248972+0000","last_peered":"2023-10-30T12:35:07.248972+0000","last_clean":"2023-10-30T12:35:07.248972+0000","last_became_active":"2023-10-30T12:35:06.213627+0000","last_became_peered":"2023-10-30T12:35:06.213627+0000","last_unstale":"2023-10-30T12:35:07.248972+0000","last_undegraded":"2023-10-30T12:35:07.248972+0000","last_fullsized":"2023-10-30T12:35:07.248972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:22:47.148407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251739+0000","last_change":"2023-10-30T12:35:06.224903+0000","last_active":"2023-10-30T12:35:07.251739+0000","last_peered":"2023-10-30T12:35:07.251739+0000","last_clean":"2023-10-30T12:35:07.251739+0000","last_became_active":"2023-10-30T12:35:06.221696+0000","last_became_peered":"2023-10-30T12:35:06.221696+0000","last_unstale":"2023-10-30T12:35:07.251739+0000","last_undegraded":"2023-10-30T12:35:07.251739+0000","last_fullsized":"2023-10-30T12:35:07.251739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:31:10.439162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246945+0000","last_change":"2023-10-30T12:35:06.198303+0000","last_active":"2023-10-30T12:35:07.246945+0000","last_peered":"2023-10-30T12:35:07.246945+0000","last_clean":"2023-10-30T12:35:07.246945+0000","last_became_active":"2023-10-30T12:35:06.198074+0000","last_became_peered":"2023-10-30T12:35:06.198074+0000","last_unstale":"2023-10-30T12:35:07.246945+0000","last_undegraded":"2023-10-30T12:35:07.246945+0000","last_fullsized":"2023-10-30T12:35:07.246945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:21:36.079586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250503+0000","last_change":"2023-10-30T12:35:06.218556+0000","last_active":"2023-10-30T12:35:07.250503+0000","last_peered":"2023-10-30T12:35:07.250503+0000","last_clean":"2023-10-30T12:35:07.250503+0000","last_became_active":"2023-10-30T12:35:06.215935+0000","last_became_peered":"2023-10-30T12:35:06.215935+0000","last_unstale":"2023-10-30T12:35:07.250503+0000","last_undegraded":"2023-10-30T12:35:07.250503+0000","last_fullsized":"2023-10-30T12:35:07.250503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:36:31.084542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162438+0000","last_change":"2023-10-30T12:35:06.201302+0000","last_active":"2023-10-30T12:35:07.162438+0000","last_peered":"2023-10-30T12:35:07.162438+0000","last_clean":"2023-10-30T12:35:07.162438+0000","last_became_active":"2023-10-30T12:35:06.199812+0000","last_became_peered":"2023-10-30T12:35:06.199812+0000","last_unstale":"2023-10-30T12:35:07.162438+0000","last_undegraded":"2023-10-30T12:35:07.162438+0000","last_fullsized":"2023-10-30T12:35:07.162438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:48.803738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160558+0000","last_change":"2023-10-30T12:35:06.217139+0000","last_active":"2023-10-30T12:35:07.160558+0000","last_peered":"2023-10-30T12:35:07.160558+0000","last_clean":"2023-10-30T12:35:07.160558+0000","last_became_active":"2023-10-30T12:35:06.216861+0000","last_became_peered":"2023-10-30T12:35:06.216861+0000","last_unstale":"2023-10-30T12:35:07.160558+0000","last_undegraded":"2023-10-30T12:35:07.160558+0000","last_fullsized":"2023-10-30T12:35:07.160558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:05:22.193180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249973+0000","last_change":"2023-10-30T12:35:06.225140+0000","last_active":"2023-10-30T12:35:07.249973+0000","last_peered":"2023-10-30T12:35:07.249973+0000","last_clean":"2023-10-30T12:35:07.249973+0000","last_became_active":"2023-10-30T12:35:06.221841+0000","last_became_peered":"2023-10-30T12:35:06.221841+0000","last_unstale":"2023-10-30T12:35:07.249973+0000","last_undegraded":"2023-10-30T12:35:07.249973+0000","last_fullsized":"2023-10-30T12:35:07.249973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:51:57.131025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247032+0000","last_change":"2023-10-30T12:35:06.182662+0000","last_active":"2023-10-30T12:35:07.247032+0000","last_peered":"2023-10-30T12:35:07.247032+0000","last_clean":"2023-10-30T12:35:07.247032+0000","last_became_active":"2023-10-30T12:35:06.182314+0000","last_became_peered":"2023-10-30T12:35:06.182314+0000","last_unstale":"2023-10-30T12:35:07.247032+0000","last_undegraded":"2023-10-30T12:35:07.247032+0000","last_fullsized":"2023-10-30T12:35:07.247032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:41:49.587856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249533+0000","last_change":"2023-10-30T12:35:06.214113+0000","last_active":"2023-10-30T12:35:07.249533+0000","last_peered":"2023-10-30T12:35:07.249533+0000","last_clean":"2023-10-30T12:35:07.249533+0000","last_became_active":"2023-10-30T12:35:06.213402+0000","last_became_peered":"2023-10-30T12:35:06.213402+0000","last_unstale":"2023-10-30T12:35:07.249533+0000","last_undegraded":"2023-10-30T12:35:07.249533+0000","last_fullsized":"2023-10-30T12:35:07.249533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:14:58.409296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247781+0000","last_change":"2023-10-30T12:35:06.188570+0000","last_active":"2023-10-30T12:35:07.247781+0000","last_peered":"2023-10-30T12:35:07.247781+0000","last_clean":"2023-10-30T12:35:07.247781+0000","last_became_active":"2023-10-30T12:35:06.188298+0000","last_became_peered":"2023-10-30T12:35:06.188298+0000","last_unstale":"2023-10-30T12:35:07.247781+0000","last_undegraded":"2023-10-30T12:35:07.247781+0000","last_fullsized":"2023-10-30T12:35:07.247781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:28.281254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163623+0000","last_change":"2023-10-30T12:35:06.208821+0000","last_active":"2023-10-30T12:35:07.163623+0000","last_peered":"2023-10-30T12:35:07.163623+0000","last_clean":"2023-10-30T12:35:07.163623+0000","last_became_active":"2023-10-30T12:35:06.202507+0000","last_became_peered":"2023-10-30T12:35:06.202507+0000","last_unstale":"2023-10-30T12:35:07.163623+0000","last_undegraded":"2023-10-30T12:35:07.163623+0000","last_fullsized":"2023-10-30T12:35:07.163623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:40:13.124316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159171+0000","last_change":"2023-10-30T12:35:06.170313+0000","last_active":"2023-10-30T12:35:07.159171+0000","last_peered":"2023-10-30T12:35:07.159171+0000","last_clean":"2023-10-30T12:35:07.159171+0000","last_became_active":"2023-10-30T12:35:06.170030+0000","last_became_peered":"2023-10-30T12:35:06.170030+0000","last_unstale":"2023-10-30T12:35:07.159171+0000","last_undegraded":"2023-10-30T12:35:07.159171+0000","last_fullsized":"2023-10-30T12:35:07.159171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:49:59.706208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247487+0000","last_change":"2023-10-30T12:35:06.189448+0000","last_active":"2023-10-30T12:35:07.247487+0000","last_peered":"2023-10-30T12:35:07.247487+0000","last_clean":"2023-10-30T12:35:07.247487+0000","last_became_active":"2023-10-30T12:35:06.189171+0000","last_became_peered":"2023-10-30T12:35:06.189171+0000","last_unstale":"2023-10-30T12:35:07.247487+0000","last_undegraded":"2023-10-30T12:35:07.247487+0000","last_fullsized":"2023-10-30T12:35:07.247487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:18:51.463273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160658+0000","last_change":"2023-10-30T12:35:06.183680+0000","last_active":"2023-10-30T12:35:07.160658+0000","last_peered":"2023-10-30T12:35:07.160658+0000","last_clean":"2023-10-30T12:35:07.160658+0000","last_became_active":"2023-10-30T12:35:06.183348+0000","last_became_peered":"2023-10-30T12:35:06.183348+0000","last_unstale":"2023-10-30T12:35:07.160658+0000","last_undegraded":"2023-10-30T12:35:07.160658+0000","last_fullsized":"2023-10-30T12:35:07.160658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:54:25.429897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250238+0000","last_change":"2023-10-30T12:35:06.206675+0000","last_active":"2023-10-30T12:35:07.250238+0000","last_peered":"2023-10-30T12:35:07.250238+0000","last_clean":"2023-10-30T12:35:07.250238+0000","last_became_active":"2023-10-30T12:35:06.196368+0000","last_became_peered":"2023-10-30T12:35:06.196368+0000","last_unstale":"2023-10-30T12:35:07.250238+0000","last_undegraded":"2023-10-30T12:35:07.250238+0000","last_fullsized":"2023-10-30T12:35:07.250238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:35.929311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246681+0000","last_change":"2023-10-30T12:35:06.214408+0000","last_active":"2023-10-30T12:35:07.246681+0000","last_peered":"2023-10-30T12:35:07.246681+0000","last_clean":"2023-10-30T12:35:07.246681+0000","last_became_active":"2023-10-30T12:35:06.213994+0000","last_became_peered":"2023-10-30T12:35:06.213994+0000","last_unstale":"2023-10-30T12:35:07.246681+0000","last_undegraded":"2023-10-30T12:35:07.246681+0000","last_fullsized":"2023-10-30T12:35:07.246681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:05:58.830698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162027+0000","last_change":"2023-10-30T12:35:06.198444+0000","last_active":"2023-10-30T12:35:07.162027+0000","last_peered":"2023-10-30T12:35:07.162027+0000","last_clean":"2023-10-30T12:35:07.162027+0000","last_became_active":"2023-10-30T12:35:06.197647+0000","last_became_peered":"2023-10-30T12:35:06.197647+0000","last_unstale":"2023-10-30T12:35:07.162027+0000","last_undegraded":"2023-10-30T12:35:07.162027+0000","last_fullsized":"2023-10-30T12:35:07.162027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:28:56.075420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163077+0000","last_change":"2023-10-30T12:35:06.168377+0000","last_active":"2023-10-30T12:35:07.163077+0000","last_peered":"2023-10-30T12:35:07.163077+0000","last_clean":"2023-10-30T12:35:07.163077+0000","last_became_active":"2023-10-30T12:35:06.167558+0000","last_became_peered":"2023-10-30T12:35:06.167558+0000","last_unstale":"2023-10-30T12:35:07.163077+0000","last_undegraded":"2023-10-30T12:35:07.163077+0000","last_fullsized":"2023-10-30T12:35:07.163077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:57:28.009164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.444498+0000","last_change":"2023-10-30T12:35:05.200844+0000","last_active":"2023-10-30T12:35:07.444498+0000","last_peered":"2023-10-30T12:35:07.444498+0000","last_clean":"2023-10-30T12:35:07.444498+0000","last_became_active":"2023-10-30T12:35:05.200169+0000","last_became_peered":"2023-10-30T12:35:05.200169+0000","last_unstale":"2023-10-30T12:35:07.444498+0000","last_undegraded":"2023-10-30T12:35:07.444498+0000","last_fullsized":"2023-10-30T12:35:07.444498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:07:19.571357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160718+0000","last_change":"2023-10-30T12:35:06.194964+0000","last_active":"2023-10-30T12:35:07.160718+0000","last_peered":"2023-10-30T12:35:07.160718+0000","last_clean":"2023-10-30T12:35:07.160718+0000","last_became_active":"2023-10-30T12:35:06.194689+0000","last_became_peered":"2023-10-30T12:35:06.194689+0000","last_unstale":"2023-10-30T12:35:07.160718+0000","last_undegraded":"2023-10-30T12:35:07.160718+0000","last_fullsized":"2023-10-30T12:35:07.160718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:30:53.410341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.461020+0000","last_change":"2023-10-30T12:35:05.183569+0000","last_active":"2023-10-30T12:35:07.461020+0000","last_peered":"2023-10-30T12:35:07.461020+0000","last_clean":"2023-10-30T12:35:07.461020+0000","last_became_active":"2023-10-30T12:35:05.182761+0000","last_became_peered":"2023-10-30T12:35:05.182761+0000","last_unstale":"2023-10-30T12:35:07.461020+0000","last_undegraded":"2023-10-30T12:35:07.461020+0000","last_fullsized":"2023-10-30T12:35:07.461020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:49:44.311814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247108+0000","last_change":"2023-10-30T12:35:05.181278+0000","last_active":"2023-10-30T12:35:07.247108+0000","last_peered":"2023-10-30T12:35:07.247108+0000","last_clean":"2023-10-30T12:35:07.247108+0000","last_became_active":"2023-10-30T12:35:05.180730+0000","last_became_peered":"2023-10-30T12:35:05.180730+0000","last_unstale":"2023-10-30T12:35:07.247108+0000","last_undegraded":"2023-10-30T12:35:07.247108+0000","last_fullsized":"2023-10-30T12:35:07.247108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:07:10.828707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246964+0000","last_change":"2023-10-30T12:35:06.180049+0000","last_active":"2023-10-30T12:35:07.246964+0000","last_peered":"2023-10-30T12:35:07.246964+0000","last_clean":"2023-10-30T12:35:07.246964+0000","last_became_active":"2023-10-30T12:35:06.179764+0000","last_became_peered":"2023-10-30T12:35:06.179764+0000","last_unstale":"2023-10-30T12:35:07.246964+0000","last_undegraded":"2023-10-30T12:35:07.246964+0000","last_fullsized":"2023-10-30T12:35:07.246964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:33.136727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.469327+0000","last_change":"2023-10-30T12:35:05.202539+0000","last_active":"2023-10-30T12:35:06.469327+0000","last_peered":"2023-10-30T12:35:06.469327+0000","last_clean":"2023-10-30T12:35:06.469327+0000","last_became_active":"2023-10-30T12:35:05.201853+0000","last_became_peered":"2023-10-30T12:35:05.201853+0000","last_unstale":"2023-10-30T12:35:06.469327+0000","last_undegraded":"2023-10-30T12:35:06.469327+0000","last_fullsized":"2023-10-30T12:35:06.469327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:33:33.151433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.158869+0000","last_change":"2023-10-30T12:35:06.173567+0000","last_active":"2023-10-30T12:35:07.158869+0000","last_peered":"2023-10-30T12:35:07.158869+0000","last_clean":"2023-10-30T12:35:07.158869+0000","last_became_active":"2023-10-30T12:35:06.170179+0000","last_became_peered":"2023-10-30T12:35:06.170179+0000","last_unstale":"2023-10-30T12:35:07.158869+0000","last_undegraded":"2023-10-30T12:35:07.158869+0000","last_fullsized":"2023-10-30T12:35:07.158869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:39:07.880510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163553+0000","last_change":"2023-10-30T12:35:05.198546+0000","last_active":"2023-10-30T12:35:07.163553+0000","last_peered":"2023-10-30T12:35:07.163553+0000","last_clean":"2023-10-30T12:35:07.163553+0000","last_became_active":"2023-10-30T12:35:05.197606+0000","last_became_peered":"2023-10-30T12:35:05.197606+0000","last_unstale":"2023-10-30T12:35:07.163553+0000","last_undegraded":"2023-10-30T12:35:07.163553+0000","last_fullsized":"2023-10-30T12:35:07.163553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:26:57.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251322+0000","last_change":"2023-10-30T12:35:06.226044+0000","last_active":"2023-10-30T12:35:07.251322+0000","last_peered":"2023-10-30T12:35:07.251322+0000","last_clean":"2023-10-30T12:35:07.251322+0000","last_became_active":"2023-10-30T12:35:06.225392+0000","last_became_peered":"2023-10-30T12:35:06.225392+0000","last_unstale":"2023-10-30T12:35:07.251322+0000","last_undegraded":"2023-10-30T12:35:07.251322+0000","last_fullsized":"2023-10-30T12:35:07.251322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:13:10.796549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.469631+0000","last_change":"2023-10-30T12:35:05.197614+0000","last_active":"2023-10-30T12:35:06.469631+0000","last_peered":"2023-10-30T12:35:06.469631+0000","last_clean":"2023-10-30T12:35:06.469631+0000","last_became_active":"2023-10-30T12:35:05.196899+0000","last_became_peered":"2023-10-30T12:35:05.196899+0000","last_unstale":"2023-10-30T12:35:06.469631+0000","last_undegraded":"2023-10-30T12:35:06.469631+0000","last_fullsized":"2023-10-30T12:35:06.469631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:21:00.152426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160715+0000","last_change":"2023-10-30T12:35:06.222344+0000","last_active":"2023-10-30T12:35:07.160715+0000","last_peered":"2023-10-30T12:35:07.160715+0000","last_clean":"2023-10-30T12:35:07.160715+0000","last_became_active":"2023-10-30T12:35:06.222057+0000","last_became_peered":"2023-10-30T12:35:06.222057+0000","last_unstale":"2023-10-30T12:35:07.160715+0000","last_undegraded":"2023-10-30T12:35:07.160715+0000","last_fullsized":"2023-10-30T12:35:07.160715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:50:46.691223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.198046+0000","last_change":"2023-10-30T12:35:05.197501+0000","last_active":"2023-10-30T12:35:11.198046+0000","last_peered":"2023-10-30T12:35:11.198046+0000","last_clean":"2023-10-30T12:35:11.198046+0000","last_became_active":"2023-10-30T12:35:05.197190+0000","last_became_peered":"2023-10-30T12:35:05.197190+0000","last_unstale":"2023-10-30T12:35:11.198046+0000","last_undegraded":"2023-10-30T12:35:11.198046+0000","last_fullsized":"2023-10-30T12:35:11.198046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:53:31.121721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164505+0000","last_change":"2023-10-30T12:35:06.204885+0000","last_active":"2023-10-30T12:35:07.164505+0000","last_peered":"2023-10-30T12:35:07.164505+0000","last_clean":"2023-10-30T12:35:07.164505+0000","last_became_active":"2023-10-30T12:35:06.204122+0000","last_became_peered":"2023-10-30T12:35:06.204122+0000","last_unstale":"2023-10-30T12:35:07.164505+0000","last_undegraded":"2023-10-30T12:35:07.164505+0000","last_fullsized":"2023-10-30T12:35:07.164505+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:27:18.433575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.201072+0000","last_change":"2023-10-30T12:35:05.180005+0000","last_active":"2023-10-30T12:35:11.201072+0000","last_peered":"2023-10-30T12:35:11.201072+0000","last_clean":"2023-10-30T12:35:11.201072+0000","last_became_active":"2023-10-30T12:35:05.179547+0000","last_became_peered":"2023-10-30T12:35:05.179547+0000","last_unstale":"2023-10-30T12:35:11.201072+0000","last_undegraded":"2023-10-30T12:35:11.201072+0000","last_fullsized":"2023-10-30T12:35:11.201072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:13:03.137433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249565+0000","last_change":"2023-10-30T12:35:06.216619+0000","last_active":"2023-10-30T12:35:07.249565+0000","last_peered":"2023-10-30T12:35:07.249565+0000","last_clean":"2023-10-30T12:35:07.249565+0000","last_became_active":"2023-10-30T12:35:06.215128+0000","last_became_peered":"2023-10-30T12:35:06.215128+0000","last_unstale":"2023-10-30T12:35:07.249565+0000","last_undegraded":"2023-10-30T12:35:07.249565+0000","last_fullsized":"2023-10-30T12:35:07.249565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:48:45.673159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.196935+0000","last_change":"2023-10-30T12:35:05.190041+0000","last_active":"2023-10-30T12:35:11.196935+0000","last_peered":"2023-10-30T12:35:11.196935+0000","last_clean":"2023-10-30T12:35:11.196935+0000","last_became_active":"2023-10-30T12:35:05.189573+0000","last_became_peered":"2023-10-30T12:35:05.189573+0000","last_unstale":"2023-10-30T12:35:11.196935+0000","last_undegraded":"2023-10-30T12:35:11.196935+0000","last_fullsized":"2023-10-30T12:35:11.196935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:02:57.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160412+0000","last_change":"2023-10-30T12:35:06.217334+0000","last_active":"2023-10-30T12:35:07.160412+0000","last_peered":"2023-10-30T12:35:07.160412+0000","last_clean":"2023-10-30T12:35:07.160412+0000","last_became_active":"2023-10-30T12:35:06.216885+0000","last_became_peered":"2023-10-30T12:35:06.216885+0000","last_unstale":"2023-10-30T12:35:07.160412+0000","last_undegraded":"2023-10-30T12:35:07.160412+0000","last_fullsized":"2023-10-30T12:35:07.160412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:38:26.745898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.469429+0000","last_change":"2023-10-30T12:35:05.203198+0000","last_active":"2023-10-30T12:35:06.469429+0000","last_peered":"2023-10-30T12:35:06.469429+0000","last_clean":"2023-10-30T12:35:06.469429+0000","last_became_active":"2023-10-30T12:35:05.202612+0000","last_became_peered":"2023-10-30T12:35:05.202612+0000","last_unstale":"2023-10-30T12:35:06.469429+0000","last_undegraded":"2023-10-30T12:35:06.469429+0000","last_fullsized":"2023-10-30T12:35:06.469429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:55.986236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249574+0000","last_change":"2023-10-30T12:35:06.216307+0000","last_active":"2023-10-30T12:35:07.249574+0000","last_peered":"2023-10-30T12:35:07.249574+0000","last_clean":"2023-10-30T12:35:07.249574+0000","last_became_active":"2023-10-30T12:35:06.215553+0000","last_became_peered":"2023-10-30T12:35:06.215553+0000","last_unstale":"2023-10-30T12:35:07.249574+0000","last_undegraded":"2023-10-30T12:35:07.249574+0000","last_fullsized":"2023-10-30T12:35:07.249574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:46:25.396759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.161910+0000","last_change":"2023-10-30T12:35:05.181882+0000","last_active":"2023-10-30T12:35:07.161910+0000","last_peered":"2023-10-30T12:35:07.161910+0000","last_clean":"2023-10-30T12:35:07.161910+0000","last_became_active":"2023-10-30T12:35:05.181115+0000","last_became_peered":"2023-10-30T12:35:05.181115+0000","last_unstale":"2023-10-30T12:35:07.161910+0000","last_undegraded":"2023-10-30T12:35:07.161910+0000","last_fullsized":"2023-10-30T12:35:07.161910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:18:55.496940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162013+0000","last_change":"2023-10-30T12:35:06.198990+0000","last_active":"2023-10-30T12:35:07.162013+0000","last_peered":"2023-10-30T12:35:07.162013+0000","last_clean":"2023-10-30T12:35:07.162013+0000","last_became_active":"2023-10-30T12:35:06.197722+0000","last_became_peered":"2023-10-30T12:35:06.197722+0000","last_unstale":"2023-10-30T12:35:07.162013+0000","last_undegraded":"2023-10-30T12:35:07.162013+0000","last_fullsized":"2023-10-30T12:35:07.162013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:56:51.757976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.197517+0000","last_change":"2023-10-30T12:35:05.169455+0000","last_active":"2023-10-30T12:35:11.197517+0000","last_peered":"2023-10-30T12:35:11.197517+0000","last_clean":"2023-10-30T12:35:11.197517+0000","last_became_active":"2023-10-30T12:35:05.169130+0000","last_became_peered":"2023-10-30T12:35:05.169130+0000","last_unstale":"2023-10-30T12:35:11.197517+0000","last_undegraded":"2023-10-30T12:35:11.197517+0000","last_fullsized":"2023-10-30T12:35:11.197517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:59:14.096315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249183+0000","last_change":"2023-10-30T12:35:06.215545+0000","last_active":"2023-10-30T12:35:07.249183+0000","last_peered":"2023-10-30T12:35:07.249183+0000","last_clean":"2023-10-30T12:35:07.249183+0000","last_became_active":"2023-10-30T12:35:06.214839+0000","last_became_peered":"2023-10-30T12:35:06.214839+0000","last_unstale":"2023-10-30T12:35:07.249183+0000","last_undegraded":"2023-10-30T12:35:07.249183+0000","last_fullsized":"2023-10-30T12:35:07.249183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:54:34.206981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.452194+0000","last_change":"2023-10-30T12:35:05.189361+0000","last_active":"2023-10-30T12:35:07.452194+0000","last_peered":"2023-10-30T12:35:07.452194+0000","last_clean":"2023-10-30T12:35:07.452194+0000","last_became_active":"2023-10-30T12:35:05.188862+0000","last_became_peered":"2023-10-30T12:35:05.188862+0000","last_unstale":"2023-10-30T12:35:07.452194+0000","last_undegraded":"2023-10-30T12:35:07.452194+0000","last_fullsized":"2023-10-30T12:35:07.452194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:13.319732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250712+0000","last_change":"2023-10-30T12:35:06.223118+0000","last_active":"2023-10-30T12:35:07.250712+0000","last_peered":"2023-10-30T12:35:07.250712+0000","last_clean":"2023-10-30T12:35:07.250712+0000","last_became_active":"2023-10-30T12:35:06.216533+0000","last_became_peered":"2023-10-30T12:35:06.216533+0000","last_unstale":"2023-10-30T12:35:07.250712+0000","last_undegraded":"2023-10-30T12:35:07.250712+0000","last_fullsized":"2023-10-30T12:35:07.250712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:32:29.374526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.195596+0000","last_change":"2023-10-30T12:35:05.177609+0000","last_active":"2023-10-30T12:35:11.195596+0000","last_peered":"2023-10-30T12:35:11.195596+0000","last_clean":"2023-10-30T12:35:11.195596+0000","last_became_active":"2023-10-30T12:35:05.177316+0000","last_became_peered":"2023-10-30T12:35:05.177316+0000","last_unstale":"2023-10-30T12:35:11.195596+0000","last_undegraded":"2023-10-30T12:35:11.195596+0000","last_fullsized":"2023-10-30T12:35:11.195596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:46.820925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251056+0000","last_change":"2023-10-30T12:35:06.222283+0000","last_active":"2023-10-30T12:35:07.251056+0000","last_peered":"2023-10-30T12:35:07.251056+0000","last_clean":"2023-10-30T12:35:07.251056+0000","last_became_active":"2023-10-30T12:35:06.221701+0000","last_became_peered":"2023-10-30T12:35:06.221701+0000","last_unstale":"2023-10-30T12:35:07.251056+0000","last_undegraded":"2023-10-30T12:35:07.251056+0000","last_fullsized":"2023-10-30T12:35:07.251056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:37:32.956627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159413+0000","last_change":"2023-10-30T12:35:05.171787+0000","last_active":"2023-10-30T12:35:07.159413+0000","last_peered":"2023-10-30T12:35:07.159413+0000","last_clean":"2023-10-30T12:35:07.159413+0000","last_became_active":"2023-10-30T12:35:05.171491+0000","last_became_peered":"2023-10-30T12:35:05.171491+0000","last_unstale":"2023-10-30T12:35:07.159413+0000","last_undegraded":"2023-10-30T12:35:07.159413+0000","last_fullsized":"2023-10-30T12:35:07.159413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:43:15.331175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250517+0000","last_change":"2023-10-30T12:35:06.214464+0000","last_active":"2023-10-30T12:35:07.250517+0000","last_peered":"2023-10-30T12:35:07.250517+0000","last_clean":"2023-10-30T12:35:07.250517+0000","last_became_active":"2023-10-30T12:35:06.213493+0000","last_became_peered":"2023-10-30T12:35:06.213493+0000","last_unstale":"2023-10-30T12:35:07.250517+0000","last_undegraded":"2023-10-30T12:35:07.250517+0000","last_fullsized":"2023-10-30T12:35:07.250517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:00:18.624720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246924+0000","last_change":"2023-10-30T12:35:05.197811+0000","last_active":"2023-10-30T12:35:07.246924+0000","last_peered":"2023-10-30T12:35:07.246924+0000","last_clean":"2023-10-30T12:35:07.246924+0000","last_became_active":"2023-10-30T12:35:05.197453+0000","last_became_peered":"2023-10-30T12:35:05.197453+0000","last_unstale":"2023-10-30T12:35:07.246924+0000","last_undegraded":"2023-10-30T12:35:07.246924+0000","last_fullsized":"2023-10-30T12:35:07.246924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:16:14.550887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249901+0000","last_change":"2023-10-30T12:35:06.220236+0000","last_active":"2023-10-30T12:35:07.249901+0000","last_peered":"2023-10-30T12:35:07.249901+0000","last_clean":"2023-10-30T12:35:07.249901+0000","last_became_active":"2023-10-30T12:35:06.219526+0000","last_became_peered":"2023-10-30T12:35:06.219526+0000","last_unstale":"2023-10-30T12:35:07.249901+0000","last_undegraded":"2023-10-30T12:35:07.249901+0000","last_fullsized":"2023-10-30T12:35:07.249901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:00:25.297590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.186752+0000","last_change":"2023-10-30T12:34:55.206976+0000","last_active":"2023-10-30T12:35:06.186752+0000","last_peered":"2023-10-30T12:35:06.186752+0000","last_clean":"2023-10-30T12:35:06.186752+0000","last_became_active":"2023-10-30T12:34:55.205923+0000","last_became_peered":"2023-10-30T12:34:55.205923+0000","last_unstale":"2023-10-30T12:35:06.186752+0000","last_undegraded":"2023-10-30T12:35:06.186752+0000","last_fullsized":"2023-10-30T12:35:06.186752+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_clean_scrub_stamp":"2023-10-30T12:34:53.444020+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:25:28.782111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.469475+0000","last_change":"2023-10-30T12:35:05.197974+0000","last_active":"2023-10-30T12:35:06.469475+0000","last_peered":"2023-10-30T12:35:06.469475+0000","last_clean":"2023-10-30T12:35:06.469475+0000","last_became_active":"2023-10-30T12:35:05.197201+0000","last_became_peered":"2023-10-30T12:35:05.197201+0000","last_unstale":"2023-10-30T12:35:06.469475+0000","last_undegraded":"2023-10-30T12:35:06.469475+0000","last_fullsized":"2023-10-30T12:35:06.469475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:39:47.119091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247224+0000","last_change":"2023-10-30T12:35:06.217973+0000","last_active":"2023-10-30T12:35:07.247224+0000","last_peered":"2023-10-30T12:35:07.247224+0000","last_clean":"2023-10-30T12:35:07.247224+0000","last_became_active":"2023-10-30T12:35:06.217687+0000","last_became_peered":"2023-10-30T12:35:06.217687+0000","last_unstale":"2023-10-30T12:35:07.247224+0000","last_undegraded":"2023-10-30T12:35:07.247224+0000","last_fullsized":"2023-10-30T12:35:07.247224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:28.392113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.248732+0000","last_change":"2023-10-30T12:35:05.208339+0000","last_active":"2023-10-30T12:35:07.248732+0000","last_peered":"2023-10-30T12:35:07.248732+0000","last_clean":"2023-10-30T12:35:07.248732+0000","last_became_active":"2023-10-30T12:35:05.207828+0000","last_became_peered":"2023-10-30T12:35:05.207828+0000","last_unstale":"2023-10-30T12:35:07.248732+0000","last_undegraded":"2023-10-30T12:35:07.248732+0000","last_fullsized":"2023-10-30T12:35:07.248732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:00.649048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160255+0000","last_change":"2023-10-30T12:35:06.195094+0000","last_active":"2023-10-30T12:35:07.160255+0000","last_peered":"2023-10-30T12:35:07.160255+0000","last_clean":"2023-10-30T12:35:07.160255+0000","last_became_active":"2023-10-30T12:35:06.194791+0000","last_became_peered":"2023-10-30T12:35:06.194791+0000","last_unstale":"2023-10-30T12:35:07.160255+0000","last_undegraded":"2023-10-30T12:35:07.160255+0000","last_fullsized":"2023-10-30T12:35:07.160255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:41:24.310349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246778+0000","last_change":"2023-10-30T12:35:05.176296+0000","last_active":"2023-10-30T12:35:07.246778+0000","last_peered":"2023-10-30T12:35:07.246778+0000","last_clean":"2023-10-30T12:35:07.246778+0000","last_became_active":"2023-10-30T12:35:05.175959+0000","last_became_peered":"2023-10-30T12:35:05.175959+0000","last_unstale":"2023-10-30T12:35:07.246778+0000","last_undegraded":"2023-10-30T12:35:07.246778+0000","last_fullsized":"2023-10-30T12:35:07.246778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:53.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159052+0000","last_change":"2023-10-30T12:35:06.167250+0000","last_active":"2023-10-30T12:35:07.159052+0000","last_peered":"2023-10-30T12:35:07.159052+0000","last_clean":"2023-10-30T12:35:07.159052+0000","last_became_active":"2023-10-30T12:35:06.164711+0000","last_became_peered":"2023-10-30T12:35:06.164711+0000","last_unstale":"2023-10-30T12:35:07.159052+0000","last_undegraded":"2023-10-30T12:35:07.159052+0000","last_fullsized":"2023-10-30T12:35:07.159052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:48:18.710410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.469452+0000","last_change":"2023-10-30T12:35:05.198964+0000","last_active":"2023-10-30T12:35:06.469452+0000","last_peered":"2023-10-30T12:35:06.469452+0000","last_clean":"2023-10-30T12:35:06.469452+0000","last_became_active":"2023-10-30T12:35:05.196873+0000","last_became_peered":"2023-10-30T12:35:05.196873+0000","last_unstale":"2023-10-30T12:35:06.469452+0000","last_undegraded":"2023-10-30T12:35:06.469452+0000","last_fullsized":"2023-10-30T12:35:06.469452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:47.793585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163526+0000","last_change":"2023-10-30T12:35:06.184123+0000","last_active":"2023-10-30T12:35:07.163526+0000","last_peered":"2023-10-30T12:35:07.163526+0000","last_clean":"2023-10-30T12:35:07.163526+0000","last_became_active":"2023-10-30T12:35:06.183225+0000","last_became_peered":"2023-10-30T12:35:06.183225+0000","last_unstale":"2023-10-30T12:35:07.163526+0000","last_undegraded":"2023-10-30T12:35:07.163526+0000","last_fullsized":"2023-10-30T12:35:07.163526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:03:47.097151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.543947+0000","last_change":"2023-10-30T12:35:05.177571+0000","last_active":"2023-10-30T12:35:06.543947+0000","last_peered":"2023-10-30T12:35:06.543947+0000","last_clean":"2023-10-30T12:35:06.543947+0000","last_became_active":"2023-10-30T12:35:05.177266+0000","last_became_peered":"2023-10-30T12:35:05.177266+0000","last_unstale":"2023-10-30T12:35:06.543947+0000","last_undegraded":"2023-10-30T12:35:06.543947+0000","last_fullsized":"2023-10-30T12:35:06.543947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:14.601934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250949+0000","last_change":"2023-10-30T12:35:06.206033+0000","last_active":"2023-10-30T12:35:07.250949+0000","last_peered":"2023-10-30T12:35:07.250949+0000","last_clean":"2023-10-30T12:35:07.250949+0000","last_became_active":"2023-10-30T12:35:06.196307+0000","last_became_peered":"2023-10-30T12:35:06.196307+0000","last_unstale":"2023-10-30T12:35:07.250949+0000","last_undegraded":"2023-10-30T12:35:07.250949+0000","last_fullsized":"2023-10-30T12:35:07.250949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:18:28.848259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250720+0000","last_change":"2023-10-30T12:35:05.207594+0000","last_active":"2023-10-30T12:35:07.250720+0000","last_peered":"2023-10-30T12:35:07.250720+0000","last_clean":"2023-10-30T12:35:07.250720+0000","last_became_active":"2023-10-30T12:35:05.206930+0000","last_became_peered":"2023-10-30T12:35:05.206930+0000","last_unstale":"2023-10-30T12:35:07.250720+0000","last_undegraded":"2023-10-30T12:35:07.250720+0000","last_fullsized":"2023-10-30T12:35:07.250720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:23:46.519077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159110+0000","last_change":"2023-10-30T12:35:06.165126+0000","last_active":"2023-10-30T12:35:07.159110+0000","last_peered":"2023-10-30T12:35:07.159110+0000","last_clean":"2023-10-30T12:35:07.159110+0000","last_became_active":"2023-10-30T12:35:06.164733+0000","last_became_peered":"2023-10-30T12:35:06.164733+0000","last_unstale":"2023-10-30T12:35:07.159110+0000","last_undegraded":"2023-10-30T12:35:07.159110+0000","last_fullsized":"2023-10-30T12:35:07.159110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:29.336555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.544006+0000","last_change":"2023-10-30T12:35:05.185034+0000","last_active":"2023-10-30T12:35:06.544006+0000","last_peered":"2023-10-30T12:35:06.544006+0000","last_clean":"2023-10-30T12:35:06.544006+0000","last_became_active":"2023-10-30T12:35:05.184695+0000","last_became_peered":"2023-10-30T12:35:05.184695+0000","last_unstale":"2023-10-30T12:35:06.544006+0000","last_undegraded":"2023-10-30T12:35:06.544006+0000","last_fullsized":"2023-10-30T12:35:06.544006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:01:42.019128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247746+0000","last_change":"2023-10-30T12:35:06.204707+0000","last_active":"2023-10-30T12:35:07.247746+0000","last_peered":"2023-10-30T12:35:07.247746+0000","last_clean":"2023-10-30T12:35:07.247746+0000","last_became_active":"2023-10-30T12:35:06.204422+0000","last_became_peered":"2023-10-30T12:35:06.204422+0000","last_unstale":"2023-10-30T12:35:07.247746+0000","last_undegraded":"2023-10-30T12:35:07.247746+0000","last_fullsized":"2023-10-30T12:35:07.247746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:58:16.758575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.201581+0000","last_change":"2023-10-30T12:35:05.200871+0000","last_active":"2023-10-30T12:35:11.201581+0000","last_peered":"2023-10-30T12:35:11.201581+0000","last_clean":"2023-10-30T12:35:11.201581+0000","last_became_active":"2023-10-30T12:35:05.200449+0000","last_became_peered":"2023-10-30T12:35:05.200449+0000","last_unstale":"2023-10-30T12:35:11.201581+0000","last_undegraded":"2023-10-30T12:35:11.201581+0000","last_fullsized":"2023-10-30T12:35:11.201581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:39:20.867587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160562+0000","last_change":"2023-10-30T12:35:06.180850+0000","last_active":"2023-10-30T12:35:07.160562+0000","last_peered":"2023-10-30T12:35:07.160562+0000","last_clean":"2023-10-30T12:35:07.160562+0000","last_became_active":"2023-10-30T12:35:06.179412+0000","last_became_peered":"2023-10-30T12:35:06.179412+0000","last_unstale":"2023-10-30T12:35:07.160562+0000","last_undegraded":"2023-10-30T12:35:07.160562+0000","last_fullsized":"2023-10-30T12:35:07.160562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:47:44.055205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.248731+0000","last_change":"2023-10-30T12:35:05.207392+0000","last_active":"2023-10-30T12:35:07.248731+0000","last_peered":"2023-10-30T12:35:07.248731+0000","last_clean":"2023-10-30T12:35:07.248731+0000","last_became_active":"2023-10-30T12:35:05.206607+0000","last_became_peered":"2023-10-30T12:35:05.206607+0000","last_unstale":"2023-10-30T12:35:07.248731+0000","last_undegraded":"2023-10-30T12:35:07.248731+0000","last_fullsized":"2023-10-30T12:35:07.248731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:36.758740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163445+0000","last_change":"2023-10-30T12:35:06.182042+0000","last_active":"2023-10-30T12:35:07.163445+0000","last_peered":"2023-10-30T12:35:07.163445+0000","last_clean":"2023-10-30T12:35:07.163445+0000","last_became_active":"2023-10-30T12:35:06.181256+0000","last_became_peered":"2023-10-30T12:35:06.181256+0000","last_unstale":"2023-10-30T12:35:07.163445+0000","last_undegraded":"2023-10-30T12:35:07.163445+0000","last_fullsized":"2023-10-30T12:35:07.163445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:13:08.109392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.543919+0000","last_change":"2023-10-30T12:35:05.201024+0000","last_active":"2023-10-30T12:35:06.543919+0000","last_peered":"2023-10-30T12:35:06.543919+0000","last_clean":"2023-10-30T12:35:06.543919+0000","last_became_active":"2023-10-30T12:35:05.200724+0000","last_became_peered":"2023-10-30T12:35:05.200724+0000","last_unstale":"2023-10-30T12:35:06.543919+0000","last_undegraded":"2023-10-30T12:35:06.543919+0000","last_fullsized":"2023-10-30T12:35:06.543919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:24.746469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247249+0000","last_change":"2023-10-30T12:35:06.174636+0000","last_active":"2023-10-30T12:35:07.247249+0000","last_peered":"2023-10-30T12:35:07.247249+0000","last_clean":"2023-10-30T12:35:07.247249+0000","last_became_active":"2023-10-30T12:35:06.173996+0000","last_became_peered":"2023-10-30T12:35:06.173996+0000","last_unstale":"2023-10-30T12:35:07.247249+0000","last_undegraded":"2023-10-30T12:35:07.247249+0000","last_fullsized":"2023-10-30T12:35:07.247249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:01:38.503494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.543930+0000","last_change":"2023-10-30T12:35:05.164079+0000","last_active":"2023-10-30T12:35:06.543930+0000","last_peered":"2023-10-30T12:35:06.543930+0000","last_clean":"2023-10-30T12:35:06.543930+0000","last_became_active":"2023-10-30T12:35:05.163794+0000","last_became_peered":"2023-10-30T12:35:05.163794+0000","last_unstale":"2023-10-30T12:35:06.543930+0000","last_undegraded":"2023-10-30T12:35:06.543930+0000","last_fullsized":"2023-10-30T12:35:06.543930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:19.795656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247344+0000","last_change":"2023-10-30T12:35:06.183531+0000","last_active":"2023-10-30T12:35:07.247344+0000","last_peered":"2023-10-30T12:35:07.247344+0000","last_clean":"2023-10-30T12:35:07.247344+0000","last_became_active":"2023-10-30T12:35:06.183226+0000","last_became_peered":"2023-10-30T12:35:06.183226+0000","last_unstale":"2023-10-30T12:35:07.247344+0000","last_undegraded":"2023-10-30T12:35:07.247344+0000","last_fullsized":"2023-10-30T12:35:07.247344+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:46.768356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.200408+0000","last_change":"2023-10-30T12:35:05.206995+0000","last_active":"2023-10-30T12:35:11.200408+0000","last_peered":"2023-10-30T12:35:11.200408+0000","last_clean":"2023-10-30T12:35:11.200408+0000","last_became_active":"2023-10-30T12:35:05.206518+0000","last_became_peered":"2023-10-30T12:35:05.206518+0000","last_unstale":"2023-10-30T12:35:11.200408+0000","last_undegraded":"2023-10-30T12:35:11.200408+0000","last_fullsized":"2023-10-30T12:35:11.200408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:41:33.703261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162528+0000","last_change":"2023-10-30T12:35:06.199557+0000","last_active":"2023-10-30T12:35:07.162528+0000","last_peered":"2023-10-30T12:35:07.162528+0000","last_clean":"2023-10-30T12:35:07.162528+0000","last_became_active":"2023-10-30T12:35:06.197861+0000","last_became_peered":"2023-10-30T12:35:06.197861+0000","last_unstale":"2023-10-30T12:35:07.162528+0000","last_undegraded":"2023-10-30T12:35:07.162528+0000","last_fullsized":"2023-10-30T12:35:07.162528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:30:46.092876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250766+0000","last_change":"2023-10-30T12:35:06.213279+0000","last_active":"2023-10-30T12:35:07.250766+0000","last_peered":"2023-10-30T12:35:07.250766+0000","last_clean":"2023-10-30T12:35:07.250766+0000","last_became_active":"2023-10-30T12:35:06.212548+0000","last_became_peered":"2023-10-30T12:35:06.212548+0000","last_unstale":"2023-10-30T12:35:07.250766+0000","last_undegraded":"2023-10-30T12:35:07.250766+0000","last_fullsized":"2023-10-30T12:35:07.250766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:01:57.127208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.200195+0000","last_change":"2023-10-30T12:35:05.206980+0000","last_active":"2023-10-30T12:35:11.200195+0000","last_peered":"2023-10-30T12:35:11.200195+0000","last_clean":"2023-10-30T12:35:11.200195+0000","last_became_active":"2023-10-30T12:35:05.206559+0000","last_became_peered":"2023-10-30T12:35:05.206559+0000","last_unstale":"2023-10-30T12:35:11.200195+0000","last_undegraded":"2023-10-30T12:35:11.200195+0000","last_fullsized":"2023-10-30T12:35:11.200195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:40.259622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250210+0000","last_change":"2023-10-30T12:35:06.213053+0000","last_active":"2023-10-30T12:35:07.250210+0000","last_peered":"2023-10-30T12:35:07.250210+0000","last_clean":"2023-10-30T12:35:07.250210+0000","last_became_active":"2023-10-30T12:35:06.212505+0000","last_became_peered":"2023-10-30T12:35:06.212505+0000","last_unstale":"2023-10-30T12:35:07.250210+0000","last_undegraded":"2023-10-30T12:35:07.250210+0000","last_fullsized":"2023-10-30T12:35:07.250210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:44:02.073639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.200471+0000","last_change":"2023-10-30T12:35:05.207851+0000","last_active":"2023-10-30T12:35:11.200471+0000","last_peered":"2023-10-30T12:35:11.200471+0000","last_clean":"2023-10-30T12:35:11.200471+0000","last_became_active":"2023-10-30T12:35:05.206879+0000","last_became_peered":"2023-10-30T12:35:05.206879+0000","last_unstale":"2023-10-30T12:35:11.200471+0000","last_undegraded":"2023-10-30T12:35:11.200471+0000","last_fullsized":"2023-10-30T12:35:11.200471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:43:15.717507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160637+0000","last_change":"2023-10-30T12:35:06.222373+0000","last_active":"2023-10-30T12:35:07.160637+0000","last_peered":"2023-10-30T12:35:07.160637+0000","last_clean":"2023-10-30T12:35:07.160637+0000","last_became_active":"2023-10-30T12:35:06.222094+0000","last_became_peered":"2023-10-30T12:35:06.222094+0000","last_unstale":"2023-10-30T12:35:07.160637+0000","last_undegraded":"2023-10-30T12:35:07.160637+0000","last_fullsized":"2023-10-30T12:35:07.160637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:42:32.482862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.444544+0000","last_change":"2023-10-30T12:35:05.174319+0000","last_active":"2023-10-30T12:35:07.444544+0000","last_peered":"2023-10-30T12:35:07.444544+0000","last_clean":"2023-10-30T12:35:07.444544+0000","last_became_active":"2023-10-30T12:35:05.173984+0000","last_became_peered":"2023-10-30T12:35:05.173984+0000","last_unstale":"2023-10-30T12:35:07.444544+0000","last_undegraded":"2023-10-30T12:35:07.444544+0000","last_fullsized":"2023-10-30T12:35:07.444544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:11.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247593+0000","last_change":"2023-10-30T12:35:06.198881+0000","last_active":"2023-10-30T12:35:07.247593+0000","last_peered":"2023-10-30T12:35:07.247593+0000","last_clean":"2023-10-30T12:35:07.247593+0000","last_became_active":"2023-10-30T12:35:06.198504+0000","last_became_peered":"2023-10-30T12:35:06.198504+0000","last_unstale":"2023-10-30T12:35:07.247593+0000","last_undegraded":"2023-10-30T12:35:07.247593+0000","last_fullsized":"2023-10-30T12:35:07.247593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:06:33.845147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.199384+0000","last_change":"2023-10-30T12:35:05.200926+0000","last_active":"2023-10-30T12:35:11.199384+0000","last_peered":"2023-10-30T12:35:11.199384+0000","last_clean":"2023-10-30T12:35:11.199384+0000","last_became_active":"2023-10-30T12:35:05.200623+0000","last_became_peered":"2023-10-30T12:35:05.200623+0000","last_unstale":"2023-10-30T12:35:11.199384+0000","last_undegraded":"2023-10-30T12:35:11.199384+0000","last_fullsized":"2023-10-30T12:35:11.199384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:38.942922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":823296,"data_stored":669814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":27654605},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4890000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5609999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5289999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":78990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":831488,"data_stored":669870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":78954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8010000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.5410000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.847}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.2939999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.3040000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3280000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8799999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":79024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8220000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6919999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2309999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0960000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:35:17.528 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-30T12:35:17.528 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-30T12:35:17.529 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-30T12:35:17.529 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-30T12:35:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa040a6b640 1 Processor -- start 2023-10-30T12:35:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa040a6b640 1 -- start start 2023-10-30T12:35:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa040a6b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa03c153c60 0x7fa03c0ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa040a6b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa03c0cb780 con 0x7fa03c0c4430 2023-10-30T12:35:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa040a6b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa03c15a2e0 con 0x7fa03c159830 2023-10-30T12:35:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa040a6b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c154040 con 0x7fa03c153c60 2023-10-30T12:35:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa039d9b640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa03c0c4430 0x7fa03c0cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:55498/0 (socket says 172.21.15.12:55498) 2023-10-30T12:35:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa039d9b640 1 -- 172.21.15.12:0/407232279 learned_addr learned my addr 172.21.15.12:0/407232279 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03a59c640 1 --2- 172.21.15.12:0/407232279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa03c153c60 0x7fa03c0ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03959a640 1 -- 172.21.15.12:0/407232279 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1829200507 0 0) 0x7fa03c0cb780 con 0x7fa03c0c4430 2023-10-30T12:35:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03959a640 1 -- 172.21.15.12:0/407232279 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa024003530 con 0x7fa03c0c4430 2023-10-30T12:35:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03959a640 1 -- 172.21.15.12:0/407232279 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1390758958 0 0) 0x7fa024003530 con 0x7fa03c0c4430 2023-10-30T12:35:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03959a640 1 -- 172.21.15.12:0/407232279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa03c153c60 msgr2=0x7fa03c0ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03959a640 1 --2- 172.21.15.12:0/407232279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa03c153c60 0x7fa03c0ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03959a640 1 -- 172.21.15.12:0/407232279 >> v1:172.21.15.154:6789/0 conn(0x7fa03c159830 legacy=0x7fa03c0cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03959a640 1 -- 172.21.15.12:0/407232279 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c15a5c0 con 0x7fa03c0c4430 2023-10-30T12:35:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03959a640 1 -- 172.21.15.12:0/407232279 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fa030001680 con 0x7fa03c0c4430 2023-10-30T12:35:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03959a640 1 -- 172.21.15.12:0/407232279 >> v1:172.21.15.12:6789/0 conn(0x7fa03c0c4430 legacy=0x7fa03c0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03959a640 1 --2- 172.21.15.12:0/407232279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0240039b0 0x7fa024003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03959a640 1 --2- 172.21.15.12:0/407232279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0240047d0 0x7fa024004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03959a640 1 --2- 172.21.15.12:0/407232279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa03c153c60 0x7fa024005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03959a640 1 -- 172.21.15.12:0/407232279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c154040 con 0x7fa0240039b0 2023-10-30T12:35:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03959a640 1 -- 172.21.15.12:0/407232279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02400ba40 con 0x7fa0240047d0 2023-10-30T12:35:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03959a640 1 -- 172.21.15.12:0/407232279 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02400bc00 con 0x7fa03c153c60 2023-10-30T12:35:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03a59c640 1 --2- 172.21.15.12:0/407232279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0240039b0 0x7fa024003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa039d9b640 1 --2- 172.21.15.12:0/407232279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0240047d0 0x7fa024004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03ad9d640 1 --2- 172.21.15.12:0/407232279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa03c153c60 0x7fa024005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03a59c640 1 -- 172.21.15.12:0/407232279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa03c153c60 msgr2=0x7fa024005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03a59c640 1 --2- 172.21.15.12:0/407232279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa03c153c60 0x7fa024005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03a59c640 1 -- 172.21.15.12:0/407232279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0240047d0 msgr2=0x7fa024004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03a59c640 1 --2- 172.21.15.12:0/407232279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0240047d0 0x7fa024004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.702+0000 7fa03a59c640 1 -- 172.21.15.12:0/407232279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa024003530 con 0x7fa0240039b0 2023-10-30T12:35:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.706+0000 7fa03a59c640 1 --2- 172.21.15.12:0/407232279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0240039b0 0x7fa024003dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009ab0 tx=0x7fa02c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c20964a8e55589a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.706+0000 7fa03959a640 1 -- 172.21.15.12:0/407232279 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c0a64d0 con 0x7fa0240039b0 2023-10-30T12:35:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.706+0000 7fa03959a640 1 -- 172.21.15.12:0/407232279 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02c0a6630 con 0x7fa0240039b0 2023-10-30T12:35:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.706+0000 7fa03959a640 1 -- 172.21.15.12:0/407232279 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c0a69e0 con 0x7fa0240039b0 2023-10-30T12:35:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.706+0000 7fa040a6b640 1 -- 172.21.15.12:0/407232279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0240039b0 msgr2=0x7fa024003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.706+0000 7fa040a6b640 1 --2- 172.21.15.12:0/407232279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0240039b0 0x7fa024003dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009ab0 tx=0x7fa02c01c9e0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.706+0000 7fa03ad9d640 1 -- 172.21.15.12:0/407232279 reap_dead start 2023-10-30T12:35:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.706+0000 7fa040a6b640 1 -- 172.21.15.12:0/407232279 shutdown_connections 2023-10-30T12:35:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.706+0000 7fa040a6b640 1 -- 172.21.15.12:0/407232279 >> 172.21.15.12:0/407232279 conn(0x7fa03c06bfe0 msgr2=0x7fa03c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.706+0000 7fa040a6b640 1 -- 172.21.15.12:0/407232279 shutdown_connections 2023-10-30T12:35:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.706+0000 7fa040a6b640 1 -- 172.21.15.12:0/407232279 wait complete. 2023-10-30T12:35:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa040a6b640 1 Processor -- start 2023-10-30T12:35:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa040a6b640 1 -- start start 2023-10-30T12:35:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa040a6b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0240039b0 0x7fa03c156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa040a6b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa03c0c9360 0x7fa03c0b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa040a6b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa03c158310 0x7fa03c0c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa040a6b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c0c7580 con 0x7fa0240039b0 2023-10-30T12:35:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa040a6b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c0b49f0 con 0x7fa03c0c9360 2023-10-30T12:35:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa040a6b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c0b3fb0 con 0x7fa03c158310 2023-10-30T12:35:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa03a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0240039b0 0x7fa03c156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa03a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0240039b0 0x7fa03c156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44750/0 (socket says 172.21.15.12:44750) 2023-10-30T12:35:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa03a59c640 1 -- 172.21.15.12:0/3726151497 learned_addr learned my addr 172.21.15.12:0/3726151497 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa039d9b640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa03c0c9360 0x7fa03c0b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa03ad9d640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa03c158310 0x7fa03c0c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa03a59c640 1 -- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa03c158310 msgr2=0x7fa03c0c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa03a59c640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa03c158310 0x7fa03c0c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa03a59c640 1 -- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa03c0c9360 msgr2=0x7fa03c0b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa03a59c640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa03c0c9360 0x7fa03c0b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa03a59c640 1 -- 172.21.15.12:0/3726151497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c0b4c90 con 0x7fa0240039b0 2023-10-30T12:35:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa039d9b640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa03c0c9360 0x7fa03c0b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa03ad9d640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa03c158310 0x7fa03c0c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:17.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa03a59c640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0240039b0 0x7fa03c156e40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa02c01cec0 tx=0x7fa024005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:17.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa02affd640 1 -- 172.21.15.12:0/3726151497 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c0a67a0 con 0x7fa0240039b0 2023-10-30T12:35:17.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa02affd640 1 -- 172.21.15.12:0/3726151497 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02c00f000 con 0x7fa0240039b0 2023-10-30T12:35:17.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa040a6b640 1 -- 172.21.15.12:0/3726151497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa03c0b4f70 con 0x7fa0240039b0 2023-10-30T12:35:17.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa02affd640 1 -- 172.21.15.12:0/3726151497 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c004220 con 0x7fa0240039b0 2023-10-30T12:35:17.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.710+0000 7fa040a6b640 1 -- 172.21.15.12:0/3726151497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa03c0c77b0 con 0x7fa0240039b0 2023-10-30T12:35:17.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.714+0000 7fa040a6b640 1 -- 172.21.15.12:0/3726151497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03c0c7970 con 0x7fa0240039b0 2023-10-30T12:35:17.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.714+0000 7fa02affd640 1 -- 172.21.15.12:0/3726151497 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa02c004380 con 0x7fa0240039b0 2023-10-30T12:35:17.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.714+0000 7fa02affd640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa01c0664a0 0x7fa01c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:17.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.714+0000 7fa02affd640 1 -- 172.21.15.12:0/3726151497 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa02c0aa070 con 0x7fa0240039b0 2023-10-30T12:35:17.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.714+0000 7fa039d9b640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa01c0664a0 0x7fa01c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:17.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.718+0000 7fa039d9b640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa01c0664a0 0x7fa01c068960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0c9760 tx=0x7fa030009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:17.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.718+0000 7fa02affd640 1 -- 172.21.15.12:0/3726151497 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa02c0eb4d0 con 0x7fa0240039b0 2023-10-30T12:35:17.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.906+0000 7fa040a6b640 1 -- 172.21.15.12:0/3726151497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa03c0b5420 con 0x7fa0240039b0 2023-10-30T12:35:17.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.910+0000 7fa02affd640 1 -- 172.21.15.12:0/3726151497 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+196 (secure 0 0 0) 0x7fa02c0be020 con 0x7fa0240039b0 2023-10-30T12:35:17.913 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:17.913 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-30T12:35:17.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.910+0000 7fa040a6b640 1 -- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa01c0664a0 msgr2=0x7fa01c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:17.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.910+0000 7fa040a6b640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa01c0664a0 0x7fa01c068960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0c9760 tx=0x7fa030009040 comp rx=0 tx=0).stop 2023-10-30T12:35:17.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.910+0000 7fa040a6b640 1 -- 172.21.15.12:0/3726151497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0240039b0 msgr2=0x7fa03c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:17.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.910+0000 7fa040a6b640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0240039b0 0x7fa03c156e40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa02c01cec0 tx=0x7fa024005800 comp rx=0 tx=0).stop 2023-10-30T12:35:17.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.910+0000 7fa040a6b640 1 -- 172.21.15.12:0/3726151497 shutdown_connections 2023-10-30T12:35:17.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.910+0000 7fa040a6b640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa01c0664a0 0x7fa01c068960 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.910+0000 7fa040a6b640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa03c158310 0x7fa03c0c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.910+0000 7fa040a6b640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa03c0c9360 0x7fa03c0b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.910+0000 7fa040a6b640 1 --2- 172.21.15.12:0/3726151497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0240039b0 0x7fa03c156e40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.910+0000 7fa040a6b640 1 -- 172.21.15.12:0/3726151497 >> 172.21.15.12:0/3726151497 conn(0x7fa03c06bfe0 msgr2=0x7fa03c151230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.910+0000 7fa040a6b640 1 -- 172.21.15.12:0/3726151497 shutdown_connections 2023-10-30T12:35:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:17.910+0000 7fa040a6b640 1 -- 172.21.15.12:0/3726151497 wait complete. 2023-10-30T12:35:20.934 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-30T12:35:21.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.082+0000 7efdfd6c9640 1 Processor -- start 2023-10-30T12:35:21.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.082+0000 7efdfd6c9640 1 -- start start 2023-10-30T12:35:21.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdfd6c9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efdf8153c60 0x7efdf80ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:21.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdfd6c9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf80c4430 0x7efdf80cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:21.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdfd6c9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf80cb780 con 0x7efdf8159830 2023-10-30T12:35:21.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdfd6c9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf8154040 con 0x7efdf8153c60 2023-10-30T12:35:21.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdfd6c9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf80cc9f0 con 0x7efdf80c4430 2023-10-30T12:35:21.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdf77fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7efdf8159830 0x7efdf80cac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58062/0 (socket says 172.21.15.12:58062) 2023-10-30T12:35:21.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdf77fe640 1 -- 172.21.15.12:0/843574908 learned_addr learned my addr 172.21.15.12:0/843574908 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:21.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdf6ffd640 1 --2- 172.21.15.12:0/843574908 >> v2:172.21.15.154:3300/0 conn(0x7efdf8153c60 0x7efdf80ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:21.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdeffff640 1 --2- 172.21.15.12:0/843574908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf80c4430 0x7efdf80cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:21.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdf67fc640 1 -- 172.21.15.12:0/843574908 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547983330 0 0) 0x7efdf80cb780 con 0x7efdf8159830 2023-10-30T12:35:21.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdf67fc640 1 -- 172.21.15.12:0/843574908 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efde4003530 con 0x7efdf8159830 2023-10-30T12:35:21.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdf6ffd640 1 -- 172.21.15.12:0/843574908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf80c4430 msgr2=0x7efdf80cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:21.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdf6ffd640 1 --2- 172.21.15.12:0/843574908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf80c4430 0x7efdf80cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:21.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdf6ffd640 1 -- 172.21.15.12:0/843574908 >> v1:172.21.15.12:6789/0 conn(0x7efdf8159830 legacy=0x7efdf80cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:21.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdf67fc640 1 -- 172.21.15.12:0/843574908 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2497903147 0 0) 0x7efde4003530 con 0x7efdf8159830 2023-10-30T12:35:21.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdf6ffd640 1 -- 172.21.15.12:0/843574908 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf815a2e0 con 0x7efdf8153c60 2023-10-30T12:35:21.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.086+0000 7efdeffff640 1 --2- 172.21.15.12:0/843574908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf80c4430 0x7efdf80cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:35:21.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.090+0000 7efdf6ffd640 1 --2- 172.21.15.12:0/843574908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf8153c60 0x7efdf80ca360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efde80097c0 tx=0x7efde801c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5906002fcdd5ffca server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:21.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.090+0000 7efdf67fc640 1 -- 172.21.15.12:0/843574908 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0001a00 con 0x7efdf8153c60 2023-10-30T12:35:21.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.090+0000 7efdf67fc640 1 -- 172.21.15.12:0/843574908 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde80a7050 con 0x7efdf8153c60 2023-10-30T12:35:21.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.090+0000 7efdf67fc640 1 -- 172.21.15.12:0/843574908 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde80ab600 con 0x7efdf8153c60 2023-10-30T12:35:21.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.090+0000 7efdfd6c9640 1 -- 172.21.15.12:0/843574908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf8153c60 msgr2=0x7efdf80ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:21.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.090+0000 7efdfd6c9640 1 --2- 172.21.15.12:0/843574908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf8153c60 0x7efdf80ca360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efde80097c0 tx=0x7efde801c3a0 comp rx=0 tx=0).stop 2023-10-30T12:35:21.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.090+0000 7efdfd6c9640 1 -- 172.21.15.12:0/843574908 shutdown_connections 2023-10-30T12:35:21.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.090+0000 7efdfd6c9640 1 --2- 172.21.15.12:0/843574908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf80c4430 0x7efdf80cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:21.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.090+0000 7efdfd6c9640 1 --2- 172.21.15.12:0/843574908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf8153c60 0x7efdf80ca360 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:21.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.090+0000 7efdfd6c9640 1 -- 172.21.15.12:0/843574908 >> 172.21.15.12:0/843574908 conn(0x7efdf806bfe0 msgr2=0x7efdf806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:21.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.090+0000 7efdfd6c9640 1 -- 172.21.15.12:0/843574908 shutdown_connections 2023-10-30T12:35:21.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.090+0000 7efdfd6c9640 1 -- 172.21.15.12:0/843574908 wait complete. 2023-10-30T12:35:21.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.090+0000 7efdfd6c9640 1 Processor -- start 2023-10-30T12:35:21.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdfd6c9640 1 -- start start 2023-10-30T12:35:21.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdfd6c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf80c4430 0x7efdf81565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:21.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdfd6c9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf8153c60 0x7efdf814d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:21.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdfd6c9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdf8159830 0x7efdf814d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:21.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdfd6c9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf814df10 con 0x7efdf8159830 2023-10-30T12:35:21.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdfd6c9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf81f1d80 con 0x7efdf80c4430 2023-10-30T12:35:21.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdfd6c9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf814e120 con 0x7efdf8153c60 2023-10-30T12:35:21.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdf77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdf8159830 0x7efdf814d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:21.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdf77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdf8159830 0x7efdf814d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52402/0 (socket says 172.21.15.12:52402) 2023-10-30T12:35:21.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdf77fe640 1 -- 172.21.15.12:0/857662437 learned_addr learned my addr 172.21.15.12:0/857662437 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:21.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdf6ffd640 1 --2- 172.21.15.12:0/857662437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf80c4430 0x7efdf81565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:21.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdeffff640 1 --2- 172.21.15.12:0/857662437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf8153c60 0x7efdf814d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:21.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdf77fe640 1 -- 172.21.15.12:0/857662437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf8153c60 msgr2=0x7efdf814d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:21.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdf77fe640 1 --2- 172.21.15.12:0/857662437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf8153c60 0x7efdf814d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:21.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdf77fe640 1 -- 172.21.15.12:0/857662437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf80c4430 msgr2=0x7efdf81565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:21.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdf77fe640 1 --2- 172.21.15.12:0/857662437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf80c4430 0x7efdf81565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:21.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdf77fe640 1 -- 172.21.15.12:0/857662437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde0009420 con 0x7efdf8159830 2023-10-30T12:35:21.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdf77fe640 1 --2- 172.21.15.12:0/857662437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdf8159830 0x7efdf814d5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efdf0001840 tx=0x7efdf000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:21.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdef7fe640 1 -- 172.21.15.12:0/857662437 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0012400 con 0x7efdf8159830 2023-10-30T12:35:21.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdfd6c9640 1 -- 172.21.15.12:0/857662437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde8009420 con 0x7efdf8159830 2023-10-30T12:35:21.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdef7fe640 1 -- 172.21.15.12:0/857662437 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf0012560 con 0x7efdf8159830 2023-10-30T12:35:21.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdef7fe640 1 -- 172.21.15.12:0/857662437 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0012910 con 0x7efdf8159830 2023-10-30T12:35:21.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdfd6c9640 1 -- 172.21.15.12:0/857662437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf814e6e0 con 0x7efdf8159830 2023-10-30T12:35:21.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.094+0000 7efdfd6c9640 1 -- 172.21.15.12:0/857662437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf81f1ef0 con 0x7efdf8159830 2023-10-30T12:35:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.098+0000 7efdef7fe640 1 -- 172.21.15.12:0/857662437 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efdf0012a70 con 0x7efdf8159830 2023-10-30T12:35:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.098+0000 7efdef7fe640 1 --2- 172.21.15.12:0/857662437 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efdd80664d0 0x7efdd8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.098+0000 7efdef7fe640 1 -- 172.21.15.12:0/857662437 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efdf0084640 con 0x7efdf8159830 2023-10-30T12:35:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.098+0000 7efdf6ffd640 1 --2- 172.21.15.12:0/857662437 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efdd80664d0 0x7efdd8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:21.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.098+0000 7efdf6ffd640 1 --2- 172.21.15.12:0/857662437 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efdd80664d0 0x7efdd8068990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efde80ad000 tx=0x7efde80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:21.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.102+0000 7efdef7fe640 1 -- 172.21.15.12:0/857662437 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efdf81f1ef0 con 0x7efdf8159830 2023-10-30T12:35:21.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.302+0000 7efdfd6c9640 1 -- 172.21.15.12:0/857662437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7efdf81f1ef0 con 0x7efdf8159830 2023-10-30T12:35:21.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.302+0000 7efdef7fe640 1 -- 172.21.15.12:0/857662437 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+196 (secure 0 0 0) 0x7efdf81f1ef0 con 0x7efdf8159830 2023-10-30T12:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-30T12:35:21.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.302+0000 7efdfd6c9640 1 -- 172.21.15.12:0/857662437 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efdd80664d0 msgr2=0x7efdd8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:21.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.302+0000 7efdfd6c9640 1 --2- 172.21.15.12:0/857662437 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efdd80664d0 0x7efdd8068990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efde80ad000 tx=0x7efde80a6000 comp rx=0 tx=0).stop 2023-10-30T12:35:21.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.302+0000 7efdfd6c9640 1 -- 172.21.15.12:0/857662437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdf8159830 msgr2=0x7efdf814d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:21.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.302+0000 7efdfd6c9640 1 --2- 172.21.15.12:0/857662437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdf8159830 0x7efdf814d5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efdf0001840 tx=0x7efdf000a430 comp rx=0 tx=0).stop 2023-10-30T12:35:21.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.302+0000 7efdfd6c9640 1 -- 172.21.15.12:0/857662437 shutdown_connections 2023-10-30T12:35:21.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.302+0000 7efdfd6c9640 1 --2- 172.21.15.12:0/857662437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdf8159830 0x7efdf814d5d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:21.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.302+0000 7efdfd6c9640 1 --2- 172.21.15.12:0/857662437 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efdd80664d0 0x7efdd8068990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:21.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.302+0000 7efdfd6c9640 1 --2- 172.21.15.12:0/857662437 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdf8153c60 0x7efdf814d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:21.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.302+0000 7efdfd6c9640 1 --2- 172.21.15.12:0/857662437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf80c4430 0x7efdf81565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:21.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.302+0000 7efdfd6c9640 1 -- 172.21.15.12:0/857662437 >> 172.21.15.12:0/857662437 conn(0x7efdf806bfe0 msgr2=0x7efdf80b4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:21.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.306+0000 7efdfd6c9640 1 -- 172.21.15.12:0/857662437 shutdown_connections 2023-10-30T12:35:21.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:21.306+0000 7efdfd6c9640 1 -- 172.21.15.12:0/857662437 wait complete. 2023-10-30T12:35:24.330 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-30T12:35:24.469 INFO:tasks.ceph.mds.e.smithi012.stderr:2023-10-30T12:35:24.466+0000 7fbd9da0c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-30T12:35:24.469 INFO:tasks.ceph.mds.e.smithi012.stderr:2023-10-30T12:35:24.466+0000 7fbd9da0c640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-30T12:35:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.482+0000 7f19143b8640 1 Processor -- start 2023-10-30T12:35:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.482+0000 7f19143b8640 1 -- start start 2023-10-30T12:35:24.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.482+0000 7f19143b8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f190c153c60 0x7f190c0ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:24.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.482+0000 7f19143b8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f190c159830 0x7f190c159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:24.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.482+0000 7f19143b8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190c0cb780 con 0x7f190c0c4430 2023-10-30T12:35:24.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.482+0000 7f19143b8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f190c154040 con 0x7f190c153c60 2023-10-30T12:35:24.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.482+0000 7f19143b8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f190c15a2e0 con 0x7f190c159830 2023-10-30T12:35:24.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.482+0000 7f1912154640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f190c153c60 0x7f190c0ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:24.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.482+0000 7f1912154640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f190c153c60 0x7f190c0ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52406/0 (socket says 172.21.15.12:52406) 2023-10-30T12:35:24.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.482+0000 7f1912154640 1 -- 172.21.15.12:0/236270521 learned_addr learned my addr 172.21.15.12:0/236270521 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:24.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.482+0000 7f1912955640 1 --2- 172.21.15.12:0/236270521 >> v2:172.21.15.154:3300/0 conn(0x7f190c159830 0x7f190c159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:24.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.482+0000 7f1912154640 1 -- 172.21.15.12:0/236270521 >> v2:172.21.15.154:3300/0 conn(0x7f190c159830 msgr2=0x7f190c159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:24.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.482+0000 7f1912154640 1 --2- 172.21.15.12:0/236270521 >> v2:172.21.15.154:3300/0 conn(0x7f190c159830 0x7f190c159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.482+0000 7f1912154640 1 -- 172.21.15.12:0/236270521 >> v1:172.21.15.154:6789/0 conn(0x7f190c0c4430 legacy=0x7f190c0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:24.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.482+0000 7f1912154640 1 -- 172.21.15.12:0/236270521 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c15a5c0 con 0x7f190c153c60 2023-10-30T12:35:24.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.486+0000 7f1912154640 1 --2- 172.21.15.12:0/236270521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f190c153c60 0x7f190c0ca360 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f190c073da0 tx=0x7f190801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3563144222025b23 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:24.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.486+0000 7f1911152640 1 -- 172.21.15.12:0/236270521 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19080ab5f0 con 0x7f190c153c60 2023-10-30T12:35:24.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.486+0000 7f1911152640 1 -- 172.21.15.12:0/236270521 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19080ab750 con 0x7f190c153c60 2023-10-30T12:35:24.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.486+0000 7f1911152640 1 -- 172.21.15.12:0/236270521 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19080abae0 con 0x7f190c153c60 2023-10-30T12:35:24.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.486+0000 7f19143b8640 1 -- 172.21.15.12:0/236270521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f190c153c60 msgr2=0x7f190c0ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:24.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.486+0000 7f19143b8640 1 --2- 172.21.15.12:0/236270521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f190c153c60 0x7f190c0ca360 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f190c073da0 tx=0x7f190801c500 comp rx=0 tx=0).stop 2023-10-30T12:35:24.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.486+0000 7f19143b8640 1 -- 172.21.15.12:0/236270521 shutdown_connections 2023-10-30T12:35:24.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.486+0000 7f19143b8640 1 --2- 172.21.15.12:0/236270521 >> v2:172.21.15.154:3300/0 conn(0x7f190c159830 0x7f190c159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.486+0000 7f19143b8640 1 --2- 172.21.15.12:0/236270521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f190c153c60 0x7f190c0ca360 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.486+0000 7f19143b8640 1 -- 172.21.15.12:0/236270521 >> 172.21.15.12:0/236270521 conn(0x7f190c06bfe0 msgr2=0x7f190c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.486+0000 7f19143b8640 1 -- 172.21.15.12:0/236270521 shutdown_connections 2023-10-30T12:35:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.486+0000 7f19143b8640 1 -- 172.21.15.12:0/236270521 wait complete. 2023-10-30T12:35:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f19143b8640 1 Processor -- start 2023-10-30T12:35:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f19143b8640 1 -- start start 2023-10-30T12:35:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f19143b8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f190c0c4430 0x7f190c0c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f19143b8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f190c153c60 0x7f190c1f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f19143b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f190c159830 0x7f190c145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f19143b8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c142530 con 0x7f190c159830 2023-10-30T12:35:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f19143b8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c144940 con 0x7f190c153c60 2023-10-30T12:35:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f19143b8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190c1439e0 con 0x7f190c0c4430 2023-10-30T12:35:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f1912955640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f190c159830 0x7f190c145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f1912955640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f190c159830 0x7f190c145e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52422/0 (socket says 172.21.15.12:52422) 2023-10-30T12:35:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f1912955640 1 -- 172.21.15.12:0/2888248399 learned_addr learned my addr 172.21.15.12:0/2888248399 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f1911953640 1 --2- 172.21.15.12:0/2888248399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f190c153c60 0x7f190c1f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f1912154640 1 --2- 172.21.15.12:0/2888248399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f190c0c4430 0x7f190c0c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f1912955640 1 -- 172.21.15.12:0/2888248399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f190c0c4430 msgr2=0x7f190c0c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f1912955640 1 --2- 172.21.15.12:0/2888248399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f190c0c4430 0x7f190c0c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f1912955640 1 -- 172.21.15.12:0/2888248399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f190c153c60 msgr2=0x7f190c1f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f1912955640 1 --2- 172.21.15.12:0/2888248399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f190c153c60 0x7f190c1f5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f1912955640 1 -- 172.21.15.12:0/2888248399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19080094d0 con 0x7f190c159830 2023-10-30T12:35:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f1912955640 1 --2- 172.21.15.12:0/2888248399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f190c159830 0x7f190c145e20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1904007ec0 tx=0x7f190400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f1902ffd640 1 -- 172.21.15.12:0/2888248399 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904014620 con 0x7f190c159830 2023-10-30T12:35:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f1902ffd640 1 -- 172.21.15.12:0/2888248399 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19040147a0 con 0x7f190c159830 2023-10-30T12:35:24.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f1902ffd640 1 -- 172.21.15.12:0/2888248399 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904014b50 con 0x7f190c159830 2023-10-30T12:35:24.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f19143b8640 1 -- 172.21.15.12:0/2888248399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190c1435f0 con 0x7f190c159830 2023-10-30T12:35:24.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.490+0000 7f19143b8640 1 -- 172.21.15.12:0/2888248399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190c14fd90 con 0x7f190c159830 2023-10-30T12:35:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.494+0000 7f19143b8640 1 -- 172.21.15.12:0/2888248399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190c04e170 con 0x7f190c159830 2023-10-30T12:35:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.494+0000 7f1902ffd640 1 -- 172.21.15.12:0/2888248399 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1904014cb0 con 0x7f190c159830 2023-10-30T12:35:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.494+0000 7f1902ffd640 1 --2- 172.21.15.12:0/2888248399 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f18f00664d0 0x7f18f0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:24.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.494+0000 7f1902ffd640 1 -- 172.21.15.12:0/2888248399 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1904084690 con 0x7f190c159830 2023-10-30T12:35:24.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.494+0000 7f1912154640 1 --2- 172.21.15.12:0/2888248399 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f18f00664d0 0x7f18f0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:24.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.494+0000 7f1912154640 1 --2- 172.21.15.12:0/2888248399 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f18f00664d0 0x7f18f0068990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f19080af000 tx=0x7f19080b6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:24.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.498+0000 7f1902ffd640 1 -- 172.21.15.12:0/2888248399 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1904051b20 con 0x7f190c159830 2023-10-30T12:35:24.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.690+0000 7f19143b8640 1 -- 172.21.15.12:0/2888248399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f190c1437d0 con 0x7f190c159830 2023-10-30T12:35:24.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.690+0000 7f1902ffd640 1 -- 172.21.15.12:0/2888248399 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1904055140 con 0x7f190c159830 2023-10-30T12:35:24.695 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:24.695 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-30T12:35:24.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.694+0000 7f19143b8640 1 -- 172.21.15.12:0/2888248399 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f18f00664d0 msgr2=0x7f18f0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:24.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.694+0000 7f19143b8640 1 --2- 172.21.15.12:0/2888248399 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f18f00664d0 0x7f18f0068990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f19080af000 tx=0x7f19080b6040 comp rx=0 tx=0).stop 2023-10-30T12:35:24.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.694+0000 7f19143b8640 1 -- 172.21.15.12:0/2888248399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f190c159830 msgr2=0x7f190c145e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:24.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.694+0000 7f19143b8640 1 --2- 172.21.15.12:0/2888248399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f190c159830 0x7f190c145e20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1904007ec0 tx=0x7f190400e4b0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.694+0000 7f19143b8640 1 -- 172.21.15.12:0/2888248399 shutdown_connections 2023-10-30T12:35:24.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.694+0000 7f19143b8640 1 --2- 172.21.15.12:0/2888248399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f190c159830 0x7f190c145e20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.694+0000 7f19143b8640 1 --2- 172.21.15.12:0/2888248399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f190c153c60 0x7f190c1f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.694+0000 7f19143b8640 1 --2- 172.21.15.12:0/2888248399 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f18f00664d0 0x7f18f0068990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.694+0000 7f19143b8640 1 --2- 172.21.15.12:0/2888248399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f190c0c4430 0x7f190c0c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.694+0000 7f19143b8640 1 -- 172.21.15.12:0/2888248399 >> 172.21.15.12:0/2888248399 conn(0x7f190c06bfe0 msgr2=0x7f190c0c56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:24.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.694+0000 7f19143b8640 1 -- 172.21.15.12:0/2888248399 shutdown_connections 2023-10-30T12:35:24.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.694+0000 7f19143b8640 1 -- 172.21.15.12:0/2888248399 wait complete. 2023-10-30T12:35:24.716 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-30T12:35:24.716 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:35:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.854+0000 7ffabcec9640 1 Processor -- start 2023-10-30T12:35:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.854+0000 7ffabcec9640 1 -- start start 2023-10-30T12:35:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffabcec9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffab81539a0 0x7ffab815a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffabcec9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffab8070a00 0x7ffab8070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:24.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffabcec9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ffab8069e90 0x7ffab806a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:24.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffabcec9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab806fbb0 con 0x7ffab8069e90 2023-10-30T12:35:24.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffabcec9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab815a8c0 con 0x7ffab8070a00 2023-10-30T12:35:24.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffabcec9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab8071380 con 0x7ffab81539a0 2023-10-30T12:35:24.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffab6d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ffab8069e90 0x7ffab806a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:24.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffab6d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ffab8069e90 0x7ffab806a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52430/0 (socket says 172.21.15.12:52430) 2023-10-30T12:35:24.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffab6d9d640 1 -- 172.21.15.12:0/3950013252 learned_addr learned my addr 172.21.15.12:0/3950013252 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:24.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffab659c640 1 --2- 172.21.15.12:0/3950013252 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffab81539a0 0x7ffab815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:24.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffaaffff640 1 --2- 172.21.15.12:0/3950013252 >> v2:172.21.15.154:3300/0 conn(0x7ffab8070a00 0x7ffab8070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:24.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffab6d9d640 1 -- 172.21.15.12:0/3950013252 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffab81539a0 msgr2=0x7ffab815a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:24.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffab6d9d640 1 --2- 172.21.15.12:0/3950013252 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffab81539a0 0x7ffab815a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffab6d9d640 1 -- 172.21.15.12:0/3950013252 >> v2:172.21.15.154:3300/0 conn(0x7ffab8070a00 msgr2=0x7ffab8070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:24.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffab6d9d640 1 --2- 172.21.15.12:0/3950013252 >> v2:172.21.15.154:3300/0 conn(0x7ffab8070a00 0x7ffab8070e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffab6d9d640 1 -- 172.21.15.12:0/3950013252 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab8153d80 con 0x7ffab8069e90 2023-10-30T12:35:24.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffab659c640 1 --2- 172.21.15.12:0/3950013252 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffab81539a0 0x7ffab815a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:24.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.858+0000 7ffaaffff640 1 --2- 172.21.15.12:0/3950013252 >> v2:172.21.15.154:3300/0 conn(0x7ffab8070a00 0x7ffab8070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.862+0000 7ffab6d9d640 1 --2- 172.21.15.12:0/3950013252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffab8069e90 0x7ffab806a300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffab806f230 tx=0x7ffab001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5e9ae41a9e32fb server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.862+0000 7ffab5d9b640 1 -- 172.21.15.12:0/3950013252 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab00a56a0 con 0x7ffab8069e90 2023-10-30T12:35:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.862+0000 7ffab5d9b640 1 -- 172.21.15.12:0/3950013252 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab00a5800 con 0x7ffab8069e90 2023-10-30T12:35:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.862+0000 7ffab5d9b640 1 -- 172.21.15.12:0/3950013252 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab00a5b90 con 0x7ffab8069e90 2023-10-30T12:35:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.862+0000 7ffabcec9640 1 -- 172.21.15.12:0/3950013252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffab8069e90 msgr2=0x7ffab806a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.862+0000 7ffabcec9640 1 --2- 172.21.15.12:0/3950013252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffab8069e90 0x7ffab806a300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffab806f230 tx=0x7ffab001c910 comp rx=0 tx=0).stop 2023-10-30T12:35:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.862+0000 7ffabcec9640 1 -- 172.21.15.12:0/3950013252 shutdown_connections 2023-10-30T12:35:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.862+0000 7ffabcec9640 1 --2- 172.21.15.12:0/3950013252 >> v2:172.21.15.154:3300/0 conn(0x7ffab8070a00 0x7ffab8070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.862+0000 7ffabcec9640 1 --2- 172.21.15.12:0/3950013252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffab8069e90 0x7ffab806a300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.862+0000 7ffabcec9640 1 --2- 172.21.15.12:0/3950013252 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffab81539a0 0x7ffab815a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.862+0000 7ffabcec9640 1 -- 172.21.15.12:0/3950013252 >> 172.21.15.12:0/3950013252 conn(0x7ffab80c7910 msgr2=0x7ffab80c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.862+0000 7ffabcec9640 1 -- 172.21.15.12:0/3950013252 shutdown_connections 2023-10-30T12:35:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.862+0000 7ffabcec9640 1 -- 172.21.15.12:0/3950013252 wait complete. 2023-10-30T12:35:24.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.862+0000 7ffabcec9640 1 Processor -- start 2023-10-30T12:35:24.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.862+0000 7ffabcec9640 1 -- start start 2023-10-30T12:35:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffabcec9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffab8069e90 0x7ffab81437b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffabcec9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffab8070a00 0x7ffab80c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffabcec9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffab81539a0 0x7ffab81f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffabcec9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab815bf20 con 0x7ffab8069e90 2023-10-30T12:35:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffabcec9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab8143320 con 0x7ffab8070a00 2023-10-30T12:35:24.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffabcec9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab81f63f0 con 0x7ffab81539a0 2023-10-30T12:35:24.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffab659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffab8069e90 0x7ffab81437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:24.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffab659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffab8069e90 0x7ffab81437b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52432/0 (socket says 172.21.15.12:52432) 2023-10-30T12:35:24.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffab659c640 1 -- 172.21.15.12:0/4095827712 learned_addr learned my addr 172.21.15.12:0/4095827712 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:24.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffaaffff640 1 --2- 172.21.15.12:0/4095827712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffab8070a00 0x7ffab80c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:24.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffab6d9d640 1 --2- 172.21.15.12:0/4095827712 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffab81539a0 0x7ffab81f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:24.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffab659c640 1 -- 172.21.15.12:0/4095827712 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffab81539a0 msgr2=0x7ffab81f5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:24.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffab659c640 1 --2- 172.21.15.12:0/4095827712 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffab81539a0 0x7ffab81f5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffab659c640 1 -- 172.21.15.12:0/4095827712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffab8070a00 msgr2=0x7ffab80c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:24.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffab659c640 1 --2- 172.21.15.12:0/4095827712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffab8070a00 0x7ffab80c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:24.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffab659c640 1 -- 172.21.15.12:0/4095827712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab0009b30 con 0x7ffab8069e90 2023-10-30T12:35:24.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffab659c640 1 --2- 172.21.15.12:0/4095827712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffab8069e90 0x7ffab81437b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800e810 tx=0x7ffaa800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:24.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffaaeffd640 1 -- 172.21.15.12:0/4095827712 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa800c870 con 0x7ffab8069e90 2023-10-30T12:35:24.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffaaeffd640 1 -- 172.21.15.12:0/4095827712 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa800c9d0 con 0x7ffab8069e90 2023-10-30T12:35:24.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffabcec9640 1 -- 172.21.15.12:0/4095827712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab8142a80 con 0x7ffab8069e90 2023-10-30T12:35:24.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffaaeffd640 1 -- 172.21.15.12:0/4095827712 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa800cd60 con 0x7ffab8069e90 2023-10-30T12:35:24.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffabcec9640 1 -- 172.21.15.12:0/4095827712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab80caa40 con 0x7ffab8069e90 2023-10-30T12:35:24.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffabcec9640 1 -- 172.21.15.12:0/4095827712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab804e170 con 0x7ffab8069e90 2023-10-30T12:35:24.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.866+0000 7ffaaeffd640 1 -- 172.21.15.12:0/4095827712 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ffaa8013410 con 0x7ffab8069e90 2023-10-30T12:35:24.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.870+0000 7ffaaeffd640 1 --2- 172.21.15.12:0/4095827712 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa980664a0 0x7ffa98068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:24.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.870+0000 7ffaaeffd640 1 -- 172.21.15.12:0/4095827712 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffaa8083b90 con 0x7ffab8069e90 2023-10-30T12:35:24.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.870+0000 7ffaaffff640 1 --2- 172.21.15.12:0/4095827712 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa980664a0 0x7ffa98068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:24.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.870+0000 7ffaaffff640 1 --2- 172.21.15.12:0/4095827712 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa980664a0 0x7ffa98068960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffab8143f00 tx=0x7ffaa0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:24.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:24.874+0000 7ffaaeffd640 1 -- 172.21.15.12:0/4095827712 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ffaa8051000 con 0x7ffab8069e90 2023-10-30T12:35:25.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.030+0000 7ffabcec9640 1 -- 172.21.15.12:0/4095827712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffab80c72f0 con 0x7ffab8069e90 2023-10-30T12:35:25.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.030+0000 7ffaaeffd640 1 -- 172.21.15.12:0/4095827712 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16505 (secure 0 0 0) 0x7ffaa8009480 con 0x7ffab8069e90 2023-10-30T12:35:25.036 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:25.036 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":22,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:35:07.153665+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:35:03.216053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:35:04.597162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:35:25.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.034+0000 7ffabcec9640 1 -- 172.21.15.12:0/4095827712 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa980664a0 msgr2=0x7ffa98068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.034+0000 7ffabcec9640 1 --2- 172.21.15.12:0/4095827712 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa980664a0 0x7ffa98068960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffab8143f00 tx=0x7ffaa0008040 comp rx=0 tx=0).stop 2023-10-30T12:35:25.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.034+0000 7ffabcec9640 1 -- 172.21.15.12:0/4095827712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffab8069e90 msgr2=0x7ffab81437b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.034+0000 7ffabcec9640 1 --2- 172.21.15.12:0/4095827712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffab8069e90 0x7ffab81437b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800e810 tx=0x7ffaa800ecd0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.034+0000 7ffabcec9640 1 -- 172.21.15.12:0/4095827712 shutdown_connections 2023-10-30T12:35:25.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.034+0000 7ffabcec9640 1 --2- 172.21.15.12:0/4095827712 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa980664a0 0x7ffa98068960 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.034+0000 7ffabcec9640 1 --2- 172.21.15.12:0/4095827712 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffab81539a0 0x7ffab81f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.034+0000 7ffabcec9640 1 --2- 172.21.15.12:0/4095827712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffab8070a00 0x7ffab80c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.034+0000 7ffabcec9640 1 --2- 172.21.15.12:0/4095827712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffab8069e90 0x7ffab81437b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.034+0000 7ffabcec9640 1 -- 172.21.15.12:0/4095827712 >> 172.21.15.12:0/4095827712 conn(0x7ffab80c7910 msgr2=0x7ffab8073480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:25.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.034+0000 7ffabcec9640 1 -- 172.21.15.12:0/4095827712 shutdown_connections 2023-10-30T12:35:25.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.034+0000 7ffabcec9640 1 -- 172.21.15.12:0/4095827712 wait complete. 2023-10-30T12:35:25.057 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:34:53.017612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-30T12:35:03.216053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-30T12:35:04.597162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:35:25.057 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:35:25.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.198+0000 7f16c6b0e640 1 Processor -- start 2023-10-30T12:35:25.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.198+0000 7f16c6b0e640 1 -- start start 2023-10-30T12:35:25.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.202+0000 7f16c6b0e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f16c00722a0 0x7f16c00bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.202+0000 7f16c6b0e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f16c00c0400 0x7f16c0156930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.202+0000 7f16c6b0e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c00c0a30 con 0x7f16c01508c0 2023-10-30T12:35:25.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.202+0000 7f16c6b0e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c00c85b0 con 0x7f16c00722a0 2023-10-30T12:35:25.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.202+0000 7f16c6b0e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c00c0850 con 0x7f16c00c0400 2023-10-30T12:35:25.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.202+0000 7f16c48aa640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f16c01508c0 0x7f16c00c6920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58074/0 (socket says 172.21.15.12:58074) 2023-10-30T12:35:25.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.202+0000 7f16c48aa640 1 -- 172.21.15.12:0/4185636722 learned_addr learned my addr 172.21.15.12:0/4185636722 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:25.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.202+0000 7f16bffff640 1 --2- 172.21.15.12:0/4185636722 >> v2:172.21.15.12:3300/0 conn(0x7f16c00722a0 0x7f16c00bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.202+0000 7f16c50ab640 1 --2- 172.21.15.12:0/4185636722 >> v2:172.21.15.154:3300/0 conn(0x7f16c00c0400 0x7f16c0156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.202+0000 7f16bf7fe640 1 -- 172.21.15.12:0/4185636722 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 627171581 0 0) 0x7f16c00c0a30 con 0x7f16c01508c0 2023-10-30T12:35:25.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.202+0000 7f16bf7fe640 1 -- 172.21.15.12:0/4185636722 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16b0003530 con 0x7f16c01508c0 2023-10-30T12:35:25.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.202+0000 7f16bffff640 1 -- 172.21.15.12:0/4185636722 >> v2:172.21.15.154:3300/0 conn(0x7f16c00c0400 msgr2=0x7f16c0156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:25.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.202+0000 7f16bffff640 1 --2- 172.21.15.12:0/4185636722 >> v2:172.21.15.154:3300/0 conn(0x7f16c00c0400 0x7f16c0156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.202+0000 7f16bffff640 1 -- 172.21.15.12:0/4185636722 >> v1:172.21.15.12:6789/0 conn(0x7f16c01508c0 legacy=0x7f16c00c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.202+0000 7f16bffff640 1 -- 172.21.15.12:0/4185636722 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c006f650 con 0x7f16c00722a0 2023-10-30T12:35:25.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.206+0000 7f16bffff640 1 --2- 172.21.15.12:0/4185636722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16c00722a0 0x7f16c00bfec0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f16c00c2140 tx=0x7f16ac01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bab29ad9605138 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:25.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.206+0000 7f16bf7fe640 1 -- 172.21.15.12:0/4185636722 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0ab600 con 0x7f16c00722a0 2023-10-30T12:35:25.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.206+0000 7f16bf7fe640 1 -- 172.21.15.12:0/4185636722 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ac0a7050 con 0x7f16c00722a0 2023-10-30T12:35:25.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.206+0000 7f16bf7fe640 1 -- 172.21.15.12:0/4185636722 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0ab980 con 0x7f16c00722a0 2023-10-30T12:35:25.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.206+0000 7f16c6b0e640 1 -- 172.21.15.12:0/4185636722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16c00722a0 msgr2=0x7f16c00bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.206+0000 7f16c6b0e640 1 --2- 172.21.15.12:0/4185636722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16c00722a0 0x7f16c00bfec0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f16c00c2140 tx=0x7f16ac01c3d0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.206+0000 7f16c6b0e640 1 -- 172.21.15.12:0/4185636722 shutdown_connections 2023-10-30T12:35:25.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.206+0000 7f16c6b0e640 1 --2- 172.21.15.12:0/4185636722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16c00722a0 0x7f16c00bfec0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.206+0000 7f16c6b0e640 1 --2- 172.21.15.12:0/4185636722 >> v2:172.21.15.154:3300/0 conn(0x7f16c00c0400 0x7f16c0156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.206+0000 7f16c6b0e640 1 -- 172.21.15.12:0/4185636722 >> 172.21.15.12:0/4185636722 conn(0x7f16c00c7910 msgr2=0x7f16c00c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:25.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.206+0000 7f16c6b0e640 1 -- 172.21.15.12:0/4185636722 shutdown_connections 2023-10-30T12:35:25.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.206+0000 7f16c6b0e640 1 -- 172.21.15.12:0/4185636722 wait complete. 2023-10-30T12:35:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.206+0000 7f16c6b0e640 1 Processor -- start 2023-10-30T12:35:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.206+0000 7f16c6b0e640 1 -- start start 2023-10-30T12:35:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c6b0e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16c00722a0 0x7f16c0154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c6b0e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16c00c0400 0x7f16c0155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c6b0e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16c01508c0 0x7f16c014bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c6b0e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c0154530 con 0x7f16c01508c0 2023-10-30T12:35:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c6b0e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c0071a50 con 0x7f16c00722a0 2023-10-30T12:35:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c6b0e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00c2210 con 0x7f16c00c0400 2023-10-30T12:35:25.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c50ab640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16c01508c0 0x7f16c014bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c50ab640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16c01508c0 0x7f16c014bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52444/0 (socket says 172.21.15.12:52444) 2023-10-30T12:35:25.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c50ab640 1 -- 172.21.15.12:0/873944412 learned_addr learned my addr 172.21.15.12:0/873944412 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:25.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c48aa640 1 --2- 172.21.15.12:0/873944412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16c00722a0 0x7f16c0154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16bffff640 1 --2- 172.21.15.12:0/873944412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16c00c0400 0x7f16c0155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c50ab640 1 -- 172.21.15.12:0/873944412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16c00c0400 msgr2=0x7f16c0155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c50ab640 1 --2- 172.21.15.12:0/873944412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16c00c0400 0x7f16c0155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c50ab640 1 -- 172.21.15.12:0/873944412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16c00722a0 msgr2=0x7f16c0154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c50ab640 1 --2- 172.21.15.12:0/873944412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16c00722a0 0x7f16c0154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c50ab640 1 -- 172.21.15.12:0/873944412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac009420 con 0x7f16c01508c0 2023-10-30T12:35:25.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c48aa640 1 --2- 172.21.15.12:0/873944412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16c00722a0 0x7f16c0154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:25.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c50ab640 1 --2- 172.21.15.12:0/873944412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16c01508c0 0x7f16c014bdb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f16b800efc0 tx=0x7f16b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:25.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16bd7fa640 1 -- 172.21.15.12:0/873944412 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b800ced0 con 0x7f16c01508c0 2023-10-30T12:35:25.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16bd7fa640 1 -- 172.21.15.12:0/873944412 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b800f040 con 0x7f16c01508c0 2023-10-30T12:35:25.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16bd7fa640 1 -- 172.21.15.12:0/873944412 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b80146c0 con 0x7f16c01508c0 2023-10-30T12:35:25.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c6b0e640 1 -- 172.21.15.12:0/873944412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c01552e0 con 0x7f16c01508c0 2023-10-30T12:35:25.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c6b0e640 1 -- 172.21.15.12:0/873944412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c00c2740 con 0x7f16c01508c0 2023-10-30T12:35:25.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16c6b0e640 1 -- 172.21.15.12:0/873944412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c0154670 con 0x7f16c01508c0 2023-10-30T12:35:25.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.210+0000 7f16bd7fa640 1 -- 172.21.15.12:0/873944412 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f16b8014820 con 0x7f16c01508c0 2023-10-30T12:35:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.214+0000 7f16bd7fa640 1 --2- 172.21.15.12:0/873944412 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f16a40664a0 0x7f16a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.214+0000 7f16bd7fa640 1 -- 172.21.15.12:0/873944412 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16b8084ab0 con 0x7f16c01508c0 2023-10-30T12:35:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.214+0000 7f16c48aa640 1 --2- 172.21.15.12:0/873944412 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f16a40664a0 0x7f16a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.214+0000 7f16c48aa640 1 --2- 172.21.15.12:0/873944412 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f16a40664a0 0x7f16a4068960 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f16b40010f0 tx=0x7f16b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.218+0000 7f16bd7fa640 1 -- 172.21.15.12:0/873944412 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f16c0154670 con 0x7f16c01508c0 2023-10-30T12:35:25.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.374+0000 7f16c6b0e640 1 -- 172.21.15.12:0/873944412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16c0154670 con 0x7f16c01508c0 2023-10-30T12:35:25.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.374+0000 7f16bd7fa640 1 -- 172.21.15.12:0/873944412 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16c0154670 con 0x7f16c01508c0 2023-10-30T12:35:25.379 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-10-30T12:35:25.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.378+0000 7f16c6b0e640 1 -- 172.21.15.12:0/873944412 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f16a40664a0 msgr2=0x7f16a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.378+0000 7f16c6b0e640 1 --2- 172.21.15.12:0/873944412 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f16a40664a0 0x7f16a4068960 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f16b40010f0 tx=0x7f16b4009040 comp rx=0 tx=0).stop 2023-10-30T12:35:25.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.378+0000 7f16c6b0e640 1 -- 172.21.15.12:0/873944412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16c01508c0 msgr2=0x7f16c014bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.378+0000 7f16c6b0e640 1 --2- 172.21.15.12:0/873944412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16c01508c0 0x7f16c014bdb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f16b800efc0 tx=0x7f16b800c490 comp rx=0 tx=0).stop 2023-10-30T12:35:25.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.378+0000 7f16c6b0e640 1 -- 172.21.15.12:0/873944412 shutdown_connections 2023-10-30T12:35:25.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.378+0000 7f16c6b0e640 1 --2- 172.21.15.12:0/873944412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16c01508c0 0x7f16c014bdb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.378+0000 7f16c6b0e640 1 --2- 172.21.15.12:0/873944412 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f16a40664a0 0x7f16a4068960 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.378+0000 7f16c6b0e640 1 --2- 172.21.15.12:0/873944412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16c00c0400 0x7f16c0155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.378+0000 7f16c6b0e640 1 --2- 172.21.15.12:0/873944412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16c00722a0 0x7f16c0154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.378+0000 7f16c6b0e640 1 -- 172.21.15.12:0/873944412 >> 172.21.15.12:0/873944412 conn(0x7f16c00c7910 msgr2=0x7f16c006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:25.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.378+0000 7f16c6b0e640 1 -- 172.21.15.12:0/873944412 shutdown_connections 2023-10-30T12:35:25.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.378+0000 7f16c6b0e640 1 -- 172.21.15.12:0/873944412 wait complete. 2023-10-30T12:35:25.400 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-30T12:35:25.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.542+0000 7f80dad0f640 1 Processor -- start 2023-10-30T12:35:25.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.542+0000 7f80dad0f640 1 -- start start 2023-10-30T12:35:25.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.542+0000 7f80dad0f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f80d41570d0 0x7f80d406afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.542+0000 7f80dad0f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80d406c9e0 0x7f80d4071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.542+0000 7f80dad0f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d406b4e0 con 0x7f80d406fdd0 2023-10-30T12:35:25.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.542+0000 7f80dad0f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d415a630 con 0x7f80d41570d0 2023-10-30T12:35:25.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.542+0000 7f80dad0f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d41599b0 con 0x7f80d406c9e0 2023-10-30T12:35:25.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.542+0000 7f80d8aab640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f80d41570d0 0x7f80d406afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.542+0000 7f80d8aab640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f80d41570d0 0x7f80d406afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52448/0 (socket says 172.21.15.12:52448) 2023-10-30T12:35:25.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.542+0000 7f80d8aab640 1 -- 172.21.15.12:0/579926382 learned_addr learned my addr 172.21.15.12:0/579926382 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:25.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.542+0000 7f80d3fff640 1 --2- 172.21.15.12:0/579926382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80d406c9e0 0x7f80d4071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.542+0000 7f80d8aab640 1 -- 172.21.15.12:0/579926382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80d406c9e0 msgr2=0x7f80d4071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:25.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.542+0000 7f80d8aab640 1 --2- 172.21.15.12:0/579926382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80d406c9e0 0x7f80d4071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.542+0000 7f80d8aab640 1 -- 172.21.15.12:0/579926382 >> v1:172.21.15.154:6789/0 conn(0x7f80d406fdd0 legacy=0x7f80d40ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.542+0000 7f80d8aab640 1 -- 172.21.15.12:0/579926382 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d4159c90 con 0x7f80d41570d0 2023-10-30T12:35:25.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.546+0000 7f80d8aab640 1 --2- 172.21.15.12:0/579926382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d41570d0 0x7f80d406afa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009870 tx=0x7f80c801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0f571942035162 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:25.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.546+0000 7f80d37fe640 1 -- 172.21.15.12:0/579926382 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c80a8a30 con 0x7f80d41570d0 2023-10-30T12:35:25.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.546+0000 7f80d37fe640 1 -- 172.21.15.12:0/579926382 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c80a8b90 con 0x7f80d41570d0 2023-10-30T12:35:25.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.546+0000 7f80d37fe640 1 -- 172.21.15.12:0/579926382 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c80b0440 con 0x7f80d41570d0 2023-10-30T12:35:25.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.546+0000 7f80dad0f640 1 -- 172.21.15.12:0/579926382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d41570d0 msgr2=0x7f80d406afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.546+0000 7f80dad0f640 1 --2- 172.21.15.12:0/579926382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d41570d0 0x7f80d406afa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009870 tx=0x7f80c801ed30 comp rx=0 tx=0).stop 2023-10-30T12:35:25.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.546+0000 7f80dad0f640 1 -- 172.21.15.12:0/579926382 shutdown_connections 2023-10-30T12:35:25.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.546+0000 7f80dad0f640 1 --2- 172.21.15.12:0/579926382 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80d406c9e0 0x7f80d4071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.546+0000 7f80dad0f640 1 --2- 172.21.15.12:0/579926382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d41570d0 0x7f80d406afa0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.546+0000 7f80dad0f640 1 -- 172.21.15.12:0/579926382 >> 172.21.15.12:0/579926382 conn(0x7f80d4069e90 msgr2=0x7f80d406a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:25.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.546+0000 7f80dad0f640 1 -- 172.21.15.12:0/579926382 shutdown_connections 2023-10-30T12:35:25.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.546+0000 7f80dad0f640 1 -- 172.21.15.12:0/579926382 wait complete. 2023-10-30T12:35:25.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80dad0f640 1 Processor -- start 2023-10-30T12:35:25.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80dad0f640 1 -- start start 2023-10-30T12:35:25.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80dad0f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80d406c9e0 0x7f80d4071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80dad0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d406fdd0 0x7f80d40cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80dad0f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80d41570d0 0x7f80d40cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80dad0f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d415cca0 con 0x7f80d406fdd0 2023-10-30T12:35:25.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80dad0f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d415b4a0 con 0x7f80d406c9e0 2023-10-30T12:35:25.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80dad0f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d415b710 con 0x7f80d41570d0 2023-10-30T12:35:25.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80d3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d406fdd0 0x7f80d40cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80d3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d406fdd0 0x7f80d40cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52452/0 (socket says 172.21.15.12:52452) 2023-10-30T12:35:25.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80d3fff640 1 -- 172.21.15.12:0/242455124 learned_addr learned my addr 172.21.15.12:0/242455124 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:25.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80d8aab640 1 --2- 172.21.15.12:0/242455124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80d406c9e0 0x7f80d4071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80d92ac640 1 --2- 172.21.15.12:0/242455124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80d41570d0 0x7f80d40cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80d3fff640 1 -- 172.21.15.12:0/242455124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80d41570d0 msgr2=0x7f80d40cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80d3fff640 1 --2- 172.21.15.12:0/242455124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80d41570d0 0x7f80d40cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80d3fff640 1 -- 172.21.15.12:0/242455124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80d406c9e0 msgr2=0x7f80d4071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80d3fff640 1 --2- 172.21.15.12:0/242455124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80d406c9e0 0x7f80d4071c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80d3fff640 1 -- 172.21.15.12:0/242455124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c80094d0 con 0x7f80d406fdd0 2023-10-30T12:35:25.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80d3fff640 1 --2- 172.21.15.12:0/242455124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d406fdd0 0x7f80d40cb8a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f80c000d7d0 tx=0x7f80c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:25.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80d17fa640 1 -- 172.21.15.12:0/242455124 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c0018070 con 0x7f80d406fdd0 2023-10-30T12:35:25.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80d17fa640 1 -- 172.21.15.12:0/242455124 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c0004020 con 0x7f80d406fdd0 2023-10-30T12:35:25.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80dad0f640 1 -- 172.21.15.12:0/242455124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d415b9f0 con 0x7f80d406fdd0 2023-10-30T12:35:25.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80d17fa640 1 -- 172.21.15.12:0/242455124 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c00041d0 con 0x7f80d406fdd0 2023-10-30T12:35:25.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.550+0000 7f80dad0f640 1 -- 172.21.15.12:0/242455124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d415bfb0 con 0x7f80d406fdd0 2023-10-30T12:35:25.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.554+0000 7f80dad0f640 1 -- 172.21.15.12:0/242455124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d404e170 con 0x7f80d406fdd0 2023-10-30T12:35:25.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.554+0000 7f80d17fa640 1 -- 172.21.15.12:0/242455124 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f80c0004330 con 0x7f80d406fdd0 2023-10-30T12:35:25.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.554+0000 7f80d17fa640 1 --2- 172.21.15.12:0/242455124 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80b80664a0 0x7f80b8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.554+0000 7f80d17fa640 1 -- 172.21.15.12:0/242455124 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80c0083da0 con 0x7f80d406fdd0 2023-10-30T12:35:25.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.554+0000 7f80d8aab640 1 --2- 172.21.15.12:0/242455124 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80b80664a0 0x7f80b8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.554+0000 7f80d8aab640 1 --2- 172.21.15.12:0/242455124 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80b80664a0 0x7f80b8068960 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f80c80b2000 tx=0x7f80c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:25.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.558+0000 7f80d17fa640 1 -- 172.21.15.12:0/242455124 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f80c00510f0 con 0x7f80d406fdd0 2023-10-30T12:35:25.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.714+0000 7f80dad0f640 1 -- 172.21.15.12:0/242455124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80d406fbb0 con 0x7f80d406fdd0 2023-10-30T12:35:25.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.714+0000 7f80d17fa640 1 -- 172.21.15.12:0/242455124 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f80c0054710 con 0x7f80d406fdd0 2023-10-30T12:35:25.720 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-10-30T12:35:25.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.718+0000 7f80dad0f640 1 -- 172.21.15.12:0/242455124 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80b80664a0 msgr2=0x7f80b8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.718+0000 7f80dad0f640 1 --2- 172.21.15.12:0/242455124 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80b80664a0 0x7f80b8068960 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f80c80b2000 tx=0x7f80c8009040 comp rx=0 tx=0).stop 2023-10-30T12:35:25.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.718+0000 7f80dad0f640 1 -- 172.21.15.12:0/242455124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d406fdd0 msgr2=0x7f80d40cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.718+0000 7f80dad0f640 1 --2- 172.21.15.12:0/242455124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d406fdd0 0x7f80d40cb8a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f80c000d7d0 tx=0x7f80c000dc90 comp rx=0 tx=0).stop 2023-10-30T12:35:25.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.718+0000 7f80dad0f640 1 -- 172.21.15.12:0/242455124 shutdown_connections 2023-10-30T12:35:25.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.718+0000 7f80dad0f640 1 --2- 172.21.15.12:0/242455124 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80b80664a0 0x7f80b8068960 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.718+0000 7f80dad0f640 1 --2- 172.21.15.12:0/242455124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80d41570d0 0x7f80d40cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.718+0000 7f80dad0f640 1 --2- 172.21.15.12:0/242455124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d406fdd0 0x7f80d40cb8a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.718+0000 7f80dad0f640 1 --2- 172.21.15.12:0/242455124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80d406c9e0 0x7f80d4071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.718+0000 7f80dad0f640 1 -- 172.21.15.12:0/242455124 >> 172.21.15.12:0/242455124 conn(0x7f80d4069e90 msgr2=0x7f80d40c9240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:25.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.718+0000 7f80dad0f640 1 -- 172.21.15.12:0/242455124 shutdown_connections 2023-10-30T12:35:25.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.718+0000 7f80dad0f640 1 -- 172.21.15.12:0/242455124 wait complete. 2023-10-30T12:35:25.741 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-30T12:35:25.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.882+0000 7f0a1f29e640 1 Processor -- start 2023-10-30T12:35:25.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.882+0000 7f0a1f29e640 1 -- start start 2023-10-30T12:35:25.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1f29e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a18153980 0x7f0a180caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1f29e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0a18159930 0x7f0a180cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1f29e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0a180cf1a0 0x7f0a180cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1f29e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1815a920 con 0x7f0a18159930 2023-10-30T12:35:25.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1f29e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a180cb930 con 0x7f0a180cf1a0 2023-10-30T12:35:25.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1f29e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a18159dc0 con 0x7f0a18153980 2023-10-30T12:35:25.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1c839640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0a18159930 0x7f0a180cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1c839640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0a18159930 0x7f0a180cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52468/0 (socket says 172.21.15.12:52468) 2023-10-30T12:35:25.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1c839640 1 -- 172.21.15.12:0/1560570877 learned_addr learned my addr 172.21.15.12:0/1560570877 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1d03a640 1 --2- 172.21.15.12:0/1560570877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a18153980 0x7f0a180caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1d83b640 1 --2- 172.21.15.12:0/1560570877 >> v2:172.21.15.154:3300/0 conn(0x7f0a180cf1a0 0x7f0a180cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1c839640 1 -- 172.21.15.12:0/1560570877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a18153980 msgr2=0x7f0a180caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1c839640 1 --2- 172.21.15.12:0/1560570877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a18153980 0x7f0a180caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1c839640 1 -- 172.21.15.12:0/1560570877 >> v2:172.21.15.154:3300/0 conn(0x7f0a180cf1a0 msgr2=0x7f0a180cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1c839640 1 --2- 172.21.15.12:0/1560570877 >> v2:172.21.15.154:3300/0 conn(0x7f0a180cf1a0 0x7f0a180cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1c839640 1 -- 172.21.15.12:0/1560570877 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1815a080 con 0x7f0a18159930 2023-10-30T12:35:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.886+0000 7f0a1d83b640 1 --2- 172.21.15.12:0/1560570877 >> v2:172.21.15.154:3300/0 conn(0x7f0a180cf1a0 0x7f0a180cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:25.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.890+0000 7f0a1c839640 1 --2- 172.21.15.12:0/1560570877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a18159930 0x7f0a180cb1e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a18072820 tx=0x7f0a0801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ca301971105dc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:25.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.890+0000 7f0a0ffff640 1 -- 172.21.15.12:0/1560570877 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a080aa5f0 con 0x7f0a18159930 2023-10-30T12:35:25.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.890+0000 7f0a0ffff640 1 -- 172.21.15.12:0/1560570877 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a080aa750 con 0x7f0a18159930 2023-10-30T12:35:25.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.890+0000 7f0a0ffff640 1 -- 172.21.15.12:0/1560570877 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a080aaae0 con 0x7f0a18159930 2023-10-30T12:35:25.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.890+0000 7f0a1f29e640 1 -- 172.21.15.12:0/1560570877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a18159930 msgr2=0x7f0a180cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.890+0000 7f0a1f29e640 1 --2- 172.21.15.12:0/1560570877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a18159930 0x7f0a180cb1e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a18072820 tx=0x7f0a0801c4d0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.890+0000 7f0a1f29e640 1 -- 172.21.15.12:0/1560570877 shutdown_connections 2023-10-30T12:35:25.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.890+0000 7f0a1f29e640 1 --2- 172.21.15.12:0/1560570877 >> v2:172.21.15.154:3300/0 conn(0x7f0a180cf1a0 0x7f0a180cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.890+0000 7f0a1f29e640 1 --2- 172.21.15.12:0/1560570877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a18159930 0x7f0a180cb1e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.890+0000 7f0a1f29e640 1 --2- 172.21.15.12:0/1560570877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a18153980 0x7f0a180caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.890+0000 7f0a1f29e640 1 -- 172.21.15.12:0/1560570877 >> 172.21.15.12:0/1560570877 conn(0x7f0a1806c0f0 msgr2=0x7f0a1806c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:25.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.890+0000 7f0a1f29e640 1 -- 172.21.15.12:0/1560570877 shutdown_connections 2023-10-30T12:35:25.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.890+0000 7f0a1f29e640 1 -- 172.21.15.12:0/1560570877 wait complete. 2023-10-30T12:35:25.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.890+0000 7f0a1f29e640 1 Processor -- start 2023-10-30T12:35:25.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.890+0000 7f0a1f29e640 1 -- start start 2023-10-30T12:35:25.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1f29e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a180cf1a0 0x7f0a18140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1f29e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a18153980 0x7f0a1813d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1f29e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a181f52f0 0x7f0a1813d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1f29e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a181418b0 con 0x7f0a180cf1a0 2023-10-30T12:35:25.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1f29e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a18141b90 con 0x7f0a18153980 2023-10-30T12:35:25.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1f29e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a18141e70 con 0x7f0a181f52f0 2023-10-30T12:35:25.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1d03a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a180cf1a0 0x7f0a18140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1d03a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a180cf1a0 0x7f0a18140be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52472/0 (socket says 172.21.15.12:52472) 2023-10-30T12:35:25.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1d03a640 1 -- 172.21.15.12:0/2221643387 learned_addr learned my addr 172.21.15.12:0/2221643387 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:25.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1c839640 1 --2- 172.21.15.12:0/2221643387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a18153980 0x7f0a1813d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1d83b640 1 --2- 172.21.15.12:0/2221643387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a181f52f0 0x7f0a1813d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1d03a640 1 -- 172.21.15.12:0/2221643387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a181f52f0 msgr2=0x7f0a1813d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1d03a640 1 --2- 172.21.15.12:0/2221643387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a181f52f0 0x7f0a1813d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1d03a640 1 -- 172.21.15.12:0/2221643387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a18153980 msgr2=0x7f0a1813d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:25.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1d03a640 1 --2- 172.21.15.12:0/2221643387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a18153980 0x7f0a1813d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:25.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1d03a640 1 -- 172.21.15.12:0/2221643387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a080094d0 con 0x7f0a180cf1a0 2023-10-30T12:35:25.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1d83b640 1 --2- 172.21.15.12:0/2221643387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a181f52f0 0x7f0a1813d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:35:25.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1d03a640 1 --2- 172.21.15.12:0/2221643387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a180cf1a0 0x7f0a18140be0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400e930 tx=0x7f0a1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:25.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a0dffb640 1 -- 172.21.15.12:0/2221643387 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14009690 con 0x7f0a180cf1a0 2023-10-30T12:35:25.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a0dffb640 1 -- 172.21.15.12:0/2221643387 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1400ca00 con 0x7f0a180cf1a0 2023-10-30T12:35:25.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1f29e640 1 -- 172.21.15.12:0/2221643387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a18142150 con 0x7f0a180cf1a0 2023-10-30T12:35:25.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a0dffb640 1 -- 172.21.15.12:0/2221643387 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1400cd70 con 0x7f0a180cf1a0 2023-10-30T12:35:25.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1f29e640 1 -- 172.21.15.12:0/2221643387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a18142710 con 0x7f0a180cf1a0 2023-10-30T12:35:25.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a1f29e640 1 -- 172.21.15.12:0/2221643387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1804e170 con 0x7f0a180cf1a0 2023-10-30T12:35:25.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.894+0000 7f0a0dffb640 1 -- 172.21.15.12:0/2221643387 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0a14013470 con 0x7f0a180cf1a0 2023-10-30T12:35:25.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.898+0000 7f0a0dffb640 1 --2- 172.21.15.12:0/2221643387 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f09fc0664a0 0x7f09fc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:25.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.898+0000 7f0a0dffb640 1 -- 172.21.15.12:0/2221643387 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a140841b0 con 0x7f0a180cf1a0 2023-10-30T12:35:25.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.898+0000 7f0a1c839640 1 --2- 172.21.15.12:0/2221643387 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f09fc0664a0 0x7f09fc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:25.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.898+0000 7f0a1c839640 1 --2- 172.21.15.12:0/2221643387 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f09fc0664a0 0x7f09fc068960 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0a080ae000 tx=0x7f0a080a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:25.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:25.902+0000 7f0a0dffb640 1 -- 172.21.15.12:0/2221643387 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0a14051650 con 0x7f0a180cf1a0 2023-10-30T12:35:26.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.058+0000 7f0a1f29e640 1 -- 172.21.15.12:0/2221643387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a180c5ff0 con 0x7f0a180cf1a0 2023-10-30T12:35:26.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.058+0000 7f0a0dffb640 1 -- 172.21.15.12:0/2221643387 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a14054c70 con 0x7f0a180cf1a0 2023-10-30T12:35:26.061 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-10-30T12:35:26.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.058+0000 7f0a1f29e640 1 -- 172.21.15.12:0/2221643387 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f09fc0664a0 msgr2=0x7f09fc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.058+0000 7f0a1f29e640 1 --2- 172.21.15.12:0/2221643387 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f09fc0664a0 0x7f09fc068960 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0a080ae000 tx=0x7f0a080a7000 comp rx=0 tx=0).stop 2023-10-30T12:35:26.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.058+0000 7f0a1f29e640 1 -- 172.21.15.12:0/2221643387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a180cf1a0 msgr2=0x7f0a18140be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.058+0000 7f0a1f29e640 1 --2- 172.21.15.12:0/2221643387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a180cf1a0 0x7f0a18140be0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400e930 tx=0x7f0a1400edf0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.058+0000 7f0a1f29e640 1 -- 172.21.15.12:0/2221643387 shutdown_connections 2023-10-30T12:35:26.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.058+0000 7f0a1f29e640 1 --2- 172.21.15.12:0/2221643387 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f09fc0664a0 0x7f09fc068960 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.058+0000 7f0a1f29e640 1 --2- 172.21.15.12:0/2221643387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a181f52f0 0x7f0a1813d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.058+0000 7f0a1f29e640 1 --2- 172.21.15.12:0/2221643387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a18153980 0x7f0a1813d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.058+0000 7f0a1f29e640 1 --2- 172.21.15.12:0/2221643387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a180cf1a0 0x7f0a18140be0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.058+0000 7f0a1f29e640 1 -- 172.21.15.12:0/2221643387 >> 172.21.15.12:0/2221643387 conn(0x7f0a1806c0f0 msgr2=0x7f0a180c9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.058+0000 7f0a1f29e640 1 -- 172.21.15.12:0/2221643387 shutdown_connections 2023-10-30T12:35:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.058+0000 7f0a1f29e640 1 -- 172.21.15.12:0/2221643387 wait complete. 2023-10-30T12:35:26.082 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:35:26.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.222+0000 7fd94aa68640 1 Processor -- start 2023-10-30T12:35:26.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.222+0000 7fd94aa68640 1 -- start start 2023-10-30T12:35:26.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.222+0000 7fd94aa68640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9440c5f60 0x7fd944071260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:26.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.222+0000 7fd94aa68640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9440700b0 con 0x7fd944153bf0 2023-10-30T12:35:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.222+0000 7fd94aa68640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9440c6530 con 0x7fd9440c6fe0 2023-10-30T12:35:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.222+0000 7fd94aa68640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9440c6810 con 0x7fd9440c5f60 2023-10-30T12:35:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.222+0000 7fd948804640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd944153bf0 0x7fd9440c74a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58084/0 (socket says 172.21.15.12:58084) 2023-10-30T12:35:26.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.222+0000 7fd948804640 1 -- 172.21.15.12:0/3162092501 learned_addr learned my addr 172.21.15.12:0/3162092501 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:26.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.222+0000 7fd949005640 1 --2- 172.21.15.12:0/3162092501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9440c5f60 0x7fd944071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:26.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.222+0000 7fd9437fe640 1 -- 172.21.15.12:0/3162092501 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643819185 0 0) 0x7fd9440700b0 con 0x7fd944153bf0 2023-10-30T12:35:26.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.222+0000 7fd9437fe640 1 -- 172.21.15.12:0/3162092501 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd934003530 con 0x7fd944153bf0 2023-10-30T12:35:26.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.222+0000 7fd9437fe640 1 -- 172.21.15.12:0/3162092501 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392566307 0 0) 0x7fd9440c6530 con 0x7fd9440c6fe0 2023-10-30T12:35:26.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.222+0000 7fd9437fe640 1 -- 172.21.15.12:0/3162092501 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9440700b0 con 0x7fd9440c6fe0 2023-10-30T12:35:26.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.226+0000 7fd949005640 1 -- 172.21.15.12:0/3162092501 >> v1:172.21.15.154:6789/0 conn(0x7fd9440c6fe0 legacy=0x7fd9440cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.226+0000 7fd9437fe640 1 -- 172.21.15.12:0/3162092501 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1853854544 0 0) 0x7fd934003530 con 0x7fd944153bf0 2023-10-30T12:35:26.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.226+0000 7fd949005640 1 -- 172.21.15.12:0/3162092501 >> v1:172.21.15.12:6789/0 conn(0x7fd944153bf0 legacy=0x7fd9440c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.226+0000 7fd949005640 1 -- 172.21.15.12:0/3162092501 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd944159b10 con 0x7fd9440c5f60 2023-10-30T12:35:26.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.226+0000 7fd949005640 1 --2- 172.21.15.12:0/3162092501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9440c5f60 0x7fd944071260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009dc0 tx=0x7fd93c01f080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=809baecad774757f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:26.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.226+0000 7fd9437fe640 1 -- 172.21.15.12:0/3162092501 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd938001680 con 0x7fd9440c5f60 2023-10-30T12:35:26.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.226+0000 7fd9437fe640 1 -- 172.21.15.12:0/3162092501 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c0a8c10 con 0x7fd9440c5f60 2023-10-30T12:35:26.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.226+0000 7fd9437fe640 1 -- 172.21.15.12:0/3162092501 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0b0440 con 0x7fd9440c5f60 2023-10-30T12:35:26.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd94aa68640 1 -- 172.21.15.12:0/3162092501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9440c5f60 msgr2=0x7fd944071260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd94aa68640 1 --2- 172.21.15.12:0/3162092501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9440c5f60 0x7fd944071260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009dc0 tx=0x7fd93c01f080 comp rx=0 tx=0).stop 2023-10-30T12:35:26.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd94aa68640 1 -- 172.21.15.12:0/3162092501 shutdown_connections 2023-10-30T12:35:26.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd94aa68640 1 --2- 172.21.15.12:0/3162092501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9440c5f60 0x7fd944071260 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd94aa68640 1 -- 172.21.15.12:0/3162092501 >> 172.21.15.12:0/3162092501 conn(0x7fd94406c6e0 msgr2=0x7fd944069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:26.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd94aa68640 1 -- 172.21.15.12:0/3162092501 shutdown_connections 2023-10-30T12:35:26.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd94aa68640 1 -- 172.21.15.12:0/3162092501 wait complete. 2023-10-30T12:35:26.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd94aa68640 1 Processor -- start 2023-10-30T12:35:26.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd94aa68640 1 -- start start 2023-10-30T12:35:26.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd94aa68640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9440c5f60 0x7fd9441515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:26.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd94aa68640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9440c6fe0 0x7fd9441560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:26.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd94aa68640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd944153bf0 0x7fd9441565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:26.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd94aa68640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd944155c10 con 0x7fd944153bf0 2023-10-30T12:35:26.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd94aa68640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd944155070 con 0x7fd9440c6fe0 2023-10-30T12:35:26.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd94aa68640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9441efea0 con 0x7fd9440c5f60 2023-10-30T12:35:26.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd949005640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd944153bf0 0x7fd9441565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:26.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd949005640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd944153bf0 0x7fd9441565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52476/0 (socket says 172.21.15.12:52476) 2023-10-30T12:35:26.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd949005640 1 -- 172.21.15.12:0/3114128138 learned_addr learned my addr 172.21.15.12:0/3114128138 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:26.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd948804640 1 --2- 172.21.15.12:0/3114128138 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9440c5f60 0x7fd9441515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:26.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd943fff640 1 --2- 172.21.15.12:0/3114128138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9440c6fe0 0x7fd9441560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:26.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd949005640 1 -- 172.21.15.12:0/3114128138 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9440c5f60 msgr2=0x7fd9441515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd949005640 1 --2- 172.21.15.12:0/3114128138 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9440c5f60 0x7fd9441515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd949005640 1 -- 172.21.15.12:0/3114128138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9440c6fe0 msgr2=0x7fd9441560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd949005640 1 --2- 172.21.15.12:0/3114128138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9440c6fe0 0x7fd9441560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.230+0000 7fd949005640 1 -- 172.21.15.12:0/3114128138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c009a70 con 0x7fd944153bf0 2023-10-30T12:35:26.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.234+0000 7fd949005640 1 --2- 172.21.15.12:0/3114128138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd944153bf0 0x7fd9441565e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009eb0 tx=0x7fd93c01be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:26.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.234+0000 7fd9417fa640 1 -- 172.21.15.12:0/3114128138 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0b0e30 con 0x7fd944153bf0 2023-10-30T12:35:26.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.234+0000 7fd9417fa640 1 -- 172.21.15.12:0/3114128138 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c0bc450 con 0x7fd944153bf0 2023-10-30T12:35:26.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.234+0000 7fd94aa68640 1 -- 172.21.15.12:0/3114128138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9441552e0 con 0x7fd944153bf0 2023-10-30T12:35:26.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.234+0000 7fd9417fa640 1 -- 172.21.15.12:0/3114128138 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0bc7c0 con 0x7fd944153bf0 2023-10-30T12:35:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.234+0000 7fd94aa68640 1 -- 172.21.15.12:0/3114128138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9441558a0 con 0x7fd944153bf0 2023-10-30T12:35:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.234+0000 7fd94aa68640 1 -- 172.21.15.12:0/3114128138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd944155d80 con 0x7fd944153bf0 2023-10-30T12:35:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.234+0000 7fd9417fa640 1 -- 172.21.15.12:0/3114128138 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd93c0bc920 con 0x7fd944153bf0 2023-10-30T12:35:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.234+0000 7fd9417fa640 1 --2- 172.21.15.12:0/3114128138 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd9280664a0 0x7fd928068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.234+0000 7fd9417fa640 1 -- 172.21.15.12:0/3114128138 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd93c122960 con 0x7fd944153bf0 2023-10-30T12:35:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.234+0000 7fd948804640 1 --2- 172.21.15.12:0/3114128138 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd9280664a0 0x7fd928068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.238+0000 7fd948804640 1 --2- 172.21.15.12:0/3114128138 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd9280664a0 0x7fd928068960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd938000f70 tx=0x7fd938009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.242+0000 7fd9417fa640 1 -- 172.21.15.12:0/3114128138 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd93c0efd40 con 0x7fd944153bf0 2023-10-30T12:35:26.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.430+0000 7fd94aa68640 1 -- 172.21.15.12:0/3114128138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd94404e170 con 0x7fd944153bf0 2023-10-30T12:35:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.430+0000 7fd9417fa640 1 -- 172.21.15.12:0/3114128138 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6525 (secure 0 0 0) 0x7fd93c0b5090 con 0x7fd944153bf0 2023-10-30T12:35:26.434 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:26.435 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:35:22.316922+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:creating","state_seq":1,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:35:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 12 2023-10-30T12:35:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.434+0000 7fd94aa68640 1 -- 172.21.15.12:0/3114128138 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd9280664a0 msgr2=0x7fd928068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.434+0000 7fd94aa68640 1 --2- 172.21.15.12:0/3114128138 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd9280664a0 0x7fd928068960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd938000f70 tx=0x7fd938009040 comp rx=0 tx=0).stop 2023-10-30T12:35:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.434+0000 7fd94aa68640 1 -- 172.21.15.12:0/3114128138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd944153bf0 msgr2=0x7fd9441565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.434+0000 7fd94aa68640 1 --2- 172.21.15.12:0/3114128138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd944153bf0 0x7fd9441565e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009eb0 tx=0x7fd93c01be20 comp rx=0 tx=0).stop 2023-10-30T12:35:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.434+0000 7fd94aa68640 1 -- 172.21.15.12:0/3114128138 shutdown_connections 2023-10-30T12:35:26.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.434+0000 7fd94aa68640 1 --2- 172.21.15.12:0/3114128138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd944153bf0 0x7fd9441565e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.434+0000 7fd94aa68640 1 --2- 172.21.15.12:0/3114128138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9440c6fe0 0x7fd9441560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.434+0000 7fd94aa68640 1 --2- 172.21.15.12:0/3114128138 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd9280664a0 0x7fd928068960 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.434+0000 7fd94aa68640 1 --2- 172.21.15.12:0/3114128138 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9440c5f60 0x7fd9441515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.434+0000 7fd94aa68640 1 -- 172.21.15.12:0/3114128138 >> 172.21.15.12:0/3114128138 conn(0x7fd94406c6e0 msgr2=0x7fd944150810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:26.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.434+0000 7fd94aa68640 1 -- 172.21.15.12:0/3114128138 shutdown_connections 2023-10-30T12:35:26.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.434+0000 7fd94aa68640 1 -- 172.21.15.12:0/3114128138 wait complete. 2023-10-30T12:35:26.457 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:35:26.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.598+0000 7fc44dc05640 1 Processor -- start 2023-10-30T12:35:26.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.598+0000 7fc44dc05640 1 -- start start 2023-10-30T12:35:26.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.598+0000 7fc44dc05640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc4481538a0 0x7fc44815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.598+0000 7fc44dc05640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4480cb800 con 0x7fc4480ca0e0 2023-10-30T12:35:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.598+0000 7fc44dc05640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44815dc30 con 0x7fc4480cf160 2023-10-30T12:35:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.598+0000 7fc44dc05640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4480cb330 con 0x7fc4481538a0 2023-10-30T12:35:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.598+0000 7fc4477fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc4481538a0 0x7fc44815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.598+0000 7fc4477fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc4481538a0 0x7fc44815d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52480/0 (socket says 172.21.15.12:52480) 2023-10-30T12:35:26.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.598+0000 7fc4477fe640 1 -- 172.21.15.12:0/1792298867 learned_addr learned my addr 172.21.15.12:0/1792298867 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:26.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.598+0000 7fc4477fe640 1 -- 172.21.15.12:0/1792298867 >> v1:172.21.15.154:6789/0 conn(0x7fc4480cf160 legacy=0x7fc448164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.598+0000 7fc4477fe640 1 -- 172.21.15.12:0/1792298867 >> v1:172.21.15.12:6789/0 conn(0x7fc4480ca0e0 legacy=0x7fc44815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.598+0000 7fc4477fe640 1 -- 172.21.15.12:0/1792298867 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4480cad70 con 0x7fc4481538a0 2023-10-30T12:35:26.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.598+0000 7fc4467fc640 1 -- 172.21.15.12:0/1792298867 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799382643 0 0) 0x7fc4480cb800 con 0x7fc4480ca0e0 2023-10-30T12:35:26.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.602+0000 7fc4477fe640 1 --2- 172.21.15.12:0/1792298867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4481538a0 0x7fc44815d6f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc438009870 tx=0x7fc43801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98dd3dbde68e7b44 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:26.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.602+0000 7fc4467fc640 1 -- 172.21.15.12:0/1792298867 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4380a8a30 con 0x7fc4481538a0 2023-10-30T12:35:26.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.602+0000 7fc4467fc640 1 -- 172.21.15.12:0/1792298867 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4380a8b90 con 0x7fc4481538a0 2023-10-30T12:35:26.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.602+0000 7fc4467fc640 1 -- 172.21.15.12:0/1792298867 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4380b0440 con 0x7fc4481538a0 2023-10-30T12:35:26.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.602+0000 7fc44dc05640 1 -- 172.21.15.12:0/1792298867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4481538a0 msgr2=0x7fc44815d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.602+0000 7fc44dc05640 1 --2- 172.21.15.12:0/1792298867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4481538a0 0x7fc44815d6f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc438009870 tx=0x7fc43801ed30 comp rx=0 tx=0).stop 2023-10-30T12:35:26.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.602+0000 7fc44dc05640 1 -- 172.21.15.12:0/1792298867 shutdown_connections 2023-10-30T12:35:26.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.602+0000 7fc44dc05640 1 --2- 172.21.15.12:0/1792298867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4481538a0 0x7fc44815d6f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.602+0000 7fc44dc05640 1 -- 172.21.15.12:0/1792298867 >> 172.21.15.12:0/1792298867 conn(0x7fc44806bfe0 msgr2=0x7fc44806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.602+0000 7fc44dc05640 1 -- 172.21.15.12:0/1792298867 shutdown_connections 2023-10-30T12:35:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.602+0000 7fc44dc05640 1 -- 172.21.15.12:0/1792298867 wait complete. 2023-10-30T12:35:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc44dc05640 1 Processor -- start 2023-10-30T12:35:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc44dc05640 1 -- start start 2023-10-30T12:35:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc44dc05640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480ca0e0 0x7fc4480b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc44dc05640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4480cf160 0x7fc448148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc44dc05640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4481538a0 0x7fc4481448b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc44dc05640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4481e5480 con 0x7fc4480ca0e0 2023-10-30T12:35:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc44dc05640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480b4d10 con 0x7fc4481538a0 2023-10-30T12:35:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc44dc05640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc448144010 con 0x7fc4480cf160 2023-10-30T12:35:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc4477fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480ca0e0 0x7fc4480b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc4477fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480ca0e0 0x7fc4480b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52486/0 (socket says 172.21.15.12:52486) 2023-10-30T12:35:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc4477fe640 1 -- 172.21.15.12:0/1293884754 learned_addr learned my addr 172.21.15.12:0/1293884754 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc447fff640 1 --2- 172.21.15.12:0/1293884754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4481538a0 0x7fc4481448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:26.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc446ffd640 1 --2- 172.21.15.12:0/1293884754 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4480cf160 0x7fc448148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:26.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc4477fe640 1 -- 172.21.15.12:0/1293884754 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4480cf160 msgr2=0x7fc448148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc4477fe640 1 --2- 172.21.15.12:0/1293884754 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4480cf160 0x7fc448148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc4477fe640 1 -- 172.21.15.12:0/1293884754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4481538a0 msgr2=0x7fc4481448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc4477fe640 1 --2- 172.21.15.12:0/1293884754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4481538a0 0x7fc4481448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc4477fe640 1 -- 172.21.15.12:0/1293884754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4380094d0 con 0x7fc4480ca0e0 2023-10-30T12:35:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc4477fe640 1 --2- 172.21.15.12:0/1293884754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480ca0e0 0x7fc4480b47d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc4380b3040 tx=0x7fc43801bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc42ffff640 1 -- 172.21.15.12:0/1293884754 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4380a8a30 con 0x7fc4480ca0e0 2023-10-30T12:35:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc42ffff640 1 -- 172.21.15.12:0/1293884754 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4380bb480 con 0x7fc4480ca0e0 2023-10-30T12:35:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc42ffff640 1 -- 172.21.15.12:0/1293884754 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4380bb850 con 0x7fc4480ca0e0 2023-10-30T12:35:26.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc44dc05640 1 -- 172.21.15.12:0/1293884754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc448144420 con 0x7fc4480ca0e0 2023-10-30T12:35:26.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.606+0000 7fc44dc05640 1 -- 172.21.15.12:0/1293884754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4481e49e0 con 0x7fc4480ca0e0 2023-10-30T12:35:26.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.610+0000 7fc44dc05640 1 -- 172.21.15.12:0/1293884754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44804e170 con 0x7fc4480ca0e0 2023-10-30T12:35:26.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.610+0000 7fc42ffff640 1 -- 172.21.15.12:0/1293884754 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc4380bb9b0 con 0x7fc4480ca0e0 2023-10-30T12:35:26.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.610+0000 7fc42ffff640 1 --2- 172.21.15.12:0/1293884754 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4340664a0 0x7fc434068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:26.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.610+0000 7fc42ffff640 1 -- 172.21.15.12:0/1293884754 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc438122410 con 0x7fc4480ca0e0 2023-10-30T12:35:26.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.610+0000 7fc446ffd640 1 --2- 172.21.15.12:0/1293884754 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4340664a0 0x7fc434068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:26.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.610+0000 7fc446ffd640 1 --2- 172.21.15.12:0/1293884754 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4340664a0 0x7fc434068960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc448151060 tx=0x7fc43c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:26.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.614+0000 7fc42ffff640 1 -- 172.21.15.12:0/1293884754 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc4380ef920 con 0x7fc4480ca0e0 2023-10-30T12:35:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.770+0000 7fc44dc05640 1 -- 172.21.15.12:0/1293884754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4480666a0 con 0x7fc4480ca0e0 2023-10-30T12:35:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.770+0000 7fc42ffff640 1 -- 172.21.15.12:0/1293884754 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16505 (secure 0 0 0) 0x7fc4380f2f40 con 0x7fc4480ca0e0 2023-10-30T12:35:26.777 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:26.777 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":22,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:35:07.153665+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:35:03.216053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:35:04.597162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:35:26.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.774+0000 7fc44dc05640 1 -- 172.21.15.12:0/1293884754 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4340664a0 msgr2=0x7fc434068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.774+0000 7fc44dc05640 1 --2- 172.21.15.12:0/1293884754 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4340664a0 0x7fc434068960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc448151060 tx=0x7fc43c009040 comp rx=0 tx=0).stop 2023-10-30T12:35:26.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.774+0000 7fc44dc05640 1 -- 172.21.15.12:0/1293884754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480ca0e0 msgr2=0x7fc4480b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.774+0000 7fc44dc05640 1 --2- 172.21.15.12:0/1293884754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480ca0e0 0x7fc4480b47d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc4380b3040 tx=0x7fc43801bad0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.774+0000 7fc44dc05640 1 -- 172.21.15.12:0/1293884754 shutdown_connections 2023-10-30T12:35:26.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.774+0000 7fc44dc05640 1 --2- 172.21.15.12:0/1293884754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4481538a0 0x7fc4481448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.774+0000 7fc44dc05640 1 --2- 172.21.15.12:0/1293884754 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4340664a0 0x7fc434068960 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.774+0000 7fc44dc05640 1 --2- 172.21.15.12:0/1293884754 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4480cf160 0x7fc448148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.774+0000 7fc44dc05640 1 --2- 172.21.15.12:0/1293884754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480ca0e0 0x7fc4480b47d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.774+0000 7fc44dc05640 1 -- 172.21.15.12:0/1293884754 >> 172.21.15.12:0/1293884754 conn(0x7fc44806bfe0 msgr2=0x7fc448142980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:26.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.774+0000 7fc44dc05640 1 -- 172.21.15.12:0/1293884754 shutdown_connections 2023-10-30T12:35:26.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.774+0000 7fc44dc05640 1 -- 172.21.15.12:0/1293884754 wait complete. 2023-10-30T12:35:26.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:34:53.017612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-30T12:35:03.216053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-30T12:35:04.597162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:35:26.798 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:35:26.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.942+0000 7f31fac09640 1 Processor -- start 2023-10-30T12:35:26.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.942+0000 7f31fac09640 1 -- start start 2023-10-30T12:35:26.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.942+0000 7f31fac09640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f31f41508c0 0x7f31f40c6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:26.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.942+0000 7f31fac09640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f40c0a30 con 0x7f31f40722a0 2023-10-30T12:35:26.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.942+0000 7f31fac09640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f406f650 con 0x7f31f40c6e60 2023-10-30T12:35:26.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.942+0000 7f31fac09640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f40c85b0 con 0x7f31f41508c0 2023-10-30T12:35:26.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.942+0000 7f31f89a5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f31f41508c0 0x7f31f40c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:26.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.942+0000 7f31f3fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f31f40722a0 0x7f31f40bfec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58104/0 (socket says 172.21.15.12:58104) 2023-10-30T12:35:26.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.942+0000 7f31f89a5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f31f41508c0 0x7f31f40c6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52502/0 (socket says 172.21.15.12:52502) 2023-10-30T12:35:26.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.942+0000 7f31f3fff640 1 -- 172.21.15.12:0/1325361930 learned_addr learned my addr 172.21.15.12:0/1325361930 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:26.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.942+0000 7f31f37fe640 1 -- 172.21.15.12:0/1325361930 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218286525 0 0) 0x7f31f40c0a30 con 0x7f31f40722a0 2023-10-30T12:35:26.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.942+0000 7f31f89a5640 1 -- 172.21.15.12:0/1325361930 >> v1:172.21.15.154:6789/0 conn(0x7f31f40c6e60 legacy=0x7f31f4156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.942+0000 7f31f89a5640 1 -- 172.21.15.12:0/1325361930 >> v1:172.21.15.12:6789/0 conn(0x7f31f40722a0 legacy=0x7f31f40bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.942+0000 7f31f89a5640 1 -- 172.21.15.12:0/1325361930 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f406f930 con 0x7f31f41508c0 2023-10-30T12:35:26.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.946+0000 7f31f89a5640 1 --2- 172.21.15.12:0/1325361930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f41508c0 0x7f31f40c6920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f31e800ba40 tx=0x7f31e801eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77220d7f5cc1e7cd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:26.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.946+0000 7f31f37fe640 1 -- 172.21.15.12:0/1325361930 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e80a7870 con 0x7f31f41508c0 2023-10-30T12:35:26.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.946+0000 7f31f37fe640 1 -- 172.21.15.12:0/1325361930 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e80a79d0 con 0x7f31f41508c0 2023-10-30T12:35:26.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.946+0000 7f31f37fe640 1 -- 172.21.15.12:0/1325361930 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e80a7d60 con 0x7f31f41508c0 2023-10-30T12:35:26.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.946+0000 7f31fac09640 1 -- 172.21.15.12:0/1325361930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f41508c0 msgr2=0x7f31f40c6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.946+0000 7f31fac09640 1 --2- 172.21.15.12:0/1325361930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f41508c0 0x7f31f40c6920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f31e800ba40 tx=0x7f31e801eaa0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.946+0000 7f31fac09640 1 -- 172.21.15.12:0/1325361930 shutdown_connections 2023-10-30T12:35:26.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.946+0000 7f31fac09640 1 --2- 172.21.15.12:0/1325361930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f41508c0 0x7f31f40c6920 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.946+0000 7f31fac09640 1 -- 172.21.15.12:0/1325361930 >> 172.21.15.12:0/1325361930 conn(0x7f31f40c7910 msgr2=0x7f31f40c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:26.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.946+0000 7f31fac09640 1 -- 172.21.15.12:0/1325361930 shutdown_connections 2023-10-30T12:35:26.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.946+0000 7f31fac09640 1 -- 172.21.15.12:0/1325361930 wait complete. 2023-10-30T12:35:26.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31fac09640 1 Processor -- start 2023-10-30T12:35:26.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31fac09640 1 -- start start 2023-10-30T12:35:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31fac09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f40722a0 0x7f31f40af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31fac09640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31f40c6e60 0x7f31f4153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31fac09640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31f41508c0 0x7f31f40ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31fac09640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f41471c0 con 0x7f31f40722a0 2023-10-30T12:35:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31fac09640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f41f2890 con 0x7f31f41508c0 2023-10-30T12:35:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31fac09640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40af8f0 con 0x7f31f40c6e60 2023-10-30T12:35:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f89a5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f40722a0 0x7f31f40af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:26.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f89a5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f40722a0 0x7f31f40af3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52508/0 (socket says 172.21.15.12:52508) 2023-10-30T12:35:26.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f89a5640 1 -- 172.21.15.12:0/336985708 learned_addr learned my addr 172.21.15.12:0/336985708 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:26.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f91a6640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31f41508c0 0x7f31f40ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:26.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f3fff640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31f40c6e60 0x7f31f4153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:26.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f89a5640 1 -- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31f40c6e60 msgr2=0x7f31f4153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f89a5640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31f40c6e60 0x7f31f4153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f89a5640 1 -- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31f41508c0 msgr2=0x7f31f40ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:26.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f89a5640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31f41508c0 0x7f31f40ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:26.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f89a5640 1 -- 172.21.15.12:0/336985708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e800b650 con 0x7f31f40722a0 2023-10-30T12:35:26.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f3fff640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31f40c6e60 0x7f31f4153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:35:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f91a6640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31f41508c0 0x7f31f40ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f89a5640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f40722a0 0x7f31f40af3b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31e80b2040 tx=0x7f31e801b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f17fa640 1 -- 172.21.15.12:0/336985708 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e80af910 con 0x7f31f40722a0 2023-10-30T12:35:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f17fa640 1 -- 172.21.15.12:0/336985708 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e80afa70 con 0x7f31f40722a0 2023-10-30T12:35:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31f17fa640 1 -- 172.21.15.12:0/336985708 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e80afe20 con 0x7f31f40722a0 2023-10-30T12:35:26.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31fac09640 1 -- 172.21.15.12:0/336985708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f4071f70 con 0x7f31f40722a0 2023-10-30T12:35:26.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31fac09640 1 -- 172.21.15.12:0/336985708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f4146ee0 con 0x7f31f40722a0 2023-10-30T12:35:26.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.950+0000 7f31fac09640 1 -- 172.21.15.12:0/336985708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f404e170 con 0x7f31f40722a0 2023-10-30T12:35:26.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.954+0000 7f31f17fa640 1 -- 172.21.15.12:0/336985708 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f31e80b6070 con 0x7f31f40722a0 2023-10-30T12:35:26.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.954+0000 7f31f17fa640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31d80664a0 0x7f31d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:26.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.954+0000 7f31f17fa640 1 -- 172.21.15.12:0/336985708 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31e811da90 con 0x7f31f40722a0 2023-10-30T12:35:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.954+0000 7f31f3fff640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31d80664a0 0x7f31d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.954+0000 7f31f3fff640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31d80664a0 0x7f31d8068960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f31f4067c10 tx=0x7f31e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:26.958+0000 7f31f17fa640 1 -- 172.21.15.12:0/336985708 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f31e80eade0 con 0x7f31f40722a0 2023-10-30T12:35:27.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.114+0000 7f31fac09640 1 -- 172.21.15.12:0/336985708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31f40b0bb0 con 0x7f31f40722a0 2023-10-30T12:35:27.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.114+0000 7f31f17fa640 1 -- 172.21.15.12:0/336985708 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31e80b4070 con 0x7f31f40722a0 2023-10-30T12:35:27.119 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-10-30T12:35:27.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.114+0000 7f31fac09640 1 -- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31d80664a0 msgr2=0x7f31d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.114+0000 7f31fac09640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31d80664a0 0x7f31d8068960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f31f4067c10 tx=0x7f31e0009040 comp rx=0 tx=0).stop 2023-10-30T12:35:27.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.114+0000 7f31fac09640 1 -- 172.21.15.12:0/336985708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f40722a0 msgr2=0x7f31f40af3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.114+0000 7f31fac09640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f40722a0 0x7f31f40af3b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31e80b2040 tx=0x7f31e801b840 comp rx=0 tx=0).stop 2023-10-30T12:35:27.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.118+0000 7f31fac09640 1 -- 172.21.15.12:0/336985708 shutdown_connections 2023-10-30T12:35:27.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.118+0000 7f31fac09640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31f41508c0 0x7f31f40ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.118+0000 7f31fac09640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31d80664a0 0x7f31d8068960 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.118+0000 7f31fac09640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31f40c6e60 0x7f31f4153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.118+0000 7f31fac09640 1 --2- 172.21.15.12:0/336985708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f40722a0 0x7f31f40af3b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.118+0000 7f31fac09640 1 -- 172.21.15.12:0/336985708 >> 172.21.15.12:0/336985708 conn(0x7f31f40c7910 msgr2=0x7f31f414eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:27.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.118+0000 7f31fac09640 1 -- 172.21.15.12:0/336985708 shutdown_connections 2023-10-30T12:35:27.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.118+0000 7f31fac09640 1 -- 172.21.15.12:0/336985708 wait complete. 2023-10-30T12:35:27.140 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-30T12:35:27.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46b289c640 1 Processor -- start 2023-10-30T12:35:27.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46b289c640 1 -- start start 2023-10-30T12:35:27.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46b289c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46ac15d570 0x7f46ac15da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46b289c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46ac15a920 con 0x7f46ac159930 2023-10-30T12:35:27.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46b289c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46ac0cb930 con 0x7f46ac153980 2023-10-30T12:35:27.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46b289c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac159dc0 con 0x7f46ac15d570 2023-10-30T12:35:27.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46ab7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f46ac159930 0x7f46ac0cf1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58116/0 (socket says 172.21.15.12:58116) 2023-10-30T12:35:27.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46ab7fe640 1 -- 172.21.15.12:0/3855966882 learned_addr learned my addr 172.21.15.12:0/3855966882 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:27.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46b0e39640 1 --2- 172.21.15.12:0/3855966882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46ac15d570 0x7f46ac15da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33329211 0 0) 0x7f46ac15a920 con 0x7f46ac159930 2023-10-30T12:35:27.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4694003530 con 0x7f46ac159930 2023-10-30T12:35:27.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148999763 0 0) 0x7f46ac0cb930 con 0x7f46ac153980 2023-10-30T12:35:27.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46ac15a920 con 0x7f46ac153980 2023-10-30T12:35:27.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2279525098 0 0) 0x7f4694003530 con 0x7f46ac159930 2023-10-30T12:35:27.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46ac15d570 msgr2=0x7f46ac15da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:27.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46aaffd640 1 --2- 172.21.15.12:0/3855966882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46ac15d570 0x7f46ac15da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 >> v1:172.21.15.154:6789/0 conn(0x7f46ac153980 legacy=0x7f46ac0caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac15a080 con 0x7f46ac159930 2023-10-30T12:35:27.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.282+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f46a0001680 con 0x7f46ac159930 2023-10-30T12:35:27.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 >> v1:172.21.15.12:6789/0 conn(0x7f46ac159930 legacy=0x7f46ac0cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46aaffd640 1 --2- 172.21.15.12:0/3855966882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4694003b40 0x7f4694003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46aaffd640 1 --2- 172.21.15.12:0/3855966882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46940047d0 0x7f4694004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46aaffd640 1 --2- 172.21.15.12:0/3855966882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46ac15d570 0x7f4694005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac159dc0 con 0x7f46940047d0 2023-10-30T12:35:27.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469400ba40 con 0x7f4694003b40 2023-10-30T12:35:27.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469400bbd0 con 0x7f46ac15d570 2023-10-30T12:35:27.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46ab7fe640 1 --2- 172.21.15.12:0/3855966882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46940047d0 0x7f4694004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46abfff640 1 --2- 172.21.15.12:0/3855966882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4694003b40 0x7f4694003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46b0e39640 1 --2- 172.21.15.12:0/3855966882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46ac15d570 0x7f4694005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46ab7fe640 1 -- 172.21.15.12:0/3855966882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46ac15d570 msgr2=0x7f4694005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:27.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46ab7fe640 1 --2- 172.21.15.12:0/3855966882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46ac15d570 0x7f4694005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46ab7fe640 1 -- 172.21.15.12:0/3855966882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4694003b40 msgr2=0x7f4694003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46ab7fe640 1 --2- 172.21.15.12:0/3855966882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4694003b40 0x7f4694003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46ab7fe640 1 -- 172.21.15.12:0/3855966882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4694003530 con 0x7f46940047d0 2023-10-30T12:35:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46abfff640 1 --2- 172.21.15.12:0/3855966882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4694003b40 0x7f4694003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:27.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46ab7fe640 1 --2- 172.21.15.12:0/3855966882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46940047d0 0x7f4694004c50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f46a000c160 tx=0x7f46a001cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc044a9a4b6e1255 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:27.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a00058e0 con 0x7f46940047d0 2023-10-30T12:35:27.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a0005a60 con 0x7f46940047d0 2023-10-30T12:35:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.286+0000 7f46aaffd640 1 -- 172.21.15.12:0/3855966882 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a0005e90 con 0x7f46940047d0 2023-10-30T12:35:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b289c640 1 -- 172.21.15.12:0/3855966882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46940047d0 msgr2=0x7f4694004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b289c640 1 --2- 172.21.15.12:0/3855966882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46940047d0 0x7f4694004c50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f46a000c160 tx=0x7f46a001cb40 comp rx=0 tx=0).stop 2023-10-30T12:35:27.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b0e39640 1 -- 172.21.15.12:0/3855966882 reap_dead start 2023-10-30T12:35:27.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b289c640 1 -- 172.21.15.12:0/3855966882 shutdown_connections 2023-10-30T12:35:27.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b289c640 1 -- 172.21.15.12:0/3855966882 >> 172.21.15.12:0/3855966882 conn(0x7f46ac06c0f0 msgr2=0x7f46ac06c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:27.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b289c640 1 -- 172.21.15.12:0/3855966882 shutdown_connections 2023-10-30T12:35:27.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b289c640 1 -- 172.21.15.12:0/3855966882 wait complete. 2023-10-30T12:35:27.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b289c640 1 Processor -- start 2023-10-30T12:35:27.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b289c640 1 -- start start 2023-10-30T12:35:27.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b289c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46940047d0 0x7f46ac14c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b289c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46ac152cb0 0x7f46ac0b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b289c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46ac1f2150 0x7f46ac146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b289c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac14ce90 con 0x7f46ac1f2150 2023-10-30T12:35:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b289c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac1573e0 con 0x7f46940047d0 2023-10-30T12:35:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b289c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac14c450 con 0x7f46ac152cb0 2023-10-30T12:35:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b0e39640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46ac1f2150 0x7f46ac146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b0e39640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46ac1f2150 0x7f46ac146bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52524/0 (socket says 172.21.15.12:52524) 2023-10-30T12:35:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b0e39640 1 -- 172.21.15.12:0/2533437584 learned_addr learned my addr 172.21.15.12:0/2533437584 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46abfff640 1 --2- 172.21.15.12:0/2533437584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46940047d0 0x7f46ac14c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46ab7fe640 1 --2- 172.21.15.12:0/2533437584 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46ac152cb0 0x7f46ac0b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b0e39640 1 -- 172.21.15.12:0/2533437584 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46ac152cb0 msgr2=0x7f46ac0b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b0e39640 1 --2- 172.21.15.12:0/2533437584 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46ac152cb0 0x7f46ac0b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b0e39640 1 -- 172.21.15.12:0/2533437584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46940047d0 msgr2=0x7f46ac14c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b0e39640 1 --2- 172.21.15.12:0/2533437584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46940047d0 0x7f46ac14c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.290+0000 7f46b0e39640 1 -- 172.21.15.12:0/2533437584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac150640 con 0x7f46ac1f2150 2023-10-30T12:35:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.294+0000 7f46b0e39640 1 --2- 172.21.15.12:0/2533437584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46ac1f2150 0x7f46ac146bc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0b1cf0 tx=0x7f46a40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.294+0000 7f46a8ff9640 1 -- 172.21.15.12:0/2533437584 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a400e070 con 0x7f46ac1f2150 2023-10-30T12:35:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.294+0000 7f46a8ff9640 1 -- 172.21.15.12:0/2533437584 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a4011ed0 con 0x7f46ac1f2150 2023-10-30T12:35:27.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.294+0000 7f46b289c640 1 -- 172.21.15.12:0/2533437584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ac14f8d0 con 0x7f46ac1f2150 2023-10-30T12:35:27.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.294+0000 7f46a8ff9640 1 -- 172.21.15.12:0/2533437584 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a40039a0 con 0x7f46ac1f2150 2023-10-30T12:35:27.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.294+0000 7f46b289c640 1 -- 172.21.15.12:0/2533437584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ac14fe90 con 0x7f46ac1f2150 2023-10-30T12:35:27.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.294+0000 7f46b289c640 1 -- 172.21.15.12:0/2533437584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ac04e170 con 0x7f46ac1f2150 2023-10-30T12:35:27.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.294+0000 7f46a8ff9640 1 -- 172.21.15.12:0/2533437584 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f46a4003b20 con 0x7f46ac1f2150 2023-10-30T12:35:27.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.294+0000 7f46a8ff9640 1 --2- 172.21.15.12:0/2533437584 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f46980664d0 0x7f4698068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.294+0000 7f46a8ff9640 1 -- 172.21.15.12:0/2533437584 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46a4084470 con 0x7f46ac1f2150 2023-10-30T12:35:27.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.298+0000 7f46abfff640 1 --2- 172.21.15.12:0/2533437584 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f46980664d0 0x7f4698068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.298+0000 7f46abfff640 1 --2- 172.21.15.12:0/2533437584 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f46980664d0 0x7f4698068990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f469c007bf0 tx=0x7f469c007120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:27.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.298+0000 7f46a8ff9640 1 -- 172.21.15.12:0/2533437584 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f46a4051850 con 0x7f46ac1f2150 2023-10-30T12:35:27.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.466+0000 7f46b289c640 1 -- 172.21.15.12:0/2533437584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f46ac0c5b90 con 0x7f46ac1f2150 2023-10-30T12:35:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.466+0000 7f46a8ff9640 1 -- 172.21.15.12:0/2533437584 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f46a4054e70 con 0x7f46ac1f2150 2023-10-30T12:35:27.471 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-10-30T12:35:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.466+0000 7f46b289c640 1 -- 172.21.15.12:0/2533437584 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f46980664d0 msgr2=0x7f4698068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.466+0000 7f46b289c640 1 --2- 172.21.15.12:0/2533437584 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f46980664d0 0x7f4698068990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f469c007bf0 tx=0x7f469c007120 comp rx=0 tx=0).stop 2023-10-30T12:35:27.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.466+0000 7f46b289c640 1 -- 172.21.15.12:0/2533437584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46ac1f2150 msgr2=0x7f46ac146bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.466+0000 7f46b289c640 1 --2- 172.21.15.12:0/2533437584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46ac1f2150 0x7f46ac146bc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0b1cf0 tx=0x7f46a40115a0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.466+0000 7f46b289c640 1 -- 172.21.15.12:0/2533437584 shutdown_connections 2023-10-30T12:35:27.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.466+0000 7f46b289c640 1 --2- 172.21.15.12:0/2533437584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46ac1f2150 0x7f46ac146bc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.466+0000 7f46b289c640 1 --2- 172.21.15.12:0/2533437584 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f46980664d0 0x7f4698068990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.466+0000 7f46b289c640 1 --2- 172.21.15.12:0/2533437584 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46ac152cb0 0x7f46ac0b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.466+0000 7f46b289c640 1 --2- 172.21.15.12:0/2533437584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46940047d0 0x7f46ac14c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.466+0000 7f46b289c640 1 -- 172.21.15.12:0/2533437584 >> 172.21.15.12:0/2533437584 conn(0x7f46ac06c0f0 msgr2=0x7f46ac158da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:27.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.466+0000 7f46b289c640 1 -- 172.21.15.12:0/2533437584 shutdown_connections 2023-10-30T12:35:27.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.466+0000 7f46b289c640 1 -- 172.21.15.12:0/2533437584 wait complete. 2023-10-30T12:35:27.490 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-30T12:35:27.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.634+0000 7fe72f71d640 1 Processor -- start 2023-10-30T12:35:27.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.634+0000 7fe72f71d640 1 -- start start 2023-10-30T12:35:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.634+0000 7fe72f71d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe7280bfec0 0x7fe7280c02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.634+0000 7fe72f71d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7280c5f20 0x7fe72815e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.634+0000 7fe72f71d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72815ebe0 con 0x7fe7280c6bf0 2023-10-30T12:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.634+0000 7fe72f71d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7280c07e0 con 0x7fe7280bfec0 2023-10-30T12:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.634+0000 7fe72f71d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe728069e90 con 0x7fe7280c5f20 2023-10-30T12:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.634+0000 7fe72dcba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7280c5f20 0x7fe72815e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.634+0000 7fe72d4b9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe7280bfec0 0x7fe7280c02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.634+0000 7fe72ccb8640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe7280c6bf0 0x7fe7280c5810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:43112/0 (socket says 172.21.15.12:43112) 2023-10-30T12:35:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.634+0000 7fe72ccb8640 1 -- 172.21.15.12:0/1331979955 learned_addr learned my addr 172.21.15.12:0/1331979955 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.634+0000 7fe72d4b9640 1 -- 172.21.15.12:0/1331979955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7280c5f20 msgr2=0x7fe72815e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.634+0000 7fe72d4b9640 1 --2- 172.21.15.12:0/1331979955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7280c5f20 0x7fe72815e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.634+0000 7fe72d4b9640 1 -- 172.21.15.12:0/1331979955 >> v1:172.21.15.154:6789/0 conn(0x7fe7280c6bf0 legacy=0x7fe7280c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.634+0000 7fe72d4b9640 1 -- 172.21.15.12:0/1331979955 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72806aa10 con 0x7fe7280bfec0 2023-10-30T12:35:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.638+0000 7fe72d4b9640 1 --2- 172.21.15.12:0/1331979955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280bfec0 0x7fe7280c02a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7240097a0 tx=0x7fe72401c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acfe84b2bf7c71bc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.638+0000 7fe71ffff640 1 -- 172.21.15.12:0/1331979955 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7240ab600 con 0x7fe7280bfec0 2023-10-30T12:35:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.638+0000 7fe71ffff640 1 -- 172.21.15.12:0/1331979955 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7240a7050 con 0x7fe7280bfec0 2023-10-30T12:35:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.638+0000 7fe71ffff640 1 -- 172.21.15.12:0/1331979955 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7240ab980 con 0x7fe7280bfec0 2023-10-30T12:35:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.638+0000 7fe72f71d640 1 -- 172.21.15.12:0/1331979955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280bfec0 msgr2=0x7fe7280c02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.638+0000 7fe72f71d640 1 --2- 172.21.15.12:0/1331979955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280bfec0 0x7fe7280c02a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7240097a0 tx=0x7fe72401c460 comp rx=0 tx=0).stop 2023-10-30T12:35:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.638+0000 7fe72f71d640 1 -- 172.21.15.12:0/1331979955 shutdown_connections 2023-10-30T12:35:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.638+0000 7fe72f71d640 1 --2- 172.21.15.12:0/1331979955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7280c5f20 0x7fe72815e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.638+0000 7fe72f71d640 1 --2- 172.21.15.12:0/1331979955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280bfec0 0x7fe7280c02a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.638+0000 7fe72f71d640 1 -- 172.21.15.12:0/1331979955 >> 172.21.15.12:0/1331979955 conn(0x7fe7280c7910 msgr2=0x7fe7280c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.638+0000 7fe72f71d640 1 -- 172.21.15.12:0/1331979955 shutdown_connections 2023-10-30T12:35:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.638+0000 7fe72f71d640 1 -- 172.21.15.12:0/1331979955 wait complete. 2023-10-30T12:35:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72f71d640 1 Processor -- start 2023-10-30T12:35:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72f71d640 1 -- start start 2023-10-30T12:35:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72f71d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280bfec0 0x7fe728151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72f71d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7280c6bf0 0x7fe7280ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72f71d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7280ca970 0x7fe72815dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72f71d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7281f6480 con 0x7fe7280ca970 2023-10-30T12:35:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72f71d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72814ac00 con 0x7fe7280bfec0 2023-10-30T12:35:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72f71d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72815b880 con 0x7fe7280c6bf0 2023-10-30T12:35:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72dcba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7280ca970 0x7fe72815dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72d4b9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280bfec0 0x7fe728151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72dcba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7280ca970 0x7fe72815dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52536/0 (socket says 172.21.15.12:52536) 2023-10-30T12:35:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72dcba640 1 -- 172.21.15.12:0/682722286 learned_addr learned my addr 172.21.15.12:0/682722286 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72ccb8640 1 --2- 172.21.15.12:0/682722286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7280c6bf0 0x7fe7280ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72dcba640 1 -- 172.21.15.12:0/682722286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7280c6bf0 msgr2=0x7fe7280ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72dcba640 1 --2- 172.21.15.12:0/682722286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7280c6bf0 0x7fe7280ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72dcba640 1 -- 172.21.15.12:0/682722286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280bfec0 msgr2=0x7fe728151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72dcba640 1 --2- 172.21.15.12:0/682722286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280bfec0 0x7fe728151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72dcba640 1 -- 172.21.15.12:0/682722286 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7240093b0 con 0x7fe7280ca970 2023-10-30T12:35:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72dcba640 1 --2- 172.21.15.12:0/682722286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7280ca970 0x7fe72815dc80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe72000efc0 tx=0x7fe72000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe71dffb640 1 -- 172.21.15.12:0/682722286 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe720009b20 con 0x7fe7280ca970 2023-10-30T12:35:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe71dffb640 1 -- 172.21.15.12:0/682722286 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72000f040 con 0x7fe7280ca970 2023-10-30T12:35:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72f71d640 1 -- 172.21.15.12:0/682722286 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72815bad0 con 0x7fe7280ca970 2023-10-30T12:35:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe71dffb640 1 -- 172.21.15.12:0/682722286 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe720014480 con 0x7fe7280ca970 2023-10-30T12:35:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.642+0000 7fe72f71d640 1 -- 172.21.15.12:0/682722286 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72815c090 con 0x7fe7280ca970 2023-10-30T12:35:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.646+0000 7fe72f71d640 1 -- 172.21.15.12:0/682722286 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7281f65c0 con 0x7fe7280ca970 2023-10-30T12:35:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.646+0000 7fe71dffb640 1 -- 172.21.15.12:0/682722286 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe7200145e0 con 0x7fe7280ca970 2023-10-30T12:35:27.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.646+0000 7fe71dffb640 1 --2- 172.21.15.12:0/682722286 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe70c0664a0 0x7fe70c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.646+0000 7fe71dffb640 1 -- 172.21.15.12:0/682722286 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe720084b00 con 0x7fe7280ca970 2023-10-30T12:35:27.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.646+0000 7fe72d4b9640 1 --2- 172.21.15.12:0/682722286 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe70c0664a0 0x7fe70c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.650+0000 7fe72d4b9640 1 --2- 172.21.15.12:0/682722286 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe70c0664a0 0x7fe70c068960 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe7240ad000 tx=0x7fe7240a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:27.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.650+0000 7fe71dffb640 1 -- 172.21.15.12:0/682722286 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe7281f65c0 con 0x7fe7280ca970 2023-10-30T12:35:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.806+0000 7fe72f71d640 1 -- 172.21.15.12:0/682722286 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7281f65c0 con 0x7fe7280ca970 2023-10-30T12:35:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.806+0000 7fe71dffb640 1 -- 172.21.15.12:0/682722286 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe7281f65c0 con 0x7fe7280ca970 2023-10-30T12:35:27.810 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-10-30T12:35:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.806+0000 7fe72f71d640 1 -- 172.21.15.12:0/682722286 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe70c0664a0 msgr2=0x7fe70c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.806+0000 7fe72f71d640 1 --2- 172.21.15.12:0/682722286 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe70c0664a0 0x7fe70c068960 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe7240ad000 tx=0x7fe7240a6000 comp rx=0 tx=0).stop 2023-10-30T12:35:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.806+0000 7fe72f71d640 1 -- 172.21.15.12:0/682722286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7280ca970 msgr2=0x7fe72815dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.806+0000 7fe72f71d640 1 --2- 172.21.15.12:0/682722286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7280ca970 0x7fe72815dc80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe72000efc0 tx=0x7fe72000c490 comp rx=0 tx=0).stop 2023-10-30T12:35:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.806+0000 7fe72f71d640 1 -- 172.21.15.12:0/682722286 shutdown_connections 2023-10-30T12:35:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.806+0000 7fe72f71d640 1 --2- 172.21.15.12:0/682722286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7280ca970 0x7fe72815dc80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.806+0000 7fe72f71d640 1 --2- 172.21.15.12:0/682722286 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe70c0664a0 0x7fe70c068960 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.806+0000 7fe72f71d640 1 --2- 172.21.15.12:0/682722286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7280c6bf0 0x7fe7280ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.806+0000 7fe72f71d640 1 --2- 172.21.15.12:0/682722286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280bfec0 0x7fe728151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.806+0000 7fe72f71d640 1 -- 172.21.15.12:0/682722286 >> 172.21.15.12:0/682722286 conn(0x7fe7280c7910 msgr2=0x7fe728071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.806+0000 7fe72f71d640 1 -- 172.21.15.12:0/682722286 shutdown_connections 2023-10-30T12:35:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.806+0000 7fe72f71d640 1 -- 172.21.15.12:0/682722286 wait complete. 2023-10-30T12:35:27.831 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:35:27.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0fd3e640 1 Processor -- start 2023-10-30T12:35:27.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0fd3e640 1 -- start start 2023-10-30T12:35:27.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0fd3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea0806f7a0 0x7fea0806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0fd3e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fea080c6bd0 0x7fea08069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0fd3e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0806a3d0 con 0x7fea08071110 2023-10-30T12:35:27.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0fd3e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea080700c0 con 0x7fea080c6bd0 2023-10-30T12:35:27.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0fd3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea08066cf0 con 0x7fea0806f7a0 2023-10-30T12:35:27.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0dada640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea0806f7a0 0x7fea0806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0e2db640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fea08071110 0x7fea08158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:43116/0 (socket says 172.21.15.12:43116) 2023-10-30T12:35:27.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0e2db640 1 -- 172.21.15.12:0/1665015811 learned_addr learned my addr 172.21.15.12:0/1665015811 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:27.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0d2d9640 1 --2- 172.21.15.12:0/1665015811 >> v2:172.21.15.154:3300/0 conn(0x7fea080c6bd0 0x7fea08069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0dada640 1 -- 172.21.15.12:0/1665015811 >> v2:172.21.15.154:3300/0 conn(0x7fea080c6bd0 msgr2=0x7fea08069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:27.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0dada640 1 --2- 172.21.15.12:0/1665015811 >> v2:172.21.15.154:3300/0 conn(0x7fea080c6bd0 0x7fea08069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0dada640 1 -- 172.21.15.12:0/1665015811 >> v1:172.21.15.154:6789/0 conn(0x7fea08071110 legacy=0x7fea08158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0cad8640 1 -- 172.21.15.12:0/1665015811 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089024367 0 0) 0x7fea0806a3d0 con 0x7fea08071110 2023-10-30T12:35:27.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.970+0000 7fea0dada640 1 -- 172.21.15.12:0/1665015811 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea080c70d0 con 0x7fea0806f7a0 2023-10-30T12:35:27.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.974+0000 7fea0dada640 1 --2- 172.21.15.12:0/1665015811 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea0806f7a0 0x7fea0806fb80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fea040097c0 tx=0x7fea0401ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c1f0d74c4dd83ca server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:27.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.974+0000 7fea0cad8640 1 -- 172.21.15.12:0/1665015811 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea040a8970 con 0x7fea0806f7a0 2023-10-30T12:35:27.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.974+0000 7fea0cad8640 1 -- 172.21.15.12:0/1665015811 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea040a8ad0 con 0x7fea0806f7a0 2023-10-30T12:35:27.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.974+0000 7fea0cad8640 1 -- 172.21.15.12:0/1665015811 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea040a8e60 con 0x7fea0806f7a0 2023-10-30T12:35:27.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.974+0000 7fea0fd3e640 1 -- 172.21.15.12:0/1665015811 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea0806f7a0 msgr2=0x7fea0806fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.974+0000 7fea0fd3e640 1 --2- 172.21.15.12:0/1665015811 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea0806f7a0 0x7fea0806fb80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fea040097c0 tx=0x7fea0401ec70 comp rx=0 tx=0).stop 2023-10-30T12:35:27.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.974+0000 7fea0fd3e640 1 -- 172.21.15.12:0/1665015811 shutdown_connections 2023-10-30T12:35:27.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.974+0000 7fea0fd3e640 1 --2- 172.21.15.12:0/1665015811 >> v2:172.21.15.154:3300/0 conn(0x7fea080c6bd0 0x7fea08069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.974+0000 7fea0fd3e640 1 --2- 172.21.15.12:0/1665015811 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea0806f7a0 0x7fea0806fb80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.974+0000 7fea0fd3e640 1 -- 172.21.15.12:0/1665015811 >> 172.21.15.12:0/1665015811 conn(0x7fea080c7910 msgr2=0x7fea080c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:27.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.974+0000 7fea0fd3e640 1 -- 172.21.15.12:0/1665015811 shutdown_connections 2023-10-30T12:35:27.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.974+0000 7fea0fd3e640 1 -- 172.21.15.12:0/1665015811 wait complete. 2023-10-30T12:35:27.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0fd3e640 1 Processor -- start 2023-10-30T12:35:27.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0fd3e640 1 -- start start 2023-10-30T12:35:27.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0fd3e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea0806f7a0 0x7fea080ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0fd3e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea08071110 0x7fea080c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0fd3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea080c6bd0 0x7fea0814d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0fd3e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea080c2090 con 0x7fea08071110 2023-10-30T12:35:27.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0fd3e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea080ade10 con 0x7fea0806f7a0 2023-10-30T12:35:27.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0fd3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea081f28f0 con 0x7fea080c6bd0 2023-10-30T12:35:27.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0d2d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea08071110 0x7fea080c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0d2d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea08071110 0x7fea080c0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52548/0 (socket says 172.21.15.12:52548) 2023-10-30T12:35:27.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0d2d9640 1 -- 172.21.15.12:0/2284200160 learned_addr learned my addr 172.21.15.12:0/2284200160 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:27.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0dada640 1 --2- 172.21.15.12:0/2284200160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea0806f7a0 0x7fea080ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0e2db640 1 --2- 172.21.15.12:0/2284200160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea080c6bd0 0x7fea0814d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0d2d9640 1 -- 172.21.15.12:0/2284200160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea080c6bd0 msgr2=0x7fea0814d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0d2d9640 1 --2- 172.21.15.12:0/2284200160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea080c6bd0 0x7fea0814d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0d2d9640 1 -- 172.21.15.12:0/2284200160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea0806f7a0 msgr2=0x7fea080ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0d2d9640 1 --2- 172.21.15.12:0/2284200160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea0806f7a0 0x7fea080ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0d2d9640 1 -- 172.21.15.12:0/2284200160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea04009420 con 0x7fea08071110 2023-10-30T12:35:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0dada640 1 --2- 172.21.15.12:0/2284200160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea0806f7a0 0x7fea080ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0d2d9640 1 --2- 172.21.15.12:0/2284200160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea08071110 0x7fea080c0da0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800e870 tx=0x7fe9f800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fe9fe7fc640 1 -- 172.21.15.12:0/2284200160 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8009510 con 0x7fea08071110 2023-10-30T12:35:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fe9fe7fc640 1 -- 172.21.15.12:0/2284200160 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f800eec0 con 0x7fea08071110 2023-10-30T12:35:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0fd3e640 1 -- 172.21.15.12:0/2284200160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea080c12e0 con 0x7fea08071110 2023-10-30T12:35:27.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fe9fe7fc640 1 -- 172.21.15.12:0/2284200160 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f800cb40 con 0x7fea08071110 2023-10-30T12:35:27.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.978+0000 7fea0fd3e640 1 -- 172.21.15.12:0/2284200160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea081ef890 con 0x7fea08071110 2023-10-30T12:35:27.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.982+0000 7fea0fd3e640 1 -- 172.21.15.12:0/2284200160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea081f2aa0 con 0x7fea08071110 2023-10-30T12:35:27.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.982+0000 7fe9fe7fc640 1 -- 172.21.15.12:0/2284200160 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe9f800cca0 con 0x7fea08071110 2023-10-30T12:35:27.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.982+0000 7fe9fe7fc640 1 --2- 172.21.15.12:0/2284200160 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe9ec0664d0 0x7fe9ec068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:27.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.982+0000 7fe9fe7fc640 1 -- 172.21.15.12:0/2284200160 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9f8083fa0 con 0x7fea08071110 2023-10-30T12:35:27.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.982+0000 7fea0dada640 1 --2- 172.21.15.12:0/2284200160 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe9ec0664d0 0x7fe9ec068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:27.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.982+0000 7fea0dada640 1 --2- 172.21.15.12:0/2284200160 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe9ec0664d0 0x7fe9ec068990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fea040b2000 tx=0x7fea040aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:27.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:27.986+0000 7fe9fe7fc640 1 -- 172.21.15.12:0/2284200160 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fea081f2aa0 con 0x7fea08071110 2023-10-30T12:35:28.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.178+0000 7fea0fd3e640 1 -- 172.21.15.12:0/2284200160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea081f2a60 con 0x7fea08071110 2023-10-30T12:35:28.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.178+0000 7fe9fe7fc640 1 -- 172.21.15.12:0/2284200160 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6525 (secure 0 0 0) 0x7fe9f8051380 con 0x7fea08071110 2023-10-30T12:35:28.184 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:28.184 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:35:22.316922+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:creating","state_seq":1,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:35:28.185 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 12 2023-10-30T12:35:28.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.182+0000 7fea0fd3e640 1 -- 172.21.15.12:0/2284200160 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe9ec0664d0 msgr2=0x7fe9ec068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:28.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.182+0000 7fea0fd3e640 1 --2- 172.21.15.12:0/2284200160 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe9ec0664d0 0x7fe9ec068990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fea040b2000 tx=0x7fea040aa070 comp rx=0 tx=0).stop 2023-10-30T12:35:28.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.182+0000 7fea0fd3e640 1 -- 172.21.15.12:0/2284200160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea08071110 msgr2=0x7fea080c0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:28.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.182+0000 7fea0fd3e640 1 --2- 172.21.15.12:0/2284200160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea08071110 0x7fea080c0da0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800e870 tx=0x7fe9f800ec70 comp rx=0 tx=0).stop 2023-10-30T12:35:28.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.182+0000 7fea0fd3e640 1 -- 172.21.15.12:0/2284200160 shutdown_connections 2023-10-30T12:35:28.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.182+0000 7fea0fd3e640 1 --2- 172.21.15.12:0/2284200160 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe9ec0664d0 0x7fe9ec068990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.182+0000 7fea0fd3e640 1 --2- 172.21.15.12:0/2284200160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea080c6bd0 0x7fea0814d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.182+0000 7fea0fd3e640 1 --2- 172.21.15.12:0/2284200160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea08071110 0x7fea080c0da0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.182+0000 7fea0fd3e640 1 --2- 172.21.15.12:0/2284200160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea0806f7a0 0x7fea080ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.182+0000 7fea0fd3e640 1 -- 172.21.15.12:0/2284200160 >> 172.21.15.12:0/2284200160 conn(0x7fea080c7910 msgr2=0x7fea080c8220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:28.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.182+0000 7fea0fd3e640 1 -- 172.21.15.12:0/2284200160 shutdown_connections 2023-10-30T12:35:28.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.182+0000 7fea0fd3e640 1 -- 172.21.15.12:0/2284200160 wait complete. 2023-10-30T12:35:28.205 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:35:28.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3dbdedc640 1 Processor -- start 2023-10-30T12:35:28.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3dbdedc640 1 -- start start 2023-10-30T12:35:28.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3dbdedc640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db81538f0 0x7f3db8159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:28.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3dbdedc640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db80ca6a0 con 0x7f3db815cd50 2023-10-30T12:35:28.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3dbdedc640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db815d300 con 0x7f3db80cefa0 2023-10-30T12:35:28.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3dbdedc640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db815d5e0 con 0x7f3db81538f0 2023-10-30T12:35:28.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db7fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3db815cd50 0x7f3db80c9f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58130/0 (socket says 172.21.15.12:58130) 2023-10-30T12:35:28.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db7fff640 1 -- 172.21.15.12:0/1133572874 learned_addr learned my addr 172.21.15.12:0/1133572874 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:28.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3dbceda640 1 --2- 172.21.15.12:0/1133572874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db81538f0 0x7f3db8159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:28.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831651189 0 0) 0x7f3db80ca6a0 con 0x7f3db815cd50 2023-10-30T12:35:28.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3da8003530 con 0x7f3db815cd50 2023-10-30T12:35:28.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155646524 0 0) 0x7f3db815d300 con 0x7f3db80cefa0 2023-10-30T12:35:28.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3db80ca6a0 con 0x7f3db80cefa0 2023-10-30T12:35:28.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 174044245 0 0) 0x7f3da8003530 con 0x7f3db815cd50 2023-10-30T12:35:28.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db81538f0 msgr2=0x7f3db8159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:28.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 --2- 172.21.15.12:0/1133572874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db81538f0 0x7f3db8159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 >> v1:172.21.15.154:6789/0 conn(0x7f3db80cefa0 legacy=0x7f3db81640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:28.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db815d8c0 con 0x7f3db815cd50 2023-10-30T12:35:28.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f3da4001680 con 0x7f3db815cd50 2023-10-30T12:35:28.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 >> v1:172.21.15.12:6789/0 conn(0x7f3db815cd50 legacy=0x7f3db80c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:28.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 --2- 172.21.15.12:0/1133572874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da8003b40 0x7f3da8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:28.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 --2- 172.21.15.12:0/1133572874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db81538f0 0x7f3da80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:28.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 --2- 172.21.15.12:0/1133572874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da8004d10 0x7f3da8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:28.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db815d5e0 con 0x7f3da8004d10 2023-10-30T12:35:28.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da800ba40 con 0x7f3da8003b40 2023-10-30T12:35:28.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da800bbb0 con 0x7f3db81538f0 2023-10-30T12:35:28.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3dbceda640 1 --2- 172.21.15.12:0/1133572874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da8003b40 0x7f3da8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:28.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3dbd6db640 1 --2- 172.21.15.12:0/1133572874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da8004d10 0x7f3da8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:28.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3db7fff640 1 --2- 172.21.15.12:0/1133572874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db81538f0 0x7f3da80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:28.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3dbd6db640 1 -- 172.21.15.12:0/1133572874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db81538f0 msgr2=0x7f3da80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:28.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3dbd6db640 1 --2- 172.21.15.12:0/1133572874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db81538f0 0x7f3da80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3dbd6db640 1 -- 172.21.15.12:0/1133572874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da8003b40 msgr2=0x7f3da8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:28.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3dbd6db640 1 --2- 172.21.15.12:0/1133572874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da8003b40 0x7f3da8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.346+0000 7f3dbd6db640 1 -- 172.21.15.12:0/1133572874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da8003530 con 0x7f3da8004d10 2023-10-30T12:35:28.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.350+0000 7f3dbd6db640 1 --2- 172.21.15.12:0/1133572874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da8004d10 0x7f3da8005190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3db0001fd0 tx=0x7f3db001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c71862410773bf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:28.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.350+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0003b80 con 0x7f3da8004d10 2023-10-30T12:35:28.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.350+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db0003d00 con 0x7f3da8004d10 2023-10-30T12:35:28.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.350+0000 7f3db77fe640 1 -- 172.21.15.12:0/1133572874 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0005310 con 0x7f3da8004d10 2023-10-30T12:35:28.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.350+0000 7f3dbdedc640 1 -- 172.21.15.12:0/1133572874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da8004d10 msgr2=0x7f3da8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:28.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.350+0000 7f3dbdedc640 1 --2- 172.21.15.12:0/1133572874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da8004d10 0x7f3da8005190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3db0001fd0 tx=0x7f3db001c8e0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.350+0000 7f3dbd6db640 1 -- 172.21.15.12:0/1133572874 reap_dead start 2023-10-30T12:35:28.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.350+0000 7f3dbdedc640 1 -- 172.21.15.12:0/1133572874 shutdown_connections 2023-10-30T12:35:28.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.350+0000 7f3dbdedc640 1 -- 172.21.15.12:0/1133572874 >> 172.21.15.12:0/1133572874 conn(0x7f3db806a640 msgr2=0x7f3db8065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:28.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.350+0000 7f3dbdedc640 1 -- 172.21.15.12:0/1133572874 shutdown_connections 2023-10-30T12:35:28.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.350+0000 7f3dbdedc640 1 -- 172.21.15.12:0/1133572874 wait complete. 2023-10-30T12:35:28.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3dbdedc640 1 Processor -- start 2023-10-30T12:35:28.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3dbdedc640 1 -- start start 2023-10-30T12:35:28.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3dbdedc640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3da8004d10 0x7f3db8145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:28.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3dbdedc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3db814f0e0 0x7f3db814f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:28.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3dbdedc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db814fde0 0x7f3db81f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:28.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3dbdedc640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db8152850 con 0x7f3db814f0e0 2023-10-30T12:35:28.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3dbdedc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db81f2e60 con 0x7f3db814fde0 2023-10-30T12:35:28.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3dbdedc640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db81f3140 con 0x7f3da8004d10 2023-10-30T12:35:28.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3db7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3db814f0e0 0x7f3db814f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:28.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3db7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3db814f0e0 0x7f3db814f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52572/0 (socket says 172.21.15.12:52572) 2023-10-30T12:35:28.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3db7fff640 1 -- 172.21.15.12:0/765019522 learned_addr learned my addr 172.21.15.12:0/765019522 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:28.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3dbceda640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3da8004d10 0x7f3db8145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:28.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3dbd6db640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db814fde0 0x7f3db81f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:28.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3db7fff640 1 -- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3da8004d10 msgr2=0x7f3db8145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:28.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3db7fff640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3da8004d10 0x7f3db8145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3db7fff640 1 -- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db814fde0 msgr2=0x7f3db81f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:28.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3db7fff640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db814fde0 0x7f3db81f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3db7fff640 1 -- 172.21.15.12:0/765019522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db814e8f0 con 0x7f3db814f0e0 2023-10-30T12:35:28.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3dbd6db640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db814fde0 0x7f3db81f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:35:28.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3dbceda640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3da8004d10 0x7f3db8145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:35:28.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3db7fff640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3db814f0e0 0x7f3db814f8a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3da4007340 tx=0x7f3da4007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:28.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3db57fa640 1 -- 172.21.15.12:0/765019522 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da400f9c0 con 0x7f3db814f0e0 2023-10-30T12:35:28.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3db57fa640 1 -- 172.21.15.12:0/765019522 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da400fb20 con 0x7f3db814f0e0 2023-10-30T12:35:28.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3db57fa640 1 -- 172.21.15.12:0/765019522 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da400fed0 con 0x7f3db814f0e0 2023-10-30T12:35:28.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3dbdedc640 1 -- 172.21.15.12:0/765019522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db814ebd0 con 0x7f3db814f0e0 2023-10-30T12:35:28.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.354+0000 7f3dbdedc640 1 -- 172.21.15.12:0/765019522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db8154540 con 0x7f3db814f0e0 2023-10-30T12:35:28.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.358+0000 7f3dbdedc640 1 -- 172.21.15.12:0/765019522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db804e170 con 0x7f3db814f0e0 2023-10-30T12:35:28.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.358+0000 7f3db57fa640 1 -- 172.21.15.12:0/765019522 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3da401d460 con 0x7f3db814f0e0 2023-10-30T12:35:28.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.358+0000 7f3db57fa640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3d9c0664a0 0x7f3d9c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:28.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.358+0000 7f3db57fa640 1 -- 172.21.15.12:0/765019522 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3da40840c0 con 0x7f3db814f0e0 2023-10-30T12:35:28.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.358+0000 7f3dbceda640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3d9c0664a0 0x7f3d9c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:28.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.358+0000 7f3dbceda640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3d9c0664a0 0x7f3d9c068960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009ad0 tx=0x7f3dac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:28.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.362+0000 7f3db57fa640 1 -- 172.21.15.12:0/765019522 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3da4051410 con 0x7f3db814f0e0 2023-10-30T12:35:28.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.526+0000 7f3dbdedc640 1 -- 172.21.15.12:0/765019522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3db8157180 con 0x7f3db814f0e0 2023-10-30T12:35:28.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.526+0000 7f3db57fa640 1 -- 172.21.15.12:0/765019522 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f3da4054a30 con 0x7f3db814f0e0 2023-10-30T12:35:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:28.531 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":22,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:35:07.153665+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:35:03.216053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:35:04.597162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:35:28.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.530+0000 7f3dbdedc640 1 -- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3d9c0664a0 msgr2=0x7f3d9c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:28.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.530+0000 7f3dbdedc640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3d9c0664a0 0x7f3d9c068960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009ad0 tx=0x7f3dac008040 comp rx=0 tx=0).stop 2023-10-30T12:35:28.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.530+0000 7f3dbdedc640 1 -- 172.21.15.12:0/765019522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3db814f0e0 msgr2=0x7f3db814f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:28.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.530+0000 7f3dbdedc640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3db814f0e0 0x7f3db814f8a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3da4007340 tx=0x7f3da4007800 comp rx=0 tx=0).stop 2023-10-30T12:35:28.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.530+0000 7f3dbdedc640 1 -- 172.21.15.12:0/765019522 shutdown_connections 2023-10-30T12:35:28.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.530+0000 7f3dbdedc640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db814fde0 0x7f3db81f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.530+0000 7f3dbdedc640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3db814f0e0 0x7f3db814f8a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.530+0000 7f3dbdedc640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3d9c0664a0 0x7f3d9c068960 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.530+0000 7f3dbdedc640 1 --2- 172.21.15.12:0/765019522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3da8004d10 0x7f3db8145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.530+0000 7f3dbdedc640 1 -- 172.21.15.12:0/765019522 >> 172.21.15.12:0/765019522 conn(0x7f3db806a640 msgr2=0x7f3db80caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.530+0000 7f3dbdedc640 1 -- 172.21.15.12:0/765019522 shutdown_connections 2023-10-30T12:35:28.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.530+0000 7f3dbdedc640 1 -- 172.21.15.12:0/765019522 wait complete. 2023-10-30T12:35:28.552 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:35:28.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.690+0000 7f33b15ab640 1 Processor -- start 2023-10-30T12:35:28.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.690+0000 7f33b15ab640 1 -- start start 2023-10-30T12:35:28.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.690+0000 7f33b15ab640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f33ac1538a0 0x7f33ac15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:28.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33b15ab640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f33ac0ca0e0 0x7f33ac15a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:28.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33b15ab640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33ac0cf160 0x7f33ac0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:28.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33b15ab640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33ac0cb800 con 0x7f33ac1538a0 2023-10-30T12:35:28.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33b15ab640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33ac15dc30 con 0x7f33ac0ca0e0 2023-10-30T12:35:28.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33b15ab640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac0cb330 con 0x7f33ac0cf160 2023-10-30T12:35:28.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33aaffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f33ac1538a0 0x7f33ac15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:28.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33aaffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f33ac1538a0 0x7f33ac15d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52584/0 (socket says 172.21.15.12:52584) 2023-10-30T12:35:28.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33aaffd640 1 -- 172.21.15.12:0/3390420262 learned_addr learned my addr 172.21.15.12:0/3390420262 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:28.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33a3fff640 1 --2- 172.21.15.12:0/3390420262 >> v2:172.21.15.154:3300/0 conn(0x7f33ac0ca0e0 0x7f33ac15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:28.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33ab7fe640 1 --2- 172.21.15.12:0/3390420262 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33ac0cf160 0x7f33ac0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:28.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33aaffd640 1 -- 172.21.15.12:0/3390420262 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33ac0cf160 msgr2=0x7f33ac0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:28.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33aaffd640 1 --2- 172.21.15.12:0/3390420262 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33ac0cf160 0x7f33ac0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33aaffd640 1 -- 172.21.15.12:0/3390420262 >> v2:172.21.15.154:3300/0 conn(0x7f33ac0ca0e0 msgr2=0x7f33ac15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:28.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33aaffd640 1 --2- 172.21.15.12:0/3390420262 >> v2:172.21.15.154:3300/0 conn(0x7f33ac0ca0e0 0x7f33ac15a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33aaffd640 1 -- 172.21.15.12:0/3390420262 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac0cad70 con 0x7f33ac1538a0 2023-10-30T12:35:28.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33a3fff640 1 --2- 172.21.15.12:0/3390420262 >> v2:172.21.15.154:3300/0 conn(0x7f33ac0ca0e0 0x7f33ac15a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:28.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.694+0000 7f33aaffd640 1 --2- 172.21.15.12:0/3390420262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33ac1538a0 0x7f33ac15d6f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0b30a0 tx=0x7f339c01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8264228beea1b01d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:28.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33aa7fc640 1 -- 172.21.15.12:0/3390420262 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c0a89b0 con 0x7f33ac1538a0 2023-10-30T12:35:28.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33aa7fc640 1 -- 172.21.15.12:0/3390420262 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c0a8b10 con 0x7f33ac1538a0 2023-10-30T12:35:28.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33aa7fc640 1 -- 172.21.15.12:0/3390420262 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c0a8ea0 con 0x7f33ac1538a0 2023-10-30T12:35:28.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 -- 172.21.15.12:0/3390420262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33ac1538a0 msgr2=0x7f33ac15d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:28.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 --2- 172.21.15.12:0/3390420262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33ac1538a0 0x7f33ac15d6f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0b30a0 tx=0x7f339c01ecf0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 -- 172.21.15.12:0/3390420262 shutdown_connections 2023-10-30T12:35:28.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 --2- 172.21.15.12:0/3390420262 >> v2:172.21.15.154:3300/0 conn(0x7f33ac0ca0e0 0x7f33ac15a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 --2- 172.21.15.12:0/3390420262 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33ac0cf160 0x7f33ac0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 --2- 172.21.15.12:0/3390420262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33ac1538a0 0x7f33ac15d6f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 -- 172.21.15.12:0/3390420262 >> 172.21.15.12:0/3390420262 conn(0x7f33ac06bfe0 msgr2=0x7f33ac06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:28.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 -- 172.21.15.12:0/3390420262 shutdown_connections 2023-10-30T12:35:28.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 -- 172.21.15.12:0/3390420262 wait complete. 2023-10-30T12:35:28.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 Processor -- start 2023-10-30T12:35:28.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 -- start start 2023-10-30T12:35:28.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33ac0ca0e0 0x7f33ac144dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:28.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33ac0cf160 0x7f33ac0b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:28.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33ac1538a0 0x7f33ac1e3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:28.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac1e3b40 con 0x7f33ac0cf160 2023-10-30T12:35:28.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac145300 con 0x7f33ac1538a0 2023-10-30T12:35:28.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33b15ab640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac1464f0 con 0x7f33ac0ca0e0 2023-10-30T12:35:28.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33a3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33ac0cf160 0x7f33ac0b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:28.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33a3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33ac0cf160 0x7f33ac0b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52594/0 (socket says 172.21.15.12:52594) 2023-10-30T12:35:28.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33a3fff640 1 -- 172.21.15.12:0/1909168268 learned_addr learned my addr 172.21.15.12:0/1909168268 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:28.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.698+0000 7f33aaffd640 1 --2- 172.21.15.12:0/1909168268 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33ac0ca0e0 0x7f33ac144dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:28.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33ab7fe640 1 --2- 172.21.15.12:0/1909168268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33ac1538a0 0x7f33ac1e3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:28.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33a3fff640 1 -- 172.21.15.12:0/1909168268 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33ac0ca0e0 msgr2=0x7f33ac144dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:28.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33a3fff640 1 --2- 172.21.15.12:0/1909168268 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33ac0ca0e0 0x7f33ac144dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33a3fff640 1 -- 172.21.15.12:0/1909168268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33ac1538a0 msgr2=0x7f33ac1e3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:28.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33a3fff640 1 --2- 172.21.15.12:0/1909168268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33ac1538a0 0x7f33ac1e3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33a3fff640 1 -- 172.21.15.12:0/1909168268 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339c0094d0 con 0x7f33ac0cf160 2023-10-30T12:35:28.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33ab7fe640 1 --2- 172.21.15.12:0/1909168268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33ac1538a0 0x7f33ac1e3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:28.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33a3fff640 1 --2- 172.21.15.12:0/1909168268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33ac0cf160 0x7f33ac0b4450 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f339400e840 tx=0x7f339400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:28.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33a37fe640 1 -- 172.21.15.12:0/1909168268 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394004030 con 0x7f33ac0cf160 2023-10-30T12:35:28.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33a37fe640 1 -- 172.21.15.12:0/1909168268 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3394004190 con 0x7f33ac0cf160 2023-10-30T12:35:28.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33a37fe640 1 -- 172.21.15.12:0/1909168268 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394004520 con 0x7f33ac0cf160 2023-10-30T12:35:28.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33b15ab640 1 -- 172.21.15.12:0/1909168268 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ac15ac10 con 0x7f33ac0cf160 2023-10-30T12:35:28.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33b15ab640 1 -- 172.21.15.12:0/1909168268 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ac145c50 con 0x7f33ac0cf160 2023-10-30T12:35:28.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33b15ab640 1 -- 172.21.15.12:0/1909168268 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ac0b2a50 con 0x7f33ac0cf160 2023-10-30T12:35:28.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33a37fe640 1 -- 172.21.15.12:0/1909168268 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f339400c670 con 0x7f33ac0cf160 2023-10-30T12:35:28.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33a37fe640 1 --2- 172.21.15.12:0/1909168268 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f338c0664a0 0x7f338c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:28.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.702+0000 7f33a37fe640 1 -- 172.21.15.12:0/1909168268 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3394083a90 con 0x7f33ac0cf160 2023-10-30T12:35:28.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.706+0000 7f33aaffd640 1 --2- 172.21.15.12:0/1909168268 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f338c0664a0 0x7f338c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:28.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.706+0000 7f33aaffd640 1 --2- 172.21.15.12:0/1909168268 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f338c0664a0 0x7f338c068960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f339c0b2000 tx=0x7f339c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:28.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.706+0000 7f33a37fe640 1 -- 172.21.15.12:0/1909168268 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3394050e90 con 0x7f33ac0cf160 2023-10-30T12:35:28.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.894+0000 7f33b15ab640 1 -- 172.21.15.12:0/1909168268 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33ac1e3c80 con 0x7f33ac0cf160 2023-10-30T12:35:28.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.894+0000 7f33a37fe640 1 -- 172.21.15.12:0/1909168268 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6523 (secure 0 0 0) 0x7f33ac1e3c80 con 0x7f33ac0cf160 2023-10-30T12:35:28.899 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:28.899 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:35:27.538464+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":9,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:35:28.900 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2023-10-30T12:35:28.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.898+0000 7f33b15ab640 1 -- 172.21.15.12:0/1909168268 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f338c0664a0 msgr2=0x7f338c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:28.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.898+0000 7f33b15ab640 1 --2- 172.21.15.12:0/1909168268 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f338c0664a0 0x7f338c068960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f339c0b2000 tx=0x7f339c009040 comp rx=0 tx=0).stop 2023-10-30T12:35:28.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.898+0000 7f33b15ab640 1 -- 172.21.15.12:0/1909168268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33ac0cf160 msgr2=0x7f33ac0b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:28.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.898+0000 7f33b15ab640 1 --2- 172.21.15.12:0/1909168268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33ac0cf160 0x7f33ac0b4450 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f339400e840 tx=0x7f339400ed00 comp rx=0 tx=0).stop 2023-10-30T12:35:28.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.898+0000 7f33b15ab640 1 -- 172.21.15.12:0/1909168268 shutdown_connections 2023-10-30T12:35:28.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.898+0000 7f33b15ab640 1 --2- 172.21.15.12:0/1909168268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33ac1538a0 0x7f33ac1e3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.898+0000 7f33b15ab640 1 --2- 172.21.15.12:0/1909168268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33ac0cf160 0x7f33ac0b4450 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.898+0000 7f33b15ab640 1 --2- 172.21.15.12:0/1909168268 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f338c0664a0 0x7f338c068960 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.898+0000 7f33b15ab640 1 --2- 172.21.15.12:0/1909168268 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33ac0ca0e0 0x7f33ac144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:28.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.898+0000 7f33b15ab640 1 -- 172.21.15.12:0/1909168268 >> 172.21.15.12:0/1909168268 conn(0x7f33ac06bfe0 msgr2=0x7f33ac0c4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:28.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.898+0000 7f33b15ab640 1 -- 172.21.15.12:0/1909168268 shutdown_connections 2023-10-30T12:35:28.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:28.898+0000 7f33b15ab640 1 -- 172.21.15.12:0/1909168268 wait complete. 2023-10-30T12:35:29.469 INFO:tasks.ceph.mds.e.smithi012.stderr:2023-10-30T12:35:29.466+0000 7fbd9da0c640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-30T12:35:29.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-30T12:35:05.553867+0000', 'modified': '2023-10-30T12:35:27.538464+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4285, 'mds_1': 4282, 'mds_2': 4269}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4269': {'gid': 4269, 'name': 'c', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.12:6837/3836803994', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6834', 'nonce': 3836803994}, {'type': 'v1', 'addr': '172.21.15.12:6837', 'nonce': 3836803994}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4282': {'gid': 4282, 'name': 'd', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.154:6839/1517672481', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 1517672481}, {'type': 'v1', 'addr': '172.21.15.154:6839', 'nonce': 1517672481}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4285': {'gid': 4285, 'name': 'e', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.12:6836/291392212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6833', 'nonce': 291392212}, {'type': 'v1', 'addr': '172.21.15.12:6836', 'nonce': 291392212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-30T12:35:29.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-10-30T12:35:29.526 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-30T12:35:29.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76bf446640 1 Processor -- start 2023-10-30T12:35:29.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76bf446640 1 -- start start 2023-10-30T12:35:29.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76bf446640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76b80c6b40 0x7f76b8161be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:29.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76bf446640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f76b80c0ed0 0x7f76b806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:29.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76bf446640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b80c72b0 con 0x7f76b8158090 2023-10-30T12:35:29.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76bf446640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b8162120 con 0x7f76b80c0ed0 2023-10-30T12:35:29.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76bf446640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80c6fd0 con 0x7f76b80c6b40 2023-10-30T12:35:29.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76bd9e3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f76b80c0ed0 0x7f76b806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:29.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76bd1e2640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f76b8158090 0x7f76b8069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58140/0 (socket says 172.21.15.12:58140) 2023-10-30T12:35:29.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76bd1e2640 1 -- 172.21.15.12:0/2953859897 learned_addr learned my addr 172.21.15.12:0/2953859897 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:29.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76bc9e1640 1 --2- 172.21.15.12:0/2953859897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76b80c6b40 0x7f76b8161be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:29.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76affff640 1 -- 172.21.15.12:0/2953859897 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268302519 0 0) 0x7f76b80c72b0 con 0x7f76b8158090 2023-10-30T12:35:29.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76affff640 1 -- 172.21.15.12:0/2953859897 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76a4003530 con 0x7f76b8158090 2023-10-30T12:35:29.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76bd9e3640 1 -- 172.21.15.12:0/2953859897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76b80c6b40 msgr2=0x7f76b8161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:29.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76bd9e3640 1 --2- 172.21.15.12:0/2953859897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76b80c6b40 0x7f76b8161be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:29.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76bd9e3640 1 -- 172.21.15.12:0/2953859897 >> v1:172.21.15.12:6789/0 conn(0x7f76b8158090 legacy=0x7f76b8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:29.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.674+0000 7f76bd9e3640 1 -- 172.21.15.12:0/2953859897 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b815dd30 con 0x7f76b80c0ed0 2023-10-30T12:35:29.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.678+0000 7f76bd9e3640 1 --2- 172.21.15.12:0/2953859897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76b80c0ed0 0x7f76b806a5a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009d90 tx=0x7f76b001f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c197adda13d22615 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:29.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.678+0000 7f76affff640 1 -- 172.21.15.12:0/2953859897 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00af600 con 0x7f76b80c0ed0 2023-10-30T12:35:29.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.678+0000 7f76affff640 1 -- 172.21.15.12:0/2953859897 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b00ab050 con 0x7f76b80c0ed0 2023-10-30T12:35:29.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.678+0000 7f76affff640 1 -- 172.21.15.12:0/2953859897 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00af980 con 0x7f76b80c0ed0 2023-10-30T12:35:29.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.678+0000 7f76bf446640 1 -- 172.21.15.12:0/2953859897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76b80c0ed0 msgr2=0x7f76b806a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:29.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.678+0000 7f76bf446640 1 --2- 172.21.15.12:0/2953859897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76b80c0ed0 0x7f76b806a5a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009d90 tx=0x7f76b001f130 comp rx=0 tx=0).stop 2023-10-30T12:35:29.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.678+0000 7f76bf446640 1 -- 172.21.15.12:0/2953859897 shutdown_connections 2023-10-30T12:35:29.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.678+0000 7f76bf446640 1 --2- 172.21.15.12:0/2953859897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76b80c0ed0 0x7f76b806a5a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:29.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.678+0000 7f76bf446640 1 --2- 172.21.15.12:0/2953859897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76b80c6b40 0x7f76b8161be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:29.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.678+0000 7f76bf446640 1 -- 172.21.15.12:0/2953859897 >> 172.21.15.12:0/2953859897 conn(0x7f76b80c7910 msgr2=0x7f76b80c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:29.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.678+0000 7f76bf446640 1 -- 172.21.15.12:0/2953859897 shutdown_connections 2023-10-30T12:35:29.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.678+0000 7f76bf446640 1 -- 172.21.15.12:0/2953859897 wait complete. 2023-10-30T12:35:29.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bf446640 1 Processor -- start 2023-10-30T12:35:29.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bf446640 1 -- start start 2023-10-30T12:35:29.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bf446640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80c0ed0 0x7f76b80c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:29.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bf446640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76b8158090 0x7f76b8156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:29.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bf446640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76b80c4f10 0x7f76b81568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:29.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bf446640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b81f6790 con 0x7f76b80c4f10 2023-10-30T12:35:29.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bf446640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80c4090 con 0x7f76b80c0ed0 2023-10-30T12:35:29.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bf446640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b806bdd0 con 0x7f76b8158090 2023-10-30T12:35:29.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bd9e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76b80c4f10 0x7f76b81568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:29.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bd9e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76b80c4f10 0x7f76b81568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52622/0 (socket says 172.21.15.12:52622) 2023-10-30T12:35:29.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bd9e3640 1 -- 172.21.15.12:0/3989150227 learned_addr learned my addr 172.21.15.12:0/3989150227 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:29.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bc9e1640 1 --2- 172.21.15.12:0/3989150227 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76b8158090 0x7f76b8156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:29.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bd1e2640 1 --2- 172.21.15.12:0/3989150227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80c0ed0 0x7f76b80c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:29.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bd9e3640 1 -- 172.21.15.12:0/3989150227 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76b8158090 msgr2=0x7f76b8156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:29.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bd9e3640 1 --2- 172.21.15.12:0/3989150227 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76b8158090 0x7f76b8156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:29.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bd9e3640 1 -- 172.21.15.12:0/3989150227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80c0ed0 msgr2=0x7f76b80c21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:29.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bd9e3640 1 --2- 172.21.15.12:0/3989150227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80c0ed0 0x7f76b80c21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:29.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bd9e3640 1 -- 172.21.15.12:0/3989150227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b0009a70 con 0x7f76b80c4f10 2023-10-30T12:35:29.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bd9e3640 1 --2- 172.21.15.12:0/3989150227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76b80c4f10 0x7f76b81568b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f76b00b2040 tx=0x7f76b001bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:29.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76adffb640 1 -- 172.21.15.12:0/3989150227 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b0009450 con 0x7f76b80c4f10 2023-10-30T12:35:29.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76adffb640 1 -- 172.21.15.12:0/3989150227 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b00b9410 con 0x7f76b80c4f10 2023-10-30T12:35:29.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76adffb640 1 -- 172.21.15.12:0/3989150227 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00b97c0 con 0x7f76b80c4f10 2023-10-30T12:35:29.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bf446640 1 -- 172.21.15.12:0/3989150227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b806c070 con 0x7f76b80c4f10 2023-10-30T12:35:29.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.682+0000 7f76bf446640 1 -- 172.21.15.12:0/3989150227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b814ca70 con 0x7f76b80c4f10 2023-10-30T12:35:29.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76adffb640 1 -- 172.21.15.12:0/3989150227 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f76b00b6050 con 0x7f76b80c4f10 2023-10-30T12:35:29.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76adffb640 1 --2- 172.21.15.12:0/3989150227 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f769c0664a0 0x7f769c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:29.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76adffb640 1 -- 172.21.15.12:0/3989150227 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76b0122410 con 0x7f76b80c4f10 2023-10-30T12:35:29.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76bd1e2640 1 --2- 172.21.15.12:0/3989150227 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f769c0664a0 0x7f769c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:29.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76c27bb000 1 Processor -- start 2023-10-30T12:35:29.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76c27bb000 1 -- start start 2023-10-30T12:35:29.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76bd1e2640 1 --2- 172.21.15.12:0/3989150227 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f769c0664a0 0x7f769c068960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f76b40011f0 tx=0x7f76b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:29.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76c27bb000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671b810 0x5603c671bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:29.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76c27bb000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671c130 0x5603c671c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:29.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76c27bb000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671ca50 0x5603c6726aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:29.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76c27bb000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c64cfcc0 con 0x5603c671ca50 2023-10-30T12:35:29.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76c27bb000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c6532920 con 0x5603c671c130 2023-10-30T12:35:29.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76c27bb000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603c652fe50 con 0x5603c671b810 2023-10-30T12:35:29.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76bd9e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671ca50 0x5603c6726aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:29.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76bd9e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671ca50 0x5603c6726aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52624/0 (socket says 172.21.15.12:52624) 2023-10-30T12:35:29.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76bd9e3640 1 -- 172.21.15.12:0/2631025351 learned_addr learned my addr 172.21.15.12:0/2631025351 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:29.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76bd1e2640 1 --2- 172.21.15.12:0/2631025351 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671b810 0x5603c671bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:29.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76bc9e1640 1 --2- 172.21.15.12:0/2631025351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671c130 0x5603c671c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:29.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76bd9e3640 1 -- 172.21.15.12:0/2631025351 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671b810 msgr2=0x5603c671bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:29.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76bd9e3640 1 --2- 172.21.15.12:0/2631025351 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671b810 0x5603c671bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:29.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76bd9e3640 1 -- 172.21.15.12:0/2631025351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671c130 msgr2=0x5603c671c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:29.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76bd9e3640 1 --2- 172.21.15.12:0/2631025351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671c130 0x5603c671c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:29.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.686+0000 7f76bd9e3640 1 -- 172.21.15.12:0/2631025351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603c6224890 con 0x5603c671ca50 2023-10-30T12:35:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76bd9e3640 1 --2- 172.21.15.12:0/2631025351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671ca50 0x5603c6726aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76b001bd40 tx=0x7f76b00f3090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb3e1ceae8eae81 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f769affd640 1 -- 172.21.15.12:0/2631025351 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00fb7b0 con 0x5603c671ca50 2023-10-30T12:35:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f769affd640 1 -- 172.21.15.12:0/2631025351 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b00f17c0 con 0x5603c671ca50 2023-10-30T12:35:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f769affd640 1 -- 172.21.15.12:0/2631025351 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00efb30 con 0x5603c671ca50 2023-10-30T12:35:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 -- 172.21.15.12:0/2631025351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671ca50 msgr2=0x5603c6726aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:29.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 --2- 172.21.15.12:0/2631025351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671ca50 0x5603c6726aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76b001bd40 tx=0x7f76b00f3090 comp rx=0 tx=0).stop 2023-10-30T12:35:29.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 -- 172.21.15.12:0/2631025351 shutdown_connections 2023-10-30T12:35:29.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 --2- 172.21.15.12:0/2631025351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671ca50 0x5603c6726aa0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:29.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 --2- 172.21.15.12:0/2631025351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671c130 0x5603c671c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:29.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 --2- 172.21.15.12:0/2631025351 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671b810 0x5603c671bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:29.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 -- 172.21.15.12:0/2631025351 >> 172.21.15.12:0/2631025351 conn(0x5603c66d1fc0 msgr2=0x5603c67195d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:29.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 -- 172.21.15.12:0/2631025351 shutdown_connections 2023-10-30T12:35:29.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 -- 172.21.15.12:0/2631025351 wait complete. 2023-10-30T12:35:29.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 -- start start 2023-10-30T12:35:29.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 Processor -- start 2023-10-30T12:35:29.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671b810 0x5603c6715b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:29.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671c130 0x5603c67160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:29.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671ca50 0x5603c67165e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:29.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c652b740 con 0x5603c671b810 2023-10-30T12:35:29.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c64ead10 con 0x5603c671ca50 2023-10-30T12:35:29.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603c64ea9b0 con 0x5603c671c130 2023-10-30T12:35:29.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76c27bb000 10 client.0 _ll_register_callbacks cb 0x5603c66c6390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-30T12:35:29.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76bd1e2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671b810 0x5603c6715b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:29.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76bd1e2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671b810 0x5603c6715b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52640/0 (socket says 172.21.15.12:52640) 2023-10-30T12:35:29.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76bd1e2640 1 -- 172.21.15.12:0/1043635149 learned_addr learned my addr 172.21.15.12:0/1043635149 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:29.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76bc9e1640 1 --2- 172.21.15.12:0/1043635149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671c130 0x5603c67160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:29.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76bd9e3640 1 --2- 172.21.15.12:0/1043635149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671ca50 0x5603c67165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:29.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76bd1e2640 1 -- 172.21.15.12:0/1043635149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671c130 msgr2=0x5603c67160a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:29.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76bd1e2640 1 --2- 172.21.15.12:0/1043635149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671c130 0x5603c67160a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:29.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76bd1e2640 1 -- 172.21.15.12:0/1043635149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671ca50 msgr2=0x5603c67165e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:29.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76bd1e2640 1 --2- 172.21.15.12:0/1043635149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671ca50 0x5603c67165e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:29.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76bd1e2640 1 -- 172.21.15.12:0/1043635149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b814ca70 con 0x5603c671b810 2023-10-30T12:35:29.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f76bd1e2640 1 --2- 172.21.15.12:0/1043635149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671b810 0x5603c6715b60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76b4005ec0 tx=0x7f76b4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b3a806f6ecdb1b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:29.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.690+0000 7f7693fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-10-30T12:35:29.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f7693fff640 1 -- 172.21.15.12:0/1043635149 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4001d90 con 0x5603c671b810 2023-10-30T12:35:29.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f7693fff640 1 -- 172.21.15.12:0/1043635149 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76b400ec50 con 0x5603c671b810 2023-10-30T12:35:29.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f76c27bb000 1 -- 172.21.15.12:0/1043635149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603c64df370 con 0x5603c671b810 2023-10-30T12:35:29.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f7693fff640 1 -- 172.21.15.12:0/1043635149 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76b4007770 con 0x5603c671b810 2023-10-30T12:35:29.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f76c27bb000 10 client.4662 fetch_fsmap learned FSMap version 13 2023-10-30T12:35:29.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f76c27bb000 1 -- 172.21.15.12:0/1043635149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x5603c64ed460 con 0x5603c671b810 2023-10-30T12:35:29.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f7693fff640 1 -- 172.21.15.12:0/1043635149 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5603c64df370 con 0x5603c671b810 2023-10-30T12:35:29.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f7693fff640 1 -- 172.21.15.12:0/1043635149 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 13) v1 ==== 4468+0+0 (secure 0 0 0) 0x7f76b4023610 con 0x5603c671b810 2023-10-30T12:35:29.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f76c27bb000 10 client.4662 fetch_fsmap finished waiting for FSMap version 13 2023-10-30T12:35:29.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f76c27bb000 10 client.4662 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-30T12:35:29.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f76c27bb000 20 client.4662 populate_metadata read hostname 'smithi012' 2023-10-30T12:35:29.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f76c27bb000 1 --2- 172.21.15.12:0/1043635149 >> [v2:172.21.15.12:6834/3836803994,v1:172.21.15.12:6837/3836803994] conn(0x5603c672f260 0x5603c6731650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:29.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f76c27bb000 4 client.4662 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-30T12:35:29.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f76c27bb000 1 -- 172.21.15.12:0/1043635149 --> [v2:172.21.15.12:6834/3836803994,v1:172.21.15.12:6837/3836803994] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603c64c52d0 con 0x5603c672f260 2023-10-30T12:35:29.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f76bc9e1640 1 --2- 172.21.15.12:0/1043635149 >> [v2:172.21.15.12:6834/3836803994,v1:172.21.15.12:6837/3836803994] conn(0x5603c672f260 0x5603c6731650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:29.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f7693fff640 10 client.4662 ms_handle_connect on v2:172.21.15.12:6834/3836803994 2023-10-30T12:35:29.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:29.694+0000 7f76bc9e1640 1 --2- 172.21.15.12:0/1043635149 >> [v2:172.21.15.12:6834/3836803994,v1:172.21.15.12:6837/3836803994] conn(0x5603c672f260 0x5603c6731650 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48144fc93ab43ba2 server_cookie=cb92c57073790553 in_seq=0 out_seq=0 2023-10-30T12:35:33.160 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:35:33.165 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:35:33.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.386+0000 7f7693fff640 1 -- 172.21.15.12:0/1043635149 <== mds.2 v2:172.21.15.12:6834/3836803994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5603c64c52d0 con 0x5603c672f260 2023-10-30T12:35:33.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.386+0000 7f7693fff640 10 client.4662 handle_command_reply: tid=0 2023-10-30T12:35:33.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.386+0000 7f76c27bb000 1 client.4662 shutdown 2023-10-30T12:35:33.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.386+0000 7f76c27bb000 1 -- 172.21.15.12:0/1043635149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671b810 msgr2=0x5603c6715b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.386+0000 7f76c27bb000 1 --2- 172.21.15.12:0/1043635149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671b810 0x5603c6715b60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76b4005ec0 tx=0x7f76b4000ee0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.386+0000 7f76c27bb000 1 -- 172.21.15.12:0/1043635149 shutdown_connections 2023-10-30T12:35:33.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.386+0000 7f76c27bb000 1 --2- 172.21.15.12:0/1043635149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671ca50 0x5603c67165e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.386+0000 7f76c27bb000 1 --2- 172.21.15.12:0/1043635149 >> [v2:172.21.15.12:6834/3836803994,v1:172.21.15.12:6837/3836803994] conn(0x5603c672f260 0x5603c6731650 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.386+0000 7f76c27bb000 1 --2- 172.21.15.12:0/1043635149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671c130 0x5603c67160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.386+0000 7f76c27bb000 1 --2- 172.21.15.12:0/1043635149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671b810 0x5603c6715b60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.386+0000 7f7693fff640 0 client.4662 ms_handle_reset on v2:172.21.15.12:6834/3836803994 2023-10-30T12:35:33.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.386+0000 7f76c27bb000 1 -- 172.21.15.12:0/1043635149 >> 172.21.15.12:0/1043635149 conn(0x5603c66d1fc0 msgr2=0x5603c671b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.386+0000 7f76c27bb000 1 -- 172.21.15.12:0/1043635149 shutdown_connections 2023-10-30T12:35:33.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.386+0000 7f76c27bb000 1 -- 172.21.15.12:0/1043635149 wait complete. 2023-10-30T12:35:33.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.386+0000 7f76c27bb000 20 client.4662 trim_cache size 0 max 16384 2023-10-30T12:35:33.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.402+0000 7f76c27bb000 1 Processor -- start 2023-10-30T12:35:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.402+0000 7f76c27bb000 1 -- start start 2023-10-30T12:35:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671b810 0x5603c6776fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671c130 0x5603c6777500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671ca50 0x5603c671ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c61e5940 con 0x5603c671ca50 2023-10-30T12:35:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c6528b40 con 0x5603c671c130 2023-10-30T12:35:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603c671b340 con 0x5603c671b810 2023-10-30T12:35:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76bd9e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671ca50 0x5603c671ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76bd9e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671ca50 0x5603c671ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57112/0 (socket says 172.21.15.12:57112) 2023-10-30T12:35:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76bd9e3640 1 -- 172.21.15.12:0/410893781 learned_addr learned my addr 172.21.15.12:0/410893781 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76bd1e2640 1 --2- 172.21.15.12:0/410893781 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671b810 0x5603c6776fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76bc9e1640 1 --2- 172.21.15.12:0/410893781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671c130 0x5603c6777500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76bd9e3640 1 -- 172.21.15.12:0/410893781 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671b810 msgr2=0x5603c6776fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76bd9e3640 1 --2- 172.21.15.12:0/410893781 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671b810 0x5603c6776fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76bd9e3640 1 -- 172.21.15.12:0/410893781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671c130 msgr2=0x5603c6777500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76bd9e3640 1 --2- 172.21.15.12:0/410893781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671c130 0x5603c6777500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76bd9e3640 1 -- 172.21.15.12:0/410893781 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603c672df50 con 0x5603c671ca50 2023-10-30T12:35:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76bd9e3640 1 --2- 172.21.15.12:0/410893781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671ca50 0x5603c671ce30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f76b00ef920 tx=0x7f76b00c9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d38c2e8fe7ca0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f769b7fe640 1 -- 172.21.15.12:0/410893781 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00f31f0 con 0x5603c671ca50 2023-10-30T12:35:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f769b7fe640 1 -- 172.21.15.12:0/410893781 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b00fb7b0 con 0x5603c671ca50 2023-10-30T12:35:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f769b7fe640 1 -- 172.21.15.12:0/410893781 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b01220c0 con 0x5603c671ca50 2023-10-30T12:35:33.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 -- 172.21.15.12:0/410893781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671ca50 msgr2=0x5603c671ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 --2- 172.21.15.12:0/410893781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671ca50 0x5603c671ce30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f76b00ef920 tx=0x7f76b00c9000 comp rx=0 tx=0).stop 2023-10-30T12:35:33.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 -- 172.21.15.12:0/410893781 shutdown_connections 2023-10-30T12:35:33.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 --2- 172.21.15.12:0/410893781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671ca50 0x5603c671ce30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 --2- 172.21.15.12:0/410893781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671c130 0x5603c6777500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 --2- 172.21.15.12:0/410893781 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671b810 0x5603c6776fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 -- 172.21.15.12:0/410893781 >> 172.21.15.12:0/410893781 conn(0x5603c66d1fc0 msgr2=0x5603c66e0490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 -- 172.21.15.12:0/410893781 shutdown_connections 2023-10-30T12:35:33.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 -- 172.21.15.12:0/410893781 wait complete. 2023-10-30T12:35:33.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 -- start start 2023-10-30T12:35:33.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.406+0000 7f76c27bb000 1 Processor -- start 2023-10-30T12:35:33.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76c27bb000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671b810 0x5603c677f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76c27bb000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671c130 0x5603c677f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76c27bb000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671ca50 0x5603c673df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76c27bb000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c6501a10 con 0x5603c671c130 2023-10-30T12:35:33.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76c27bb000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c652f740 con 0x5603c671b810 2023-10-30T12:35:33.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76c27bb000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603c64f2d10 con 0x5603c671ca50 2023-10-30T12:35:33.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76c27bb000 10 client.0 _ll_register_callbacks cb 0x7f76b00ef760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-30T12:35:33.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76bc9e1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671c130 0x5603c677f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76bc9e1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671c130 0x5603c677f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57126/0 (socket says 172.21.15.12:57126) 2023-10-30T12:35:33.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76bc9e1640 1 -- 172.21.15.12:0/2581202154 learned_addr learned my addr 172.21.15.12:0/2581202154 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:33.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76bd1e2640 1 --2- 172.21.15.12:0/2581202154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671b810 0x5603c677f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76bd9e3640 1 --2- 172.21.15.12:0/2581202154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671ca50 0x5603c673df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76bc9e1640 1 -- 172.21.15.12:0/2581202154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671ca50 msgr2=0x5603c673df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76bc9e1640 1 --2- 172.21.15.12:0/2581202154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671ca50 0x5603c673df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76bc9e1640 1 -- 172.21.15.12:0/2581202154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671b810 msgr2=0x5603c677f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76bc9e1640 1 --2- 172.21.15.12:0/2581202154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671b810 0x5603c677f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76bc9e1640 1 -- 172.21.15.12:0/2581202154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603c6224890 con 0x5603c671c130 2023-10-30T12:35:33.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76bd9e3640 1 --2- 172.21.15.12:0/2581202154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671ca50 0x5603c673df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:33.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76bc9e1640 1 --2- 172.21.15.12:0/2581202154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671c130 0x5603c677f5a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f76a8000c10 tx=0x7f76a801d510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7aedf67751c3ec server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f7693fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-10-30T12:35:33.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f7693fff640 1 -- 172.21.15.12:0/2581202154 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8004230 con 0x5603c671c130 2023-10-30T12:35:33.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f7693fff640 1 -- 172.21.15.12:0/2581202154 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76a8004390 con 0x5603c671c130 2023-10-30T12:35:33.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76c27bb000 1 -- 172.21.15.12:0/2581202154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603c64c36f0 con 0x5603c671c130 2023-10-30T12:35:33.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f7693fff640 1 -- 172.21.15.12:0/2581202154 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76a8003620 con 0x5603c671c130 2023-10-30T12:35:33.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76c27bb000 10 client.4668 fetch_fsmap learned FSMap version 13 2023-10-30T12:35:33.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f76c27bb000 1 -- 172.21.15.12:0/2581202154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x5603c64d3760 con 0x5603c671c130 2023-10-30T12:35:33.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.410+0000 7f7693fff640 1 -- 172.21.15.12:0/2581202154 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5603c64c36f0 con 0x5603c671c130 2023-10-30T12:35:33.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f7693fff640 1 -- 172.21.15.12:0/2581202154 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 13) v1 ==== 4468+0+0 (secure 0 0 0) 0x7f76a8003780 con 0x5603c671c130 2023-10-30T12:35:33.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 10 client.4668 fetch_fsmap finished waiting for FSMap version 13 2023-10-30T12:35:33.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 10 client.4668 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-30T12:35:33.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 20 client.4668 populate_metadata read hostname 'smithi012' 2023-10-30T12:35:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 1 --2- 172.21.15.12:0/2581202154 >> [v2:172.21.15.12:6834/3836803994,v1:172.21.15.12:6837/3836803994] conn(0x5603c6785ec0 0x5603c673ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 4 client.4668 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-30T12:35:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 1 -- 172.21.15.12:0/2581202154 --> [v2:172.21.15.12:6834/3836803994,v1:172.21.15.12:6837/3836803994] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5603c6524720 con 0x5603c6785ec0 2023-10-30T12:35:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76bd1e2640 1 --2- 172.21.15.12:0/2581202154 >> [v2:172.21.15.12:6834/3836803994,v1:172.21.15.12:6837/3836803994] conn(0x5603c6785ec0 0x5603c673ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76bd1e2640 1 --2- 172.21.15.12:0/2581202154 >> [v2:172.21.15.12:6834/3836803994,v1:172.21.15.12:6837/3836803994] conn(0x5603c6785ec0 0x5603c673ee70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3173f4ef21dbd01c server_cookie=fce50567cbfb9d9b in_seq=0 out_seq=0 2023-10-30T12:35:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f7693fff640 10 client.4668 ms_handle_connect on v2:172.21.15.12:6834/3836803994 2023-10-30T12:35:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f7693fff640 1 -- 172.21.15.12:0/2581202154 <== mds.2 v2:172.21.15.12:6834/3836803994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5603c6524720 con 0x5603c6785ec0 2023-10-30T12:35:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f7693fff640 10 client.4668 handle_command_reply: tid=0 2023-10-30T12:35:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 1 client.4668 shutdown 2023-10-30T12:35:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 1 -- 172.21.15.12:0/2581202154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671c130 msgr2=0x5603c677f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 1 --2- 172.21.15.12:0/2581202154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671c130 0x5603c677f5a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f76a8000c10 tx=0x7f76a801d510 comp rx=0 tx=0).stop 2023-10-30T12:35:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 1 -- 172.21.15.12:0/2581202154 shutdown_connections 2023-10-30T12:35:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 1 --2- 172.21.15.12:0/2581202154 >> [v2:172.21.15.12:6834/3836803994,v1:172.21.15.12:6837/3836803994] conn(0x5603c6785ec0 0x5603c673ee70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 1 --2- 172.21.15.12:0/2581202154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x5603c671ca50 0x5603c673df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 1 --2- 172.21.15.12:0/2581202154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603c671c130 0x5603c677f5a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f7693fff640 0 client.4668 ms_handle_reset on v2:172.21.15.12:6834/3836803994 2023-10-30T12:35:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 1 --2- 172.21.15.12:0/2581202154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5603c671b810 0x5603c677f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 1 -- 172.21.15.12:0/2581202154 >> 172.21.15.12:0/2581202154 conn(0x5603c66d1fc0 msgr2=0x5603c6721290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 1 -- 172.21.15.12:0/2581202154 shutdown_connections 2023-10-30T12:35:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 1 -- 172.21.15.12:0/2581202154 wait complete. 2023-10-30T12:35:33.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76c27bb000 20 client.4668 trim_cache size 0 max 16384 2023-10-30T12:35:33.432 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-30T12:35:33.432 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "97b69bd8-a5ea-4668-9ffd-462651b43fe5", 2023-10-30T12:35:33.432 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2023-10-30T12:35:33.433 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 4269, 2023-10-30T12:35:33.433 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2023-10-30T12:35:33.433 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2023-10-30T12:35:33.433 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2023-10-30T12:35:33.433 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 11.095124915, 2023-10-30T12:35:33.434 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 13, 2023-10-30T12:35:33.434 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 22, 2023-10-30T12:35:33.434 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 22, 2023-10-30T12:35:33.434 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 34.972006059000002 2023-10-30T12:35:33.434 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-30T12:35:33.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76bf446640 1 -- 172.21.15.12:0/3989150227 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f769c0664a0 msgr2=0x7f769c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76bf446640 1 --2- 172.21.15.12:0/3989150227 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f769c0664a0 0x7f769c068960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f76b40011f0 tx=0x7f76b4009040 comp rx=0 tx=0).stop 2023-10-30T12:35:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76bf446640 1 -- 172.21.15.12:0/3989150227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76b80c4f10 msgr2=0x7f76b81568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76bf446640 1 --2- 172.21.15.12:0/3989150227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76b80c4f10 0x7f76b81568b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f76b00b2040 tx=0x7f76b001bed0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76bf446640 1 -- 172.21.15.12:0/3989150227 shutdown_connections 2023-10-30T12:35:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76bf446640 1 --2- 172.21.15.12:0/3989150227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76b80c4f10 0x7f76b81568b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76bf446640 1 --2- 172.21.15.12:0/3989150227 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f769c0664a0 0x7f769c068960 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76bf446640 1 --2- 172.21.15.12:0/3989150227 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76b8158090 0x7f76b8156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76bf446640 1 --2- 172.21.15.12:0/3989150227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80c0ed0 0x7f76b80c21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76bf446640 1 -- 172.21.15.12:0/3989150227 >> 172.21.15.12:0/3989150227 conn(0x7f76b80c7910 msgr2=0x7f76b80c12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76bf446640 1 -- 172.21.15.12:0/3989150227 shutdown_connections 2023-10-30T12:35:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.414+0000 7f76bf446640 1 -- 172.21.15.12:0/3989150227 wait complete. 2023-10-30T12:35:33.442 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-30T12:35:33.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.586+0000 7f4072017640 1 Processor -- start 2023-10-30T12:35:33.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.586+0000 7f4072017640 1 -- start start 2023-10-30T12:35:33.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f4072017640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f406c06e430 0x7f406c06ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f4072017640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406c06d710 con 0x7f406c0cce00 2023-10-30T12:35:33.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f4072017640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406c06b560 con 0x7f406c0c7830 2023-10-30T12:35:33.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f4072017640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406c06dd30 con 0x7f406c06e430 2023-10-30T12:35:33.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406b7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f406c0cce00 0x7f406c159850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54408/0 (socket says 172.21.15.12:54408) 2023-10-30T12:35:33.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406b7fe640 1 -- 172.21.15.12:0/4221040459 learned_addr learned my addr 172.21.15.12:0/4221040459 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406bfff640 1 --2- 172.21.15.12:0/4221040459 >> v2:172.21.15.154:3300/0 conn(0x7f406c06e430 0x7f406c06ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437427873 0 0) 0x7f406c06d710 con 0x7f406c0cce00 2023-10-30T12:35:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4054003530 con 0x7f406c0cce00 2023-10-30T12:35:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552072568 0 0) 0x7f406c06b560 con 0x7f406c0c7830 2023-10-30T12:35:33.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f406c06d710 con 0x7f406c0c7830 2023-10-30T12:35:33.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1312233797 0 0) 0x7f4054003530 con 0x7f406c0cce00 2023-10-30T12:35:33.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 >> v2:172.21.15.154:3300/0 conn(0x7f406c06e430 msgr2=0x7f406c06ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 --2- 172.21.15.12:0/4221040459 >> v2:172.21.15.154:3300/0 conn(0x7f406c06e430 0x7f406c06ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 >> v1:172.21.15.154:6789/0 conn(0x7f406c0c7830 legacy=0x7f406c159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c0c3c80 con 0x7f406c0cce00 2023-10-30T12:35:33.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f405c001680 con 0x7f406c0cce00 2023-10-30T12:35:33.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 >> v1:172.21.15.12:6789/0 conn(0x7f406c0cce00 legacy=0x7f406c159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 --2- 172.21.15.12:0/4221040459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4054003b10 0x7f4054003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 --2- 172.21.15.12:0/4221040459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40540047d0 0x7f4054004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 --2- 172.21.15.12:0/4221040459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4054005190 0x7f405400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c06dd30 con 0x7f4054003b10 2023-10-30T12:35:33.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405400c0c0 con 0x7f4054005190 2023-10-30T12:35:33.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405400c280 con 0x7f40540047d0 2023-10-30T12:35:33.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406b7fe640 1 --2- 172.21.15.12:0/4221040459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4054003b10 0x7f4054003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406bfff640 1 --2- 172.21.15.12:0/4221040459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4054005190 0x7f405400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406affd640 1 --2- 172.21.15.12:0/4221040459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40540047d0 0x7f4054004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406b7fe640 1 -- 172.21.15.12:0/4221040459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40540047d0 msgr2=0x7f4054004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406b7fe640 1 --2- 172.21.15.12:0/4221040459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40540047d0 0x7f4054004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406b7fe640 1 -- 172.21.15.12:0/4221040459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4054005190 msgr2=0x7f405400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406b7fe640 1 --2- 172.21.15.12:0/4221040459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4054005190 0x7f405400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406b7fe640 1 -- 172.21.15.12:0/4221040459 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4054003530 con 0x7f4054003b10 2023-10-30T12:35:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406bfff640 1 --2- 172.21.15.12:0/4221040459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4054005190 0x7f405400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.590+0000 7f406bfff640 1 -- 172.21.15.12:0/4221040459 reap_dead start 2023-10-30T12:35:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.594+0000 7f406b7fe640 1 --2- 172.21.15.12:0/4221040459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4054003b10 0x7f4054003f10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f405c00c190 tx=0x7f405c01cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21df0568ea3cafbf server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.594+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c0058e0 con 0x7f4054003b10 2023-10-30T12:35:33.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.594+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c005a40 con 0x7f4054003b10 2023-10-30T12:35:33.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.594+0000 7f406a7fc640 1 -- 172.21.15.12:0/4221040459 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c005e50 con 0x7f4054003b10 2023-10-30T12:35:33.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.594+0000 7f4072017640 1 -- 172.21.15.12:0/4221040459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4054003b10 msgr2=0x7f4054003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.594+0000 7f4072017640 1 --2- 172.21.15.12:0/4221040459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4054003b10 0x7f4054003f10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f405c00c190 tx=0x7f405c01cbc0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.594+0000 7f4072017640 1 -- 172.21.15.12:0/4221040459 shutdown_connections 2023-10-30T12:35:33.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.594+0000 7f4072017640 1 --2- 172.21.15.12:0/4221040459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4054003b10 0x7f4054003f10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.594+0000 7f4072017640 1 -- 172.21.15.12:0/4221040459 >> 172.21.15.12:0/4221040459 conn(0x7f406c071ea0 msgr2=0x7f406c065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.594+0000 7f4072017640 1 -- 172.21.15.12:0/4221040459 shutdown_connections 2023-10-30T12:35:33.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.594+0000 7f4072017640 1 -- 172.21.15.12:0/4221040459 wait complete. 2023-10-30T12:35:33.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f4072017640 1 Processor -- start 2023-10-30T12:35:33.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f4072017640 1 -- start start 2023-10-30T12:35:33.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f4072017640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4054003b10 0x7f406c1f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f4072017640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c15a9f0 0x7f406c0c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f4072017640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f406c1f2c70 0x7f406c14b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f4072017640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c0c8d60 con 0x7f406c15a9f0 2023-10-30T12:35:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f4072017640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c15b5c0 con 0x7f406c1f2c70 2023-10-30T12:35:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f4072017640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406c15b8a0 con 0x7f4054003b10 2023-10-30T12:35:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f406affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c15a9f0 0x7f406c0c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f406affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c15a9f0 0x7f406c0c32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57138/0 (socket says 172.21.15.12:57138) 2023-10-30T12:35:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f406affd640 1 -- 172.21.15.12:0/1571375400 learned_addr learned my addr 172.21.15.12:0/1571375400 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f406b7fe640 1 --2- 172.21.15.12:0/1571375400 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4054003b10 0x7f406c1f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f406bfff640 1 --2- 172.21.15.12:0/1571375400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f406c1f2c70 0x7f406c14b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f406affd640 1 -- 172.21.15.12:0/1571375400 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4054003b10 msgr2=0x7f406c1f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f406affd640 1 --2- 172.21.15.12:0/1571375400 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4054003b10 0x7f406c1f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f406affd640 1 -- 172.21.15.12:0/1571375400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f406c1f2c70 msgr2=0x7f406c14b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f406affd640 1 --2- 172.21.15.12:0/1571375400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f406c1f2c70 0x7f406c14b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f406affd640 1 -- 172.21.15.12:0/1571375400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c15bb80 con 0x7f406c15a9f0 2023-10-30T12:35:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f406affd640 1 --2- 172.21.15.12:0/1571375400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c15a9f0 0x7f406c0c32c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f406000e7b0 tx=0x7f406000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f4053fff640 1 -- 172.21.15.12:0/1571375400 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406000ee60 con 0x7f406c15a9f0 2023-10-30T12:35:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f4053fff640 1 -- 172.21.15.12:0/1571375400 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40600052e0 con 0x7f406c15a9f0 2023-10-30T12:35:33.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f4053fff640 1 -- 172.21.15.12:0/1571375400 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406000c900 con 0x7f406c15a9f0 2023-10-30T12:35:33.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f4072017640 1 -- 172.21.15.12:0/1571375400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406c1f2470 con 0x7f406c15a9f0 2023-10-30T12:35:33.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.598+0000 7f4072017640 1 -- 172.21.15.12:0/1571375400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406c15be60 con 0x7f406c15a9f0 2023-10-30T12:35:33.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f4053fff640 1 -- 172.21.15.12:0/1571375400 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f406000ca60 con 0x7f406c15a9f0 2023-10-30T12:35:33.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f4053fff640 1 --2- 172.21.15.12:0/1571375400 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f40580664d0 0x7f4058068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f4053fff640 1 -- 172.21.15.12:0/1571375400 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4060083d80 con 0x7f406c15a9f0 2023-10-30T12:35:33.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f40753aa000 1 Processor -- start 2023-10-30T12:35:33.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f40753aa000 1 -- start start 2023-10-30T12:35:33.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f406b7fe640 1 --2- 172.21.15.12:0/1571375400 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f40580664d0 0x7f4058068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f40753aa000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 0x55e993020bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f40753aa000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e993021510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f40753aa000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e99302baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f40753aa000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e992dd4cc0 con 0x55e993021130 2023-10-30T12:35:33.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f40753aa000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e992e37920 con 0x55e993020810 2023-10-30T12:35:33.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f40753aa000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e992e34e50 con 0x55e993021a50 2023-10-30T12:35:33.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f406b7fe640 1 --2- 172.21.15.12:0/1571375400 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f40580664d0 0x7f4058068990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f405c0aa000 tx=0x7f406c06c970 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f406affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e993021510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f406affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e993021510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57150/0 (socket says 172.21.15.12:57150) 2023-10-30T12:35:33.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.602+0000 7f406affd640 1 -- 172.21.15.12:0/4210726622 learned_addr learned my addr 172.21.15.12:0/4210726622 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:33.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f406b7fe640 1 --2- 172.21.15.12:0/4210726622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 0x55e993020bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f406bfff640 1 --2- 172.21.15.12:0/4210726622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e99302baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f406affd640 1 -- 172.21.15.12:0/4210726622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 msgr2=0x55e99302baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f406affd640 1 --2- 172.21.15.12:0/4210726622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e99302baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f406affd640 1 -- 172.21.15.12:0/4210726622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 msgr2=0x55e993020bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f406affd640 1 --2- 172.21.15.12:0/4210726622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 0x55e993020bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f406affd640 1 -- 172.21.15.12:0/4210726622 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e992b29890 con 0x55e993021130 2023-10-30T12:35:33.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f406b7fe640 1 --2- 172.21.15.12:0/4210726622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 0x55e993020bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f406affd640 1 --2- 172.21.15.12:0/4210726622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e993021510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f40600050a0 tx=0x7f4060054860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f120e0c98a0ada server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40517fa640 1 -- 172.21.15.12:0/4210726622 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406007d070 con 0x55e993021130 2023-10-30T12:35:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40517fa640 1 -- 172.21.15.12:0/4210726622 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406005d120 con 0x55e993021130 2023-10-30T12:35:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40517fa640 1 -- 172.21.15.12:0/4210726622 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060058070 con 0x55e993021130 2023-10-30T12:35:33.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 -- 172.21.15.12:0/4210726622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 msgr2=0x55e993021510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 --2- 172.21.15.12:0/4210726622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e993021510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f40600050a0 tx=0x7f4060054860 comp rx=0 tx=0).stop 2023-10-30T12:35:33.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 -- 172.21.15.12:0/4210726622 shutdown_connections 2023-10-30T12:35:33.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 --2- 172.21.15.12:0/4210726622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e99302baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 --2- 172.21.15.12:0/4210726622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e993021510 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 --2- 172.21.15.12:0/4210726622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 0x55e993020bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 -- 172.21.15.12:0/4210726622 >> 172.21.15.12:0/4210726622 conn(0x55e992fd6fc0 msgr2=0x55e99301e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 -- 172.21.15.12:0/4210726622 shutdown_connections 2023-10-30T12:35:33.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 -- 172.21.15.12:0/4210726622 wait complete. 2023-10-30T12:35:33.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 -- start start 2023-10-30T12:35:33.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 Processor -- start 2023-10-30T12:35:33.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993020810 0x55e99301ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993021130 0x55e99301b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e99301b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e992e30740 con 0x55e993020810 2023-10-30T12:35:33.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e992defd10 con 0x55e993021130 2023-10-30T12:35:33.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e992def9b0 con 0x55e993021a50 2023-10-30T12:35:33.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f40753aa000 10 client.0 _ll_register_callbacks cb 0x55e992fcb390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-30T12:35:33.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f406b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993020810 0x55e99301ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f406b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993020810 0x55e99301ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57158/0 (socket says 172.21.15.12:57158) 2023-10-30T12:35:33.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f406b7fe640 1 -- 172.21.15.12:0/808612483 learned_addr learned my addr 172.21.15.12:0/808612483 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:33.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f406bfff640 1 --2- 172.21.15.12:0/808612483 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e99301b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.606+0000 7f406affd640 1 --2- 172.21.15.12:0/808612483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993021130 0x55e99301b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f406b7fe640 1 -- 172.21.15.12:0/808612483 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 msgr2=0x55e99301b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f406b7fe640 1 --2- 172.21.15.12:0/808612483 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e99301b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f406b7fe640 1 -- 172.21.15.12:0/808612483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993021130 msgr2=0x55e99301b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f406b7fe640 1 --2- 172.21.15.12:0/808612483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993021130 0x55e99301b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f406b7fe640 1 -- 172.21.15.12:0/808612483 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406c15be60 con 0x55e993020810 2023-10-30T12:35:33.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f406b7fe640 1 --2- 172.21.15.12:0/808612483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993020810 0x55e99301ab60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f405c01c9f0 tx=0x7f406c15a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a837aa1b1c41ebe server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f404a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-10-30T12:35:33.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f404a7fc640 1 -- 172.21.15.12:0/808612483 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c005cf0 con 0x55e993020810 2023-10-30T12:35:33.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f40753aa000 1 -- 172.21.15.12:0/808612483 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e992de4370 con 0x55e993020810 2023-10-30T12:35:33.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f404a7fc640 1 -- 172.21.15.12:0/808612483 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f405c0070d0 con 0x55e993020810 2023-10-30T12:35:33.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f404a7fc640 1 -- 172.21.15.12:0/808612483 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f405c005e50 con 0x55e993020810 2023-10-30T12:35:33.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f40753aa000 10 client.4680 fetch_fsmap learned FSMap version 13 2023-10-30T12:35:33.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f40753aa000 1 -- 172.21.15.12:0/808612483 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55e992df2460 con 0x55e993020810 2023-10-30T12:35:33.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f404a7fc640 1 -- 172.21.15.12:0/808612483 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c0ca020 con 0x55e993020810 2023-10-30T12:35:33.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f404a7fc640 1 -- 172.21.15.12:0/808612483 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 13) v1 ==== 4468+0+0 (secure 0 0 0) 0x7f405c007290 con 0x55e993020810 2023-10-30T12:35:33.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f40753aa000 10 client.4680 fetch_fsmap finished waiting for FSMap version 13 2023-10-30T12:35:33.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f40753aa000 10 client.4680 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-30T12:35:33.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f40753aa000 20 client.4680 populate_metadata read hostname 'smithi012' 2023-10-30T12:35:33.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f40753aa000 1 --2- 172.21.15.12:0/808612483 >> [v2:172.21.15.154:6836/1517672481,v1:172.21.15.154:6839/1517672481] conn(0x55e993034260 0x55e993036650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f40753aa000 4 client.4680 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-30T12:35:33.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f40753aa000 1 -- 172.21.15.12:0/808612483 --> [v2:172.21.15.154:6836/1517672481,v1:172.21.15.154:6839/1517672481] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e992dca2d0 con 0x55e993034260 2023-10-30T12:35:33.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f406affd640 1 --2- 172.21.15.12:0/808612483 >> [v2:172.21.15.154:6836/1517672481,v1:172.21.15.154:6839/1517672481] conn(0x55e993034260 0x55e993036650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f406affd640 1 --2- 172.21.15.12:0/808612483 >> [v2:172.21.15.154:6836/1517672481,v1:172.21.15.154:6839/1517672481] conn(0x55e993034260 0x55e993036650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d897f5005e56f22 server_cookie=8f2686c8181dac31 in_seq=0 out_seq=0 2023-10-30T12:35:33.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.610+0000 7f404a7fc640 10 client.4680 ms_handle_connect on v2:172.21.15.154:6836/1517672481 2023-10-30T12:35:33.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.614+0000 7f404a7fc640 1 -- 172.21.15.12:0/808612483 <== mds.1 v2:172.21.15.154:6836/1517672481 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e992dca2d0 con 0x55e993034260 2023-10-30T12:35:33.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.614+0000 7f404a7fc640 10 client.4680 handle_command_reply: tid=0 2023-10-30T12:35:33.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.614+0000 7f40753aa000 1 client.4680 shutdown 2023-10-30T12:35:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.614+0000 7f40753aa000 1 -- 172.21.15.12:0/808612483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993020810 msgr2=0x55e99301ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.614+0000 7f40753aa000 1 --2- 172.21.15.12:0/808612483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993020810 0x55e99301ab60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f405c01c9f0 tx=0x7f406c15a860 comp rx=0 tx=0).stop 2023-10-30T12:35:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.614+0000 7f40753aa000 1 -- 172.21.15.12:0/808612483 shutdown_connections 2023-10-30T12:35:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.614+0000 7f40753aa000 1 --2- 172.21.15.12:0/808612483 >> [v2:172.21.15.154:6836/1517672481,v1:172.21.15.154:6839/1517672481] conn(0x55e993034260 0x55e993036650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.614+0000 7f40753aa000 1 --2- 172.21.15.12:0/808612483 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e99301b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.614+0000 7f40753aa000 1 --2- 172.21.15.12:0/808612483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993021130 0x55e99301b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.614+0000 7f40753aa000 1 --2- 172.21.15.12:0/808612483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993020810 0x55e99301ab60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.614+0000 7f40753aa000 1 -- 172.21.15.12:0/808612483 >> 172.21.15.12:0/808612483 conn(0x55e992fd6fc0 msgr2=0x55e993020090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.614+0000 7f404a7fc640 0 client.4680 ms_handle_reset on v2:172.21.15.154:6836/1517672481 2023-10-30T12:35:33.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.614+0000 7f40753aa000 1 -- 172.21.15.12:0/808612483 shutdown_connections 2023-10-30T12:35:33.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.614+0000 7f40753aa000 1 -- 172.21.15.12:0/808612483 wait complete. 2023-10-30T12:35:33.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.614+0000 7f40753aa000 20 client.4680 trim_cache size 0 max 16384 2023-10-30T12:35:33.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 Processor -- start 2023-10-30T12:35:33.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 -- start start 2023-10-30T12:35:33.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 0x55e99307fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e993080340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e993080880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e992aea940 con 0x55e993021130 2023-10-30T12:35:33.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99301fc60 con 0x55e993020810 2023-10-30T12:35:33.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e992e36e60 con 0x55e993021a50 2023-10-30T12:35:33.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f406affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e993080340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f406affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e993080340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57172/0 (socket says 172.21.15.12:57172) 2023-10-30T12:35:33.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f406affd640 1 -- 172.21.15.12:0/2559988654 learned_addr learned my addr 172.21.15.12:0/2559988654 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:33.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f406b7fe640 1 --2- 172.21.15.12:0/2559988654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 0x55e99307fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f406bfff640 1 --2- 172.21.15.12:0/2559988654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e993080880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f406affd640 1 -- 172.21.15.12:0/2559988654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 msgr2=0x55e993080880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f406affd640 1 --2- 172.21.15.12:0/2559988654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e993080880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f406affd640 1 -- 172.21.15.12:0/2559988654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 msgr2=0x55e99307fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f406affd640 1 --2- 172.21.15.12:0/2559988654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 0x55e99307fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f406affd640 1 -- 172.21.15.12:0/2559988654 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e993032f50 con 0x55e993021130 2023-10-30T12:35:33.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f406affd640 1 --2- 172.21.15.12:0/2559988654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e993080340 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4060054d80 tx=0x7f40600512b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb4f5647be3b766 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f4051ffb640 1 -- 172.21.15.12:0/2559988654 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060072040 con 0x55e993021130 2023-10-30T12:35:33.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f4051ffb640 1 -- 172.21.15.12:0/2559988654 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406005d1a0 con 0x55e993021130 2023-10-30T12:35:33.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f4051ffb640 1 -- 172.21.15.12:0/2559988654 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406007d040 con 0x55e993021130 2023-10-30T12:35:33.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 -- 172.21.15.12:0/2559988654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 msgr2=0x55e993080340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 --2- 172.21.15.12:0/2559988654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e993080340 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4060054d80 tx=0x7f40600512b0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 -- 172.21.15.12:0/2559988654 shutdown_connections 2023-10-30T12:35:33.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 --2- 172.21.15.12:0/2559988654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e993080880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 --2- 172.21.15.12:0/2559988654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e993080340 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 --2- 172.21.15.12:0/2559988654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 0x55e99307fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 -- 172.21.15.12:0/2559988654 >> 172.21.15.12:0/2559988654 conn(0x55e992fd6fc0 msgr2=0x55e993026080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 -- 172.21.15.12:0/2559988654 shutdown_connections 2023-10-30T12:35:33.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 -- 172.21.15.12:0/2559988654 wait complete. 2023-10-30T12:35:33.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 -- start start 2023-10-30T12:35:33.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.630+0000 7f40753aa000 1 Processor -- start 2023-10-30T12:35:33.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 0x55e99307dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e99307e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e99307e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e992dd84a0 con 0x55e993021130 2023-10-30T12:35:33.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e992def4d0 con 0x55e993020810 2023-10-30T12:35:33.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e992e3a200 con 0x55e993021a50 2023-10-30T12:35:33.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 10 client.0 _ll_register_callbacks cb 0x7f4060007f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-30T12:35:33.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f406affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e99307e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f406affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e99307e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57180/0 (socket says 172.21.15.12:57180) 2023-10-30T12:35:33.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f406affd640 1 -- 172.21.15.12:0/2454197850 learned_addr learned my addr 172.21.15.12:0/2454197850 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:33.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f406bfff640 1 --2- 172.21.15.12:0/2454197850 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e99307e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f406b7fe640 1 --2- 172.21.15.12:0/2454197850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 0x55e99307dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f406affd640 1 -- 172.21.15.12:0/2454197850 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 msgr2=0x55e99307e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f406affd640 1 --2- 172.21.15.12:0/2454197850 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e99307e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f406affd640 1 -- 172.21.15.12:0/2454197850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 msgr2=0x55e99307dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f406affd640 1 --2- 172.21.15.12:0/2454197850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 0x55e99307dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f406affd640 1 -- 172.21.15.12:0/2454197850 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e992b29890 con 0x55e993021130 2023-10-30T12:35:33.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f406affd640 1 --2- 172.21.15.12:0/2454197850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e99307e260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4060051340 tx=0x7f40600012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838ac3350802decb server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f404a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-10-30T12:35:33.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f404a7fc640 1 -- 172.21.15.12:0/2454197850 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406007d070 con 0x55e993021130 2023-10-30T12:35:33.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f404a7fc640 1 -- 172.21.15.12:0/2454197850 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4060071070 con 0x55e993021130 2023-10-30T12:35:33.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 1 -- 172.21.15.12:0/2454197850 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e993032950 con 0x55e993021130 2023-10-30T12:35:33.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f404a7fc640 1 -- 172.21.15.12:0/2454197850 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f406000cd10 con 0x55e993021130 2023-10-30T12:35:33.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 10 client.4686 fetch_fsmap learned FSMap version 13 2023-10-30T12:35:33.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 1 -- 172.21.15.12:0/2454197850 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55e992dd8760 con 0x55e993021130 2023-10-30T12:35:33.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f404a7fc640 1 -- 172.21.15.12:0/2454197850 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e993032950 con 0x55e993021130 2023-10-30T12:35:33.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f404a7fc640 1 -- 172.21.15.12:0/2454197850 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 13) v1 ==== 4468+0+0 (secure 0 0 0) 0x7f4060051550 con 0x55e993021130 2023-10-30T12:35:33.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 10 client.4686 fetch_fsmap finished waiting for FSMap version 13 2023-10-30T12:35:33.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 10 client.4686 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-30T12:35:33.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 20 client.4686 populate_metadata read hostname 'smithi012' 2023-10-30T12:35:33.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 1 --2- 172.21.15.12:0/2454197850 >> [v2:172.21.15.154:6836/1517672481,v1:172.21.15.154:6839/1517672481] conn(0x55e99307f380 0x55e99307f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 4 client.4686 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-30T12:35:33.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f40753aa000 1 -- 172.21.15.12:0/2454197850 --> [v2:172.21.15.154:6836/1517672481,v1:172.21.15.154:6839/1517672481] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e992dc86f0 con 0x55e99307f380 2023-10-30T12:35:33.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f406b7fe640 1 --2- 172.21.15.12:0/2454197850 >> [v2:172.21.15.154:6836/1517672481,v1:172.21.15.154:6839/1517672481] conn(0x55e99307f380 0x55e99307f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f406b7fe640 1 --2- 172.21.15.12:0/2454197850 >> [v2:172.21.15.154:6836/1517672481,v1:172.21.15.154:6839/1517672481] conn(0x55e99307f380 0x55e99307f760 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=92f0b562a668383d server_cookie=d4fc2240966c531a in_seq=0 out_seq=0 2023-10-30T12:35:33.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.634+0000 7f404a7fc640 10 client.4686 ms_handle_connect on v2:172.21.15.154:6836/1517672481 2023-10-30T12:35:33.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f404a7fc640 1 -- 172.21.15.12:0/2454197850 <== mds.1 v2:172.21.15.154:6836/1517672481 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55e992dc86f0 con 0x55e99307f380 2023-10-30T12:35:33.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f404a7fc640 10 client.4686 handle_command_reply: tid=0 2023-10-30T12:35:33.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f40753aa000 1 client.4686 shutdown 2023-10-30T12:35:33.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f40753aa000 1 -- 172.21.15.12:0/2454197850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 msgr2=0x55e99307e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f40753aa000 1 --2- 172.21.15.12:0/2454197850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e99307e260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4060051340 tx=0x7f40600012e0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f40753aa000 1 -- 172.21.15.12:0/2454197850 shutdown_connections 2023-10-30T12:35:33.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f40753aa000 1 --2- 172.21.15.12:0/2454197850 >> [v2:172.21.15.154:6836/1517672481,v1:172.21.15.154:6839/1517672481] conn(0x55e99307f380 0x55e99307f760 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f40753aa000 1 --2- 172.21.15.12:0/2454197850 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55e993021a50 0x55e99307e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f404a7fc640 0 client.4686 ms_handle_reset on v2:172.21.15.154:6836/1517672481 2023-10-30T12:35:33.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f40753aa000 1 --2- 172.21.15.12:0/2454197850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e993021130 0x55e99307e260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f40753aa000 1 --2- 172.21.15.12:0/2454197850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e993020810 0x55e99307dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f40753aa000 1 -- 172.21.15.12:0/2454197850 >> 172.21.15.12:0/2454197850 conn(0x55e992fd6fc0 msgr2=0x55e993023e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f40753aa000 1 -- 172.21.15.12:0/2454197850 shutdown_connections 2023-10-30T12:35:33.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f40753aa000 1 -- 172.21.15.12:0/2454197850 wait complete. 2023-10-30T12:35:33.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f40753aa000 20 client.4686 trim_cache size 0 max 16384 2023-10-30T12:35:33.671 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-30T12:35:33.671 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "97b69bd8-a5ea-4668-9ffd-462651b43fe5", 2023-10-30T12:35:33.672 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2023-10-30T12:35:33.672 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 4282, 2023-10-30T12:35:33.672 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2023-10-30T12:35:33.672 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2023-10-30T12:35:33.672 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2023-10-30T12:35:33.672 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 17.352556274000001, 2023-10-30T12:35:33.673 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 13, 2023-10-30T12:35:33.673 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 22, 2023-10-30T12:35:33.673 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 22, 2023-10-30T12:35:33.673 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 35.200791201999998 2023-10-30T12:35:33.673 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-30T12:35:33.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f4072017640 1 -- 172.21.15.12:0/1571375400 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f40580664d0 msgr2=0x7f4058068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f4072017640 1 --2- 172.21.15.12:0/1571375400 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f40580664d0 0x7f4058068990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f405c0aa000 tx=0x7f406c06c970 comp rx=0 tx=0).stop 2023-10-30T12:35:33.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f4072017640 1 -- 172.21.15.12:0/1571375400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c15a9f0 msgr2=0x7f406c0c32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f4072017640 1 --2- 172.21.15.12:0/1571375400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c15a9f0 0x7f406c0c32c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f406000e7b0 tx=0x7f406000ec70 comp rx=0 tx=0).stop 2023-10-30T12:35:33.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f4072017640 1 -- 172.21.15.12:0/1571375400 shutdown_connections 2023-10-30T12:35:33.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f4072017640 1 --2- 172.21.15.12:0/1571375400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f406c1f2c70 0x7f406c14b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f4072017640 1 --2- 172.21.15.12:0/1571375400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f406c15a9f0 0x7f406c0c32c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f4072017640 1 --2- 172.21.15.12:0/1571375400 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f40580664d0 0x7f4058068990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f4072017640 1 --2- 172.21.15.12:0/1571375400 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4054003b10 0x7f406c1f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f4072017640 1 -- 172.21.15.12:0/1571375400 >> 172.21.15.12:0/1571375400 conn(0x7f406c071ea0 msgr2=0x7f406c158a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f4072017640 1 -- 172.21.15.12:0/1571375400 shutdown_connections 2023-10-30T12:35:33.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.638+0000 7f4072017640 1 -- 172.21.15.12:0/1571375400 wait complete. 2023-10-30T12:35:33.677 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-10-30T12:35:33.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa18a2bb640 1 Processor -- start 2023-10-30T12:35:33.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa18a2bb640 1 -- start start 2023-10-30T12:35:33.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa18a2bb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1840c5550 0x7fa1840cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa18a2bb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1840cab00 0x7fa1840cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa18a2bb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1840cba30 con 0x7fa1840cb050 2023-10-30T12:35:33.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa18a2bb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1840ca290 con 0x7fa1840c5550 2023-10-30T12:35:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa18a2bb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840c9cf0 con 0x7fa1840cab00 2023-10-30T12:35:33.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa189aba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1840cab00 0x7fa1840cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa1892b9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1840c5550 0x7fa1840cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa189aba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1840cab00 0x7fa1840cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:37924/0 (socket says 172.21.15.12:37924) 2023-10-30T12:35:33.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa189aba640 1 -- 172.21.15.12:0/3529761157 learned_addr learned my addr 172.21.15.12:0/3529761157 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:33.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa188ab8640 1 --1- 172.21.15.12:0/3529761157 >> v1:172.21.15.154:6789/0 conn(0x7fa1840cb050 0x7fa1840cbec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:51026/0 (socket says 172.21.15.12:51026) 2023-10-30T12:35:33.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa189aba640 1 -- 172.21.15.12:0/3529761157 >> v2:172.21.15.154:3300/0 conn(0x7fa1840c5550 msgr2=0x7fa1840cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa189aba640 1 --2- 172.21.15.12:0/3529761157 >> v2:172.21.15.154:3300/0 conn(0x7fa1840c5550 0x7fa1840cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa189aba640 1 -- 172.21.15.12:0/3529761157 >> v1:172.21.15.154:6789/0 conn(0x7fa1840cb050 legacy=0x7fa1840cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa189aba640 1 -- 172.21.15.12:0/3529761157 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18415d990 con 0x7fa1840cab00 2023-10-30T12:35:33.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.818+0000 7fa1892b9640 1 --2- 172.21.15.12:0/3529761157 >> v2:172.21.15.154:3300/0 conn(0x7fa1840c5550 0x7fa1840cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:35:33.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.822+0000 7fa189aba640 1 --2- 172.21.15.12:0/3529761157 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1840cab00 0x7fa1840cf440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009e50 tx=0x7fa17c01ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f625b8f674947d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.822+0000 7fa17bfff640 1 -- 172.21.15.12:0/3529761157 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0a57d0 con 0x7fa1840cab00 2023-10-30T12:35:33.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.822+0000 7fa17bfff640 1 -- 172.21.15.12:0/3529761157 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c0a5930 con 0x7fa1840cab00 2023-10-30T12:35:33.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.822+0000 7fa17bfff640 1 -- 172.21.15.12:0/3529761157 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0a5cc0 con 0x7fa1840cab00 2023-10-30T12:35:33.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.822+0000 7fa18a2bb640 1 -- 172.21.15.12:0/3529761157 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1840cab00 msgr2=0x7fa1840cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.822+0000 7fa18a2bb640 1 --2- 172.21.15.12:0/3529761157 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1840cab00 0x7fa1840cf440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009e50 tx=0x7fa17c01ca10 comp rx=0 tx=0).stop 2023-10-30T12:35:33.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.822+0000 7fa18a2bb640 1 -- 172.21.15.12:0/3529761157 shutdown_connections 2023-10-30T12:35:33.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.822+0000 7fa18a2bb640 1 --2- 172.21.15.12:0/3529761157 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1840cab00 0x7fa1840cf440 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.822+0000 7fa18a2bb640 1 --2- 172.21.15.12:0/3529761157 >> v2:172.21.15.154:3300/0 conn(0x7fa1840c5550 0x7fa1840cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.822+0000 7fa18a2bb640 1 -- 172.21.15.12:0/3529761157 >> 172.21.15.12:0/3529761157 conn(0x7fa18406bed0 msgr2=0x7fa18406c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa18a2bb640 1 -- 172.21.15.12:0/3529761157 shutdown_connections 2023-10-30T12:35:33.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa18a2bb640 1 -- 172.21.15.12:0/3529761157 wait complete. 2023-10-30T12:35:33.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa18a2bb640 1 Processor -- start 2023-10-30T12:35:33.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa18a2bb640 1 -- start start 2023-10-30T12:35:33.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa18a2bb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1840c5550 0x7fa1841f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa18a2bb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1840cab00 0x7fa18414fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa18a2bb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1840cb050 0x7fa184148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa18a2bb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1841f21f0 con 0x7fa1840cb050 2023-10-30T12:35:33.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa18a2bb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1841504b0 con 0x7fa1840c5550 2023-10-30T12:35:33.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa18a2bb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa184150c80 con 0x7fa1840cab00 2023-10-30T12:35:33.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa189aba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1840cb050 0x7fa184148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa1892b9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1840c5550 0x7fa1841f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa189aba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1840cb050 0x7fa184148470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57182/0 (socket says 172.21.15.12:57182) 2023-10-30T12:35:33.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa189aba640 1 -- 172.21.15.12:0/2184551395 learned_addr learned my addr 172.21.15.12:0/2184551395 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:33.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa188ab8640 1 --2- 172.21.15.12:0/2184551395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1840cab00 0x7fa18414fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa189aba640 1 -- 172.21.15.12:0/2184551395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1840cab00 msgr2=0x7fa18414fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa189aba640 1 --2- 172.21.15.12:0/2184551395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1840cab00 0x7fa18414fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa189aba640 1 -- 172.21.15.12:0/2184551395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1840c5550 msgr2=0x7fa1841f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa189aba640 1 --2- 172.21.15.12:0/2184551395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1840c5550 0x7fa1841f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa189aba640 1 -- 172.21.15.12:0/2184551395 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa180009420 con 0x7fa1840cb050 2023-10-30T12:35:33.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.826+0000 7fa189aba640 1 --2- 172.21.15.12:0/2184551395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1840cb050 0x7fa184148470 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa17c01a640 tx=0x7fa17c0197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.830+0000 7fa179ffb640 1 -- 172.21.15.12:0/2184551395 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0b8670 con 0x7fa1840cb050 2023-10-30T12:35:33.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.830+0000 7fa179ffb640 1 -- 172.21.15.12:0/2184551395 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c0b87f0 con 0x7fa1840cb050 2023-10-30T12:35:33.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.830+0000 7fa179ffb640 1 -- 172.21.15.12:0/2184551395 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0b8ba0 con 0x7fa1840cb050 2023-10-30T12:35:33.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.830+0000 7fa18a2bb640 1 -- 172.21.15.12:0/2184551395 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17c009b30 con 0x7fa1840cb050 2023-10-30T12:35:33.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.830+0000 7fa18a2bb640 1 -- 172.21.15.12:0/2184551395 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa184150eb0 con 0x7fa1840cb050 2023-10-30T12:35:33.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.830+0000 7fa179ffb640 1 -- 172.21.15.12:0/2184551395 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa17c0b8d00 con 0x7fa1840cb050 2023-10-30T12:35:33.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.830+0000 7fa179ffb640 1 --2- 172.21.15.12:0/2184551395 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa1680664a0 0x7fa168068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.830+0000 7fa179ffb640 1 -- 172.21.15.12:0/2184551395 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa17c11f540 con 0x7fa1840cb050 2023-10-30T12:35:33.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.830+0000 7fa18e89c000 1 Processor -- start 2023-10-30T12:35:33.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.830+0000 7fa1892b9640 1 --2- 172.21.15.12:0/2184551395 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa1680664a0 0x7fa168068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.830+0000 7fa18e89c000 1 -- start start 2023-10-30T12:35:33.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa1892b9640 1 --2- 172.21.15.12:0/2184551395 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa1680664a0 0x7fa168068960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa180009cf0 tx=0x7fa180009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5a810 0x55aa2ca5abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca5b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5ba50 0x55aa2ca65aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c80ecc0 con 0x55aa2ca5b130 2023-10-30T12:35:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c871920 con 0x55aa2ca5ba50 2023-10-30T12:35:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c86ee50 con 0x55aa2ca5a810 2023-10-30T12:35:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa188ab8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca5b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa188ab8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca5b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57188/0 (socket says 172.21.15.12:57188) 2023-10-30T12:35:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa188ab8640 1 -- 172.21.15.12:0/3991649151 learned_addr learned my addr 172.21.15.12:0/3991649151 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa1892b9640 1 --2- 172.21.15.12:0/3991649151 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5a810 0x55aa2ca5abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa189aba640 1 --2- 172.21.15.12:0/3991649151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5ba50 0x55aa2ca65aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa188ab8640 1 -- 172.21.15.12:0/3991649151 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5a810 msgr2=0x55aa2ca5abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa188ab8640 1 --2- 172.21.15.12:0/3991649151 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5a810 0x55aa2ca5abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa188ab8640 1 -- 172.21.15.12:0/3991649151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5ba50 msgr2=0x55aa2ca65aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa188ab8640 1 --2- 172.21.15.12:0/3991649151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5ba50 0x55aa2ca65aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa188ab8640 1 -- 172.21.15.12:0/3991649151 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa2c563890 con 0x55aa2ca5b130 2023-10-30T12:35:33.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa188ab8640 1 --2- 172.21.15.12:0/3991649151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca5b510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa174000fc0 tx=0x7fa174007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe97ffdb60068ff server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa166ffd640 1 -- 172.21.15.12:0/3991649151 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174004e70 con 0x55aa2ca5b130 2023-10-30T12:35:33.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa166ffd640 1 -- 172.21.15.12:0/3991649151 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa174007700 con 0x55aa2ca5b130 2023-10-30T12:35:33.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa166ffd640 1 -- 172.21.15.12:0/3991649151 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174001a20 con 0x55aa2ca5b130 2023-10-30T12:35:33.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 -- 172.21.15.12:0/3991649151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 msgr2=0x55aa2ca5b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 --2- 172.21.15.12:0/3991649151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca5b510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa174000fc0 tx=0x7fa174007e40 comp rx=0 tx=0).stop 2023-10-30T12:35:33.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 -- 172.21.15.12:0/3991649151 shutdown_connections 2023-10-30T12:35:33.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 --2- 172.21.15.12:0/3991649151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5ba50 0x55aa2ca65aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 --2- 172.21.15.12:0/3991649151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca5b510 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 --2- 172.21.15.12:0/3991649151 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5a810 0x55aa2ca5abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 -- 172.21.15.12:0/3991649151 >> 172.21.15.12:0/3991649151 conn(0x55aa2ca10fc0 msgr2=0x55aa2ca585d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 -- 172.21.15.12:0/3991649151 shutdown_connections 2023-10-30T12:35:33.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 -- 172.21.15.12:0/3991649151 wait complete. 2023-10-30T12:35:33.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 -- start start 2023-10-30T12:35:33.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.834+0000 7fa18e89c000 1 Processor -- start 2023-10-30T12:35:33.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5a810 0x55aa2ca54f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5b130 0x55aa2ca55490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5ba50 0x55aa2ca60c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c86a740 con 0x55aa2ca5ba50 2023-10-30T12:35:33.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c829d10 con 0x55aa2ca5b130 2023-10-30T12:35:33.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c8299b0 con 0x55aa2ca5a810 2023-10-30T12:35:33.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 10 client.0 _ll_register_callbacks cb 0x55aa2ca05390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-30T12:35:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa189aba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5ba50 0x55aa2ca60c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa189aba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5ba50 0x55aa2ca60c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57194/0 (socket says 172.21.15.12:57194) 2023-10-30T12:35:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa189aba640 1 -- 172.21.15.12:0/1639865702 learned_addr learned my addr 172.21.15.12:0/1639865702 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa188ab8640 1 --2- 172.21.15.12:0/1639865702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5b130 0x55aa2ca55490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa1892b9640 1 --2- 172.21.15.12:0/1639865702 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5a810 0x55aa2ca54f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa189aba640 1 -- 172.21.15.12:0/1639865702 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5a810 msgr2=0x55aa2ca54f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa189aba640 1 --2- 172.21.15.12:0/1639865702 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5a810 0x55aa2ca54f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa189aba640 1 -- 172.21.15.12:0/1639865702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5b130 msgr2=0x55aa2ca55490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa189aba640 1 --2- 172.21.15.12:0/1639865702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5b130 0x55aa2ca55490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa189aba640 1 -- 172.21.15.12:0/1639865702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1840cba30 con 0x55aa2ca5ba50 2023-10-30T12:35:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa188ab8640 1 --2- 172.21.15.12:0/1639865702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5b130 0x55aa2ca55490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa189aba640 1 --2- 172.21.15.12:0/1639865702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5ba50 0x55aa2ca60c30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0ecd70 tx=0x7fa17c0f4910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32d0ed879a26cc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa15ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-10-30T12:35:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa15ffff640 1 -- 172.21.15.12:0/1639865702 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0eecd0 con 0x55aa2ca5ba50 2023-10-30T12:35:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa15ffff640 1 -- 172.21.15.12:0/1639865702 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa17c0efd60 con 0x55aa2ca5ba50 2023-10-30T12:35:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 1 -- 172.21.15.12:0/1639865702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa2c81e370 con 0x55aa2ca5ba50 2023-10-30T12:35:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa15ffff640 1 -- 172.21.15.12:0/1639865702 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa17c0eff20 con 0x55aa2ca5ba50 2023-10-30T12:35:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 10 client.4695 fetch_fsmap learned FSMap version 13 2023-10-30T12:35:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 1 -- 172.21.15.12:0/1639865702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55aa2c82c460 con 0x55aa2ca5ba50 2023-10-30T12:35:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa15ffff640 1 -- 172.21.15.12:0/1639865702 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aa2c81e370 con 0x55aa2ca5ba50 2023-10-30T12:35:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa15ffff640 1 -- 172.21.15.12:0/1639865702 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 13) v1 ==== 4468+0+0 (secure 0 0 0) 0x7fa17c005d20 con 0x55aa2ca5ba50 2023-10-30T12:35:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 10 client.4695 fetch_fsmap finished waiting for FSMap version 13 2023-10-30T12:35:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 10 client.4695 resolve_mds: resolved name 'e' to daemon mds.e 2023-10-30T12:35:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 20 client.4695 populate_metadata read hostname 'smithi012' 2023-10-30T12:35:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1639865702 >> [v2:172.21.15.12:6833/291392212,v1:172.21.15.12:6836/291392212] conn(0x55aa2ca619a0 0x55aa2ca61d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 4 client.4695 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-30T12:35:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa18e89c000 1 -- 172.21.15.12:0/1639865702 --> [v2:172.21.15.12:6833/291392212,v1:172.21.15.12:6836/291392212] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa2c8042d0 con 0x55aa2ca619a0 2023-10-30T12:35:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa1892b9640 1 --2- 172.21.15.12:0/1639865702 >> [v2:172.21.15.12:6833/291392212,v1:172.21.15.12:6836/291392212] conn(0x55aa2ca619a0 0x55aa2ca61d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa1892b9640 1 --2- 172.21.15.12:0/1639865702 >> [v2:172.21.15.12:6833/291392212,v1:172.21.15.12:6836/291392212] conn(0x55aa2ca619a0 0x55aa2ca61d80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dde0626ff53f3874 server_cookie=b25e3d28c88c6d01 in_seq=0 out_seq=0 2023-10-30T12:35:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa15ffff640 10 client.4695 ms_handle_connect on v2:172.21.15.12:6833/291392212 2023-10-30T12:35:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa15ffff640 1 -- 172.21.15.12:0/1639865702 <== mds.0 v2:172.21.15.12:6833/291392212 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aa2c8042d0 con 0x55aa2ca619a0 2023-10-30T12:35:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.838+0000 7fa15ffff640 10 client.4695 handle_command_reply: tid=0 2023-10-30T12:35:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.842+0000 7fa18e89c000 1 client.4695 shutdown 2023-10-30T12:35:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.842+0000 7fa18e89c000 1 -- 172.21.15.12:0/1639865702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5ba50 msgr2=0x55aa2ca60c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.842+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1639865702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5ba50 0x55aa2ca60c30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0ecd70 tx=0x7fa17c0f4910 comp rx=0 tx=0).stop 2023-10-30T12:35:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.842+0000 7fa18e89c000 1 -- 172.21.15.12:0/1639865702 shutdown_connections 2023-10-30T12:35:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.842+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1639865702 >> [v2:172.21.15.12:6833/291392212,v1:172.21.15.12:6836/291392212] conn(0x55aa2ca619a0 0x55aa2ca61d80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.842+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1639865702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5ba50 0x55aa2ca60c30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.842+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1639865702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5b130 0x55aa2ca55490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.842+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1639865702 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5a810 0x55aa2ca54f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.842+0000 7fa15ffff640 0 client.4695 ms_handle_reset on v2:172.21.15.12:6833/291392212 2023-10-30T12:35:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.842+0000 7fa18e89c000 1 -- 172.21.15.12:0/1639865702 >> 172.21.15.12:0/1639865702 conn(0x55aa2ca10fc0 msgr2=0x55aa2ca5a090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.842+0000 7fa18e89c000 1 -- 172.21.15.12:0/1639865702 shutdown_connections 2023-10-30T12:35:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.842+0000 7fa18e89c000 1 -- 172.21.15.12:0/1639865702 wait complete. 2023-10-30T12:35:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.842+0000 7fa18e89c000 20 client.4695 trim_cache size 0 max 16384 2023-10-30T12:35:33.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.854+0000 7fa18e89c000 1 Processor -- start 2023-10-30T12:35:33.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 -- start start 2023-10-30T12:35:33.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5a810 0x55aa2ca1f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca1f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5ba50 0x55aa2ca5e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c524940 con 0x55aa2ca5b130 2023-10-30T12:35:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c8044c0 con 0x55aa2ca5ba50 2023-10-30T12:35:33.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c8294d0 con 0x55aa2ca5a810 2023-10-30T12:35:33.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa188ab8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca1f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa188ab8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca1f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57208/0 (socket says 172.21.15.12:57208) 2023-10-30T12:35:33.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa188ab8640 1 -- 172.21.15.12:0/1251258001 learned_addr learned my addr 172.21.15.12:0/1251258001 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:33.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa189aba640 1 --2- 172.21.15.12:0/1251258001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5ba50 0x55aa2ca5e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa1892b9640 1 --2- 172.21.15.12:0/1251258001 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5a810 0x55aa2ca1f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa188ab8640 1 -- 172.21.15.12:0/1251258001 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5a810 msgr2=0x55aa2ca1f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa188ab8640 1 --2- 172.21.15.12:0/1251258001 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5a810 0x55aa2ca1f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa188ab8640 1 -- 172.21.15.12:0/1251258001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5ba50 msgr2=0x55aa2ca5e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa188ab8640 1 --2- 172.21.15.12:0/1251258001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5ba50 0x55aa2ca5e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa188ab8640 1 -- 172.21.15.12:0/1251258001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa2ca6cf50 con 0x55aa2ca5b130 2023-10-30T12:35:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa188ab8640 1 --2- 172.21.15.12:0/1251258001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca1f9e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa174014000 tx=0x7fa17400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9279ec6d3b3cc1f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa1677fe640 1 -- 172.21.15.12:0/1251258001 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174001a20 con 0x55aa2ca5b130 2023-10-30T12:35:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa1677fe640 1 -- 172.21.15.12:0/1251258001 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17400b070 con 0x55aa2ca5b130 2023-10-30T12:35:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa1677fe640 1 -- 172.21.15.12:0/1251258001 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174001dc0 con 0x55aa2ca5b130 2023-10-30T12:35:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 -- 172.21.15.12:0/1251258001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 msgr2=0x55aa2ca1f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1251258001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca1f9e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa174014000 tx=0x7fa17400c000 comp rx=0 tx=0).stop 2023-10-30T12:35:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 -- 172.21.15.12:0/1251258001 shutdown_connections 2023-10-30T12:35:33.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1251258001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5ba50 0x55aa2ca5e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1251258001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca1f9e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1251258001 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5a810 0x55aa2ca1f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 -- 172.21.15.12:0/1251258001 >> 172.21.15.12:0/1251258001 conn(0x55aa2ca10fc0 msgr2=0x55aa2ca631b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 -- 172.21.15.12:0/1251258001 shutdown_connections 2023-10-30T12:35:33.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 -- 172.21.15.12:0/1251258001 wait complete. 2023-10-30T12:35:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 -- start start 2023-10-30T12:35:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.858+0000 7fa18e89c000 1 Processor -- start 2023-10-30T12:35:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5a810 0x55aa2ca5ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca5f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5ba50 0x55aa2ca5f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c840a10 con 0x55aa2ca5b130 2023-10-30T12:35:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c86e740 con 0x55aa2ca5a810 2023-10-30T12:35:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c8660b0 con 0x55aa2ca5ba50 2023-10-30T12:35:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 10 client.0 _ll_register_callbacks cb 0x7fa174004880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-30T12:35:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa188ab8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca5f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa188ab8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca5f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57214/0 (socket says 172.21.15.12:57214) 2023-10-30T12:35:33.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa188ab8640 1 -- 172.21.15.12:0/1038324601 learned_addr learned my addr 172.21.15.12:0/1038324601 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:33.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa1892b9640 1 --2- 172.21.15.12:0/1038324601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5a810 0x55aa2ca5ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa189aba640 1 --2- 172.21.15.12:0/1038324601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5ba50 0x55aa2ca5f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa188ab8640 1 -- 172.21.15.12:0/1038324601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5ba50 msgr2=0x55aa2ca5f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa188ab8640 1 --2- 172.21.15.12:0/1038324601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5ba50 0x55aa2ca5f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa188ab8640 1 -- 172.21.15.12:0/1038324601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5a810 msgr2=0x55aa2ca5ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:33.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa188ab8640 1 --2- 172.21.15.12:0/1038324601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5a810 0x55aa2ca5ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa188ab8640 1 -- 172.21.15.12:0/1038324601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa2c563890 con 0x55aa2ca5b130 2023-10-30T12:35:33.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa188ab8640 1 --2- 172.21.15.12:0/1038324601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca5f110 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55aa2c9fa230 tx=0x7fa1740010e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8951760fe0f8a906 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa15ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-10-30T12:35:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa15ffff640 1 -- 172.21.15.12:0/1038324601 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17402b650 con 0x55aa2ca5b130 2023-10-30T12:35:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa15ffff640 1 -- 172.21.15.12:0/1038324601 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa174001d60 con 0x55aa2ca5b130 2023-10-30T12:35:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 1 -- 172.21.15.12:0/1038324601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa2ca759b0 con 0x55aa2ca5b130 2023-10-30T12:35:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa15ffff640 1 -- 172.21.15.12:0/1038324601 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa17402b7b0 con 0x55aa2ca5b130 2023-10-30T12:35:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 10 client.4701 fetch_fsmap learned FSMap version 13 2023-10-30T12:35:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 1 -- 172.21.15.12:0/1038324601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55aa2c812760 con 0x55aa2ca5b130 2023-10-30T12:35:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa15ffff640 1 -- 172.21.15.12:0/1038324601 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aa2ca759b0 con 0x55aa2ca5b130 2023-10-30T12:35:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa15ffff640 1 -- 172.21.15.12:0/1038324601 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 13) v1 ==== 4468+0+0 (secure 0 0 0) 0x7fa174012070 con 0x55aa2ca5b130 2023-10-30T12:35:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 10 client.4701 fetch_fsmap finished waiting for FSMap version 13 2023-10-30T12:35:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 10 client.4701 resolve_mds: resolved name 'e' to daemon mds.e 2023-10-30T12:35:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 20 client.4701 populate_metadata read hostname 'smithi012' 2023-10-30T12:35:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1038324601 >> [v2:172.21.15.12:6833/291392212,v1:172.21.15.12:6836/291392212] conn(0x55aa2cac3ae0 0x55aa2cac5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 4 client.4701 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-30T12:35:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 1 -- 172.21.15.12:0/1038324601 --> [v2:172.21.15.12:6833/291392212,v1:172.21.15.12:6836/291392212] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aa2c8026f0 con 0x55aa2cac3ae0 2023-10-30T12:35:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa1892b9640 1 --2- 172.21.15.12:0/1038324601 >> [v2:172.21.15.12:6833/291392212,v1:172.21.15.12:6836/291392212] conn(0x55aa2cac3ae0 0x55aa2cac5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa1892b9640 1 --2- 172.21.15.12:0/1038324601 >> [v2:172.21.15.12:6833/291392212,v1:172.21.15.12:6836/291392212] conn(0x55aa2cac3ae0 0x55aa2cac5ed0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fea3bd68e54552af server_cookie=64edb89d52316427 in_seq=0 out_seq=0 2023-10-30T12:35:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa15ffff640 10 client.4701 ms_handle_connect on v2:172.21.15.12:6833/291392212 2023-10-30T12:35:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa15ffff640 1 -- 172.21.15.12:0/1038324601 <== mds.0 v2:172.21.15.12:6833/291392212 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55aa2c8026f0 con 0x55aa2cac3ae0 2023-10-30T12:35:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa15ffff640 10 client.4701 handle_command_reply: tid=0 2023-10-30T12:35:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.862+0000 7fa18e89c000 1 client.4701 shutdown 2023-10-30T12:35:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18e89c000 1 -- 172.21.15.12:0/1038324601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 msgr2=0x55aa2ca5f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1038324601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca5f110 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55aa2c9fa230 tx=0x7fa1740010e0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18e89c000 1 -- 172.21.15.12:0/1038324601 shutdown_connections 2023-10-30T12:35:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1038324601 >> [v2:172.21.15.12:6833/291392212,v1:172.21.15.12:6836/291392212] conn(0x55aa2cac3ae0 0x55aa2cac5ed0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1038324601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55aa2ca5ba50 0x55aa2ca5f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1038324601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa2ca5b130 0x55aa2ca5f110 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18e89c000 1 --2- 172.21.15.12:0/1038324601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55aa2ca5a810 0x55aa2ca5ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18e89c000 1 -- 172.21.15.12:0/1038324601 >> 172.21.15.12:0/1038324601 conn(0x55aa2ca10fc0 msgr2=0x55aa2ca61d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa15ffff640 0 client.4701 ms_handle_reset on v2:172.21.15.12:6833/291392212 2023-10-30T12:35:33.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18e89c000 1 -- 172.21.15.12:0/1038324601 shutdown_connections 2023-10-30T12:35:33.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18e89c000 1 -- 172.21.15.12:0/1038324601 wait complete. 2023-10-30T12:35:33.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18e89c000 20 client.4701 trim_cache size 0 max 16384 2023-10-30T12:35:33.887 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-30T12:35:33.888 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "97b69bd8-a5ea-4668-9ffd-462651b43fe5", 2023-10-30T12:35:33.888 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2023-10-30T12:35:33.888 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 4285, 2023-10-30T12:35:33.888 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2023-10-30T12:35:33.888 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2023-10-30T12:35:33.889 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2023-10-30T12:35:33.889 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 27.685900109999999, 2023-10-30T12:35:33.889 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 13, 2023-10-30T12:35:33.889 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 22, 2023-10-30T12:35:33.889 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 22, 2023-10-30T12:35:33.889 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 35.424411835000001 2023-10-30T12:35:33.890 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-30T12:35:33.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18a2bb640 1 -- 172.21.15.12:0/2184551395 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa1680664a0 msgr2=0x7fa168068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18a2bb640 1 --2- 172.21.15.12:0/2184551395 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa1680664a0 0x7fa168068960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa180009cf0 tx=0x7fa180009190 comp rx=0 tx=0).stop 2023-10-30T12:35:33.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18a2bb640 1 -- 172.21.15.12:0/2184551395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1840cb050 msgr2=0x7fa184148470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:33.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18a2bb640 1 --2- 172.21.15.12:0/2184551395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1840cb050 0x7fa184148470 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa17c01a640 tx=0x7fa17c0197b0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18a2bb640 1 -- 172.21.15.12:0/2184551395 shutdown_connections 2023-10-30T12:35:33.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18a2bb640 1 --2- 172.21.15.12:0/2184551395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1840cb050 0x7fa184148470 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18a2bb640 1 --2- 172.21.15.12:0/2184551395 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa1680664a0 0x7fa168068960 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18a2bb640 1 --2- 172.21.15.12:0/2184551395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1840cab00 0x7fa18414fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18a2bb640 1 --2- 172.21.15.12:0/2184551395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1840c5550 0x7fa1841f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:33.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18a2bb640 1 -- 172.21.15.12:0/2184551395 >> 172.21.15.12:0/2184551395 conn(0x7fa18406bed0 msgr2=0x7fa1840cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:33.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18a2bb640 1 -- 172.21.15.12:0/2184551395 shutdown_connections 2023-10-30T12:35:33.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:33.866+0000 7fa18a2bb640 1 -- 172.21.15.12:0/2184551395 wait complete. 2023-10-30T12:35:33.894 INFO:teuthology.run_tasks:Running task kclient... 2023-10-30T12:35:33.921 INFO:tasks.kclient:Mounting kernel clients... 2023-10-30T12:35:33.921 INFO:tasks.kclient:config is {'client.0': {}} 2023-10-30T12:35:33.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:33.922 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2023-10-30T12:35:33.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:33.933 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2023-10-30T12:35:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:Cannot find device "ceph-brx" 2023-10-30T12:35:34.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:35:34.003 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-10-30T12:35:34.003 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-30T12:35:34.004 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-30T12:35:34.004 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-30T12:35:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.182+0000 7fab833ab640 1 Processor -- start 2023-10-30T12:35:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.182+0000 7fab833ab640 1 -- start start 2023-10-30T12:35:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.182+0000 7fab833ab640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fab7c1538a0 0x7fab7c15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.182+0000 7fab833ab640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab7c0cb800 con 0x7fab7c0cf160 2023-10-30T12:35:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.182+0000 7fab833ab640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab7c15dc30 con 0x7fab7c0ca0e0 2023-10-30T12:35:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.182+0000 7fab833ab640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab7c0cb330 con 0x7fab7c1538a0 2023-10-30T12:35:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.182+0000 7fab81147640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fab7c1538a0 0x7fab7c15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:34.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.182+0000 7fab81147640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fab7c1538a0 0x7fab7c15d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57222/0 (socket says 172.21.15.12:57222) 2023-10-30T12:35:34.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.182+0000 7fab81147640 1 -- 172.21.15.12:0/1966184123 learned_addr learned my addr 172.21.15.12:0/1966184123 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:34.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.182+0000 7fab81948640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fab7c0cf160 0x7fab7c164070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54412/0 (socket says 172.21.15.12:54412) 2023-10-30T12:35:34.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.186+0000 7fab81147640 1 -- 172.21.15.12:0/1966184123 >> v1:172.21.15.154:6789/0 conn(0x7fab7c0ca0e0 legacy=0x7fab7c15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:34.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.186+0000 7fab81147640 1 -- 172.21.15.12:0/1966184123 >> v1:172.21.15.12:6789/0 conn(0x7fab7c0cf160 legacy=0x7fab7c164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:34.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.186+0000 7fab73fff640 1 -- 172.21.15.12:0/1966184123 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333342872 0 0) 0x7fab7c0cb800 con 0x7fab7c0cf160 2023-10-30T12:35:34.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.186+0000 7fab81147640 1 -- 172.21.15.12:0/1966184123 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c0cad70 con 0x7fab7c1538a0 2023-10-30T12:35:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.186+0000 7fab81147640 1 --2- 172.21.15.12:0/1966184123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab7c1538a0 0x7fab7c15d6f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fab78009870 tx=0x7fab7801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6383bf942f1fcb1a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.186+0000 7fab73fff640 1 -- 172.21.15.12:0/1966184123 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab780a8a30 con 0x7fab7c1538a0 2023-10-30T12:35:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.186+0000 7fab73fff640 1 -- 172.21.15.12:0/1966184123 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab780a8b90 con 0x7fab7c1538a0 2023-10-30T12:35:34.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.186+0000 7fab73fff640 1 -- 172.21.15.12:0/1966184123 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab780b0440 con 0x7fab7c1538a0 2023-10-30T12:35:34.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.186+0000 7fab833ab640 1 -- 172.21.15.12:0/1966184123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab7c1538a0 msgr2=0x7fab7c15d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:34.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.186+0000 7fab833ab640 1 --2- 172.21.15.12:0/1966184123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab7c1538a0 0x7fab7c15d6f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fab78009870 tx=0x7fab7801ed30 comp rx=0 tx=0).stop 2023-10-30T12:35:34.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab833ab640 1 -- 172.21.15.12:0/1966184123 shutdown_connections 2023-10-30T12:35:34.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab833ab640 1 --2- 172.21.15.12:0/1966184123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab7c1538a0 0x7fab7c15d6f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:34.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab833ab640 1 -- 172.21.15.12:0/1966184123 >> 172.21.15.12:0/1966184123 conn(0x7fab7c06bfe0 msgr2=0x7fab7c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:34.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab833ab640 1 -- 172.21.15.12:0/1966184123 shutdown_connections 2023-10-30T12:35:34.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab833ab640 1 -- 172.21.15.12:0/1966184123 wait complete. 2023-10-30T12:35:34.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab833ab640 1 Processor -- start 2023-10-30T12:35:34.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab833ab640 1 -- start start 2023-10-30T12:35:34.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab833ab640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab7c0ca0e0 0x7fab7c0c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:34.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab833ab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab7c0cf160 0x7fab7c1507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:34.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab833ab640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab7c1538a0 0x7fab7c0b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:34.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab833ab640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c157830 con 0x7fab7c1538a0 2023-10-30T12:35:34.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab833ab640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c1575c0 con 0x7fab7c0cf160 2023-10-30T12:35:34.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab833ab640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c0b3d90 con 0x7fab7c0ca0e0 2023-10-30T12:35:34.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab81948640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab7c1538a0 0x7fab7c0b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:34.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab81948640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab7c1538a0 0x7fab7c0b42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57232/0 (socket says 172.21.15.12:57232) 2023-10-30T12:35:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab81948640 1 -- 172.21.15.12:0/3049628666 learned_addr learned my addr 172.21.15.12:0/3049628666 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab81147640 1 --2- 172.21.15.12:0/3049628666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab7c0ca0e0 0x7fab7c0c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab80946640 1 --2- 172.21.15.12:0/3049628666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab7c0cf160 0x7fab7c1507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab81948640 1 -- 172.21.15.12:0/3049628666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab7c0ca0e0 msgr2=0x7fab7c0c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab81948640 1 --2- 172.21.15.12:0/3049628666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab7c0ca0e0 0x7fab7c0c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab81948640 1 -- 172.21.15.12:0/3049628666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab7c0cf160 msgr2=0x7fab7c1507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab81948640 1 --2- 172.21.15.12:0/3049628666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab7c0cf160 0x7fab7c1507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab81948640 1 -- 172.21.15.12:0/3049628666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab780094d0 con 0x7fab7c1538a0 2023-10-30T12:35:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.190+0000 7fab81948640 1 --2- 172.21.15.12:0/3049628666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab7c1538a0 0x7fab7c0b42e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fab74001570 tx=0x7fab7400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:34.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.194+0000 7fab71ffb640 1 -- 172.21.15.12:0/3049628666 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7400cea0 con 0x7fab7c1538a0 2023-10-30T12:35:34.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.194+0000 7fab71ffb640 1 -- 172.21.15.12:0/3049628666 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7400a5e0 con 0x7fab7c1538a0 2023-10-30T12:35:34.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.194+0000 7fab71ffb640 1 -- 172.21.15.12:0/3049628666 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7400a990 con 0x7fab7c1538a0 2023-10-30T12:35:34.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.194+0000 7fab833ab640 1 -- 172.21.15.12:0/3049628666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7c159040 con 0x7fab7c1538a0 2023-10-30T12:35:34.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.194+0000 7fab833ab640 1 -- 172.21.15.12:0/3049628666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7c0b4010 con 0x7fab7c1538a0 2023-10-30T12:35:34.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.194+0000 7fab833ab640 1 -- 172.21.15.12:0/3049628666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab7c04e170 con 0x7fab7c1538a0 2023-10-30T12:35:34.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.194+0000 7fab71ffb640 1 -- 172.21.15.12:0/3049628666 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fab7400aaf0 con 0x7fab7c1538a0 2023-10-30T12:35:34.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.194+0000 7fab71ffb640 1 --2- 172.21.15.12:0/3049628666 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fab600664a0 0x7fab60068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:34.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.194+0000 7fab71ffb640 1 -- 172.21.15.12:0/3049628666 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab74083390 con 0x7fab7c1538a0 2023-10-30T12:35:34.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.194+0000 7fab81147640 1 --2- 172.21.15.12:0/3049628666 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fab600664a0 0x7fab60068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:34.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.198+0000 7fab81147640 1 --2- 172.21.15.12:0/3049628666 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fab600664a0 0x7fab60068960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fab780b2000 tx=0x7fab78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:34.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.198+0000 7fab71ffb640 1 -- 172.21.15.12:0/3049628666 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fab74050820 con 0x7fab7c1538a0 2023-10-30T12:35:34.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.390+0000 7fab833ab640 1 -- 172.21.15.12:0/3049628666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fab7c159b40 con 0x7fab7c1538a0 2023-10-30T12:35:34.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.390+0000 7fab71ffb640 1 -- 172.21.15.12:0/3049628666 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7fab74018070 con 0x7fab7c1538a0 2023-10-30T12:35:34.396 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-30T12:35:34.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.394+0000 7fab833ab640 1 -- 172.21.15.12:0/3049628666 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fab600664a0 msgr2=0x7fab60068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:34.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.394+0000 7fab833ab640 1 --2- 172.21.15.12:0/3049628666 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fab600664a0 0x7fab60068960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fab780b2000 tx=0x7fab78009040 comp rx=0 tx=0).stop 2023-10-30T12:35:34.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.394+0000 7fab833ab640 1 -- 172.21.15.12:0/3049628666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab7c1538a0 msgr2=0x7fab7c0b42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:34.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.394+0000 7fab833ab640 1 --2- 172.21.15.12:0/3049628666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab7c1538a0 0x7fab7c0b42e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fab74001570 tx=0x7fab7400cc50 comp rx=0 tx=0).stop 2023-10-30T12:35:34.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.394+0000 7fab833ab640 1 -- 172.21.15.12:0/3049628666 shutdown_connections 2023-10-30T12:35:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.394+0000 7fab833ab640 1 --2- 172.21.15.12:0/3049628666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab7c1538a0 0x7fab7c0b42e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.394+0000 7fab833ab640 1 --2- 172.21.15.12:0/3049628666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab7c0cf160 0x7fab7c1507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.394+0000 7fab833ab640 1 --2- 172.21.15.12:0/3049628666 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fab600664a0 0x7fab60068960 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.394+0000 7fab833ab640 1 --2- 172.21.15.12:0/3049628666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab7c0ca0e0 0x7fab7c0c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.394+0000 7fab833ab640 1 -- 172.21.15.12:0/3049628666 >> 172.21.15.12:0/3049628666 conn(0x7fab7c06bfe0 msgr2=0x7fab7c152310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.394+0000 7fab833ab640 1 -- 172.21.15.12:0/3049628666 shutdown_connections 2023-10-30T12:35:34.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:34.394+0000 7fab833ab640 1 -- 172.21.15.12:0/3049628666 wait complete. 2023-10-30T12:35:34.417 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-30T12:35:34.417 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-30T12:35:34.417 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2023-10-30T12:35:34.418 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-30T12:35:34.418 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-30T12:35:34.418 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-30T12:35:34.418 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-30T12:35:34.419 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-30T12:35:34.419 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-30T12:35:34.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:34.420 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2023-10-30T12:35:34.430 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-30T12:35:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-30T12:35:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2023-10-30T12:35:34.430 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-10-30T12:35:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2023-10-30T12:35:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-10-30T12:35:34.431 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-30T12:35:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2023-10-30T12:35:34.431 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2023-10-30T12:35:34.432 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-30T12:35:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2023-10-30T12:35:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2023-10-30T12:35:34.432 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-30T12:35:34.432 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2023-10-30T12:35:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2023-10-30T12:35:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-30T12:35:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42403sec preferred_lft 42403sec 2023-10-30T12:35:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link 2023-10-30T12:35:34.433 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-10-30T12:35:34.433 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-30T12:35:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2023-10-30T12:35:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2023-10-30T12:35:34.434 INFO:teuthology.orchestra.run.smithi012.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-30T12:35:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 02:42:8b:7b:27:2b brd ff:ff:ff:ff:ff:ff 2023-10-30T12:35:34.434 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-30T12:35:34.435 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-10-30T12:35:34.435 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-30T12:35:34.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:34.436 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-10-30T12:35:34.436 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-10-30T12:35:34.436 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2023-10-30T12:35:34.437 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2023-10-30T12:35:34.437 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2023-10-30T12:35:34.437 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-30T12:35:34.438 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-10-30T12:35:34.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:34.544 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-30T12:35:34.565 INFO:teuthology.orchestra.run.smithi012.stdout:1 2023-10-30T12:35:34.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:34.567 DEBUG:teuthology.orchestra.run.smithi012:> route 2023-10-30T12:35:34.642 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2023-10-30T12:35:34.642 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-30T12:35:34.643 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-30T12:35:34.643 INFO:teuthology.orchestra.run.smithi012.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-30T12:35:34.643 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-30T12:35:34.643 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-30T12:35:34.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:34.644 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-10-30T12:35:34.644 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-10-30T12:35:34.645 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-30T12:35:34.645 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-30T12:35:34.645 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-30T12:35:34.646 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-10-30T12:35:34.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:34.737 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2023-10-30T12:35:34.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:34.747 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2023-10-30T12:35:34.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:34.799 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-10-30T12:35:34.800 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-10-30T12:35:34.800 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-30T12:35:34.800 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-30T12:35:34.800 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-10-30T12:35:34.880 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-30T12:35:34.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:34.881 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-10-30T12:35:34.881 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-10-30T12:35:34.882 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-30T12:35:34.882 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-30T12:35:34.882 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-30T12:35:34.883 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-30T12:35:34.883 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-30T12:35:34.883 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-10-30T12:35:35.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:35.056 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-10-30T12:35:35.056 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-10-30T12:35:35.057 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2023-10-30T12:35:35.057 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2023-10-30T12:35:35.057 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-10-30T12:35:35.101 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-30T12:35:35.102 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-30T12:35:35.153 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-30T12:35:35.153 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-30T12:35:35.154 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-30T12:35:35.208 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-30T12:35:35.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:35.209 DEBUG:teuthology.orchestra.run.smithi012:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-30T12:35:35.311 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-30T12:35:35.311 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-30T12:35:35.311 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2023-10-30T12:35:35.421 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-30T12:35:35.422 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-30T12:35:35.422 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2023-10-30T12:35:35.422 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-30T12:35:35.423 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-30T12:35:35.436 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-30T12:35:35.436 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-30T12:35:35.437 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2023-10-30T12:35:35.437 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-30T12:35:35.437 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-30T12:35:35.437 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:6789,172.21.15.154:6789,172.21.15.154:6790" 2023-10-30T12:35:35.437 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.12:6789,172.21.15.154:6789,172.21.15.154:6790:/ 2023-10-30T12:35:35.438 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=97b69bd8-a5ea-4668-9ffd-462651b43fe5" will pass to kernel 2023-10-30T12:35:35.440 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-30T12:35:35.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:35.441 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-30T12:35:35.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:35.470 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-30T12:35:35.541 INFO:teuthology.orchestra.run.smithi012.stdout:client4713 2023-10-30T12:35:35.541 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:35:35.542 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-30T12:35:35.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:35.593 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-30T12:35:35.656 INFO:teuthology.orchestra.run.smithi012.stdout:client4713 2023-10-30T12:35:35.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:35.657 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-30T12:35:36.214 INFO:teuthology.orchestra.run.smithi012.stdout:client4713 2023-10-30T12:35:36.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:35:36.214 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/97b69bd8-a5ea-4668-9ffd-462651b43fe5.client4713/status') 2023-10-30T12:35:36.245 INFO:teuthology.orchestra.run.smithi012.stdout:instance: client.4713 (3)192.168.0.1:0/3271102098 2023-10-30T12:35:36.245 INFO:teuthology.orchestra.run.smithi012.stdout:blocklisted: false 2023-10-30T12:35:36.246 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records in 2023-10-30T12:35:36.246 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records out 2023-10-30T12:35:36.246 INFO:teuthology.orchestra.run.smithi012.stderr:69 bytes copied, 5.905e-05 s, 1.2 MB/s 2023-10-30T12:35:36.247 INFO:teuthology.run_tasks:Running task mds_thrash... 2023-10-30T12:35:36.259 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:35:36.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.430+0000 7fd2eb1f5640 1 Processor -- start 2023-10-30T12:35:36.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.430+0000 7fd2eb1f5640 1 -- start start 2023-10-30T12:35:36.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.434+0000 7fd2eb1f5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd2e40c6b80 0x7fd2e40bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:36.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.434+0000 7fd2eb1f5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e40c0a50 con 0x7fd2e40c6f60 2023-10-30T12:35:36.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.434+0000 7fd2eb1f5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e406f6e0 con 0x7fd2e413f4a0 2023-10-30T12:35:36.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.434+0000 7fd2eb1f5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e406f9c0 con 0x7fd2e40c6b80 2023-10-30T12:35:36.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.434+0000 7fd2e8f91640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd2e40c6b80 0x7fd2e40bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:36.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.434+0000 7fd2e8f91640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd2e40c6b80 0x7fd2e40bfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57240/0 (socket says 172.21.15.12:57240) 2023-10-30T12:35:36.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.434+0000 7fd2e8f91640 1 -- 172.21.15.12:0/3794271529 learned_addr learned my addr 172.21.15.12:0/3794271529 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:36.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.434+0000 7fd2e3fff640 1 --1- 172.21.15.12:0/3794271529 >> v1:172.21.15.12:6789/0 conn(0x7fd2e40c6f60 0x7fd2e413ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54416/0 (socket says 172.21.15.12:54416) 2023-10-30T12:35:36.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.434+0000 7fd2e8f91640 1 -- 172.21.15.12:0/3794271529 >> v1:172.21.15.154:6789/0 conn(0x7fd2e413f4a0 legacy=0x7fd2e41566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:36.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.434+0000 7fd2e8f91640 1 -- 172.21.15.12:0/3794271529 >> v1:172.21.15.12:6789/0 conn(0x7fd2e40c6f60 legacy=0x7fd2e413ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:36.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.434+0000 7fd2e8f91640 1 -- 172.21.15.12:0/3794271529 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e406fca0 con 0x7fd2e40c6b80 2023-10-30T12:35:36.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.438+0000 7fd2e8f91640 1 --2- 172.21.15.12:0/3794271529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e40c6b80 0x7fd2e40bfec0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009870 tx=0x7fd2d801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f01d352e5c3d06 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:36.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.438+0000 7fd2e37fe640 1 -- 172.21.15.12:0/3794271529 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d80ab650 con 0x7fd2e40c6b80 2023-10-30T12:35:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.438+0000 7fd2e37fe640 1 -- 172.21.15.12:0/3794271529 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d80ab7b0 con 0x7fd2e40c6b80 2023-10-30T12:35:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.438+0000 7fd2e37fe640 1 -- 172.21.15.12:0/3794271529 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d80abb40 con 0x7fd2e40c6b80 2023-10-30T12:35:36.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.438+0000 7fd2eb1f5640 1 -- 172.21.15.12:0/3794271529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e40c6b80 msgr2=0x7fd2e40bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:36.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.438+0000 7fd2eb1f5640 1 --2- 172.21.15.12:0/3794271529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e40c6b80 0x7fd2e40bfec0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009870 tx=0x7fd2d801c540 comp rx=0 tx=0).stop 2023-10-30T12:35:36.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.438+0000 7fd2eb1f5640 1 -- 172.21.15.12:0/3794271529 shutdown_connections 2023-10-30T12:35:36.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.438+0000 7fd2eb1f5640 1 --2- 172.21.15.12:0/3794271529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e40c6b80 0x7fd2e40bfec0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.438+0000 7fd2eb1f5640 1 -- 172.21.15.12:0/3794271529 >> 172.21.15.12:0/3794271529 conn(0x7fd2e40c7910 msgr2=0x7fd2e40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:36.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.438+0000 7fd2eb1f5640 1 -- 172.21.15.12:0/3794271529 shutdown_connections 2023-10-30T12:35:36.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.438+0000 7fd2eb1f5640 1 -- 172.21.15.12:0/3794271529 wait complete. 2023-10-30T12:35:36.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.438+0000 7fd2eb1f5640 1 Processor -- start 2023-10-30T12:35:36.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.438+0000 7fd2eb1f5640 1 -- start start 2023-10-30T12:35:36.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2eb1f5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2e40c6b80 0x7fd2e4155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:36.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2eb1f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e40c6f60 0x7fd2e4156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:36.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2eb1f5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2e413f4a0 0x7fd2e4134600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:36.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2eb1f5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e4155e50 con 0x7fd2e40c6f60 2023-10-30T12:35:36.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2eb1f5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e41561b0 con 0x7fd2e413f4a0 2023-10-30T12:35:36.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2eb1f5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e41547e0 con 0x7fd2e40c6b80 2023-10-30T12:35:36.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2e3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e40c6f60 0x7fd2e4156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:36.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2e3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e40c6f60 0x7fd2e4156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57244/0 (socket says 172.21.15.12:57244) 2023-10-30T12:35:36.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2e3fff640 1 -- 172.21.15.12:0/1881730624 learned_addr learned my addr 172.21.15.12:0/1881730624 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:36.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2e8f91640 1 --2- 172.21.15.12:0/1881730624 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2e40c6b80 0x7fd2e4155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:36.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2e9792640 1 --2- 172.21.15.12:0/1881730624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2e413f4a0 0x7fd2e4134600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:36.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2e3fff640 1 -- 172.21.15.12:0/1881730624 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2e40c6b80 msgr2=0x7fd2e4155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:36.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2e3fff640 1 --2- 172.21.15.12:0/1881730624 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2e40c6b80 0x7fd2e4155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2e3fff640 1 -- 172.21.15.12:0/1881730624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2e413f4a0 msgr2=0x7fd2e4134600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:36.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2e3fff640 1 --2- 172.21.15.12:0/1881730624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2e413f4a0 0x7fd2e4134600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2e3fff640 1 -- 172.21.15.12:0/1881730624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d80094d0 con 0x7fd2e40c6f60 2023-10-30T12:35:36.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2e3fff640 1 --2- 172.21.15.12:0/1881730624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e40c6f60 0x7fd2e4156660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000c4c0 tx=0x7fd2d000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:36.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2e17fa640 1 -- 172.21.15.12:0/1881730624 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0007480 con 0x7fd2e40c6f60 2023-10-30T12:35:36.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2e17fa640 1 -- 172.21.15.12:0/1881730624 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d00075e0 con 0x7fd2e40c6f60 2023-10-30T12:35:36.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2eb1f5640 1 -- 172.21.15.12:0/1881730624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e4156ba0 con 0x7fd2e40c6f60 2023-10-30T12:35:36.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2e17fa640 1 -- 172.21.15.12:0/1881730624 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0004a10 con 0x7fd2e40c6f60 2023-10-30T12:35:36.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2eb1f5640 1 -- 172.21.15.12:0/1881730624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e40ca500 con 0x7fd2e40c6f60 2023-10-30T12:35:36.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2eb1f5640 1 -- 172.21.15.12:0/1881730624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e4156340 con 0x7fd2e40c6f60 2023-10-30T12:35:36.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.442+0000 7fd2e17fa640 1 -- 172.21.15.12:0/1881730624 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd2d0004b70 con 0x7fd2e40c6f60 2023-10-30T12:35:36.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.446+0000 7fd2e17fa640 1 --2- 172.21.15.12:0/1881730624 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd2c80664a0 0x7fd2c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:36.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.446+0000 7fd2e17fa640 1 -- 172.21.15.12:0/1881730624 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2d00833c0 con 0x7fd2e40c6f60 2023-10-30T12:35:36.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.446+0000 7fd2e8f91640 1 --2- 172.21.15.12:0/1881730624 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd2c80664a0 0x7fd2c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:36.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.446+0000 7fd2e8f91640 1 --2- 172.21.15.12:0/1881730624 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd2c80664a0 0x7fd2c8068960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80af000 tx=0x7fd2d80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:36.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.450+0000 7fd2e17fa640 1 -- 172.21.15.12:0/1881730624 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd2d00507a0 con 0x7fd2e40c6f60 2023-10-30T12:35:36.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.606+0000 7fd2eb1f5640 1 -- 172.21.15.12:0/1881730624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2e404e170 con 0x7fd2e40c6f60 2023-10-30T12:35:36.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.610+0000 7fd2e17fa640 1 -- 172.21.15.12:0/1881730624 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16505 (secure 0 0 0) 0x7fd2d0053dc0 con 0x7fd2e40c6f60 2023-10-30T12:35:36.612 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:36.613 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":22,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:35:07.153665+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:35:03.216053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:35:04.597162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:35:36.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.610+0000 7fd2eb1f5640 1 -- 172.21.15.12:0/1881730624 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd2c80664a0 msgr2=0x7fd2c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:36.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.610+0000 7fd2eb1f5640 1 --2- 172.21.15.12:0/1881730624 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd2c80664a0 0x7fd2c8068960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80af000 tx=0x7fd2d80a7000 comp rx=0 tx=0).stop 2023-10-30T12:35:36.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.610+0000 7fd2eb1f5640 1 -- 172.21.15.12:0/1881730624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e40c6f60 msgr2=0x7fd2e4156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:36.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.610+0000 7fd2eb1f5640 1 --2- 172.21.15.12:0/1881730624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e40c6f60 0x7fd2e4156660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000c4c0 tx=0x7fd2d000c980 comp rx=0 tx=0).stop 2023-10-30T12:35:36.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.610+0000 7fd2eb1f5640 1 -- 172.21.15.12:0/1881730624 shutdown_connections 2023-10-30T12:35:36.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.610+0000 7fd2eb1f5640 1 --2- 172.21.15.12:0/1881730624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2e413f4a0 0x7fd2e4134600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.610+0000 7fd2eb1f5640 1 --2- 172.21.15.12:0/1881730624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e40c6f60 0x7fd2e4156660 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.610+0000 7fd2eb1f5640 1 --2- 172.21.15.12:0/1881730624 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd2c80664a0 0x7fd2c8068960 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.610+0000 7fd2eb1f5640 1 --2- 172.21.15.12:0/1881730624 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2e40c6b80 0x7fd2e4155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.610+0000 7fd2eb1f5640 1 -- 172.21.15.12:0/1881730624 >> 172.21.15.12:0/1881730624 conn(0x7fd2e40c7910 msgr2=0x7fd2e4138bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:36.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.610+0000 7fd2eb1f5640 1 -- 172.21.15.12:0/1881730624 shutdown_connections 2023-10-30T12:35:36.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.610+0000 7fd2eb1f5640 1 -- 172.21.15.12:0/1881730624 wait complete. 2023-10-30T12:35:36.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:34:53.017612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-30T12:35:03.216053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-30T12:35:04.597162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:35:36.635 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:35:36.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.774+0000 7f67b139d640 1 Processor -- start 2023-10-30T12:35:36.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.774+0000 7f67b139d640 1 -- start start 2023-10-30T12:35:36.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67b139d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67ac153980 0x7f67ac0caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:36.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67b139d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f67ac159930 0x7f67ac0cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:36.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67b139d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ac15a920 con 0x7f67ac0cf1a0 2023-10-30T12:35:36.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67b139d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67ac0cb930 con 0x7f67ac159930 2023-10-30T12:35:36.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67b139d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac159dc0 con 0x7f67ac153980 2023-10-30T12:35:36.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67aaffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67ac153980 0x7f67ac0caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:36.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67a3fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f67ac159930 0x7f67ac0cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:36.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67ab7fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f67ac0cf1a0 0x7f67ac15d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:51050/0 (socket says 172.21.15.12:51050) 2023-10-30T12:35:36.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67ab7fe640 1 -- 172.21.15.12:0/873163086 learned_addr learned my addr 172.21.15.12:0/873163086 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:36.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67aaffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67ac153980 0x7f67ac0caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:37976/0 (socket says 172.21.15.12:37976) 2023-10-30T12:35:36.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67aa7fc640 1 -- 172.21.15.12:0/873163086 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070481968 0 0) 0x7f67ac15a920 con 0x7f67ac0cf1a0 2023-10-30T12:35:36.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67aa7fc640 1 -- 172.21.15.12:0/873163086 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6798003530 con 0x7f67ac0cf1a0 2023-10-30T12:35:36.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67aaffd640 1 -- 172.21.15.12:0/873163086 >> v2:172.21.15.154:3300/0 conn(0x7f67ac159930 msgr2=0x7f67ac0cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:36.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67aaffd640 1 --2- 172.21.15.12:0/873163086 >> v2:172.21.15.154:3300/0 conn(0x7f67ac159930 0x7f67ac0cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67aaffd640 1 -- 172.21.15.12:0/873163086 >> v1:172.21.15.154:6789/0 conn(0x7f67ac0cf1a0 legacy=0x7f67ac15d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:36.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.778+0000 7f67aaffd640 1 -- 172.21.15.12:0/873163086 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac15a080 con 0x7f67ac153980 2023-10-30T12:35:36.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.782+0000 7f67aaffd640 1 --2- 172.21.15.12:0/873163086 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67ac153980 0x7f67ac0caca0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f679c0097c0 tx=0x7f679c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1cabcd7c3403410 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:36.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.782+0000 7f67aa7fc640 1 -- 172.21.15.12:0/873163086 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c0ab650 con 0x7f67ac153980 2023-10-30T12:35:36.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.782+0000 7f67aa7fc640 1 -- 172.21.15.12:0/873163086 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c0ab7b0 con 0x7f67ac153980 2023-10-30T12:35:36.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.782+0000 7f67aa7fc640 1 -- 172.21.15.12:0/873163086 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c0abb40 con 0x7f67ac153980 2023-10-30T12:35:36.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.782+0000 7f67b139d640 1 -- 172.21.15.12:0/873163086 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67ac153980 msgr2=0x7f67ac0caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:36.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.782+0000 7f67b139d640 1 --2- 172.21.15.12:0/873163086 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67ac153980 0x7f67ac0caca0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f679c0097c0 tx=0x7f679c01c450 comp rx=0 tx=0).stop 2023-10-30T12:35:36.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.782+0000 7f67b139d640 1 -- 172.21.15.12:0/873163086 shutdown_connections 2023-10-30T12:35:36.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.782+0000 7f67b139d640 1 --2- 172.21.15.12:0/873163086 >> v2:172.21.15.154:3300/0 conn(0x7f67ac159930 0x7f67ac0cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.782+0000 7f67b139d640 1 --2- 172.21.15.12:0/873163086 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67ac153980 0x7f67ac0caca0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.782+0000 7f67b139d640 1 -- 172.21.15.12:0/873163086 >> 172.21.15.12:0/873163086 conn(0x7f67ac06c0f0 msgr2=0x7f67ac06c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:36.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.782+0000 7f67b139d640 1 -- 172.21.15.12:0/873163086 shutdown_connections 2023-10-30T12:35:36.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.782+0000 7f67b139d640 1 -- 172.21.15.12:0/873163086 wait complete. 2023-10-30T12:35:36.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67b139d640 1 Processor -- start 2023-10-30T12:35:36.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67b139d640 1 -- start start 2023-10-30T12:35:36.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67b139d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67ac0cf1a0 0x7f67ac1f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:36.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67b139d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67ac153980 0x7f67ac1474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:36.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67b139d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67ac0c51f0 0x7f67ac0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:36.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67b139d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac0cd330 con 0x7f67ac0c51f0 2023-10-30T12:35:36.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67b139d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac147ac0 con 0x7f67ac153980 2023-10-30T12:35:36.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67b139d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac147da0 con 0x7f67ac0cf1a0 2023-10-30T12:35:36.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67ab7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67ac0c51f0 0x7f67ac0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:36.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67ab7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67ac0c51f0 0x7f67ac0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57260/0 (socket says 172.21.15.12:57260) 2023-10-30T12:35:36.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67ab7fe640 1 -- 172.21.15.12:0/794110009 learned_addr learned my addr 172.21.15.12:0/794110009 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:36.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67a3fff640 1 --2- 172.21.15.12:0/794110009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67ac153980 0x7f67ac1474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:36.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67aaffd640 1 --2- 172.21.15.12:0/794110009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67ac0cf1a0 0x7f67ac1f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:36.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67ab7fe640 1 -- 172.21.15.12:0/794110009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67ac0cf1a0 msgr2=0x7f67ac1f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:36.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67ab7fe640 1 --2- 172.21.15.12:0/794110009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67ac0cf1a0 0x7f67ac1f1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67ab7fe640 1 -- 172.21.15.12:0/794110009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67ac153980 msgr2=0x7f67ac1474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:36.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67ab7fe640 1 --2- 172.21.15.12:0/794110009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67ac153980 0x7f67ac1474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67ab7fe640 1 -- 172.21.15.12:0/794110009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c009420 con 0x7f67ac0c51f0 2023-10-30T12:35:36.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67ab7fe640 1 --2- 172.21.15.12:0/794110009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67ac0c51f0 0x7f67ac0cd7c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f67a400cb40 tx=0x7f67a400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:36.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67a37fe640 1 -- 172.21.15.12:0/794110009 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a400aca0 con 0x7f67ac0c51f0 2023-10-30T12:35:36.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67a37fe640 1 -- 172.21.15.12:0/794110009 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a400ae00 con 0x7f67ac0c51f0 2023-10-30T12:35:36.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67a37fe640 1 -- 172.21.15.12:0/794110009 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4012620 con 0x7f67ac0c51f0 2023-10-30T12:35:36.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67b139d640 1 -- 172.21.15.12:0/794110009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ac148080 con 0x7f67ac0c51f0 2023-10-30T12:35:36.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67b139d640 1 -- 172.21.15.12:0/794110009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ac148640 con 0x7f67ac0c51f0 2023-10-30T12:35:36.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.786+0000 7f67b139d640 1 -- 172.21.15.12:0/794110009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ac04e170 con 0x7f67ac0c51f0 2023-10-30T12:35:36.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.790+0000 7f67a37fe640 1 -- 172.21.15.12:0/794110009 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f67a4007070 con 0x7f67ac0c51f0 2023-10-30T12:35:36.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.790+0000 7f67a37fe640 1 --2- 172.21.15.12:0/794110009 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f678c0664a0 0x7f678c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:36.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.790+0000 7f67a37fe640 1 -- 172.21.15.12:0/794110009 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67a4083e60 con 0x7f67ac0c51f0 2023-10-30T12:35:36.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.790+0000 7f67aaffd640 1 --2- 172.21.15.12:0/794110009 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f678c0664a0 0x7f678c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:36.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.790+0000 7f67aaffd640 1 --2- 172.21.15.12:0/794110009 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f678c0664a0 0x7f678c068960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f679c0af000 tx=0x7f679c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:36.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.794+0000 7f67a37fe640 1 -- 172.21.15.12:0/794110009 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f67a4051380 con 0x7f67ac0c51f0 2023-10-30T12:35:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.950+0000 7f67b139d640 1 -- 172.21.15.12:0/794110009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67ac0cd470 con 0x7f67ac0c51f0 2023-10-30T12:35:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.950+0000 7f67a37fe640 1 -- 172.21.15.12:0/794110009 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f67a40170a0 con 0x7f67ac0c51f0 2023-10-30T12:35:36.955 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-10-30T12:35:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.954+0000 7f67b139d640 1 -- 172.21.15.12:0/794110009 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f678c0664a0 msgr2=0x7f678c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.954+0000 7f67b139d640 1 --2- 172.21.15.12:0/794110009 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f678c0664a0 0x7f678c068960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f679c0af000 tx=0x7f679c0a7000 comp rx=0 tx=0).stop 2023-10-30T12:35:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.954+0000 7f67b139d640 1 -- 172.21.15.12:0/794110009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67ac0c51f0 msgr2=0x7f67ac0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.954+0000 7f67b139d640 1 --2- 172.21.15.12:0/794110009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67ac0c51f0 0x7f67ac0cd7c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f67a400cb40 tx=0x7f67a400a060 comp rx=0 tx=0).stop 2023-10-30T12:35:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.954+0000 7f67b139d640 1 -- 172.21.15.12:0/794110009 shutdown_connections 2023-10-30T12:35:36.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.954+0000 7f67b139d640 1 --2- 172.21.15.12:0/794110009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67ac0c51f0 0x7f67ac0cd7c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.954+0000 7f67b139d640 1 --2- 172.21.15.12:0/794110009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67ac153980 0x7f67ac1474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.954+0000 7f67b139d640 1 --2- 172.21.15.12:0/794110009 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f678c0664a0 0x7f678c068960 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.954+0000 7f67b139d640 1 --2- 172.21.15.12:0/794110009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67ac0cf1a0 0x7f67ac1f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:36.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.954+0000 7f67b139d640 1 -- 172.21.15.12:0/794110009 >> 172.21.15.12:0/794110009 conn(0x7f67ac06c0f0 msgr2=0x7f67ac0c9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:36.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.954+0000 7f67b139d640 1 -- 172.21.15.12:0/794110009 shutdown_connections 2023-10-30T12:35:36.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:36.954+0000 7f67b139d640 1 -- 172.21.15.12:0/794110009 wait complete. 2023-10-30T12:35:36.976 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-30T12:35:37.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.118+0000 7f88d533e640 1 Processor -- start 2023-10-30T12:35:37.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.118+0000 7f88d533e640 1 -- start start 2023-10-30T12:35:37.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.118+0000 7f88d533e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f88d006f880 0x7f88d006a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.118+0000 7f88d533e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f88d006a7b0 0x7f88d0161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.118+0000 7f88d533e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d00c6cb0 con 0x7f88d00c5810 2023-10-30T12:35:37.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.118+0000 7f88d533e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d0071d90 con 0x7f88d006a7b0 2023-10-30T12:35:37.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.118+0000 7f88d533e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d01585d0 con 0x7f88d006f880 2023-10-30T12:35:37.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.118+0000 7f88cf7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f88d006a7b0 0x7f88d0161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.118+0000 7f88cf7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f88d006a7b0 0x7f88d0161f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57272/0 (socket says 172.21.15.12:57272) 2023-10-30T12:35:37.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.118+0000 7f88cf7fe640 1 -- 172.21.15.12:0/1916756666 learned_addr learned my addr 172.21.15.12:0/1916756666 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:37.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.118+0000 7f88c7fff640 1 --2- 172.21.15.12:0/1916756666 >> v2:172.21.15.154:3300/0 conn(0x7f88d006f880 0x7f88d006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.118+0000 7f88cf7fe640 1 -- 172.21.15.12:0/1916756666 >> v2:172.21.15.154:3300/0 conn(0x7f88d006f880 msgr2=0x7f88d006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:37.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.118+0000 7f88cf7fe640 1 --2- 172.21.15.12:0/1916756666 >> v2:172.21.15.154:3300/0 conn(0x7f88d006f880 0x7f88d006a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.118+0000 7f88cf7fe640 1 -- 172.21.15.12:0/1916756666 >> v1:172.21.15.154:6789/0 conn(0x7f88d00c5810 legacy=0x7f88d00c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.118+0000 7f88cf7fe640 1 -- 172.21.15.12:0/1916756666 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d00c6620 con 0x7f88d006a7b0 2023-10-30T12:35:37.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.122+0000 7f88cf7fe640 1 --2- 172.21.15.12:0/1916756666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d006a7b0 0x7f88d0161f10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f88d00c8cc0 tx=0x7f88c801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667d514440d0fb07 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:37.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.122+0000 7f88ce7fc640 1 -- 172.21.15.12:0/1916756666 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c80a5680 con 0x7f88d006a7b0 2023-10-30T12:35:37.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.122+0000 7f88ce7fc640 1 -- 172.21.15.12:0/1916756666 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c80a57e0 con 0x7f88d006a7b0 2023-10-30T12:35:37.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.122+0000 7f88ce7fc640 1 -- 172.21.15.12:0/1916756666 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c80a5b70 con 0x7f88d006a7b0 2023-10-30T12:35:37.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.122+0000 7f88d533e640 1 -- 172.21.15.12:0/1916756666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d006a7b0 msgr2=0x7f88d0161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.122+0000 7f88d533e640 1 --2- 172.21.15.12:0/1916756666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d006a7b0 0x7f88d0161f10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f88d00c8cc0 tx=0x7f88c801c8b0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.122+0000 7f88d533e640 1 -- 172.21.15.12:0/1916756666 shutdown_connections 2023-10-30T12:35:37.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.122+0000 7f88d533e640 1 --2- 172.21.15.12:0/1916756666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d006a7b0 0x7f88d0161f10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.122+0000 7f88d533e640 1 --2- 172.21.15.12:0/1916756666 >> v2:172.21.15.154:3300/0 conn(0x7f88d006f880 0x7f88d006a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.122+0000 7f88d533e640 1 -- 172.21.15.12:0/1916756666 >> 172.21.15.12:0/1916756666 conn(0x7f88d00c7910 msgr2=0x7f88d00c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:37.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.122+0000 7f88d533e640 1 -- 172.21.15.12:0/1916756666 shutdown_connections 2023-10-30T12:35:37.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.122+0000 7f88d533e640 1 -- 172.21.15.12:0/1916756666 wait complete. 2023-10-30T12:35:37.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88d533e640 1 Processor -- start 2023-10-30T12:35:37.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88d533e640 1 -- start start 2023-10-30T12:35:37.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88d533e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88d006a7b0 0x7f88d0150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88d533e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d006f880 0x7f88d00c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88d533e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d00c5810 0x7f88d0143800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88d533e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00ca540 con 0x7f88d00c5810 2023-10-30T12:35:37.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88d533e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d0149ce0 con 0x7f88d006a7b0 2023-10-30T12:35:37.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88d533e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00c92e0 con 0x7f88d006f880 2023-10-30T12:35:37.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88cf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d00c5810 0x7f88d0143800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88cf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d00c5810 0x7f88d0143800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57286/0 (socket says 172.21.15.12:57286) 2023-10-30T12:35:37.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88cf7fe640 1 -- 172.21.15.12:0/4017403214 learned_addr learned my addr 172.21.15.12:0/4017403214 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:37.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88ceffd640 1 --2- 172.21.15.12:0/4017403214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88d006a7b0 0x7f88d0150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88c7fff640 1 --2- 172.21.15.12:0/4017403214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d006f880 0x7f88d00c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88cf7fe640 1 -- 172.21.15.12:0/4017403214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d006f880 msgr2=0x7f88d00c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88cf7fe640 1 --2- 172.21.15.12:0/4017403214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d006f880 0x7f88d00c25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88cf7fe640 1 -- 172.21.15.12:0/4017403214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88d006a7b0 msgr2=0x7f88d0150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88cf7fe640 1 --2- 172.21.15.12:0/4017403214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88d006a7b0 0x7f88d0150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88cf7fe640 1 -- 172.21.15.12:0/4017403214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c8009b30 con 0x7f88d00c5810 2023-10-30T12:35:37.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88ceffd640 1 --2- 172.21.15.12:0/4017403214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88d006a7b0 0x7f88d0150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:37.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88cf7fe640 1 --2- 172.21.15.12:0/4017403214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d00c5810 0x7f88d0143800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f88c80194c0 tx=0x7f88c8019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:37.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88c77fe640 1 -- 172.21.15.12:0/4017403214 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c80b8670 con 0x7f88d00c5810 2023-10-30T12:35:37.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88c77fe640 1 -- 172.21.15.12:0/4017403214 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c80b87f0 con 0x7f88d00c5810 2023-10-30T12:35:37.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88d533e640 1 -- 172.21.15.12:0/4017403214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d00c9810 con 0x7f88d00c5810 2023-10-30T12:35:37.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88c77fe640 1 -- 172.21.15.12:0/4017403214 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c80b8ba0 con 0x7f88d00c5810 2023-10-30T12:35:37.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.126+0000 7f88d533e640 1 -- 172.21.15.12:0/4017403214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d00c9b60 con 0x7f88d00c5810 2023-10-30T12:35:37.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.130+0000 7f88d533e640 1 -- 172.21.15.12:0/4017403214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d00ca6b0 con 0x7f88d00c5810 2023-10-30T12:35:37.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.130+0000 7f88c77fe640 1 -- 172.21.15.12:0/4017403214 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f88c80b8d00 con 0x7f88d00c5810 2023-10-30T12:35:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.130+0000 7f88c77fe640 1 --2- 172.21.15.12:0/4017403214 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f88b00664a0 0x7f88b0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.130+0000 7f88c77fe640 1 -- 172.21.15.12:0/4017403214 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88c811f260 con 0x7f88d00c5810 2023-10-30T12:35:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.130+0000 7f88ceffd640 1 --2- 172.21.15.12:0/4017403214 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f88b00664a0 0x7f88b0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.134+0000 7f88ceffd640 1 --2- 172.21.15.12:0/4017403214 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f88b00664a0 0x7f88b0068960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f88c0007fd0 tx=0x7f88c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:37.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.134+0000 7f88c77fe640 1 -- 172.21.15.12:0/4017403214 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f88d00ca6b0 con 0x7f88d00c5810 2023-10-30T12:35:37.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.290+0000 7f88d533e640 1 -- 172.21.15.12:0/4017403214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f88d00ca680 con 0x7f88d00c5810 2023-10-30T12:35:37.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.290+0000 7f88c77fe640 1 -- 172.21.15.12:0/4017403214 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f88c80ec6f0 con 0x7f88d00c5810 2023-10-30T12:35:37.296 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-10-30T12:35:37.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.294+0000 7f88d533e640 1 -- 172.21.15.12:0/4017403214 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f88b00664a0 msgr2=0x7f88b0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.294+0000 7f88d533e640 1 --2- 172.21.15.12:0/4017403214 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f88b00664a0 0x7f88b0068960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f88c0007fd0 tx=0x7f88c0009040 comp rx=0 tx=0).stop 2023-10-30T12:35:37.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.294+0000 7f88d533e640 1 -- 172.21.15.12:0/4017403214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d00c5810 msgr2=0x7f88d0143800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.294+0000 7f88d533e640 1 --2- 172.21.15.12:0/4017403214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d00c5810 0x7f88d0143800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f88c80194c0 tx=0x7f88c8019650 comp rx=0 tx=0).stop 2023-10-30T12:35:37.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.294+0000 7f88d533e640 1 -- 172.21.15.12:0/4017403214 shutdown_connections 2023-10-30T12:35:37.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.294+0000 7f88d533e640 1 --2- 172.21.15.12:0/4017403214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d00c5810 0x7f88d0143800 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.294+0000 7f88d533e640 1 --2- 172.21.15.12:0/4017403214 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f88b00664a0 0x7f88b0068960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.294+0000 7f88d533e640 1 --2- 172.21.15.12:0/4017403214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d006f880 0x7f88d00c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.294+0000 7f88d533e640 1 --2- 172.21.15.12:0/4017403214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88d006a7b0 0x7f88d0150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.294+0000 7f88d533e640 1 -- 172.21.15.12:0/4017403214 >> 172.21.15.12:0/4017403214 conn(0x7f88d00c7910 msgr2=0x7f88d0155e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:37.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.294+0000 7f88d533e640 1 -- 172.21.15.12:0/4017403214 shutdown_connections 2023-10-30T12:35:37.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.294+0000 7f88d533e640 1 -- 172.21.15.12:0/4017403214 wait complete. 2023-10-30T12:35:37.316 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-30T12:35:37.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.458+0000 7f1fb3e54640 1 Processor -- start 2023-10-30T12:35:37.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.462+0000 7f1fb3e54640 1 -- start start 2023-10-30T12:35:37.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.462+0000 7f1fb3e54640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1fac159930 0x7f1fac0cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.462+0000 7f1fb3e54640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1fac0cf6e0 0x7f1fac15d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.462+0000 7f1fb3e54640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fac15a920 con 0x7f1fac153980 2023-10-30T12:35:37.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.462+0000 7f1fb3e54640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fac0cb930 con 0x7f1fac159930 2023-10-30T12:35:37.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.462+0000 7f1fb3e54640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fac159dc0 con 0x7f1fac0cf6e0 2023-10-30T12:35:37.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.462+0000 7f1fb1bf0640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f1fac153980 0x7f1fac0caca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54422/0 (socket says 172.21.15.12:54422) 2023-10-30T12:35:37.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.462+0000 7f1fb1bf0640 1 -- 172.21.15.12:0/3945471859 learned_addr learned my addr 172.21.15.12:0/3945471859 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:37.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.462+0000 7f1fb13ef640 1 --2- 172.21.15.12:0/3945471859 >> v2:172.21.15.12:3300/0 conn(0x7f1fac159930 0x7f1fac0cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.462+0000 7f1fb23f1640 1 --2- 172.21.15.12:0/3945471859 >> v2:172.21.15.154:3300/0 conn(0x7f1fac0cf6e0 0x7f1fac15d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.462+0000 7f1fb13ef640 1 -- 172.21.15.12:0/3945471859 >> v2:172.21.15.154:3300/0 conn(0x7f1fac0cf6e0 msgr2=0x7f1fac15d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:37.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.462+0000 7f1fb13ef640 1 --2- 172.21.15.12:0/3945471859 >> v2:172.21.15.154:3300/0 conn(0x7f1fac0cf6e0 0x7f1fac15d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.462+0000 7f1fb0bee640 1 -- 172.21.15.12:0/3945471859 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574089232 0 0) 0x7f1fac15a920 con 0x7f1fac153980 2023-10-30T12:35:37.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.462+0000 7f1fb13ef640 1 -- 172.21.15.12:0/3945471859 >> v1:172.21.15.12:6789/0 conn(0x7f1fac153980 legacy=0x7f1fac0caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.462+0000 7f1fb13ef640 1 -- 172.21.15.12:0/3945471859 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac15a080 con 0x7f1fac159930 2023-10-30T12:35:37.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.466+0000 7f1fb13ef640 1 --2- 172.21.15.12:0/3945471859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac159930 0x7f1fac0cf1a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0b1cf0 tx=0x7f1f9c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa41dc1aead4a3f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:37.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.466+0000 7f1fb0bee640 1 -- 172.21.15.12:0/3945471859 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0ab600 con 0x7f1fac159930 2023-10-30T12:35:37.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.466+0000 7f1fb0bee640 1 -- 172.21.15.12:0/3945471859 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c0a7050 con 0x7f1fac159930 2023-10-30T12:35:37.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.466+0000 7f1fb0bee640 1 -- 172.21.15.12:0/3945471859 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0ab980 con 0x7f1fac159930 2023-10-30T12:35:37.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.466+0000 7f1fb3e54640 1 -- 172.21.15.12:0/3945471859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac159930 msgr2=0x7f1fac0cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.466+0000 7f1fb3e54640 1 --2- 172.21.15.12:0/3945471859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac159930 0x7f1fac0cf1a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0b1cf0 tx=0x7f1f9c01c430 comp rx=0 tx=0).stop 2023-10-30T12:35:37.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.466+0000 7f1fb3e54640 1 -- 172.21.15.12:0/3945471859 shutdown_connections 2023-10-30T12:35:37.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.466+0000 7f1fb3e54640 1 --2- 172.21.15.12:0/3945471859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac159930 0x7f1fac0cf1a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.466+0000 7f1fb3e54640 1 --2- 172.21.15.12:0/3945471859 >> v2:172.21.15.154:3300/0 conn(0x7f1fac0cf6e0 0x7f1fac15d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.466+0000 7f1fb3e54640 1 -- 172.21.15.12:0/3945471859 >> 172.21.15.12:0/3945471859 conn(0x7f1fac06c0f0 msgr2=0x7f1fac06c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:37.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.466+0000 7f1fb3e54640 1 -- 172.21.15.12:0/3945471859 shutdown_connections 2023-10-30T12:35:37.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.466+0000 7f1fb3e54640 1 -- 172.21.15.12:0/3945471859 wait complete. 2023-10-30T12:35:37.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.466+0000 7f1fb3e54640 1 Processor -- start 2023-10-30T12:35:37.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.466+0000 7f1fb3e54640 1 -- start start 2023-10-30T12:35:37.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb3e54640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac0cf6e0 0x7f1fac0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb3e54640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fac153980 0x7f1fac1f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb3e54640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac0cdd00 0x7f1fac13dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb3e54640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0ce350 con 0x7f1fac0cdd00 2023-10-30T12:35:37.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb3e54640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac14a4c0 con 0x7f1fac153980 2023-10-30T12:35:37.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb3e54640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac14a1e0 con 0x7f1fac0cf6e0 2023-10-30T12:35:37.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb23f1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac0cdd00 0x7f1fac13dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb23f1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac0cdd00 0x7f1fac13dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57302/0 (socket says 172.21.15.12:57302) 2023-10-30T12:35:37.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb23f1640 1 -- 172.21.15.12:0/448202217 learned_addr learned my addr 172.21.15.12:0/448202217 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:37.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb13ef640 1 --2- 172.21.15.12:0/448202217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fac153980 0x7f1fac1f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb1bf0640 1 --2- 172.21.15.12:0/448202217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac0cf6e0 0x7f1fac0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb23f1640 1 -- 172.21.15.12:0/448202217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac0cf6e0 msgr2=0x7f1fac0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb23f1640 1 --2- 172.21.15.12:0/448202217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac0cf6e0 0x7f1fac0ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb23f1640 1 -- 172.21.15.12:0/448202217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fac153980 msgr2=0x7f1fac1f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb23f1640 1 --2- 172.21.15.12:0/448202217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fac153980 0x7f1fac1f3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb23f1640 1 -- 172.21.15.12:0/448202217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c009420 con 0x7f1fac0cdd00 2023-10-30T12:35:37.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb23f1640 1 --2- 172.21.15.12:0/448202217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac0cdd00 0x7f1fac13dba0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4007e40 tx=0x7f1fa400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:37.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fa27fc640 1 -- 172.21.15.12:0/448202217 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4014620 con 0x7f1fac0cdd00 2023-10-30T12:35:37.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fa27fc640 1 -- 172.21.15.12:0/448202217 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa400ee90 con 0x7f1fac0cdd00 2023-10-30T12:35:37.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fa27fc640 1 -- 172.21.15.12:0/448202217 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa40149e0 con 0x7f1fac0cdd00 2023-10-30T12:35:37.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb3e54640 1 -- 172.21.15.12:0/448202217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fac143320 con 0x7f1fac0cdd00 2023-10-30T12:35:37.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb3e54640 1 -- 172.21.15.12:0/448202217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fac1438e0 con 0x7f1fac0cdd00 2023-10-30T12:35:37.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.470+0000 7f1fb3e54640 1 -- 172.21.15.12:0/448202217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fac04e170 con 0x7f1fac0cdd00 2023-10-30T12:35:37.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.474+0000 7f1fa27fc640 1 -- 172.21.15.12:0/448202217 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1fa4014b40 con 0x7f1fac0cdd00 2023-10-30T12:35:37.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.474+0000 7f1fa27fc640 1 --2- 172.21.15.12:0/448202217 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1f900664a0 0x7f1f90068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.474+0000 7f1fa27fc640 1 -- 172.21.15.12:0/448202217 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fa4084710 con 0x7f1fac0cdd00 2023-10-30T12:35:37.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.474+0000 7f1fb1bf0640 1 --2- 172.21.15.12:0/448202217 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1f900664a0 0x7f1f90068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.474+0000 7f1fb1bf0640 1 --2- 172.21.15.12:0/448202217 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1f900664a0 0x7f1f90068960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8000f70 tx=0x7f1fa8007b60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:37.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.478+0000 7f1fa27fc640 1 -- 172.21.15.12:0/448202217 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1fa4051c20 con 0x7f1fac0cdd00 2023-10-30T12:35:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.634+0000 7f1fb3e54640 1 -- 172.21.15.12:0/448202217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1fac0caa80 con 0x7f1fac0cdd00 2023-10-30T12:35:37.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.634+0000 7f1fa27fc640 1 -- 172.21.15.12:0/448202217 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1fa4055240 con 0x7f1fac0cdd00 2023-10-30T12:35:37.638 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-10-30T12:35:37.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.634+0000 7f1fb3e54640 1 -- 172.21.15.12:0/448202217 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1f900664a0 msgr2=0x7f1f90068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.634+0000 7f1fb3e54640 1 --2- 172.21.15.12:0/448202217 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1f900664a0 0x7f1f90068960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8000f70 tx=0x7f1fa8007b60 comp rx=0 tx=0).stop 2023-10-30T12:35:37.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.634+0000 7f1fb3e54640 1 -- 172.21.15.12:0/448202217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac0cdd00 msgr2=0x7f1fac13dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.634+0000 7f1fb3e54640 1 --2- 172.21.15.12:0/448202217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac0cdd00 0x7f1fac13dba0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4007e40 tx=0x7f1fa400e4b0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.634+0000 7f1fb3e54640 1 -- 172.21.15.12:0/448202217 shutdown_connections 2023-10-30T12:35:37.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.634+0000 7f1fb3e54640 1 --2- 172.21.15.12:0/448202217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac0cdd00 0x7f1fac13dba0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.634+0000 7f1fb3e54640 1 --2- 172.21.15.12:0/448202217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fac153980 0x7f1fac1f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.634+0000 7f1fb3e54640 1 --2- 172.21.15.12:0/448202217 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1f900664a0 0x7f1f90068960 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.634+0000 7f1fb3e54640 1 --2- 172.21.15.12:0/448202217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fac0cf6e0 0x7f1fac0ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.634+0000 7f1fb3e54640 1 -- 172.21.15.12:0/448202217 >> 172.21.15.12:0/448202217 conn(0x7f1fac06c0f0 msgr2=0x7f1fac158a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:37.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.638+0000 7f1fb3e54640 1 -- 172.21.15.12:0/448202217 shutdown_connections 2023-10-30T12:35:37.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.638+0000 7f1fb3e54640 1 -- 172.21.15.12:0/448202217 wait complete. 2023-10-30T12:35:37.659 INFO:tasks.mds_thrash:mds thrasher using random seed: 1698669337 2023-10-30T12:35:37.660 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:35:37.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.798+0000 7f20d6279640 1 Processor -- start 2023-10-30T12:35:37.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.798+0000 7f20d6279640 1 -- start start 2023-10-30T12:35:37.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.798+0000 7f20d6279640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f20d014fd50 0x7f20d00c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.798+0000 7f20d6279640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20d0071960 0x7f20d00cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.798+0000 7f20d6279640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d01560a0 con 0x7f20d00cb550 2023-10-30T12:35:37.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.798+0000 7f20d6279640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d0156380 con 0x7f20d014fd50 2023-10-30T12:35:37.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.798+0000 7f20d6279640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d0156660 con 0x7f20d0071960 2023-10-30T12:35:37.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.798+0000 7f20cffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f20d014fd50 0x7f20d00c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.798+0000 7f20cffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f20d014fd50 0x7f20d00c65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57314/0 (socket says 172.21.15.12:57314) 2023-10-30T12:35:37.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.798+0000 7f20cffff640 1 -- 172.21.15.12:0/484529785 learned_addr learned my addr 172.21.15.12:0/484529785 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:37.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.802+0000 7f20d4816640 1 --2- 172.21.15.12:0/484529785 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20d0071960 0x7f20d00cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.802+0000 7f20cffff640 1 -- 172.21.15.12:0/484529785 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20d0071960 msgr2=0x7f20d00cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:37.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.802+0000 7f20cffff640 1 --2- 172.21.15.12:0/484529785 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20d0071960 0x7f20d00cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.802+0000 7f20cffff640 1 -- 172.21.15.12:0/484529785 >> v1:172.21.15.154:6789/0 conn(0x7f20d00cb550 legacy=0x7f20d0071250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.802+0000 7f20cffff640 1 -- 172.21.15.12:0/484529785 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d0156940 con 0x7f20d014fd50 2023-10-30T12:35:37.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.802+0000 7f20cffff640 1 --2- 172.21.15.12:0/484529785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d014fd50 0x7f20d00c65d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009870 tx=0x7f20c001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bd6ebb5a6e56c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:37.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.802+0000 7f20ceffd640 1 -- 172.21.15.12:0/484529785 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c00ab650 con 0x7f20d014fd50 2023-10-30T12:35:37.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.802+0000 7f20ceffd640 1 -- 172.21.15.12:0/484529785 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c00ab7b0 con 0x7f20d014fd50 2023-10-30T12:35:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.802+0000 7f20ceffd640 1 -- 172.21.15.12:0/484529785 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c00abb40 con 0x7f20d014fd50 2023-10-30T12:35:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.802+0000 7f20d6279640 1 -- 172.21.15.12:0/484529785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d014fd50 msgr2=0x7f20d00c65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.802+0000 7f20d6279640 1 --2- 172.21.15.12:0/484529785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d014fd50 0x7f20d00c65d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009870 tx=0x7f20c001c510 comp rx=0 tx=0).stop 2023-10-30T12:35:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d6279640 1 -- 172.21.15.12:0/484529785 shutdown_connections 2023-10-30T12:35:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d6279640 1 --2- 172.21.15.12:0/484529785 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20d0071960 0x7f20d00cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d6279640 1 --2- 172.21.15.12:0/484529785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d014fd50 0x7f20d00c65d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d6279640 1 -- 172.21.15.12:0/484529785 >> 172.21.15.12:0/484529785 conn(0x7f20d006c6e0 msgr2=0x7f20d0069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d6279640 1 -- 172.21.15.12:0/484529785 shutdown_connections 2023-10-30T12:35:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d6279640 1 -- 172.21.15.12:0/484529785 wait complete. 2023-10-30T12:35:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d6279640 1 Processor -- start 2023-10-30T12:35:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d6279640 1 -- start start 2023-10-30T12:35:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d6279640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20d0071960 0x7f20d01f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d6279640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20d00cb550 0x7f20d0145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d6279640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d014fd50 0x7f20d00c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d6279640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d00c38f0 con 0x7f20d014fd50 2023-10-30T12:35:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d6279640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d00c7ae0 con 0x7f20d0071960 2023-10-30T12:35:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d6279640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d00c7dc0 con 0x7f20d00cb550 2023-10-30T12:35:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d4816640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d014fd50 0x7f20d00c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d4816640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d014fd50 0x7f20d00c3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57326/0 (socket says 172.21.15.12:57326) 2023-10-30T12:35:37.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d4816640 1 -- 172.21.15.12:0/3858608356 learned_addr learned my addr 172.21.15.12:0/3858608356 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:37.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20cffff640 1 --2- 172.21.15.12:0/3858608356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20d0071960 0x7f20d01f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20cf7fe640 1 --2- 172.21.15.12:0/3858608356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20d00cb550 0x7f20d0145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d4816640 1 -- 172.21.15.12:0/3858608356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20d00cb550 msgr2=0x7f20d0145100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d4816640 1 --2- 172.21.15.12:0/3858608356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20d00cb550 0x7f20d0145100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d4816640 1 -- 172.21.15.12:0/3858608356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20d0071960 msgr2=0x7f20d01f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d4816640 1 --2- 172.21.15.12:0/3858608356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20d0071960 0x7f20d01f2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.806+0000 7f20d4816640 1 -- 172.21.15.12:0/3858608356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c00094d0 con 0x7f20d014fd50 2023-10-30T12:35:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.810+0000 7f20d4816640 1 --2- 172.21.15.12:0/3858608356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d014fd50 0x7f20d00c3d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f20c8007e60 tx=0x7f20c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.810+0000 7f20ccff9640 1 -- 172.21.15.12:0/3858608356 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8014420 con 0x7f20d014fd50 2023-10-30T12:35:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.810+0000 7f20ccff9640 1 -- 172.21.15.12:0/3858608356 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c80145a0 con 0x7f20d014fd50 2023-10-30T12:35:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.810+0000 7f20ccff9640 1 -- 172.21.15.12:0/3858608356 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8014950 con 0x7f20d014fd50 2023-10-30T12:35:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.810+0000 7f20d6279640 1 -- 172.21.15.12:0/3858608356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d00c80a0 con 0x7f20d014fd50 2023-10-30T12:35:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.810+0000 7f20d6279640 1 -- 172.21.15.12:0/3858608356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d00c9d90 con 0x7f20d014fd50 2023-10-30T12:35:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.810+0000 7f20d6279640 1 -- 172.21.15.12:0/3858608356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d00c3aa0 con 0x7f20d014fd50 2023-10-30T12:35:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.810+0000 7f20ccff9640 1 -- 172.21.15.12:0/3858608356 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f20c8014ab0 con 0x7f20d014fd50 2023-10-30T12:35:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.810+0000 7f20ccff9640 1 --2- 172.21.15.12:0/3858608356 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20bc0664a0 0x7f20bc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.810+0000 7f20ccff9640 1 -- 172.21.15.12:0/3858608356 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20c8084600 con 0x7f20d014fd50 2023-10-30T12:35:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.814+0000 7f20cffff640 1 --2- 172.21.15.12:0/3858608356 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20bc0664a0 0x7f20bc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.814+0000 7f20cffff640 1 --2- 172.21.15.12:0/3858608356 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20bc0664a0 0x7f20bc068960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f20c00af000 tx=0x7f20c00b6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.818+0000 7f20ccff9640 1 -- 172.21.15.12:0/3858608356 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f20d00c3aa0 con 0x7f20d014fd50 2023-10-30T12:35:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.974+0000 7f20d6279640 1 -- 172.21.15.12:0/3858608356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20d004e170 con 0x7f20d014fd50 2023-10-30T12:35:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.974+0000 7f20ccff9640 1 -- 172.21.15.12:0/3858608356 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f20c8051b30 con 0x7f20d014fd50 2023-10-30T12:35:37.978 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:37.979 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":22,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:35:07.153665+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:35:03.216053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:35:04.597162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:35:37.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.978+0000 7f20d6279640 1 -- 172.21.15.12:0/3858608356 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20bc0664a0 msgr2=0x7f20bc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.978+0000 7f20d6279640 1 --2- 172.21.15.12:0/3858608356 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20bc0664a0 0x7f20bc068960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f20c00af000 tx=0x7f20c00b6040 comp rx=0 tx=0).stop 2023-10-30T12:35:37.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.978+0000 7f20d6279640 1 -- 172.21.15.12:0/3858608356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d014fd50 msgr2=0x7f20d00c3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:37.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.978+0000 7f20d6279640 1 --2- 172.21.15.12:0/3858608356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d014fd50 0x7f20d00c3d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f20c8007e60 tx=0x7f20c800e4b0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.978+0000 7f20d6279640 1 -- 172.21.15.12:0/3858608356 shutdown_connections 2023-10-30T12:35:37.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.978+0000 7f20d6279640 1 --2- 172.21.15.12:0/3858608356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d014fd50 0x7f20d00c3d80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.978+0000 7f20d6279640 1 --2- 172.21.15.12:0/3858608356 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20bc0664a0 0x7f20bc068960 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.978+0000 7f20d6279640 1 --2- 172.21.15.12:0/3858608356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20d00cb550 0x7f20d0145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.978+0000 7f20d6279640 1 --2- 172.21.15.12:0/3858608356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20d0071960 0x7f20d01f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:37.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.978+0000 7f20d6279640 1 -- 172.21.15.12:0/3858608356 >> 172.21.15.12:0/3858608356 conn(0x7f20d006c6e0 msgr2=0x7f20d0069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:37.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.978+0000 7f20d6279640 1 -- 172.21.15.12:0/3858608356 shutdown_connections 2023-10-30T12:35:37.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:37.978+0000 7f20d6279640 1 -- 172.21.15.12:0/3858608356 wait complete. 2023-10-30T12:35:38.000 INFO:tasks.mds_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:34:53.017612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-30T12:35:03.216053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-30T12:35:04.597162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:35:38.000 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:35:38.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba22cea640 1 Processor -- start 2023-10-30T12:35:38.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba22cea640 1 -- start start 2023-10-30T12:35:38.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba22cea640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fba1c153980 0x7fba1c0caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba22cea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba1c0cb1e0 0x7fba1c15d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba22cea640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1c15a920 con 0x7fba1c159930 2023-10-30T12:35:38.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba22cea640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1c0cb930 con 0x7fba1c153980 2023-10-30T12:35:38.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba22cea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c159dc0 con 0x7fba1c0cb1e0 2023-10-30T12:35:38.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba20a86640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fba1c153980 0x7fba1c0caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba1bfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fba1c159930 0x7fba1c0cf1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54432/0 (socket says 172.21.15.12:54432) 2023-10-30T12:35:38.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba1bfff640 1 -- 172.21.15.12:0/3185585199 learned_addr learned my addr 172.21.15.12:0/3185585199 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:38.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba20a86640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fba1c153980 0x7fba1c0caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57338/0 (socket says 172.21.15.12:57338) 2023-10-30T12:35:38.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba21287640 1 --2- 172.21.15.12:0/3185585199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba1c0cb1e0 0x7fba1c15d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba20a86640 1 -- 172.21.15.12:0/3185585199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba1c0cb1e0 msgr2=0x7fba1c15d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:38.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba20a86640 1 --2- 172.21.15.12:0/3185585199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba1c0cb1e0 0x7fba1c15d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba20a86640 1 -- 172.21.15.12:0/3185585199 >> v1:172.21.15.12:6789/0 conn(0x7fba1c159930 legacy=0x7fba1c0cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba20a86640 1 -- 172.21.15.12:0/3185585199 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c15a080 con 0x7fba1c153980 2023-10-30T12:35:38.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba21287640 1 --2- 172.21.15.12:0/3185585199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba1c0cb1e0 0x7fba1c15d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:38.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.142+0000 7fba1b7fe640 1 -- 172.21.15.12:0/3185585199 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108802960 0 0) 0x7fba1c15a920 con 0x7fba1c159930 2023-10-30T12:35:38.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.146+0000 7fba20a86640 1 --2- 172.21.15.12:0/3185585199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1c153980 0x7fba1c0caca0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fba100097c0 tx=0x7fba1001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1d3321679da8e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:38.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.146+0000 7fba1b7fe640 1 -- 172.21.15.12:0/3185585199 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba100ad660 con 0x7fba1c153980 2023-10-30T12:35:38.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.146+0000 7fba1b7fe640 1 -- 172.21.15.12:0/3185585199 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba100ad7c0 con 0x7fba1c153980 2023-10-30T12:35:38.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.146+0000 7fba1b7fe640 1 -- 172.21.15.12:0/3185585199 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba100adb50 con 0x7fba1c153980 2023-10-30T12:35:38.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.146+0000 7fba22cea640 1 -- 172.21.15.12:0/3185585199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1c153980 msgr2=0x7fba1c0caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.146+0000 7fba22cea640 1 --2- 172.21.15.12:0/3185585199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1c153980 0x7fba1c0caca0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fba100097c0 tx=0x7fba1001c450 comp rx=0 tx=0).stop 2023-10-30T12:35:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.146+0000 7fba22cea640 1 -- 172.21.15.12:0/3185585199 shutdown_connections 2023-10-30T12:35:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.146+0000 7fba22cea640 1 --2- 172.21.15.12:0/3185585199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba1c0cb1e0 0x7fba1c15d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.146+0000 7fba22cea640 1 --2- 172.21.15.12:0/3185585199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1c153980 0x7fba1c0caca0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.146+0000 7fba22cea640 1 -- 172.21.15.12:0/3185585199 >> 172.21.15.12:0/3185585199 conn(0x7fba1c06c0f0 msgr2=0x7fba1c06c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.146+0000 7fba22cea640 1 -- 172.21.15.12:0/3185585199 shutdown_connections 2023-10-30T12:35:38.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.146+0000 7fba22cea640 1 -- 172.21.15.12:0/3185585199 wait complete. 2023-10-30T12:35:38.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba22cea640 1 Processor -- start 2023-10-30T12:35:38.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba22cea640 1 -- start start 2023-10-30T12:35:38.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba22cea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1c0cb1e0 0x7fba1c0c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba22cea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba1c153980 0x7fba1c0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba22cea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba1c0c6740 0x7fba1c15b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba22cea640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0c7bc0 con 0x7fba1c0cb1e0 2023-10-30T12:35:38.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba22cea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c15aaf0 con 0x7fba1c0c6740 2023-10-30T12:35:38.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba22cea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0c7e80 con 0x7fba1c153980 2023-10-30T12:35:38.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba20a86640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1c0cb1e0 0x7fba1c0c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba20a86640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1c0cb1e0 0x7fba1c0c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57352/0 (socket says 172.21.15.12:57352) 2023-10-30T12:35:38.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba20a86640 1 -- 172.21.15.12:0/2192187332 learned_addr learned my addr 172.21.15.12:0/2192187332 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:38.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba1bfff640 1 --2- 172.21.15.12:0/2192187332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba1c153980 0x7fba1c0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba21287640 1 --2- 172.21.15.12:0/2192187332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba1c0c6740 0x7fba1c15b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba20a86640 1 -- 172.21.15.12:0/2192187332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba1c153980 msgr2=0x7fba1c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba20a86640 1 --2- 172.21.15.12:0/2192187332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba1c153980 0x7fba1c0c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba20a86640 1 -- 172.21.15.12:0/2192187332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba1c0c6740 msgr2=0x7fba1c15b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba20a86640 1 --2- 172.21.15.12:0/2192187332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba1c0c6740 0x7fba1c15b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba20a86640 1 -- 172.21.15.12:0/2192187332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba10009420 con 0x7fba1c0cb1e0 2023-10-30T12:35:38.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba20a86640 1 --2- 172.21.15.12:0/2192187332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1c0cb1e0 0x7fba1c0c88e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fba100b0040 tx=0x7fba1001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:38.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba197fa640 1 -- 172.21.15.12:0/2192187332 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba100be070 con 0x7fba1c0cb1e0 2023-10-30T12:35:38.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba197fa640 1 -- 172.21.15.12:0/2192187332 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba100b9450 con 0x7fba1c0cb1e0 2023-10-30T12:35:38.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba22cea640 1 -- 172.21.15.12:0/2192187332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1c15adb0 con 0x7fba1c0cb1e0 2023-10-30T12:35:38.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.150+0000 7fba197fa640 1 -- 172.21.15.12:0/2192187332 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba100b9820 con 0x7fba1c0cb1e0 2023-10-30T12:35:38.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.154+0000 7fba22cea640 1 -- 172.21.15.12:0/2192187332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c15b370 con 0x7fba1c0cb1e0 2023-10-30T12:35:38.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.154+0000 7fba22cea640 1 -- 172.21.15.12:0/2192187332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1c04e170 con 0x7fba1c0cb1e0 2023-10-30T12:35:38.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.154+0000 7fba197fa640 1 -- 172.21.15.12:0/2192187332 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fba100b9980 con 0x7fba1c0cb1e0 2023-10-30T12:35:38.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.154+0000 7fba197fa640 1 --2- 172.21.15.12:0/2192187332 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba000664a0 0x7fba00068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.154+0000 7fba197fa640 1 -- 172.21.15.12:0/2192187332 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba1011e0e0 con 0x7fba1c0cb1e0 2023-10-30T12:35:38.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.154+0000 7fba1bfff640 1 --2- 172.21.15.12:0/2192187332 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba000664a0 0x7fba00068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.154+0000 7fba1bfff640 1 --2- 172.21.15.12:0/2192187332 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba000664a0 0x7fba00068960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0d0060 tx=0x7fba08007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:38.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.158+0000 7fba197fa640 1 -- 172.21.15.12:0/2192187332 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fba100eb550 con 0x7fba1c0cb1e0 2023-10-30T12:35:38.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.318+0000 7fba22cea640 1 -- 172.21.15.12:0/2192187332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba1c0caf50 con 0x7fba1c0cb1e0 2023-10-30T12:35:38.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.318+0000 7fba197fa640 1 -- 172.21.15.12:0/2192187332 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba100eeb70 con 0x7fba1c0cb1e0 2023-10-30T12:35:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-10-30T12:35:38.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.318+0000 7fba22cea640 1 -- 172.21.15.12:0/2192187332 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba000664a0 msgr2=0x7fba00068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.318+0000 7fba22cea640 1 --2- 172.21.15.12:0/2192187332 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba000664a0 0x7fba00068960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0d0060 tx=0x7fba08007c10 comp rx=0 tx=0).stop 2023-10-30T12:35:38.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.318+0000 7fba22cea640 1 -- 172.21.15.12:0/2192187332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1c0cb1e0 msgr2=0x7fba1c0c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.318+0000 7fba22cea640 1 --2- 172.21.15.12:0/2192187332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1c0cb1e0 0x7fba1c0c88e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fba100b0040 tx=0x7fba1001ac90 comp rx=0 tx=0).stop 2023-10-30T12:35:38.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.318+0000 7fba22cea640 1 -- 172.21.15.12:0/2192187332 shutdown_connections 2023-10-30T12:35:38.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.318+0000 7fba22cea640 1 --2- 172.21.15.12:0/2192187332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba1c0c6740 0x7fba1c15b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.318+0000 7fba22cea640 1 --2- 172.21.15.12:0/2192187332 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba000664a0 0x7fba00068960 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.318+0000 7fba22cea640 1 --2- 172.21.15.12:0/2192187332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba1c153980 0x7fba1c0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.318+0000 7fba22cea640 1 --2- 172.21.15.12:0/2192187332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1c0cb1e0 0x7fba1c0c88e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.318+0000 7fba22cea640 1 -- 172.21.15.12:0/2192187332 >> 172.21.15.12:0/2192187332 conn(0x7fba1c06c0f0 msgr2=0x7fba1c157a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:38.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.318+0000 7fba22cea640 1 -- 172.21.15.12:0/2192187332 shutdown_connections 2023-10-30T12:35:38.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.318+0000 7fba22cea640 1 -- 172.21.15.12:0/2192187332 wait complete. 2023-10-30T12:35:38.343 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-30T12:35:38.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1cef217640 1 Processor -- start 2023-10-30T12:35:38.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1cef217640 1 -- start start 2023-10-30T12:35:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1cef217640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1ce8153980 0x7f1ce80caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1cef217640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ce8159930 0x7f1ce80cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1cef217640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce815a920 con 0x7f1ce80cf1a0 2023-10-30T12:35:38.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1cef217640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80cb930 con 0x7f1ce8153980 2023-10-30T12:35:38.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1cef217640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce8159dc0 con 0x7f1ce8159930 2023-10-30T12:35:38.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ced7b4640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f1ce80cf1a0 0x7f1ce815d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54444/0 (socket says 172.21.15.12:54444) 2023-10-30T12:35:38.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ced7b4640 1 -- 172.21.15.12:0/3146942105 learned_addr learned my addr 172.21.15.12:0/3146942105 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1cecfb3640 1 --2- 172.21.15.12:0/3146942105 >> v2:172.21.15.154:3300/0 conn(0x7f1ce8153980 0x7f1ce80caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ce7fff640 1 --2- 172.21.15.12:0/3146942105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ce8159930 0x7f1ce80cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ce77fe640 1 -- 172.21.15.12:0/3146942105 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358622985 0 0) 0x7f1ce815a920 con 0x7f1ce80cf1a0 2023-10-30T12:35:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ce77fe640 1 -- 172.21.15.12:0/3146942105 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cd8003530 con 0x7f1ce80cf1a0 2023-10-30T12:35:38.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ce77fe640 1 -- 172.21.15.12:0/3146942105 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2403689462 0 0) 0x7f1cd8003530 con 0x7f1ce80cf1a0 2023-10-30T12:35:38.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ce77fe640 1 -- 172.21.15.12:0/3146942105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ce8159930 msgr2=0x7f1ce80cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:38.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ce77fe640 1 --2- 172.21.15.12:0/3146942105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ce8159930 0x7f1ce80cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ce77fe640 1 -- 172.21.15.12:0/3146942105 >> v2:172.21.15.154:3300/0 conn(0x7f1ce8153980 msgr2=0x7f1ce80caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:38.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ce77fe640 1 --2- 172.21.15.12:0/3146942105 >> v2:172.21.15.154:3300/0 conn(0x7f1ce8153980 0x7f1ce80caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ce77fe640 1 -- 172.21.15.12:0/3146942105 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce815a080 con 0x7f1ce80cf1a0 2023-10-30T12:35:38.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ce77fe640 1 -- 172.21.15.12:0/3146942105 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f1ce0001a00 con 0x7f1ce80cf1a0 2023-10-30T12:35:38.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ce7fff640 1 --2- 172.21.15.12:0/3146942105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ce8159930 0x7f1ce80cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:35:38.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ce77fe640 1 -- 172.21.15.12:0/3146942105 >> v1:172.21.15.12:6789/0 conn(0x7f1ce80cf1a0 legacy=0x7f1ce815d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ce77fe640 1 --2- 172.21.15.12:0/3146942105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd8003980 0x7f1cd8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.486+0000 7f1ce77fe640 1 --2- 172.21.15.12:0/3146942105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cd80047d0 0x7f1cd8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1ce77fe640 1 --2- 172.21.15.12:0/3146942105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1cd8005190 0x7f1cd800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1ce77fe640 1 -- 172.21.15.12:0/3146942105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce80cb930 con 0x7f1cd80047d0 2023-10-30T12:35:38.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1ce77fe640 1 -- 172.21.15.12:0/3146942105 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce8159dc0 con 0x7f1cd8003980 2023-10-30T12:35:38.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1ce77fe640 1 -- 172.21.15.12:0/3146942105 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd8004520 con 0x7f1cd8005190 2023-10-30T12:35:38.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1ce7fff640 1 --2- 172.21.15.12:0/3146942105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cd80047d0 0x7f1cd8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1cecfb3640 1 --2- 172.21.15.12:0/3146942105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd8003980 0x7f1cd8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1ced7b4640 1 --2- 172.21.15.12:0/3146942105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1cd8005190 0x7f1cd800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1ce7fff640 1 -- 172.21.15.12:0/3146942105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1cd8005190 msgr2=0x7f1cd800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1ce7fff640 1 --2- 172.21.15.12:0/3146942105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1cd8005190 0x7f1cd800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1ce7fff640 1 -- 172.21.15.12:0/3146942105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd8003980 msgr2=0x7f1cd8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:38.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1ce7fff640 1 --2- 172.21.15.12:0/3146942105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd8003980 0x7f1cd8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1ce7fff640 1 -- 172.21.15.12:0/3146942105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd4009420 con 0x7f1cd80047d0 2023-10-30T12:35:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1ce7fff640 1 --2- 172.21.15.12:0/3146942105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cd80047d0 0x7f1cd8004c50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4009cb0 tx=0x7f1cd401cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a125c242e8de84 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1ce77fe640 1 -- 172.21.15.12:0/3146942105 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40a57a0 con 0x7f1cd80047d0 2023-10-30T12:35:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1ce77fe640 1 -- 172.21.15.12:0/3146942105 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd40a5900 con 0x7f1cd80047d0 2023-10-30T12:35:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.490+0000 7f1ce77fe640 1 -- 172.21.15.12:0/3146942105 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40a5cb0 con 0x7f1cd80047d0 2023-10-30T12:35:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1cef217640 1 -- 172.21.15.12:0/3146942105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cd80047d0 msgr2=0x7f1cd8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1cef217640 1 --2- 172.21.15.12:0/3146942105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cd80047d0 0x7f1cd8004c50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4009cb0 tx=0x7f1cd401cbf0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1ced7b4640 1 -- 172.21.15.12:0/3146942105 reap_dead start 2023-10-30T12:35:38.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1cef217640 1 -- 172.21.15.12:0/3146942105 shutdown_connections 2023-10-30T12:35:38.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1cef217640 1 -- 172.21.15.12:0/3146942105 >> 172.21.15.12:0/3146942105 conn(0x7f1ce806c0f0 msgr2=0x7f1ce806c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:38.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1cef217640 1 -- 172.21.15.12:0/3146942105 shutdown_connections 2023-10-30T12:35:38.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1cef217640 1 -- 172.21.15.12:0/3146942105 wait complete. 2023-10-30T12:35:38.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1cef217640 1 Processor -- start 2023-10-30T12:35:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1cef217640 1 -- start start 2023-10-30T12:35:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1cef217640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd80047d0 0x7f1ce815b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1cef217640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ce81eea90 0x7f1ce81f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1cef217640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce814dda0 0x7f1ce815c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1cef217640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce815ad80 con 0x7f1ce814dda0 2023-10-30T12:35:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1cef217640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce81eeef0 con 0x7f1cd80047d0 2023-10-30T12:35:38.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1cef217640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce81f2000 con 0x7f1ce81eea90 2023-10-30T12:35:38.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1ced7b4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce814dda0 0x7f1ce815c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1ced7b4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce814dda0 0x7f1ce815c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57378/0 (socket says 172.21.15.12:57378) 2023-10-30T12:35:38.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1ced7b4640 1 -- 172.21.15.12:0/1431148483 learned_addr learned my addr 172.21.15.12:0/1431148483 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:38.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1cecfb3640 1 --2- 172.21.15.12:0/1431148483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd80047d0 0x7f1ce815b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1ced7b4640 1 -- 172.21.15.12:0/1431148483 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ce81eea90 msgr2=0x7f1ce81f2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1ced7b4640 1 --2- 172.21.15.12:0/1431148483 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ce81eea90 0x7f1ce81f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1ced7b4640 1 -- 172.21.15.12:0/1431148483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd80047d0 msgr2=0x7f1ce815b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1ced7b4640 1 --2- 172.21.15.12:0/1431148483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd80047d0 0x7f1ce815b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.494+0000 7f1ced7b4640 1 -- 172.21.15.12:0/1431148483 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce815ba00 con 0x7f1ce814dda0 2023-10-30T12:35:38.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.498+0000 7f1ced7b4640 1 --2- 172.21.15.12:0/1431148483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce814dda0 0x7f1ce815c590 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0002050 tx=0x7f1ce0011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:38.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.498+0000 7f1ce57fa640 1 -- 172.21.15.12:0/1431148483 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce000f840 con 0x7f1ce814dda0 2023-10-30T12:35:38.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.498+0000 7f1ce57fa640 1 -- 172.21.15.12:0/1431148483 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce000f9a0 con 0x7f1ce814dda0 2023-10-30T12:35:38.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.498+0000 7f1ce57fa640 1 -- 172.21.15.12:0/1431148483 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce000fd50 con 0x7f1ce814dda0 2023-10-30T12:35:38.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.498+0000 7f1cef217640 1 -- 172.21.15.12:0/1431148483 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce815bce0 con 0x7f1ce814dda0 2023-10-30T12:35:38.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.498+0000 7f1cef217640 1 -- 172.21.15.12:0/1431148483 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce81f2250 con 0x7f1ce814dda0 2023-10-30T12:35:38.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.498+0000 7f1cef217640 1 -- 172.21.15.12:0/1431148483 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce804e170 con 0x7f1ce814dda0 2023-10-30T12:35:38.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.498+0000 7f1ce57fa640 1 -- 172.21.15.12:0/1431148483 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1ce001f460 con 0x7f1ce814dda0 2023-10-30T12:35:38.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.498+0000 7f1ce57fa640 1 --2- 172.21.15.12:0/1431148483 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1ccc0664d0 0x7f1ccc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.498+0000 7f1ce57fa640 1 -- 172.21.15.12:0/1431148483 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ce0086f00 con 0x7f1ce814dda0 2023-10-30T12:35:38.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.498+0000 7f1cecfb3640 1 --2- 172.21.15.12:0/1431148483 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1ccc0664d0 0x7f1ccc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.502+0000 7f1cecfb3640 1 --2- 172.21.15.12:0/1431148483 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1ccc0664d0 0x7f1ccc068990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009ba0 tx=0x7f1cdc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:38.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.502+0000 7f1ce57fa640 1 -- 172.21.15.12:0/1431148483 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1ce00542e0 con 0x7f1ce814dda0 2023-10-30T12:35:38.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.658+0000 7f1cef217640 1 -- 172.21.15.12:0/1431148483 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ce8153d60 con 0x7f1ce814dda0 2023-10-30T12:35:38.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.658+0000 7f1ce57fa640 1 -- 172.21.15.12:0/1431148483 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1ce0057900 con 0x7f1ce814dda0 2023-10-30T12:35:38.664 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-10-30T12:35:38.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.662+0000 7f1cef217640 1 -- 172.21.15.12:0/1431148483 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1ccc0664d0 msgr2=0x7f1ccc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.662+0000 7f1cef217640 1 --2- 172.21.15.12:0/1431148483 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1ccc0664d0 0x7f1ccc068990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009ba0 tx=0x7f1cdc008040 comp rx=0 tx=0).stop 2023-10-30T12:35:38.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.662+0000 7f1cef217640 1 -- 172.21.15.12:0/1431148483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce814dda0 msgr2=0x7f1ce815c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.662+0000 7f1cef217640 1 --2- 172.21.15.12:0/1431148483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce814dda0 0x7f1ce815c590 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0002050 tx=0x7f1ce0011c20 comp rx=0 tx=0).stop 2023-10-30T12:35:38.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.662+0000 7f1cef217640 1 -- 172.21.15.12:0/1431148483 shutdown_connections 2023-10-30T12:35:38.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.662+0000 7f1cef217640 1 --2- 172.21.15.12:0/1431148483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce814dda0 0x7f1ce815c590 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.662+0000 7f1cef217640 1 --2- 172.21.15.12:0/1431148483 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1ccc0664d0 0x7f1ccc068990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.662+0000 7f1cef217640 1 --2- 172.21.15.12:0/1431148483 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ce81eea90 0x7f1ce81f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.662+0000 7f1cef217640 1 --2- 172.21.15.12:0/1431148483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd80047d0 0x7f1ce815b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.662+0000 7f1cef217640 1 -- 172.21.15.12:0/1431148483 >> 172.21.15.12:0/1431148483 conn(0x7f1ce806c0f0 msgr2=0x7f1ce8150500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:38.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.662+0000 7f1cef217640 1 -- 172.21.15.12:0/1431148483 shutdown_connections 2023-10-30T12:35:38.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.662+0000 7f1cef217640 1 -- 172.21.15.12:0/1431148483 wait complete. 2023-10-30T12:35:38.684 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-30T12:35:38.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.826+0000 7f7ccb59e640 1 Processor -- start 2023-10-30T12:35:38.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.826+0000 7f7ccb59e640 1 -- start start 2023-10-30T12:35:38.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7ccb59e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ccc153c20 0x7f7ccc15a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7ccb59e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ccc15a6b0 con 0x7f7ccc153840 2023-10-30T12:35:38.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7ccb59e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ccc15d730 con 0x7f7ccc0cb4c0 2023-10-30T12:35:38.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7ccb59e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc15dbc0 con 0x7f7ccc153c20 2023-10-30T12:35:38.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cca59c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f7ccc153840 0x7f7ccc0c3eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54460/0 (socket says 172.21.15.12:54460) 2023-10-30T12:35:38.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cca59c640 1 -- 172.21.15.12:0/894210087 learned_addr learned my addr 172.21.15.12:0/894210087 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:38.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc3fff640 1 --2- 172.21.15.12:0/894210087 >> v2:172.21.15.154:3300/0 conn(0x7f7ccc153c20 0x7f7ccc15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc9d9b640 1 -- 172.21.15.12:0/894210087 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735430452 0 0) 0x7f7cbc001120 con 0x7f7ccc153840 2023-10-30T12:35:38.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc9d9b640 1 -- 172.21.15.12:0/894210087 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cb8003530 con 0x7f7ccc153840 2023-10-30T12:35:38.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc9d9b640 1 -- 172.21.15.12:0/894210087 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1687674250 0 0) 0x7f7cb8003530 con 0x7f7ccc153840 2023-10-30T12:35:38.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc9d9b640 1 -- 172.21.15.12:0/894210087 >> v2:172.21.15.154:3300/0 conn(0x7f7ccc153c20 msgr2=0x7f7ccc15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:38.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc9d9b640 1 --2- 172.21.15.12:0/894210087 >> v2:172.21.15.154:3300/0 conn(0x7f7ccc153c20 0x7f7ccc15a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc9d9b640 1 -- 172.21.15.12:0/894210087 >> v1:172.21.15.154:6789/0 conn(0x7f7ccc0cb4c0 legacy=0x7f7ccc0cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc9d9b640 1 -- 172.21.15.12:0/894210087 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc0ca8f0 con 0x7f7ccc153840 2023-10-30T12:35:38.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc9d9b640 1 -- 172.21.15.12:0/894210087 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f7cbc001810 con 0x7f7ccc153840 2023-10-30T12:35:38.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc9d9b640 1 -- 172.21.15.12:0/894210087 >> v1:172.21.15.12:6789/0 conn(0x7f7ccc153840 legacy=0x7f7ccc0c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc9d9b640 1 --2- 172.21.15.12:0/894210087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cb8003980 0x7f7cb8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc9d9b640 1 --2- 172.21.15.12:0/894210087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cb80047d0 0x7f7cb8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc9d9b640 1 --2- 172.21.15.12:0/894210087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cb8005190 0x7f7cb800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc9d9b640 1 -- 172.21.15.12:0/894210087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc15dbc0 con 0x7f7cb8003980 2023-10-30T12:35:38.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc9d9b640 1 -- 172.21.15.12:0/894210087 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb800c040 con 0x7f7cb80047d0 2023-10-30T12:35:38.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc9d9b640 1 -- 172.21.15.12:0/894210087 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb800c200 con 0x7f7cb8005190 2023-10-30T12:35:38.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cca59c640 1 --2- 172.21.15.12:0/894210087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cb8003980 0x7f7cb8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc3fff640 1 --2- 172.21.15.12:0/894210087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cb80047d0 0x7f7cb8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7ccad9d640 1 --2- 172.21.15.12:0/894210087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cb8005190 0x7f7cb800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cca59c640 1 -- 172.21.15.12:0/894210087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cb8005190 msgr2=0x7f7cb800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:38.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cca59c640 1 --2- 172.21.15.12:0/894210087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cb8005190 0x7f7cb800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cca59c640 1 -- 172.21.15.12:0/894210087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cb80047d0 msgr2=0x7f7cb8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:38.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cca59c640 1 --2- 172.21.15.12:0/894210087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cb80047d0 0x7f7cb8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cca59c640 1 -- 172.21.15.12:0/894210087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb8003530 con 0x7f7cb8003980 2023-10-30T12:35:38.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7cc3fff640 1 --2- 172.21.15.12:0/894210087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cb80047d0 0x7f7cb8004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:38.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.830+0000 7f7ccad9d640 1 -- 172.21.15.12:0/894210087 reap_dead start 2023-10-30T12:35:38.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.834+0000 7f7cca59c640 1 --2- 172.21.15.12:0/894210087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cb8003980 0x7f7cb8003da0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00c240 tx=0x7f7cbc01f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2eb147c839753a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:38.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.834+0000 7f7cc9d9b640 1 -- 172.21.15.12:0/894210087 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc005ae0 con 0x7f7cb8003980 2023-10-30T12:35:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.834+0000 7f7cc9d9b640 1 -- 172.21.15.12:0/894210087 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cbc0ab070 con 0x7f7cb8003980 2023-10-30T12:35:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.834+0000 7f7cc9d9b640 1 -- 172.21.15.12:0/894210087 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc007070 con 0x7f7cb8003980 2023-10-30T12:35:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.834+0000 7f7ccb59e640 1 -- 172.21.15.12:0/894210087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cb8003980 msgr2=0x7f7cb8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.834+0000 7f7ccb59e640 1 --2- 172.21.15.12:0/894210087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cb8003980 0x7f7cb8003da0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00c240 tx=0x7f7cbc01f3b0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.834+0000 7f7ccb59e640 1 -- 172.21.15.12:0/894210087 shutdown_connections 2023-10-30T12:35:38.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.834+0000 7f7ccb59e640 1 --2- 172.21.15.12:0/894210087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cb8003980 0x7f7cb8003da0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.834+0000 7f7ccb59e640 1 -- 172.21.15.12:0/894210087 >> 172.21.15.12:0/894210087 conn(0x7f7ccc06be40 msgr2=0x7f7ccc06c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:38.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.834+0000 7f7ccb59e640 1 -- 172.21.15.12:0/894210087 shutdown_connections 2023-10-30T12:35:38.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.834+0000 7f7ccb59e640 1 -- 172.21.15.12:0/894210087 wait complete. 2023-10-30T12:35:38.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7ccb59e640 1 Processor -- start 2023-10-30T12:35:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7ccb59e640 1 -- start start 2023-10-30T12:35:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7ccb59e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cb8003980 0x7f7ccc0ce690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7ccb59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ccc1589b0 0x7f7ccc14ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7ccb59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ccc14dd80 0x7f7ccc0cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7ccb59e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0ce1a0 con 0x7f7ccc1589b0 2023-10-30T12:35:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7ccb59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc157600 con 0x7f7ccc14dd80 2023-10-30T12:35:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7ccb59e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0ce3b0 con 0x7f7cb8003980 2023-10-30T12:35:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7cc3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ccc1589b0 0x7f7ccc14ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7cc3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ccc1589b0 0x7f7ccc14ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57398/0 (socket says 172.21.15.12:57398) 2023-10-30T12:35:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7cc3fff640 1 -- 172.21.15.12:0/208812289 learned_addr learned my addr 172.21.15.12:0/208812289 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:38.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7ccad9d640 1 --2- 172.21.15.12:0/208812289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ccc14dd80 0x7f7ccc0cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7cca59c640 1 --2- 172.21.15.12:0/208812289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cb8003980 0x7f7ccc0ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7cc3fff640 1 -- 172.21.15.12:0/208812289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cb8003980 msgr2=0x7f7ccc0ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7cc3fff640 1 --2- 172.21.15.12:0/208812289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cb8003980 0x7f7ccc0ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7cc3fff640 1 -- 172.21.15.12:0/208812289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ccc14dd80 msgr2=0x7f7ccc0cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:38.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7cc3fff640 1 --2- 172.21.15.12:0/208812289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ccc14dd80 0x7f7ccc0cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:38.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7cc3fff640 1 -- 172.21.15.12:0/208812289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc1578a0 con 0x7f7ccc1589b0 2023-10-30T12:35:38.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7cc3fff640 1 --2- 172.21.15.12:0/208812289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ccc1589b0 0x7f7ccc14ff00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4011530 tx=0x7f7cb40119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:38.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7cc2ffd640 1 -- 172.21.15.12:0/208812289 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4004220 con 0x7f7ccc1589b0 2023-10-30T12:35:38.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7cc2ffd640 1 -- 172.21.15.12:0/208812289 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb4004380 con 0x7f7ccc1589b0 2023-10-30T12:35:38.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7cc2ffd640 1 -- 172.21.15.12:0/208812289 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4015870 con 0x7f7ccc1589b0 2023-10-30T12:35:38.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7ccb59e640 1 -- 172.21.15.12:0/208812289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc157b80 con 0x7f7ccc1589b0 2023-10-30T12:35:38.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7ccb59e640 1 -- 172.21.15.12:0/208812289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc15a850 con 0x7f7ccc1589b0 2023-10-30T12:35:38.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.838+0000 7f7ccb59e640 1 -- 172.21.15.12:0/208812289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc0ccfd0 con 0x7f7ccc1589b0 2023-10-30T12:35:38.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.842+0000 7f7cc2ffd640 1 -- 172.21.15.12:0/208812289 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7cb40159d0 con 0x7f7ccc1589b0 2023-10-30T12:35:38.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.842+0000 7f7cc2ffd640 1 --2- 172.21.15.12:0/208812289 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cac0664a0 0x7f7cac068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:38.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.842+0000 7f7cc2ffd640 1 -- 172.21.15.12:0/208812289 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cb4084300 con 0x7f7ccc1589b0 2023-10-30T12:35:38.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.842+0000 7f7cca59c640 1 --2- 172.21.15.12:0/208812289 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cac0664a0 0x7f7cac068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:38.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.842+0000 7f7cca59c640 1 --2- 172.21.15.12:0/208812289 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cac0664a0 0x7f7cac068960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0af000 tx=0x7f7cbc0aa000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:38.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:38.846+0000 7f7cc2ffd640 1 -- 172.21.15.12:0/208812289 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7ccc0ccfd0 con 0x7f7ccc1589b0 2023-10-30T12:35:39.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.002+0000 7f7ccb59e640 1 -- 172.21.15.12:0/208812289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7ccc0ccfd0 con 0x7f7ccc1589b0 2023-10-30T12:35:39.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.002+0000 7f7cc2ffd640 1 -- 172.21.15.12:0/208812289 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7ccc0ccfd0 con 0x7f7ccc1589b0 2023-10-30T12:35:39.005 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-10-30T12:35:39.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.002+0000 7f7ccb59e640 1 -- 172.21.15.12:0/208812289 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cac0664a0 msgr2=0x7f7cac068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.002+0000 7f7ccb59e640 1 --2- 172.21.15.12:0/208812289 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cac0664a0 0x7f7cac068960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0af000 tx=0x7f7cbc0aa000 comp rx=0 tx=0).stop 2023-10-30T12:35:39.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.002+0000 7f7ccb59e640 1 -- 172.21.15.12:0/208812289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ccc1589b0 msgr2=0x7f7ccc14ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.002+0000 7f7ccb59e640 1 --2- 172.21.15.12:0/208812289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ccc1589b0 0x7f7ccc14ff00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4011530 tx=0x7f7cb40119f0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.002+0000 7f7ccb59e640 1 -- 172.21.15.12:0/208812289 shutdown_connections 2023-10-30T12:35:39.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.002+0000 7f7ccb59e640 1 --2- 172.21.15.12:0/208812289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ccc14dd80 0x7f7ccc0cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.002+0000 7f7ccb59e640 1 --2- 172.21.15.12:0/208812289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ccc1589b0 0x7f7ccc14ff00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.002+0000 7f7ccb59e640 1 --2- 172.21.15.12:0/208812289 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7cac0664a0 0x7f7cac068960 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.002+0000 7f7ccb59e640 1 --2- 172.21.15.12:0/208812289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cb8003980 0x7f7ccc0ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.002+0000 7f7ccb59e640 1 -- 172.21.15.12:0/208812289 >> 172.21.15.12:0/208812289 conn(0x7f7ccc06be40 msgr2=0x7f7ccc14e850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:39.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.002+0000 7f7ccb59e640 1 -- 172.21.15.12:0/208812289 shutdown_connections 2023-10-30T12:35:39.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.002+0000 7f7ccb59e640 1 -- 172.21.15.12:0/208812289 wait complete. 2023-10-30T12:35:39.027 INFO:tasks.mds_thrash:Wait for all MDSs to reach steady state... 2023-10-30T12:35:39.027 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:35:39.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.166+0000 7f4390e68640 1 Processor -- start 2023-10-30T12:35:39.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.166+0000 7f4390e68640 1 -- start start 2023-10-30T12:35:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.166+0000 7f4390e68640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438c0cf160 0x7f438c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.166+0000 7f4390e68640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438c0cb800 con 0x7f438c1538a0 2023-10-30T12:35:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.166+0000 7f4390e68640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438c0cb330 con 0x7f438c15d6f0 2023-10-30T12:35:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.166+0000 7f4390e68640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438c0cad70 con 0x7f438c0cf160 2023-10-30T12:35:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.166+0000 7f438a59c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f438c1538a0 0x7f438c0ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54474/0 (socket says 172.21.15.12:54474) 2023-10-30T12:35:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.166+0000 7f438a59c640 1 -- 172.21.15.12:0/2504232330 learned_addr learned my addr 172.21.15.12:0/2504232330 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:39.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438ad9d640 1 --2- 172.21.15.12:0/2504232330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438c0cf160 0x7f438c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846915857 0 0) 0x7f438c0cb800 con 0x7f438c1538a0 2023-10-30T12:35:39.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4374003530 con 0x7f438c1538a0 2023-10-30T12:35:39.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134899603 0 0) 0x7f438c0cb330 con 0x7f438c15d6f0 2023-10-30T12:35:39.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f438c0cb800 con 0x7f438c15d6f0 2023-10-30T12:35:39.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3474679366 0 0) 0x7f4374003530 con 0x7f438c1538a0 2023-10-30T12:35:39.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438c0cf160 msgr2=0x7f438c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:39.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 --2- 172.21.15.12:0/2504232330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438c0cf160 0x7f438c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 >> v1:172.21.15.154:6789/0 conn(0x7f438c15d6f0 legacy=0x7f438c15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438c0cb050 con 0x7f438c1538a0 2023-10-30T12:35:39.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f437c001680 con 0x7f438c1538a0 2023-10-30T12:35:39.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 >> v1:172.21.15.12:6789/0 conn(0x7f438c1538a0 legacy=0x7f438c0ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 --2- 172.21.15.12:0/2504232330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4374003b40 0x7f4374003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 --2- 172.21.15.12:0/2504232330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43740047d0 0x7f4374004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 --2- 172.21.15.12:0/2504232330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438c0cf160 0x7f4374005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c0cad70 con 0x7f43740047d0 2023-10-30T12:35:39.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437400ba40 con 0x7f4374003b40 2023-10-30T12:35:39.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437400bbd0 con 0x7f438c0cf160 2023-10-30T12:35:39.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f4389d9b640 1 --2- 172.21.15.12:0/2504232330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43740047d0 0x7f4374004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438a59c640 1 --2- 172.21.15.12:0/2504232330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4374003b40 0x7f4374003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f438ad9d640 1 --2- 172.21.15.12:0/2504232330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438c0cf160 0x7f4374005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f4389d9b640 1 -- 172.21.15.12:0/2504232330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438c0cf160 msgr2=0x7f4374005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f4389d9b640 1 --2- 172.21.15.12:0/2504232330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438c0cf160 0x7f4374005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f4389d9b640 1 -- 172.21.15.12:0/2504232330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4374003b40 msgr2=0x7f4374003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f4389d9b640 1 --2- 172.21.15.12:0/2504232330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4374003b40 0x7f4374003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.170+0000 7f4389d9b640 1 -- 172.21.15.12:0/2504232330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4374003530 con 0x7f43740047d0 2023-10-30T12:35:39.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.174+0000 7f4389d9b640 1 --2- 172.21.15.12:0/2504232330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43740047d0 0x7f4374004c50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f43800018d0 tx=0x7f438001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fd762058ba33e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:39.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.174+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437c007ea0 con 0x7f43740047d0 2023-10-30T12:35:39.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.174+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43800a7050 con 0x7f43740047d0 2023-10-30T12:35:39.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.174+0000 7f438959a640 1 -- 172.21.15.12:0/2504232330 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4380003600 con 0x7f43740047d0 2023-10-30T12:35:39.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.174+0000 7f4390e68640 1 -- 172.21.15.12:0/2504232330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43740047d0 msgr2=0x7f4374004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.174+0000 7f4390e68640 1 --2- 172.21.15.12:0/2504232330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43740047d0 0x7f4374004c50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f43800018d0 tx=0x7f438001c500 comp rx=0 tx=0).stop 2023-10-30T12:35:39.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.174+0000 7f438ad9d640 1 -- 172.21.15.12:0/2504232330 reap_dead start 2023-10-30T12:35:39.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.174+0000 7f4390e68640 1 -- 172.21.15.12:0/2504232330 shutdown_connections 2023-10-30T12:35:39.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.174+0000 7f4390e68640 1 -- 172.21.15.12:0/2504232330 >> 172.21.15.12:0/2504232330 conn(0x7f438c06bfe0 msgr2=0x7f438c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:39.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.174+0000 7f4390e68640 1 -- 172.21.15.12:0/2504232330 shutdown_connections 2023-10-30T12:35:39.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.174+0000 7f4390e68640 1 -- 172.21.15.12:0/2504232330 wait complete. 2023-10-30T12:35:39.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.174+0000 7f4390e68640 1 Processor -- start 2023-10-30T12:35:39.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.174+0000 7f4390e68640 1 -- start start 2023-10-30T12:35:39.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f4390e68640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43740047d0 0x7f438c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f4390e68640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438c14ef90 0x7f438c143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f4390e68640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f438c158ef0 0x7f438c14f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f4390e68640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c14db50 con 0x7f438c158ef0 2023-10-30T12:35:39.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f4390e68640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c150ba0 con 0x7f43740047d0 2023-10-30T12:35:39.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f4390e68640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438c15a8b0 con 0x7f438c14ef90 2023-10-30T12:35:39.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f438ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f438c158ef0 0x7f438c14f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f438ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f438c158ef0 0x7f438c14f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57410/0 (socket says 172.21.15.12:57410) 2023-10-30T12:35:39.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f438ad9d640 1 -- 172.21.15.12:0/3074982022 learned_addr learned my addr 172.21.15.12:0/3074982022 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:39.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f438a59c640 1 --2- 172.21.15.12:0/3074982022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43740047d0 0x7f438c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f4389d9b640 1 --2- 172.21.15.12:0/3074982022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438c14ef90 0x7f438c143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f438ad9d640 1 -- 172.21.15.12:0/3074982022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438c14ef90 msgr2=0x7f438c143fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f438ad9d640 1 --2- 172.21.15.12:0/3074982022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438c14ef90 0x7f438c143fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f438ad9d640 1 -- 172.21.15.12:0/3074982022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43740047d0 msgr2=0x7f438c0b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f438ad9d640 1 --2- 172.21.15.12:0/3074982022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43740047d0 0x7f438c0b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f438ad9d640 1 -- 172.21.15.12:0/3074982022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438c1508c0 con 0x7f438c158ef0 2023-10-30T12:35:39.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f438ad9d640 1 --2- 172.21.15.12:0/3074982022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f438c158ef0 0x7f438c14f8a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f438c06cb00 tx=0x7f438400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:39.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f437affd640 1 -- 172.21.15.12:0/3074982022 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438400e040 con 0x7f438c158ef0 2023-10-30T12:35:39.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f437affd640 1 -- 172.21.15.12:0/3074982022 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438400f040 con 0x7f438c158ef0 2023-10-30T12:35:39.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f437affd640 1 -- 172.21.15.12:0/3074982022 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4384015430 con 0x7f438c158ef0 2023-10-30T12:35:39.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f4390e68640 1 -- 172.21.15.12:0/3074982022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438c0b4770 con 0x7f438c158ef0 2023-10-30T12:35:39.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f4390e68640 1 -- 172.21.15.12:0/3074982022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438c14dda0 con 0x7f438c158ef0 2023-10-30T12:35:39.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f4390e68640 1 -- 172.21.15.12:0/3074982022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438c14df90 con 0x7f438c158ef0 2023-10-30T12:35:39.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.178+0000 7f437affd640 1 -- 172.21.15.12:0/3074982022 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f43840157a0 con 0x7f438c158ef0 2023-10-30T12:35:39.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.182+0000 7f437affd640 1 --2- 172.21.15.12:0/3074982022 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f436c0664a0 0x7f436c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.182+0000 7f437affd640 1 -- 172.21.15.12:0/3074982022 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4384084a80 con 0x7f438c158ef0 2023-10-30T12:35:39.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.182+0000 7f438a59c640 1 --2- 172.21.15.12:0/3074982022 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f436c0664a0 0x7f436c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.182+0000 7f438a59c640 1 --2- 172.21.15.12:0/3074982022 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f436c0664a0 0x7f436c068960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f437c001060 tx=0x7f437c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:39.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.186+0000 7f437affd640 1 -- 172.21.15.12:0/3074982022 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4384051ef0 con 0x7f438c158ef0 2023-10-30T12:35:39.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.378+0000 7f4390e68640 1 -- 172.21.15.12:0/3074982022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f438c04e170 con 0x7f438c158ef0 2023-10-30T12:35:39.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.378+0000 7f437affd640 1 -- 172.21.15.12:0/3074982022 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6523 (secure 0 0 0) 0x7f4384055510 con 0x7f438c158ef0 2023-10-30T12:35:39.383 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:39.384 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:35:27.538464+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":9,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:35:39.384 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2023-10-30T12:35:39.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.382+0000 7f4390e68640 1 -- 172.21.15.12:0/3074982022 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f436c0664a0 msgr2=0x7f436c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.382+0000 7f4390e68640 1 --2- 172.21.15.12:0/3074982022 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f436c0664a0 0x7f436c068960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f437c001060 tx=0x7f437c009040 comp rx=0 tx=0).stop 2023-10-30T12:35:39.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.382+0000 7f4390e68640 1 -- 172.21.15.12:0/3074982022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f438c158ef0 msgr2=0x7f438c14f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.382+0000 7f4390e68640 1 --2- 172.21.15.12:0/3074982022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f438c158ef0 0x7f438c14f8a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f438c06cb00 tx=0x7f438400f580 comp rx=0 tx=0).stop 2023-10-30T12:35:39.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.382+0000 7f4390e68640 1 -- 172.21.15.12:0/3074982022 shutdown_connections 2023-10-30T12:35:39.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.382+0000 7f4390e68640 1 --2- 172.21.15.12:0/3074982022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f438c158ef0 0x7f438c14f8a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.382+0000 7f4390e68640 1 --2- 172.21.15.12:0/3074982022 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f436c0664a0 0x7f436c068960 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.382+0000 7f4390e68640 1 --2- 172.21.15.12:0/3074982022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438c14ef90 0x7f438c143fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.382+0000 7f4390e68640 1 --2- 172.21.15.12:0/3074982022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43740047d0 0x7f438c0b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.382+0000 7f4390e68640 1 -- 172.21.15.12:0/3074982022 >> 172.21.15.12:0/3074982022 conn(0x7f438c06bfe0 msgr2=0x7f438c152170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:39.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.382+0000 7f4390e68640 1 -- 172.21.15.12:0/3074982022 shutdown_connections 2023-10-30T12:35:39.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.382+0000 7f4390e68640 1 -- 172.21.15.12:0/3074982022 wait complete. 2023-10-30T12:35:39.404 INFO:tasks.mds_thrash:Ready to start thrashing 2023-10-30T12:35:39.405 INFO:tasks.mds_thrash.ceph_manager:waiting for clean 2023-10-30T12:35:39.405 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:35:39.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.546+0000 7f00db73e640 1 Processor -- start 2023-10-30T12:35:39.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.546+0000 7f00db73e640 1 -- start start 2023-10-30T12:35:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.546+0000 7f00db73e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00d40c6b80 0x7f00d40bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.546+0000 7f00db73e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f00d413f4a0 0x7f00d40c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.546+0000 7f00db73e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d40c0a50 con 0x7f00d40c6f60 2023-10-30T12:35:39.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.546+0000 7f00db73e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d406f6e0 con 0x7f00d413f4a0 2023-10-30T12:35:39.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.546+0000 7f00db73e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d406f9c0 con 0x7f00d40c6b80 2023-10-30T12:35:39.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.546+0000 7f00d8cd9640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f00d40c6f60 0x7f00d413ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54480/0 (socket says 172.21.15.12:54480) 2023-10-30T12:35:39.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.546+0000 7f00d8cd9640 1 -- 172.21.15.12:0/1271237080 learned_addr learned my addr 172.21.15.12:0/1271237080 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:39.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.546+0000 7f00d94da640 1 --2- 172.21.15.12:0/1271237080 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00d40c6b80 0x7f00d40bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.546+0000 7f00d9cdb640 1 --2- 172.21.15.12:0/1271237080 >> v2:172.21.15.154:3300/0 conn(0x7f00d413f4a0 0x7f00d40c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.550+0000 7f00cbfff640 1 -- 172.21.15.12:0/1271237080 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206102038 0 0) 0x7f00d40c0a50 con 0x7f00d40c6f60 2023-10-30T12:35:39.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.550+0000 7f00cbfff640 1 -- 172.21.15.12:0/1271237080 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00c0003530 con 0x7f00d40c6f60 2023-10-30T12:35:39.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.550+0000 7f00d94da640 1 -- 172.21.15.12:0/1271237080 >> v2:172.21.15.154:3300/0 conn(0x7f00d413f4a0 msgr2=0x7f00d40c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:39.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.550+0000 7f00d94da640 1 --2- 172.21.15.12:0/1271237080 >> v2:172.21.15.154:3300/0 conn(0x7f00d413f4a0 0x7f00d40c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.550+0000 7f00d94da640 1 -- 172.21.15.12:0/1271237080 >> v1:172.21.15.12:6789/0 conn(0x7f00d40c6f60 legacy=0x7f00d413ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.550+0000 7f00cbfff640 1 -- 172.21.15.12:0/1271237080 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 840131065 0 0) 0x7f00c0003530 con 0x7f00d40c6f60 2023-10-30T12:35:39.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.550+0000 7f00d94da640 1 -- 172.21.15.12:0/1271237080 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d406fca0 con 0x7f00d40c6b80 2023-10-30T12:35:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.550+0000 7f00d94da640 1 --2- 172.21.15.12:0/1271237080 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00d40c6b80 0x7f00d40bfec0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f00d00097c0 tx=0x7f00d001c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2452961bc2b01f01 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.550+0000 7f00cbfff640 1 -- 172.21.15.12:0/1271237080 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4001680 con 0x7f00d40c6b80 2023-10-30T12:35:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.550+0000 7f00cbfff640 1 -- 172.21.15.12:0/1271237080 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d00a7050 con 0x7f00d40c6b80 2023-10-30T12:35:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.550+0000 7f00cbfff640 1 -- 172.21.15.12:0/1271237080 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d00ab620 con 0x7f00d40c6b80 2023-10-30T12:35:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 -- 172.21.15.12:0/1271237080 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00d40c6b80 msgr2=0x7f00d40bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 --2- 172.21.15.12:0/1271237080 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00d40c6b80 0x7f00d40bfec0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f00d00097c0 tx=0x7f00d001c3a0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 -- 172.21.15.12:0/1271237080 shutdown_connections 2023-10-30T12:35:39.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 --2- 172.21.15.12:0/1271237080 >> v2:172.21.15.154:3300/0 conn(0x7f00d413f4a0 0x7f00d40c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 --2- 172.21.15.12:0/1271237080 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00d40c6b80 0x7f00d40bfec0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 -- 172.21.15.12:0/1271237080 >> 172.21.15.12:0/1271237080 conn(0x7f00d40c7910 msgr2=0x7f00d40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:39.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 -- 172.21.15.12:0/1271237080 shutdown_connections 2023-10-30T12:35:39.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 -- 172.21.15.12:0/1271237080 wait complete. 2023-10-30T12:35:39.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 Processor -- start 2023-10-30T12:35:39.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 -- start start 2023-10-30T12:35:39.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00d40c6b80 0x7f00d413e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00d40c6f60 0x7f00d41f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00d413f4a0 0x7f00d4159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d40c19b0 con 0x7f00d413f4a0 2023-10-30T12:35:39.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d41f1960 con 0x7f00d40c6f60 2023-10-30T12:35:39.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00db73e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d41f28b0 con 0x7f00d40c6b80 2023-10-30T12:35:39.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00d9cdb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00d413f4a0 0x7f00d4159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00d9cdb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00d413f4a0 0x7f00d4159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57424/0 (socket says 172.21.15.12:57424) 2023-10-30T12:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00d9cdb640 1 -- 172.21.15.12:0/1026802343 learned_addr learned my addr 172.21.15.12:0/1026802343 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00d94da640 1 --2- 172.21.15.12:0/1026802343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00d40c6b80 0x7f00d413e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.554+0000 7f00d8cd9640 1 --2- 172.21.15.12:0/1026802343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00d40c6f60 0x7f00d41f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.558+0000 7f00d9cdb640 1 -- 172.21.15.12:0/1026802343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00d40c6b80 msgr2=0x7f00d413e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.558+0000 7f00d9cdb640 1 --2- 172.21.15.12:0/1026802343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00d40c6b80 0x7f00d413e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.558+0000 7f00d9cdb640 1 -- 172.21.15.12:0/1026802343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00d40c6f60 msgr2=0x7f00d41f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.558+0000 7f00d9cdb640 1 --2- 172.21.15.12:0/1026802343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00d40c6f60 0x7f00d41f2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.558+0000 7f00d9cdb640 1 -- 172.21.15.12:0/1026802343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d0009420 con 0x7f00d413f4a0 2023-10-30T12:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.558+0000 7f00d9cdb640 1 --2- 172.21.15.12:0/1026802343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00d413f4a0 0x7f00d4159320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00ef90 tx=0x7f00cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.558+0000 7f00c9ffb640 1 -- 172.21.15.12:0/1026802343 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc009b20 con 0x7f00d413f4a0 2023-10-30T12:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.558+0000 7f00c9ffb640 1 -- 172.21.15.12:0/1026802343 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00cc00f040 con 0x7f00d413f4a0 2023-10-30T12:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.558+0000 7f00c9ffb640 1 -- 172.21.15.12:0/1026802343 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc0146c0 con 0x7f00d413f4a0 2023-10-30T12:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.558+0000 7f00db73e640 1 -- 172.21.15.12:0/1026802343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d41f5d20 con 0x7f00d413f4a0 2023-10-30T12:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.558+0000 7f00db73e640 1 -- 172.21.15.12:0/1026802343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d40c16d0 con 0x7f00d413f4a0 2023-10-30T12:35:39.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.558+0000 7f00db73e640 1 -- 172.21.15.12:0/1026802343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d404e170 con 0x7f00d413f4a0 2023-10-30T12:35:39.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.558+0000 7f00c9ffb640 1 -- 172.21.15.12:0/1026802343 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f00cc014820 con 0x7f00d413f4a0 2023-10-30T12:35:39.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.558+0000 7f00c9ffb640 1 --2- 172.21.15.12:0/1026802343 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f00b80664f0 0x7f00b80689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.558+0000 7f00c9ffb640 1 -- 172.21.15.12:0/1026802343 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00cc084bc0 con 0x7f00d413f4a0 2023-10-30T12:35:39.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.562+0000 7f00d94da640 1 --2- 172.21.15.12:0/1026802343 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f00b80664f0 0x7f00b80689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.562+0000 7f00d94da640 1 --2- 172.21.15.12:0/1026802343 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f00b80664f0 0x7f00b80689b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f00d00ad000 tx=0x7f00d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:39.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.562+0000 7f00c9ffb640 1 -- 172.21.15.12:0/1026802343 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f00cc052040 con 0x7f00d413f4a0 2023-10-30T12:35:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.718+0000 7f00db73e640 1 -- 172.21.15.12:0/1026802343 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00d4071cc0 con 0x7f00b80664f0 2023-10-30T12:35:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.738+0000 7f00c9ffb640 1 -- 172.21.15.12:0/1026802343 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264700 (secure 0 0 0) 0x7f00d4071cc0 con 0x7f00b80664f0 2023-10-30T12:35:39.745 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-10-30T12:35:39.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.742+0000 7f00db73e640 1 -- 172.21.15.12:0/1026802343 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f00b80664f0 msgr2=0x7f00b80689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.742+0000 7f00db73e640 1 --2- 172.21.15.12:0/1026802343 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f00b80664f0 0x7f00b80689b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f00d00ad000 tx=0x7f00d0008040 comp rx=0 tx=0).stop 2023-10-30T12:35:39.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.742+0000 7f00db73e640 1 -- 172.21.15.12:0/1026802343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00d413f4a0 msgr2=0x7f00d4159320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.742+0000 7f00db73e640 1 --2- 172.21.15.12:0/1026802343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00d413f4a0 0x7f00d4159320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00ef90 tx=0x7f00cc00c490 comp rx=0 tx=0).stop 2023-10-30T12:35:39.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.742+0000 7f00db73e640 1 -- 172.21.15.12:0/1026802343 shutdown_connections 2023-10-30T12:35:39.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.742+0000 7f00db73e640 1 --2- 172.21.15.12:0/1026802343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00d413f4a0 0x7f00d4159320 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.742+0000 7f00db73e640 1 --2- 172.21.15.12:0/1026802343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00d40c6f60 0x7f00d41f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.742+0000 7f00db73e640 1 --2- 172.21.15.12:0/1026802343 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f00b80664f0 0x7f00b80689b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.742+0000 7f00db73e640 1 --2- 172.21.15.12:0/1026802343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00d40c6b80 0x7f00d413e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.742+0000 7f00db73e640 1 -- 172.21.15.12:0/1026802343 >> 172.21.15.12:0/1026802343 conn(0x7f00d40c7910 msgr2=0x7f00d4160770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:39.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.742+0000 7f00db73e640 1 -- 172.21.15.12:0/1026802343 shutdown_connections 2023-10-30T12:35:39.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.742+0000 7f00db73e640 1 -- 172.21.15.12:0/1026802343 wait complete. 2023-10-30T12:35:39.765 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-10-30T12:35:38.947034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596921,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":288,"num_write_kb":2458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219208,"kb_used_data":3080,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755512,"statfs":{"total":773094113280,"available":772869644288,"internally_reserved":0,"allocated":3153920,"data_stored":1824754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12729,"internal_metadata":221236807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2737,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004266"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250527+0000","last_change":"2023-10-30T12:35:06.223711+0000","last_active":"2023-10-30T12:35:07.250527+0000","last_peered":"2023-10-30T12:35:07.250527+0000","last_clean":"2023-10-30T12:35:07.250527+0000","last_became_active":"2023-10-30T12:35:06.222724+0000","last_became_peered":"2023-10-30T12:35:06.222724+0000","last_unstale":"2023-10-30T12:35:07.250527+0000","last_undegraded":"2023-10-30T12:35:07.250527+0000","last_fullsized":"2023-10-30T12:35:07.250527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:34:35.217274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159376+0000","last_change":"2023-10-30T12:35:06.182748+0000","last_active":"2023-10-30T12:35:07.159376+0000","last_peered":"2023-10-30T12:35:07.159376+0000","last_clean":"2023-10-30T12:35:07.159376+0000","last_became_active":"2023-10-30T12:35:06.182441+0000","last_became_peered":"2023-10-30T12:35:06.182441+0000","last_unstale":"2023-10-30T12:35:07.159376+0000","last_undegraded":"2023-10-30T12:35:07.159376+0000","last_fullsized":"2023-10-30T12:35:07.159376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:26:28.943768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160588+0000","last_change":"2023-10-30T12:35:06.180426+0000","last_active":"2023-10-30T12:35:07.160588+0000","last_peered":"2023-10-30T12:35:07.160588+0000","last_clean":"2023-10-30T12:35:07.160588+0000","last_became_active":"2023-10-30T12:35:06.180132+0000","last_became_peered":"2023-10-30T12:35:06.180132+0000","last_unstale":"2023-10-30T12:35:07.160588+0000","last_undegraded":"2023-10-30T12:35:07.160588+0000","last_fullsized":"2023-10-30T12:35:07.160588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:33:22.216739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250158+0000","last_change":"2023-10-30T12:35:06.217986+0000","last_active":"2023-10-30T12:35:07.250158+0000","last_peered":"2023-10-30T12:35:07.250158+0000","last_clean":"2023-10-30T12:35:07.250158+0000","last_became_active":"2023-10-30T12:35:06.213987+0000","last_became_peered":"2023-10-30T12:35:06.213987+0000","last_unstale":"2023-10-30T12:35:07.250158+0000","last_undegraded":"2023-10-30T12:35:07.250158+0000","last_fullsized":"2023-10-30T12:35:07.250158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:51:20.515999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249406+0000","last_change":"2023-10-30T12:35:06.216098+0000","last_active":"2023-10-30T12:35:07.249406+0000","last_peered":"2023-10-30T12:35:07.249406+0000","last_clean":"2023-10-30T12:35:07.249406+0000","last_became_active":"2023-10-30T12:35:06.214964+0000","last_became_peered":"2023-10-30T12:35:06.214964+0000","last_unstale":"2023-10-30T12:35:07.249406+0000","last_undegraded":"2023-10-30T12:35:07.249406+0000","last_fullsized":"2023-10-30T12:35:07.249406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:37:43.156964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249861+0000","last_change":"2023-10-30T12:35:06.217608+0000","last_active":"2023-10-30T12:35:07.249861+0000","last_peered":"2023-10-30T12:35:07.249861+0000","last_clean":"2023-10-30T12:35:07.249861+0000","last_became_active":"2023-10-30T12:35:06.214219+0000","last_became_peered":"2023-10-30T12:35:06.214219+0000","last_unstale":"2023-10-30T12:35:07.249861+0000","last_undegraded":"2023-10-30T12:35:07.249861+0000","last_fullsized":"2023-10-30T12:35:07.249861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:35:27.431813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249000+0000","last_change":"2023-10-30T12:35:06.217120+0000","last_active":"2023-10-30T12:35:07.249000+0000","last_peered":"2023-10-30T12:35:07.249000+0000","last_clean":"2023-10-30T12:35:07.249000+0000","last_became_active":"2023-10-30T12:35:06.216468+0000","last_became_peered":"2023-10-30T12:35:06.216468+0000","last_unstale":"2023-10-30T12:35:07.249000+0000","last_undegraded":"2023-10-30T12:35:07.249000+0000","last_fullsized":"2023-10-30T12:35:07.249000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:26:07.313967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246707+0000","last_change":"2023-10-30T12:35:06.214767+0000","last_active":"2023-10-30T12:35:07.246707+0000","last_peered":"2023-10-30T12:35:07.246707+0000","last_clean":"2023-10-30T12:35:07.246707+0000","last_became_active":"2023-10-30T12:35:06.214384+0000","last_became_peered":"2023-10-30T12:35:06.214384+0000","last_unstale":"2023-10-30T12:35:07.246707+0000","last_undegraded":"2023-10-30T12:35:07.246707+0000","last_fullsized":"2023-10-30T12:35:07.246707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:29.866538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250843+0000","last_change":"2023-10-30T12:35:06.223503+0000","last_active":"2023-10-30T12:35:07.250843+0000","last_peered":"2023-10-30T12:35:07.250843+0000","last_clean":"2023-10-30T12:35:07.250843+0000","last_became_active":"2023-10-30T12:35:06.222692+0000","last_became_peered":"2023-10-30T12:35:06.222692+0000","last_unstale":"2023-10-30T12:35:07.250843+0000","last_undegraded":"2023-10-30T12:35:07.250843+0000","last_fullsized":"2023-10-30T12:35:07.250843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:31:36.926720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163304+0000","last_change":"2023-10-30T12:35:06.215938+0000","last_active":"2023-10-30T12:35:07.163304+0000","last_peered":"2023-10-30T12:35:07.163304+0000","last_clean":"2023-10-30T12:35:07.163304+0000","last_became_active":"2023-10-30T12:35:06.215384+0000","last_became_peered":"2023-10-30T12:35:06.215384+0000","last_unstale":"2023-10-30T12:35:07.163304+0000","last_undegraded":"2023-10-30T12:35:07.163304+0000","last_fullsized":"2023-10-30T12:35:07.163304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:46:21.253555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162425+0000","last_change":"2023-10-30T12:35:06.215965+0000","last_active":"2023-10-30T12:35:07.162425+0000","last_peered":"2023-10-30T12:35:07.162425+0000","last_clean":"2023-10-30T12:35:07.162425+0000","last_became_active":"2023-10-30T12:35:06.215355+0000","last_became_peered":"2023-10-30T12:35:06.215355+0000","last_unstale":"2023-10-30T12:35:07.162425+0000","last_undegraded":"2023-10-30T12:35:07.162425+0000","last_fullsized":"2023-10-30T12:35:07.162425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:07.929526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164185+0000","last_change":"2023-10-30T12:35:06.197278+0000","last_active":"2023-10-30T12:35:07.164185+0000","last_peered":"2023-10-30T12:35:07.164185+0000","last_clean":"2023-10-30T12:35:07.164185+0000","last_became_active":"2023-10-30T12:35:06.196503+0000","last_became_peered":"2023-10-30T12:35:06.196503+0000","last_unstale":"2023-10-30T12:35:07.164185+0000","last_undegraded":"2023-10-30T12:35:07.164185+0000","last_fullsized":"2023-10-30T12:35:07.164185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:06.928264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247199+0000","last_change":"2023-10-30T12:35:06.195721+0000","last_active":"2023-10-30T12:35:07.247199+0000","last_peered":"2023-10-30T12:35:07.247199+0000","last_clean":"2023-10-30T12:35:07.247199+0000","last_became_active":"2023-10-30T12:35:06.195408+0000","last_became_peered":"2023-10-30T12:35:06.195408+0000","last_unstale":"2023-10-30T12:35:07.247199+0000","last_undegraded":"2023-10-30T12:35:07.247199+0000","last_fullsized":"2023-10-30T12:35:07.247199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:59:02.872646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160258+0000","last_change":"2023-10-30T12:35:06.182939+0000","last_active":"2023-10-30T12:35:07.160258+0000","last_peered":"2023-10-30T12:35:07.160258+0000","last_clean":"2023-10-30T12:35:07.160258+0000","last_became_active":"2023-10-30T12:35:06.182573+0000","last_became_peered":"2023-10-30T12:35:06.182573+0000","last_unstale":"2023-10-30T12:35:07.160258+0000","last_undegraded":"2023-10-30T12:35:07.160258+0000","last_fullsized":"2023-10-30T12:35:07.160258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:50.345142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.158834+0000","last_change":"2023-10-30T12:35:06.185636+0000","last_active":"2023-10-30T12:35:07.158834+0000","last_peered":"2023-10-30T12:35:07.158834+0000","last_clean":"2023-10-30T12:35:07.158834+0000","last_became_active":"2023-10-30T12:35:06.185290+0000","last_became_peered":"2023-10-30T12:35:06.185290+0000","last_unstale":"2023-10-30T12:35:07.158834+0000","last_undegraded":"2023-10-30T12:35:07.158834+0000","last_fullsized":"2023-10-30T12:35:07.158834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:26:44.783714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.248972+0000","last_change":"2023-10-30T12:35:06.214488+0000","last_active":"2023-10-30T12:35:07.248972+0000","last_peered":"2023-10-30T12:35:07.248972+0000","last_clean":"2023-10-30T12:35:07.248972+0000","last_became_active":"2023-10-30T12:35:06.213627+0000","last_became_peered":"2023-10-30T12:35:06.213627+0000","last_unstale":"2023-10-30T12:35:07.248972+0000","last_undegraded":"2023-10-30T12:35:07.248972+0000","last_fullsized":"2023-10-30T12:35:07.248972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:22:47.148407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251739+0000","last_change":"2023-10-30T12:35:06.224903+0000","last_active":"2023-10-30T12:35:07.251739+0000","last_peered":"2023-10-30T12:35:07.251739+0000","last_clean":"2023-10-30T12:35:07.251739+0000","last_became_active":"2023-10-30T12:35:06.221696+0000","last_became_peered":"2023-10-30T12:35:06.221696+0000","last_unstale":"2023-10-30T12:35:07.251739+0000","last_undegraded":"2023-10-30T12:35:07.251739+0000","last_fullsized":"2023-10-30T12:35:07.251739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:31:10.439162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246945+0000","last_change":"2023-10-30T12:35:06.198303+0000","last_active":"2023-10-30T12:35:07.246945+0000","last_peered":"2023-10-30T12:35:07.246945+0000","last_clean":"2023-10-30T12:35:07.246945+0000","last_became_active":"2023-10-30T12:35:06.198074+0000","last_became_peered":"2023-10-30T12:35:06.198074+0000","last_unstale":"2023-10-30T12:35:07.246945+0000","last_undegraded":"2023-10-30T12:35:07.246945+0000","last_fullsized":"2023-10-30T12:35:07.246945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:21:36.079586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250503+0000","last_change":"2023-10-30T12:35:06.218556+0000","last_active":"2023-10-30T12:35:07.250503+0000","last_peered":"2023-10-30T12:35:07.250503+0000","last_clean":"2023-10-30T12:35:07.250503+0000","last_became_active":"2023-10-30T12:35:06.215935+0000","last_became_peered":"2023-10-30T12:35:06.215935+0000","last_unstale":"2023-10-30T12:35:07.250503+0000","last_undegraded":"2023-10-30T12:35:07.250503+0000","last_fullsized":"2023-10-30T12:35:07.250503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:36:31.084542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162438+0000","last_change":"2023-10-30T12:35:06.201302+0000","last_active":"2023-10-30T12:35:07.162438+0000","last_peered":"2023-10-30T12:35:07.162438+0000","last_clean":"2023-10-30T12:35:07.162438+0000","last_became_active":"2023-10-30T12:35:06.199812+0000","last_became_peered":"2023-10-30T12:35:06.199812+0000","last_unstale":"2023-10-30T12:35:07.162438+0000","last_undegraded":"2023-10-30T12:35:07.162438+0000","last_fullsized":"2023-10-30T12:35:07.162438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:48.803738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160558+0000","last_change":"2023-10-30T12:35:06.217139+0000","last_active":"2023-10-30T12:35:07.160558+0000","last_peered":"2023-10-30T12:35:07.160558+0000","last_clean":"2023-10-30T12:35:07.160558+0000","last_became_active":"2023-10-30T12:35:06.216861+0000","last_became_peered":"2023-10-30T12:35:06.216861+0000","last_unstale":"2023-10-30T12:35:07.160558+0000","last_undegraded":"2023-10-30T12:35:07.160558+0000","last_fullsized":"2023-10-30T12:35:07.160558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:05:22.193180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249973+0000","last_change":"2023-10-30T12:35:06.225140+0000","last_active":"2023-10-30T12:35:07.249973+0000","last_peered":"2023-10-30T12:35:07.249973+0000","last_clean":"2023-10-30T12:35:07.249973+0000","last_became_active":"2023-10-30T12:35:06.221841+0000","last_became_peered":"2023-10-30T12:35:06.221841+0000","last_unstale":"2023-10-30T12:35:07.249973+0000","last_undegraded":"2023-10-30T12:35:07.249973+0000","last_fullsized":"2023-10-30T12:35:07.249973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:51:57.131025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247032+0000","last_change":"2023-10-30T12:35:06.182662+0000","last_active":"2023-10-30T12:35:07.247032+0000","last_peered":"2023-10-30T12:35:07.247032+0000","last_clean":"2023-10-30T12:35:07.247032+0000","last_became_active":"2023-10-30T12:35:06.182314+0000","last_became_peered":"2023-10-30T12:35:06.182314+0000","last_unstale":"2023-10-30T12:35:07.247032+0000","last_undegraded":"2023-10-30T12:35:07.247032+0000","last_fullsized":"2023-10-30T12:35:07.247032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:41:49.587856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249533+0000","last_change":"2023-10-30T12:35:06.214113+0000","last_active":"2023-10-30T12:35:07.249533+0000","last_peered":"2023-10-30T12:35:07.249533+0000","last_clean":"2023-10-30T12:35:07.249533+0000","last_became_active":"2023-10-30T12:35:06.213402+0000","last_became_peered":"2023-10-30T12:35:06.213402+0000","last_unstale":"2023-10-30T12:35:07.249533+0000","last_undegraded":"2023-10-30T12:35:07.249533+0000","last_fullsized":"2023-10-30T12:35:07.249533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:14:58.409296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247781+0000","last_change":"2023-10-30T12:35:06.188570+0000","last_active":"2023-10-30T12:35:07.247781+0000","last_peered":"2023-10-30T12:35:07.247781+0000","last_clean":"2023-10-30T12:35:07.247781+0000","last_became_active":"2023-10-30T12:35:06.188298+0000","last_became_peered":"2023-10-30T12:35:06.188298+0000","last_unstale":"2023-10-30T12:35:07.247781+0000","last_undegraded":"2023-10-30T12:35:07.247781+0000","last_fullsized":"2023-10-30T12:35:07.247781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:28.281254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163623+0000","last_change":"2023-10-30T12:35:06.208821+0000","last_active":"2023-10-30T12:35:07.163623+0000","last_peered":"2023-10-30T12:35:07.163623+0000","last_clean":"2023-10-30T12:35:07.163623+0000","last_became_active":"2023-10-30T12:35:06.202507+0000","last_became_peered":"2023-10-30T12:35:06.202507+0000","last_unstale":"2023-10-30T12:35:07.163623+0000","last_undegraded":"2023-10-30T12:35:07.163623+0000","last_fullsized":"2023-10-30T12:35:07.163623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:40:13.124316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159171+0000","last_change":"2023-10-30T12:35:06.170313+0000","last_active":"2023-10-30T12:35:07.159171+0000","last_peered":"2023-10-30T12:35:07.159171+0000","last_clean":"2023-10-30T12:35:07.159171+0000","last_became_active":"2023-10-30T12:35:06.170030+0000","last_became_peered":"2023-10-30T12:35:06.170030+0000","last_unstale":"2023-10-30T12:35:07.159171+0000","last_undegraded":"2023-10-30T12:35:07.159171+0000","last_fullsized":"2023-10-30T12:35:07.159171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:49:59.706208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247487+0000","last_change":"2023-10-30T12:35:06.189448+0000","last_active":"2023-10-30T12:35:07.247487+0000","last_peered":"2023-10-30T12:35:07.247487+0000","last_clean":"2023-10-30T12:35:07.247487+0000","last_became_active":"2023-10-30T12:35:06.189171+0000","last_became_peered":"2023-10-30T12:35:06.189171+0000","last_unstale":"2023-10-30T12:35:07.247487+0000","last_undegraded":"2023-10-30T12:35:07.247487+0000","last_fullsized":"2023-10-30T12:35:07.247487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:18:51.463273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160658+0000","last_change":"2023-10-30T12:35:06.183680+0000","last_active":"2023-10-30T12:35:07.160658+0000","last_peered":"2023-10-30T12:35:07.160658+0000","last_clean":"2023-10-30T12:35:07.160658+0000","last_became_active":"2023-10-30T12:35:06.183348+0000","last_became_peered":"2023-10-30T12:35:06.183348+0000","last_unstale":"2023-10-30T12:35:07.160658+0000","last_undegraded":"2023-10-30T12:35:07.160658+0000","last_fullsized":"2023-10-30T12:35:07.160658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:54:25.429897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250238+0000","last_change":"2023-10-30T12:35:06.206675+0000","last_active":"2023-10-30T12:35:07.250238+0000","last_peered":"2023-10-30T12:35:07.250238+0000","last_clean":"2023-10-30T12:35:07.250238+0000","last_became_active":"2023-10-30T12:35:06.196368+0000","last_became_peered":"2023-10-30T12:35:06.196368+0000","last_unstale":"2023-10-30T12:35:07.250238+0000","last_undegraded":"2023-10-30T12:35:07.250238+0000","last_fullsized":"2023-10-30T12:35:07.250238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:35.929311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246681+0000","last_change":"2023-10-30T12:35:06.214408+0000","last_active":"2023-10-30T12:35:07.246681+0000","last_peered":"2023-10-30T12:35:07.246681+0000","last_clean":"2023-10-30T12:35:07.246681+0000","last_became_active":"2023-10-30T12:35:06.213994+0000","last_became_peered":"2023-10-30T12:35:06.213994+0000","last_unstale":"2023-10-30T12:35:07.246681+0000","last_undegraded":"2023-10-30T12:35:07.246681+0000","last_fullsized":"2023-10-30T12:35:07.246681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:05:58.830698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162027+0000","last_change":"2023-10-30T12:35:06.198444+0000","last_active":"2023-10-30T12:35:07.162027+0000","last_peered":"2023-10-30T12:35:07.162027+0000","last_clean":"2023-10-30T12:35:07.162027+0000","last_became_active":"2023-10-30T12:35:06.197647+0000","last_became_peered":"2023-10-30T12:35:06.197647+0000","last_unstale":"2023-10-30T12:35:07.162027+0000","last_undegraded":"2023-10-30T12:35:07.162027+0000","last_fullsized":"2023-10-30T12:35:07.162027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:28:56.075420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163077+0000","last_change":"2023-10-30T12:35:06.168377+0000","last_active":"2023-10-30T12:35:07.163077+0000","last_peered":"2023-10-30T12:35:07.163077+0000","last_clean":"2023-10-30T12:35:07.163077+0000","last_became_active":"2023-10-30T12:35:06.167558+0000","last_became_peered":"2023-10-30T12:35:06.167558+0000","last_unstale":"2023-10-30T12:35:07.163077+0000","last_undegraded":"2023-10-30T12:35:07.163077+0000","last_fullsized":"2023-10-30T12:35:07.163077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:57:28.009164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.310153+0000","last_change":"2023-10-30T12:35:05.200844+0000","last_active":"2023-10-30T12:35:21.310153+0000","last_peered":"2023-10-30T12:35:21.310153+0000","last_clean":"2023-10-30T12:35:21.310153+0000","last_became_active":"2023-10-30T12:35:05.200169+0000","last_became_peered":"2023-10-30T12:35:05.200169+0000","last_unstale":"2023-10-30T12:35:21.310153+0000","last_undegraded":"2023-10-30T12:35:21.310153+0000","last_fullsized":"2023-10-30T12:35:21.310153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:07:19.571357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160718+0000","last_change":"2023-10-30T12:35:06.194964+0000","last_active":"2023-10-30T12:35:07.160718+0000","last_peered":"2023-10-30T12:35:07.160718+0000","last_clean":"2023-10-30T12:35:07.160718+0000","last_became_active":"2023-10-30T12:35:06.194689+0000","last_became_peered":"2023-10-30T12:35:06.194689+0000","last_unstale":"2023-10-30T12:35:07.160718+0000","last_undegraded":"2023-10-30T12:35:07.160718+0000","last_fullsized":"2023-10-30T12:35:07.160718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:30:53.410341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.336271+0000","last_change":"2023-10-30T12:35:05.183569+0000","last_active":"2023-10-30T12:35:27.336271+0000","last_peered":"2023-10-30T12:35:27.336271+0000","last_clean":"2023-10-30T12:35:27.336271+0000","last_became_active":"2023-10-30T12:35:05.182761+0000","last_became_peered":"2023-10-30T12:35:05.182761+0000","last_unstale":"2023-10-30T12:35:27.336271+0000","last_undegraded":"2023-10-30T12:35:27.336271+0000","last_fullsized":"2023-10-30T12:35:27.336271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:49:44.311814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.536230+0000","last_change":"2023-10-30T12:35:05.181278+0000","last_active":"2023-10-30T12:35:27.536230+0000","last_peered":"2023-10-30T12:35:27.536230+0000","last_clean":"2023-10-30T12:35:27.536230+0000","last_became_active":"2023-10-30T12:35:05.180730+0000","last_became_peered":"2023-10-30T12:35:05.180730+0000","last_unstale":"2023-10-30T12:35:27.536230+0000","last_undegraded":"2023-10-30T12:35:27.536230+0000","last_fullsized":"2023-10-30T12:35:27.536230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:07:10.828707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246964+0000","last_change":"2023-10-30T12:35:06.180049+0000","last_active":"2023-10-30T12:35:07.246964+0000","last_peered":"2023-10-30T12:35:07.246964+0000","last_clean":"2023-10-30T12:35:07.246964+0000","last_became_active":"2023-10-30T12:35:06.179764+0000","last_became_peered":"2023-10-30T12:35:06.179764+0000","last_unstale":"2023-10-30T12:35:07.246964+0000","last_undegraded":"2023-10-30T12:35:07.246964+0000","last_fullsized":"2023-10-30T12:35:07.246964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:33.136727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249880+0000","last_change":"2023-10-30T12:35:05.202539+0000","last_active":"2023-10-30T12:35:07.249880+0000","last_peered":"2023-10-30T12:35:07.249880+0000","last_clean":"2023-10-30T12:35:07.249880+0000","last_became_active":"2023-10-30T12:35:05.201853+0000","last_became_peered":"2023-10-30T12:35:05.201853+0000","last_unstale":"2023-10-30T12:35:07.249880+0000","last_undegraded":"2023-10-30T12:35:07.249880+0000","last_fullsized":"2023-10-30T12:35:07.249880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:33:33.151433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.158869+0000","last_change":"2023-10-30T12:35:06.173567+0000","last_active":"2023-10-30T12:35:07.158869+0000","last_peered":"2023-10-30T12:35:07.158869+0000","last_clean":"2023-10-30T12:35:07.158869+0000","last_became_active":"2023-10-30T12:35:06.170179+0000","last_became_peered":"2023-10-30T12:35:06.170179+0000","last_unstale":"2023-10-30T12:35:07.158869+0000","last_undegraded":"2023-10-30T12:35:07.158869+0000","last_fullsized":"2023-10-30T12:35:07.158869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:39:07.880510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.306226+0000","last_change":"2023-10-30T12:35:05.198546+0000","last_active":"2023-10-30T12:35:21.306226+0000","last_peered":"2023-10-30T12:35:21.306226+0000","last_clean":"2023-10-30T12:35:21.306226+0000","last_became_active":"2023-10-30T12:35:05.197606+0000","last_became_peered":"2023-10-30T12:35:05.197606+0000","last_unstale":"2023-10-30T12:35:21.306226+0000","last_undegraded":"2023-10-30T12:35:21.306226+0000","last_fullsized":"2023-10-30T12:35:21.306226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:26:57.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251322+0000","last_change":"2023-10-30T12:35:06.226044+0000","last_active":"2023-10-30T12:35:07.251322+0000","last_peered":"2023-10-30T12:35:07.251322+0000","last_clean":"2023-10-30T12:35:07.251322+0000","last_became_active":"2023-10-30T12:35:06.225392+0000","last_became_peered":"2023-10-30T12:35:06.225392+0000","last_unstale":"2023-10-30T12:35:07.251322+0000","last_undegraded":"2023-10-30T12:35:07.251322+0000","last_fullsized":"2023-10-30T12:35:07.251322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:13:10.796549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.313047+0000","last_change":"2023-10-30T12:35:05.197614+0000","last_active":"2023-10-30T12:35:21.313047+0000","last_peered":"2023-10-30T12:35:21.313047+0000","last_clean":"2023-10-30T12:35:21.313047+0000","last_became_active":"2023-10-30T12:35:05.196899+0000","last_became_peered":"2023-10-30T12:35:05.196899+0000","last_unstale":"2023-10-30T12:35:21.313047+0000","last_undegraded":"2023-10-30T12:35:21.313047+0000","last_fullsized":"2023-10-30T12:35:21.313047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:21:00.152426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160715+0000","last_change":"2023-10-30T12:35:06.222344+0000","last_active":"2023-10-30T12:35:07.160715+0000","last_peered":"2023-10-30T12:35:07.160715+0000","last_clean":"2023-10-30T12:35:07.160715+0000","last_became_active":"2023-10-30T12:35:06.222057+0000","last_became_peered":"2023-10-30T12:35:06.222057+0000","last_unstale":"2023-10-30T12:35:07.160715+0000","last_undegraded":"2023-10-30T12:35:07.160715+0000","last_fullsized":"2023-10-30T12:35:07.160715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:50:46.691223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.201054+0000","last_change":"2023-10-30T12:35:05.197501+0000","last_active":"2023-10-30T12:35:11.201054+0000","last_peered":"2023-10-30T12:35:11.201054+0000","last_clean":"2023-10-30T12:35:11.201054+0000","last_became_active":"2023-10-30T12:35:05.197190+0000","last_became_peered":"2023-10-30T12:35:05.197190+0000","last_unstale":"2023-10-30T12:35:11.201054+0000","last_undegraded":"2023-10-30T12:35:11.201054+0000","last_fullsized":"2023-10-30T12:35:11.201054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:53:31.121721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164505+0000","last_change":"2023-10-30T12:35:06.204885+0000","last_active":"2023-10-30T12:35:07.164505+0000","last_peered":"2023-10-30T12:35:07.164505+0000","last_clean":"2023-10-30T12:35:07.164505+0000","last_became_active":"2023-10-30T12:35:06.204122+0000","last_became_peered":"2023-10-30T12:35:06.204122+0000","last_unstale":"2023-10-30T12:35:07.164505+0000","last_undegraded":"2023-10-30T12:35:07.164505+0000","last_fullsized":"2023-10-30T12:35:07.164505+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:27:18.433575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'11","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:35.466455+0000","last_change":"2023-10-30T12:35:05.180005+0000","last_active":"2023-10-30T12:35:35.466455+0000","last_peered":"2023-10-30T12:35:35.466455+0000","last_clean":"2023-10-30T12:35:35.466455+0000","last_became_active":"2023-10-30T12:35:05.179547+0000","last_became_peered":"2023-10-30T12:35:05.179547+0000","last_unstale":"2023-10-30T12:35:35.466455+0000","last_undegraded":"2023-10-30T12:35:35.466455+0000","last_fullsized":"2023-10-30T12:35:35.466455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:13:03.137433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2685,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249565+0000","last_change":"2023-10-30T12:35:06.216619+0000","last_active":"2023-10-30T12:35:07.249565+0000","last_peered":"2023-10-30T12:35:07.249565+0000","last_clean":"2023-10-30T12:35:07.249565+0000","last_became_active":"2023-10-30T12:35:06.215128+0000","last_became_peered":"2023-10-30T12:35:06.215128+0000","last_unstale":"2023-10-30T12:35:07.249565+0000","last_undegraded":"2023-10-30T12:35:07.249565+0000","last_fullsized":"2023-10-30T12:35:07.249565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:48:45.673159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.335411+0000","last_change":"2023-10-30T12:35:05.190041+0000","last_active":"2023-10-30T12:35:27.335411+0000","last_peered":"2023-10-30T12:35:27.335411+0000","last_clean":"2023-10-30T12:35:27.335411+0000","last_became_active":"2023-10-30T12:35:05.189573+0000","last_became_peered":"2023-10-30T12:35:05.189573+0000","last_unstale":"2023-10-30T12:35:27.335411+0000","last_undegraded":"2023-10-30T12:35:27.335411+0000","last_fullsized":"2023-10-30T12:35:27.335411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:02:57.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160412+0000","last_change":"2023-10-30T12:35:06.217334+0000","last_active":"2023-10-30T12:35:07.160412+0000","last_peered":"2023-10-30T12:35:07.160412+0000","last_clean":"2023-10-30T12:35:07.160412+0000","last_became_active":"2023-10-30T12:35:06.216885+0000","last_became_peered":"2023-10-30T12:35:06.216885+0000","last_unstale":"2023-10-30T12:35:07.160412+0000","last_undegraded":"2023-10-30T12:35:07.160412+0000","last_fullsized":"2023-10-30T12:35:07.160412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:38:26.745898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.313580+0000","last_change":"2023-10-30T12:35:05.203198+0000","last_active":"2023-10-30T12:35:21.313580+0000","last_peered":"2023-10-30T12:35:21.313580+0000","last_clean":"2023-10-30T12:35:21.313580+0000","last_became_active":"2023-10-30T12:35:05.202612+0000","last_became_peered":"2023-10-30T12:35:05.202612+0000","last_unstale":"2023-10-30T12:35:21.313580+0000","last_undegraded":"2023-10-30T12:35:21.313580+0000","last_fullsized":"2023-10-30T12:35:21.313580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:55.986236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249574+0000","last_change":"2023-10-30T12:35:06.216307+0000","last_active":"2023-10-30T12:35:07.249574+0000","last_peered":"2023-10-30T12:35:07.249574+0000","last_clean":"2023-10-30T12:35:07.249574+0000","last_became_active":"2023-10-30T12:35:06.215553+0000","last_became_peered":"2023-10-30T12:35:06.215553+0000","last_unstale":"2023-10-30T12:35:07.249574+0000","last_undegraded":"2023-10-30T12:35:07.249574+0000","last_fullsized":"2023-10-30T12:35:07.249574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:46:25.396759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.336548+0000","last_change":"2023-10-30T12:35:05.181882+0000","last_active":"2023-10-30T12:35:27.336548+0000","last_peered":"2023-10-30T12:35:27.336548+0000","last_clean":"2023-10-30T12:35:27.336548+0000","last_became_active":"2023-10-30T12:35:05.181115+0000","last_became_peered":"2023-10-30T12:35:05.181115+0000","last_unstale":"2023-10-30T12:35:27.336548+0000","last_undegraded":"2023-10-30T12:35:27.336548+0000","last_fullsized":"2023-10-30T12:35:27.336548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:18:55.496940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162013+0000","last_change":"2023-10-30T12:35:06.198990+0000","last_active":"2023-10-30T12:35:07.162013+0000","last_peered":"2023-10-30T12:35:07.162013+0000","last_clean":"2023-10-30T12:35:07.162013+0000","last_became_active":"2023-10-30T12:35:06.197722+0000","last_became_peered":"2023-10-30T12:35:06.197722+0000","last_unstale":"2023-10-30T12:35:07.162013+0000","last_undegraded":"2023-10-30T12:35:07.162013+0000","last_fullsized":"2023-10-30T12:35:07.162013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:56:51.757976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'15","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.338386+0000","last_change":"2023-10-30T12:35:05.169455+0000","last_active":"2023-10-30T12:35:27.338386+0000","last_peered":"2023-10-30T12:35:27.338386+0000","last_clean":"2023-10-30T12:35:27.338386+0000","last_became_active":"2023-10-30T12:35:05.169130+0000","last_became_peered":"2023-10-30T12:35:05.169130+0000","last_unstale":"2023-10-30T12:35:27.338386+0000","last_undegraded":"2023-10-30T12:35:27.338386+0000","last_fullsized":"2023-10-30T12:35:27.338386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:59:14.096315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249183+0000","last_change":"2023-10-30T12:35:06.215545+0000","last_active":"2023-10-30T12:35:07.249183+0000","last_peered":"2023-10-30T12:35:07.249183+0000","last_clean":"2023-10-30T12:35:07.249183+0000","last_became_active":"2023-10-30T12:35:06.214839+0000","last_became_peered":"2023-10-30T12:35:06.214839+0000","last_unstale":"2023-10-30T12:35:07.249183+0000","last_undegraded":"2023-10-30T12:35:07.249183+0000","last_fullsized":"2023-10-30T12:35:07.249183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:54:34.206981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.199110+0000","last_change":"2023-10-30T12:35:05.189361+0000","last_active":"2023-10-30T12:35:11.199110+0000","last_peered":"2023-10-30T12:35:11.199110+0000","last_clean":"2023-10-30T12:35:11.199110+0000","last_became_active":"2023-10-30T12:35:05.188862+0000","last_became_peered":"2023-10-30T12:35:05.188862+0000","last_unstale":"2023-10-30T12:35:11.199110+0000","last_undegraded":"2023-10-30T12:35:11.199110+0000","last_fullsized":"2023-10-30T12:35:11.199110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:13.319732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250712+0000","last_change":"2023-10-30T12:35:06.223118+0000","last_active":"2023-10-30T12:35:07.250712+0000","last_peered":"2023-10-30T12:35:07.250712+0000","last_clean":"2023-10-30T12:35:07.250712+0000","last_became_active":"2023-10-30T12:35:06.216533+0000","last_became_peered":"2023-10-30T12:35:06.216533+0000","last_unstale":"2023-10-30T12:35:07.250712+0000","last_undegraded":"2023-10-30T12:35:07.250712+0000","last_fullsized":"2023-10-30T12:35:07.250712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:32:29.374526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.195596+0000","last_change":"2023-10-30T12:35:05.177609+0000","last_active":"2023-10-30T12:35:11.195596+0000","last_peered":"2023-10-30T12:35:11.195596+0000","last_clean":"2023-10-30T12:35:11.195596+0000","last_became_active":"2023-10-30T12:35:05.177316+0000","last_became_peered":"2023-10-30T12:35:05.177316+0000","last_unstale":"2023-10-30T12:35:11.195596+0000","last_undegraded":"2023-10-30T12:35:11.195596+0000","last_fullsized":"2023-10-30T12:35:11.195596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:46.820925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251056+0000","last_change":"2023-10-30T12:35:06.222283+0000","last_active":"2023-10-30T12:35:07.251056+0000","last_peered":"2023-10-30T12:35:07.251056+0000","last_clean":"2023-10-30T12:35:07.251056+0000","last_became_active":"2023-10-30T12:35:06.221701+0000","last_became_peered":"2023-10-30T12:35:06.221701+0000","last_unstale":"2023-10-30T12:35:07.251056+0000","last_undegraded":"2023-10-30T12:35:07.251056+0000","last_fullsized":"2023-10-30T12:35:07.251056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:37:32.956627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159413+0000","last_change":"2023-10-30T12:35:05.171787+0000","last_active":"2023-10-30T12:35:07.159413+0000","last_peered":"2023-10-30T12:35:07.159413+0000","last_clean":"2023-10-30T12:35:07.159413+0000","last_became_active":"2023-10-30T12:35:05.171491+0000","last_became_peered":"2023-10-30T12:35:05.171491+0000","last_unstale":"2023-10-30T12:35:07.159413+0000","last_undegraded":"2023-10-30T12:35:07.159413+0000","last_fullsized":"2023-10-30T12:35:07.159413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:43:15.331175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250517+0000","last_change":"2023-10-30T12:35:06.214464+0000","last_active":"2023-10-30T12:35:07.250517+0000","last_peered":"2023-10-30T12:35:07.250517+0000","last_clean":"2023-10-30T12:35:07.250517+0000","last_became_active":"2023-10-30T12:35:06.213493+0000","last_became_peered":"2023-10-30T12:35:06.213493+0000","last_unstale":"2023-10-30T12:35:07.250517+0000","last_undegraded":"2023-10-30T12:35:07.250517+0000","last_fullsized":"2023-10-30T12:35:07.250517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:00:18.624720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.337571+0000","last_change":"2023-10-30T12:35:05.197811+0000","last_active":"2023-10-30T12:35:27.337571+0000","last_peered":"2023-10-30T12:35:27.337571+0000","last_clean":"2023-10-30T12:35:27.337571+0000","last_became_active":"2023-10-30T12:35:05.197453+0000","last_became_peered":"2023-10-30T12:35:05.197453+0000","last_unstale":"2023-10-30T12:35:27.337571+0000","last_undegraded":"2023-10-30T12:35:27.337571+0000","last_fullsized":"2023-10-30T12:35:27.337571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:16:14.550887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249901+0000","last_change":"2023-10-30T12:35:06.220236+0000","last_active":"2023-10-30T12:35:07.249901+0000","last_peered":"2023-10-30T12:35:07.249901+0000","last_clean":"2023-10-30T12:35:07.249901+0000","last_became_active":"2023-10-30T12:35:06.219526+0000","last_became_peered":"2023-10-30T12:35:06.219526+0000","last_unstale":"2023-10-30T12:35:07.249901+0000","last_undegraded":"2023-10-30T12:35:07.249901+0000","last_fullsized":"2023-10-30T12:35:07.249901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:00:25.297590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.186752+0000","last_change":"2023-10-30T12:34:55.206976+0000","last_active":"2023-10-30T12:35:06.186752+0000","last_peered":"2023-10-30T12:35:06.186752+0000","last_clean":"2023-10-30T12:35:06.186752+0000","last_became_active":"2023-10-30T12:34:55.205923+0000","last_became_peered":"2023-10-30T12:34:55.205923+0000","last_unstale":"2023-10-30T12:35:06.186752+0000","last_undegraded":"2023-10-30T12:35:06.186752+0000","last_fullsized":"2023-10-30T12:35:06.186752+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_clean_scrub_stamp":"2023-10-30T12:34:53.444020+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:25:28.782111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.313312+0000","last_change":"2023-10-30T12:35:05.197974+0000","last_active":"2023-10-30T12:35:21.313312+0000","last_peered":"2023-10-30T12:35:21.313312+0000","last_clean":"2023-10-30T12:35:21.313312+0000","last_became_active":"2023-10-30T12:35:05.197201+0000","last_became_peered":"2023-10-30T12:35:05.197201+0000","last_unstale":"2023-10-30T12:35:21.313312+0000","last_undegraded":"2023-10-30T12:35:21.313312+0000","last_fullsized":"2023-10-30T12:35:21.313312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:39:47.119091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247224+0000","last_change":"2023-10-30T12:35:06.217973+0000","last_active":"2023-10-30T12:35:07.247224+0000","last_peered":"2023-10-30T12:35:07.247224+0000","last_clean":"2023-10-30T12:35:07.247224+0000","last_became_active":"2023-10-30T12:35:06.217687+0000","last_became_peered":"2023-10-30T12:35:06.217687+0000","last_unstale":"2023-10-30T12:35:07.247224+0000","last_undegraded":"2023-10-30T12:35:07.247224+0000","last_fullsized":"2023-10-30T12:35:07.247224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:28.392113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.333670+0000","last_change":"2023-10-30T12:35:05.208339+0000","last_active":"2023-10-30T12:35:27.333670+0000","last_peered":"2023-10-30T12:35:27.333670+0000","last_clean":"2023-10-30T12:35:27.333670+0000","last_became_active":"2023-10-30T12:35:05.207828+0000","last_became_peered":"2023-10-30T12:35:05.207828+0000","last_unstale":"2023-10-30T12:35:27.333670+0000","last_undegraded":"2023-10-30T12:35:27.333670+0000","last_fullsized":"2023-10-30T12:35:27.333670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:00.649048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160255+0000","last_change":"2023-10-30T12:35:06.195094+0000","last_active":"2023-10-30T12:35:07.160255+0000","last_peered":"2023-10-30T12:35:07.160255+0000","last_clean":"2023-10-30T12:35:07.160255+0000","last_became_active":"2023-10-30T12:35:06.194791+0000","last_became_peered":"2023-10-30T12:35:06.194791+0000","last_unstale":"2023-10-30T12:35:07.160255+0000","last_undegraded":"2023-10-30T12:35:07.160255+0000","last_fullsized":"2023-10-30T12:35:07.160255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:41:24.310349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.336946+0000","last_change":"2023-10-30T12:35:05.176296+0000","last_active":"2023-10-30T12:35:27.336946+0000","last_peered":"2023-10-30T12:35:27.336946+0000","last_clean":"2023-10-30T12:35:27.336946+0000","last_became_active":"2023-10-30T12:35:05.175959+0000","last_became_peered":"2023-10-30T12:35:05.175959+0000","last_unstale":"2023-10-30T12:35:27.336946+0000","last_undegraded":"2023-10-30T12:35:27.336946+0000","last_fullsized":"2023-10-30T12:35:27.336946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:53.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159052+0000","last_change":"2023-10-30T12:35:06.167250+0000","last_active":"2023-10-30T12:35:07.159052+0000","last_peered":"2023-10-30T12:35:07.159052+0000","last_clean":"2023-10-30T12:35:07.159052+0000","last_became_active":"2023-10-30T12:35:06.164711+0000","last_became_peered":"2023-10-30T12:35:06.164711+0000","last_unstale":"2023-10-30T12:35:07.159052+0000","last_undegraded":"2023-10-30T12:35:07.159052+0000","last_fullsized":"2023-10-30T12:35:07.159052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:48:18.710410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250361+0000","last_change":"2023-10-30T12:35:05.198964+0000","last_active":"2023-10-30T12:35:07.250361+0000","last_peered":"2023-10-30T12:35:07.250361+0000","last_clean":"2023-10-30T12:35:07.250361+0000","last_became_active":"2023-10-30T12:35:05.196873+0000","last_became_peered":"2023-10-30T12:35:05.196873+0000","last_unstale":"2023-10-30T12:35:07.250361+0000","last_undegraded":"2023-10-30T12:35:07.250361+0000","last_fullsized":"2023-10-30T12:35:07.250361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:47.793585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163526+0000","last_change":"2023-10-30T12:35:06.184123+0000","last_active":"2023-10-30T12:35:07.163526+0000","last_peered":"2023-10-30T12:35:07.163526+0000","last_clean":"2023-10-30T12:35:07.163526+0000","last_became_active":"2023-10-30T12:35:06.183225+0000","last_became_peered":"2023-10-30T12:35:06.183225+0000","last_unstale":"2023-10-30T12:35:07.163526+0000","last_undegraded":"2023-10-30T12:35:07.163526+0000","last_fullsized":"2023-10-30T12:35:07.163526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:03:47.097151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.543947+0000","last_change":"2023-10-30T12:35:05.177571+0000","last_active":"2023-10-30T12:35:06.543947+0000","last_peered":"2023-10-30T12:35:06.543947+0000","last_clean":"2023-10-30T12:35:06.543947+0000","last_became_active":"2023-10-30T12:35:05.177266+0000","last_became_peered":"2023-10-30T12:35:05.177266+0000","last_unstale":"2023-10-30T12:35:06.543947+0000","last_undegraded":"2023-10-30T12:35:06.543947+0000","last_fullsized":"2023-10-30T12:35:06.543947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:14.601934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250949+0000","last_change":"2023-10-30T12:35:06.206033+0000","last_active":"2023-10-30T12:35:07.250949+0000","last_peered":"2023-10-30T12:35:07.250949+0000","last_clean":"2023-10-30T12:35:07.250949+0000","last_became_active":"2023-10-30T12:35:06.196307+0000","last_became_peered":"2023-10-30T12:35:06.196307+0000","last_unstale":"2023-10-30T12:35:07.250949+0000","last_undegraded":"2023-10-30T12:35:07.250949+0000","last_fullsized":"2023-10-30T12:35:07.250949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:18:28.848259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.460771+0000","last_change":"2023-10-30T12:35:05.207594+0000","last_active":"2023-10-30T12:35:07.460771+0000","last_peered":"2023-10-30T12:35:07.460771+0000","last_clean":"2023-10-30T12:35:07.460771+0000","last_became_active":"2023-10-30T12:35:05.206930+0000","last_became_peered":"2023-10-30T12:35:05.206930+0000","last_unstale":"2023-10-30T12:35:07.460771+0000","last_undegraded":"2023-10-30T12:35:07.460771+0000","last_fullsized":"2023-10-30T12:35:07.460771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:23:46.519077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159110+0000","last_change":"2023-10-30T12:35:06.165126+0000","last_active":"2023-10-30T12:35:07.159110+0000","last_peered":"2023-10-30T12:35:07.159110+0000","last_clean":"2023-10-30T12:35:07.159110+0000","last_became_active":"2023-10-30T12:35:06.164733+0000","last_became_peered":"2023-10-30T12:35:06.164733+0000","last_unstale":"2023-10-30T12:35:07.159110+0000","last_undegraded":"2023-10-30T12:35:07.159110+0000","last_fullsized":"2023-10-30T12:35:07.159110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:29.336555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.337400+0000","last_change":"2023-10-30T12:35:05.185034+0000","last_active":"2023-10-30T12:35:27.337400+0000","last_peered":"2023-10-30T12:35:27.337400+0000","last_clean":"2023-10-30T12:35:27.337400+0000","last_became_active":"2023-10-30T12:35:05.184695+0000","last_became_peered":"2023-10-30T12:35:05.184695+0000","last_unstale":"2023-10-30T12:35:27.337400+0000","last_undegraded":"2023-10-30T12:35:27.337400+0000","last_fullsized":"2023-10-30T12:35:27.337400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:01:42.019128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247746+0000","last_change":"2023-10-30T12:35:06.204707+0000","last_active":"2023-10-30T12:35:07.247746+0000","last_peered":"2023-10-30T12:35:07.247746+0000","last_clean":"2023-10-30T12:35:07.247746+0000","last_became_active":"2023-10-30T12:35:06.204422+0000","last_became_peered":"2023-10-30T12:35:06.204422+0000","last_unstale":"2023-10-30T12:35:07.247746+0000","last_undegraded":"2023-10-30T12:35:07.247746+0000","last_fullsized":"2023-10-30T12:35:07.247746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:58:16.758575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.304904+0000","last_change":"2023-10-30T12:35:05.200871+0000","last_active":"2023-10-30T12:35:21.304904+0000","last_peered":"2023-10-30T12:35:21.304904+0000","last_clean":"2023-10-30T12:35:21.304904+0000","last_became_active":"2023-10-30T12:35:05.200449+0000","last_became_peered":"2023-10-30T12:35:05.200449+0000","last_unstale":"2023-10-30T12:35:21.304904+0000","last_undegraded":"2023-10-30T12:35:21.304904+0000","last_fullsized":"2023-10-30T12:35:21.304904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:39:20.867587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160562+0000","last_change":"2023-10-30T12:35:06.180850+0000","last_active":"2023-10-30T12:35:07.160562+0000","last_peered":"2023-10-30T12:35:07.160562+0000","last_clean":"2023-10-30T12:35:07.160562+0000","last_became_active":"2023-10-30T12:35:06.179412+0000","last_became_peered":"2023-10-30T12:35:06.179412+0000","last_unstale":"2023-10-30T12:35:07.160562+0000","last_undegraded":"2023-10-30T12:35:07.160562+0000","last_fullsized":"2023-10-30T12:35:07.160562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:47:44.055205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.304568+0000","last_change":"2023-10-30T12:35:05.207392+0000","last_active":"2023-10-30T12:35:21.304568+0000","last_peered":"2023-10-30T12:35:21.304568+0000","last_clean":"2023-10-30T12:35:21.304568+0000","last_became_active":"2023-10-30T12:35:05.206607+0000","last_became_peered":"2023-10-30T12:35:05.206607+0000","last_unstale":"2023-10-30T12:35:21.304568+0000","last_undegraded":"2023-10-30T12:35:21.304568+0000","last_fullsized":"2023-10-30T12:35:21.304568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:36.758740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163445+0000","last_change":"2023-10-30T12:35:06.182042+0000","last_active":"2023-10-30T12:35:07.163445+0000","last_peered":"2023-10-30T12:35:07.163445+0000","last_clean":"2023-10-30T12:35:07.163445+0000","last_became_active":"2023-10-30T12:35:06.181256+0000","last_became_peered":"2023-10-30T12:35:06.181256+0000","last_unstale":"2023-10-30T12:35:07.163445+0000","last_undegraded":"2023-10-30T12:35:07.163445+0000","last_fullsized":"2023-10-30T12:35:07.163445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:13:08.109392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.305680+0000","last_change":"2023-10-30T12:35:05.201024+0000","last_active":"2023-10-30T12:35:21.305680+0000","last_peered":"2023-10-30T12:35:21.305680+0000","last_clean":"2023-10-30T12:35:21.305680+0000","last_became_active":"2023-10-30T12:35:05.200724+0000","last_became_peered":"2023-10-30T12:35:05.200724+0000","last_unstale":"2023-10-30T12:35:21.305680+0000","last_undegraded":"2023-10-30T12:35:21.305680+0000","last_fullsized":"2023-10-30T12:35:21.305680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:24.746469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247249+0000","last_change":"2023-10-30T12:35:06.174636+0000","last_active":"2023-10-30T12:35:07.247249+0000","last_peered":"2023-10-30T12:35:07.247249+0000","last_clean":"2023-10-30T12:35:07.247249+0000","last_became_active":"2023-10-30T12:35:06.173996+0000","last_became_peered":"2023-10-30T12:35:06.173996+0000","last_unstale":"2023-10-30T12:35:07.247249+0000","last_undegraded":"2023-10-30T12:35:07.247249+0000","last_fullsized":"2023-10-30T12:35:07.247249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:01:38.503494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159100+0000","last_change":"2023-10-30T12:35:05.164079+0000","last_active":"2023-10-30T12:35:07.159100+0000","last_peered":"2023-10-30T12:35:07.159100+0000","last_clean":"2023-10-30T12:35:07.159100+0000","last_became_active":"2023-10-30T12:35:05.163794+0000","last_became_peered":"2023-10-30T12:35:05.163794+0000","last_unstale":"2023-10-30T12:35:07.159100+0000","last_undegraded":"2023-10-30T12:35:07.159100+0000","last_fullsized":"2023-10-30T12:35:07.159100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:19.795656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247344+0000","last_change":"2023-10-30T12:35:06.183531+0000","last_active":"2023-10-30T12:35:07.247344+0000","last_peered":"2023-10-30T12:35:07.247344+0000","last_clean":"2023-10-30T12:35:07.247344+0000","last_became_active":"2023-10-30T12:35:06.183226+0000","last_became_peered":"2023-10-30T12:35:06.183226+0000","last_unstale":"2023-10-30T12:35:07.247344+0000","last_undegraded":"2023-10-30T12:35:07.247344+0000","last_fullsized":"2023-10-30T12:35:07.247344+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:46.768356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.303852+0000","last_change":"2023-10-30T12:35:05.206995+0000","last_active":"2023-10-30T12:35:21.303852+0000","last_peered":"2023-10-30T12:35:21.303852+0000","last_clean":"2023-10-30T12:35:21.303852+0000","last_became_active":"2023-10-30T12:35:05.206518+0000","last_became_peered":"2023-10-30T12:35:05.206518+0000","last_unstale":"2023-10-30T12:35:21.303852+0000","last_undegraded":"2023-10-30T12:35:21.303852+0000","last_fullsized":"2023-10-30T12:35:21.303852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:41:33.703261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162528+0000","last_change":"2023-10-30T12:35:06.199557+0000","last_active":"2023-10-30T12:35:07.162528+0000","last_peered":"2023-10-30T12:35:07.162528+0000","last_clean":"2023-10-30T12:35:07.162528+0000","last_became_active":"2023-10-30T12:35:06.197861+0000","last_became_peered":"2023-10-30T12:35:06.197861+0000","last_unstale":"2023-10-30T12:35:07.162528+0000","last_undegraded":"2023-10-30T12:35:07.162528+0000","last_fullsized":"2023-10-30T12:35:07.162528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:30:46.092876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250766+0000","last_change":"2023-10-30T12:35:06.213279+0000","last_active":"2023-10-30T12:35:07.250766+0000","last_peered":"2023-10-30T12:35:07.250766+0000","last_clean":"2023-10-30T12:35:07.250766+0000","last_became_active":"2023-10-30T12:35:06.212548+0000","last_became_peered":"2023-10-30T12:35:06.212548+0000","last_unstale":"2023-10-30T12:35:07.250766+0000","last_undegraded":"2023-10-30T12:35:07.250766+0000","last_fullsized":"2023-10-30T12:35:07.250766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:01:57.127208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.200195+0000","last_change":"2023-10-30T12:35:05.206980+0000","last_active":"2023-10-30T12:35:11.200195+0000","last_peered":"2023-10-30T12:35:11.200195+0000","last_clean":"2023-10-30T12:35:11.200195+0000","last_became_active":"2023-10-30T12:35:05.206559+0000","last_became_peered":"2023-10-30T12:35:05.206559+0000","last_unstale":"2023-10-30T12:35:11.200195+0000","last_undegraded":"2023-10-30T12:35:11.200195+0000","last_fullsized":"2023-10-30T12:35:11.200195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:40.259622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250210+0000","last_change":"2023-10-30T12:35:06.213053+0000","last_active":"2023-10-30T12:35:07.250210+0000","last_peered":"2023-10-30T12:35:07.250210+0000","last_clean":"2023-10-30T12:35:07.250210+0000","last_became_active":"2023-10-30T12:35:06.212505+0000","last_became_peered":"2023-10-30T12:35:06.212505+0000","last_unstale":"2023-10-30T12:35:07.250210+0000","last_undegraded":"2023-10-30T12:35:07.250210+0000","last_fullsized":"2023-10-30T12:35:07.250210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:44:02.073639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.333919+0000","last_change":"2023-10-30T12:35:05.207851+0000","last_active":"2023-10-30T12:35:27.333919+0000","last_peered":"2023-10-30T12:35:27.333919+0000","last_clean":"2023-10-30T12:35:27.333919+0000","last_became_active":"2023-10-30T12:35:05.206879+0000","last_became_peered":"2023-10-30T12:35:05.206879+0000","last_unstale":"2023-10-30T12:35:27.333919+0000","last_undegraded":"2023-10-30T12:35:27.333919+0000","last_fullsized":"2023-10-30T12:35:27.333919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:43:15.717507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160637+0000","last_change":"2023-10-30T12:35:06.222373+0000","last_active":"2023-10-30T12:35:07.160637+0000","last_peered":"2023-10-30T12:35:07.160637+0000","last_clean":"2023-10-30T12:35:07.160637+0000","last_became_active":"2023-10-30T12:35:06.222094+0000","last_became_peered":"2023-10-30T12:35:06.222094+0000","last_unstale":"2023-10-30T12:35:07.160637+0000","last_undegraded":"2023-10-30T12:35:07.160637+0000","last_fullsized":"2023-10-30T12:35:07.160637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:42:32.482862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.195154+0000","last_change":"2023-10-30T12:35:05.174319+0000","last_active":"2023-10-30T12:35:11.195154+0000","last_peered":"2023-10-30T12:35:11.195154+0000","last_clean":"2023-10-30T12:35:11.195154+0000","last_became_active":"2023-10-30T12:35:05.173984+0000","last_became_peered":"2023-10-30T12:35:05.173984+0000","last_unstale":"2023-10-30T12:35:11.195154+0000","last_undegraded":"2023-10-30T12:35:11.195154+0000","last_fullsized":"2023-10-30T12:35:11.195154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:11.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247593+0000","last_change":"2023-10-30T12:35:06.198881+0000","last_active":"2023-10-30T12:35:07.247593+0000","last_peered":"2023-10-30T12:35:07.247593+0000","last_clean":"2023-10-30T12:35:07.247593+0000","last_became_active":"2023-10-30T12:35:06.198504+0000","last_became_peered":"2023-10-30T12:35:06.198504+0000","last_unstale":"2023-10-30T12:35:07.247593+0000","last_undegraded":"2023-10-30T12:35:07.247593+0000","last_fullsized":"2023-10-30T12:35:07.247593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:06:33.845147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'9","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.335834+0000","last_change":"2023-10-30T12:35:05.200926+0000","last_active":"2023-10-30T12:35:27.335834+0000","last_peered":"2023-10-30T12:35:27.335834+0000","last_clean":"2023-10-30T12:35:27.335834+0000","last_became_active":"2023-10-30T12:35:05.200623+0000","last_became_peered":"2023-10-30T12:35:05.200623+0000","last_unstale":"2023-10-30T12:35:27.335834+0000","last_undegraded":"2023-10-30T12:35:27.335834+0000","last_fullsized":"2023-10-30T12:35:27.335834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:38.942922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6553,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":13106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":839680,"data_stored":670806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":27654605},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542157,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":82227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]}]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":241664,"data_stored":79578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":79682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":669892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":80242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":249856,"data_stored":82519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":249856,"data_stored":79808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:35:39.772 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:35:39.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.946+0000 7f31151d3640 1 Processor -- start 2023-10-30T12:35:39.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.946+0000 7f31151d3640 1 -- start start 2023-10-30T12:35:39.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.946+0000 7f31151d3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f31101538a0 0x7f311015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.946+0000 7f31151d3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31100cb800 con 0x7f31100ca0e0 2023-10-30T12:35:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.946+0000 7f31151d3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311015dc30 con 0x7f31100cf160 2023-10-30T12:35:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.946+0000 7f31151d3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31100cb330 con 0x7f31101538a0 2023-10-30T12:35:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.946+0000 7f310ed9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f31101538a0 0x7f311015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.946+0000 7f310ed9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f31101538a0 0x7f311015d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57438/0 (socket says 172.21.15.12:57438) 2023-10-30T12:35:39.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.946+0000 7f310e59c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f31100ca0e0 0x7f311015a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54492/0 (socket says 172.21.15.12:54492) 2023-10-30T12:35:39.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.946+0000 7f310ed9d640 1 -- 172.21.15.12:0/386650577 learned_addr learned my addr 172.21.15.12:0/386650577 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:39.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.946+0000 7f310ed9d640 1 -- 172.21.15.12:0/386650577 >> v1:172.21.15.154:6789/0 conn(0x7f31100cf160 legacy=0x7f3110164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.946+0000 7f310ed9d640 1 -- 172.21.15.12:0/386650577 >> v1:172.21.15.12:6789/0 conn(0x7f31100ca0e0 legacy=0x7f311015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.946+0000 7f310ed9d640 1 -- 172.21.15.12:0/386650577 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31100cad70 con 0x7f31101538a0 2023-10-30T12:35:39.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.950+0000 7f310ed9d640 1 --2- 172.21.15.12:0/386650577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31101538a0 0x7f311015d6f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3100009870 tx=0x7f310001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e5a308fed12501 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:39.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.950+0000 7f310dd9b640 1 -- 172.21.15.12:0/386650577 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31000a8a30 con 0x7f31101538a0 2023-10-30T12:35:39.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.950+0000 7f310dd9b640 1 -- 172.21.15.12:0/386650577 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31000a8b90 con 0x7f31101538a0 2023-10-30T12:35:39.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.950+0000 7f310dd9b640 1 -- 172.21.15.12:0/386650577 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31000b0440 con 0x7f31101538a0 2023-10-30T12:35:39.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.950+0000 7f31151d3640 1 -- 172.21.15.12:0/386650577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31101538a0 msgr2=0x7f311015d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.950+0000 7f31151d3640 1 --2- 172.21.15.12:0/386650577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31101538a0 0x7f311015d6f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3100009870 tx=0x7f310001ed30 comp rx=0 tx=0).stop 2023-10-30T12:35:39.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.950+0000 7f31151d3640 1 -- 172.21.15.12:0/386650577 shutdown_connections 2023-10-30T12:35:39.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.950+0000 7f31151d3640 1 --2- 172.21.15.12:0/386650577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31101538a0 0x7f311015d6f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.950+0000 7f31151d3640 1 -- 172.21.15.12:0/386650577 >> 172.21.15.12:0/386650577 conn(0x7f311006bfe0 msgr2=0x7f311006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:39.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.950+0000 7f31151d3640 1 -- 172.21.15.12:0/386650577 shutdown_connections 2023-10-30T12:35:39.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.950+0000 7f31151d3640 1 -- 172.21.15.12:0/386650577 wait complete. 2023-10-30T12:35:39.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f31151d3640 1 Processor -- start 2023-10-30T12:35:39.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f31151d3640 1 -- start start 2023-10-30T12:35:39.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f31151d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31100ca0e0 0x7f31101e2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f31151d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31100cf160 0x7f3110157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f31151d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31101538a0 0x7f31100b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f31151d3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31100c9670 con 0x7f31100cf160 2023-10-30T12:35:39.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f31151d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3110158470 con 0x7f31100ca0e0 2023-10-30T12:35:39.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f31151d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31101e5340 con 0x7f31101538a0 2023-10-30T12:35:39.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f310e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31100cf160 0x7f3110157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f310e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31100cf160 0x7f3110157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57444/0 (socket says 172.21.15.12:57444) 2023-10-30T12:35:39.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f310e59c640 1 -- 172.21.15.12:0/2743085173 learned_addr learned my addr 172.21.15.12:0/2743085173 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:39.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f310ed9d640 1 --2- 172.21.15.12:0/2743085173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31100ca0e0 0x7f31101e2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f310f59e640 1 --2- 172.21.15.12:0/2743085173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31101538a0 0x7f31100b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f310e59c640 1 -- 172.21.15.12:0/2743085173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31101538a0 msgr2=0x7f31100b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f310e59c640 1 --2- 172.21.15.12:0/2743085173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31101538a0 0x7f31100b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f310e59c640 1 -- 172.21.15.12:0/2743085173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31100ca0e0 msgr2=0x7f31101e2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:39.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f310e59c640 1 --2- 172.21.15.12:0/2743085173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31100ca0e0 0x7f31101e2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:39.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f310e59c640 1 -- 172.21.15.12:0/2743085173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31000094d0 con 0x7f31100cf160 2023-10-30T12:35:39.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f310e59c640 1 --2- 172.21.15.12:0/2743085173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31100cf160 0x7f3110157f30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f310400c460 tx=0x7f310400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:39.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f30f77fe640 1 -- 172.21.15.12:0/2743085173 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104007480 con 0x7f31100cf160 2023-10-30T12:35:39.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f30f77fe640 1 -- 172.21.15.12:0/2743085173 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31040075e0 con 0x7f31100cf160 2023-10-30T12:35:39.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f30f77fe640 1 -- 172.21.15.12:0/2743085173 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104004a10 con 0x7f31100cf160 2023-10-30T12:35:39.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f31151d3640 1 -- 172.21.15.12:0/2743085173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31101e4eb0 con 0x7f31100cf160 2023-10-30T12:35:39.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.954+0000 7f31151d3640 1 -- 172.21.15.12:0/2743085173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31101e42a0 con 0x7f31100cf160 2023-10-30T12:35:39.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.958+0000 7f31151d3640 1 -- 172.21.15.12:0/2743085173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31101e5040 con 0x7f31100cf160 2023-10-30T12:35:39.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.958+0000 7f30f77fe640 1 -- 172.21.15.12:0/2743085173 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3104004b70 con 0x7f31100cf160 2023-10-30T12:35:39.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.958+0000 7f30f77fe640 1 --2- 172.21.15.12:0/2743085173 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f30fc0664a0 0x7f30fc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:39.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.958+0000 7f30f77fe640 1 -- 172.21.15.12:0/2743085173 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3104083060 con 0x7f31100cf160 2023-10-30T12:35:39.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.958+0000 7f310ed9d640 1 --2- 172.21.15.12:0/2743085173 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f30fc0664a0 0x7f30fc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:39.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.958+0000 7f310ed9d640 1 --2- 172.21.15.12:0/2743085173 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f30fc0664a0 0x7f30fc068960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f31000b2000 tx=0x7f3100009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:39.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:39.962+0000 7f30f77fe640 1 -- 172.21.15.12:0/2743085173 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f31040b4090 con 0x7f31100cf160 2023-10-30T12:35:40.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.114+0000 7f31151d3640 1 -- 172.21.15.12:0/2743085173 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f31100cb050 con 0x7f30fc0664a0 2023-10-30T12:35:40.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.134+0000 7f30f77fe640 1 -- 172.21.15.12:0/2743085173 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264700 (secure 0 0 0) 0x7f31000b0620 con 0x7f30fc0664a0 2023-10-30T12:35:40.141 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:40.142 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-10-30T12:35:40.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.138+0000 7f31151d3640 1 -- 172.21.15.12:0/2743085173 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f30fc0664a0 msgr2=0x7f30fc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:40.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.138+0000 7f31151d3640 1 --2- 172.21.15.12:0/2743085173 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f30fc0664a0 0x7f30fc068960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f31000b2000 tx=0x7f3100009040 comp rx=0 tx=0).stop 2023-10-30T12:35:40.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.138+0000 7f31151d3640 1 -- 172.21.15.12:0/2743085173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31100cf160 msgr2=0x7f3110157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:40.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.138+0000 7f31151d3640 1 --2- 172.21.15.12:0/2743085173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31100cf160 0x7f3110157f30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f310400c460 tx=0x7f310400c920 comp rx=0 tx=0).stop 2023-10-30T12:35:40.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.138+0000 7f31151d3640 1 -- 172.21.15.12:0/2743085173 shutdown_connections 2023-10-30T12:35:40.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.138+0000 7f31151d3640 1 --2- 172.21.15.12:0/2743085173 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f30fc0664a0 0x7f30fc068960 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.138+0000 7f31151d3640 1 --2- 172.21.15.12:0/2743085173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31101538a0 0x7f31100b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.138+0000 7f31151d3640 1 --2- 172.21.15.12:0/2743085173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31100cf160 0x7f3110157f30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.138+0000 7f31151d3640 1 --2- 172.21.15.12:0/2743085173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31100ca0e0 0x7f31101e2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.138+0000 7f31151d3640 1 -- 172.21.15.12:0/2743085173 >> 172.21.15.12:0/2743085173 conn(0x7f311006bfe0 msgr2=0x7f31100c54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:40.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.138+0000 7f31151d3640 1 -- 172.21.15.12:0/2743085173 shutdown_connections 2023-10-30T12:35:40.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.138+0000 7f31151d3640 1 -- 172.21.15.12:0/2743085173 wait complete. 2023-10-30T12:35:40.161 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-10-30T12:35:38.947034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596921,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":288,"num_write_kb":2458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219208,"kb_used_data":3080,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755512,"statfs":{"total":773094113280,"available":772869644288,"internally_reserved":0,"allocated":3153920,"data_stored":1824754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12729,"internal_metadata":221236807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2737,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004266"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250527+0000","last_change":"2023-10-30T12:35:06.223711+0000","last_active":"2023-10-30T12:35:07.250527+0000","last_peered":"2023-10-30T12:35:07.250527+0000","last_clean":"2023-10-30T12:35:07.250527+0000","last_became_active":"2023-10-30T12:35:06.222724+0000","last_became_peered":"2023-10-30T12:35:06.222724+0000","last_unstale":"2023-10-30T12:35:07.250527+0000","last_undegraded":"2023-10-30T12:35:07.250527+0000","last_fullsized":"2023-10-30T12:35:07.250527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:34:35.217274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159376+0000","last_change":"2023-10-30T12:35:06.182748+0000","last_active":"2023-10-30T12:35:07.159376+0000","last_peered":"2023-10-30T12:35:07.159376+0000","last_clean":"2023-10-30T12:35:07.159376+0000","last_became_active":"2023-10-30T12:35:06.182441+0000","last_became_peered":"2023-10-30T12:35:06.182441+0000","last_unstale":"2023-10-30T12:35:07.159376+0000","last_undegraded":"2023-10-30T12:35:07.159376+0000","last_fullsized":"2023-10-30T12:35:07.159376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:26:28.943768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160588+0000","last_change":"2023-10-30T12:35:06.180426+0000","last_active":"2023-10-30T12:35:07.160588+0000","last_peered":"2023-10-30T12:35:07.160588+0000","last_clean":"2023-10-30T12:35:07.160588+0000","last_became_active":"2023-10-30T12:35:06.180132+0000","last_became_peered":"2023-10-30T12:35:06.180132+0000","last_unstale":"2023-10-30T12:35:07.160588+0000","last_undegraded":"2023-10-30T12:35:07.160588+0000","last_fullsized":"2023-10-30T12:35:07.160588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:33:22.216739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250158+0000","last_change":"2023-10-30T12:35:06.217986+0000","last_active":"2023-10-30T12:35:07.250158+0000","last_peered":"2023-10-30T12:35:07.250158+0000","last_clean":"2023-10-30T12:35:07.250158+0000","last_became_active":"2023-10-30T12:35:06.213987+0000","last_became_peered":"2023-10-30T12:35:06.213987+0000","last_unstale":"2023-10-30T12:35:07.250158+0000","last_undegraded":"2023-10-30T12:35:07.250158+0000","last_fullsized":"2023-10-30T12:35:07.250158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:51:20.515999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249406+0000","last_change":"2023-10-30T12:35:06.216098+0000","last_active":"2023-10-30T12:35:07.249406+0000","last_peered":"2023-10-30T12:35:07.249406+0000","last_clean":"2023-10-30T12:35:07.249406+0000","last_became_active":"2023-10-30T12:35:06.214964+0000","last_became_peered":"2023-10-30T12:35:06.214964+0000","last_unstale":"2023-10-30T12:35:07.249406+0000","last_undegraded":"2023-10-30T12:35:07.249406+0000","last_fullsized":"2023-10-30T12:35:07.249406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:37:43.156964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249861+0000","last_change":"2023-10-30T12:35:06.217608+0000","last_active":"2023-10-30T12:35:07.249861+0000","last_peered":"2023-10-30T12:35:07.249861+0000","last_clean":"2023-10-30T12:35:07.249861+0000","last_became_active":"2023-10-30T12:35:06.214219+0000","last_became_peered":"2023-10-30T12:35:06.214219+0000","last_unstale":"2023-10-30T12:35:07.249861+0000","last_undegraded":"2023-10-30T12:35:07.249861+0000","last_fullsized":"2023-10-30T12:35:07.249861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:35:27.431813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249000+0000","last_change":"2023-10-30T12:35:06.217120+0000","last_active":"2023-10-30T12:35:07.249000+0000","last_peered":"2023-10-30T12:35:07.249000+0000","last_clean":"2023-10-30T12:35:07.249000+0000","last_became_active":"2023-10-30T12:35:06.216468+0000","last_became_peered":"2023-10-30T12:35:06.216468+0000","last_unstale":"2023-10-30T12:35:07.249000+0000","last_undegraded":"2023-10-30T12:35:07.249000+0000","last_fullsized":"2023-10-30T12:35:07.249000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:26:07.313967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246707+0000","last_change":"2023-10-30T12:35:06.214767+0000","last_active":"2023-10-30T12:35:07.246707+0000","last_peered":"2023-10-30T12:35:07.246707+0000","last_clean":"2023-10-30T12:35:07.246707+0000","last_became_active":"2023-10-30T12:35:06.214384+0000","last_became_peered":"2023-10-30T12:35:06.214384+0000","last_unstale":"2023-10-30T12:35:07.246707+0000","last_undegraded":"2023-10-30T12:35:07.246707+0000","last_fullsized":"2023-10-30T12:35:07.246707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:29.866538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250843+0000","last_change":"2023-10-30T12:35:06.223503+0000","last_active":"2023-10-30T12:35:07.250843+0000","last_peered":"2023-10-30T12:35:07.250843+0000","last_clean":"2023-10-30T12:35:07.250843+0000","last_became_active":"2023-10-30T12:35:06.222692+0000","last_became_peered":"2023-10-30T12:35:06.222692+0000","last_unstale":"2023-10-30T12:35:07.250843+0000","last_undegraded":"2023-10-30T12:35:07.250843+0000","last_fullsized":"2023-10-30T12:35:07.250843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:31:36.926720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163304+0000","last_change":"2023-10-30T12:35:06.215938+0000","last_active":"2023-10-30T12:35:07.163304+0000","last_peered":"2023-10-30T12:35:07.163304+0000","last_clean":"2023-10-30T12:35:07.163304+0000","last_became_active":"2023-10-30T12:35:06.215384+0000","last_became_peered":"2023-10-30T12:35:06.215384+0000","last_unstale":"2023-10-30T12:35:07.163304+0000","last_undegraded":"2023-10-30T12:35:07.163304+0000","last_fullsized":"2023-10-30T12:35:07.163304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:46:21.253555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162425+0000","last_change":"2023-10-30T12:35:06.215965+0000","last_active":"2023-10-30T12:35:07.162425+0000","last_peered":"2023-10-30T12:35:07.162425+0000","last_clean":"2023-10-30T12:35:07.162425+0000","last_became_active":"2023-10-30T12:35:06.215355+0000","last_became_peered":"2023-10-30T12:35:06.215355+0000","last_unstale":"2023-10-30T12:35:07.162425+0000","last_undegraded":"2023-10-30T12:35:07.162425+0000","last_fullsized":"2023-10-30T12:35:07.162425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:07.929526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164185+0000","last_change":"2023-10-30T12:35:06.197278+0000","last_active":"2023-10-30T12:35:07.164185+0000","last_peered":"2023-10-30T12:35:07.164185+0000","last_clean":"2023-10-30T12:35:07.164185+0000","last_became_active":"2023-10-30T12:35:06.196503+0000","last_became_peered":"2023-10-30T12:35:06.196503+0000","last_unstale":"2023-10-30T12:35:07.164185+0000","last_undegraded":"2023-10-30T12:35:07.164185+0000","last_fullsized":"2023-10-30T12:35:07.164185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:06.928264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247199+0000","last_change":"2023-10-30T12:35:06.195721+0000","last_active":"2023-10-30T12:35:07.247199+0000","last_peered":"2023-10-30T12:35:07.247199+0000","last_clean":"2023-10-30T12:35:07.247199+0000","last_became_active":"2023-10-30T12:35:06.195408+0000","last_became_peered":"2023-10-30T12:35:06.195408+0000","last_unstale":"2023-10-30T12:35:07.247199+0000","last_undegraded":"2023-10-30T12:35:07.247199+0000","last_fullsized":"2023-10-30T12:35:07.247199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:59:02.872646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160258+0000","last_change":"2023-10-30T12:35:06.182939+0000","last_active":"2023-10-30T12:35:07.160258+0000","last_peered":"2023-10-30T12:35:07.160258+0000","last_clean":"2023-10-30T12:35:07.160258+0000","last_became_active":"2023-10-30T12:35:06.182573+0000","last_became_peered":"2023-10-30T12:35:06.182573+0000","last_unstale":"2023-10-30T12:35:07.160258+0000","last_undegraded":"2023-10-30T12:35:07.160258+0000","last_fullsized":"2023-10-30T12:35:07.160258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:50.345142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.158834+0000","last_change":"2023-10-30T12:35:06.185636+0000","last_active":"2023-10-30T12:35:07.158834+0000","last_peered":"2023-10-30T12:35:07.158834+0000","last_clean":"2023-10-30T12:35:07.158834+0000","last_became_active":"2023-10-30T12:35:06.185290+0000","last_became_peered":"2023-10-30T12:35:06.185290+0000","last_unstale":"2023-10-30T12:35:07.158834+0000","last_undegraded":"2023-10-30T12:35:07.158834+0000","last_fullsized":"2023-10-30T12:35:07.158834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:26:44.783714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.248972+0000","last_change":"2023-10-30T12:35:06.214488+0000","last_active":"2023-10-30T12:35:07.248972+0000","last_peered":"2023-10-30T12:35:07.248972+0000","last_clean":"2023-10-30T12:35:07.248972+0000","last_became_active":"2023-10-30T12:35:06.213627+0000","last_became_peered":"2023-10-30T12:35:06.213627+0000","last_unstale":"2023-10-30T12:35:07.248972+0000","last_undegraded":"2023-10-30T12:35:07.248972+0000","last_fullsized":"2023-10-30T12:35:07.248972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:22:47.148407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251739+0000","last_change":"2023-10-30T12:35:06.224903+0000","last_active":"2023-10-30T12:35:07.251739+0000","last_peered":"2023-10-30T12:35:07.251739+0000","last_clean":"2023-10-30T12:35:07.251739+0000","last_became_active":"2023-10-30T12:35:06.221696+0000","last_became_peered":"2023-10-30T12:35:06.221696+0000","last_unstale":"2023-10-30T12:35:07.251739+0000","last_undegraded":"2023-10-30T12:35:07.251739+0000","last_fullsized":"2023-10-30T12:35:07.251739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:31:10.439162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246945+0000","last_change":"2023-10-30T12:35:06.198303+0000","last_active":"2023-10-30T12:35:07.246945+0000","last_peered":"2023-10-30T12:35:07.246945+0000","last_clean":"2023-10-30T12:35:07.246945+0000","last_became_active":"2023-10-30T12:35:06.198074+0000","last_became_peered":"2023-10-30T12:35:06.198074+0000","last_unstale":"2023-10-30T12:35:07.246945+0000","last_undegraded":"2023-10-30T12:35:07.246945+0000","last_fullsized":"2023-10-30T12:35:07.246945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:21:36.079586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250503+0000","last_change":"2023-10-30T12:35:06.218556+0000","last_active":"2023-10-30T12:35:07.250503+0000","last_peered":"2023-10-30T12:35:07.250503+0000","last_clean":"2023-10-30T12:35:07.250503+0000","last_became_active":"2023-10-30T12:35:06.215935+0000","last_became_peered":"2023-10-30T12:35:06.215935+0000","last_unstale":"2023-10-30T12:35:07.250503+0000","last_undegraded":"2023-10-30T12:35:07.250503+0000","last_fullsized":"2023-10-30T12:35:07.250503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:36:31.084542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162438+0000","last_change":"2023-10-30T12:35:06.201302+0000","last_active":"2023-10-30T12:35:07.162438+0000","last_peered":"2023-10-30T12:35:07.162438+0000","last_clean":"2023-10-30T12:35:07.162438+0000","last_became_active":"2023-10-30T12:35:06.199812+0000","last_became_peered":"2023-10-30T12:35:06.199812+0000","last_unstale":"2023-10-30T12:35:07.162438+0000","last_undegraded":"2023-10-30T12:35:07.162438+0000","last_fullsized":"2023-10-30T12:35:07.162438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:48.803738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160558+0000","last_change":"2023-10-30T12:35:06.217139+0000","last_active":"2023-10-30T12:35:07.160558+0000","last_peered":"2023-10-30T12:35:07.160558+0000","last_clean":"2023-10-30T12:35:07.160558+0000","last_became_active":"2023-10-30T12:35:06.216861+0000","last_became_peered":"2023-10-30T12:35:06.216861+0000","last_unstale":"2023-10-30T12:35:07.160558+0000","last_undegraded":"2023-10-30T12:35:07.160558+0000","last_fullsized":"2023-10-30T12:35:07.160558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:05:22.193180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249973+0000","last_change":"2023-10-30T12:35:06.225140+0000","last_active":"2023-10-30T12:35:07.249973+0000","last_peered":"2023-10-30T12:35:07.249973+0000","last_clean":"2023-10-30T12:35:07.249973+0000","last_became_active":"2023-10-30T12:35:06.221841+0000","last_became_peered":"2023-10-30T12:35:06.221841+0000","last_unstale":"2023-10-30T12:35:07.249973+0000","last_undegraded":"2023-10-30T12:35:07.249973+0000","last_fullsized":"2023-10-30T12:35:07.249973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:51:57.131025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247032+0000","last_change":"2023-10-30T12:35:06.182662+0000","last_active":"2023-10-30T12:35:07.247032+0000","last_peered":"2023-10-30T12:35:07.247032+0000","last_clean":"2023-10-30T12:35:07.247032+0000","last_became_active":"2023-10-30T12:35:06.182314+0000","last_became_peered":"2023-10-30T12:35:06.182314+0000","last_unstale":"2023-10-30T12:35:07.247032+0000","last_undegraded":"2023-10-30T12:35:07.247032+0000","last_fullsized":"2023-10-30T12:35:07.247032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:41:49.587856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249533+0000","last_change":"2023-10-30T12:35:06.214113+0000","last_active":"2023-10-30T12:35:07.249533+0000","last_peered":"2023-10-30T12:35:07.249533+0000","last_clean":"2023-10-30T12:35:07.249533+0000","last_became_active":"2023-10-30T12:35:06.213402+0000","last_became_peered":"2023-10-30T12:35:06.213402+0000","last_unstale":"2023-10-30T12:35:07.249533+0000","last_undegraded":"2023-10-30T12:35:07.249533+0000","last_fullsized":"2023-10-30T12:35:07.249533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:14:58.409296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247781+0000","last_change":"2023-10-30T12:35:06.188570+0000","last_active":"2023-10-30T12:35:07.247781+0000","last_peered":"2023-10-30T12:35:07.247781+0000","last_clean":"2023-10-30T12:35:07.247781+0000","last_became_active":"2023-10-30T12:35:06.188298+0000","last_became_peered":"2023-10-30T12:35:06.188298+0000","last_unstale":"2023-10-30T12:35:07.247781+0000","last_undegraded":"2023-10-30T12:35:07.247781+0000","last_fullsized":"2023-10-30T12:35:07.247781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:28.281254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163623+0000","last_change":"2023-10-30T12:35:06.208821+0000","last_active":"2023-10-30T12:35:07.163623+0000","last_peered":"2023-10-30T12:35:07.163623+0000","last_clean":"2023-10-30T12:35:07.163623+0000","last_became_active":"2023-10-30T12:35:06.202507+0000","last_became_peered":"2023-10-30T12:35:06.202507+0000","last_unstale":"2023-10-30T12:35:07.163623+0000","last_undegraded":"2023-10-30T12:35:07.163623+0000","last_fullsized":"2023-10-30T12:35:07.163623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:40:13.124316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159171+0000","last_change":"2023-10-30T12:35:06.170313+0000","last_active":"2023-10-30T12:35:07.159171+0000","last_peered":"2023-10-30T12:35:07.159171+0000","last_clean":"2023-10-30T12:35:07.159171+0000","last_became_active":"2023-10-30T12:35:06.170030+0000","last_became_peered":"2023-10-30T12:35:06.170030+0000","last_unstale":"2023-10-30T12:35:07.159171+0000","last_undegraded":"2023-10-30T12:35:07.159171+0000","last_fullsized":"2023-10-30T12:35:07.159171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:49:59.706208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247487+0000","last_change":"2023-10-30T12:35:06.189448+0000","last_active":"2023-10-30T12:35:07.247487+0000","last_peered":"2023-10-30T12:35:07.247487+0000","last_clean":"2023-10-30T12:35:07.247487+0000","last_became_active":"2023-10-30T12:35:06.189171+0000","last_became_peered":"2023-10-30T12:35:06.189171+0000","last_unstale":"2023-10-30T12:35:07.247487+0000","last_undegraded":"2023-10-30T12:35:07.247487+0000","last_fullsized":"2023-10-30T12:35:07.247487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:18:51.463273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160658+0000","last_change":"2023-10-30T12:35:06.183680+0000","last_active":"2023-10-30T12:35:07.160658+0000","last_peered":"2023-10-30T12:35:07.160658+0000","last_clean":"2023-10-30T12:35:07.160658+0000","last_became_active":"2023-10-30T12:35:06.183348+0000","last_became_peered":"2023-10-30T12:35:06.183348+0000","last_unstale":"2023-10-30T12:35:07.160658+0000","last_undegraded":"2023-10-30T12:35:07.160658+0000","last_fullsized":"2023-10-30T12:35:07.160658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:54:25.429897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250238+0000","last_change":"2023-10-30T12:35:06.206675+0000","last_active":"2023-10-30T12:35:07.250238+0000","last_peered":"2023-10-30T12:35:07.250238+0000","last_clean":"2023-10-30T12:35:07.250238+0000","last_became_active":"2023-10-30T12:35:06.196368+0000","last_became_peered":"2023-10-30T12:35:06.196368+0000","last_unstale":"2023-10-30T12:35:07.250238+0000","last_undegraded":"2023-10-30T12:35:07.250238+0000","last_fullsized":"2023-10-30T12:35:07.250238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:35.929311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246681+0000","last_change":"2023-10-30T12:35:06.214408+0000","last_active":"2023-10-30T12:35:07.246681+0000","last_peered":"2023-10-30T12:35:07.246681+0000","last_clean":"2023-10-30T12:35:07.246681+0000","last_became_active":"2023-10-30T12:35:06.213994+0000","last_became_peered":"2023-10-30T12:35:06.213994+0000","last_unstale":"2023-10-30T12:35:07.246681+0000","last_undegraded":"2023-10-30T12:35:07.246681+0000","last_fullsized":"2023-10-30T12:35:07.246681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:05:58.830698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162027+0000","last_change":"2023-10-30T12:35:06.198444+0000","last_active":"2023-10-30T12:35:07.162027+0000","last_peered":"2023-10-30T12:35:07.162027+0000","last_clean":"2023-10-30T12:35:07.162027+0000","last_became_active":"2023-10-30T12:35:06.197647+0000","last_became_peered":"2023-10-30T12:35:06.197647+0000","last_unstale":"2023-10-30T12:35:07.162027+0000","last_undegraded":"2023-10-30T12:35:07.162027+0000","last_fullsized":"2023-10-30T12:35:07.162027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:28:56.075420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163077+0000","last_change":"2023-10-30T12:35:06.168377+0000","last_active":"2023-10-30T12:35:07.163077+0000","last_peered":"2023-10-30T12:35:07.163077+0000","last_clean":"2023-10-30T12:35:07.163077+0000","last_became_active":"2023-10-30T12:35:06.167558+0000","last_became_peered":"2023-10-30T12:35:06.167558+0000","last_unstale":"2023-10-30T12:35:07.163077+0000","last_undegraded":"2023-10-30T12:35:07.163077+0000","last_fullsized":"2023-10-30T12:35:07.163077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:57:28.009164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.310153+0000","last_change":"2023-10-30T12:35:05.200844+0000","last_active":"2023-10-30T12:35:21.310153+0000","last_peered":"2023-10-30T12:35:21.310153+0000","last_clean":"2023-10-30T12:35:21.310153+0000","last_became_active":"2023-10-30T12:35:05.200169+0000","last_became_peered":"2023-10-30T12:35:05.200169+0000","last_unstale":"2023-10-30T12:35:21.310153+0000","last_undegraded":"2023-10-30T12:35:21.310153+0000","last_fullsized":"2023-10-30T12:35:21.310153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:07:19.571357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160718+0000","last_change":"2023-10-30T12:35:06.194964+0000","last_active":"2023-10-30T12:35:07.160718+0000","last_peered":"2023-10-30T12:35:07.160718+0000","last_clean":"2023-10-30T12:35:07.160718+0000","last_became_active":"2023-10-30T12:35:06.194689+0000","last_became_peered":"2023-10-30T12:35:06.194689+0000","last_unstale":"2023-10-30T12:35:07.160718+0000","last_undegraded":"2023-10-30T12:35:07.160718+0000","last_fullsized":"2023-10-30T12:35:07.160718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:30:53.410341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.336271+0000","last_change":"2023-10-30T12:35:05.183569+0000","last_active":"2023-10-30T12:35:27.336271+0000","last_peered":"2023-10-30T12:35:27.336271+0000","last_clean":"2023-10-30T12:35:27.336271+0000","last_became_active":"2023-10-30T12:35:05.182761+0000","last_became_peered":"2023-10-30T12:35:05.182761+0000","last_unstale":"2023-10-30T12:35:27.336271+0000","last_undegraded":"2023-10-30T12:35:27.336271+0000","last_fullsized":"2023-10-30T12:35:27.336271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:49:44.311814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.536230+0000","last_change":"2023-10-30T12:35:05.181278+0000","last_active":"2023-10-30T12:35:27.536230+0000","last_peered":"2023-10-30T12:35:27.536230+0000","last_clean":"2023-10-30T12:35:27.536230+0000","last_became_active":"2023-10-30T12:35:05.180730+0000","last_became_peered":"2023-10-30T12:35:05.180730+0000","last_unstale":"2023-10-30T12:35:27.536230+0000","last_undegraded":"2023-10-30T12:35:27.536230+0000","last_fullsized":"2023-10-30T12:35:27.536230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:07:10.828707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246964+0000","last_change":"2023-10-30T12:35:06.180049+0000","last_active":"2023-10-30T12:35:07.246964+0000","last_peered":"2023-10-30T12:35:07.246964+0000","last_clean":"2023-10-30T12:35:07.246964+0000","last_became_active":"2023-10-30T12:35:06.179764+0000","last_became_peered":"2023-10-30T12:35:06.179764+0000","last_unstale":"2023-10-30T12:35:07.246964+0000","last_undegraded":"2023-10-30T12:35:07.246964+0000","last_fullsized":"2023-10-30T12:35:07.246964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:33.136727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249880+0000","last_change":"2023-10-30T12:35:05.202539+0000","last_active":"2023-10-30T12:35:07.249880+0000","last_peered":"2023-10-30T12:35:07.249880+0000","last_clean":"2023-10-30T12:35:07.249880+0000","last_became_active":"2023-10-30T12:35:05.201853+0000","last_became_peered":"2023-10-30T12:35:05.201853+0000","last_unstale":"2023-10-30T12:35:07.249880+0000","last_undegraded":"2023-10-30T12:35:07.249880+0000","last_fullsized":"2023-10-30T12:35:07.249880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:33:33.151433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.158869+0000","last_change":"2023-10-30T12:35:06.173567+0000","last_active":"2023-10-30T12:35:07.158869+0000","last_peered":"2023-10-30T12:35:07.158869+0000","last_clean":"2023-10-30T12:35:07.158869+0000","last_became_active":"2023-10-30T12:35:06.170179+0000","last_became_peered":"2023-10-30T12:35:06.170179+0000","last_unstale":"2023-10-30T12:35:07.158869+0000","last_undegraded":"2023-10-30T12:35:07.158869+0000","last_fullsized":"2023-10-30T12:35:07.158869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:39:07.880510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.306226+0000","last_change":"2023-10-30T12:35:05.198546+0000","last_active":"2023-10-30T12:35:21.306226+0000","last_peered":"2023-10-30T12:35:21.306226+0000","last_clean":"2023-10-30T12:35:21.306226+0000","last_became_active":"2023-10-30T12:35:05.197606+0000","last_became_peered":"2023-10-30T12:35:05.197606+0000","last_unstale":"2023-10-30T12:35:21.306226+0000","last_undegraded":"2023-10-30T12:35:21.306226+0000","last_fullsized":"2023-10-30T12:35:21.306226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:26:57.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251322+0000","last_change":"2023-10-30T12:35:06.226044+0000","last_active":"2023-10-30T12:35:07.251322+0000","last_peered":"2023-10-30T12:35:07.251322+0000","last_clean":"2023-10-30T12:35:07.251322+0000","last_became_active":"2023-10-30T12:35:06.225392+0000","last_became_peered":"2023-10-30T12:35:06.225392+0000","last_unstale":"2023-10-30T12:35:07.251322+0000","last_undegraded":"2023-10-30T12:35:07.251322+0000","last_fullsized":"2023-10-30T12:35:07.251322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:13:10.796549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.313047+0000","last_change":"2023-10-30T12:35:05.197614+0000","last_active":"2023-10-30T12:35:21.313047+0000","last_peered":"2023-10-30T12:35:21.313047+0000","last_clean":"2023-10-30T12:35:21.313047+0000","last_became_active":"2023-10-30T12:35:05.196899+0000","last_became_peered":"2023-10-30T12:35:05.196899+0000","last_unstale":"2023-10-30T12:35:21.313047+0000","last_undegraded":"2023-10-30T12:35:21.313047+0000","last_fullsized":"2023-10-30T12:35:21.313047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:21:00.152426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160715+0000","last_change":"2023-10-30T12:35:06.222344+0000","last_active":"2023-10-30T12:35:07.160715+0000","last_peered":"2023-10-30T12:35:07.160715+0000","last_clean":"2023-10-30T12:35:07.160715+0000","last_became_active":"2023-10-30T12:35:06.222057+0000","last_became_peered":"2023-10-30T12:35:06.222057+0000","last_unstale":"2023-10-30T12:35:07.160715+0000","last_undegraded":"2023-10-30T12:35:07.160715+0000","last_fullsized":"2023-10-30T12:35:07.160715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:50:46.691223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.201054+0000","last_change":"2023-10-30T12:35:05.197501+0000","last_active":"2023-10-30T12:35:11.201054+0000","last_peered":"2023-10-30T12:35:11.201054+0000","last_clean":"2023-10-30T12:35:11.201054+0000","last_became_active":"2023-10-30T12:35:05.197190+0000","last_became_peered":"2023-10-30T12:35:05.197190+0000","last_unstale":"2023-10-30T12:35:11.201054+0000","last_undegraded":"2023-10-30T12:35:11.201054+0000","last_fullsized":"2023-10-30T12:35:11.201054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:53:31.121721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164505+0000","last_change":"2023-10-30T12:35:06.204885+0000","last_active":"2023-10-30T12:35:07.164505+0000","last_peered":"2023-10-30T12:35:07.164505+0000","last_clean":"2023-10-30T12:35:07.164505+0000","last_became_active":"2023-10-30T12:35:06.204122+0000","last_became_peered":"2023-10-30T12:35:06.204122+0000","last_unstale":"2023-10-30T12:35:07.164505+0000","last_undegraded":"2023-10-30T12:35:07.164505+0000","last_fullsized":"2023-10-30T12:35:07.164505+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:27:18.433575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'11","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:35.466455+0000","last_change":"2023-10-30T12:35:05.180005+0000","last_active":"2023-10-30T12:35:35.466455+0000","last_peered":"2023-10-30T12:35:35.466455+0000","last_clean":"2023-10-30T12:35:35.466455+0000","last_became_active":"2023-10-30T12:35:05.179547+0000","last_became_peered":"2023-10-30T12:35:05.179547+0000","last_unstale":"2023-10-30T12:35:35.466455+0000","last_undegraded":"2023-10-30T12:35:35.466455+0000","last_fullsized":"2023-10-30T12:35:35.466455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:13:03.137433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2685,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249565+0000","last_change":"2023-10-30T12:35:06.216619+0000","last_active":"2023-10-30T12:35:07.249565+0000","last_peered":"2023-10-30T12:35:07.249565+0000","last_clean":"2023-10-30T12:35:07.249565+0000","last_became_active":"2023-10-30T12:35:06.215128+0000","last_became_peered":"2023-10-30T12:35:06.215128+0000","last_unstale":"2023-10-30T12:35:07.249565+0000","last_undegraded":"2023-10-30T12:35:07.249565+0000","last_fullsized":"2023-10-30T12:35:07.249565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:48:45.673159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.335411+0000","last_change":"2023-10-30T12:35:05.190041+0000","last_active":"2023-10-30T12:35:27.335411+0000","last_peered":"2023-10-30T12:35:27.335411+0000","last_clean":"2023-10-30T12:35:27.335411+0000","last_became_active":"2023-10-30T12:35:05.189573+0000","last_became_peered":"2023-10-30T12:35:05.189573+0000","last_unstale":"2023-10-30T12:35:27.335411+0000","last_undegraded":"2023-10-30T12:35:27.335411+0000","last_fullsized":"2023-10-30T12:35:27.335411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:02:57.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160412+0000","last_change":"2023-10-30T12:35:06.217334+0000","last_active":"2023-10-30T12:35:07.160412+0000","last_peered":"2023-10-30T12:35:07.160412+0000","last_clean":"2023-10-30T12:35:07.160412+0000","last_became_active":"2023-10-30T12:35:06.216885+0000","last_became_peered":"2023-10-30T12:35:06.216885+0000","last_unstale":"2023-10-30T12:35:07.160412+0000","last_undegraded":"2023-10-30T12:35:07.160412+0000","last_fullsized":"2023-10-30T12:35:07.160412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:38:26.745898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.313580+0000","last_change":"2023-10-30T12:35:05.203198+0000","last_active":"2023-10-30T12:35:21.313580+0000","last_peered":"2023-10-30T12:35:21.313580+0000","last_clean":"2023-10-30T12:35:21.313580+0000","last_became_active":"2023-10-30T12:35:05.202612+0000","last_became_peered":"2023-10-30T12:35:05.202612+0000","last_unstale":"2023-10-30T12:35:21.313580+0000","last_undegraded":"2023-10-30T12:35:21.313580+0000","last_fullsized":"2023-10-30T12:35:21.313580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:55.986236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249574+0000","last_change":"2023-10-30T12:35:06.216307+0000","last_active":"2023-10-30T12:35:07.249574+0000","last_peered":"2023-10-30T12:35:07.249574+0000","last_clean":"2023-10-30T12:35:07.249574+0000","last_became_active":"2023-10-30T12:35:06.215553+0000","last_became_peered":"2023-10-30T12:35:06.215553+0000","last_unstale":"2023-10-30T12:35:07.249574+0000","last_undegraded":"2023-10-30T12:35:07.249574+0000","last_fullsized":"2023-10-30T12:35:07.249574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:46:25.396759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.336548+0000","last_change":"2023-10-30T12:35:05.181882+0000","last_active":"2023-10-30T12:35:27.336548+0000","last_peered":"2023-10-30T12:35:27.336548+0000","last_clean":"2023-10-30T12:35:27.336548+0000","last_became_active":"2023-10-30T12:35:05.181115+0000","last_became_peered":"2023-10-30T12:35:05.181115+0000","last_unstale":"2023-10-30T12:35:27.336548+0000","last_undegraded":"2023-10-30T12:35:27.336548+0000","last_fullsized":"2023-10-30T12:35:27.336548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:18:55.496940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162013+0000","last_change":"2023-10-30T12:35:06.198990+0000","last_active":"2023-10-30T12:35:07.162013+0000","last_peered":"2023-10-30T12:35:07.162013+0000","last_clean":"2023-10-30T12:35:07.162013+0000","last_became_active":"2023-10-30T12:35:06.197722+0000","last_became_peered":"2023-10-30T12:35:06.197722+0000","last_unstale":"2023-10-30T12:35:07.162013+0000","last_undegraded":"2023-10-30T12:35:07.162013+0000","last_fullsized":"2023-10-30T12:35:07.162013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:56:51.757976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'15","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.338386+0000","last_change":"2023-10-30T12:35:05.169455+0000","last_active":"2023-10-30T12:35:27.338386+0000","last_peered":"2023-10-30T12:35:27.338386+0000","last_clean":"2023-10-30T12:35:27.338386+0000","last_became_active":"2023-10-30T12:35:05.169130+0000","last_became_peered":"2023-10-30T12:35:05.169130+0000","last_unstale":"2023-10-30T12:35:27.338386+0000","last_undegraded":"2023-10-30T12:35:27.338386+0000","last_fullsized":"2023-10-30T12:35:27.338386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:59:14.096315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249183+0000","last_change":"2023-10-30T12:35:06.215545+0000","last_active":"2023-10-30T12:35:07.249183+0000","last_peered":"2023-10-30T12:35:07.249183+0000","last_clean":"2023-10-30T12:35:07.249183+0000","last_became_active":"2023-10-30T12:35:06.214839+0000","last_became_peered":"2023-10-30T12:35:06.214839+0000","last_unstale":"2023-10-30T12:35:07.249183+0000","last_undegraded":"2023-10-30T12:35:07.249183+0000","last_fullsized":"2023-10-30T12:35:07.249183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:54:34.206981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.199110+0000","last_change":"2023-10-30T12:35:05.189361+0000","last_active":"2023-10-30T12:35:11.199110+0000","last_peered":"2023-10-30T12:35:11.199110+0000","last_clean":"2023-10-30T12:35:11.199110+0000","last_became_active":"2023-10-30T12:35:05.188862+0000","last_became_peered":"2023-10-30T12:35:05.188862+0000","last_unstale":"2023-10-30T12:35:11.199110+0000","last_undegraded":"2023-10-30T12:35:11.199110+0000","last_fullsized":"2023-10-30T12:35:11.199110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:13.319732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250712+0000","last_change":"2023-10-30T12:35:06.223118+0000","last_active":"2023-10-30T12:35:07.250712+0000","last_peered":"2023-10-30T12:35:07.250712+0000","last_clean":"2023-10-30T12:35:07.250712+0000","last_became_active":"2023-10-30T12:35:06.216533+0000","last_became_peered":"2023-10-30T12:35:06.216533+0000","last_unstale":"2023-10-30T12:35:07.250712+0000","last_undegraded":"2023-10-30T12:35:07.250712+0000","last_fullsized":"2023-10-30T12:35:07.250712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:32:29.374526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.195596+0000","last_change":"2023-10-30T12:35:05.177609+0000","last_active":"2023-10-30T12:35:11.195596+0000","last_peered":"2023-10-30T12:35:11.195596+0000","last_clean":"2023-10-30T12:35:11.195596+0000","last_became_active":"2023-10-30T12:35:05.177316+0000","last_became_peered":"2023-10-30T12:35:05.177316+0000","last_unstale":"2023-10-30T12:35:11.195596+0000","last_undegraded":"2023-10-30T12:35:11.195596+0000","last_fullsized":"2023-10-30T12:35:11.195596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:46.820925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251056+0000","last_change":"2023-10-30T12:35:06.222283+0000","last_active":"2023-10-30T12:35:07.251056+0000","last_peered":"2023-10-30T12:35:07.251056+0000","last_clean":"2023-10-30T12:35:07.251056+0000","last_became_active":"2023-10-30T12:35:06.221701+0000","last_became_peered":"2023-10-30T12:35:06.221701+0000","last_unstale":"2023-10-30T12:35:07.251056+0000","last_undegraded":"2023-10-30T12:35:07.251056+0000","last_fullsized":"2023-10-30T12:35:07.251056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:37:32.956627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159413+0000","last_change":"2023-10-30T12:35:05.171787+0000","last_active":"2023-10-30T12:35:07.159413+0000","last_peered":"2023-10-30T12:35:07.159413+0000","last_clean":"2023-10-30T12:35:07.159413+0000","last_became_active":"2023-10-30T12:35:05.171491+0000","last_became_peered":"2023-10-30T12:35:05.171491+0000","last_unstale":"2023-10-30T12:35:07.159413+0000","last_undegraded":"2023-10-30T12:35:07.159413+0000","last_fullsized":"2023-10-30T12:35:07.159413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:43:15.331175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250517+0000","last_change":"2023-10-30T12:35:06.214464+0000","last_active":"2023-10-30T12:35:07.250517+0000","last_peered":"2023-10-30T12:35:07.250517+0000","last_clean":"2023-10-30T12:35:07.250517+0000","last_became_active":"2023-10-30T12:35:06.213493+0000","last_became_peered":"2023-10-30T12:35:06.213493+0000","last_unstale":"2023-10-30T12:35:07.250517+0000","last_undegraded":"2023-10-30T12:35:07.250517+0000","last_fullsized":"2023-10-30T12:35:07.250517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:00:18.624720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.337571+0000","last_change":"2023-10-30T12:35:05.197811+0000","last_active":"2023-10-30T12:35:27.337571+0000","last_peered":"2023-10-30T12:35:27.337571+0000","last_clean":"2023-10-30T12:35:27.337571+0000","last_became_active":"2023-10-30T12:35:05.197453+0000","last_became_peered":"2023-10-30T12:35:05.197453+0000","last_unstale":"2023-10-30T12:35:27.337571+0000","last_undegraded":"2023-10-30T12:35:27.337571+0000","last_fullsized":"2023-10-30T12:35:27.337571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:16:14.550887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249901+0000","last_change":"2023-10-30T12:35:06.220236+0000","last_active":"2023-10-30T12:35:07.249901+0000","last_peered":"2023-10-30T12:35:07.249901+0000","last_clean":"2023-10-30T12:35:07.249901+0000","last_became_active":"2023-10-30T12:35:06.219526+0000","last_became_peered":"2023-10-30T12:35:06.219526+0000","last_unstale":"2023-10-30T12:35:07.249901+0000","last_undegraded":"2023-10-30T12:35:07.249901+0000","last_fullsized":"2023-10-30T12:35:07.249901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:00:25.297590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.186752+0000","last_change":"2023-10-30T12:34:55.206976+0000","last_active":"2023-10-30T12:35:06.186752+0000","last_peered":"2023-10-30T12:35:06.186752+0000","last_clean":"2023-10-30T12:35:06.186752+0000","last_became_active":"2023-10-30T12:34:55.205923+0000","last_became_peered":"2023-10-30T12:34:55.205923+0000","last_unstale":"2023-10-30T12:35:06.186752+0000","last_undegraded":"2023-10-30T12:35:06.186752+0000","last_fullsized":"2023-10-30T12:35:06.186752+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_clean_scrub_stamp":"2023-10-30T12:34:53.444020+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:25:28.782111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.313312+0000","last_change":"2023-10-30T12:35:05.197974+0000","last_active":"2023-10-30T12:35:21.313312+0000","last_peered":"2023-10-30T12:35:21.313312+0000","last_clean":"2023-10-30T12:35:21.313312+0000","last_became_active":"2023-10-30T12:35:05.197201+0000","last_became_peered":"2023-10-30T12:35:05.197201+0000","last_unstale":"2023-10-30T12:35:21.313312+0000","last_undegraded":"2023-10-30T12:35:21.313312+0000","last_fullsized":"2023-10-30T12:35:21.313312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:39:47.119091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247224+0000","last_change":"2023-10-30T12:35:06.217973+0000","last_active":"2023-10-30T12:35:07.247224+0000","last_peered":"2023-10-30T12:35:07.247224+0000","last_clean":"2023-10-30T12:35:07.247224+0000","last_became_active":"2023-10-30T12:35:06.217687+0000","last_became_peered":"2023-10-30T12:35:06.217687+0000","last_unstale":"2023-10-30T12:35:07.247224+0000","last_undegraded":"2023-10-30T12:35:07.247224+0000","last_fullsized":"2023-10-30T12:35:07.247224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:28.392113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.333670+0000","last_change":"2023-10-30T12:35:05.208339+0000","last_active":"2023-10-30T12:35:27.333670+0000","last_peered":"2023-10-30T12:35:27.333670+0000","last_clean":"2023-10-30T12:35:27.333670+0000","last_became_active":"2023-10-30T12:35:05.207828+0000","last_became_peered":"2023-10-30T12:35:05.207828+0000","last_unstale":"2023-10-30T12:35:27.333670+0000","last_undegraded":"2023-10-30T12:35:27.333670+0000","last_fullsized":"2023-10-30T12:35:27.333670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:00.649048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160255+0000","last_change":"2023-10-30T12:35:06.195094+0000","last_active":"2023-10-30T12:35:07.160255+0000","last_peered":"2023-10-30T12:35:07.160255+0000","last_clean":"2023-10-30T12:35:07.160255+0000","last_became_active":"2023-10-30T12:35:06.194791+0000","last_became_peered":"2023-10-30T12:35:06.194791+0000","last_unstale":"2023-10-30T12:35:07.160255+0000","last_undegraded":"2023-10-30T12:35:07.160255+0000","last_fullsized":"2023-10-30T12:35:07.160255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:41:24.310349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.336946+0000","last_change":"2023-10-30T12:35:05.176296+0000","last_active":"2023-10-30T12:35:27.336946+0000","last_peered":"2023-10-30T12:35:27.336946+0000","last_clean":"2023-10-30T12:35:27.336946+0000","last_became_active":"2023-10-30T12:35:05.175959+0000","last_became_peered":"2023-10-30T12:35:05.175959+0000","last_unstale":"2023-10-30T12:35:27.336946+0000","last_undegraded":"2023-10-30T12:35:27.336946+0000","last_fullsized":"2023-10-30T12:35:27.336946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:53.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159052+0000","last_change":"2023-10-30T12:35:06.167250+0000","last_active":"2023-10-30T12:35:07.159052+0000","last_peered":"2023-10-30T12:35:07.159052+0000","last_clean":"2023-10-30T12:35:07.159052+0000","last_became_active":"2023-10-30T12:35:06.164711+0000","last_became_peered":"2023-10-30T12:35:06.164711+0000","last_unstale":"2023-10-30T12:35:07.159052+0000","last_undegraded":"2023-10-30T12:35:07.159052+0000","last_fullsized":"2023-10-30T12:35:07.159052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:48:18.710410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250361+0000","last_change":"2023-10-30T12:35:05.198964+0000","last_active":"2023-10-30T12:35:07.250361+0000","last_peered":"2023-10-30T12:35:07.250361+0000","last_clean":"2023-10-30T12:35:07.250361+0000","last_became_active":"2023-10-30T12:35:05.196873+0000","last_became_peered":"2023-10-30T12:35:05.196873+0000","last_unstale":"2023-10-30T12:35:07.250361+0000","last_undegraded":"2023-10-30T12:35:07.250361+0000","last_fullsized":"2023-10-30T12:35:07.250361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:47.793585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163526+0000","last_change":"2023-10-30T12:35:06.184123+0000","last_active":"2023-10-30T12:35:07.163526+0000","last_peered":"2023-10-30T12:35:07.163526+0000","last_clean":"2023-10-30T12:35:07.163526+0000","last_became_active":"2023-10-30T12:35:06.183225+0000","last_became_peered":"2023-10-30T12:35:06.183225+0000","last_unstale":"2023-10-30T12:35:07.163526+0000","last_undegraded":"2023-10-30T12:35:07.163526+0000","last_fullsized":"2023-10-30T12:35:07.163526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:03:47.097151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.543947+0000","last_change":"2023-10-30T12:35:05.177571+0000","last_active":"2023-10-30T12:35:06.543947+0000","last_peered":"2023-10-30T12:35:06.543947+0000","last_clean":"2023-10-30T12:35:06.543947+0000","last_became_active":"2023-10-30T12:35:05.177266+0000","last_became_peered":"2023-10-30T12:35:05.177266+0000","last_unstale":"2023-10-30T12:35:06.543947+0000","last_undegraded":"2023-10-30T12:35:06.543947+0000","last_fullsized":"2023-10-30T12:35:06.543947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:14.601934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250949+0000","last_change":"2023-10-30T12:35:06.206033+0000","last_active":"2023-10-30T12:35:07.250949+0000","last_peered":"2023-10-30T12:35:07.250949+0000","last_clean":"2023-10-30T12:35:07.250949+0000","last_became_active":"2023-10-30T12:35:06.196307+0000","last_became_peered":"2023-10-30T12:35:06.196307+0000","last_unstale":"2023-10-30T12:35:07.250949+0000","last_undegraded":"2023-10-30T12:35:07.250949+0000","last_fullsized":"2023-10-30T12:35:07.250949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:18:28.848259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.460771+0000","last_change":"2023-10-30T12:35:05.207594+0000","last_active":"2023-10-30T12:35:07.460771+0000","last_peered":"2023-10-30T12:35:07.460771+0000","last_clean":"2023-10-30T12:35:07.460771+0000","last_became_active":"2023-10-30T12:35:05.206930+0000","last_became_peered":"2023-10-30T12:35:05.206930+0000","last_unstale":"2023-10-30T12:35:07.460771+0000","last_undegraded":"2023-10-30T12:35:07.460771+0000","last_fullsized":"2023-10-30T12:35:07.460771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:23:46.519077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159110+0000","last_change":"2023-10-30T12:35:06.165126+0000","last_active":"2023-10-30T12:35:07.159110+0000","last_peered":"2023-10-30T12:35:07.159110+0000","last_clean":"2023-10-30T12:35:07.159110+0000","last_became_active":"2023-10-30T12:35:06.164733+0000","last_became_peered":"2023-10-30T12:35:06.164733+0000","last_unstale":"2023-10-30T12:35:07.159110+0000","last_undegraded":"2023-10-30T12:35:07.159110+0000","last_fullsized":"2023-10-30T12:35:07.159110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:29.336555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.337400+0000","last_change":"2023-10-30T12:35:05.185034+0000","last_active":"2023-10-30T12:35:27.337400+0000","last_peered":"2023-10-30T12:35:27.337400+0000","last_clean":"2023-10-30T12:35:27.337400+0000","last_became_active":"2023-10-30T12:35:05.184695+0000","last_became_peered":"2023-10-30T12:35:05.184695+0000","last_unstale":"2023-10-30T12:35:27.337400+0000","last_undegraded":"2023-10-30T12:35:27.337400+0000","last_fullsized":"2023-10-30T12:35:27.337400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:01:42.019128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247746+0000","last_change":"2023-10-30T12:35:06.204707+0000","last_active":"2023-10-30T12:35:07.247746+0000","last_peered":"2023-10-30T12:35:07.247746+0000","last_clean":"2023-10-30T12:35:07.247746+0000","last_became_active":"2023-10-30T12:35:06.204422+0000","last_became_peered":"2023-10-30T12:35:06.204422+0000","last_unstale":"2023-10-30T12:35:07.247746+0000","last_undegraded":"2023-10-30T12:35:07.247746+0000","last_fullsized":"2023-10-30T12:35:07.247746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:58:16.758575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.304904+0000","last_change":"2023-10-30T12:35:05.200871+0000","last_active":"2023-10-30T12:35:21.304904+0000","last_peered":"2023-10-30T12:35:21.304904+0000","last_clean":"2023-10-30T12:35:21.304904+0000","last_became_active":"2023-10-30T12:35:05.200449+0000","last_became_peered":"2023-10-30T12:35:05.200449+0000","last_unstale":"2023-10-30T12:35:21.304904+0000","last_undegraded":"2023-10-30T12:35:21.304904+0000","last_fullsized":"2023-10-30T12:35:21.304904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:39:20.867587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160562+0000","last_change":"2023-10-30T12:35:06.180850+0000","last_active":"2023-10-30T12:35:07.160562+0000","last_peered":"2023-10-30T12:35:07.160562+0000","last_clean":"2023-10-30T12:35:07.160562+0000","last_became_active":"2023-10-30T12:35:06.179412+0000","last_became_peered":"2023-10-30T12:35:06.179412+0000","last_unstale":"2023-10-30T12:35:07.160562+0000","last_undegraded":"2023-10-30T12:35:07.160562+0000","last_fullsized":"2023-10-30T12:35:07.160562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:47:44.055205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.304568+0000","last_change":"2023-10-30T12:35:05.207392+0000","last_active":"2023-10-30T12:35:21.304568+0000","last_peered":"2023-10-30T12:35:21.304568+0000","last_clean":"2023-10-30T12:35:21.304568+0000","last_became_active":"2023-10-30T12:35:05.206607+0000","last_became_peered":"2023-10-30T12:35:05.206607+0000","last_unstale":"2023-10-30T12:35:21.304568+0000","last_undegraded":"2023-10-30T12:35:21.304568+0000","last_fullsized":"2023-10-30T12:35:21.304568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:36.758740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163445+0000","last_change":"2023-10-30T12:35:06.182042+0000","last_active":"2023-10-30T12:35:07.163445+0000","last_peered":"2023-10-30T12:35:07.163445+0000","last_clean":"2023-10-30T12:35:07.163445+0000","last_became_active":"2023-10-30T12:35:06.181256+0000","last_became_peered":"2023-10-30T12:35:06.181256+0000","last_unstale":"2023-10-30T12:35:07.163445+0000","last_undegraded":"2023-10-30T12:35:07.163445+0000","last_fullsized":"2023-10-30T12:35:07.163445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:13:08.109392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.305680+0000","last_change":"2023-10-30T12:35:05.201024+0000","last_active":"2023-10-30T12:35:21.305680+0000","last_peered":"2023-10-30T12:35:21.305680+0000","last_clean":"2023-10-30T12:35:21.305680+0000","last_became_active":"2023-10-30T12:35:05.200724+0000","last_became_peered":"2023-10-30T12:35:05.200724+0000","last_unstale":"2023-10-30T12:35:21.305680+0000","last_undegraded":"2023-10-30T12:35:21.305680+0000","last_fullsized":"2023-10-30T12:35:21.305680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:24.746469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247249+0000","last_change":"2023-10-30T12:35:06.174636+0000","last_active":"2023-10-30T12:35:07.247249+0000","last_peered":"2023-10-30T12:35:07.247249+0000","last_clean":"2023-10-30T12:35:07.247249+0000","last_became_active":"2023-10-30T12:35:06.173996+0000","last_became_peered":"2023-10-30T12:35:06.173996+0000","last_unstale":"2023-10-30T12:35:07.247249+0000","last_undegraded":"2023-10-30T12:35:07.247249+0000","last_fullsized":"2023-10-30T12:35:07.247249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:01:38.503494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159100+0000","last_change":"2023-10-30T12:35:05.164079+0000","last_active":"2023-10-30T12:35:07.159100+0000","last_peered":"2023-10-30T12:35:07.159100+0000","last_clean":"2023-10-30T12:35:07.159100+0000","last_became_active":"2023-10-30T12:35:05.163794+0000","last_became_peered":"2023-10-30T12:35:05.163794+0000","last_unstale":"2023-10-30T12:35:07.159100+0000","last_undegraded":"2023-10-30T12:35:07.159100+0000","last_fullsized":"2023-10-30T12:35:07.159100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:19.795656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247344+0000","last_change":"2023-10-30T12:35:06.183531+0000","last_active":"2023-10-30T12:35:07.247344+0000","last_peered":"2023-10-30T12:35:07.247344+0000","last_clean":"2023-10-30T12:35:07.247344+0000","last_became_active":"2023-10-30T12:35:06.183226+0000","last_became_peered":"2023-10-30T12:35:06.183226+0000","last_unstale":"2023-10-30T12:35:07.247344+0000","last_undegraded":"2023-10-30T12:35:07.247344+0000","last_fullsized":"2023-10-30T12:35:07.247344+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:46.768356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.303852+0000","last_change":"2023-10-30T12:35:05.206995+0000","last_active":"2023-10-30T12:35:21.303852+0000","last_peered":"2023-10-30T12:35:21.303852+0000","last_clean":"2023-10-30T12:35:21.303852+0000","last_became_active":"2023-10-30T12:35:05.206518+0000","last_became_peered":"2023-10-30T12:35:05.206518+0000","last_unstale":"2023-10-30T12:35:21.303852+0000","last_undegraded":"2023-10-30T12:35:21.303852+0000","last_fullsized":"2023-10-30T12:35:21.303852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:41:33.703261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162528+0000","last_change":"2023-10-30T12:35:06.199557+0000","last_active":"2023-10-30T12:35:07.162528+0000","last_peered":"2023-10-30T12:35:07.162528+0000","last_clean":"2023-10-30T12:35:07.162528+0000","last_became_active":"2023-10-30T12:35:06.197861+0000","last_became_peered":"2023-10-30T12:35:06.197861+0000","last_unstale":"2023-10-30T12:35:07.162528+0000","last_undegraded":"2023-10-30T12:35:07.162528+0000","last_fullsized":"2023-10-30T12:35:07.162528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:30:46.092876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250766+0000","last_change":"2023-10-30T12:35:06.213279+0000","last_active":"2023-10-30T12:35:07.250766+0000","last_peered":"2023-10-30T12:35:07.250766+0000","last_clean":"2023-10-30T12:35:07.250766+0000","last_became_active":"2023-10-30T12:35:06.212548+0000","last_became_peered":"2023-10-30T12:35:06.212548+0000","last_unstale":"2023-10-30T12:35:07.250766+0000","last_undegraded":"2023-10-30T12:35:07.250766+0000","last_fullsized":"2023-10-30T12:35:07.250766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:01:57.127208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.200195+0000","last_change":"2023-10-30T12:35:05.206980+0000","last_active":"2023-10-30T12:35:11.200195+0000","last_peered":"2023-10-30T12:35:11.200195+0000","last_clean":"2023-10-30T12:35:11.200195+0000","last_became_active":"2023-10-30T12:35:05.206559+0000","last_became_peered":"2023-10-30T12:35:05.206559+0000","last_unstale":"2023-10-30T12:35:11.200195+0000","last_undegraded":"2023-10-30T12:35:11.200195+0000","last_fullsized":"2023-10-30T12:35:11.200195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:40.259622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250210+0000","last_change":"2023-10-30T12:35:06.213053+0000","last_active":"2023-10-30T12:35:07.250210+0000","last_peered":"2023-10-30T12:35:07.250210+0000","last_clean":"2023-10-30T12:35:07.250210+0000","last_became_active":"2023-10-30T12:35:06.212505+0000","last_became_peered":"2023-10-30T12:35:06.212505+0000","last_unstale":"2023-10-30T12:35:07.250210+0000","last_undegraded":"2023-10-30T12:35:07.250210+0000","last_fullsized":"2023-10-30T12:35:07.250210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:44:02.073639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.333919+0000","last_change":"2023-10-30T12:35:05.207851+0000","last_active":"2023-10-30T12:35:27.333919+0000","last_peered":"2023-10-30T12:35:27.333919+0000","last_clean":"2023-10-30T12:35:27.333919+0000","last_became_active":"2023-10-30T12:35:05.206879+0000","last_became_peered":"2023-10-30T12:35:05.206879+0000","last_unstale":"2023-10-30T12:35:27.333919+0000","last_undegraded":"2023-10-30T12:35:27.333919+0000","last_fullsized":"2023-10-30T12:35:27.333919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:43:15.717507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160637+0000","last_change":"2023-10-30T12:35:06.222373+0000","last_active":"2023-10-30T12:35:07.160637+0000","last_peered":"2023-10-30T12:35:07.160637+0000","last_clean":"2023-10-30T12:35:07.160637+0000","last_became_active":"2023-10-30T12:35:06.222094+0000","last_became_peered":"2023-10-30T12:35:06.222094+0000","last_unstale":"2023-10-30T12:35:07.160637+0000","last_undegraded":"2023-10-30T12:35:07.160637+0000","last_fullsized":"2023-10-30T12:35:07.160637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:42:32.482862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.195154+0000","last_change":"2023-10-30T12:35:05.174319+0000","last_active":"2023-10-30T12:35:11.195154+0000","last_peered":"2023-10-30T12:35:11.195154+0000","last_clean":"2023-10-30T12:35:11.195154+0000","last_became_active":"2023-10-30T12:35:05.173984+0000","last_became_peered":"2023-10-30T12:35:05.173984+0000","last_unstale":"2023-10-30T12:35:11.195154+0000","last_undegraded":"2023-10-30T12:35:11.195154+0000","last_fullsized":"2023-10-30T12:35:11.195154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:11.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247593+0000","last_change":"2023-10-30T12:35:06.198881+0000","last_active":"2023-10-30T12:35:07.247593+0000","last_peered":"2023-10-30T12:35:07.247593+0000","last_clean":"2023-10-30T12:35:07.247593+0000","last_became_active":"2023-10-30T12:35:06.198504+0000","last_became_peered":"2023-10-30T12:35:06.198504+0000","last_unstale":"2023-10-30T12:35:07.247593+0000","last_undegraded":"2023-10-30T12:35:07.247593+0000","last_fullsized":"2023-10-30T12:35:07.247593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:06:33.845147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'9","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.335834+0000","last_change":"2023-10-30T12:35:05.200926+0000","last_active":"2023-10-30T12:35:27.335834+0000","last_peered":"2023-10-30T12:35:27.335834+0000","last_clean":"2023-10-30T12:35:27.335834+0000","last_became_active":"2023-10-30T12:35:05.200623+0000","last_became_peered":"2023-10-30T12:35:05.200623+0000","last_unstale":"2023-10-30T12:35:27.335834+0000","last_undegraded":"2023-10-30T12:35:27.335834+0000","last_fullsized":"2023-10-30T12:35:27.335834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:38.942922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6553,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":13106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":839680,"data_stored":670806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":27654605},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542157,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":82227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]}]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":241664,"data_stored":79578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":79682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":669892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":80242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":249856,"data_stored":82519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":249856,"data_stored":79808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:35:40.166 INFO:tasks.mds_thrash.ceph_manager:clean! 2023-10-30T12:35:40.167 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:35:40.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f63925f5640 1 Processor -- start 2023-10-30T12:35:40.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f63925f5640 1 -- start start 2023-10-30T12:35:40.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f63925f5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f638c0c6b80 0x7f638c0bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:40.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f63925f5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f638c0c6f60 0x7f638c0c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:40.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f63925f5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f638c0c8430 0x7f638c13ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:40.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f63925f5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638c0c0a50 con 0x7f638c0c8430 2023-10-30T12:35:40.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f63925f5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638c06f6e0 con 0x7f638c0c6b80 2023-10-30T12:35:40.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f63925f5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c06f9c0 con 0x7f638c0c6f60 2023-10-30T12:35:40.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f6390b92640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f638c0c8430 0x7f638c13ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:40.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f6390b92640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f638c0c8430 0x7f638c13ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57452/0 (socket says 172.21.15.12:57452) 2023-10-30T12:35:40.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f6390b92640 1 -- 172.21.15.12:0/2968408727 learned_addr learned my addr 172.21.15.12:0/2968408727 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:40.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f638bfff640 1 --2- 172.21.15.12:0/2968408727 >> v2:172.21.15.154:3300/0 conn(0x7f638c0c6b80 0x7f638c0bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:40.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f638b7fe640 1 --2- 172.21.15.12:0/2968408727 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f638c0c6f60 0x7f638c0c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:40.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f6390b92640 1 -- 172.21.15.12:0/2968408727 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f638c0c6f60 msgr2=0x7f638c0c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:40.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f6390b92640 1 --2- 172.21.15.12:0/2968408727 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f638c0c6f60 0x7f638c0c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f6390b92640 1 -- 172.21.15.12:0/2968408727 >> v2:172.21.15.154:3300/0 conn(0x7f638c0c6b80 msgr2=0x7f638c0bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:40.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f6390b92640 1 --2- 172.21.15.12:0/2968408727 >> v2:172.21.15.154:3300/0 conn(0x7f638c0c6b80 0x7f638c0bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.346+0000 7f6390b92640 1 -- 172.21.15.12:0/2968408727 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c06fca0 con 0x7f638c0c8430 2023-10-30T12:35:40.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.350+0000 7f6390b92640 1 --2- 172.21.15.12:0/2968408727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f638c0c8430 0x7f638c13ed90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f638c06f1b0 tx=0x7f638401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a51d05c99463ef2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:40.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.350+0000 7f638affd640 1 -- 172.21.15.12:0/2968408727 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63840a56d0 con 0x7f638c0c8430 2023-10-30T12:35:40.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.350+0000 7f638affd640 1 -- 172.21.15.12:0/2968408727 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63840a5830 con 0x7f638c0c8430 2023-10-30T12:35:40.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.350+0000 7f638affd640 1 -- 172.21.15.12:0/2968408727 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63840a5bc0 con 0x7f638c0c8430 2023-10-30T12:35:40.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.350+0000 7f63925f5640 1 -- 172.21.15.12:0/2968408727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f638c0c8430 msgr2=0x7f638c13ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:40.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.350+0000 7f63925f5640 1 --2- 172.21.15.12:0/2968408727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f638c0c8430 0x7f638c13ed90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f638c06f1b0 tx=0x7f638401c940 comp rx=0 tx=0).stop 2023-10-30T12:35:40.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.350+0000 7f63925f5640 1 -- 172.21.15.12:0/2968408727 shutdown_connections 2023-10-30T12:35:40.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.350+0000 7f63925f5640 1 --2- 172.21.15.12:0/2968408727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f638c0c8430 0x7f638c13ed90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.350+0000 7f63925f5640 1 --2- 172.21.15.12:0/2968408727 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f638c0c6f60 0x7f638c0c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.350+0000 7f63925f5640 1 --2- 172.21.15.12:0/2968408727 >> v2:172.21.15.154:3300/0 conn(0x7f638c0c6b80 0x7f638c0bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.350+0000 7f63925f5640 1 -- 172.21.15.12:0/2968408727 >> 172.21.15.12:0/2968408727 conn(0x7f638c0c7910 msgr2=0x7f638c0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:40.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.350+0000 7f63925f5640 1 -- 172.21.15.12:0/2968408727 shutdown_connections 2023-10-30T12:35:40.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.350+0000 7f63925f5640 1 -- 172.21.15.12:0/2968408727 wait complete. 2023-10-30T12:35:40.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f63925f5640 1 Processor -- start 2023-10-30T12:35:40.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f63925f5640 1 -- start start 2023-10-30T12:35:40.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f63925f5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f638c0c6b80 0x7f638c134d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:40.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f63925f5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f638c0c6f60 0x7f638c15e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:40.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f63925f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f638c0c8430 0x7f638c15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:40.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f63925f5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c15e430 con 0x7f638c0c8430 2023-10-30T12:35:40.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f63925f5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c1583b0 con 0x7f638c0c6b80 2023-10-30T12:35:40.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f63925f5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c158670 con 0x7f638c0c6f60 2023-10-30T12:35:40.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f6390b92640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f638c0c8430 0x7f638c15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:40.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f6390b92640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f638c0c8430 0x7f638c15ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57462/0 (socket says 172.21.15.12:57462) 2023-10-30T12:35:40.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f6390b92640 1 -- 172.21.15.12:0/1082588517 learned_addr learned my addr 172.21.15.12:0/1082588517 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:40.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f638bfff640 1 --2- 172.21.15.12:0/1082588517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f638c0c6b80 0x7f638c134d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:40.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f638b7fe640 1 --2- 172.21.15.12:0/1082588517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f638c0c6f60 0x7f638c15e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:40.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f6390b92640 1 -- 172.21.15.12:0/1082588517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f638c0c6f60 msgr2=0x7f638c15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:40.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f6390b92640 1 --2- 172.21.15.12:0/1082588517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f638c0c6f60 0x7f638c15e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f6390b92640 1 -- 172.21.15.12:0/1082588517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f638c0c6b80 msgr2=0x7f638c134d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:40.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f6390b92640 1 --2- 172.21.15.12:0/1082588517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f638c0c6b80 0x7f638c134d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f6390b92640 1 -- 172.21.15.12:0/1082588517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6384009b30 con 0x7f638c0c8430 2023-10-30T12:35:40.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f6390b92640 1 --2- 172.21.15.12:0/1082588517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f638c0c8430 0x7f638c15ed90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f638401d1e0 tx=0x7f63840196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:40.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f6388ff9640 1 -- 172.21.15.12:0/1082588517 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63840b6890 con 0x7f638c0c8430 2023-10-30T12:35:40.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f6388ff9640 1 -- 172.21.15.12:0/1082588517 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63840b6a30 con 0x7f638c0c8430 2023-10-30T12:35:40.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f6388ff9640 1 -- 172.21.15.12:0/1082588517 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63840b6de0 con 0x7f638c0c8430 2023-10-30T12:35:40.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f63925f5640 1 -- 172.21.15.12:0/1082588517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f638c15e090 con 0x7f638c0c8430 2023-10-30T12:35:40.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.354+0000 7f63925f5640 1 -- 172.21.15.12:0/1082588517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638c159650 con 0x7f638c0c8430 2023-10-30T12:35:40.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.358+0000 7f63925f5640 1 -- 172.21.15.12:0/1082588517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638c15e570 con 0x7f638c0c8430 2023-10-30T12:35:40.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.358+0000 7f6388ff9640 1 -- 172.21.15.12:0/1082588517 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f63840c0410 con 0x7f638c0c8430 2023-10-30T12:35:40.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.358+0000 7f6388ff9640 1 --2- 172.21.15.12:0/1082588517 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f63780664a0 0x7f6378068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:40.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.358+0000 7f6388ff9640 1 -- 172.21.15.12:0/1082588517 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f638411eb00 con 0x7f638c0c8430 2023-10-30T12:35:40.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.358+0000 7f638bfff640 1 --2- 172.21.15.12:0/1082588517 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f63780664a0 0x7f6378068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:40.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.358+0000 7f638bfff640 1 --2- 172.21.15.12:0/1082588517 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f63780664a0 0x7f6378068960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f637c0045f0 tx=0x7f637c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:40.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.362+0000 7f6388ff9640 1 -- 172.21.15.12:0/1082588517 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f638c15e570 con 0x7f638c0c8430 2023-10-30T12:35:40.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.518+0000 7f63925f5640 1 -- 172.21.15.12:0/1082588517 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f638c15e570 con 0x7f63780664a0 2023-10-30T12:35:40.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.538+0000 7f6388ff9640 1 -- 172.21.15.12:0/1082588517 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264700 (secure 0 0 0) 0x7f638c15e570 con 0x7f63780664a0 2023-10-30T12:35:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:40.544 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-10-30T12:35:40.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.538+0000 7f63925f5640 1 -- 172.21.15.12:0/1082588517 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f63780664a0 msgr2=0x7f6378068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:40.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.538+0000 7f63925f5640 1 --2- 172.21.15.12:0/1082588517 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f63780664a0 0x7f6378068960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f637c0045f0 tx=0x7f637c008040 comp rx=0 tx=0).stop 2023-10-30T12:35:40.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.538+0000 7f63925f5640 1 -- 172.21.15.12:0/1082588517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f638c0c8430 msgr2=0x7f638c15ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:40.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.538+0000 7f63925f5640 1 --2- 172.21.15.12:0/1082588517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f638c0c8430 0x7f638c15ed90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f638401d1e0 tx=0x7f63840196e0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.542+0000 7f63925f5640 1 -- 172.21.15.12:0/1082588517 shutdown_connections 2023-10-30T12:35:40.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.542+0000 7f63925f5640 1 --2- 172.21.15.12:0/1082588517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f638c0c8430 0x7f638c15ed90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.542+0000 7f63925f5640 1 --2- 172.21.15.12:0/1082588517 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f63780664a0 0x7f6378068960 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.542+0000 7f63925f5640 1 --2- 172.21.15.12:0/1082588517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f638c0c6f60 0x7f638c15e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.542+0000 7f63925f5640 1 --2- 172.21.15.12:0/1082588517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f638c0c6b80 0x7f638c134d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.542+0000 7f63925f5640 1 -- 172.21.15.12:0/1082588517 >> 172.21.15.12:0/1082588517 conn(0x7f638c0c7910 msgr2=0x7f638c0b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:40.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.542+0000 7f63925f5640 1 -- 172.21.15.12:0/1082588517 shutdown_connections 2023-10-30T12:35:40.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.542+0000 7f63925f5640 1 -- 172.21.15.12:0/1082588517 wait complete. 2023-10-30T12:35:40.563 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-10-30T12:35:38.947034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596921,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":288,"num_write_kb":2458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219208,"kb_used_data":3080,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755512,"statfs":{"total":773094113280,"available":772869644288,"internally_reserved":0,"allocated":3153920,"data_stored":1824754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12729,"internal_metadata":221236807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2737,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004266"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250527+0000","last_change":"2023-10-30T12:35:06.223711+0000","last_active":"2023-10-30T12:35:07.250527+0000","last_peered":"2023-10-30T12:35:07.250527+0000","last_clean":"2023-10-30T12:35:07.250527+0000","last_became_active":"2023-10-30T12:35:06.222724+0000","last_became_peered":"2023-10-30T12:35:06.222724+0000","last_unstale":"2023-10-30T12:35:07.250527+0000","last_undegraded":"2023-10-30T12:35:07.250527+0000","last_fullsized":"2023-10-30T12:35:07.250527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:34:35.217274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159376+0000","last_change":"2023-10-30T12:35:06.182748+0000","last_active":"2023-10-30T12:35:07.159376+0000","last_peered":"2023-10-30T12:35:07.159376+0000","last_clean":"2023-10-30T12:35:07.159376+0000","last_became_active":"2023-10-30T12:35:06.182441+0000","last_became_peered":"2023-10-30T12:35:06.182441+0000","last_unstale":"2023-10-30T12:35:07.159376+0000","last_undegraded":"2023-10-30T12:35:07.159376+0000","last_fullsized":"2023-10-30T12:35:07.159376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:26:28.943768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160588+0000","last_change":"2023-10-30T12:35:06.180426+0000","last_active":"2023-10-30T12:35:07.160588+0000","last_peered":"2023-10-30T12:35:07.160588+0000","last_clean":"2023-10-30T12:35:07.160588+0000","last_became_active":"2023-10-30T12:35:06.180132+0000","last_became_peered":"2023-10-30T12:35:06.180132+0000","last_unstale":"2023-10-30T12:35:07.160588+0000","last_undegraded":"2023-10-30T12:35:07.160588+0000","last_fullsized":"2023-10-30T12:35:07.160588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:33:22.216739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250158+0000","last_change":"2023-10-30T12:35:06.217986+0000","last_active":"2023-10-30T12:35:07.250158+0000","last_peered":"2023-10-30T12:35:07.250158+0000","last_clean":"2023-10-30T12:35:07.250158+0000","last_became_active":"2023-10-30T12:35:06.213987+0000","last_became_peered":"2023-10-30T12:35:06.213987+0000","last_unstale":"2023-10-30T12:35:07.250158+0000","last_undegraded":"2023-10-30T12:35:07.250158+0000","last_fullsized":"2023-10-30T12:35:07.250158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:51:20.515999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249406+0000","last_change":"2023-10-30T12:35:06.216098+0000","last_active":"2023-10-30T12:35:07.249406+0000","last_peered":"2023-10-30T12:35:07.249406+0000","last_clean":"2023-10-30T12:35:07.249406+0000","last_became_active":"2023-10-30T12:35:06.214964+0000","last_became_peered":"2023-10-30T12:35:06.214964+0000","last_unstale":"2023-10-30T12:35:07.249406+0000","last_undegraded":"2023-10-30T12:35:07.249406+0000","last_fullsized":"2023-10-30T12:35:07.249406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:37:43.156964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249861+0000","last_change":"2023-10-30T12:35:06.217608+0000","last_active":"2023-10-30T12:35:07.249861+0000","last_peered":"2023-10-30T12:35:07.249861+0000","last_clean":"2023-10-30T12:35:07.249861+0000","last_became_active":"2023-10-30T12:35:06.214219+0000","last_became_peered":"2023-10-30T12:35:06.214219+0000","last_unstale":"2023-10-30T12:35:07.249861+0000","last_undegraded":"2023-10-30T12:35:07.249861+0000","last_fullsized":"2023-10-30T12:35:07.249861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:35:27.431813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249000+0000","last_change":"2023-10-30T12:35:06.217120+0000","last_active":"2023-10-30T12:35:07.249000+0000","last_peered":"2023-10-30T12:35:07.249000+0000","last_clean":"2023-10-30T12:35:07.249000+0000","last_became_active":"2023-10-30T12:35:06.216468+0000","last_became_peered":"2023-10-30T12:35:06.216468+0000","last_unstale":"2023-10-30T12:35:07.249000+0000","last_undegraded":"2023-10-30T12:35:07.249000+0000","last_fullsized":"2023-10-30T12:35:07.249000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:26:07.313967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246707+0000","last_change":"2023-10-30T12:35:06.214767+0000","last_active":"2023-10-30T12:35:07.246707+0000","last_peered":"2023-10-30T12:35:07.246707+0000","last_clean":"2023-10-30T12:35:07.246707+0000","last_became_active":"2023-10-30T12:35:06.214384+0000","last_became_peered":"2023-10-30T12:35:06.214384+0000","last_unstale":"2023-10-30T12:35:07.246707+0000","last_undegraded":"2023-10-30T12:35:07.246707+0000","last_fullsized":"2023-10-30T12:35:07.246707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:29.866538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250843+0000","last_change":"2023-10-30T12:35:06.223503+0000","last_active":"2023-10-30T12:35:07.250843+0000","last_peered":"2023-10-30T12:35:07.250843+0000","last_clean":"2023-10-30T12:35:07.250843+0000","last_became_active":"2023-10-30T12:35:06.222692+0000","last_became_peered":"2023-10-30T12:35:06.222692+0000","last_unstale":"2023-10-30T12:35:07.250843+0000","last_undegraded":"2023-10-30T12:35:07.250843+0000","last_fullsized":"2023-10-30T12:35:07.250843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:31:36.926720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163304+0000","last_change":"2023-10-30T12:35:06.215938+0000","last_active":"2023-10-30T12:35:07.163304+0000","last_peered":"2023-10-30T12:35:07.163304+0000","last_clean":"2023-10-30T12:35:07.163304+0000","last_became_active":"2023-10-30T12:35:06.215384+0000","last_became_peered":"2023-10-30T12:35:06.215384+0000","last_unstale":"2023-10-30T12:35:07.163304+0000","last_undegraded":"2023-10-30T12:35:07.163304+0000","last_fullsized":"2023-10-30T12:35:07.163304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:46:21.253555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162425+0000","last_change":"2023-10-30T12:35:06.215965+0000","last_active":"2023-10-30T12:35:07.162425+0000","last_peered":"2023-10-30T12:35:07.162425+0000","last_clean":"2023-10-30T12:35:07.162425+0000","last_became_active":"2023-10-30T12:35:06.215355+0000","last_became_peered":"2023-10-30T12:35:06.215355+0000","last_unstale":"2023-10-30T12:35:07.162425+0000","last_undegraded":"2023-10-30T12:35:07.162425+0000","last_fullsized":"2023-10-30T12:35:07.162425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:07.929526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164185+0000","last_change":"2023-10-30T12:35:06.197278+0000","last_active":"2023-10-30T12:35:07.164185+0000","last_peered":"2023-10-30T12:35:07.164185+0000","last_clean":"2023-10-30T12:35:07.164185+0000","last_became_active":"2023-10-30T12:35:06.196503+0000","last_became_peered":"2023-10-30T12:35:06.196503+0000","last_unstale":"2023-10-30T12:35:07.164185+0000","last_undegraded":"2023-10-30T12:35:07.164185+0000","last_fullsized":"2023-10-30T12:35:07.164185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:06.928264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247199+0000","last_change":"2023-10-30T12:35:06.195721+0000","last_active":"2023-10-30T12:35:07.247199+0000","last_peered":"2023-10-30T12:35:07.247199+0000","last_clean":"2023-10-30T12:35:07.247199+0000","last_became_active":"2023-10-30T12:35:06.195408+0000","last_became_peered":"2023-10-30T12:35:06.195408+0000","last_unstale":"2023-10-30T12:35:07.247199+0000","last_undegraded":"2023-10-30T12:35:07.247199+0000","last_fullsized":"2023-10-30T12:35:07.247199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:59:02.872646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160258+0000","last_change":"2023-10-30T12:35:06.182939+0000","last_active":"2023-10-30T12:35:07.160258+0000","last_peered":"2023-10-30T12:35:07.160258+0000","last_clean":"2023-10-30T12:35:07.160258+0000","last_became_active":"2023-10-30T12:35:06.182573+0000","last_became_peered":"2023-10-30T12:35:06.182573+0000","last_unstale":"2023-10-30T12:35:07.160258+0000","last_undegraded":"2023-10-30T12:35:07.160258+0000","last_fullsized":"2023-10-30T12:35:07.160258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:50.345142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.158834+0000","last_change":"2023-10-30T12:35:06.185636+0000","last_active":"2023-10-30T12:35:07.158834+0000","last_peered":"2023-10-30T12:35:07.158834+0000","last_clean":"2023-10-30T12:35:07.158834+0000","last_became_active":"2023-10-30T12:35:06.185290+0000","last_became_peered":"2023-10-30T12:35:06.185290+0000","last_unstale":"2023-10-30T12:35:07.158834+0000","last_undegraded":"2023-10-30T12:35:07.158834+0000","last_fullsized":"2023-10-30T12:35:07.158834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:26:44.783714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.248972+0000","last_change":"2023-10-30T12:35:06.214488+0000","last_active":"2023-10-30T12:35:07.248972+0000","last_peered":"2023-10-30T12:35:07.248972+0000","last_clean":"2023-10-30T12:35:07.248972+0000","last_became_active":"2023-10-30T12:35:06.213627+0000","last_became_peered":"2023-10-30T12:35:06.213627+0000","last_unstale":"2023-10-30T12:35:07.248972+0000","last_undegraded":"2023-10-30T12:35:07.248972+0000","last_fullsized":"2023-10-30T12:35:07.248972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:22:47.148407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251739+0000","last_change":"2023-10-30T12:35:06.224903+0000","last_active":"2023-10-30T12:35:07.251739+0000","last_peered":"2023-10-30T12:35:07.251739+0000","last_clean":"2023-10-30T12:35:07.251739+0000","last_became_active":"2023-10-30T12:35:06.221696+0000","last_became_peered":"2023-10-30T12:35:06.221696+0000","last_unstale":"2023-10-30T12:35:07.251739+0000","last_undegraded":"2023-10-30T12:35:07.251739+0000","last_fullsized":"2023-10-30T12:35:07.251739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:31:10.439162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246945+0000","last_change":"2023-10-30T12:35:06.198303+0000","last_active":"2023-10-30T12:35:07.246945+0000","last_peered":"2023-10-30T12:35:07.246945+0000","last_clean":"2023-10-30T12:35:07.246945+0000","last_became_active":"2023-10-30T12:35:06.198074+0000","last_became_peered":"2023-10-30T12:35:06.198074+0000","last_unstale":"2023-10-30T12:35:07.246945+0000","last_undegraded":"2023-10-30T12:35:07.246945+0000","last_fullsized":"2023-10-30T12:35:07.246945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:21:36.079586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250503+0000","last_change":"2023-10-30T12:35:06.218556+0000","last_active":"2023-10-30T12:35:07.250503+0000","last_peered":"2023-10-30T12:35:07.250503+0000","last_clean":"2023-10-30T12:35:07.250503+0000","last_became_active":"2023-10-30T12:35:06.215935+0000","last_became_peered":"2023-10-30T12:35:06.215935+0000","last_unstale":"2023-10-30T12:35:07.250503+0000","last_undegraded":"2023-10-30T12:35:07.250503+0000","last_fullsized":"2023-10-30T12:35:07.250503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:36:31.084542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162438+0000","last_change":"2023-10-30T12:35:06.201302+0000","last_active":"2023-10-30T12:35:07.162438+0000","last_peered":"2023-10-30T12:35:07.162438+0000","last_clean":"2023-10-30T12:35:07.162438+0000","last_became_active":"2023-10-30T12:35:06.199812+0000","last_became_peered":"2023-10-30T12:35:06.199812+0000","last_unstale":"2023-10-30T12:35:07.162438+0000","last_undegraded":"2023-10-30T12:35:07.162438+0000","last_fullsized":"2023-10-30T12:35:07.162438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:48.803738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160558+0000","last_change":"2023-10-30T12:35:06.217139+0000","last_active":"2023-10-30T12:35:07.160558+0000","last_peered":"2023-10-30T12:35:07.160558+0000","last_clean":"2023-10-30T12:35:07.160558+0000","last_became_active":"2023-10-30T12:35:06.216861+0000","last_became_peered":"2023-10-30T12:35:06.216861+0000","last_unstale":"2023-10-30T12:35:07.160558+0000","last_undegraded":"2023-10-30T12:35:07.160558+0000","last_fullsized":"2023-10-30T12:35:07.160558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:05:22.193180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249973+0000","last_change":"2023-10-30T12:35:06.225140+0000","last_active":"2023-10-30T12:35:07.249973+0000","last_peered":"2023-10-30T12:35:07.249973+0000","last_clean":"2023-10-30T12:35:07.249973+0000","last_became_active":"2023-10-30T12:35:06.221841+0000","last_became_peered":"2023-10-30T12:35:06.221841+0000","last_unstale":"2023-10-30T12:35:07.249973+0000","last_undegraded":"2023-10-30T12:35:07.249973+0000","last_fullsized":"2023-10-30T12:35:07.249973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:51:57.131025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247032+0000","last_change":"2023-10-30T12:35:06.182662+0000","last_active":"2023-10-30T12:35:07.247032+0000","last_peered":"2023-10-30T12:35:07.247032+0000","last_clean":"2023-10-30T12:35:07.247032+0000","last_became_active":"2023-10-30T12:35:06.182314+0000","last_became_peered":"2023-10-30T12:35:06.182314+0000","last_unstale":"2023-10-30T12:35:07.247032+0000","last_undegraded":"2023-10-30T12:35:07.247032+0000","last_fullsized":"2023-10-30T12:35:07.247032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:41:49.587856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249533+0000","last_change":"2023-10-30T12:35:06.214113+0000","last_active":"2023-10-30T12:35:07.249533+0000","last_peered":"2023-10-30T12:35:07.249533+0000","last_clean":"2023-10-30T12:35:07.249533+0000","last_became_active":"2023-10-30T12:35:06.213402+0000","last_became_peered":"2023-10-30T12:35:06.213402+0000","last_unstale":"2023-10-30T12:35:07.249533+0000","last_undegraded":"2023-10-30T12:35:07.249533+0000","last_fullsized":"2023-10-30T12:35:07.249533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:14:58.409296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247781+0000","last_change":"2023-10-30T12:35:06.188570+0000","last_active":"2023-10-30T12:35:07.247781+0000","last_peered":"2023-10-30T12:35:07.247781+0000","last_clean":"2023-10-30T12:35:07.247781+0000","last_became_active":"2023-10-30T12:35:06.188298+0000","last_became_peered":"2023-10-30T12:35:06.188298+0000","last_unstale":"2023-10-30T12:35:07.247781+0000","last_undegraded":"2023-10-30T12:35:07.247781+0000","last_fullsized":"2023-10-30T12:35:07.247781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:28.281254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163623+0000","last_change":"2023-10-30T12:35:06.208821+0000","last_active":"2023-10-30T12:35:07.163623+0000","last_peered":"2023-10-30T12:35:07.163623+0000","last_clean":"2023-10-30T12:35:07.163623+0000","last_became_active":"2023-10-30T12:35:06.202507+0000","last_became_peered":"2023-10-30T12:35:06.202507+0000","last_unstale":"2023-10-30T12:35:07.163623+0000","last_undegraded":"2023-10-30T12:35:07.163623+0000","last_fullsized":"2023-10-30T12:35:07.163623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:40:13.124316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159171+0000","last_change":"2023-10-30T12:35:06.170313+0000","last_active":"2023-10-30T12:35:07.159171+0000","last_peered":"2023-10-30T12:35:07.159171+0000","last_clean":"2023-10-30T12:35:07.159171+0000","last_became_active":"2023-10-30T12:35:06.170030+0000","last_became_peered":"2023-10-30T12:35:06.170030+0000","last_unstale":"2023-10-30T12:35:07.159171+0000","last_undegraded":"2023-10-30T12:35:07.159171+0000","last_fullsized":"2023-10-30T12:35:07.159171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:49:59.706208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247487+0000","last_change":"2023-10-30T12:35:06.189448+0000","last_active":"2023-10-30T12:35:07.247487+0000","last_peered":"2023-10-30T12:35:07.247487+0000","last_clean":"2023-10-30T12:35:07.247487+0000","last_became_active":"2023-10-30T12:35:06.189171+0000","last_became_peered":"2023-10-30T12:35:06.189171+0000","last_unstale":"2023-10-30T12:35:07.247487+0000","last_undegraded":"2023-10-30T12:35:07.247487+0000","last_fullsized":"2023-10-30T12:35:07.247487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:18:51.463273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160658+0000","last_change":"2023-10-30T12:35:06.183680+0000","last_active":"2023-10-30T12:35:07.160658+0000","last_peered":"2023-10-30T12:35:07.160658+0000","last_clean":"2023-10-30T12:35:07.160658+0000","last_became_active":"2023-10-30T12:35:06.183348+0000","last_became_peered":"2023-10-30T12:35:06.183348+0000","last_unstale":"2023-10-30T12:35:07.160658+0000","last_undegraded":"2023-10-30T12:35:07.160658+0000","last_fullsized":"2023-10-30T12:35:07.160658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:54:25.429897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250238+0000","last_change":"2023-10-30T12:35:06.206675+0000","last_active":"2023-10-30T12:35:07.250238+0000","last_peered":"2023-10-30T12:35:07.250238+0000","last_clean":"2023-10-30T12:35:07.250238+0000","last_became_active":"2023-10-30T12:35:06.196368+0000","last_became_peered":"2023-10-30T12:35:06.196368+0000","last_unstale":"2023-10-30T12:35:07.250238+0000","last_undegraded":"2023-10-30T12:35:07.250238+0000","last_fullsized":"2023-10-30T12:35:07.250238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:35.929311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246681+0000","last_change":"2023-10-30T12:35:06.214408+0000","last_active":"2023-10-30T12:35:07.246681+0000","last_peered":"2023-10-30T12:35:07.246681+0000","last_clean":"2023-10-30T12:35:07.246681+0000","last_became_active":"2023-10-30T12:35:06.213994+0000","last_became_peered":"2023-10-30T12:35:06.213994+0000","last_unstale":"2023-10-30T12:35:07.246681+0000","last_undegraded":"2023-10-30T12:35:07.246681+0000","last_fullsized":"2023-10-30T12:35:07.246681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:05:58.830698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162027+0000","last_change":"2023-10-30T12:35:06.198444+0000","last_active":"2023-10-30T12:35:07.162027+0000","last_peered":"2023-10-30T12:35:07.162027+0000","last_clean":"2023-10-30T12:35:07.162027+0000","last_became_active":"2023-10-30T12:35:06.197647+0000","last_became_peered":"2023-10-30T12:35:06.197647+0000","last_unstale":"2023-10-30T12:35:07.162027+0000","last_undegraded":"2023-10-30T12:35:07.162027+0000","last_fullsized":"2023-10-30T12:35:07.162027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:28:56.075420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163077+0000","last_change":"2023-10-30T12:35:06.168377+0000","last_active":"2023-10-30T12:35:07.163077+0000","last_peered":"2023-10-30T12:35:07.163077+0000","last_clean":"2023-10-30T12:35:07.163077+0000","last_became_active":"2023-10-30T12:35:06.167558+0000","last_became_peered":"2023-10-30T12:35:06.167558+0000","last_unstale":"2023-10-30T12:35:07.163077+0000","last_undegraded":"2023-10-30T12:35:07.163077+0000","last_fullsized":"2023-10-30T12:35:07.163077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:57:28.009164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.310153+0000","last_change":"2023-10-30T12:35:05.200844+0000","last_active":"2023-10-30T12:35:21.310153+0000","last_peered":"2023-10-30T12:35:21.310153+0000","last_clean":"2023-10-30T12:35:21.310153+0000","last_became_active":"2023-10-30T12:35:05.200169+0000","last_became_peered":"2023-10-30T12:35:05.200169+0000","last_unstale":"2023-10-30T12:35:21.310153+0000","last_undegraded":"2023-10-30T12:35:21.310153+0000","last_fullsized":"2023-10-30T12:35:21.310153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:07:19.571357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160718+0000","last_change":"2023-10-30T12:35:06.194964+0000","last_active":"2023-10-30T12:35:07.160718+0000","last_peered":"2023-10-30T12:35:07.160718+0000","last_clean":"2023-10-30T12:35:07.160718+0000","last_became_active":"2023-10-30T12:35:06.194689+0000","last_became_peered":"2023-10-30T12:35:06.194689+0000","last_unstale":"2023-10-30T12:35:07.160718+0000","last_undegraded":"2023-10-30T12:35:07.160718+0000","last_fullsized":"2023-10-30T12:35:07.160718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:30:53.410341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.336271+0000","last_change":"2023-10-30T12:35:05.183569+0000","last_active":"2023-10-30T12:35:27.336271+0000","last_peered":"2023-10-30T12:35:27.336271+0000","last_clean":"2023-10-30T12:35:27.336271+0000","last_became_active":"2023-10-30T12:35:05.182761+0000","last_became_peered":"2023-10-30T12:35:05.182761+0000","last_unstale":"2023-10-30T12:35:27.336271+0000","last_undegraded":"2023-10-30T12:35:27.336271+0000","last_fullsized":"2023-10-30T12:35:27.336271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:49:44.311814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.536230+0000","last_change":"2023-10-30T12:35:05.181278+0000","last_active":"2023-10-30T12:35:27.536230+0000","last_peered":"2023-10-30T12:35:27.536230+0000","last_clean":"2023-10-30T12:35:27.536230+0000","last_became_active":"2023-10-30T12:35:05.180730+0000","last_became_peered":"2023-10-30T12:35:05.180730+0000","last_unstale":"2023-10-30T12:35:27.536230+0000","last_undegraded":"2023-10-30T12:35:27.536230+0000","last_fullsized":"2023-10-30T12:35:27.536230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:07:10.828707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.246964+0000","last_change":"2023-10-30T12:35:06.180049+0000","last_active":"2023-10-30T12:35:07.246964+0000","last_peered":"2023-10-30T12:35:07.246964+0000","last_clean":"2023-10-30T12:35:07.246964+0000","last_became_active":"2023-10-30T12:35:06.179764+0000","last_became_peered":"2023-10-30T12:35:06.179764+0000","last_unstale":"2023-10-30T12:35:07.246964+0000","last_undegraded":"2023-10-30T12:35:07.246964+0000","last_fullsized":"2023-10-30T12:35:07.246964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:33.136727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249880+0000","last_change":"2023-10-30T12:35:05.202539+0000","last_active":"2023-10-30T12:35:07.249880+0000","last_peered":"2023-10-30T12:35:07.249880+0000","last_clean":"2023-10-30T12:35:07.249880+0000","last_became_active":"2023-10-30T12:35:05.201853+0000","last_became_peered":"2023-10-30T12:35:05.201853+0000","last_unstale":"2023-10-30T12:35:07.249880+0000","last_undegraded":"2023-10-30T12:35:07.249880+0000","last_fullsized":"2023-10-30T12:35:07.249880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:33:33.151433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.158869+0000","last_change":"2023-10-30T12:35:06.173567+0000","last_active":"2023-10-30T12:35:07.158869+0000","last_peered":"2023-10-30T12:35:07.158869+0000","last_clean":"2023-10-30T12:35:07.158869+0000","last_became_active":"2023-10-30T12:35:06.170179+0000","last_became_peered":"2023-10-30T12:35:06.170179+0000","last_unstale":"2023-10-30T12:35:07.158869+0000","last_undegraded":"2023-10-30T12:35:07.158869+0000","last_fullsized":"2023-10-30T12:35:07.158869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:39:07.880510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.306226+0000","last_change":"2023-10-30T12:35:05.198546+0000","last_active":"2023-10-30T12:35:21.306226+0000","last_peered":"2023-10-30T12:35:21.306226+0000","last_clean":"2023-10-30T12:35:21.306226+0000","last_became_active":"2023-10-30T12:35:05.197606+0000","last_became_peered":"2023-10-30T12:35:05.197606+0000","last_unstale":"2023-10-30T12:35:21.306226+0000","last_undegraded":"2023-10-30T12:35:21.306226+0000","last_fullsized":"2023-10-30T12:35:21.306226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:26:57.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251322+0000","last_change":"2023-10-30T12:35:06.226044+0000","last_active":"2023-10-30T12:35:07.251322+0000","last_peered":"2023-10-30T12:35:07.251322+0000","last_clean":"2023-10-30T12:35:07.251322+0000","last_became_active":"2023-10-30T12:35:06.225392+0000","last_became_peered":"2023-10-30T12:35:06.225392+0000","last_unstale":"2023-10-30T12:35:07.251322+0000","last_undegraded":"2023-10-30T12:35:07.251322+0000","last_fullsized":"2023-10-30T12:35:07.251322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:13:10.796549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.313047+0000","last_change":"2023-10-30T12:35:05.197614+0000","last_active":"2023-10-30T12:35:21.313047+0000","last_peered":"2023-10-30T12:35:21.313047+0000","last_clean":"2023-10-30T12:35:21.313047+0000","last_became_active":"2023-10-30T12:35:05.196899+0000","last_became_peered":"2023-10-30T12:35:05.196899+0000","last_unstale":"2023-10-30T12:35:21.313047+0000","last_undegraded":"2023-10-30T12:35:21.313047+0000","last_fullsized":"2023-10-30T12:35:21.313047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:21:00.152426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160715+0000","last_change":"2023-10-30T12:35:06.222344+0000","last_active":"2023-10-30T12:35:07.160715+0000","last_peered":"2023-10-30T12:35:07.160715+0000","last_clean":"2023-10-30T12:35:07.160715+0000","last_became_active":"2023-10-30T12:35:06.222057+0000","last_became_peered":"2023-10-30T12:35:06.222057+0000","last_unstale":"2023-10-30T12:35:07.160715+0000","last_undegraded":"2023-10-30T12:35:07.160715+0000","last_fullsized":"2023-10-30T12:35:07.160715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:50:46.691223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.201054+0000","last_change":"2023-10-30T12:35:05.197501+0000","last_active":"2023-10-30T12:35:11.201054+0000","last_peered":"2023-10-30T12:35:11.201054+0000","last_clean":"2023-10-30T12:35:11.201054+0000","last_became_active":"2023-10-30T12:35:05.197190+0000","last_became_peered":"2023-10-30T12:35:05.197190+0000","last_unstale":"2023-10-30T12:35:11.201054+0000","last_undegraded":"2023-10-30T12:35:11.201054+0000","last_fullsized":"2023-10-30T12:35:11.201054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:53:31.121721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.164505+0000","last_change":"2023-10-30T12:35:06.204885+0000","last_active":"2023-10-30T12:35:07.164505+0000","last_peered":"2023-10-30T12:35:07.164505+0000","last_clean":"2023-10-30T12:35:07.164505+0000","last_became_active":"2023-10-30T12:35:06.204122+0000","last_became_peered":"2023-10-30T12:35:06.204122+0000","last_unstale":"2023-10-30T12:35:07.164505+0000","last_undegraded":"2023-10-30T12:35:07.164505+0000","last_fullsized":"2023-10-30T12:35:07.164505+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:27:18.433575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'11","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:35.466455+0000","last_change":"2023-10-30T12:35:05.180005+0000","last_active":"2023-10-30T12:35:35.466455+0000","last_peered":"2023-10-30T12:35:35.466455+0000","last_clean":"2023-10-30T12:35:35.466455+0000","last_became_active":"2023-10-30T12:35:05.179547+0000","last_became_peered":"2023-10-30T12:35:05.179547+0000","last_unstale":"2023-10-30T12:35:35.466455+0000","last_undegraded":"2023-10-30T12:35:35.466455+0000","last_fullsized":"2023-10-30T12:35:35.466455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:13:03.137433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2685,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249565+0000","last_change":"2023-10-30T12:35:06.216619+0000","last_active":"2023-10-30T12:35:07.249565+0000","last_peered":"2023-10-30T12:35:07.249565+0000","last_clean":"2023-10-30T12:35:07.249565+0000","last_became_active":"2023-10-30T12:35:06.215128+0000","last_became_peered":"2023-10-30T12:35:06.215128+0000","last_unstale":"2023-10-30T12:35:07.249565+0000","last_undegraded":"2023-10-30T12:35:07.249565+0000","last_fullsized":"2023-10-30T12:35:07.249565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:48:45.673159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.335411+0000","last_change":"2023-10-30T12:35:05.190041+0000","last_active":"2023-10-30T12:35:27.335411+0000","last_peered":"2023-10-30T12:35:27.335411+0000","last_clean":"2023-10-30T12:35:27.335411+0000","last_became_active":"2023-10-30T12:35:05.189573+0000","last_became_peered":"2023-10-30T12:35:05.189573+0000","last_unstale":"2023-10-30T12:35:27.335411+0000","last_undegraded":"2023-10-30T12:35:27.335411+0000","last_fullsized":"2023-10-30T12:35:27.335411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:02:57.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160412+0000","last_change":"2023-10-30T12:35:06.217334+0000","last_active":"2023-10-30T12:35:07.160412+0000","last_peered":"2023-10-30T12:35:07.160412+0000","last_clean":"2023-10-30T12:35:07.160412+0000","last_became_active":"2023-10-30T12:35:06.216885+0000","last_became_peered":"2023-10-30T12:35:06.216885+0000","last_unstale":"2023-10-30T12:35:07.160412+0000","last_undegraded":"2023-10-30T12:35:07.160412+0000","last_fullsized":"2023-10-30T12:35:07.160412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:38:26.745898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.313580+0000","last_change":"2023-10-30T12:35:05.203198+0000","last_active":"2023-10-30T12:35:21.313580+0000","last_peered":"2023-10-30T12:35:21.313580+0000","last_clean":"2023-10-30T12:35:21.313580+0000","last_became_active":"2023-10-30T12:35:05.202612+0000","last_became_peered":"2023-10-30T12:35:05.202612+0000","last_unstale":"2023-10-30T12:35:21.313580+0000","last_undegraded":"2023-10-30T12:35:21.313580+0000","last_fullsized":"2023-10-30T12:35:21.313580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:55.986236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249574+0000","last_change":"2023-10-30T12:35:06.216307+0000","last_active":"2023-10-30T12:35:07.249574+0000","last_peered":"2023-10-30T12:35:07.249574+0000","last_clean":"2023-10-30T12:35:07.249574+0000","last_became_active":"2023-10-30T12:35:06.215553+0000","last_became_peered":"2023-10-30T12:35:06.215553+0000","last_unstale":"2023-10-30T12:35:07.249574+0000","last_undegraded":"2023-10-30T12:35:07.249574+0000","last_fullsized":"2023-10-30T12:35:07.249574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:46:25.396759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.336548+0000","last_change":"2023-10-30T12:35:05.181882+0000","last_active":"2023-10-30T12:35:27.336548+0000","last_peered":"2023-10-30T12:35:27.336548+0000","last_clean":"2023-10-30T12:35:27.336548+0000","last_became_active":"2023-10-30T12:35:05.181115+0000","last_became_peered":"2023-10-30T12:35:05.181115+0000","last_unstale":"2023-10-30T12:35:27.336548+0000","last_undegraded":"2023-10-30T12:35:27.336548+0000","last_fullsized":"2023-10-30T12:35:27.336548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:18:55.496940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162013+0000","last_change":"2023-10-30T12:35:06.198990+0000","last_active":"2023-10-30T12:35:07.162013+0000","last_peered":"2023-10-30T12:35:07.162013+0000","last_clean":"2023-10-30T12:35:07.162013+0000","last_became_active":"2023-10-30T12:35:06.197722+0000","last_became_peered":"2023-10-30T12:35:06.197722+0000","last_unstale":"2023-10-30T12:35:07.162013+0000","last_undegraded":"2023-10-30T12:35:07.162013+0000","last_fullsized":"2023-10-30T12:35:07.162013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:56:51.757976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'15","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.338386+0000","last_change":"2023-10-30T12:35:05.169455+0000","last_active":"2023-10-30T12:35:27.338386+0000","last_peered":"2023-10-30T12:35:27.338386+0000","last_clean":"2023-10-30T12:35:27.338386+0000","last_became_active":"2023-10-30T12:35:05.169130+0000","last_became_peered":"2023-10-30T12:35:05.169130+0000","last_unstale":"2023-10-30T12:35:27.338386+0000","last_undegraded":"2023-10-30T12:35:27.338386+0000","last_fullsized":"2023-10-30T12:35:27.338386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:59:14.096315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249183+0000","last_change":"2023-10-30T12:35:06.215545+0000","last_active":"2023-10-30T12:35:07.249183+0000","last_peered":"2023-10-30T12:35:07.249183+0000","last_clean":"2023-10-30T12:35:07.249183+0000","last_became_active":"2023-10-30T12:35:06.214839+0000","last_became_peered":"2023-10-30T12:35:06.214839+0000","last_unstale":"2023-10-30T12:35:07.249183+0000","last_undegraded":"2023-10-30T12:35:07.249183+0000","last_fullsized":"2023-10-30T12:35:07.249183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:54:34.206981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.199110+0000","last_change":"2023-10-30T12:35:05.189361+0000","last_active":"2023-10-30T12:35:11.199110+0000","last_peered":"2023-10-30T12:35:11.199110+0000","last_clean":"2023-10-30T12:35:11.199110+0000","last_became_active":"2023-10-30T12:35:05.188862+0000","last_became_peered":"2023-10-30T12:35:05.188862+0000","last_unstale":"2023-10-30T12:35:11.199110+0000","last_undegraded":"2023-10-30T12:35:11.199110+0000","last_fullsized":"2023-10-30T12:35:11.199110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:13.319732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250712+0000","last_change":"2023-10-30T12:35:06.223118+0000","last_active":"2023-10-30T12:35:07.250712+0000","last_peered":"2023-10-30T12:35:07.250712+0000","last_clean":"2023-10-30T12:35:07.250712+0000","last_became_active":"2023-10-30T12:35:06.216533+0000","last_became_peered":"2023-10-30T12:35:06.216533+0000","last_unstale":"2023-10-30T12:35:07.250712+0000","last_undegraded":"2023-10-30T12:35:07.250712+0000","last_fullsized":"2023-10-30T12:35:07.250712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:32:29.374526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.195596+0000","last_change":"2023-10-30T12:35:05.177609+0000","last_active":"2023-10-30T12:35:11.195596+0000","last_peered":"2023-10-30T12:35:11.195596+0000","last_clean":"2023-10-30T12:35:11.195596+0000","last_became_active":"2023-10-30T12:35:05.177316+0000","last_became_peered":"2023-10-30T12:35:05.177316+0000","last_unstale":"2023-10-30T12:35:11.195596+0000","last_undegraded":"2023-10-30T12:35:11.195596+0000","last_fullsized":"2023-10-30T12:35:11.195596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:46.820925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.251056+0000","last_change":"2023-10-30T12:35:06.222283+0000","last_active":"2023-10-30T12:35:07.251056+0000","last_peered":"2023-10-30T12:35:07.251056+0000","last_clean":"2023-10-30T12:35:07.251056+0000","last_became_active":"2023-10-30T12:35:06.221701+0000","last_became_peered":"2023-10-30T12:35:06.221701+0000","last_unstale":"2023-10-30T12:35:07.251056+0000","last_undegraded":"2023-10-30T12:35:07.251056+0000","last_fullsized":"2023-10-30T12:35:07.251056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:37:32.956627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159413+0000","last_change":"2023-10-30T12:35:05.171787+0000","last_active":"2023-10-30T12:35:07.159413+0000","last_peered":"2023-10-30T12:35:07.159413+0000","last_clean":"2023-10-30T12:35:07.159413+0000","last_became_active":"2023-10-30T12:35:05.171491+0000","last_became_peered":"2023-10-30T12:35:05.171491+0000","last_unstale":"2023-10-30T12:35:07.159413+0000","last_undegraded":"2023-10-30T12:35:07.159413+0000","last_fullsized":"2023-10-30T12:35:07.159413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:43:15.331175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250517+0000","last_change":"2023-10-30T12:35:06.214464+0000","last_active":"2023-10-30T12:35:07.250517+0000","last_peered":"2023-10-30T12:35:07.250517+0000","last_clean":"2023-10-30T12:35:07.250517+0000","last_became_active":"2023-10-30T12:35:06.213493+0000","last_became_peered":"2023-10-30T12:35:06.213493+0000","last_unstale":"2023-10-30T12:35:07.250517+0000","last_undegraded":"2023-10-30T12:35:07.250517+0000","last_fullsized":"2023-10-30T12:35:07.250517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:00:18.624720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.337571+0000","last_change":"2023-10-30T12:35:05.197811+0000","last_active":"2023-10-30T12:35:27.337571+0000","last_peered":"2023-10-30T12:35:27.337571+0000","last_clean":"2023-10-30T12:35:27.337571+0000","last_became_active":"2023-10-30T12:35:05.197453+0000","last_became_peered":"2023-10-30T12:35:05.197453+0000","last_unstale":"2023-10-30T12:35:27.337571+0000","last_undegraded":"2023-10-30T12:35:27.337571+0000","last_fullsized":"2023-10-30T12:35:27.337571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:16:14.550887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.249901+0000","last_change":"2023-10-30T12:35:06.220236+0000","last_active":"2023-10-30T12:35:07.249901+0000","last_peered":"2023-10-30T12:35:07.249901+0000","last_clean":"2023-10-30T12:35:07.249901+0000","last_became_active":"2023-10-30T12:35:06.219526+0000","last_became_peered":"2023-10-30T12:35:06.219526+0000","last_unstale":"2023-10-30T12:35:07.249901+0000","last_undegraded":"2023-10-30T12:35:07.249901+0000","last_fullsized":"2023-10-30T12:35:07.249901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:00:25.297590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.186752+0000","last_change":"2023-10-30T12:34:55.206976+0000","last_active":"2023-10-30T12:35:06.186752+0000","last_peered":"2023-10-30T12:35:06.186752+0000","last_clean":"2023-10-30T12:35:06.186752+0000","last_became_active":"2023-10-30T12:34:55.205923+0000","last_became_peered":"2023-10-30T12:34:55.205923+0000","last_unstale":"2023-10-30T12:35:06.186752+0000","last_undegraded":"2023-10-30T12:35:06.186752+0000","last_fullsized":"2023-10-30T12:35:06.186752+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_clean_scrub_stamp":"2023-10-30T12:34:53.444020+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:25:28.782111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.313312+0000","last_change":"2023-10-30T12:35:05.197974+0000","last_active":"2023-10-30T12:35:21.313312+0000","last_peered":"2023-10-30T12:35:21.313312+0000","last_clean":"2023-10-30T12:35:21.313312+0000","last_became_active":"2023-10-30T12:35:05.197201+0000","last_became_peered":"2023-10-30T12:35:05.197201+0000","last_unstale":"2023-10-30T12:35:21.313312+0000","last_undegraded":"2023-10-30T12:35:21.313312+0000","last_fullsized":"2023-10-30T12:35:21.313312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:39:47.119091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247224+0000","last_change":"2023-10-30T12:35:06.217973+0000","last_active":"2023-10-30T12:35:07.247224+0000","last_peered":"2023-10-30T12:35:07.247224+0000","last_clean":"2023-10-30T12:35:07.247224+0000","last_became_active":"2023-10-30T12:35:06.217687+0000","last_became_peered":"2023-10-30T12:35:06.217687+0000","last_unstale":"2023-10-30T12:35:07.247224+0000","last_undegraded":"2023-10-30T12:35:07.247224+0000","last_fullsized":"2023-10-30T12:35:07.247224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:28.392113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.333670+0000","last_change":"2023-10-30T12:35:05.208339+0000","last_active":"2023-10-30T12:35:27.333670+0000","last_peered":"2023-10-30T12:35:27.333670+0000","last_clean":"2023-10-30T12:35:27.333670+0000","last_became_active":"2023-10-30T12:35:05.207828+0000","last_became_peered":"2023-10-30T12:35:05.207828+0000","last_unstale":"2023-10-30T12:35:27.333670+0000","last_undegraded":"2023-10-30T12:35:27.333670+0000","last_fullsized":"2023-10-30T12:35:27.333670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:00.649048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160255+0000","last_change":"2023-10-30T12:35:06.195094+0000","last_active":"2023-10-30T12:35:07.160255+0000","last_peered":"2023-10-30T12:35:07.160255+0000","last_clean":"2023-10-30T12:35:07.160255+0000","last_became_active":"2023-10-30T12:35:06.194791+0000","last_became_peered":"2023-10-30T12:35:06.194791+0000","last_unstale":"2023-10-30T12:35:07.160255+0000","last_undegraded":"2023-10-30T12:35:07.160255+0000","last_fullsized":"2023-10-30T12:35:07.160255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:41:24.310349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.336946+0000","last_change":"2023-10-30T12:35:05.176296+0000","last_active":"2023-10-30T12:35:27.336946+0000","last_peered":"2023-10-30T12:35:27.336946+0000","last_clean":"2023-10-30T12:35:27.336946+0000","last_became_active":"2023-10-30T12:35:05.175959+0000","last_became_peered":"2023-10-30T12:35:05.175959+0000","last_unstale":"2023-10-30T12:35:27.336946+0000","last_undegraded":"2023-10-30T12:35:27.336946+0000","last_fullsized":"2023-10-30T12:35:27.336946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:53.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159052+0000","last_change":"2023-10-30T12:35:06.167250+0000","last_active":"2023-10-30T12:35:07.159052+0000","last_peered":"2023-10-30T12:35:07.159052+0000","last_clean":"2023-10-30T12:35:07.159052+0000","last_became_active":"2023-10-30T12:35:06.164711+0000","last_became_peered":"2023-10-30T12:35:06.164711+0000","last_unstale":"2023-10-30T12:35:07.159052+0000","last_undegraded":"2023-10-30T12:35:07.159052+0000","last_fullsized":"2023-10-30T12:35:07.159052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:48:18.710410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250361+0000","last_change":"2023-10-30T12:35:05.198964+0000","last_active":"2023-10-30T12:35:07.250361+0000","last_peered":"2023-10-30T12:35:07.250361+0000","last_clean":"2023-10-30T12:35:07.250361+0000","last_became_active":"2023-10-30T12:35:05.196873+0000","last_became_peered":"2023-10-30T12:35:05.196873+0000","last_unstale":"2023-10-30T12:35:07.250361+0000","last_undegraded":"2023-10-30T12:35:07.250361+0000","last_fullsized":"2023-10-30T12:35:07.250361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:47.793585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163526+0000","last_change":"2023-10-30T12:35:06.184123+0000","last_active":"2023-10-30T12:35:07.163526+0000","last_peered":"2023-10-30T12:35:07.163526+0000","last_clean":"2023-10-30T12:35:07.163526+0000","last_became_active":"2023-10-30T12:35:06.183225+0000","last_became_peered":"2023-10-30T12:35:06.183225+0000","last_unstale":"2023-10-30T12:35:07.163526+0000","last_undegraded":"2023-10-30T12:35:07.163526+0000","last_fullsized":"2023-10-30T12:35:07.163526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:03:47.097151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-30T12:35:06.543947+0000","last_change":"2023-10-30T12:35:05.177571+0000","last_active":"2023-10-30T12:35:06.543947+0000","last_peered":"2023-10-30T12:35:06.543947+0000","last_clean":"2023-10-30T12:35:06.543947+0000","last_became_active":"2023-10-30T12:35:05.177266+0000","last_became_peered":"2023-10-30T12:35:05.177266+0000","last_unstale":"2023-10-30T12:35:06.543947+0000","last_undegraded":"2023-10-30T12:35:06.543947+0000","last_fullsized":"2023-10-30T12:35:06.543947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:14.601934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250949+0000","last_change":"2023-10-30T12:35:06.206033+0000","last_active":"2023-10-30T12:35:07.250949+0000","last_peered":"2023-10-30T12:35:07.250949+0000","last_clean":"2023-10-30T12:35:07.250949+0000","last_became_active":"2023-10-30T12:35:06.196307+0000","last_became_peered":"2023-10-30T12:35:06.196307+0000","last_unstale":"2023-10-30T12:35:07.250949+0000","last_undegraded":"2023-10-30T12:35:07.250949+0000","last_fullsized":"2023-10-30T12:35:07.250949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:18:28.848259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.460771+0000","last_change":"2023-10-30T12:35:05.207594+0000","last_active":"2023-10-30T12:35:07.460771+0000","last_peered":"2023-10-30T12:35:07.460771+0000","last_clean":"2023-10-30T12:35:07.460771+0000","last_became_active":"2023-10-30T12:35:05.206930+0000","last_became_peered":"2023-10-30T12:35:05.206930+0000","last_unstale":"2023-10-30T12:35:07.460771+0000","last_undegraded":"2023-10-30T12:35:07.460771+0000","last_fullsized":"2023-10-30T12:35:07.460771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:23:46.519077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159110+0000","last_change":"2023-10-30T12:35:06.165126+0000","last_active":"2023-10-30T12:35:07.159110+0000","last_peered":"2023-10-30T12:35:07.159110+0000","last_clean":"2023-10-30T12:35:07.159110+0000","last_became_active":"2023-10-30T12:35:06.164733+0000","last_became_peered":"2023-10-30T12:35:06.164733+0000","last_unstale":"2023-10-30T12:35:07.159110+0000","last_undegraded":"2023-10-30T12:35:07.159110+0000","last_fullsized":"2023-10-30T12:35:07.159110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:29.336555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.337400+0000","last_change":"2023-10-30T12:35:05.185034+0000","last_active":"2023-10-30T12:35:27.337400+0000","last_peered":"2023-10-30T12:35:27.337400+0000","last_clean":"2023-10-30T12:35:27.337400+0000","last_became_active":"2023-10-30T12:35:05.184695+0000","last_became_peered":"2023-10-30T12:35:05.184695+0000","last_unstale":"2023-10-30T12:35:27.337400+0000","last_undegraded":"2023-10-30T12:35:27.337400+0000","last_fullsized":"2023-10-30T12:35:27.337400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:01:42.019128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247746+0000","last_change":"2023-10-30T12:35:06.204707+0000","last_active":"2023-10-30T12:35:07.247746+0000","last_peered":"2023-10-30T12:35:07.247746+0000","last_clean":"2023-10-30T12:35:07.247746+0000","last_became_active":"2023-10-30T12:35:06.204422+0000","last_became_peered":"2023-10-30T12:35:06.204422+0000","last_unstale":"2023-10-30T12:35:07.247746+0000","last_undegraded":"2023-10-30T12:35:07.247746+0000","last_fullsized":"2023-10-30T12:35:07.247746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:58:16.758575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.304904+0000","last_change":"2023-10-30T12:35:05.200871+0000","last_active":"2023-10-30T12:35:21.304904+0000","last_peered":"2023-10-30T12:35:21.304904+0000","last_clean":"2023-10-30T12:35:21.304904+0000","last_became_active":"2023-10-30T12:35:05.200449+0000","last_became_peered":"2023-10-30T12:35:05.200449+0000","last_unstale":"2023-10-30T12:35:21.304904+0000","last_undegraded":"2023-10-30T12:35:21.304904+0000","last_fullsized":"2023-10-30T12:35:21.304904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:39:20.867587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160562+0000","last_change":"2023-10-30T12:35:06.180850+0000","last_active":"2023-10-30T12:35:07.160562+0000","last_peered":"2023-10-30T12:35:07.160562+0000","last_clean":"2023-10-30T12:35:07.160562+0000","last_became_active":"2023-10-30T12:35:06.179412+0000","last_became_peered":"2023-10-30T12:35:06.179412+0000","last_unstale":"2023-10-30T12:35:07.160562+0000","last_undegraded":"2023-10-30T12:35:07.160562+0000","last_fullsized":"2023-10-30T12:35:07.160562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:47:44.055205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.304568+0000","last_change":"2023-10-30T12:35:05.207392+0000","last_active":"2023-10-30T12:35:21.304568+0000","last_peered":"2023-10-30T12:35:21.304568+0000","last_clean":"2023-10-30T12:35:21.304568+0000","last_became_active":"2023-10-30T12:35:05.206607+0000","last_became_peered":"2023-10-30T12:35:05.206607+0000","last_unstale":"2023-10-30T12:35:21.304568+0000","last_undegraded":"2023-10-30T12:35:21.304568+0000","last_fullsized":"2023-10-30T12:35:21.304568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:36.758740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.163445+0000","last_change":"2023-10-30T12:35:06.182042+0000","last_active":"2023-10-30T12:35:07.163445+0000","last_peered":"2023-10-30T12:35:07.163445+0000","last_clean":"2023-10-30T12:35:07.163445+0000","last_became_active":"2023-10-30T12:35:06.181256+0000","last_became_peered":"2023-10-30T12:35:06.181256+0000","last_unstale":"2023-10-30T12:35:07.163445+0000","last_undegraded":"2023-10-30T12:35:07.163445+0000","last_fullsized":"2023-10-30T12:35:07.163445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:13:08.109392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.305680+0000","last_change":"2023-10-30T12:35:05.201024+0000","last_active":"2023-10-30T12:35:21.305680+0000","last_peered":"2023-10-30T12:35:21.305680+0000","last_clean":"2023-10-30T12:35:21.305680+0000","last_became_active":"2023-10-30T12:35:05.200724+0000","last_became_peered":"2023-10-30T12:35:05.200724+0000","last_unstale":"2023-10-30T12:35:21.305680+0000","last_undegraded":"2023-10-30T12:35:21.305680+0000","last_fullsized":"2023-10-30T12:35:21.305680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:24.746469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247249+0000","last_change":"2023-10-30T12:35:06.174636+0000","last_active":"2023-10-30T12:35:07.247249+0000","last_peered":"2023-10-30T12:35:07.247249+0000","last_clean":"2023-10-30T12:35:07.247249+0000","last_became_active":"2023-10-30T12:35:06.173996+0000","last_became_peered":"2023-10-30T12:35:06.173996+0000","last_unstale":"2023-10-30T12:35:07.247249+0000","last_undegraded":"2023-10-30T12:35:07.247249+0000","last_fullsized":"2023-10-30T12:35:07.247249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:01:38.503494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.159100+0000","last_change":"2023-10-30T12:35:05.164079+0000","last_active":"2023-10-30T12:35:07.159100+0000","last_peered":"2023-10-30T12:35:07.159100+0000","last_clean":"2023-10-30T12:35:07.159100+0000","last_became_active":"2023-10-30T12:35:05.163794+0000","last_became_peered":"2023-10-30T12:35:05.163794+0000","last_unstale":"2023-10-30T12:35:07.159100+0000","last_undegraded":"2023-10-30T12:35:07.159100+0000","last_fullsized":"2023-10-30T12:35:07.159100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:19.795656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247344+0000","last_change":"2023-10-30T12:35:06.183531+0000","last_active":"2023-10-30T12:35:07.247344+0000","last_peered":"2023-10-30T12:35:07.247344+0000","last_clean":"2023-10-30T12:35:07.247344+0000","last_became_active":"2023-10-30T12:35:06.183226+0000","last_became_peered":"2023-10-30T12:35:06.183226+0000","last_unstale":"2023-10-30T12:35:07.247344+0000","last_undegraded":"2023-10-30T12:35:07.247344+0000","last_fullsized":"2023-10-30T12:35:07.247344+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:46.768356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:21.303852+0000","last_change":"2023-10-30T12:35:05.206995+0000","last_active":"2023-10-30T12:35:21.303852+0000","last_peered":"2023-10-30T12:35:21.303852+0000","last_clean":"2023-10-30T12:35:21.303852+0000","last_became_active":"2023-10-30T12:35:05.206518+0000","last_became_peered":"2023-10-30T12:35:05.206518+0000","last_unstale":"2023-10-30T12:35:21.303852+0000","last_undegraded":"2023-10-30T12:35:21.303852+0000","last_fullsized":"2023-10-30T12:35:21.303852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:41:33.703261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.162528+0000","last_change":"2023-10-30T12:35:06.199557+0000","last_active":"2023-10-30T12:35:07.162528+0000","last_peered":"2023-10-30T12:35:07.162528+0000","last_clean":"2023-10-30T12:35:07.162528+0000","last_became_active":"2023-10-30T12:35:06.197861+0000","last_became_peered":"2023-10-30T12:35:06.197861+0000","last_unstale":"2023-10-30T12:35:07.162528+0000","last_undegraded":"2023-10-30T12:35:07.162528+0000","last_fullsized":"2023-10-30T12:35:07.162528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:30:46.092876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250766+0000","last_change":"2023-10-30T12:35:06.213279+0000","last_active":"2023-10-30T12:35:07.250766+0000","last_peered":"2023-10-30T12:35:07.250766+0000","last_clean":"2023-10-30T12:35:07.250766+0000","last_became_active":"2023-10-30T12:35:06.212548+0000","last_became_peered":"2023-10-30T12:35:06.212548+0000","last_unstale":"2023-10-30T12:35:07.250766+0000","last_undegraded":"2023-10-30T12:35:07.250766+0000","last_fullsized":"2023-10-30T12:35:07.250766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:01:57.127208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.200195+0000","last_change":"2023-10-30T12:35:05.206980+0000","last_active":"2023-10-30T12:35:11.200195+0000","last_peered":"2023-10-30T12:35:11.200195+0000","last_clean":"2023-10-30T12:35:11.200195+0000","last_became_active":"2023-10-30T12:35:05.206559+0000","last_became_peered":"2023-10-30T12:35:05.206559+0000","last_unstale":"2023-10-30T12:35:11.200195+0000","last_undegraded":"2023-10-30T12:35:11.200195+0000","last_fullsized":"2023-10-30T12:35:11.200195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:40.259622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.250210+0000","last_change":"2023-10-30T12:35:06.213053+0000","last_active":"2023-10-30T12:35:07.250210+0000","last_peered":"2023-10-30T12:35:07.250210+0000","last_clean":"2023-10-30T12:35:07.250210+0000","last_became_active":"2023-10-30T12:35:06.212505+0000","last_became_peered":"2023-10-30T12:35:06.212505+0000","last_unstale":"2023-10-30T12:35:07.250210+0000","last_undegraded":"2023-10-30T12:35:07.250210+0000","last_fullsized":"2023-10-30T12:35:07.250210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:44:02.073639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.333919+0000","last_change":"2023-10-30T12:35:05.207851+0000","last_active":"2023-10-30T12:35:27.333919+0000","last_peered":"2023-10-30T12:35:27.333919+0000","last_clean":"2023-10-30T12:35:27.333919+0000","last_became_active":"2023-10-30T12:35:05.206879+0000","last_became_peered":"2023-10-30T12:35:05.206879+0000","last_unstale":"2023-10-30T12:35:27.333919+0000","last_undegraded":"2023-10-30T12:35:27.333919+0000","last_fullsized":"2023-10-30T12:35:27.333919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:43:15.717507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.160637+0000","last_change":"2023-10-30T12:35:06.222373+0000","last_active":"2023-10-30T12:35:07.160637+0000","last_peered":"2023-10-30T12:35:07.160637+0000","last_clean":"2023-10-30T12:35:07.160637+0000","last_became_active":"2023-10-30T12:35:06.222094+0000","last_became_peered":"2023-10-30T12:35:06.222094+0000","last_unstale":"2023-10-30T12:35:07.160637+0000","last_undegraded":"2023-10-30T12:35:07.160637+0000","last_fullsized":"2023-10-30T12:35:07.160637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:42:32.482862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:11.195154+0000","last_change":"2023-10-30T12:35:05.174319+0000","last_active":"2023-10-30T12:35:11.195154+0000","last_peered":"2023-10-30T12:35:11.195154+0000","last_clean":"2023-10-30T12:35:11.195154+0000","last_became_active":"2023-10-30T12:35:05.173984+0000","last_became_peered":"2023-10-30T12:35:05.173984+0000","last_unstale":"2023-10-30T12:35:11.195154+0000","last_undegraded":"2023-10-30T12:35:11.195154+0000","last_fullsized":"2023-10-30T12:35:11.195154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:11.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:07.247593+0000","last_change":"2023-10-30T12:35:06.198881+0000","last_active":"2023-10-30T12:35:07.247593+0000","last_peered":"2023-10-30T12:35:07.247593+0000","last_clean":"2023-10-30T12:35:07.247593+0000","last_became_active":"2023-10-30T12:35:06.198504+0000","last_became_peered":"2023-10-30T12:35:06.198504+0000","last_unstale":"2023-10-30T12:35:07.247593+0000","last_undegraded":"2023-10-30T12:35:07.247593+0000","last_fullsized":"2023-10-30T12:35:07.247593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:06:33.845147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'9","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-30T12:35:27.335834+0000","last_change":"2023-10-30T12:35:05.200926+0000","last_active":"2023-10-30T12:35:27.335834+0000","last_peered":"2023-10-30T12:35:27.335834+0000","last_clean":"2023-10-30T12:35:27.335834+0000","last_became_active":"2023-10-30T12:35:05.200623+0000","last_became_peered":"2023-10-30T12:35:05.200623+0000","last_unstale":"2023-10-30T12:35:27.335834+0000","last_undegraded":"2023-10-30T12:35:27.335834+0000","last_fullsized":"2023-10-30T12:35:27.335834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:38.942922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6553,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":13106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":839680,"data_stored":670806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":27654605},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542157,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":82227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]}]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":241664,"data_stored":79578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":79682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":669892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":80242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":249856,"data_stored":82519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":249856,"data_stored":79808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:35:40.569 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:35:40.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.750+0000 7fbe7510f640 1 Processor -- start 2023-10-30T12:35:40.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.750+0000 7fbe7510f640 1 -- start start 2023-10-30T12:35:40.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.750+0000 7fbe7510f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbe700cf160 0x7fbe700cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:40.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.750+0000 7fbe7510f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe700cb800 con 0x7fbe7015d6f0 2023-10-30T12:35:40.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.750+0000 7fbe7510f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe700cb330 con 0x7fbe701538a0 2023-10-30T12:35:40.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.750+0000 7fbe7510f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe700cad70 con 0x7fbe700cf160 2023-10-30T12:35:40.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.750+0000 7fbe67fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbe7015d6f0 0x7fbe7015a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54496/0 (socket says 172.21.15.12:54496) 2023-10-30T12:35:40.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.750+0000 7fbe67fff640 1 -- 172.21.15.12:0/490766539 learned_addr learned my addr 172.21.15.12:0/490766539 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:40.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.750+0000 7fbe6f59e640 1 --2- 172.21.15.12:0/490766539 >> v2:172.21.15.154:3300/0 conn(0x7fbe700cf160 0x7fbe700cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:40.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.750+0000 7fbe6e59c640 1 -- 172.21.15.12:0/490766539 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581264627 0 0) 0x7fbe700cb800 con 0x7fbe7015d6f0 2023-10-30T12:35:40.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.750+0000 7fbe6e59c640 1 -- 172.21.15.12:0/490766539 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe5c003530 con 0x7fbe7015d6f0 2023-10-30T12:35:40.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.750+0000 7fbe6f59e640 1 -- 172.21.15.12:0/490766539 >> v1:172.21.15.154:6789/0 conn(0x7fbe701538a0 legacy=0x7fbe700ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:40.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.750+0000 7fbe6f59e640 1 -- 172.21.15.12:0/490766539 >> v1:172.21.15.12:6789/0 conn(0x7fbe7015d6f0 legacy=0x7fbe7015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:40.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.750+0000 7fbe6f59e640 1 -- 172.21.15.12:0/490766539 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe700cb050 con 0x7fbe700cf160 2023-10-30T12:35:40.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.754+0000 7fbe6f59e640 1 --2- 172.21.15.12:0/490766539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe700cf160 0x7fbe700cf590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009d90 tx=0x7fbe6801f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=639927ff29bebb1e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:40.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.754+0000 7fbe6e59c640 1 -- 172.21.15.12:0/490766539 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe680af600 con 0x7fbe700cf160 2023-10-30T12:35:40.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.754+0000 7fbe6e59c640 1 -- 172.21.15.12:0/490766539 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe680ab050 con 0x7fbe700cf160 2023-10-30T12:35:40.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.754+0000 7fbe6e59c640 1 -- 172.21.15.12:0/490766539 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe680af960 con 0x7fbe700cf160 2023-10-30T12:35:40.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.754+0000 7fbe7510f640 1 -- 172.21.15.12:0/490766539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe700cf160 msgr2=0x7fbe700cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:40.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.754+0000 7fbe7510f640 1 --2- 172.21.15.12:0/490766539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe700cf160 0x7fbe700cf590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009d90 tx=0x7fbe6801f130 comp rx=0 tx=0).stop 2023-10-30T12:35:40.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.754+0000 7fbe7510f640 1 -- 172.21.15.12:0/490766539 shutdown_connections 2023-10-30T12:35:40.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.754+0000 7fbe7510f640 1 --2- 172.21.15.12:0/490766539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe700cf160 0x7fbe700cf590 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.754+0000 7fbe7510f640 1 -- 172.21.15.12:0/490766539 >> 172.21.15.12:0/490766539 conn(0x7fbe7006bfe0 msgr2=0x7fbe7006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:40.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.754+0000 7fbe7510f640 1 -- 172.21.15.12:0/490766539 shutdown_connections 2023-10-30T12:35:40.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.754+0000 7fbe7510f640 1 -- 172.21.15.12:0/490766539 wait complete. 2023-10-30T12:35:40.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe7510f640 1 Processor -- start 2023-10-30T12:35:40.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe7510f640 1 -- start start 2023-10-30T12:35:40.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe7510f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe700cf160 0x7fbe7014f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:40.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe7510f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe701538a0 0x7fbe700b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:40.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe7510f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7015d6f0 0x7fbe700b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:40.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe7510f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe70150c70 con 0x7fbe700cf160 2023-10-30T12:35:40.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe7510f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe701502a0 con 0x7fbe701538a0 2023-10-30T12:35:40.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe7510f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7014e700 con 0x7fbe7015d6f0 2023-10-30T12:35:40.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe6ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe700cf160 0x7fbe7014f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:40.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe6ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe700cf160 0x7fbe7014f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57464/0 (socket says 172.21.15.12:57464) 2023-10-30T12:35:40.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe6ed9d640 1 -- 172.21.15.12:0/2722114031 learned_addr learned my addr 172.21.15.12:0/2722114031 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:40.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe67fff640 1 --2- 172.21.15.12:0/2722114031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe701538a0 0x7fbe700b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:40.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe6f59e640 1 --2- 172.21.15.12:0/2722114031 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7015d6f0 0x7fbe700b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:40.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe6ed9d640 1 -- 172.21.15.12:0/2722114031 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7015d6f0 msgr2=0x7fbe700b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:40.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe6ed9d640 1 --2- 172.21.15.12:0/2722114031 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7015d6f0 0x7fbe700b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe6ed9d640 1 -- 172.21.15.12:0/2722114031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe701538a0 msgr2=0x7fbe700b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:40.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe6ed9d640 1 --2- 172.21.15.12:0/2722114031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe701538a0 0x7fbe700b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe6ed9d640 1 -- 172.21.15.12:0/2722114031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe68009a70 con 0x7fbe700cf160 2023-10-30T12:35:40.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe6ed9d640 1 --2- 172.21.15.12:0/2722114031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe700cf160 0x7fbe7014f740 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000c500 tx=0x7fbe6000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:40.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe677fe640 1 -- 172.21.15.12:0/2722114031 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe60005220 con 0x7fbe700cf160 2023-10-30T12:35:40.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe677fe640 1 -- 172.21.15.12:0/2722114031 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe60005380 con 0x7fbe700cf160 2023-10-30T12:35:40.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe677fe640 1 -- 172.21.15.12:0/2722114031 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe60007270 con 0x7fbe700cf160 2023-10-30T12:35:40.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.758+0000 7fbe7510f640 1 -- 172.21.15.12:0/2722114031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe70150990 con 0x7fbe700cf160 2023-10-30T12:35:40.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.762+0000 7fbe7510f640 1 -- 172.21.15.12:0/2722114031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7014ec80 con 0x7fbe700cf160 2023-10-30T12:35:40.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.762+0000 7fbe7510f640 1 -- 172.21.15.12:0/2722114031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe7004e170 con 0x7fbe700cf160 2023-10-30T12:35:40.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.762+0000 7fbe677fe640 1 -- 172.21.15.12:0/2722114031 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbe600073d0 con 0x7fbe700cf160 2023-10-30T12:35:40.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.762+0000 7fbe677fe640 1 --2- 172.21.15.12:0/2722114031 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbe500664a0 0x7fbe50068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:40.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.762+0000 7fbe677fe640 1 -- 172.21.15.12:0/2722114031 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe60083620 con 0x7fbe700cf160 2023-10-30T12:35:40.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.762+0000 7fbe67fff640 1 --2- 172.21.15.12:0/2722114031 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbe500664a0 0x7fbe50068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:40.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.766+0000 7fbe67fff640 1 --2- 172.21.15.12:0/2722114031 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbe500664a0 0x7fbe50068960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbe700c54c0 tx=0x7fbe58007d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:40.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.766+0000 7fbe677fe640 1 -- 172.21.15.12:0/2722114031 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbe60050b60 con 0x7fbe700cf160 2023-10-30T12:35:40.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.926+0000 7fbe7510f640 1 -- 172.21.15.12:0/2722114031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe700ca5d0 con 0x7fbe700cf160 2023-10-30T12:35:40.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.926+0000 7fbe677fe640 1 -- 172.21.15.12:0/2722114031 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16505 (secure 0 0 0) 0x7fbe60054180 con 0x7fbe700cf160 2023-10-30T12:35:40.930 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:40.930 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":22,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:35:07.153665+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:35:03.216053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:35:04.597162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:35:40.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.926+0000 7fbe7510f640 1 -- 172.21.15.12:0/2722114031 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbe500664a0 msgr2=0x7fbe50068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:40.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.926+0000 7fbe7510f640 1 --2- 172.21.15.12:0/2722114031 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbe500664a0 0x7fbe50068960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbe700c54c0 tx=0x7fbe58007d40 comp rx=0 tx=0).stop 2023-10-30T12:35:40.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.926+0000 7fbe7510f640 1 -- 172.21.15.12:0/2722114031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe700cf160 msgr2=0x7fbe7014f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:40.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.926+0000 7fbe7510f640 1 --2- 172.21.15.12:0/2722114031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe700cf160 0x7fbe7014f740 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000c500 tx=0x7fbe6000c9c0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.926+0000 7fbe7510f640 1 -- 172.21.15.12:0/2722114031 shutdown_connections 2023-10-30T12:35:40.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.926+0000 7fbe7510f640 1 --2- 172.21.15.12:0/2722114031 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbe500664a0 0x7fbe50068960 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.926+0000 7fbe7510f640 1 --2- 172.21.15.12:0/2722114031 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe7015d6f0 0x7fbe700b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.926+0000 7fbe7510f640 1 --2- 172.21.15.12:0/2722114031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe701538a0 0x7fbe700b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.926+0000 7fbe7510f640 1 --2- 172.21.15.12:0/2722114031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe700cf160 0x7fbe7014f740 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:40.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.926+0000 7fbe7510f640 1 -- 172.21.15.12:0/2722114031 >> 172.21.15.12:0/2722114031 conn(0x7fbe7006bfe0 msgr2=0x7fbe700cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:40.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.926+0000 7fbe7510f640 1 -- 172.21.15.12:0/2722114031 shutdown_connections 2023-10-30T12:35:40.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:40.926+0000 7fbe7510f640 1 -- 172.21.15.12:0/2722114031 wait complete. 2023-10-30T12:35:40.952 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:34:53.017612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-30T12:35:03.216053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-30T12:35:04.597162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:35:40.952 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:35:41.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82d04ea640 1 Processor -- start 2023-10-30T12:35:41.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82d04ea640 1 -- start start 2023-10-30T12:35:41.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82d04ea640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82c80c5810 0x7f82c80c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82d04ea640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c80c6cb0 con 0x7f82c806a980 2023-10-30T12:35:41.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82d04ea640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c80c6620 con 0x7f82c80c6130 2023-10-30T12:35:41.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82d04ea640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c8071d90 con 0x7f82c80c5810 2023-10-30T12:35:41.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82cea87640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f82c806a980 0x7f82c8161f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42956/0 (socket says 172.21.15.12:42956) 2023-10-30T12:35:41.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82cea87640 1 -- 172.21.15.12:0/4030592423 learned_addr learned my addr 172.21.15.12:0/4030592423 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:41.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82ce286640 1 --2- 172.21.15.12:0/4030592423 >> v2:172.21.15.154:3300/0 conn(0x7f82c80c5810 0x7f82c80c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82cd284640 1 -- 172.21.15.12:0/4030592423 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715052731 0 0) 0x7f82c80c6cb0 con 0x7f82c806a980 2023-10-30T12:35:41.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82cd284640 1 -- 172.21.15.12:0/4030592423 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82b4003530 con 0x7f82c806a980 2023-10-30T12:35:41.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82cd284640 1 -- 172.21.15.12:0/4030592423 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1196305394 0 0) 0x7f82b4003530 con 0x7f82c806a980 2023-10-30T12:35:41.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82cd284640 1 -- 172.21.15.12:0/4030592423 >> v2:172.21.15.154:3300/0 conn(0x7f82c80c5810 msgr2=0x7f82c80c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:41.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82cd284640 1 --2- 172.21.15.12:0/4030592423 >> v2:172.21.15.154:3300/0 conn(0x7f82c80c5810 0x7f82c80c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82cd284640 1 -- 172.21.15.12:0/4030592423 >> v1:172.21.15.154:6789/0 conn(0x7f82c80c6130 legacy=0x7f82c806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82cd284640 1 -- 172.21.15.12:0/4030592423 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c81582f0 con 0x7f82c806a980 2023-10-30T12:35:41.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.094+0000 7f82cd284640 1 -- 172.21.15.12:0/4030592423 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f82c0001a00 con 0x7f82c806a980 2023-10-30T12:35:41.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cd284640 1 -- 172.21.15.12:0/4030592423 >> v1:172.21.15.12:6789/0 conn(0x7f82c806a980 legacy=0x7f82c8161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cd284640 1 --2- 172.21.15.12:0/4030592423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b4003980 0x7f82b4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cd284640 1 --2- 172.21.15.12:0/4030592423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82b40047d0 0x7f82b4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cd284640 1 --2- 172.21.15.12:0/4030592423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82b40052b0 0x7f82b400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cd284640 1 -- 172.21.15.12:0/4030592423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b400c150 con 0x7f82b40047d0 2023-10-30T12:35:41.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cd284640 1 -- 172.21.15.12:0/4030592423 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b400c2c0 con 0x7f82b4003980 2023-10-30T12:35:41.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cd284640 1 -- 172.21.15.12:0/4030592423 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b400c480 con 0x7f82b40052b0 2023-10-30T12:35:41.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cda85640 1 --2- 172.21.15.12:0/4030592423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82b40047d0 0x7f82b4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82ce286640 1 --2- 172.21.15.12:0/4030592423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b4003980 0x7f82b4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cea87640 1 --2- 172.21.15.12:0/4030592423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82b40052b0 0x7f82b400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cda85640 1 -- 172.21.15.12:0/4030592423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82b40052b0 msgr2=0x7f82b400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:41.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cda85640 1 --2- 172.21.15.12:0/4030592423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82b40052b0 0x7f82b400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cda85640 1 -- 172.21.15.12:0/4030592423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b4003980 msgr2=0x7f82b4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:41.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cda85640 1 --2- 172.21.15.12:0/4030592423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b4003980 0x7f82b4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cda85640 1 -- 172.21.15.12:0/4030592423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b4003530 con 0x7f82b40047d0 2023-10-30T12:35:41.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82ce286640 1 --2- 172.21.15.12:0/4030592423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b4003980 0x7f82b4003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:41.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cea87640 1 -- 172.21.15.12:0/4030592423 reap_dead start 2023-10-30T12:35:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cda85640 1 --2- 172.21.15.12:0/4030592423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82b40047d0 0x7f82b4004c50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f82c806f5d0 tx=0x7f82b801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da50fe0a8d8311b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cd284640 1 -- 172.21.15.12:0/4030592423 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80052d0 con 0x7f82b40047d0 2023-10-30T12:35:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cd284640 1 -- 172.21.15.12:0/4030592423 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b80a7050 con 0x7f82b40047d0 2023-10-30T12:35:41.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.098+0000 7f82cd284640 1 -- 172.21.15.12:0/4030592423 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8003600 con 0x7f82b40047d0 2023-10-30T12:35:41.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82d04ea640 1 -- 172.21.15.12:0/4030592423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82b40047d0 msgr2=0x7f82b4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82d04ea640 1 --2- 172.21.15.12:0/4030592423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82b40047d0 0x7f82b4004c50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f82c806f5d0 tx=0x7f82b801c360 comp rx=0 tx=0).stop 2023-10-30T12:35:41.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82d04ea640 1 -- 172.21.15.12:0/4030592423 shutdown_connections 2023-10-30T12:35:41.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82d04ea640 1 --2- 172.21.15.12:0/4030592423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82b40047d0 0x7f82b4004c50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82d04ea640 1 -- 172.21.15.12:0/4030592423 >> 172.21.15.12:0/4030592423 conn(0x7f82c80c7910 msgr2=0x7f82c80c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:41.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82d04ea640 1 -- 172.21.15.12:0/4030592423 shutdown_connections 2023-10-30T12:35:41.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82d04ea640 1 -- 172.21.15.12:0/4030592423 wait complete. 2023-10-30T12:35:41.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82d04ea640 1 Processor -- start 2023-10-30T12:35:41.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82d04ea640 1 -- start start 2023-10-30T12:35:41.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82d04ea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b40047d0 0x7f82c80c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82d04ea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c814e610 0x7f82c8143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82d04ea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c81558e0 0x7f82c8155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82d04ea640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c814cef0 con 0x7f82c814e610 2023-10-30T12:35:41.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82d04ea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c814eb50 con 0x7f82b40047d0 2023-10-30T12:35:41.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82d04ea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c814cb40 con 0x7f82c81558e0 2023-10-30T12:35:41.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82cda85640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c814e610 0x7f82c8143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82cda85640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c814e610 0x7f82c8143dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50588/0 (socket says 172.21.15.12:50588) 2023-10-30T12:35:41.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82cda85640 1 -- 172.21.15.12:0/581102453 learned_addr learned my addr 172.21.15.12:0/581102453 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:41.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82ce286640 1 --2- 172.21.15.12:0/581102453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b40047d0 0x7f82c80c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.102+0000 7f82cea87640 1 --2- 172.21.15.12:0/581102453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c81558e0 0x7f82c8155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.106+0000 7f82cda85640 1 -- 172.21.15.12:0/581102453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c81558e0 msgr2=0x7f82c8155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.106+0000 7f82cda85640 1 --2- 172.21.15.12:0/581102453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c81558e0 0x7f82c8155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.106+0000 7f82cda85640 1 -- 172.21.15.12:0/581102453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b40047d0 msgr2=0x7f82c80c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.106+0000 7f82cda85640 1 --2- 172.21.15.12:0/581102453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b40047d0 0x7f82c80c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.106+0000 7f82cda85640 1 -- 172.21.15.12:0/581102453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c80c49a0 con 0x7f82c814e610 2023-10-30T12:35:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.106+0000 7f82cda85640 1 --2- 172.21.15.12:0/581102453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c814e610 0x7f82c8143dc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f82b801c190 tx=0x7f82b4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.106+0000 7f82beffd640 1 -- 172.21.15.12:0/581102453 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80072f0 con 0x7f82c814e610 2023-10-30T12:35:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.106+0000 7f82beffd640 1 -- 172.21.15.12:0/581102453 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b8007450 con 0x7f82c814e610 2023-10-30T12:35:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.106+0000 7f82d04ea640 1 -- 172.21.15.12:0/581102453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c814d100 con 0x7f82c814e610 2023-10-30T12:35:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.106+0000 7f82beffd640 1 -- 172.21.15.12:0/581102453 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80067e0 con 0x7f82c814e610 2023-10-30T12:35:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.106+0000 7f82d04ea640 1 -- 172.21.15.12:0/581102453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c814d7c0 con 0x7f82c814e610 2023-10-30T12:35:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.106+0000 7f82d04ea640 1 -- 172.21.15.12:0/581102453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c804e170 con 0x7f82c814e610 2023-10-30T12:35:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.106+0000 7f82beffd640 1 -- 172.21.15.12:0/581102453 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f82b8006940 con 0x7f82c814e610 2023-10-30T12:35:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.106+0000 7f82beffd640 1 --2- 172.21.15.12:0/581102453 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82ac0664a0 0x7f82ac068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.110+0000 7f82beffd640 1 -- 172.21.15.12:0/581102453 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82b811de40 con 0x7f82c814e610 2023-10-30T12:35:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.110+0000 7f82ce286640 1 --2- 172.21.15.12:0/581102453 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82ac0664a0 0x7f82ac068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.110+0000 7f82ce286640 1 --2- 172.21.15.12:0/581102453 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82ac0664a0 0x7f82ac068960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f82c804f480 tx=0x7f82c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.110+0000 7f82beffd640 1 -- 172.21.15.12:0/581102453 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f82b80eb190 con 0x7f82c814e610 2023-10-30T12:35:41.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.266+0000 7f82d04ea640 1 -- 172.21.15.12:0/581102453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82c8154fb0 con 0x7f82c814e610 2023-10-30T12:35:41.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.266+0000 7f82beffd640 1 -- 172.21.15.12:0/581102453 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82b80ee7b0 con 0x7f82c814e610 2023-10-30T12:35:41.272 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-10-30T12:35:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.270+0000 7f82d04ea640 1 -- 172.21.15.12:0/581102453 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82ac0664a0 msgr2=0x7f82ac068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.270+0000 7f82d04ea640 1 --2- 172.21.15.12:0/581102453 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82ac0664a0 0x7f82ac068960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f82c804f480 tx=0x7f82c4008040 comp rx=0 tx=0).stop 2023-10-30T12:35:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.270+0000 7f82d04ea640 1 -- 172.21.15.12:0/581102453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c814e610 msgr2=0x7f82c8143dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.270+0000 7f82d04ea640 1 --2- 172.21.15.12:0/581102453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c814e610 0x7f82c8143dc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f82b801c190 tx=0x7f82b4005800 comp rx=0 tx=0).stop 2023-10-30T12:35:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.270+0000 7f82d04ea640 1 -- 172.21.15.12:0/581102453 shutdown_connections 2023-10-30T12:35:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.270+0000 7f82d04ea640 1 --2- 172.21.15.12:0/581102453 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82ac0664a0 0x7f82ac068960 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.270+0000 7f82d04ea640 1 --2- 172.21.15.12:0/581102453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c81558e0 0x7f82c8155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.270+0000 7f82d04ea640 1 --2- 172.21.15.12:0/581102453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c814e610 0x7f82c8143dc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.270+0000 7f82d04ea640 1 --2- 172.21.15.12:0/581102453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b40047d0 0x7f82c80c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.270+0000 7f82d04ea640 1 -- 172.21.15.12:0/581102453 >> 172.21.15.12:0/581102453 conn(0x7f82c80c7910 msgr2=0x7f82c815b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:41.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.270+0000 7f82d04ea640 1 -- 172.21.15.12:0/581102453 shutdown_connections 2023-10-30T12:35:41.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.270+0000 7f82d04ea640 1 -- 172.21.15.12:0/581102453 wait complete. 2023-10-30T12:35:41.293 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-30T12:35:41.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.434+0000 7fae0eb1d640 1 Processor -- start 2023-10-30T12:35:41.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.434+0000 7fae0eb1d640 1 -- start start 2023-10-30T12:35:41.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae0eb1d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fae08153980 0x7fae080caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae0eb1d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fae080cb1e0 0x7fae0815d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae0eb1d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0815a920 con 0x7fae08159930 2023-10-30T12:35:41.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae0eb1d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae080cb930 con 0x7fae08153980 2023-10-30T12:35:41.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae0eb1d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae08159dc0 con 0x7fae080cb1e0 2023-10-30T12:35:41.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae0c8b9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fae08153980 0x7fae080caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae07fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fae08159930 0x7fae080cf1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42964/0 (socket says 172.21.15.12:42964) 2023-10-30T12:35:41.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae0c8b9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fae08153980 0x7fae080caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50602/0 (socket says 172.21.15.12:50602) 2023-10-30T12:35:41.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae07fff640 1 -- 172.21.15.12:0/1358801818 learned_addr learned my addr 172.21.15.12:0/1358801818 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:41.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae0d0ba640 1 --2- 172.21.15.12:0/1358801818 >> v2:172.21.15.154:3300/0 conn(0x7fae080cb1e0 0x7fae0815d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae0c8b9640 1 -- 172.21.15.12:0/1358801818 >> v2:172.21.15.154:3300/0 conn(0x7fae080cb1e0 msgr2=0x7fae0815d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae0c8b9640 1 --2- 172.21.15.12:0/1358801818 >> v2:172.21.15.154:3300/0 conn(0x7fae080cb1e0 0x7fae0815d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae0c8b9640 1 -- 172.21.15.12:0/1358801818 >> v1:172.21.15.12:6789/0 conn(0x7fae08159930 legacy=0x7fae080cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae0c8b9640 1 -- 172.21.15.12:0/1358801818 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0815a080 con 0x7fae08153980 2023-10-30T12:35:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae077fe640 1 -- 172.21.15.12:0/1358801818 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666207550 0 0) 0x7fae0815a920 con 0x7fae08159930 2023-10-30T12:35:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.438+0000 7fae0d0ba640 1 --2- 172.21.15.12:0/1358801818 >> v2:172.21.15.154:3300/0 conn(0x7fae080cb1e0 0x7fae0815d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.442+0000 7fae0c8b9640 1 --2- 172.21.15.12:0/1358801818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae08153980 0x7fae080caca0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fae080b1cf0 tx=0x7fadfc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f75195423d00242 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.442+0000 7fae077fe640 1 -- 172.21.15.12:0/1358801818 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0ab600 con 0x7fae08153980 2023-10-30T12:35:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.442+0000 7fae077fe640 1 -- 172.21.15.12:0/1358801818 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadfc0a7050 con 0x7fae08153980 2023-10-30T12:35:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.442+0000 7fae077fe640 1 -- 172.21.15.12:0/1358801818 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0ab980 con 0x7fae08153980 2023-10-30T12:35:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.442+0000 7fae0eb1d640 1 -- 172.21.15.12:0/1358801818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae08153980 msgr2=0x7fae080caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.442+0000 7fae0eb1d640 1 --2- 172.21.15.12:0/1358801818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae08153980 0x7fae080caca0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fae080b1cf0 tx=0x7fadfc01c430 comp rx=0 tx=0).stop 2023-10-30T12:35:41.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.442+0000 7fae0eb1d640 1 -- 172.21.15.12:0/1358801818 shutdown_connections 2023-10-30T12:35:41.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.442+0000 7fae0eb1d640 1 --2- 172.21.15.12:0/1358801818 >> v2:172.21.15.154:3300/0 conn(0x7fae080cb1e0 0x7fae0815d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.442+0000 7fae0eb1d640 1 --2- 172.21.15.12:0/1358801818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae08153980 0x7fae080caca0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.442+0000 7fae0eb1d640 1 -- 172.21.15.12:0/1358801818 >> 172.21.15.12:0/1358801818 conn(0x7fae0806c0f0 msgr2=0x7fae0806c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:41.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.442+0000 7fae0eb1d640 1 -- 172.21.15.12:0/1358801818 shutdown_connections 2023-10-30T12:35:41.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.442+0000 7fae0eb1d640 1 -- 172.21.15.12:0/1358801818 wait complete. 2023-10-30T12:35:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.442+0000 7fae0eb1d640 1 Processor -- start 2023-10-30T12:35:41.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.442+0000 7fae0eb1d640 1 -- start start 2023-10-30T12:35:41.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae0eb1d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae080cb1e0 0x7fae08152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae0eb1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae08153980 0x7fae081f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae0eb1d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae080c75a0 0x7fae0815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae0eb1d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae081529c0 con 0x7fae08153980 2023-10-30T12:35:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae0eb1d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae08151060 con 0x7fae080c75a0 2023-10-30T12:35:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae0eb1d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae081512c0 con 0x7fae080cb1e0 2023-10-30T12:35:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae07fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae08153980 0x7fae081f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae07fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae08153980 0x7fae081f4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50604/0 (socket says 172.21.15.12:50604) 2023-10-30T12:35:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae07fff640 1 -- 172.21.15.12:0/2624806856 learned_addr learned my addr 172.21.15.12:0/2624806856 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae0c8b9640 1 --2- 172.21.15.12:0/2624806856 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae080cb1e0 0x7fae08152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae0d0ba640 1 --2- 172.21.15.12:0/2624806856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae080c75a0 0x7fae0815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae07fff640 1 -- 172.21.15.12:0/2624806856 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae080cb1e0 msgr2=0x7fae08152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae07fff640 1 --2- 172.21.15.12:0/2624806856 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae080cb1e0 0x7fae08152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae07fff640 1 -- 172.21.15.12:0/2624806856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae080c75a0 msgr2=0x7fae0815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae07fff640 1 --2- 172.21.15.12:0/2624806856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae080c75a0 0x7fae0815c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae07fff640 1 -- 172.21.15.12:0/2624806856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc009420 con 0x7fae08153980 2023-10-30T12:35:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae0d0ba640 1 --2- 172.21.15.12:0/2624806856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae080c75a0 0x7fae0815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae07fff640 1 --2- 172.21.15.12:0/2624806856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae08153980 0x7fae081f4ed0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fadf400c480 tx=0x7fadf400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.446+0000 7fae057fa640 1 -- 172.21.15.12:0/2624806856 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf40074e0 con 0x7fae08153980 2023-10-30T12:35:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.450+0000 7fae057fa640 1 -- 172.21.15.12:0/2624806856 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf4007660 con 0x7fae08153980 2023-10-30T12:35:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.450+0000 7fae057fa640 1 -- 172.21.15.12:0/2624806856 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4004a30 con 0x7fae08153980 2023-10-30T12:35:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.450+0000 7fae0eb1d640 1 -- 172.21.15.12:0/2624806856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae081515a0 con 0x7fae08153980 2023-10-30T12:35:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.450+0000 7fae0eb1d640 1 -- 172.21.15.12:0/2624806856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae08152650 con 0x7fae08153980 2023-10-30T12:35:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.450+0000 7fae0eb1d640 1 -- 172.21.15.12:0/2624806856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae08152810 con 0x7fae08153980 2023-10-30T12:35:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.450+0000 7fae057fa640 1 -- 172.21.15.12:0/2624806856 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fadf4004b90 con 0x7fae08153980 2023-10-30T12:35:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.450+0000 7fae057fa640 1 --2- 172.21.15.12:0/2624806856 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fadec0664a0 0x7fadec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.450+0000 7fae057fa640 1 -- 172.21.15.12:0/2624806856 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadf4082ff0 con 0x7fae08153980 2023-10-30T12:35:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.450+0000 7fae0c8b9640 1 --2- 172.21.15.12:0/2624806856 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fadec0664a0 0x7fadec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.454+0000 7fae0c8b9640 1 --2- 172.21.15.12:0/2624806856 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fadec0664a0 0x7fadec068960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fae080742c0 tx=0x7fadfc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:41.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.454+0000 7fae057fa640 1 -- 172.21.15.12:0/2624806856 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fadf40b4090 con 0x7fae08153980 2023-10-30T12:35:41.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.614+0000 7fae0eb1d640 1 -- 172.21.15.12:0/2624806856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae0815d050 con 0x7fae08153980 2023-10-30T12:35:41.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.614+0000 7fae057fa640 1 -- 172.21.15.12:0/2624806856 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fadf4053f50 con 0x7fae08153980 2023-10-30T12:35:41.617 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-10-30T12:35:41.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.614+0000 7fae0eb1d640 1 -- 172.21.15.12:0/2624806856 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fadec0664a0 msgr2=0x7fadec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.614+0000 7fae0eb1d640 1 --2- 172.21.15.12:0/2624806856 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fadec0664a0 0x7fadec068960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fae080742c0 tx=0x7fadfc0a6000 comp rx=0 tx=0).stop 2023-10-30T12:35:41.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.614+0000 7fae0eb1d640 1 -- 172.21.15.12:0/2624806856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae08153980 msgr2=0x7fae081f4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.614+0000 7fae0eb1d640 1 --2- 172.21.15.12:0/2624806856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae08153980 0x7fae081f4ed0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fadf400c480 tx=0x7fadf400c940 comp rx=0 tx=0).stop 2023-10-30T12:35:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.614+0000 7fae0eb1d640 1 -- 172.21.15.12:0/2624806856 shutdown_connections 2023-10-30T12:35:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.614+0000 7fae0eb1d640 1 --2- 172.21.15.12:0/2624806856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae080c75a0 0x7fae0815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.614+0000 7fae0eb1d640 1 --2- 172.21.15.12:0/2624806856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae08153980 0x7fae081f4ed0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.614+0000 7fae0eb1d640 1 --2- 172.21.15.12:0/2624806856 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fadec0664a0 0x7fadec068960 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.614+0000 7fae0eb1d640 1 --2- 172.21.15.12:0/2624806856 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae080cb1e0 0x7fae08152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.614+0000 7fae0eb1d640 1 -- 172.21.15.12:0/2624806856 >> 172.21.15.12:0/2624806856 conn(0x7fae0806c0f0 msgr2=0x7fae080c4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:41.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.614+0000 7fae0eb1d640 1 -- 172.21.15.12:0/2624806856 shutdown_connections 2023-10-30T12:35:41.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.614+0000 7fae0eb1d640 1 -- 172.21.15.12:0/2624806856 wait complete. 2023-10-30T12:35:41.638 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-30T12:35:41.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.778+0000 7f9705c74640 1 Processor -- start 2023-10-30T12:35:41.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.778+0000 7f9705c74640 1 -- start start 2023-10-30T12:35:41.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f9705c74640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9700153c20 0x7f970015a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f9705c74640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f970015a6b0 con 0x7f97000cb4c0 2023-10-30T12:35:41.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f9705c74640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f970015d730 con 0x7f9700153840 2023-10-30T12:35:41.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f9705c74640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970015dbc0 con 0x7f9700153c20 2023-10-30T12:35:41.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f9705473640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f97000cb4c0 0x7f97000cef40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42974/0 (socket says 172.21.15.12:42974) 2023-10-30T12:35:41.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f9705473640 1 -- 172.21.15.12:0/3640682499 learned_addr learned my addr 172.21.15.12:0/3640682499 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:41.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96fffff640 1 --2- 172.21.15.12:0/3640682499 >> v2:172.21.15.154:3300/0 conn(0x7f9700153c20 0x7f970015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96ff7fe640 1 -- 172.21.15.12:0/3640682499 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973886333 0 0) 0x7f96f80014a0 con 0x7f97000cb4c0 2023-10-30T12:35:41.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96ff7fe640 1 -- 172.21.15.12:0/3640682499 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96f0003530 con 0x7f97000cb4c0 2023-10-30T12:35:41.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96ff7fe640 1 -- 172.21.15.12:0/3640682499 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3462271637 0 0) 0x7f96f0003530 con 0x7f97000cb4c0 2023-10-30T12:35:41.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96ff7fe640 1 -- 172.21.15.12:0/3640682499 >> v2:172.21.15.154:3300/0 conn(0x7f9700153c20 msgr2=0x7f970015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:41.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96ff7fe640 1 --2- 172.21.15.12:0/3640682499 >> v2:172.21.15.154:3300/0 conn(0x7f9700153c20 0x7f970015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96ff7fe640 1 -- 172.21.15.12:0/3640682499 >> v1:172.21.15.154:6789/0 conn(0x7f9700153840 legacy=0x7f97000c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96ff7fe640 1 -- 172.21.15.12:0/3640682499 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97000ca8f0 con 0x7f97000cb4c0 2023-10-30T12:35:41.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96ff7fe640 1 -- 172.21.15.12:0/3640682499 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f96f8001b90 con 0x7f97000cb4c0 2023-10-30T12:35:41.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96ff7fe640 1 -- 172.21.15.12:0/3640682499 >> v1:172.21.15.12:6789/0 conn(0x7f97000cb4c0 legacy=0x7f97000cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96ff7fe640 1 --2- 172.21.15.12:0/3640682499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96f0003980 0x7f96f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96ff7fe640 1 --2- 172.21.15.12:0/3640682499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f00047d0 0x7f96f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96ff7fe640 1 --2- 172.21.15.12:0/3640682499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96f0005190 0x7f96f000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96ff7fe640 1 -- 172.21.15.12:0/3640682499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970015dbc0 con 0x7f96f0005190 2023-10-30T12:35:41.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96ff7fe640 1 -- 172.21.15.12:0/3640682499 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f000c040 con 0x7f96f00047d0 2023-10-30T12:35:41.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96ff7fe640 1 -- 172.21.15.12:0/3640682499 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f000c200 con 0x7f96f0003980 2023-10-30T12:35:41.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f9705473640 1 --2- 172.21.15.12:0/3640682499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96f0005190 0x7f96f000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f96fffff640 1 --2- 172.21.15.12:0/3640682499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f00047d0 0x7f96f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f9704c72640 1 --2- 172.21.15.12:0/3640682499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96f0003980 0x7f96f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f9705473640 1 -- 172.21.15.12:0/3640682499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96f0003980 msgr2=0x7f96f0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:41.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f9705473640 1 --2- 172.21.15.12:0/3640682499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96f0003980 0x7f96f0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f9705473640 1 -- 172.21.15.12:0/3640682499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f00047d0 msgr2=0x7f96f0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:41.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f9705473640 1 --2- 172.21.15.12:0/3640682499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f00047d0 0x7f96f0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.782+0000 7f9705473640 1 -- 172.21.15.12:0/3640682499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f0003530 con 0x7f96f0005190 2023-10-30T12:35:41.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.786+0000 7f9705473640 1 -- 172.21.15.12:0/3640682499 reap_dead start 2023-10-30T12:35:41.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.786+0000 7f9705473640 1 --2- 172.21.15.12:0/3640682499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96f0005190 0x7f96f000b990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f96f800a830 tx=0x7f96f801d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabd6be668fbf776 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:41.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.786+0000 7f96ff7fe640 1 -- 172.21.15.12:0/3640682499 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f80ad630 con 0x7f96f0005190 2023-10-30T12:35:41.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.786+0000 7f96ff7fe640 1 -- 172.21.15.12:0/3640682499 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f80a8070 con 0x7f96f0005190 2023-10-30T12:35:41.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.786+0000 7f96ff7fe640 1 -- 172.21.15.12:0/3640682499 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f80ad9f0 con 0x7f96f0005190 2023-10-30T12:35:41.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.786+0000 7f9705c74640 1 -- 172.21.15.12:0/3640682499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96f0005190 msgr2=0x7f96f000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.786+0000 7f9705c74640 1 --2- 172.21.15.12:0/3640682499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96f0005190 0x7f96f000b990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f96f800a830 tx=0x7f96f801d340 comp rx=0 tx=0).stop 2023-10-30T12:35:41.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.786+0000 7f9705c74640 1 -- 172.21.15.12:0/3640682499 shutdown_connections 2023-10-30T12:35:41.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.786+0000 7f9705c74640 1 --2- 172.21.15.12:0/3640682499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96f0005190 0x7f96f000b990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.786+0000 7f9705c74640 1 -- 172.21.15.12:0/3640682499 >> 172.21.15.12:0/3640682499 conn(0x7f970006be40 msgr2=0x7f970006c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:41.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.786+0000 7f9705c74640 1 -- 172.21.15.12:0/3640682499 shutdown_connections 2023-10-30T12:35:41.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.786+0000 7f9705c74640 1 -- 172.21.15.12:0/3640682499 wait complete. 2023-10-30T12:35:41.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.786+0000 7f9705c74640 1 Processor -- start 2023-10-30T12:35:41.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.786+0000 7f9705c74640 1 -- start start 2023-10-30T12:35:41.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705c74640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f0005190 0x7f9700149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705c74640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97000cdb10 0x7f97001eeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705c74640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97001ef3f0 0x7f9700156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705c74640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97001555a0 con 0x7f97001ef3f0 2023-10-30T12:35:41.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705c74640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9700155ad0 con 0x7f96f0005190 2023-10-30T12:35:41.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705c74640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97001ef870 con 0x7f97000cdb10 2023-10-30T12:35:41.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705473640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97001ef3f0 0x7f9700156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705473640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97001ef3f0 0x7f9700156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50622/0 (socket says 172.21.15.12:50622) 2023-10-30T12:35:41.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705473640 1 -- 172.21.15.12:0/858773919 learned_addr learned my addr 172.21.15.12:0/858773919 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:41.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9704c72640 1 --2- 172.21.15.12:0/858773919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f0005190 0x7f9700149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f96fffff640 1 --2- 172.21.15.12:0/858773919 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97000cdb10 0x7f97001eeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705473640 1 -- 172.21.15.12:0/858773919 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97000cdb10 msgr2=0x7f97001eeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705473640 1 --2- 172.21.15.12:0/858773919 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97000cdb10 0x7f97001eeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705473640 1 -- 172.21.15.12:0/858773919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f0005190 msgr2=0x7f9700149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705473640 1 --2- 172.21.15.12:0/858773919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f0005190 0x7f9700149c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705473640 1 -- 172.21.15.12:0/858773919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97000c6220 con 0x7f97001ef3f0 2023-10-30T12:35:41.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9704c72640 1 --2- 172.21.15.12:0/858773919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f0005190 0x7f9700149c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:41.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705473640 1 --2- 172.21.15.12:0/858773919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97001ef3f0 0x7f9700156550 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f96f801dbe0 tx=0x7f96f0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:41.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f96fd7fa640 1 -- 172.21.15.12:0/858773919 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f80ad630 con 0x7f97001ef3f0 2023-10-30T12:35:41.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f96fd7fa640 1 -- 172.21.15.12:0/858773919 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f80ad790 con 0x7f97001ef3f0 2023-10-30T12:35:41.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f96fd7fa640 1 -- 172.21.15.12:0/858773919 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f80adb00 con 0x7f97001ef3f0 2023-10-30T12:35:41.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705c74640 1 -- 172.21.15.12:0/858773919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97001555a0 con 0x7f97001ef3f0 2023-10-30T12:35:41.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705c74640 1 -- 172.21.15.12:0/858773919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97001f2dd0 con 0x7f97001ef3f0 2023-10-30T12:35:41.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.790+0000 7f9705c74640 1 -- 172.21.15.12:0/858773919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970004e9c0 con 0x7f97001ef3f0 2023-10-30T12:35:41.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.794+0000 7f96fd7fa640 1 -- 172.21.15.12:0/858773919 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f96f80aa030 con 0x7f97001ef3f0 2023-10-30T12:35:41.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.794+0000 7f96fd7fa640 1 --2- 172.21.15.12:0/858773919 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f96e40664a0 0x7f96e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:41.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.794+0000 7f96fd7fa640 1 -- 172.21.15.12:0/858773919 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96f811def0 con 0x7f97001ef3f0 2023-10-30T12:35:41.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.794+0000 7f9704c72640 1 --2- 172.21.15.12:0/858773919 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f96e40664a0 0x7f96e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:41.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.794+0000 7f9704c72640 1 --2- 172.21.15.12:0/858773919 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f96e40664a0 0x7f96e4068960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f96f4000c10 tx=0x7f96f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:41.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.798+0000 7f96fd7fa640 1 -- 172.21.15.12:0/858773919 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f96f80eb1b0 con 0x7f97001ef3f0 2023-10-30T12:35:41.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.954+0000 7f9705c74640 1 -- 172.21.15.12:0/858773919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f970015a6b0 con 0x7f97001ef3f0 2023-10-30T12:35:41.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.954+0000 7f96fd7fa640 1 -- 172.21.15.12:0/858773919 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f970015a6b0 con 0x7f97001ef3f0 2023-10-30T12:35:41.958 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-10-30T12:35:41.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.954+0000 7f9705c74640 1 -- 172.21.15.12:0/858773919 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f96e40664a0 msgr2=0x7f96e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.954+0000 7f9705c74640 1 --2- 172.21.15.12:0/858773919 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f96e40664a0 0x7f96e4068960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f96f4000c10 tx=0x7f96f4009040 comp rx=0 tx=0).stop 2023-10-30T12:35:41.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.954+0000 7f9705c74640 1 -- 172.21.15.12:0/858773919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97001ef3f0 msgr2=0x7f9700156550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:41.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.954+0000 7f9705c74640 1 --2- 172.21.15.12:0/858773919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97001ef3f0 0x7f9700156550 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f96f801dbe0 tx=0x7f96f0005800 comp rx=0 tx=0).stop 2023-10-30T12:35:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.954+0000 7f9705c74640 1 -- 172.21.15.12:0/858773919 shutdown_connections 2023-10-30T12:35:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.954+0000 7f9705c74640 1 --2- 172.21.15.12:0/858773919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97001ef3f0 0x7f9700156550 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.954+0000 7f9705c74640 1 --2- 172.21.15.12:0/858773919 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f96e40664a0 0x7f96e4068960 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.954+0000 7f9705c74640 1 --2- 172.21.15.12:0/858773919 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97000cdb10 0x7f97001eeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.954+0000 7f9705c74640 1 --2- 172.21.15.12:0/858773919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f0005190 0x7f9700149c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:41.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.954+0000 7f9705c74640 1 -- 172.21.15.12:0/858773919 >> 172.21.15.12:0/858773919 conn(0x7f970006be40 msgr2=0x7f97000cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:41.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.958+0000 7f9705c74640 1 -- 172.21.15.12:0/858773919 shutdown_connections 2023-10-30T12:35:41.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:41.958+0000 7f9705c74640 1 -- 172.21.15.12:0/858773919 wait complete. 2023-10-30T12:35:41.980 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:35:42.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.118+0000 7fab6b068640 1 Processor -- start 2023-10-30T12:35:42.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.118+0000 7fab6b068640 1 -- start start 2023-10-30T12:35:42.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.122+0000 7fab6b068640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fab640c6b80 0x7fab640bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:42.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.122+0000 7fab6b068640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab6413f4a0 0x7fab640c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:42.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.122+0000 7fab6b068640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab640c0a50 con 0x7fab640c6f60 2023-10-30T12:35:42.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.122+0000 7fab6b068640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6406f6e0 con 0x7fab640c6b80 2023-10-30T12:35:42.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.122+0000 7fab6b068640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6406f9c0 con 0x7fab6413f4a0 2023-10-30T12:35:42.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.122+0000 7fab63fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fab640c6f60 0x7fab6413ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42990/0 (socket says 172.21.15.12:42990) 2023-10-30T12:35:42.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.122+0000 7fab63fff640 1 -- 172.21.15.12:0/4196028708 learned_addr learned my addr 172.21.15.12:0/4196028708 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:42.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.122+0000 7fab68e04640 1 --2- 172.21.15.12:0/4196028708 >> v2:172.21.15.12:3300/0 conn(0x7fab640c6b80 0x7fab640bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:42.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.122+0000 7fab69605640 1 --2- 172.21.15.12:0/4196028708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab6413f4a0 0x7fab640c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:42.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.122+0000 7fab637fe640 1 -- 172.21.15.12:0/4196028708 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269633174 0 0) 0x7fab640c0a50 con 0x7fab640c6f60 2023-10-30T12:35:42.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.122+0000 7fab637fe640 1 -- 172.21.15.12:0/4196028708 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab54003530 con 0x7fab640c6f60 2023-10-30T12:35:42.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.122+0000 7fab68e04640 1 -- 172.21.15.12:0/4196028708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab6413f4a0 msgr2=0x7fab640c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:42.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.122+0000 7fab68e04640 1 --2- 172.21.15.12:0/4196028708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab6413f4a0 0x7fab640c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.122+0000 7fab68e04640 1 -- 172.21.15.12:0/4196028708 >> v1:172.21.15.12:6789/0 conn(0x7fab640c6f60 legacy=0x7fab6413ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:42.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.122+0000 7fab68e04640 1 -- 172.21.15.12:0/4196028708 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6406fca0 con 0x7fab640c6b80 2023-10-30T12:35:42.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.126+0000 7fab68e04640 1 --2- 172.21.15.12:0/4196028708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab640c6b80 0x7fab640bfec0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fab580097c0 tx=0x7fab5801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1432976dba5488a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:42.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.126+0000 7fab637fe640 1 -- 172.21.15.12:0/4196028708 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab580ad640 con 0x7fab640c6b80 2023-10-30T12:35:42.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.126+0000 7fab637fe640 1 -- 172.21.15.12:0/4196028708 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab580ad7a0 con 0x7fab640c6b80 2023-10-30T12:35:42.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.126+0000 7fab637fe640 1 -- 172.21.15.12:0/4196028708 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab580adb30 con 0x7fab640c6b80 2023-10-30T12:35:42.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.126+0000 7fab6b068640 1 -- 172.21.15.12:0/4196028708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab640c6b80 msgr2=0x7fab640bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:42.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.126+0000 7fab6b068640 1 --2- 172.21.15.12:0/4196028708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab640c6b80 0x7fab640bfec0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fab580097c0 tx=0x7fab5801c480 comp rx=0 tx=0).stop 2023-10-30T12:35:42.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.126+0000 7fab6b068640 1 -- 172.21.15.12:0/4196028708 shutdown_connections 2023-10-30T12:35:42.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.126+0000 7fab6b068640 1 --2- 172.21.15.12:0/4196028708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab6413f4a0 0x7fab640c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.126+0000 7fab6b068640 1 --2- 172.21.15.12:0/4196028708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab640c6b80 0x7fab640bfec0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.126+0000 7fab6b068640 1 -- 172.21.15.12:0/4196028708 >> 172.21.15.12:0/4196028708 conn(0x7fab640c7910 msgr2=0x7fab640c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:42.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.126+0000 7fab6b068640 1 -- 172.21.15.12:0/4196028708 shutdown_connections 2023-10-30T12:35:42.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.126+0000 7fab6b068640 1 -- 172.21.15.12:0/4196028708 wait complete. 2023-10-30T12:35:42.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.126+0000 7fab6b068640 1 Processor -- start 2023-10-30T12:35:42.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.126+0000 7fab6b068640 1 -- start start 2023-10-30T12:35:42.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.126+0000 7fab6b068640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab640c6b80 0x7fab640ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:42.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab6b068640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab640c6f60 0x7fab64155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:42.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab6b068640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab6413f4a0 0x7fab64162690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:42.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab6b068640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab64154da0 con 0x7fab640c6f60 2023-10-30T12:35:42.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab6b068640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6413ce80 con 0x7fab640c6b80 2023-10-30T12:35:42.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab6b068640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab64154aa0 con 0x7fab6413f4a0 2023-10-30T12:35:42.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab63fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab640c6f60 0x7fab64155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:42.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab63fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab640c6f60 0x7fab64155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50640/0 (socket says 172.21.15.12:50640) 2023-10-30T12:35:42.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab63fff640 1 -- 172.21.15.12:0/3395457337 learned_addr learned my addr 172.21.15.12:0/3395457337 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:42.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab68e04640 1 --2- 172.21.15.12:0/3395457337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab640c6b80 0x7fab640ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:42.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab69605640 1 --2- 172.21.15.12:0/3395457337 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab6413f4a0 0x7fab64162690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:42.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab63fff640 1 -- 172.21.15.12:0/3395457337 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab6413f4a0 msgr2=0x7fab64162690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:42.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab63fff640 1 --2- 172.21.15.12:0/3395457337 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab6413f4a0 0x7fab64162690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab63fff640 1 -- 172.21.15.12:0/3395457337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab640c6b80 msgr2=0x7fab640ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:42.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab63fff640 1 --2- 172.21.15.12:0/3395457337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab640c6b80 0x7fab640ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab63fff640 1 -- 172.21.15.12:0/3395457337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab58009420 con 0x7fab640c6f60 2023-10-30T12:35:42.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab63fff640 1 --2- 172.21.15.12:0/3395457337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab640c6f60 0x7fab64155080 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fab5000c500 tx=0x7fab5000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:42.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab617fa640 1 -- 172.21.15.12:0/3395457337 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50005cf0 con 0x7fab640c6f60 2023-10-30T12:35:42.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab617fa640 1 -- 172.21.15.12:0/3395457337 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab50005e50 con 0x7fab640c6f60 2023-10-30T12:35:42.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab6b068640 1 -- 172.21.15.12:0/3395457337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab64154290 con 0x7fab640c6f60 2023-10-30T12:35:42.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab617fa640 1 -- 172.21.15.12:0/3395457337 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab500072f0 con 0x7fab640c6f60 2023-10-30T12:35:42.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab6b068640 1 -- 172.21.15.12:0/3395457337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab64162200 con 0x7fab640c6f60 2023-10-30T12:35:42.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab6b068640 1 -- 172.21.15.12:0/3395457337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab64162390 con 0x7fab640c6f60 2023-10-30T12:35:42.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.130+0000 7fab617fa640 1 -- 172.21.15.12:0/3395457337 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fab50007450 con 0x7fab640c6f60 2023-10-30T12:35:42.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.134+0000 7fab617fa640 1 --2- 172.21.15.12:0/3395457337 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fab480664a0 0x7fab48068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:42.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.134+0000 7fab617fa640 1 -- 172.21.15.12:0/3395457337 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab500833e0 con 0x7fab640c6f60 2023-10-30T12:35:42.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.134+0000 7fab68e04640 1 --2- 172.21.15.12:0/3395457337 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fab480664a0 0x7fab48068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:42.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.134+0000 7fab68e04640 1 --2- 172.21.15.12:0/3395457337 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fab480664a0 0x7fab48068960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fab580af000 tx=0x7fab58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:42.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.134+0000 7fab617fa640 1 -- 172.21.15.12:0/3395457337 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fab50050900 con 0x7fab640c6f60 2023-10-30T12:35:42.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.326+0000 7fab6b068640 1 -- 172.21.15.12:0/3395457337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab64162020 con 0x7fab640c6f60 2023-10-30T12:35:42.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.326+0000 7fab617fa640 1 -- 172.21.15.12:0/3395457337 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6523 (secure 0 0 0) 0x7fab50053f20 con 0x7fab640c6f60 2023-10-30T12:35:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:42.331 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:35:27.538464+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":9,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:35:42.331 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2023-10-30T12:35:42.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.326+0000 7fab6b068640 1 -- 172.21.15.12:0/3395457337 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fab480664a0 msgr2=0x7fab48068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:42.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.326+0000 7fab6b068640 1 --2- 172.21.15.12:0/3395457337 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fab480664a0 0x7fab48068960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fab580af000 tx=0x7fab58008040 comp rx=0 tx=0).stop 2023-10-30T12:35:42.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.326+0000 7fab6b068640 1 -- 172.21.15.12:0/3395457337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab640c6f60 msgr2=0x7fab64155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:42.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.326+0000 7fab6b068640 1 --2- 172.21.15.12:0/3395457337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab640c6f60 0x7fab64155080 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fab5000c500 tx=0x7fab5000c9c0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.326+0000 7fab6b068640 1 -- 172.21.15.12:0/3395457337 shutdown_connections 2023-10-30T12:35:42.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.326+0000 7fab6b068640 1 --2- 172.21.15.12:0/3395457337 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fab480664a0 0x7fab48068960 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.326+0000 7fab6b068640 1 --2- 172.21.15.12:0/3395457337 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab6413f4a0 0x7fab64162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.326+0000 7fab6b068640 1 --2- 172.21.15.12:0/3395457337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab640c6f60 0x7fab64155080 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.326+0000 7fab6b068640 1 --2- 172.21.15.12:0/3395457337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab640c6b80 0x7fab640ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.326+0000 7fab6b068640 1 -- 172.21.15.12:0/3395457337 >> 172.21.15.12:0/3395457337 conn(0x7fab640c7910 msgr2=0x7fab6413ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:42.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.330+0000 7fab6b068640 1 -- 172.21.15.12:0/3395457337 shutdown_connections 2023-10-30T12:35:42.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.330+0000 7fab6b068640 1 -- 172.21.15.12:0/3395457337 wait complete. 2023-10-30T12:35:42.351 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:35:42.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.490+0000 7fed5a93e640 1 Processor -- start 2023-10-30T12:35:42.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.490+0000 7fed5a93e640 1 -- start start 2023-10-30T12:35:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.494+0000 7fed5a93e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fed5406f7a0 0x7fed5406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.494+0000 7fed5a93e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed54071110 0x7fed54158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.494+0000 7fed5a93e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed54066cf0 con 0x7fed540c6bd0 2023-10-30T12:35:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.494+0000 7fed5a93e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed540700c0 con 0x7fed5406f7a0 2023-10-30T12:35:42.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.494+0000 7fed5a93e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed540c70d0 con 0x7fed54071110 2023-10-30T12:35:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.494+0000 7fed58edb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed54071110 0x7fed54158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.494+0000 7fed53fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fed5406f7a0 0x7fed5406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.494+0000 7fed537fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fed540c6bd0 0x7fed54069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:55932/0 (socket says 172.21.15.12:55932) 2023-10-30T12:35:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.494+0000 7fed537fe640 1 -- 172.21.15.12:0/2980339522 learned_addr learned my addr 172.21.15.12:0/2980339522 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:35:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.494+0000 7fed58edb640 1 --2- 172.21.15.12:0/2980339522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed54071110 0x7fed54158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:39110/0 (socket says 172.21.15.12:39110) 2023-10-30T12:35:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.494+0000 7fed52ffd640 1 -- 172.21.15.12:0/2980339522 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193493293 0 0) 0x7fed54066cf0 con 0x7fed540c6bd0 2023-10-30T12:35:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.494+0000 7fed58edb640 1 -- 172.21.15.12:0/2980339522 >> v2:172.21.15.154:3300/0 conn(0x7fed5406f7a0 msgr2=0x7fed5406fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:35:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.494+0000 7fed58edb640 1 --2- 172.21.15.12:0/2980339522 >> v2:172.21.15.154:3300/0 conn(0x7fed5406f7a0 0x7fed5406fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.494+0000 7fed58edb640 1 -- 172.21.15.12:0/2980339522 >> v1:172.21.15.154:6789/0 conn(0x7fed540c6bd0 legacy=0x7fed54069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.494+0000 7fed58edb640 1 -- 172.21.15.12:0/2980339522 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed540c7390 con 0x7fed54071110 2023-10-30T12:35:42.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.498+0000 7fed58edb640 1 --2- 172.21.15.12:0/2980339522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed54071110 0x7fed54158110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009d90 tx=0x7fed4c01f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=900cc5c92ce96667 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:42.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.498+0000 7fed52ffd640 1 -- 172.21.15.12:0/2980339522 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0af600 con 0x7fed54071110 2023-10-30T12:35:42.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.498+0000 7fed52ffd640 1 -- 172.21.15.12:0/2980339522 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4c0ab050 con 0x7fed54071110 2023-10-30T12:35:42.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.498+0000 7fed52ffd640 1 -- 172.21.15.12:0/2980339522 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0af980 con 0x7fed54071110 2023-10-30T12:35:42.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.498+0000 7fed5a93e640 1 -- 172.21.15.12:0/2980339522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed54071110 msgr2=0x7fed54158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:42.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.498+0000 7fed5a93e640 1 --2- 172.21.15.12:0/2980339522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed54071110 0x7fed54158110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009d90 tx=0x7fed4c01f130 comp rx=0 tx=0).stop 2023-10-30T12:35:42.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.498+0000 7fed5a93e640 1 -- 172.21.15.12:0/2980339522 shutdown_connections 2023-10-30T12:35:42.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.498+0000 7fed5a93e640 1 --2- 172.21.15.12:0/2980339522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed54071110 0x7fed54158110 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.498+0000 7fed5a93e640 1 --2- 172.21.15.12:0/2980339522 >> v2:172.21.15.154:3300/0 conn(0x7fed5406f7a0 0x7fed5406fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.498+0000 7fed5a93e640 1 -- 172.21.15.12:0/2980339522 >> 172.21.15.12:0/2980339522 conn(0x7fed540c7910 msgr2=0x7fed540c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:42.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.498+0000 7fed5a93e640 1 -- 172.21.15.12:0/2980339522 shutdown_connections 2023-10-30T12:35:42.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.498+0000 7fed5a93e640 1 -- 172.21.15.12:0/2980339522 wait complete. 2023-10-30T12:35:42.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed5a93e640 1 Processor -- start 2023-10-30T12:35:42.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed5a93e640 1 -- start start 2023-10-30T12:35:42.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed5a93e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed5406f7a0 0x7fed541eb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:42.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed5a93e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed54071110 0x7fed54152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:42.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed5a93e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed540c6bd0 0x7fed54152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:42.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed5a93e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed540aa070 con 0x7fed5406f7a0 2023-10-30T12:35:42.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed5a93e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed541eba70 con 0x7fed540c6bd0 2023-10-30T12:35:42.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed5a93e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed540a95f0 con 0x7fed54071110 2023-10-30T12:35:42.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed53fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed5406f7a0 0x7fed541eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:42.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed53fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed5406f7a0 0x7fed541eb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50650/0 (socket says 172.21.15.12:50650) 2023-10-30T12:35:42.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed53fff640 1 -- 172.21.15.12:0/3086624403 learned_addr learned my addr 172.21.15.12:0/3086624403 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:35:42.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed537fe640 1 --2- 172.21.15.12:0/3086624403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed54071110 0x7fed54152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:42.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed58edb640 1 --2- 172.21.15.12:0/3086624403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed540c6bd0 0x7fed54152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:42.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed53fff640 1 -- 172.21.15.12:0/3086624403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed54071110 msgr2=0x7fed54152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:42.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed53fff640 1 --2- 172.21.15.12:0/3086624403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed54071110 0x7fed54152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed53fff640 1 -- 172.21.15.12:0/3086624403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed540c6bd0 msgr2=0x7fed54152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:42.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed53fff640 1 --2- 172.21.15.12:0/3086624403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed540c6bd0 0x7fed54152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed53fff640 1 -- 172.21.15.12:0/3086624403 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4c009a70 con 0x7fed5406f7a0 2023-10-30T12:35:42.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed537fe640 1 --2- 172.21.15.12:0/3086624403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed54071110 0x7fed54152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:35:42.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed53fff640 1 --2- 172.21.15.12:0/3086624403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed5406f7a0 0x7fed541eb530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fed44012b00 tx=0x7fed440100c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:42.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed50ff9640 1 -- 172.21.15.12:0/3086624403 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4400d9b0 con 0x7fed5406f7a0 2023-10-30T12:35:42.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed50ff9640 1 -- 172.21.15.12:0/3086624403 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed44013040 con 0x7fed5406f7a0 2023-10-30T12:35:42.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed50ff9640 1 -- 172.21.15.12:0/3086624403 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44010cf0 con 0x7fed5406f7a0 2023-10-30T12:35:42.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed5a93e640 1 -- 172.21.15.12:0/3086624403 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed54151a60 con 0x7fed5406f7a0 2023-10-30T12:35:42.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.502+0000 7fed5a93e640 1 -- 172.21.15.12:0/3086624403 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed54151d00 con 0x7fed5406f7a0 2023-10-30T12:35:42.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.506+0000 7fed5a93e640 1 -- 172.21.15.12:0/3086624403 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5404e170 con 0x7fed5406f7a0 2023-10-30T12:35:42.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.506+0000 7fed50ff9640 1 -- 172.21.15.12:0/3086624403 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fed44018480 con 0x7fed5406f7a0 2023-10-30T12:35:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.506+0000 7fed50ff9640 1 --2- 172.21.15.12:0/3086624403 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fed400664a0 0x7fed40068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:35:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.506+0000 7fed50ff9640 1 -- 172.21.15.12:0/3086624403 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed44087e40 con 0x7fed5406f7a0 2023-10-30T12:35:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.506+0000 7fed537fe640 1 --2- 172.21.15.12:0/3086624403 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fed400664a0 0x7fed40068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:35:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.506+0000 7fed537fe640 1 --2- 172.21.15.12:0/3086624403 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fed400664a0 0x7fed40068960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fed541e5420 tx=0x7fed48007c80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:35:42.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.510+0000 7fed50ff9640 1 -- 172.21.15.12:0/3086624403 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fed44055350 con 0x7fed5406f7a0 2023-10-30T12:35:42.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.666+0000 7fed5a93e640 1 -- 172.21.15.12:0/3086624403 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed540c8400 con 0x7fed5406f7a0 2023-10-30T12:35:42.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.670+0000 7fed50ff9640 1 -- 172.21.15.12:0/3086624403 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16505 (secure 0 0 0) 0x7fed44058970 con 0x7fed5406f7a0 2023-10-30T12:35:42.672 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:35:42.673 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":22,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:35:07.153665+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:35:03.216053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:35:04.597162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:35:42.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.670+0000 7fed5a93e640 1 -- 172.21.15.12:0/3086624403 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fed400664a0 msgr2=0x7fed40068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.670+0000 7fed5a93e640 1 --2- 172.21.15.12:0/3086624403 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fed400664a0 0x7fed40068960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fed541e5420 tx=0x7fed48007c80 comp rx=0 tx=0).stop 2023-10-30T12:35:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.670+0000 7fed5a93e640 1 -- 172.21.15.12:0/3086624403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed5406f7a0 msgr2=0x7fed541eb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:35:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.670+0000 7fed5a93e640 1 --2- 172.21.15.12:0/3086624403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed5406f7a0 0x7fed541eb530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fed44012b00 tx=0x7fed440100c0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.670+0000 7fed5a93e640 1 -- 172.21.15.12:0/3086624403 shutdown_connections 2023-10-30T12:35:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.670+0000 7fed5a93e640 1 --2- 172.21.15.12:0/3086624403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed540c6bd0 0x7fed54152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.670+0000 7fed5a93e640 1 --2- 172.21.15.12:0/3086624403 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fed400664a0 0x7fed40068960 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.670+0000 7fed5a93e640 1 --2- 172.21.15.12:0/3086624403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed54071110 0x7fed54152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.670+0000 7fed5a93e640 1 --2- 172.21.15.12:0/3086624403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed5406f7a0 0x7fed541eb530 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:35:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.670+0000 7fed5a93e640 1 -- 172.21.15.12:0/3086624403 >> 172.21.15.12:0/3086624403 conn(0x7fed540c7910 msgr2=0x7fed540c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:35:42.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.670+0000 7fed5a93e640 1 -- 172.21.15.12:0/3086624403 shutdown_connections 2023-10-30T12:35:42.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:35:42.670+0000 7fed5a93e640 1 -- 172.21.15.12:0/3086624403 wait complete. 2023-10-30T12:35:42.694 DEBUG:tasks.mds_thrash:Yielding 2023-10-30T12:35:42.695 INFO:teuthology.run_tasks:Running task workunit... 2023-10-30T12:35:42.707 INFO:tasks.workunit:Pulling workunits from ref 00c1c13cea21a686b046584596fbeac5315a7d71 2023-10-30T12:35:42.708 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-10-30T12:35:42.708 INFO:tasks.workunit:timeout=6h 2023-10-30T12:35:42.708 INFO:tasks.workunit:cleanup=True 2023-10-30T12:35:42.709 DEBUG:teuthology.orchestra.run.smithi012:> stat -- /home/ubuntu/cephtest/mnt.0 2023-10-30T12:35:42.710 INFO:tasks.mds_thrash.fs.[cephfs]:starting mds_do_thrash for fs cephfs 2023-10-30T12:35:42.710 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 114 secs before thrashing 2023-10-30T12:35:42.719 INFO:teuthology.orchestra.run.smithi012.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-10-30T12:35:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-10-30T12:35:42.720 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 2bh/43d Inode: 1 Links: 2 2023-10-30T12:35:42.720 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:35:42.720 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2023-10-30T12:35:42.721 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-10-30 12:35:11.183844710 +0000 2023-10-30T12:35:42.721 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-10-30 12:35:35.458312901 +0000 2023-10-30T12:35:42.721 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2023-10-30 12:35:11.183844710 +0000 2023-10-30T12:35:42.721 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-10-30T12:35:42.721 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-10-30T12:35:42.797 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 00c1c13cea21a686b046584596fbeac5315a7d71 2023-10-30T12:35:42.854 INFO:tasks.workunit.client.0.smithi012.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-10-30T12:36:03.222 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:36:03.227 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:36:33.276 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:36:33.285 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:36:45.533 INFO:tasks.workunit.client.0.smithi012.stderr:Updating files: 65% (7882/12071) Updating files: 66% (7967/12071) Updating files: 67% (8088/12071) Updating files: 68% (8209/12071) Updating files: 69% (8329/12071) Updating files: 70% (8450/12071) Updating files: 71% (8571/12071) Updating files: 72% (8692/12071) Updating files: 73% (8812/12071) Updating files: 74% (8933/12071) Updating files: 75% (9054/12071) Updating files: 76% (9174/12071) Updating files: 77% (9295/12071) Updating files: 78% (9416/12071) Updating files: 79% (9537/12071) Updating files: 80% (9657/12071) Updating files: 81% (9778/12071) Updating files: 82% (9899/12071) Updating files: 83% (10019/12071) Updating files: 84% (10140/12071) Updating files: 85% (10261/12071) Updating files: 86% (10382/12071) Updating files: 87% (10502/12071) Updating files: 88% (10623/12071) Updating files: 89% (10744/12071) Updating files: 90% (10864/12071) Updating files: 91% (10985/12071) Updating files: 92% (11106/12071) Updating files: 93% (11227/12071) Updating files: 94% (11347/12071) Updating files: 95% (11468/12071) Updating files: 96% (11589/12071) Updating files: 97% (11709/12071) Updating files: 98% (11830/12071) Updating files: 99% (11951/12071) Updating files: 100% (12071/12071) Updating files: 100% (12071/12071), done. 2023-10-30T12:36:45.958 INFO:tasks.workunit.client.0.smithi012.stderr:Note: switching to '00c1c13cea21a686b046584596fbeac5315a7d71'. 2023-10-30T12:36:45.958 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-10-30T12:36:45.958 INFO:tasks.workunit.client.0.smithi012.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-10-30T12:36:45.958 INFO:tasks.workunit.client.0.smithi012.stderr:changes and commit them, and you can discard any commits you make in this 2023-10-30T12:36:45.959 INFO:tasks.workunit.client.0.smithi012.stderr:state without impacting any branches by switching back to a branch. 2023-10-30T12:36:45.959 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-10-30T12:36:45.959 INFO:tasks.workunit.client.0.smithi012.stderr:If you want to create a new branch to retain commits you create, you may 2023-10-30T12:36:45.959 INFO:tasks.workunit.client.0.smithi012.stderr:do so (now or later) by using -c with the switch command. Example: 2023-10-30T12:36:45.959 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-10-30T12:36:45.960 INFO:tasks.workunit.client.0.smithi012.stderr: git switch -c 2023-10-30T12:36:45.960 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-10-30T12:36:45.960 INFO:tasks.workunit.client.0.smithi012.stderr:Or undo this operation with: 2023-10-30T12:36:45.960 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-10-30T12:36:45.960 INFO:tasks.workunit.client.0.smithi012.stderr: git switch - 2023-10-30T12:36:45.960 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-10-30T12:36:45.961 INFO:tasks.workunit.client.0.smithi012.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-10-30T12:36:45.961 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-10-30T12:36:45.961 INFO:tasks.workunit.client.0.smithi012.stderr:HEAD is now at 00c1c13cea2 mds/MDBalancer: ignore queued callbacks if MDS is not active 2023-10-30T12:36:45.963 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-10-30T12:36:45.973 INFO:tasks.workunit.client.0.smithi012.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-10-30T12:36:45.975 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-30T12:36:45.976 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-10-30T12:36:46.716 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-10-30T12:36:46.833 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-10-30T12:36:46.890 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-30T12:36:46.892 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-30T12:36:46.893 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-10-30T12:36:46.948 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-30T12:36:46.953 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:36:46.954 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-10-30T12:36:47.009 INFO:tasks.workunit:Running workunits matching fs/snaps on client.0... 2023-10-30T12:36:47.011 INFO:tasks.workunit:Running workunit fs/snaps/snap-rm-diff.sh... 2023-10-30T12:36:47.011 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snap-rm-diff.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh 2023-10-30T12:36:47.070 INFO:tasks.workunit.client.0.smithi012.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2023-10-30T12:36:49.412 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir foo 2023-10-30T12:36:49.420 INFO:tasks.workunit.client.0.smithi012.stderr:+ cp linux-2.6.33.tar.bz2 foo 2023-10-30T12:36:49.515 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir foo/.snap/barsnap 2023-10-30T12:36:49.528 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm foo/linux-2.6.33.tar.bz2 2023-10-30T12:36:49.537 INFO:tasks.workunit.client.0.smithi012.stderr:+ diff -q foo/.snap/barsnap/linux-2.6.33.tar.bz2 linux-2.6.33.tar.bz2 2023-10-30T12:36:49.916 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo passed: files are identical 2023-10-30T12:36:49.916 INFO:tasks.workunit.client.0.smithi012.stdout:passed: files are identical 2023-10-30T12:36:49.917 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir foo/.snap/barsnap 2023-10-30T12:36:49.946 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-10-30T12:36:49.947 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo OK 2023-10-30T12:36:49.947 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:36:49.948 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:36:50.527 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-1.sh... 2023-10-30T12:36:50.527 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-1.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-1.sh 2023-10-30T12:36:50.547 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 1 2023-10-30T12:36:50.548 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 2 2023-10-30T12:36:50.549 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 3 2023-10-30T12:36:50.549 INFO:tasks.workunit.client.0.smithi012.stderr:+ '[' -e file4 ']' 2023-10-30T12:36:50.549 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir .snap/snap1 2023-10-30T12:36:51.196 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 4 2023-10-30T12:36:51.197 INFO:tasks.workunit.client.0.smithi012.stderr:++ ls 2023-10-30T12:36:51.204 INFO:tasks.workunit.client.0.smithi012.stderr:+ now='file1 2023-10-30T12:36:51.205 INFO:tasks.workunit.client.0.smithi012.stderr:file2 2023-10-30T12:36:51.205 INFO:tasks.workunit.client.0.smithi012.stderr:file3 2023-10-30T12:36:51.205 INFO:tasks.workunit.client.0.smithi012.stderr:file4' 2023-10-30T12:36:51.205 INFO:tasks.workunit.client.0.smithi012.stderr:++ ls .snap/snap1 2023-10-30T12:36:51.210 INFO:tasks.workunit.client.0.smithi012.stderr:+ then='file1 2023-10-30T12:36:51.210 INFO:tasks.workunit.client.0.smithi012.stderr:file2 2023-10-30T12:36:51.211 INFO:tasks.workunit.client.0.smithi012.stderr:file3' 2023-10-30T12:36:51.211 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir .snap/snap1 2023-10-30T12:36:51.221 INFO:tasks.workunit.client.0.smithi012.stderr:+ '[' 'file1 2023-10-30T12:36:51.221 INFO:tasks.workunit.client.0.smithi012.stderr:file2 2023-10-30T12:36:51.221 INFO:tasks.workunit.client.0.smithi012.stderr:file3 2023-10-30T12:36:51.221 INFO:tasks.workunit.client.0.smithi012.stderr:file4' = 'file1 2023-10-30T12:36:51.221 INFO:tasks.workunit.client.0.smithi012.stderr:file2 2023-10-30T12:36:51.222 INFO:tasks.workunit.client.0.smithi012.stderr:file3' ']' 2023-10-30T12:36:51.222 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 1 2023-10-30T12:36:51.236 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 2 2023-10-30T12:36:51.245 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 3 2023-10-30T12:36:51.257 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir .snap/snap1 2023-10-30T12:36:51.268 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 4 2023-10-30T12:36:51.278 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir .snap/snap1 2023-10-30T12:36:51.288 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm file1 file2 file3 file4 2023-10-30T12:36:51.295 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo OK 2023-10-30T12:36:51.296 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-10-30T12:36:51.296 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:36:51.297 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:36:51.366 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-2.sh... 2023-10-30T12:36:51.366 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-2.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-2.sh 2023-10-30T12:36:51.427 INFO:tasks.workunit.client.0.smithi012.stdout:Create dir 100 to 199 ... 2023-10-30T12:36:51.429 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 100 2023-10-30T12:36:51.435 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 101 2023-10-30T12:36:51.445 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 102 2023-10-30T12:36:51.455 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 103 2023-10-30T12:36:51.465 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 104 2023-10-30T12:36:51.473 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 105 2023-10-30T12:36:51.483 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 106 2023-10-30T12:36:51.492 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 107 2023-10-30T12:36:51.750 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 108 2023-10-30T12:36:52.007 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 109 2023-10-30T12:36:52.016 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 110 2023-10-30T12:36:52.026 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 111 2023-10-30T12:36:52.035 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 112 2023-10-30T12:36:52.044 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 113 2023-10-30T12:36:52.236 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 114 2023-10-30T12:36:52.363 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 115 2023-10-30T12:36:52.823 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 116 2023-10-30T12:36:52.879 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 117 2023-10-30T12:36:52.889 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 118 2023-10-30T12:36:52.899 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 119 2023-10-30T12:36:53.371 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 120 2023-10-30T12:36:53.382 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 121 2023-10-30T12:36:53.393 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 122 2023-10-30T12:36:53.400 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 123 2023-10-30T12:36:53.409 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 124 2023-10-30T12:36:53.418 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 125 2023-10-30T12:36:53.428 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 126 2023-10-30T12:36:53.691 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 127 2023-10-30T12:36:53.701 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 128 2023-10-30T12:36:53.710 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 129 2023-10-30T12:36:53.719 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 130 2023-10-30T12:36:53.728 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 131 2023-10-30T12:36:53.737 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 132 2023-10-30T12:36:53.746 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 133 2023-10-30T12:36:53.755 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 134 2023-10-30T12:36:53.764 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 135 2023-10-30T12:36:53.774 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 136 2023-10-30T12:36:53.783 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 137 2023-10-30T12:36:53.792 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 138 2023-10-30T12:36:53.801 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 139 2023-10-30T12:36:53.810 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 140 2023-10-30T12:36:53.819 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 141 2023-10-30T12:36:53.828 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 142 2023-10-30T12:36:53.837 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 143 2023-10-30T12:36:53.846 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 144 2023-10-30T12:36:53.855 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 145 2023-10-30T12:36:53.864 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 146 2023-10-30T12:36:53.873 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 147 2023-10-30T12:36:53.882 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 148 2023-10-30T12:36:53.891 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 149 2023-10-30T12:36:53.901 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 150 2023-10-30T12:36:53.910 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 151 2023-10-30T12:36:53.919 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 152 2023-10-30T12:36:53.928 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 153 2023-10-30T12:36:53.937 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 154 2023-10-30T12:36:53.946 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 155 2023-10-30T12:36:53.955 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 156 2023-10-30T12:36:53.964 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 157 2023-10-30T12:36:53.973 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 158 2023-10-30T12:36:53.982 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 159 2023-10-30T12:36:53.991 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 160 2023-10-30T12:36:54.000 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 161 2023-10-30T12:36:54.010 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 162 2023-10-30T12:36:54.676 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 163 2023-10-30T12:36:54.688 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 164 2023-10-30T12:36:54.698 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 165 2023-10-30T12:36:54.708 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 166 2023-10-30T12:36:54.723 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 167 2023-10-30T12:36:54.734 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 168 2023-10-30T12:36:54.744 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 169 2023-10-30T12:36:54.755 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 170 2023-10-30T12:36:54.765 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 171 2023-10-30T12:36:54.776 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 172 2023-10-30T12:36:54.786 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 173 2023-10-30T12:36:54.796 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 174 2023-10-30T12:36:55.056 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 175 2023-10-30T12:36:55.066 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 176 2023-10-30T12:36:55.076 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 177 2023-10-30T12:36:55.338 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 178 2023-10-30T12:36:55.599 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 179 2023-10-30T12:36:55.831 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 180 2023-10-30T12:36:56.238 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 181 2023-10-30T12:36:56.250 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 182 2023-10-30T12:36:56.826 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 183 2023-10-30T12:36:56.852 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 184 2023-10-30T12:36:56.862 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 185 2023-10-30T12:36:56.872 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 186 2023-10-30T12:36:56.881 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 187 2023-10-30T12:36:56.891 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 188 2023-10-30T12:36:56.900 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 189 2023-10-30T12:36:56.910 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 190 2023-10-30T12:36:56.919 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 191 2023-10-30T12:36:57.179 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 192 2023-10-30T12:36:57.192 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 193 2023-10-30T12:36:57.212 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 194 2023-10-30T12:36:57.226 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 195 2023-10-30T12:36:57.487 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 196 2023-10-30T12:36:57.501 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 197 2023-10-30T12:36:57.515 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 198 2023-10-30T12:36:57.529 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 199 2023-10-30T12:36:57.543 INFO:tasks.workunit.client.0.smithi012.stdout:Take first snapshot .snap/test1 2023-10-30T12:36:57.645 INFO:tasks.workunit.client.0.smithi012.stdout:Create dir 200 to 299 ... 2023-10-30T12:36:57.647 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 200 2023-10-30T12:36:57.653 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 201 2023-10-30T12:36:57.662 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 202 2023-10-30T12:36:57.751 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 203 2023-10-30T12:36:57.815 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 204 2023-10-30T12:36:57.870 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 205 2023-10-30T12:36:57.882 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 206 2023-10-30T12:36:57.895 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 207 2023-10-30T12:36:57.908 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 208 2023-10-30T12:36:57.920 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 209 2023-10-30T12:36:57.933 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 210 2023-10-30T12:36:57.946 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 211 2023-10-30T12:36:57.959 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 212 2023-10-30T12:36:57.972 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 213 2023-10-30T12:36:57.985 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 214 2023-10-30T12:36:57.997 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 215 2023-10-30T12:36:58.009 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 216 2023-10-30T12:36:58.018 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 217 2023-10-30T12:36:58.027 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 218 2023-10-30T12:36:58.534 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 219 2023-10-30T12:36:58.544 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 220 2023-10-30T12:36:58.554 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 221 2023-10-30T12:36:58.563 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 222 2023-10-30T12:36:58.573 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 223 2023-10-30T12:36:58.582 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 224 2023-10-30T12:36:58.842 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 225 2023-10-30T12:36:58.852 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 226 2023-10-30T12:36:58.861 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 227 2023-10-30T12:36:59.831 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 228 2023-10-30T12:36:59.853 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 229 2023-10-30T12:36:59.863 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 230 2023-10-30T12:36:59.873 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 231 2023-10-30T12:37:00.495 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 232 2023-10-30T12:37:00.506 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 233 2023-10-30T12:37:00.517 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 234 2023-10-30T12:37:00.526 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 235 2023-10-30T12:37:00.536 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 236 2023-10-30T12:37:00.546 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 237 2023-10-30T12:37:00.556 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 238 2023-10-30T12:37:00.823 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 239 2023-10-30T12:37:01.627 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 240 2023-10-30T12:37:01.636 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 241 2023-10-30T12:37:01.646 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 242 2023-10-30T12:37:01.655 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 243 2023-10-30T12:37:01.664 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 244 2023-10-30T12:37:01.674 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 245 2023-10-30T12:37:01.683 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 246 2023-10-30T12:37:01.692 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 247 2023-10-30T12:37:01.703 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 248 2023-10-30T12:37:01.713 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 249 2023-10-30T12:37:02.398 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 250 2023-10-30T12:37:02.839 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 251 2023-10-30T12:37:02.848 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 252 2023-10-30T12:37:02.858 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 253 2023-10-30T12:37:02.867 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 254 2023-10-30T12:37:02.876 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 255 2023-10-30T12:37:02.884 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 256 2023-10-30T12:37:02.893 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 257 2023-10-30T12:37:02.901 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 258 2023-10-30T12:37:02.910 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 259 2023-10-30T12:37:02.919 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 260 2023-10-30T12:37:02.927 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 261 2023-10-30T12:37:02.936 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 262 2023-10-30T12:37:02.944 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 263 2023-10-30T12:37:02.953 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 264 2023-10-30T12:37:02.962 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 265 2023-10-30T12:37:02.970 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 266 2023-10-30T12:37:03.340 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:37:03.344 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:37:03.831 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 267 2023-10-30T12:37:04.095 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 268 2023-10-30T12:37:04.109 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 269 2023-10-30T12:37:04.123 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 270 2023-10-30T12:37:04.137 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 271 2023-10-30T12:37:04.150 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 272 2023-10-30T12:37:04.164 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 273 2023-10-30T12:37:04.177 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 274 2023-10-30T12:37:04.190 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 275 2023-10-30T12:37:04.204 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 276 2023-10-30T12:37:04.217 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 277 2023-10-30T12:37:04.230 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 278 2023-10-30T12:37:04.241 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 279 2023-10-30T12:37:04.254 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 280 2023-10-30T12:37:04.267 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 281 2023-10-30T12:37:04.278 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 282 2023-10-30T12:37:04.287 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 283 2023-10-30T12:37:04.296 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 284 2023-10-30T12:37:04.306 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 285 2023-10-30T12:37:04.316 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 286 2023-10-30T12:37:04.328 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 287 2023-10-30T12:37:04.341 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 288 2023-10-30T12:37:04.354 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 289 2023-10-30T12:37:04.367 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 290 2023-10-30T12:37:04.378 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 291 2023-10-30T12:37:04.389 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 292 2023-10-30T12:37:04.399 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 293 2023-10-30T12:37:04.409 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 294 2023-10-30T12:37:04.667 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 295 2023-10-30T12:37:04.678 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 296 2023-10-30T12:37:04.689 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 297 2023-10-30T12:37:04.702 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 298 2023-10-30T12:37:04.716 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 299 2023-10-30T12:37:04.734 INFO:tasks.workunit.client.0.smithi012.stdout:Create a snapshot in every first level dir ... 2023-10-30T12:37:04.910 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/.snap/snap-subdir-test 2023-10-30T12:37:04.936 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/30 file after the snapshot 2023-10-30T12:37:04.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/31 file after the snapshot 2023-10-30T12:37:04.938 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/32 file after the snapshot 2023-10-30T12:37:04.938 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/33 file after the snapshot 2023-10-30T12:37:04.938 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/34 file after the snapshot 2023-10-30T12:37:04.939 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/35 file after the snapshot 2023-10-30T12:37:04.939 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/36 file after the snapshot 2023-10-30T12:37:04.939 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/37 file after the snapshot 2023-10-30T12:37:04.939 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/38 file after the snapshot 2023-10-30T12:37:04.939 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/39 file after the snapshot 2023-10-30T12:37:04.939 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/.snap/snap-subdir-test 2023-10-30T12:37:04.979 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/30 file after the snapshot 2023-10-30T12:37:04.981 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/31 file after the snapshot 2023-10-30T12:37:04.981 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/32 file after the snapshot 2023-10-30T12:37:04.982 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/33 file after the snapshot 2023-10-30T12:37:04.982 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/34 file after the snapshot 2023-10-30T12:37:04.982 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/35 file after the snapshot 2023-10-30T12:37:04.982 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/36 file after the snapshot 2023-10-30T12:37:04.982 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/37 file after the snapshot 2023-10-30T12:37:04.983 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/38 file after the snapshot 2023-10-30T12:37:04.983 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/39 file after the snapshot 2023-10-30T12:37:04.983 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/.snap/snap-subdir-test 2023-10-30T12:37:05.021 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/30 file after the snapshot 2023-10-30T12:37:05.023 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/31 file after the snapshot 2023-10-30T12:37:05.023 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/32 file after the snapshot 2023-10-30T12:37:05.023 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/33 file after the snapshot 2023-10-30T12:37:05.023 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/34 file after the snapshot 2023-10-30T12:37:05.024 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/35 file after the snapshot 2023-10-30T12:37:05.024 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/36 file after the snapshot 2023-10-30T12:37:05.024 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/37 file after the snapshot 2023-10-30T12:37:05.024 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/38 file after the snapshot 2023-10-30T12:37:05.024 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/39 file after the snapshot 2023-10-30T12:37:05.025 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/.snap/snap-subdir-test 2023-10-30T12:37:05.055 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/30 file after the snapshot 2023-10-30T12:37:05.057 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/31 file after the snapshot 2023-10-30T12:37:05.057 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/32 file after the snapshot 2023-10-30T12:37:05.057 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/33 file after the snapshot 2023-10-30T12:37:05.057 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/34 file after the snapshot 2023-10-30T12:37:05.058 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/35 file after the snapshot 2023-10-30T12:37:05.058 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/36 file after the snapshot 2023-10-30T12:37:05.058 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/37 file after the snapshot 2023-10-30T12:37:05.058 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/38 file after the snapshot 2023-10-30T12:37:05.058 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/39 file after the snapshot 2023-10-30T12:37:05.059 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/.snap/snap-subdir-test 2023-10-30T12:37:05.089 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/30 file after the snapshot 2023-10-30T12:37:05.091 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/31 file after the snapshot 2023-10-30T12:37:05.091 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/32 file after the snapshot 2023-10-30T12:37:05.092 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/33 file after the snapshot 2023-10-30T12:37:05.092 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/34 file after the snapshot 2023-10-30T12:37:05.092 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/35 file after the snapshot 2023-10-30T12:37:05.092 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/36 file after the snapshot 2023-10-30T12:37:05.092 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/37 file after the snapshot 2023-10-30T12:37:05.092 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/38 file after the snapshot 2023-10-30T12:37:05.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/39 file after the snapshot 2023-10-30T12:37:05.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/.snap/snap-subdir-test 2023-10-30T12:37:05.126 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/30 file after the snapshot 2023-10-30T12:37:05.128 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/31 file after the snapshot 2023-10-30T12:37:05.128 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/32 file after the snapshot 2023-10-30T12:37:05.128 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/33 file after the snapshot 2023-10-30T12:37:05.131 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/34 file after the snapshot 2023-10-30T12:37:05.132 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/35 file after the snapshot 2023-10-30T12:37:05.132 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/36 file after the snapshot 2023-10-30T12:37:05.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/37 file after the snapshot 2023-10-30T12:37:05.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/38 file after the snapshot 2023-10-30T12:37:05.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/39 file after the snapshot 2023-10-30T12:37:05.134 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/.snap/snap-subdir-test 2023-10-30T12:37:05.411 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/30 file after the snapshot 2023-10-30T12:37:05.413 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/31 file after the snapshot 2023-10-30T12:37:05.414 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/32 file after the snapshot 2023-10-30T12:37:05.414 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/33 file after the snapshot 2023-10-30T12:37:05.414 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/34 file after the snapshot 2023-10-30T12:37:05.414 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/35 file after the snapshot 2023-10-30T12:37:05.415 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/36 file after the snapshot 2023-10-30T12:37:05.415 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/37 file after the snapshot 2023-10-30T12:37:05.415 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/38 file after the snapshot 2023-10-30T12:37:05.415 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/39 file after the snapshot 2023-10-30T12:37:05.416 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/.snap/snap-subdir-test 2023-10-30T12:37:06.077 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/30 file after the snapshot 2023-10-30T12:37:06.079 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/31 file after the snapshot 2023-10-30T12:37:06.079 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/32 file after the snapshot 2023-10-30T12:37:06.080 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/33 file after the snapshot 2023-10-30T12:37:06.080 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/34 file after the snapshot 2023-10-30T12:37:06.080 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/35 file after the snapshot 2023-10-30T12:37:06.080 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/36 file after the snapshot 2023-10-30T12:37:06.080 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/37 file after the snapshot 2023-10-30T12:37:06.081 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/38 file after the snapshot 2023-10-30T12:37:06.081 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/39 file after the snapshot 2023-10-30T12:37:06.081 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/.snap/snap-subdir-test 2023-10-30T12:37:06.120 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/30 file after the snapshot 2023-10-30T12:37:06.122 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/31 file after the snapshot 2023-10-30T12:37:06.122 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/32 file after the snapshot 2023-10-30T12:37:06.122 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/33 file after the snapshot 2023-10-30T12:37:06.122 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/34 file after the snapshot 2023-10-30T12:37:06.123 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/35 file after the snapshot 2023-10-30T12:37:06.124 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/36 file after the snapshot 2023-10-30T12:37:06.124 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/37 file after the snapshot 2023-10-30T12:37:06.124 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/38 file after the snapshot 2023-10-30T12:37:06.124 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/39 file after the snapshot 2023-10-30T12:37:06.124 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/.snap/snap-subdir-test 2023-10-30T12:37:06.168 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/30 file after the snapshot 2023-10-30T12:37:06.170 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/31 file after the snapshot 2023-10-30T12:37:06.170 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/32 file after the snapshot 2023-10-30T12:37:06.171 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/33 file after the snapshot 2023-10-30T12:37:06.171 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/34 file after the snapshot 2023-10-30T12:37:06.171 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/35 file after the snapshot 2023-10-30T12:37:06.171 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/36 file after the snapshot 2023-10-30T12:37:06.171 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/37 file after the snapshot 2023-10-30T12:37:06.172 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/38 file after the snapshot 2023-10-30T12:37:06.172 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/39 file after the snapshot 2023-10-30T12:37:06.172 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/.snap/snap-subdir-test 2023-10-30T12:37:06.211 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/30 file after the snapshot 2023-10-30T12:37:06.212 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/31 file after the snapshot 2023-10-30T12:37:06.212 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/32 file after the snapshot 2023-10-30T12:37:06.212 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/33 file after the snapshot 2023-10-30T12:37:06.213 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/34 file after the snapshot 2023-10-30T12:37:06.213 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/35 file after the snapshot 2023-10-30T12:37:06.213 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/36 file after the snapshot 2023-10-30T12:37:06.213 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/37 file after the snapshot 2023-10-30T12:37:06.214 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/38 file after the snapshot 2023-10-30T12:37:06.214 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/39 file after the snapshot 2023-10-30T12:37:06.214 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/.snap/snap-subdir-test 2023-10-30T12:37:06.619 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/30 file after the snapshot 2023-10-30T12:37:06.622 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/31 file after the snapshot 2023-10-30T12:37:06.622 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/32 file after the snapshot 2023-10-30T12:37:06.622 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/33 file after the snapshot 2023-10-30T12:37:06.623 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/34 file after the snapshot 2023-10-30T12:37:06.623 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/35 file after the snapshot 2023-10-30T12:37:06.623 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/36 file after the snapshot 2023-10-30T12:37:06.623 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/37 file after the snapshot 2023-10-30T12:37:06.623 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/38 file after the snapshot 2023-10-30T12:37:06.624 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/39 file after the snapshot 2023-10-30T12:37:06.624 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/.snap/snap-subdir-test 2023-10-30T12:37:06.663 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/30 file after the snapshot 2023-10-30T12:37:06.665 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/31 file after the snapshot 2023-10-30T12:37:06.665 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/32 file after the snapshot 2023-10-30T12:37:06.665 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/33 file after the snapshot 2023-10-30T12:37:06.666 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/34 file after the snapshot 2023-10-30T12:37:06.666 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/35 file after the snapshot 2023-10-30T12:37:06.666 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/36 file after the snapshot 2023-10-30T12:37:06.666 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/37 file after the snapshot 2023-10-30T12:37:06.667 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/38 file after the snapshot 2023-10-30T12:37:06.667 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/39 file after the snapshot 2023-10-30T12:37:06.667 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/.snap/snap-subdir-test 2023-10-30T12:37:06.699 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/30 file after the snapshot 2023-10-30T12:37:06.701 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/31 file after the snapshot 2023-10-30T12:37:06.702 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/32 file after the snapshot 2023-10-30T12:37:06.702 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/33 file after the snapshot 2023-10-30T12:37:06.702 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/34 file after the snapshot 2023-10-30T12:37:06.703 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/35 file after the snapshot 2023-10-30T12:37:06.703 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/36 file after the snapshot 2023-10-30T12:37:06.703 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/37 file after the snapshot 2023-10-30T12:37:06.703 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/38 file after the snapshot 2023-10-30T12:37:06.703 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/39 file after the snapshot 2023-10-30T12:37:06.704 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/.snap/snap-subdir-test 2023-10-30T12:37:06.735 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/30 file after the snapshot 2023-10-30T12:37:06.736 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/31 file after the snapshot 2023-10-30T12:37:06.737 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/32 file after the snapshot 2023-10-30T12:37:06.737 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/33 file after the snapshot 2023-10-30T12:37:06.737 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/34 file after the snapshot 2023-10-30T12:37:06.737 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/35 file after the snapshot 2023-10-30T12:37:06.738 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/36 file after the snapshot 2023-10-30T12:37:06.738 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/37 file after the snapshot 2023-10-30T12:37:06.738 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/38 file after the snapshot 2023-10-30T12:37:06.738 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/39 file after the snapshot 2023-10-30T12:37:06.738 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/.snap/snap-subdir-test 2023-10-30T12:37:07.286 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/30 file after the snapshot 2023-10-30T12:37:07.288 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/31 file after the snapshot 2023-10-30T12:37:07.289 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/32 file after the snapshot 2023-10-30T12:37:07.289 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/33 file after the snapshot 2023-10-30T12:37:07.289 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/34 file after the snapshot 2023-10-30T12:37:07.289 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/35 file after the snapshot 2023-10-30T12:37:07.289 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/36 file after the snapshot 2023-10-30T12:37:07.290 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/37 file after the snapshot 2023-10-30T12:37:07.290 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/38 file after the snapshot 2023-10-30T12:37:07.290 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/39 file after the snapshot 2023-10-30T12:37:07.290 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/.snap/snap-subdir-test 2023-10-30T12:37:07.322 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/30 file after the snapshot 2023-10-30T12:37:07.325 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/31 file after the snapshot 2023-10-30T12:37:07.325 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/32 file after the snapshot 2023-10-30T12:37:07.325 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/33 file after the snapshot 2023-10-30T12:37:07.325 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/34 file after the snapshot 2023-10-30T12:37:07.326 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/35 file after the snapshot 2023-10-30T12:37:07.326 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/36 file after the snapshot 2023-10-30T12:37:07.326 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/37 file after the snapshot 2023-10-30T12:37:07.326 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/38 file after the snapshot 2023-10-30T12:37:07.326 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/39 file after the snapshot 2023-10-30T12:37:07.327 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/.snap/snap-subdir-test 2023-10-30T12:37:08.534 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/30 file after the snapshot 2023-10-30T12:37:08.683 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/31 file after the snapshot 2023-10-30T12:37:08.684 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/32 file after the snapshot 2023-10-30T12:37:08.684 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/33 file after the snapshot 2023-10-30T12:37:08.684 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/34 file after the snapshot 2023-10-30T12:37:08.684 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/35 file after the snapshot 2023-10-30T12:37:08.685 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/36 file after the snapshot 2023-10-30T12:37:08.685 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/37 file after the snapshot 2023-10-30T12:37:08.685 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/38 file after the snapshot 2023-10-30T12:37:08.685 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/39 file after the snapshot 2023-10-30T12:37:08.685 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/.snap/snap-subdir-test 2023-10-30T12:37:08.719 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/30 file after the snapshot 2023-10-30T12:37:08.722 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/31 file after the snapshot 2023-10-30T12:37:08.722 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/32 file after the snapshot 2023-10-30T12:37:08.722 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/33 file after the snapshot 2023-10-30T12:37:08.722 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/34 file after the snapshot 2023-10-30T12:37:08.723 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/35 file after the snapshot 2023-10-30T12:37:08.723 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/36 file after the snapshot 2023-10-30T12:37:08.723 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/37 file after the snapshot 2023-10-30T12:37:08.723 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/38 file after the snapshot 2023-10-30T12:37:08.723 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/39 file after the snapshot 2023-10-30T12:37:08.724 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/.snap/snap-subdir-test 2023-10-30T12:37:09.010 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/30 file after the snapshot 2023-10-30T12:37:09.013 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/31 file after the snapshot 2023-10-30T12:37:09.013 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/32 file after the snapshot 2023-10-30T12:37:09.013 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/33 file after the snapshot 2023-10-30T12:37:09.013 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/34 file after the snapshot 2023-10-30T12:37:09.014 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/35 file after the snapshot 2023-10-30T12:37:09.014 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/36 file after the snapshot 2023-10-30T12:37:09.014 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/37 file after the snapshot 2023-10-30T12:37:09.014 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/38 file after the snapshot 2023-10-30T12:37:09.015 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/39 file after the snapshot 2023-10-30T12:37:09.015 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/.snap/snap-subdir-test 2023-10-30T12:37:09.048 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/30 file after the snapshot 2023-10-30T12:37:09.050 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/31 file after the snapshot 2023-10-30T12:37:09.051 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/32 file after the snapshot 2023-10-30T12:37:09.051 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/33 file after the snapshot 2023-10-30T12:37:09.051 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/34 file after the snapshot 2023-10-30T12:37:09.051 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/35 file after the snapshot 2023-10-30T12:37:09.052 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/36 file after the snapshot 2023-10-30T12:37:09.052 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/37 file after the snapshot 2023-10-30T12:37:09.052 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/38 file after the snapshot 2023-10-30T12:37:09.052 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/39 file after the snapshot 2023-10-30T12:37:09.052 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/.snap/snap-subdir-test 2023-10-30T12:37:09.084 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/30 file after the snapshot 2023-10-30T12:37:09.086 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/31 file after the snapshot 2023-10-30T12:37:09.086 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/32 file after the snapshot 2023-10-30T12:37:09.086 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/33 file after the snapshot 2023-10-30T12:37:09.086 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/34 file after the snapshot 2023-10-30T12:37:09.086 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/35 file after the snapshot 2023-10-30T12:37:09.087 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/36 file after the snapshot 2023-10-30T12:37:09.087 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/37 file after the snapshot 2023-10-30T12:37:09.087 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/38 file after the snapshot 2023-10-30T12:37:09.087 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/39 file after the snapshot 2023-10-30T12:37:09.088 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/.snap/snap-subdir-test 2023-10-30T12:37:09.118 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/30 file after the snapshot 2023-10-30T12:37:09.120 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/31 file after the snapshot 2023-10-30T12:37:09.120 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/32 file after the snapshot 2023-10-30T12:37:09.120 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/33 file after the snapshot 2023-10-30T12:37:09.120 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/34 file after the snapshot 2023-10-30T12:37:09.121 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/35 file after the snapshot 2023-10-30T12:37:09.121 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/36 file after the snapshot 2023-10-30T12:37:09.121 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/37 file after the snapshot 2023-10-30T12:37:09.121 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/38 file after the snapshot 2023-10-30T12:37:09.121 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/39 file after the snapshot 2023-10-30T12:37:09.122 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/.snap/snap-subdir-test 2023-10-30T12:37:09.152 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/30 file after the snapshot 2023-10-30T12:37:09.153 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/31 file after the snapshot 2023-10-30T12:37:09.153 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/32 file after the snapshot 2023-10-30T12:37:09.154 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/33 file after the snapshot 2023-10-30T12:37:09.154 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/34 file after the snapshot 2023-10-30T12:37:09.154 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/35 file after the snapshot 2023-10-30T12:37:09.155 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/36 file after the snapshot 2023-10-30T12:37:09.155 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/37 file after the snapshot 2023-10-30T12:37:09.155 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/38 file after the snapshot 2023-10-30T12:37:09.156 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/39 file after the snapshot 2023-10-30T12:37:09.156 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/.snap/snap-subdir-test 2023-10-30T12:37:09.186 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/30 file after the snapshot 2023-10-30T12:37:09.190 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/31 file after the snapshot 2023-10-30T12:37:09.190 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/32 file after the snapshot 2023-10-30T12:37:09.191 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/33 file after the snapshot 2023-10-30T12:37:09.191 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/34 file after the snapshot 2023-10-30T12:37:09.191 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/35 file after the snapshot 2023-10-30T12:37:09.191 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/36 file after the snapshot 2023-10-30T12:37:09.191 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/37 file after the snapshot 2023-10-30T12:37:09.192 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/38 file after the snapshot 2023-10-30T12:37:09.192 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/39 file after the snapshot 2023-10-30T12:37:09.192 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/.snap/snap-subdir-test 2023-10-30T12:37:09.849 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/30 file after the snapshot 2023-10-30T12:37:09.850 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/31 file after the snapshot 2023-10-30T12:37:09.850 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/32 file after the snapshot 2023-10-30T12:37:09.851 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/33 file after the snapshot 2023-10-30T12:37:09.851 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/34 file after the snapshot 2023-10-30T12:37:09.851 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/35 file after the snapshot 2023-10-30T12:37:09.851 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/36 file after the snapshot 2023-10-30T12:37:09.851 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/37 file after the snapshot 2023-10-30T12:37:09.852 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/38 file after the snapshot 2023-10-30T12:37:09.852 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/39 file after the snapshot 2023-10-30T12:37:09.852 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/.snap/snap-subdir-test 2023-10-30T12:37:09.892 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/30 file after the snapshot 2023-10-30T12:37:09.893 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/31 file after the snapshot 2023-10-30T12:37:09.893 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/32 file after the snapshot 2023-10-30T12:37:09.894 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/33 file after the snapshot 2023-10-30T12:37:09.895 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/34 file after the snapshot 2023-10-30T12:37:09.895 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/35 file after the snapshot 2023-10-30T12:37:09.895 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/36 file after the snapshot 2023-10-30T12:37:09.895 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/37 file after the snapshot 2023-10-30T12:37:09.896 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/38 file after the snapshot 2023-10-30T12:37:09.896 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/39 file after the snapshot 2023-10-30T12:37:09.896 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/.snap/snap-subdir-test 2023-10-30T12:37:09.929 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/30 file after the snapshot 2023-10-30T12:37:09.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/31 file after the snapshot 2023-10-30T12:37:09.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/32 file after the snapshot 2023-10-30T12:37:09.932 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/33 file after the snapshot 2023-10-30T12:37:09.932 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/34 file after the snapshot 2023-10-30T12:37:09.932 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/35 file after the snapshot 2023-10-30T12:37:09.932 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/36 file after the snapshot 2023-10-30T12:37:09.932 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/37 file after the snapshot 2023-10-30T12:37:09.933 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/38 file after the snapshot 2023-10-30T12:37:09.933 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/39 file after the snapshot 2023-10-30T12:37:09.933 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/.snap/snap-subdir-test 2023-10-30T12:37:09.965 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/30 file after the snapshot 2023-10-30T12:37:09.967 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/31 file after the snapshot 2023-10-30T12:37:09.967 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/32 file after the snapshot 2023-10-30T12:37:09.967 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/33 file after the snapshot 2023-10-30T12:37:09.968 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/34 file after the snapshot 2023-10-30T12:37:09.968 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/35 file after the snapshot 2023-10-30T12:37:09.968 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/36 file after the snapshot 2023-10-30T12:37:09.968 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/37 file after the snapshot 2023-10-30T12:37:09.968 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/38 file after the snapshot 2023-10-30T12:37:09.969 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/39 file after the snapshot 2023-10-30T12:37:09.969 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/.snap/snap-subdir-test 2023-10-30T12:37:10.003 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/30 file after the snapshot 2023-10-30T12:37:10.004 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/31 file after the snapshot 2023-10-30T12:37:10.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/32 file after the snapshot 2023-10-30T12:37:10.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/33 file after the snapshot 2023-10-30T12:37:10.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/34 file after the snapshot 2023-10-30T12:37:10.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/35 file after the snapshot 2023-10-30T12:37:10.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/36 file after the snapshot 2023-10-30T12:37:10.006 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/37 file after the snapshot 2023-10-30T12:37:10.006 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/38 file after the snapshot 2023-10-30T12:37:10.006 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/39 file after the snapshot 2023-10-30T12:37:10.006 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/.snap/snap-subdir-test 2023-10-30T12:37:10.038 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/30 file after the snapshot 2023-10-30T12:37:10.040 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/31 file after the snapshot 2023-10-30T12:37:10.040 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/32 file after the snapshot 2023-10-30T12:37:10.040 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/33 file after the snapshot 2023-10-30T12:37:10.040 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/34 file after the snapshot 2023-10-30T12:37:10.041 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/35 file after the snapshot 2023-10-30T12:37:10.041 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/36 file after the snapshot 2023-10-30T12:37:10.041 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/37 file after the snapshot 2023-10-30T12:37:10.041 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/38 file after the snapshot 2023-10-30T12:37:10.041 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/39 file after the snapshot 2023-10-30T12:37:10.041 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/.snap/snap-subdir-test 2023-10-30T12:37:10.077 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/30 file after the snapshot 2023-10-30T12:37:10.079 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/31 file after the snapshot 2023-10-30T12:37:10.079 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/32 file after the snapshot 2023-10-30T12:37:10.080 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/33 file after the snapshot 2023-10-30T12:37:10.080 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/34 file after the snapshot 2023-10-30T12:37:10.080 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/35 file after the snapshot 2023-10-30T12:37:10.080 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/36 file after the snapshot 2023-10-30T12:37:10.080 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/37 file after the snapshot 2023-10-30T12:37:10.081 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/38 file after the snapshot 2023-10-30T12:37:10.081 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/39 file after the snapshot 2023-10-30T12:37:10.081 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/.snap/snap-subdir-test 2023-10-30T12:37:10.118 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/30 file after the snapshot 2023-10-30T12:37:10.119 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/31 file after the snapshot 2023-10-30T12:37:10.119 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/32 file after the snapshot 2023-10-30T12:37:10.120 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/33 file after the snapshot 2023-10-30T12:37:10.120 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/34 file after the snapshot 2023-10-30T12:37:10.120 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/35 file after the snapshot 2023-10-30T12:37:10.120 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/36 file after the snapshot 2023-10-30T12:37:10.120 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/37 file after the snapshot 2023-10-30T12:37:10.121 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/38 file after the snapshot 2023-10-30T12:37:10.121 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/39 file after the snapshot 2023-10-30T12:37:10.121 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/.snap/snap-subdir-test 2023-10-30T12:37:10.152 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/30 file after the snapshot 2023-10-30T12:37:10.154 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/31 file after the snapshot 2023-10-30T12:37:10.154 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/32 file after the snapshot 2023-10-30T12:37:10.154 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/33 file after the snapshot 2023-10-30T12:37:10.154 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/34 file after the snapshot 2023-10-30T12:37:10.155 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/35 file after the snapshot 2023-10-30T12:37:10.155 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/36 file after the snapshot 2023-10-30T12:37:10.155 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/37 file after the snapshot 2023-10-30T12:37:10.155 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/38 file after the snapshot 2023-10-30T12:37:10.155 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/39 file after the snapshot 2023-10-30T12:37:10.156 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/.snap/snap-subdir-test 2023-10-30T12:37:10.198 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/30 file after the snapshot 2023-10-30T12:37:10.199 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/31 file after the snapshot 2023-10-30T12:37:10.200 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/32 file after the snapshot 2023-10-30T12:37:10.200 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/33 file after the snapshot 2023-10-30T12:37:10.200 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/34 file after the snapshot 2023-10-30T12:37:10.200 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/35 file after the snapshot 2023-10-30T12:37:10.200 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/36 file after the snapshot 2023-10-30T12:37:10.201 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/37 file after the snapshot 2023-10-30T12:37:10.201 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/38 file after the snapshot 2023-10-30T12:37:10.202 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/39 file after the snapshot 2023-10-30T12:37:10.202 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/.snap/snap-subdir-test 2023-10-30T12:37:10.235 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/30 file after the snapshot 2023-10-30T12:37:10.238 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/31 file after the snapshot 2023-10-30T12:37:10.238 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/32 file after the snapshot 2023-10-30T12:37:10.238 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/33 file after the snapshot 2023-10-30T12:37:10.238 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/34 file after the snapshot 2023-10-30T12:37:10.239 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/35 file after the snapshot 2023-10-30T12:37:10.239 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/36 file after the snapshot 2023-10-30T12:37:10.239 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/37 file after the snapshot 2023-10-30T12:37:10.239 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/38 file after the snapshot 2023-10-30T12:37:10.240 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/39 file after the snapshot 2023-10-30T12:37:10.240 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/.snap/snap-subdir-test 2023-10-30T12:37:10.273 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/30 file after the snapshot 2023-10-30T12:37:10.276 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/31 file after the snapshot 2023-10-30T12:37:10.276 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/32 file after the snapshot 2023-10-30T12:37:10.276 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/33 file after the snapshot 2023-10-30T12:37:10.276 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/34 file after the snapshot 2023-10-30T12:37:10.276 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/35 file after the snapshot 2023-10-30T12:37:10.277 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/36 file after the snapshot 2023-10-30T12:37:10.277 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/37 file after the snapshot 2023-10-30T12:37:10.277 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/38 file after the snapshot 2023-10-30T12:37:10.277 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/39 file after the snapshot 2023-10-30T12:37:10.278 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/.snap/snap-subdir-test 2023-10-30T12:37:10.606 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/30 file after the snapshot 2023-10-30T12:37:10.607 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/31 file after the snapshot 2023-10-30T12:37:10.608 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/32 file after the snapshot 2023-10-30T12:37:10.608 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/33 file after the snapshot 2023-10-30T12:37:10.608 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/34 file after the snapshot 2023-10-30T12:37:10.608 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/35 file after the snapshot 2023-10-30T12:37:10.608 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/36 file after the snapshot 2023-10-30T12:37:10.609 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/37 file after the snapshot 2023-10-30T12:37:10.609 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/38 file after the snapshot 2023-10-30T12:37:10.609 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/39 file after the snapshot 2023-10-30T12:37:10.609 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/.snap/snap-subdir-test 2023-10-30T12:37:10.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/30 file after the snapshot 2023-10-30T12:37:10.643 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/31 file after the snapshot 2023-10-30T12:37:10.644 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/32 file after the snapshot 2023-10-30T12:37:10.644 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/33 file after the snapshot 2023-10-30T12:37:10.644 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/34 file after the snapshot 2023-10-30T12:37:10.644 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/35 file after the snapshot 2023-10-30T12:37:10.644 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/36 file after the snapshot 2023-10-30T12:37:10.645 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/37 file after the snapshot 2023-10-30T12:37:10.645 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/38 file after the snapshot 2023-10-30T12:37:10.645 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/39 file after the snapshot 2023-10-30T12:37:10.645 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/.snap/snap-subdir-test 2023-10-30T12:37:10.677 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/30 file after the snapshot 2023-10-30T12:37:10.678 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/31 file after the snapshot 2023-10-30T12:37:10.678 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/32 file after the snapshot 2023-10-30T12:37:10.678 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/33 file after the snapshot 2023-10-30T12:37:10.679 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/34 file after the snapshot 2023-10-30T12:37:10.679 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/35 file after the snapshot 2023-10-30T12:37:10.679 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/36 file after the snapshot 2023-10-30T12:37:10.680 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/37 file after the snapshot 2023-10-30T12:37:10.680 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/38 file after the snapshot 2023-10-30T12:37:10.680 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/39 file after the snapshot 2023-10-30T12:37:10.680 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/.snap/snap-subdir-test 2023-10-30T12:37:10.723 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/30 file after the snapshot 2023-10-30T12:37:10.725 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/31 file after the snapshot 2023-10-30T12:37:10.725 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/32 file after the snapshot 2023-10-30T12:37:10.725 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/33 file after the snapshot 2023-10-30T12:37:10.725 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/34 file after the snapshot 2023-10-30T12:37:10.725 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/35 file after the snapshot 2023-10-30T12:37:10.726 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/36 file after the snapshot 2023-10-30T12:37:10.726 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/37 file after the snapshot 2023-10-30T12:37:10.726 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/38 file after the snapshot 2023-10-30T12:37:10.726 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/39 file after the snapshot 2023-10-30T12:37:10.727 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/.snap/snap-subdir-test 2023-10-30T12:37:10.769 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/30 file after the snapshot 2023-10-30T12:37:10.771 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/31 file after the snapshot 2023-10-30T12:37:10.771 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/32 file after the snapshot 2023-10-30T12:37:10.771 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/33 file after the snapshot 2023-10-30T12:37:10.771 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/34 file after the snapshot 2023-10-30T12:37:10.771 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/35 file after the snapshot 2023-10-30T12:37:10.772 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/36 file after the snapshot 2023-10-30T12:37:10.772 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/37 file after the snapshot 2023-10-30T12:37:10.772 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/38 file after the snapshot 2023-10-30T12:37:10.772 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/39 file after the snapshot 2023-10-30T12:37:10.772 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/.snap/snap-subdir-test 2023-10-30T12:37:10.841 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/30 file after the snapshot 2023-10-30T12:37:10.842 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/31 file after the snapshot 2023-10-30T12:37:10.843 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/32 file after the snapshot 2023-10-30T12:37:10.843 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/33 file after the snapshot 2023-10-30T12:37:10.843 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/34 file after the snapshot 2023-10-30T12:37:10.843 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/35 file after the snapshot 2023-10-30T12:37:10.844 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/36 file after the snapshot 2023-10-30T12:37:10.844 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/37 file after the snapshot 2023-10-30T12:37:10.844 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/38 file after the snapshot 2023-10-30T12:37:10.844 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/39 file after the snapshot 2023-10-30T12:37:10.844 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/.snap/snap-subdir-test 2023-10-30T12:37:10.877 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/30 file after the snapshot 2023-10-30T12:37:10.879 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/31 file after the snapshot 2023-10-30T12:37:10.879 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/32 file after the snapshot 2023-10-30T12:37:10.879 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/33 file after the snapshot 2023-10-30T12:37:10.879 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/34 file after the snapshot 2023-10-30T12:37:10.880 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/35 file after the snapshot 2023-10-30T12:37:10.880 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/36 file after the snapshot 2023-10-30T12:37:10.880 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/37 file after the snapshot 2023-10-30T12:37:10.880 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/38 file after the snapshot 2023-10-30T12:37:10.880 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/39 file after the snapshot 2023-10-30T12:37:10.881 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/.snap/snap-subdir-test 2023-10-30T12:37:10.914 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/30 file after the snapshot 2023-10-30T12:37:10.916 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/31 file after the snapshot 2023-10-30T12:37:10.916 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/32 file after the snapshot 2023-10-30T12:37:10.916 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/33 file after the snapshot 2023-10-30T12:37:10.916 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/34 file after the snapshot 2023-10-30T12:37:10.917 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/35 file after the snapshot 2023-10-30T12:37:10.917 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/36 file after the snapshot 2023-10-30T12:37:10.917 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/37 file after the snapshot 2023-10-30T12:37:10.917 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/38 file after the snapshot 2023-10-30T12:37:10.917 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/39 file after the snapshot 2023-10-30T12:37:10.918 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/.snap/snap-subdir-test 2023-10-30T12:37:11.639 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/30 file after the snapshot 2023-10-30T12:37:11.641 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/31 file after the snapshot 2023-10-30T12:37:11.641 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/32 file after the snapshot 2023-10-30T12:37:11.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/33 file after the snapshot 2023-10-30T12:37:11.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/34 file after the snapshot 2023-10-30T12:37:11.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/35 file after the snapshot 2023-10-30T12:37:11.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/36 file after the snapshot 2023-10-30T12:37:11.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/37 file after the snapshot 2023-10-30T12:37:11.643 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/38 file after the snapshot 2023-10-30T12:37:11.643 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/39 file after the snapshot 2023-10-30T12:37:11.643 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/.snap/snap-subdir-test 2023-10-30T12:37:11.679 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/30 file after the snapshot 2023-10-30T12:37:11.681 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/31 file after the snapshot 2023-10-30T12:37:11.681 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/32 file after the snapshot 2023-10-30T12:37:11.681 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/33 file after the snapshot 2023-10-30T12:37:11.682 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/34 file after the snapshot 2023-10-30T12:37:11.682 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/35 file after the snapshot 2023-10-30T12:37:11.683 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/36 file after the snapshot 2023-10-30T12:37:11.683 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/37 file after the snapshot 2023-10-30T12:37:11.683 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/38 file after the snapshot 2023-10-30T12:37:11.683 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/39 file after the snapshot 2023-10-30T12:37:11.683 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/.snap/snap-subdir-test 2023-10-30T12:37:11.724 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/30 file after the snapshot 2023-10-30T12:37:11.976 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/31 file after the snapshot 2023-10-30T12:37:11.976 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/32 file after the snapshot 2023-10-30T12:37:11.977 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/33 file after the snapshot 2023-10-30T12:37:11.977 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/34 file after the snapshot 2023-10-30T12:37:11.977 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/35 file after the snapshot 2023-10-30T12:37:11.977 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/36 file after the snapshot 2023-10-30T12:37:11.977 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/37 file after the snapshot 2023-10-30T12:37:11.978 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/38 file after the snapshot 2023-10-30T12:37:11.978 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/39 file after the snapshot 2023-10-30T12:37:11.978 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/.snap/snap-subdir-test 2023-10-30T12:37:12.019 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/30 file after the snapshot 2023-10-30T12:37:12.021 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/31 file after the snapshot 2023-10-30T12:37:12.021 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/32 file after the snapshot 2023-10-30T12:37:12.021 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/33 file after the snapshot 2023-10-30T12:37:12.021 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/34 file after the snapshot 2023-10-30T12:37:12.022 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/35 file after the snapshot 2023-10-30T12:37:12.022 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/36 file after the snapshot 2023-10-30T12:37:12.022 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/37 file after the snapshot 2023-10-30T12:37:12.023 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/38 file after the snapshot 2023-10-30T12:37:12.023 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/39 file after the snapshot 2023-10-30T12:37:12.024 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/.snap/snap-subdir-test 2023-10-30T12:37:12.062 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/30 file after the snapshot 2023-10-30T12:37:12.064 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/31 file after the snapshot 2023-10-30T12:37:12.064 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/32 file after the snapshot 2023-10-30T12:37:12.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/33 file after the snapshot 2023-10-30T12:37:12.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/34 file after the snapshot 2023-10-30T12:37:12.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/35 file after the snapshot 2023-10-30T12:37:12.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/36 file after the snapshot 2023-10-30T12:37:12.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/37 file after the snapshot 2023-10-30T12:37:12.066 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/38 file after the snapshot 2023-10-30T12:37:12.066 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/39 file after the snapshot 2023-10-30T12:37:12.066 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/.snap/snap-subdir-test 2023-10-30T12:37:12.182 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/30 file after the snapshot 2023-10-30T12:37:12.184 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/31 file after the snapshot 2023-10-30T12:37:12.185 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/32 file after the snapshot 2023-10-30T12:37:12.185 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/33 file after the snapshot 2023-10-30T12:37:12.185 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/34 file after the snapshot 2023-10-30T12:37:12.186 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/35 file after the snapshot 2023-10-30T12:37:12.186 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/36 file after the snapshot 2023-10-30T12:37:12.187 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/37 file after the snapshot 2023-10-30T12:37:12.187 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/38 file after the snapshot 2023-10-30T12:37:12.187 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/39 file after the snapshot 2023-10-30T12:37:12.187 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/.snap/snap-subdir-test 2023-10-30T12:37:12.264 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/30 file after the snapshot 2023-10-30T12:37:12.267 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/31 file after the snapshot 2023-10-30T12:37:12.267 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/32 file after the snapshot 2023-10-30T12:37:12.268 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/33 file after the snapshot 2023-10-30T12:37:12.268 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/34 file after the snapshot 2023-10-30T12:37:12.268 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/35 file after the snapshot 2023-10-30T12:37:12.268 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/36 file after the snapshot 2023-10-30T12:37:12.268 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/37 file after the snapshot 2023-10-30T12:37:12.269 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/38 file after the snapshot 2023-10-30T12:37:12.269 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/39 file after the snapshot 2023-10-30T12:37:12.269 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/.snap/snap-subdir-test 2023-10-30T12:37:12.297 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/30 file after the snapshot 2023-10-30T12:37:12.298 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/31 file after the snapshot 2023-10-30T12:37:12.299 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/32 file after the snapshot 2023-10-30T12:37:12.299 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/33 file after the snapshot 2023-10-30T12:37:12.299 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/34 file after the snapshot 2023-10-30T12:37:12.299 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/35 file after the snapshot 2023-10-30T12:37:12.299 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/36 file after the snapshot 2023-10-30T12:37:12.299 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/37 file after the snapshot 2023-10-30T12:37:12.300 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/38 file after the snapshot 2023-10-30T12:37:12.300 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/39 file after the snapshot 2023-10-30T12:37:12.300 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/.snap/snap-subdir-test 2023-10-30T12:37:12.327 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/30 file after the snapshot 2023-10-30T12:37:12.330 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/31 file after the snapshot 2023-10-30T12:37:12.330 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/32 file after the snapshot 2023-10-30T12:37:12.330 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/33 file after the snapshot 2023-10-30T12:37:12.330 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/34 file after the snapshot 2023-10-30T12:37:12.331 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/35 file after the snapshot 2023-10-30T12:37:12.331 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/36 file after the snapshot 2023-10-30T12:37:12.331 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/37 file after the snapshot 2023-10-30T12:37:12.331 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/38 file after the snapshot 2023-10-30T12:37:12.331 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/39 file after the snapshot 2023-10-30T12:37:12.332 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/.snap/snap-subdir-test 2023-10-30T12:37:13.909 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/30 file after the snapshot 2023-10-30T12:37:14.149 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/31 file after the snapshot 2023-10-30T12:37:14.150 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/32 file after the snapshot 2023-10-30T12:37:14.150 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/33 file after the snapshot 2023-10-30T12:37:14.150 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/34 file after the snapshot 2023-10-30T12:37:14.151 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/35 file after the snapshot 2023-10-30T12:37:14.151 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/36 file after the snapshot 2023-10-30T12:37:14.151 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/37 file after the snapshot 2023-10-30T12:37:14.151 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/38 file after the snapshot 2023-10-30T12:37:14.151 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/39 file after the snapshot 2023-10-30T12:37:14.152 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/.snap/snap-subdir-test 2023-10-30T12:37:14.337 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/30 file after the snapshot 2023-10-30T12:37:14.338 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/31 file after the snapshot 2023-10-30T12:37:14.339 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/32 file after the snapshot 2023-10-30T12:37:14.339 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/33 file after the snapshot 2023-10-30T12:37:14.339 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/34 file after the snapshot 2023-10-30T12:37:14.339 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/35 file after the snapshot 2023-10-30T12:37:14.339 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/36 file after the snapshot 2023-10-30T12:37:14.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/37 file after the snapshot 2023-10-30T12:37:14.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/38 file after the snapshot 2023-10-30T12:37:14.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/39 file after the snapshot 2023-10-30T12:37:14.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/.snap/snap-subdir-test 2023-10-30T12:37:14.870 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/30 file after the snapshot 2023-10-30T12:37:14.871 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/31 file after the snapshot 2023-10-30T12:37:14.871 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/32 file after the snapshot 2023-10-30T12:37:14.872 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/33 file after the snapshot 2023-10-30T12:37:14.872 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/34 file after the snapshot 2023-10-30T12:37:14.872 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/35 file after the snapshot 2023-10-30T12:37:14.872 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/36 file after the snapshot 2023-10-30T12:37:14.872 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/37 file after the snapshot 2023-10-30T12:37:14.873 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/38 file after the snapshot 2023-10-30T12:37:14.873 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/39 file after the snapshot 2023-10-30T12:37:14.873 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/.snap/snap-subdir-test 2023-10-30T12:37:14.905 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/30 file after the snapshot 2023-10-30T12:37:14.906 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/31 file after the snapshot 2023-10-30T12:37:14.906 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/32 file after the snapshot 2023-10-30T12:37:14.907 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/33 file after the snapshot 2023-10-30T12:37:14.907 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/34 file after the snapshot 2023-10-30T12:37:14.907 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/35 file after the snapshot 2023-10-30T12:37:14.907 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/36 file after the snapshot 2023-10-30T12:37:14.907 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/37 file after the snapshot 2023-10-30T12:37:14.908 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/38 file after the snapshot 2023-10-30T12:37:14.908 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/39 file after the snapshot 2023-10-30T12:37:14.908 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/.snap/snap-subdir-test 2023-10-30T12:37:14.945 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/30 file after the snapshot 2023-10-30T12:37:14.947 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/31 file after the snapshot 2023-10-30T12:37:14.947 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/32 file after the snapshot 2023-10-30T12:37:14.947 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/33 file after the snapshot 2023-10-30T12:37:14.947 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/34 file after the snapshot 2023-10-30T12:37:14.948 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/35 file after the snapshot 2023-10-30T12:37:14.948 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/36 file after the snapshot 2023-10-30T12:37:14.948 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/37 file after the snapshot 2023-10-30T12:37:14.948 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/38 file after the snapshot 2023-10-30T12:37:14.948 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/39 file after the snapshot 2023-10-30T12:37:14.949 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/.snap/snap-subdir-test 2023-10-30T12:37:14.980 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/30 file after the snapshot 2023-10-30T12:37:14.982 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/31 file after the snapshot 2023-10-30T12:37:14.982 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/32 file after the snapshot 2023-10-30T12:37:14.982 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/33 file after the snapshot 2023-10-30T12:37:14.982 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/34 file after the snapshot 2023-10-30T12:37:14.983 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/35 file after the snapshot 2023-10-30T12:37:14.983 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/36 file after the snapshot 2023-10-30T12:37:14.983 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/37 file after the snapshot 2023-10-30T12:37:14.983 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/38 file after the snapshot 2023-10-30T12:37:14.983 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/39 file after the snapshot 2023-10-30T12:37:14.984 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/.snap/snap-subdir-test 2023-10-30T12:37:15.013 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/30 file after the snapshot 2023-10-30T12:37:15.015 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/31 file after the snapshot 2023-10-30T12:37:15.015 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/32 file after the snapshot 2023-10-30T12:37:15.015 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/33 file after the snapshot 2023-10-30T12:37:15.015 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/34 file after the snapshot 2023-10-30T12:37:15.016 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/35 file after the snapshot 2023-10-30T12:37:15.016 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/36 file after the snapshot 2023-10-30T12:37:15.016 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/37 file after the snapshot 2023-10-30T12:37:15.016 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/38 file after the snapshot 2023-10-30T12:37:15.016 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/39 file after the snapshot 2023-10-30T12:37:15.017 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/.snap/snap-subdir-test 2023-10-30T12:37:15.045 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/30 file after the snapshot 2023-10-30T12:37:15.047 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/31 file after the snapshot 2023-10-30T12:37:15.047 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/32 file after the snapshot 2023-10-30T12:37:15.048 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/33 file after the snapshot 2023-10-30T12:37:15.048 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/34 file after the snapshot 2023-10-30T12:37:15.048 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/35 file after the snapshot 2023-10-30T12:37:15.048 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/36 file after the snapshot 2023-10-30T12:37:15.049 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/37 file after the snapshot 2023-10-30T12:37:15.049 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/38 file after the snapshot 2023-10-30T12:37:15.049 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/39 file after the snapshot 2023-10-30T12:37:15.049 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/.snap/snap-subdir-test 2023-10-30T12:37:15.811 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/30 file after the snapshot 2023-10-30T12:37:15.812 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/31 file after the snapshot 2023-10-30T12:37:15.813 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/32 file after the snapshot 2023-10-30T12:37:15.813 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/33 file after the snapshot 2023-10-30T12:37:15.813 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/34 file after the snapshot 2023-10-30T12:37:15.813 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/35 file after the snapshot 2023-10-30T12:37:15.813 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/36 file after the snapshot 2023-10-30T12:37:15.813 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/37 file after the snapshot 2023-10-30T12:37:15.814 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/38 file after the snapshot 2023-10-30T12:37:15.814 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/39 file after the snapshot 2023-10-30T12:37:15.814 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/.snap/snap-subdir-test 2023-10-30T12:37:15.844 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/30 file after the snapshot 2023-10-30T12:37:15.845 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/31 file after the snapshot 2023-10-30T12:37:15.846 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/32 file after the snapshot 2023-10-30T12:37:15.846 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/33 file after the snapshot 2023-10-30T12:37:15.846 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/34 file after the snapshot 2023-10-30T12:37:15.846 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/35 file after the snapshot 2023-10-30T12:37:15.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/36 file after the snapshot 2023-10-30T12:37:15.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/37 file after the snapshot 2023-10-30T12:37:15.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/38 file after the snapshot 2023-10-30T12:37:15.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/39 file after the snapshot 2023-10-30T12:37:15.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/.snap/snap-subdir-test 2023-10-30T12:37:15.873 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/30 file after the snapshot 2023-10-30T12:37:15.875 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/31 file after the snapshot 2023-10-30T12:37:15.875 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/32 file after the snapshot 2023-10-30T12:37:15.876 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/33 file after the snapshot 2023-10-30T12:37:15.876 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/34 file after the snapshot 2023-10-30T12:37:15.876 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/35 file after the snapshot 2023-10-30T12:37:15.876 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/36 file after the snapshot 2023-10-30T12:37:15.876 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/37 file after the snapshot 2023-10-30T12:37:15.877 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/38 file after the snapshot 2023-10-30T12:37:15.877 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/39 file after the snapshot 2023-10-30T12:37:15.877 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/.snap/snap-subdir-test 2023-10-30T12:37:15.907 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/30 file after the snapshot 2023-10-30T12:37:15.908 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/31 file after the snapshot 2023-10-30T12:37:15.908 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/32 file after the snapshot 2023-10-30T12:37:15.909 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/33 file after the snapshot 2023-10-30T12:37:15.909 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/34 file after the snapshot 2023-10-30T12:37:15.909 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/35 file after the snapshot 2023-10-30T12:37:15.909 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/36 file after the snapshot 2023-10-30T12:37:15.909 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/37 file after the snapshot 2023-10-30T12:37:15.910 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/38 file after the snapshot 2023-10-30T12:37:15.910 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/39 file after the snapshot 2023-10-30T12:37:15.910 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/.snap/snap-subdir-test 2023-10-30T12:37:15.934 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/30 file after the snapshot 2023-10-30T12:37:15.935 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/31 file after the snapshot 2023-10-30T12:37:15.936 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/32 file after the snapshot 2023-10-30T12:37:15.936 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/33 file after the snapshot 2023-10-30T12:37:15.936 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/34 file after the snapshot 2023-10-30T12:37:15.936 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/35 file after the snapshot 2023-10-30T12:37:15.936 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/36 file after the snapshot 2023-10-30T12:37:15.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/37 file after the snapshot 2023-10-30T12:37:15.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/38 file after the snapshot 2023-10-30T12:37:15.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/39 file after the snapshot 2023-10-30T12:37:15.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/.snap/snap-subdir-test 2023-10-30T12:37:15.967 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/30 file after the snapshot 2023-10-30T12:37:15.969 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/31 file after the snapshot 2023-10-30T12:37:15.969 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/32 file after the snapshot 2023-10-30T12:37:15.970 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/33 file after the snapshot 2023-10-30T12:37:15.970 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/34 file after the snapshot 2023-10-30T12:37:15.970 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/35 file after the snapshot 2023-10-30T12:37:15.970 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/36 file after the snapshot 2023-10-30T12:37:15.970 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/37 file after the snapshot 2023-10-30T12:37:15.971 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/38 file after the snapshot 2023-10-30T12:37:15.971 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/39 file after the snapshot 2023-10-30T12:37:15.971 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/.snap/snap-subdir-test 2023-10-30T12:37:16.001 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/30 file after the snapshot 2023-10-30T12:37:16.003 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/31 file after the snapshot 2023-10-30T12:37:16.004 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/32 file after the snapshot 2023-10-30T12:37:16.004 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/33 file after the snapshot 2023-10-30T12:37:16.004 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/34 file after the snapshot 2023-10-30T12:37:16.004 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/35 file after the snapshot 2023-10-30T12:37:16.004 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/36 file after the snapshot 2023-10-30T12:37:16.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/37 file after the snapshot 2023-10-30T12:37:16.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/38 file after the snapshot 2023-10-30T12:37:16.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/39 file after the snapshot 2023-10-30T12:37:16.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/.snap/snap-subdir-test 2023-10-30T12:37:16.027 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/30 file after the snapshot 2023-10-30T12:37:16.279 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/31 file after the snapshot 2023-10-30T12:37:16.279 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/32 file after the snapshot 2023-10-30T12:37:16.280 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/33 file after the snapshot 2023-10-30T12:37:16.280 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/34 file after the snapshot 2023-10-30T12:37:16.280 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/35 file after the snapshot 2023-10-30T12:37:16.280 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/36 file after the snapshot 2023-10-30T12:37:16.281 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/37 file after the snapshot 2023-10-30T12:37:16.281 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/38 file after the snapshot 2023-10-30T12:37:16.281 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/39 file after the snapshot 2023-10-30T12:37:16.282 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/.snap/snap-subdir-test 2023-10-30T12:37:16.311 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/30 file after the snapshot 2023-10-30T12:37:16.313 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/31 file after the snapshot 2023-10-30T12:37:16.313 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/32 file after the snapshot 2023-10-30T12:37:16.313 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/33 file after the snapshot 2023-10-30T12:37:16.313 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/34 file after the snapshot 2023-10-30T12:37:16.314 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/35 file after the snapshot 2023-10-30T12:37:16.314 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/36 file after the snapshot 2023-10-30T12:37:16.314 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/37 file after the snapshot 2023-10-30T12:37:16.314 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/38 file after the snapshot 2023-10-30T12:37:16.314 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/39 file after the snapshot 2023-10-30T12:37:16.315 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/.snap/snap-subdir-test 2023-10-30T12:37:16.338 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/30 file after the snapshot 2023-10-30T12:37:16.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/31 file after the snapshot 2023-10-30T12:37:16.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/32 file after the snapshot 2023-10-30T12:37:16.341 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/33 file after the snapshot 2023-10-30T12:37:16.341 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/34 file after the snapshot 2023-10-30T12:37:16.341 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/35 file after the snapshot 2023-10-30T12:37:16.342 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/36 file after the snapshot 2023-10-30T12:37:16.342 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/37 file after the snapshot 2023-10-30T12:37:16.342 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/38 file after the snapshot 2023-10-30T12:37:16.342 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/39 file after the snapshot 2023-10-30T12:37:16.343 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/.snap/snap-subdir-test 2023-10-30T12:37:17.818 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/30 file after the snapshot 2023-10-30T12:37:17.821 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/31 file after the snapshot 2023-10-30T12:37:17.821 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/32 file after the snapshot 2023-10-30T12:37:17.821 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/33 file after the snapshot 2023-10-30T12:37:17.821 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/34 file after the snapshot 2023-10-30T12:37:17.821 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/35 file after the snapshot 2023-10-30T12:37:17.822 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/36 file after the snapshot 2023-10-30T12:37:17.822 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/37 file after the snapshot 2023-10-30T12:37:17.822 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/38 file after the snapshot 2023-10-30T12:37:17.822 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/39 file after the snapshot 2023-10-30T12:37:17.822 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/.snap/snap-subdir-test 2023-10-30T12:37:18.149 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/30 file after the snapshot 2023-10-30T12:37:18.150 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/31 file after the snapshot 2023-10-30T12:37:18.150 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/32 file after the snapshot 2023-10-30T12:37:18.151 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/33 file after the snapshot 2023-10-30T12:37:18.151 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/34 file after the snapshot 2023-10-30T12:37:18.151 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/35 file after the snapshot 2023-10-30T12:37:18.151 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/36 file after the snapshot 2023-10-30T12:37:18.152 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/37 file after the snapshot 2023-10-30T12:37:18.152 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/38 file after the snapshot 2023-10-30T12:37:18.152 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/39 file after the snapshot 2023-10-30T12:37:18.152 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/.snap/snap-subdir-test 2023-10-30T12:37:18.177 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/30 file after the snapshot 2023-10-30T12:37:18.179 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/31 file after the snapshot 2023-10-30T12:37:18.179 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/32 file after the snapshot 2023-10-30T12:37:18.179 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/33 file after the snapshot 2023-10-30T12:37:18.180 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/34 file after the snapshot 2023-10-30T12:37:18.180 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/35 file after the snapshot 2023-10-30T12:37:18.180 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/36 file after the snapshot 2023-10-30T12:37:18.180 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/37 file after the snapshot 2023-10-30T12:37:18.180 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/38 file after the snapshot 2023-10-30T12:37:18.180 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/39 file after the snapshot 2023-10-30T12:37:18.181 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/.snap/snap-subdir-test 2023-10-30T12:37:18.208 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/30 file after the snapshot 2023-10-30T12:37:18.210 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/31 file after the snapshot 2023-10-30T12:37:18.211 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/32 file after the snapshot 2023-10-30T12:37:18.211 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/33 file after the snapshot 2023-10-30T12:37:18.211 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/34 file after the snapshot 2023-10-30T12:37:18.211 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/35 file after the snapshot 2023-10-30T12:37:18.212 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/36 file after the snapshot 2023-10-30T12:37:18.212 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/37 file after the snapshot 2023-10-30T12:37:18.212 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/38 file after the snapshot 2023-10-30T12:37:18.212 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/39 file after the snapshot 2023-10-30T12:37:18.212 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/.snap/snap-subdir-test 2023-10-30T12:37:18.235 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/30 file after the snapshot 2023-10-30T12:37:18.249 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/31 file after the snapshot 2023-10-30T12:37:18.252 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/32 file after the snapshot 2023-10-30T12:37:18.257 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/33 file after the snapshot 2023-10-30T12:37:18.257 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/34 file after the snapshot 2023-10-30T12:37:18.258 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/35 file after the snapshot 2023-10-30T12:37:18.258 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/36 file after the snapshot 2023-10-30T12:37:18.258 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/37 file after the snapshot 2023-10-30T12:37:18.258 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/38 file after the snapshot 2023-10-30T12:37:18.258 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/39 file after the snapshot 2023-10-30T12:37:18.259 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/.snap/snap-subdir-test 2023-10-30T12:37:18.334 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/30 file after the snapshot 2023-10-30T12:37:18.335 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/31 file after the snapshot 2023-10-30T12:37:18.336 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/32 file after the snapshot 2023-10-30T12:37:18.336 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/33 file after the snapshot 2023-10-30T12:37:18.337 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/34 file after the snapshot 2023-10-30T12:37:18.337 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/35 file after the snapshot 2023-10-30T12:37:18.337 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/36 file after the snapshot 2023-10-30T12:37:18.337 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/37 file after the snapshot 2023-10-30T12:37:18.337 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/38 file after the snapshot 2023-10-30T12:37:18.337 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/39 file after the snapshot 2023-10-30T12:37:18.338 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/.snap/snap-subdir-test 2023-10-30T12:37:18.369 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/30 file after the snapshot 2023-10-30T12:37:18.371 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/31 file after the snapshot 2023-10-30T12:37:18.371 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/32 file after the snapshot 2023-10-30T12:37:18.371 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/33 file after the snapshot 2023-10-30T12:37:18.371 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/34 file after the snapshot 2023-10-30T12:37:18.372 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/35 file after the snapshot 2023-10-30T12:37:18.372 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/36 file after the snapshot 2023-10-30T12:37:18.372 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/37 file after the snapshot 2023-10-30T12:37:18.372 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/38 file after the snapshot 2023-10-30T12:37:18.372 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/39 file after the snapshot 2023-10-30T12:37:18.373 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/.snap/snap-subdir-test 2023-10-30T12:37:18.399 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/30 file after the snapshot 2023-10-30T12:37:18.401 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/31 file after the snapshot 2023-10-30T12:37:18.402 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/32 file after the snapshot 2023-10-30T12:37:18.402 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/33 file after the snapshot 2023-10-30T12:37:18.402 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/34 file after the snapshot 2023-10-30T12:37:18.402 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/35 file after the snapshot 2023-10-30T12:37:18.402 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/36 file after the snapshot 2023-10-30T12:37:18.403 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/37 file after the snapshot 2023-10-30T12:37:18.403 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/38 file after the snapshot 2023-10-30T12:37:18.404 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/39 file after the snapshot 2023-10-30T12:37:18.404 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/.snap/snap-subdir-test 2023-10-30T12:37:18.690 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/30 file after the snapshot 2023-10-30T12:37:18.692 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/31 file after the snapshot 2023-10-30T12:37:18.693 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/32 file after the snapshot 2023-10-30T12:37:18.693 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/33 file after the snapshot 2023-10-30T12:37:18.693 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/34 file after the snapshot 2023-10-30T12:37:18.693 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/35 file after the snapshot 2023-10-30T12:37:18.694 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/36 file after the snapshot 2023-10-30T12:37:18.694 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/37 file after the snapshot 2023-10-30T12:37:18.694 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/38 file after the snapshot 2023-10-30T12:37:18.694 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/39 file after the snapshot 2023-10-30T12:37:18.694 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/.snap/snap-subdir-test 2023-10-30T12:37:18.712 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/30 file after the snapshot 2023-10-30T12:37:18.714 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/31 file after the snapshot 2023-10-30T12:37:18.714 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/32 file after the snapshot 2023-10-30T12:37:18.714 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/33 file after the snapshot 2023-10-30T12:37:18.715 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/34 file after the snapshot 2023-10-30T12:37:18.715 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/35 file after the snapshot 2023-10-30T12:37:18.715 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/36 file after the snapshot 2023-10-30T12:37:18.715 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/37 file after the snapshot 2023-10-30T12:37:18.715 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/38 file after the snapshot 2023-10-30T12:37:18.716 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/39 file after the snapshot 2023-10-30T12:37:18.716 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/.snap/snap-subdir-test 2023-10-30T12:37:18.747 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/30 file after the snapshot 2023-10-30T12:37:18.750 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/31 file after the snapshot 2023-10-30T12:37:18.750 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/32 file after the snapshot 2023-10-30T12:37:18.751 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/33 file after the snapshot 2023-10-30T12:37:18.751 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/34 file after the snapshot 2023-10-30T12:37:18.751 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/35 file after the snapshot 2023-10-30T12:37:18.751 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/36 file after the snapshot 2023-10-30T12:37:18.751 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/37 file after the snapshot 2023-10-30T12:37:18.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/38 file after the snapshot 2023-10-30T12:37:18.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/39 file after the snapshot 2023-10-30T12:37:18.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/.snap/snap-subdir-test 2023-10-30T12:37:18.767 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/30 file after the snapshot 2023-10-30T12:37:18.769 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/31 file after the snapshot 2023-10-30T12:37:18.769 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/32 file after the snapshot 2023-10-30T12:37:18.769 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/33 file after the snapshot 2023-10-30T12:37:18.770 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/34 file after the snapshot 2023-10-30T12:37:18.770 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/35 file after the snapshot 2023-10-30T12:37:18.770 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/36 file after the snapshot 2023-10-30T12:37:18.770 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/37 file after the snapshot 2023-10-30T12:37:18.770 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/38 file after the snapshot 2023-10-30T12:37:18.771 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/39 file after the snapshot 2023-10-30T12:37:18.771 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/.snap/snap-subdir-test 2023-10-30T12:37:19.301 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/30 file after the snapshot 2023-10-30T12:37:19.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/31 file after the snapshot 2023-10-30T12:37:19.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/32 file after the snapshot 2023-10-30T12:37:19.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/33 file after the snapshot 2023-10-30T12:37:19.305 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/34 file after the snapshot 2023-10-30T12:37:19.305 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/35 file after the snapshot 2023-10-30T12:37:19.305 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/36 file after the snapshot 2023-10-30T12:37:19.305 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/37 file after the snapshot 2023-10-30T12:37:19.305 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/38 file after the snapshot 2023-10-30T12:37:19.306 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/39 file after the snapshot 2023-10-30T12:37:19.306 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/.snap/snap-subdir-test 2023-10-30T12:37:19.348 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/30 file after the snapshot 2023-10-30T12:37:19.351 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/31 file after the snapshot 2023-10-30T12:37:19.352 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/32 file after the snapshot 2023-10-30T12:37:19.352 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/33 file after the snapshot 2023-10-30T12:37:19.352 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/34 file after the snapshot 2023-10-30T12:37:19.352 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/35 file after the snapshot 2023-10-30T12:37:19.353 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/36 file after the snapshot 2023-10-30T12:37:19.353 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/37 file after the snapshot 2023-10-30T12:37:19.353 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/38 file after the snapshot 2023-10-30T12:37:19.353 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/39 file after the snapshot 2023-10-30T12:37:19.353 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/.snap/snap-subdir-test 2023-10-30T12:37:19.375 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/30 file after the snapshot 2023-10-30T12:37:19.377 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/31 file after the snapshot 2023-10-30T12:37:19.378 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/32 file after the snapshot 2023-10-30T12:37:19.378 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/33 file after the snapshot 2023-10-30T12:37:19.378 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/34 file after the snapshot 2023-10-30T12:37:19.378 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/35 file after the snapshot 2023-10-30T12:37:19.379 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/36 file after the snapshot 2023-10-30T12:37:19.379 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/37 file after the snapshot 2023-10-30T12:37:19.379 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/38 file after the snapshot 2023-10-30T12:37:19.379 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/39 file after the snapshot 2023-10-30T12:37:19.379 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/.snap/snap-subdir-test 2023-10-30T12:37:19.420 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/30 file after the snapshot 2023-10-30T12:37:19.423 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/31 file after the snapshot 2023-10-30T12:37:19.423 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/32 file after the snapshot 2023-10-30T12:37:19.424 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/33 file after the snapshot 2023-10-30T12:37:19.424 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/34 file after the snapshot 2023-10-30T12:37:19.424 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/35 file after the snapshot 2023-10-30T12:37:19.425 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/36 file after the snapshot 2023-10-30T12:37:19.425 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/37 file after the snapshot 2023-10-30T12:37:19.425 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/38 file after the snapshot 2023-10-30T12:37:19.425 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/39 file after the snapshot 2023-10-30T12:37:19.426 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/.snap/snap-subdir-test 2023-10-30T12:37:19.815 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/30 file after the snapshot 2023-10-30T12:37:19.817 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/31 file after the snapshot 2023-10-30T12:37:19.817 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/32 file after the snapshot 2023-10-30T12:37:19.817 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/33 file after the snapshot 2023-10-30T12:37:19.818 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/34 file after the snapshot 2023-10-30T12:37:19.818 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/35 file after the snapshot 2023-10-30T12:37:19.818 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/36 file after the snapshot 2023-10-30T12:37:19.818 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/37 file after the snapshot 2023-10-30T12:37:19.818 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/38 file after the snapshot 2023-10-30T12:37:19.819 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/39 file after the snapshot 2023-10-30T12:37:19.819 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/.snap/snap-subdir-test 2023-10-30T12:37:19.853 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/30 file after the snapshot 2023-10-30T12:37:19.856 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/31 file after the snapshot 2023-10-30T12:37:19.856 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/32 file after the snapshot 2023-10-30T12:37:19.857 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/33 file after the snapshot 2023-10-30T12:37:19.857 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/34 file after the snapshot 2023-10-30T12:37:19.857 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/35 file after the snapshot 2023-10-30T12:37:19.857 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/36 file after the snapshot 2023-10-30T12:37:19.857 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/37 file after the snapshot 2023-10-30T12:37:19.858 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/38 file after the snapshot 2023-10-30T12:37:19.858 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/39 file after the snapshot 2023-10-30T12:37:19.858 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/.snap/snap-subdir-test 2023-10-30T12:37:20.370 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/30 file after the snapshot 2023-10-30T12:37:20.372 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/31 file after the snapshot 2023-10-30T12:37:20.373 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/32 file after the snapshot 2023-10-30T12:37:20.373 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/33 file after the snapshot 2023-10-30T12:37:20.373 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/34 file after the snapshot 2023-10-30T12:37:20.373 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/35 file after the snapshot 2023-10-30T12:37:20.373 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/36 file after the snapshot 2023-10-30T12:37:20.374 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/37 file after the snapshot 2023-10-30T12:37:20.374 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/38 file after the snapshot 2023-10-30T12:37:20.374 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/39 file after the snapshot 2023-10-30T12:37:20.374 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/.snap/snap-subdir-test 2023-10-30T12:37:20.416 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/30 file after the snapshot 2023-10-30T12:37:20.419 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/31 file after the snapshot 2023-10-30T12:37:20.420 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/32 file after the snapshot 2023-10-30T12:37:20.420 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/33 file after the snapshot 2023-10-30T12:37:20.420 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/34 file after the snapshot 2023-10-30T12:37:20.420 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/35 file after the snapshot 2023-10-30T12:37:20.421 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/36 file after the snapshot 2023-10-30T12:37:20.421 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/37 file after the snapshot 2023-10-30T12:37:20.421 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/38 file after the snapshot 2023-10-30T12:37:20.421 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/39 file after the snapshot 2023-10-30T12:37:20.421 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/.snap/snap-subdir-test 2023-10-30T12:37:20.465 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/30 file after the snapshot 2023-10-30T12:37:20.468 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/31 file after the snapshot 2023-10-30T12:37:20.468 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/32 file after the snapshot 2023-10-30T12:37:20.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/33 file after the snapshot 2023-10-30T12:37:20.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/34 file after the snapshot 2023-10-30T12:37:20.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/35 file after the snapshot 2023-10-30T12:37:20.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/36 file after the snapshot 2023-10-30T12:37:20.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/37 file after the snapshot 2023-10-30T12:37:20.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/38 file after the snapshot 2023-10-30T12:37:20.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/39 file after the snapshot 2023-10-30T12:37:20.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/.snap/snap-subdir-test 2023-10-30T12:37:20.489 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/30 file after the snapshot 2023-10-30T12:37:20.491 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/31 file after the snapshot 2023-10-30T12:37:20.491 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/32 file after the snapshot 2023-10-30T12:37:20.491 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/33 file after the snapshot 2023-10-30T12:37:20.491 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/34 file after the snapshot 2023-10-30T12:37:20.491 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/35 file after the snapshot 2023-10-30T12:37:20.492 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/36 file after the snapshot 2023-10-30T12:37:20.492 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/37 file after the snapshot 2023-10-30T12:37:20.492 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/38 file after the snapshot 2023-10-30T12:37:20.492 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/39 file after the snapshot 2023-10-30T12:37:20.492 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/.snap/snap-subdir-test 2023-10-30T12:37:20.522 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/30 file after the snapshot 2023-10-30T12:37:20.525 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/31 file after the snapshot 2023-10-30T12:37:20.526 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/32 file after the snapshot 2023-10-30T12:37:20.526 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/33 file after the snapshot 2023-10-30T12:37:20.526 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/34 file after the snapshot 2023-10-30T12:37:20.526 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/35 file after the snapshot 2023-10-30T12:37:20.526 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/36 file after the snapshot 2023-10-30T12:37:20.527 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/37 file after the snapshot 2023-10-30T12:37:20.527 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/38 file after the snapshot 2023-10-30T12:37:20.527 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/39 file after the snapshot 2023-10-30T12:37:20.527 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/.snap/snap-subdir-test 2023-10-30T12:37:20.546 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/30 file after the snapshot 2023-10-30T12:37:20.548 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/31 file after the snapshot 2023-10-30T12:37:20.548 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/32 file after the snapshot 2023-10-30T12:37:20.548 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/33 file after the snapshot 2023-10-30T12:37:20.549 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/34 file after the snapshot 2023-10-30T12:37:20.549 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/35 file after the snapshot 2023-10-30T12:37:20.549 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/36 file after the snapshot 2023-10-30T12:37:20.549 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/37 file after the snapshot 2023-10-30T12:37:20.549 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/38 file after the snapshot 2023-10-30T12:37:20.550 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/39 file after the snapshot 2023-10-30T12:37:20.550 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/.snap/snap-subdir-test 2023-10-30T12:37:20.580 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/30 file after the snapshot 2023-10-30T12:37:20.582 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/31 file after the snapshot 2023-10-30T12:37:20.582 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/32 file after the snapshot 2023-10-30T12:37:20.583 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/33 file after the snapshot 2023-10-30T12:37:20.583 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/34 file after the snapshot 2023-10-30T12:37:20.583 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/35 file after the snapshot 2023-10-30T12:37:20.583 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/36 file after the snapshot 2023-10-30T12:37:20.583 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/37 file after the snapshot 2023-10-30T12:37:20.584 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/38 file after the snapshot 2023-10-30T12:37:20.584 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/39 file after the snapshot 2023-10-30T12:37:20.584 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/.snap/snap-subdir-test 2023-10-30T12:37:21.376 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/30 file after the snapshot 2023-10-30T12:37:21.378 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/31 file after the snapshot 2023-10-30T12:37:21.379 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/32 file after the snapshot 2023-10-30T12:37:21.379 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/33 file after the snapshot 2023-10-30T12:37:21.379 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/34 file after the snapshot 2023-10-30T12:37:21.379 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/35 file after the snapshot 2023-10-30T12:37:21.380 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/36 file after the snapshot 2023-10-30T12:37:21.380 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/37 file after the snapshot 2023-10-30T12:37:21.380 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/38 file after the snapshot 2023-10-30T12:37:21.381 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/39 file after the snapshot 2023-10-30T12:37:21.381 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/.snap/snap-subdir-test 2023-10-30T12:37:21.441 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/30 file after the snapshot 2023-10-30T12:37:21.444 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/31 file after the snapshot 2023-10-30T12:37:21.444 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/32 file after the snapshot 2023-10-30T12:37:21.445 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/33 file after the snapshot 2023-10-30T12:37:21.445 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/34 file after the snapshot 2023-10-30T12:37:21.445 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/35 file after the snapshot 2023-10-30T12:37:21.445 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/36 file after the snapshot 2023-10-30T12:37:21.445 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/37 file after the snapshot 2023-10-30T12:37:21.446 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/38 file after the snapshot 2023-10-30T12:37:21.446 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/39 file after the snapshot 2023-10-30T12:37:21.446 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/.snap/snap-subdir-test 2023-10-30T12:37:21.464 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/30 file after the snapshot 2023-10-30T12:37:21.465 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/31 file after the snapshot 2023-10-30T12:37:21.465 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/32 file after the snapshot 2023-10-30T12:37:21.466 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/33 file after the snapshot 2023-10-30T12:37:21.466 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/34 file after the snapshot 2023-10-30T12:37:21.466 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/35 file after the snapshot 2023-10-30T12:37:21.466 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/36 file after the snapshot 2023-10-30T12:37:21.466 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/37 file after the snapshot 2023-10-30T12:37:21.467 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/38 file after the snapshot 2023-10-30T12:37:21.467 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/39 file after the snapshot 2023-10-30T12:37:21.467 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/.snap/snap-subdir-test 2023-10-30T12:37:21.490 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/30 file after the snapshot 2023-10-30T12:37:21.492 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/31 file after the snapshot 2023-10-30T12:37:21.492 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/32 file after the snapshot 2023-10-30T12:37:21.492 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/33 file after the snapshot 2023-10-30T12:37:21.493 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/34 file after the snapshot 2023-10-30T12:37:21.493 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/35 file after the snapshot 2023-10-30T12:37:21.493 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/36 file after the snapshot 2023-10-30T12:37:21.493 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/37 file after the snapshot 2023-10-30T12:37:21.493 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/38 file after the snapshot 2023-10-30T12:37:21.493 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/39 file after the snapshot 2023-10-30T12:37:21.494 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/.snap/snap-subdir-test 2023-10-30T12:37:21.516 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/30 file after the snapshot 2023-10-30T12:37:21.518 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/31 file after the snapshot 2023-10-30T12:37:21.518 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/32 file after the snapshot 2023-10-30T12:37:21.519 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/33 file after the snapshot 2023-10-30T12:37:21.519 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/34 file after the snapshot 2023-10-30T12:37:21.519 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/35 file after the snapshot 2023-10-30T12:37:21.519 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/36 file after the snapshot 2023-10-30T12:37:21.519 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/37 file after the snapshot 2023-10-30T12:37:21.520 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/38 file after the snapshot 2023-10-30T12:37:21.520 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/39 file after the snapshot 2023-10-30T12:37:21.520 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/.snap/snap-subdir-test 2023-10-30T12:37:21.557 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/30 file after the snapshot 2023-10-30T12:37:21.560 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/31 file after the snapshot 2023-10-30T12:37:21.560 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/32 file after the snapshot 2023-10-30T12:37:21.560 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/33 file after the snapshot 2023-10-30T12:37:21.561 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/34 file after the snapshot 2023-10-30T12:37:21.561 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/35 file after the snapshot 2023-10-30T12:37:21.561 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/36 file after the snapshot 2023-10-30T12:37:21.561 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/37 file after the snapshot 2023-10-30T12:37:21.561 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/38 file after the snapshot 2023-10-30T12:37:21.562 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/39 file after the snapshot 2023-10-30T12:37:21.562 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/.snap/snap-subdir-test 2023-10-30T12:37:21.579 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/30 file after the snapshot 2023-10-30T12:37:21.581 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/31 file after the snapshot 2023-10-30T12:37:21.581 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/32 file after the snapshot 2023-10-30T12:37:21.581 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/33 file after the snapshot 2023-10-30T12:37:21.582 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/34 file after the snapshot 2023-10-30T12:37:21.582 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/35 file after the snapshot 2023-10-30T12:37:21.582 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/36 file after the snapshot 2023-10-30T12:37:21.582 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/37 file after the snapshot 2023-10-30T12:37:21.582 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/38 file after the snapshot 2023-10-30T12:37:21.583 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/39 file after the snapshot 2023-10-30T12:37:21.583 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/.snap/snap-subdir-test 2023-10-30T12:37:21.623 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/30 file after the snapshot 2023-10-30T12:37:21.627 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/31 file after the snapshot 2023-10-30T12:37:21.627 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/32 file after the snapshot 2023-10-30T12:37:21.627 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/33 file after the snapshot 2023-10-30T12:37:21.627 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/34 file after the snapshot 2023-10-30T12:37:21.627 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/35 file after the snapshot 2023-10-30T12:37:21.628 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/36 file after the snapshot 2023-10-30T12:37:21.628 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/37 file after the snapshot 2023-10-30T12:37:21.628 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/38 file after the snapshot 2023-10-30T12:37:21.628 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/39 file after the snapshot 2023-10-30T12:37:21.628 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/.snap/snap-subdir-test 2023-10-30T12:37:21.646 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/30 file after the snapshot 2023-10-30T12:37:21.648 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/31 file after the snapshot 2023-10-30T12:37:21.648 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/32 file after the snapshot 2023-10-30T12:37:21.648 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/33 file after the snapshot 2023-10-30T12:37:21.649 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/34 file after the snapshot 2023-10-30T12:37:21.649 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/35 file after the snapshot 2023-10-30T12:37:21.649 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/36 file after the snapshot 2023-10-30T12:37:21.649 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/37 file after the snapshot 2023-10-30T12:37:21.649 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/38 file after the snapshot 2023-10-30T12:37:21.649 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/39 file after the snapshot 2023-10-30T12:37:21.650 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/.snap/snap-subdir-test 2023-10-30T12:37:21.671 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/30 file after the snapshot 2023-10-30T12:37:21.673 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/31 file after the snapshot 2023-10-30T12:37:21.673 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/32 file after the snapshot 2023-10-30T12:37:21.673 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/33 file after the snapshot 2023-10-30T12:37:21.673 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/34 file after the snapshot 2023-10-30T12:37:21.673 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/35 file after the snapshot 2023-10-30T12:37:21.674 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/36 file after the snapshot 2023-10-30T12:37:21.674 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/37 file after the snapshot 2023-10-30T12:37:21.674 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/38 file after the snapshot 2023-10-30T12:37:21.674 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/39 file after the snapshot 2023-10-30T12:37:21.674 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/.snap/snap-subdir-test 2023-10-30T12:37:21.698 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/30 file after the snapshot 2023-10-30T12:37:21.699 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/31 file after the snapshot 2023-10-30T12:37:21.699 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/32 file after the snapshot 2023-10-30T12:37:21.700 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/33 file after the snapshot 2023-10-30T12:37:21.700 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/34 file after the snapshot 2023-10-30T12:37:21.700 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/35 file after the snapshot 2023-10-30T12:37:21.700 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/36 file after the snapshot 2023-10-30T12:37:21.700 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/37 file after the snapshot 2023-10-30T12:37:21.701 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/38 file after the snapshot 2023-10-30T12:37:21.701 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/39 file after the snapshot 2023-10-30T12:37:21.701 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/.snap/snap-subdir-test 2023-10-30T12:37:21.722 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/30 file after the snapshot 2023-10-30T12:37:21.724 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/31 file after the snapshot 2023-10-30T12:37:21.724 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/32 file after the snapshot 2023-10-30T12:37:21.724 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/33 file after the snapshot 2023-10-30T12:37:21.725 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/34 file after the snapshot 2023-10-30T12:37:21.725 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/35 file after the snapshot 2023-10-30T12:37:21.725 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/36 file after the snapshot 2023-10-30T12:37:21.725 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/37 file after the snapshot 2023-10-30T12:37:21.726 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/38 file after the snapshot 2023-10-30T12:37:21.726 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/39 file after the snapshot 2023-10-30T12:37:21.726 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/.snap/snap-subdir-test 2023-10-30T12:37:21.748 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/30 file after the snapshot 2023-10-30T12:37:21.750 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/31 file after the snapshot 2023-10-30T12:37:21.750 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/32 file after the snapshot 2023-10-30T12:37:21.751 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/33 file after the snapshot 2023-10-30T12:37:21.751 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/34 file after the snapshot 2023-10-30T12:37:21.751 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/35 file after the snapshot 2023-10-30T12:37:21.751 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/36 file after the snapshot 2023-10-30T12:37:21.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/37 file after the snapshot 2023-10-30T12:37:21.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/38 file after the snapshot 2023-10-30T12:37:21.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/39 file after the snapshot 2023-10-30T12:37:21.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/.snap/snap-subdir-test 2023-10-30T12:37:22.844 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/30 file after the snapshot 2023-10-30T12:37:22.846 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/31 file after the snapshot 2023-10-30T12:37:22.846 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/32 file after the snapshot 2023-10-30T12:37:22.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/33 file after the snapshot 2023-10-30T12:37:22.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/34 file after the snapshot 2023-10-30T12:37:22.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/35 file after the snapshot 2023-10-30T12:37:22.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/36 file after the snapshot 2023-10-30T12:37:22.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/37 file after the snapshot 2023-10-30T12:37:22.848 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/38 file after the snapshot 2023-10-30T12:37:22.848 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/39 file after the snapshot 2023-10-30T12:37:22.848 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/.snap/snap-subdir-test 2023-10-30T12:37:23.412 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/30 file after the snapshot 2023-10-30T12:37:23.412 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/31 file after the snapshot 2023-10-30T12:37:23.412 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/32 file after the snapshot 2023-10-30T12:37:23.412 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/33 file after the snapshot 2023-10-30T12:37:23.413 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/34 file after the snapshot 2023-10-30T12:37:23.413 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/35 file after the snapshot 2023-10-30T12:37:23.413 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/36 file after the snapshot 2023-10-30T12:37:23.413 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/37 file after the snapshot 2023-10-30T12:37:23.413 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/38 file after the snapshot 2023-10-30T12:37:23.414 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/39 file after the snapshot 2023-10-30T12:37:23.414 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/.snap/snap-subdir-test 2023-10-30T12:37:23.433 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/30 file after the snapshot 2023-10-30T12:37:23.439 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/31 file after the snapshot 2023-10-30T12:37:23.440 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/32 file after the snapshot 2023-10-30T12:37:23.441 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/33 file after the snapshot 2023-10-30T12:37:23.441 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/34 file after the snapshot 2023-10-30T12:37:23.441 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/35 file after the snapshot 2023-10-30T12:37:23.441 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/36 file after the snapshot 2023-10-30T12:37:23.441 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/37 file after the snapshot 2023-10-30T12:37:23.442 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/38 file after the snapshot 2023-10-30T12:37:23.442 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/39 file after the snapshot 2023-10-30T12:37:23.442 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/.snap/snap-subdir-test 2023-10-30T12:37:23.467 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/30 file after the snapshot 2023-10-30T12:37:23.468 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/31 file after the snapshot 2023-10-30T12:37:23.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/32 file after the snapshot 2023-10-30T12:37:23.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/33 file after the snapshot 2023-10-30T12:37:23.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/34 file after the snapshot 2023-10-30T12:37:23.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/35 file after the snapshot 2023-10-30T12:37:23.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/36 file after the snapshot 2023-10-30T12:37:23.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/37 file after the snapshot 2023-10-30T12:37:23.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/38 file after the snapshot 2023-10-30T12:37:23.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/39 file after the snapshot 2023-10-30T12:37:23.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/.snap/snap-subdir-test 2023-10-30T12:37:23.486 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/30 file after the snapshot 2023-10-30T12:37:23.488 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/31 file after the snapshot 2023-10-30T12:37:23.488 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/32 file after the snapshot 2023-10-30T12:37:23.488 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/33 file after the snapshot 2023-10-30T12:37:23.489 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/34 file after the snapshot 2023-10-30T12:37:23.489 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/35 file after the snapshot 2023-10-30T12:37:23.489 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/36 file after the snapshot 2023-10-30T12:37:23.489 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/37 file after the snapshot 2023-10-30T12:37:23.489 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/38 file after the snapshot 2023-10-30T12:37:23.490 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/39 file after the snapshot 2023-10-30T12:37:23.490 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/.snap/snap-subdir-test 2023-10-30T12:37:23.513 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/30 file after the snapshot 2023-10-30T12:37:23.515 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/31 file after the snapshot 2023-10-30T12:37:23.516 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/32 file after the snapshot 2023-10-30T12:37:23.516 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/33 file after the snapshot 2023-10-30T12:37:23.516 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/34 file after the snapshot 2023-10-30T12:37:23.516 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/35 file after the snapshot 2023-10-30T12:37:23.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/36 file after the snapshot 2023-10-30T12:37:23.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/37 file after the snapshot 2023-10-30T12:37:23.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/38 file after the snapshot 2023-10-30T12:37:23.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/39 file after the snapshot 2023-10-30T12:37:23.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/.snap/snap-subdir-test 2023-10-30T12:37:23.539 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/30 file after the snapshot 2023-10-30T12:37:23.541 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/31 file after the snapshot 2023-10-30T12:37:23.541 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/32 file after the snapshot 2023-10-30T12:37:23.541 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/33 file after the snapshot 2023-10-30T12:37:23.541 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/34 file after the snapshot 2023-10-30T12:37:23.542 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/35 file after the snapshot 2023-10-30T12:37:23.542 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/36 file after the snapshot 2023-10-30T12:37:23.542 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/37 file after the snapshot 2023-10-30T12:37:23.542 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/38 file after the snapshot 2023-10-30T12:37:23.542 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/39 file after the snapshot 2023-10-30T12:37:23.543 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/.snap/snap-subdir-test 2023-10-30T12:37:23.569 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/30 file after the snapshot 2023-10-30T12:37:23.571 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/31 file after the snapshot 2023-10-30T12:37:23.572 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/32 file after the snapshot 2023-10-30T12:37:23.572 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/33 file after the snapshot 2023-10-30T12:37:23.572 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/34 file after the snapshot 2023-10-30T12:37:23.572 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/35 file after the snapshot 2023-10-30T12:37:23.572 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/36 file after the snapshot 2023-10-30T12:37:23.573 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/37 file after the snapshot 2023-10-30T12:37:23.573 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/38 file after the snapshot 2023-10-30T12:37:23.573 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/39 file after the snapshot 2023-10-30T12:37:23.573 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/.snap/snap-subdir-test 2023-10-30T12:37:23.603 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/30 file after the snapshot 2023-10-30T12:37:23.605 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/31 file after the snapshot 2023-10-30T12:37:23.605 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/32 file after the snapshot 2023-10-30T12:37:23.605 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/33 file after the snapshot 2023-10-30T12:37:23.605 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/34 file after the snapshot 2023-10-30T12:37:23.605 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/35 file after the snapshot 2023-10-30T12:37:23.606 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/36 file after the snapshot 2023-10-30T12:37:23.606 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/37 file after the snapshot 2023-10-30T12:37:23.606 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/38 file after the snapshot 2023-10-30T12:37:23.606 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/39 file after the snapshot 2023-10-30T12:37:23.607 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/.snap/snap-subdir-test 2023-10-30T12:37:23.639 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/30 file after the snapshot 2023-10-30T12:37:23.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/31 file after the snapshot 2023-10-30T12:37:23.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/32 file after the snapshot 2023-10-30T12:37:23.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/33 file after the snapshot 2023-10-30T12:37:23.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/34 file after the snapshot 2023-10-30T12:37:23.643 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/35 file after the snapshot 2023-10-30T12:37:23.643 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/36 file after the snapshot 2023-10-30T12:37:23.643 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/37 file after the snapshot 2023-10-30T12:37:23.643 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/38 file after the snapshot 2023-10-30T12:37:23.643 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/39 file after the snapshot 2023-10-30T12:37:23.644 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/.snap/snap-subdir-test 2023-10-30T12:37:23.666 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/30 file after the snapshot 2023-10-30T12:37:23.667 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/31 file after the snapshot 2023-10-30T12:37:23.668 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/32 file after the snapshot 2023-10-30T12:37:23.668 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/33 file after the snapshot 2023-10-30T12:37:23.668 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/34 file after the snapshot 2023-10-30T12:37:23.668 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/35 file after the snapshot 2023-10-30T12:37:23.668 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/36 file after the snapshot 2023-10-30T12:37:23.669 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/37 file after the snapshot 2023-10-30T12:37:23.669 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/38 file after the snapshot 2023-10-30T12:37:23.669 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/39 file after the snapshot 2023-10-30T12:37:23.669 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/.snap/snap-subdir-test 2023-10-30T12:37:23.687 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/30 file after the snapshot 2023-10-30T12:37:23.689 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/31 file after the snapshot 2023-10-30T12:37:23.689 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/32 file after the snapshot 2023-10-30T12:37:23.689 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/33 file after the snapshot 2023-10-30T12:37:23.689 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/34 file after the snapshot 2023-10-30T12:37:23.690 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/35 file after the snapshot 2023-10-30T12:37:23.690 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/36 file after the snapshot 2023-10-30T12:37:23.690 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/37 file after the snapshot 2023-10-30T12:37:23.690 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/38 file after the snapshot 2023-10-30T12:37:23.690 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/39 file after the snapshot 2023-10-30T12:37:23.691 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/.snap/snap-subdir-test 2023-10-30T12:37:23.715 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/30 file after the snapshot 2023-10-30T12:37:23.717 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/31 file after the snapshot 2023-10-30T12:37:23.717 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/32 file after the snapshot 2023-10-30T12:37:23.717 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/33 file after the snapshot 2023-10-30T12:37:23.717 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/34 file after the snapshot 2023-10-30T12:37:23.718 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/35 file after the snapshot 2023-10-30T12:37:23.718 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/36 file after the snapshot 2023-10-30T12:37:23.718 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/37 file after the snapshot 2023-10-30T12:37:23.718 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/38 file after the snapshot 2023-10-30T12:37:23.718 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/39 file after the snapshot 2023-10-30T12:37:23.719 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/.snap/snap-subdir-test 2023-10-30T12:37:24.153 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/30 file after the snapshot 2023-10-30T12:37:24.155 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/31 file after the snapshot 2023-10-30T12:37:24.155 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/32 file after the snapshot 2023-10-30T12:37:24.155 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/33 file after the snapshot 2023-10-30T12:37:24.156 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/34 file after the snapshot 2023-10-30T12:37:24.156 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/35 file after the snapshot 2023-10-30T12:37:24.156 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/36 file after the snapshot 2023-10-30T12:37:24.156 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/37 file after the snapshot 2023-10-30T12:37:24.156 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/38 file after the snapshot 2023-10-30T12:37:24.156 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/39 file after the snapshot 2023-10-30T12:37:24.157 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/.snap/snap-subdir-test 2023-10-30T12:37:24.174 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/30 file after the snapshot 2023-10-30T12:37:24.176 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/31 file after the snapshot 2023-10-30T12:37:24.176 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/32 file after the snapshot 2023-10-30T12:37:24.176 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/33 file after the snapshot 2023-10-30T12:37:24.176 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/34 file after the snapshot 2023-10-30T12:37:24.176 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/35 file after the snapshot 2023-10-30T12:37:24.177 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/36 file after the snapshot 2023-10-30T12:37:24.177 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/37 file after the snapshot 2023-10-30T12:37:24.177 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/38 file after the snapshot 2023-10-30T12:37:24.177 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/39 file after the snapshot 2023-10-30T12:37:24.177 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/.snap/snap-subdir-test 2023-10-30T12:37:24.535 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/30 file after the snapshot 2023-10-30T12:37:24.539 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/31 file after the snapshot 2023-10-30T12:37:24.539 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/32 file after the snapshot 2023-10-30T12:37:24.539 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/33 file after the snapshot 2023-10-30T12:37:24.540 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/34 file after the snapshot 2023-10-30T12:37:24.540 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/35 file after the snapshot 2023-10-30T12:37:24.540 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/36 file after the snapshot 2023-10-30T12:37:24.540 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/37 file after the snapshot 2023-10-30T12:37:24.540 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/38 file after the snapshot 2023-10-30T12:37:24.541 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/39 file after the snapshot 2023-10-30T12:37:24.541 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/.snap/snap-subdir-test 2023-10-30T12:37:24.564 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/30 file after the snapshot 2023-10-30T12:37:24.565 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/31 file after the snapshot 2023-10-30T12:37:24.566 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/32 file after the snapshot 2023-10-30T12:37:24.566 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/33 file after the snapshot 2023-10-30T12:37:24.566 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/34 file after the snapshot 2023-10-30T12:37:24.566 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/35 file after the snapshot 2023-10-30T12:37:24.566 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/36 file after the snapshot 2023-10-30T12:37:24.567 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/37 file after the snapshot 2023-10-30T12:37:24.567 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/38 file after the snapshot 2023-10-30T12:37:24.567 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/39 file after the snapshot 2023-10-30T12:37:24.567 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/.snap/snap-subdir-test 2023-10-30T12:37:25.300 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/30 file after the snapshot 2023-10-30T12:37:25.303 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/31 file after the snapshot 2023-10-30T12:37:25.303 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/32 file after the snapshot 2023-10-30T12:37:25.303 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/33 file after the snapshot 2023-10-30T12:37:25.303 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/34 file after the snapshot 2023-10-30T12:37:25.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/35 file after the snapshot 2023-10-30T12:37:25.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/36 file after the snapshot 2023-10-30T12:37:25.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/37 file after the snapshot 2023-10-30T12:37:25.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/38 file after the snapshot 2023-10-30T12:37:25.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/39 file after the snapshot 2023-10-30T12:37:25.305 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/.snap/snap-subdir-test 2023-10-30T12:37:25.359 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/30 file after the snapshot 2023-10-30T12:37:25.362 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/31 file after the snapshot 2023-10-30T12:37:25.363 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/32 file after the snapshot 2023-10-30T12:37:25.363 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/33 file after the snapshot 2023-10-30T12:37:25.363 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/34 file after the snapshot 2023-10-30T12:37:25.363 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/35 file after the snapshot 2023-10-30T12:37:25.363 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/36 file after the snapshot 2023-10-30T12:37:25.363 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/37 file after the snapshot 2023-10-30T12:37:25.364 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/38 file after the snapshot 2023-10-30T12:37:25.364 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/39 file after the snapshot 2023-10-30T12:37:25.364 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/.snap/snap-subdir-test 2023-10-30T12:37:25.421 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/30 file after the snapshot 2023-10-30T12:37:25.424 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/31 file after the snapshot 2023-10-30T12:37:25.424 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/32 file after the snapshot 2023-10-30T12:37:25.424 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/33 file after the snapshot 2023-10-30T12:37:25.424 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/34 file after the snapshot 2023-10-30T12:37:25.425 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/35 file after the snapshot 2023-10-30T12:37:25.425 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/36 file after the snapshot 2023-10-30T12:37:25.425 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/37 file after the snapshot 2023-10-30T12:37:25.425 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/38 file after the snapshot 2023-10-30T12:37:25.425 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/39 file after the snapshot 2023-10-30T12:37:25.426 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/.snap/snap-subdir-test 2023-10-30T12:37:25.481 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/30 file after the snapshot 2023-10-30T12:37:25.484 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/31 file after the snapshot 2023-10-30T12:37:25.485 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/32 file after the snapshot 2023-10-30T12:37:25.485 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/33 file after the snapshot 2023-10-30T12:37:25.485 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/34 file after the snapshot 2023-10-30T12:37:25.485 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/35 file after the snapshot 2023-10-30T12:37:25.486 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/36 file after the snapshot 2023-10-30T12:37:25.486 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/37 file after the snapshot 2023-10-30T12:37:25.486 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/38 file after the snapshot 2023-10-30T12:37:25.486 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/39 file after the snapshot 2023-10-30T12:37:25.486 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/.snap/snap-subdir-test 2023-10-30T12:37:25.510 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/30 file after the snapshot 2023-10-30T12:37:25.513 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/31 file after the snapshot 2023-10-30T12:37:25.513 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/32 file after the snapshot 2023-10-30T12:37:25.513 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/33 file after the snapshot 2023-10-30T12:37:25.513 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/34 file after the snapshot 2023-10-30T12:37:25.513 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/35 file after the snapshot 2023-10-30T12:37:25.514 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/36 file after the snapshot 2023-10-30T12:37:25.514 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/37 file after the snapshot 2023-10-30T12:37:25.514 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/38 file after the snapshot 2023-10-30T12:37:25.514 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/39 file after the snapshot 2023-10-30T12:37:25.514 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/.snap/snap-subdir-test 2023-10-30T12:37:25.545 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/30 file after the snapshot 2023-10-30T12:37:25.550 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/31 file after the snapshot 2023-10-30T12:37:25.550 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/32 file after the snapshot 2023-10-30T12:37:25.550 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/33 file after the snapshot 2023-10-30T12:37:25.550 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/34 file after the snapshot 2023-10-30T12:37:25.550 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/35 file after the snapshot 2023-10-30T12:37:25.551 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/36 file after the snapshot 2023-10-30T12:37:25.551 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/37 file after the snapshot 2023-10-30T12:37:25.551 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/38 file after the snapshot 2023-10-30T12:37:25.551 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/39 file after the snapshot 2023-10-30T12:37:25.551 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/.snap/snap-subdir-test 2023-10-30T12:37:25.612 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/30 file after the snapshot 2023-10-30T12:37:25.617 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/31 file after the snapshot 2023-10-30T12:37:25.617 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/32 file after the snapshot 2023-10-30T12:37:25.617 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/33 file after the snapshot 2023-10-30T12:37:25.618 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/34 file after the snapshot 2023-10-30T12:37:25.618 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/35 file after the snapshot 2023-10-30T12:37:25.618 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/36 file after the snapshot 2023-10-30T12:37:25.618 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/37 file after the snapshot 2023-10-30T12:37:25.618 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/38 file after the snapshot 2023-10-30T12:37:25.619 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/39 file after the snapshot 2023-10-30T12:37:25.619 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/.snap/snap-subdir-test 2023-10-30T12:37:25.676 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/30 file after the snapshot 2023-10-30T12:37:25.679 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/31 file after the snapshot 2023-10-30T12:37:25.679 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/32 file after the snapshot 2023-10-30T12:37:25.679 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/33 file after the snapshot 2023-10-30T12:37:25.679 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/34 file after the snapshot 2023-10-30T12:37:25.680 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/35 file after the snapshot 2023-10-30T12:37:25.680 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/36 file after the snapshot 2023-10-30T12:37:25.680 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/37 file after the snapshot 2023-10-30T12:37:25.680 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/38 file after the snapshot 2023-10-30T12:37:25.680 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/39 file after the snapshot 2023-10-30T12:37:25.681 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/.snap/snap-subdir-test 2023-10-30T12:37:25.703 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/30 file after the snapshot 2023-10-30T12:37:25.705 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/31 file after the snapshot 2023-10-30T12:37:25.705 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/32 file after the snapshot 2023-10-30T12:37:25.705 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/33 file after the snapshot 2023-10-30T12:37:25.705 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/34 file after the snapshot 2023-10-30T12:37:25.706 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/35 file after the snapshot 2023-10-30T12:37:25.706 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/36 file after the snapshot 2023-10-30T12:37:25.706 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/37 file after the snapshot 2023-10-30T12:37:25.706 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/38 file after the snapshot 2023-10-30T12:37:25.707 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/39 file after the snapshot 2023-10-30T12:37:25.707 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/.snap/snap-subdir-test 2023-10-30T12:37:25.975 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/30 file after the snapshot 2023-10-30T12:37:25.976 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/31 file after the snapshot 2023-10-30T12:37:25.977 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/32 file after the snapshot 2023-10-30T12:37:25.977 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/33 file after the snapshot 2023-10-30T12:37:25.977 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/34 file after the snapshot 2023-10-30T12:37:25.977 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/35 file after the snapshot 2023-10-30T12:37:25.977 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/36 file after the snapshot 2023-10-30T12:37:25.978 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/37 file after the snapshot 2023-10-30T12:37:25.978 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/38 file after the snapshot 2023-10-30T12:37:25.978 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/39 file after the snapshot 2023-10-30T12:37:25.978 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/.snap/snap-subdir-test 2023-10-30T12:37:26.002 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/30 file after the snapshot 2023-10-30T12:37:26.004 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/31 file after the snapshot 2023-10-30T12:37:26.004 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/32 file after the snapshot 2023-10-30T12:37:26.004 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/33 file after the snapshot 2023-10-30T12:37:26.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/34 file after the snapshot 2023-10-30T12:37:26.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/35 file after the snapshot 2023-10-30T12:37:26.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/36 file after the snapshot 2023-10-30T12:37:26.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/37 file after the snapshot 2023-10-30T12:37:26.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/38 file after the snapshot 2023-10-30T12:37:26.006 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/39 file after the snapshot 2023-10-30T12:37:26.006 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/.snap/snap-subdir-test 2023-10-30T12:37:26.036 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/30 file after the snapshot 2023-10-30T12:37:26.039 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/31 file after the snapshot 2023-10-30T12:37:26.039 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/32 file after the snapshot 2023-10-30T12:37:26.039 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/33 file after the snapshot 2023-10-30T12:37:26.039 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/34 file after the snapshot 2023-10-30T12:37:26.039 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/35 file after the snapshot 2023-10-30T12:37:26.040 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/36 file after the snapshot 2023-10-30T12:37:26.040 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/37 file after the snapshot 2023-10-30T12:37:26.040 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/38 file after the snapshot 2023-10-30T12:37:26.040 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/39 file after the snapshot 2023-10-30T12:37:26.040 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/.snap/snap-subdir-test 2023-10-30T12:37:26.061 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/30 file after the snapshot 2023-10-30T12:37:26.063 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/31 file after the snapshot 2023-10-30T12:37:26.063 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/32 file after the snapshot 2023-10-30T12:37:26.063 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/33 file after the snapshot 2023-10-30T12:37:26.064 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/34 file after the snapshot 2023-10-30T12:37:26.064 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/35 file after the snapshot 2023-10-30T12:37:26.064 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/36 file after the snapshot 2023-10-30T12:37:26.064 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/37 file after the snapshot 2023-10-30T12:37:26.064 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/38 file after the snapshot 2023-10-30T12:37:26.064 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/39 file after the snapshot 2023-10-30T12:37:26.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/.snap/snap-subdir-test 2023-10-30T12:37:26.116 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/30 file after the snapshot 2023-10-30T12:37:26.117 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/31 file after the snapshot 2023-10-30T12:37:26.118 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/32 file after the snapshot 2023-10-30T12:37:26.118 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/33 file after the snapshot 2023-10-30T12:37:26.118 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/34 file after the snapshot 2023-10-30T12:37:26.118 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/35 file after the snapshot 2023-10-30T12:37:26.119 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/36 file after the snapshot 2023-10-30T12:37:26.119 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/37 file after the snapshot 2023-10-30T12:37:26.119 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/38 file after the snapshot 2023-10-30T12:37:26.119 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/39 file after the snapshot 2023-10-30T12:37:26.119 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/.snap/snap-subdir-test 2023-10-30T12:37:26.151 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/30 file after the snapshot 2023-10-30T12:37:26.154 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/31 file after the snapshot 2023-10-30T12:37:26.154 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/32 file after the snapshot 2023-10-30T12:37:26.155 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/33 file after the snapshot 2023-10-30T12:37:26.155 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/34 file after the snapshot 2023-10-30T12:37:26.155 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/35 file after the snapshot 2023-10-30T12:37:26.155 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/36 file after the snapshot 2023-10-30T12:37:26.156 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/37 file after the snapshot 2023-10-30T12:37:26.156 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/38 file after the snapshot 2023-10-30T12:37:26.156 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/39 file after the snapshot 2023-10-30T12:37:26.156 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/.snap/snap-subdir-test 2023-10-30T12:37:26.178 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/30 file after the snapshot 2023-10-30T12:37:26.180 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/31 file after the snapshot 2023-10-30T12:37:26.180 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/32 file after the snapshot 2023-10-30T12:37:26.180 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/33 file after the snapshot 2023-10-30T12:37:26.180 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/34 file after the snapshot 2023-10-30T12:37:26.180 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/35 file after the snapshot 2023-10-30T12:37:26.181 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/36 file after the snapshot 2023-10-30T12:37:26.181 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/37 file after the snapshot 2023-10-30T12:37:26.181 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/38 file after the snapshot 2023-10-30T12:37:26.181 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/39 file after the snapshot 2023-10-30T12:37:26.181 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/.snap/snap-subdir-test 2023-10-30T12:37:26.206 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/30 file after the snapshot 2023-10-30T12:37:26.208 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/31 file after the snapshot 2023-10-30T12:37:26.208 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/32 file after the snapshot 2023-10-30T12:37:26.208 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/33 file after the snapshot 2023-10-30T12:37:26.209 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/34 file after the snapshot 2023-10-30T12:37:26.209 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/35 file after the snapshot 2023-10-30T12:37:26.209 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/36 file after the snapshot 2023-10-30T12:37:26.209 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/37 file after the snapshot 2023-10-30T12:37:26.209 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/38 file after the snapshot 2023-10-30T12:37:26.209 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/39 file after the snapshot 2023-10-30T12:37:26.210 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/.snap/snap-subdir-test 2023-10-30T12:37:26.235 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/30 file after the snapshot 2023-10-30T12:37:26.236 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/31 file after the snapshot 2023-10-30T12:37:26.237 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/32 file after the snapshot 2023-10-30T12:37:26.237 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/33 file after the snapshot 2023-10-30T12:37:26.237 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/34 file after the snapshot 2023-10-30T12:37:26.237 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/35 file after the snapshot 2023-10-30T12:37:26.237 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/36 file after the snapshot 2023-10-30T12:37:26.238 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/37 file after the snapshot 2023-10-30T12:37:26.238 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/38 file after the snapshot 2023-10-30T12:37:26.238 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/39 file after the snapshot 2023-10-30T12:37:26.238 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/.snap/snap-subdir-test 2023-10-30T12:37:26.259 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/30 file after the snapshot 2023-10-30T12:37:26.261 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/31 file after the snapshot 2023-10-30T12:37:26.261 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/32 file after the snapshot 2023-10-30T12:37:26.261 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/33 file after the snapshot 2023-10-30T12:37:26.262 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/34 file after the snapshot 2023-10-30T12:37:26.262 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/35 file after the snapshot 2023-10-30T12:37:26.262 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/36 file after the snapshot 2023-10-30T12:37:26.262 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/37 file after the snapshot 2023-10-30T12:37:26.262 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/38 file after the snapshot 2023-10-30T12:37:26.263 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/39 file after the snapshot 2023-10-30T12:37:26.263 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/.snap/snap-subdir-test 2023-10-30T12:37:26.290 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/30 file after the snapshot 2023-10-30T12:37:26.293 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/31 file after the snapshot 2023-10-30T12:37:26.293 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/32 file after the snapshot 2023-10-30T12:37:26.293 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/33 file after the snapshot 2023-10-30T12:37:26.293 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/34 file after the snapshot 2023-10-30T12:37:26.294 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/35 file after the snapshot 2023-10-30T12:37:26.294 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/36 file after the snapshot 2023-10-30T12:37:26.294 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/37 file after the snapshot 2023-10-30T12:37:26.294 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/38 file after the snapshot 2023-10-30T12:37:26.294 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/39 file after the snapshot 2023-10-30T12:37:26.295 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/.snap/snap-subdir-test 2023-10-30T12:37:26.516 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/30 file after the snapshot 2023-10-30T12:37:26.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/31 file after the snapshot 2023-10-30T12:37:26.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/32 file after the snapshot 2023-10-30T12:37:26.518 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/33 file after the snapshot 2023-10-30T12:37:26.518 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/34 file after the snapshot 2023-10-30T12:37:26.518 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/35 file after the snapshot 2023-10-30T12:37:26.518 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/36 file after the snapshot 2023-10-30T12:37:26.518 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/37 file after the snapshot 2023-10-30T12:37:26.519 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/38 file after the snapshot 2023-10-30T12:37:26.519 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/39 file after the snapshot 2023-10-30T12:37:26.519 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/.snap/snap-subdir-test 2023-10-30T12:37:26.614 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/30 file after the snapshot 2023-10-30T12:37:26.616 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/31 file after the snapshot 2023-10-30T12:37:26.616 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/32 file after the snapshot 2023-10-30T12:37:26.616 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/33 file after the snapshot 2023-10-30T12:37:26.616 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/34 file after the snapshot 2023-10-30T12:37:26.616 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/35 file after the snapshot 2023-10-30T12:37:26.617 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/36 file after the snapshot 2023-10-30T12:37:26.617 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/37 file after the snapshot 2023-10-30T12:37:26.617 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/38 file after the snapshot 2023-10-30T12:37:26.617 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/39 file after the snapshot 2023-10-30T12:37:26.617 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/.snap/snap-subdir-test 2023-10-30T12:37:26.641 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/30 file after the snapshot 2023-10-30T12:37:26.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/31 file after the snapshot 2023-10-30T12:37:26.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/32 file after the snapshot 2023-10-30T12:37:26.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/33 file after the snapshot 2023-10-30T12:37:26.643 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/34 file after the snapshot 2023-10-30T12:37:26.643 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/35 file after the snapshot 2023-10-30T12:37:26.643 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/36 file after the snapshot 2023-10-30T12:37:26.643 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/37 file after the snapshot 2023-10-30T12:37:26.643 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/38 file after the snapshot 2023-10-30T12:37:26.644 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/39 file after the snapshot 2023-10-30T12:37:26.644 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/.snap/snap-subdir-test 2023-10-30T12:37:26.669 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/30 file after the snapshot 2023-10-30T12:37:26.671 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/31 file after the snapshot 2023-10-30T12:37:26.671 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/32 file after the snapshot 2023-10-30T12:37:26.671 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/33 file after the snapshot 2023-10-30T12:37:26.672 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/34 file after the snapshot 2023-10-30T12:37:26.672 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/35 file after the snapshot 2023-10-30T12:37:26.672 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/36 file after the snapshot 2023-10-30T12:37:26.672 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/37 file after the snapshot 2023-10-30T12:37:26.672 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/38 file after the snapshot 2023-10-30T12:37:26.672 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/39 file after the snapshot 2023-10-30T12:37:26.673 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/.snap/snap-subdir-test 2023-10-30T12:37:27.209 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/30 file after the snapshot 2023-10-30T12:37:27.211 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/31 file after the snapshot 2023-10-30T12:37:27.211 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/32 file after the snapshot 2023-10-30T12:37:27.211 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/33 file after the snapshot 2023-10-30T12:37:27.212 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/34 file after the snapshot 2023-10-30T12:37:27.212 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/35 file after the snapshot 2023-10-30T12:37:27.212 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/36 file after the snapshot 2023-10-30T12:37:27.212 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/37 file after the snapshot 2023-10-30T12:37:27.212 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/38 file after the snapshot 2023-10-30T12:37:27.213 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/39 file after the snapshot 2023-10-30T12:37:27.213 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/.snap/snap-subdir-test 2023-10-30T12:37:27.440 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/30 file after the snapshot 2023-10-30T12:37:27.442 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/31 file after the snapshot 2023-10-30T12:37:27.442 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/32 file after the snapshot 2023-10-30T12:37:27.443 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/33 file after the snapshot 2023-10-30T12:37:27.443 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/34 file after the snapshot 2023-10-30T12:37:27.443 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/35 file after the snapshot 2023-10-30T12:37:27.443 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/36 file after the snapshot 2023-10-30T12:37:27.443 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/37 file after the snapshot 2023-10-30T12:37:27.444 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/38 file after the snapshot 2023-10-30T12:37:27.444 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/39 file after the snapshot 2023-10-30T12:37:27.444 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/.snap/snap-subdir-test 2023-10-30T12:37:27.473 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/30 file after the snapshot 2023-10-30T12:37:27.475 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/31 file after the snapshot 2023-10-30T12:37:27.475 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/32 file after the snapshot 2023-10-30T12:37:27.475 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/33 file after the snapshot 2023-10-30T12:37:27.475 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/34 file after the snapshot 2023-10-30T12:37:27.475 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/35 file after the snapshot 2023-10-30T12:37:27.476 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/36 file after the snapshot 2023-10-30T12:37:27.476 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/37 file after the snapshot 2023-10-30T12:37:27.476 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/38 file after the snapshot 2023-10-30T12:37:27.476 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/39 file after the snapshot 2023-10-30T12:37:27.476 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/.snap/snap-subdir-test 2023-10-30T12:37:27.505 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/30 file after the snapshot 2023-10-30T12:37:27.506 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/31 file after the snapshot 2023-10-30T12:37:27.507 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/32 file after the snapshot 2023-10-30T12:37:27.507 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/33 file after the snapshot 2023-10-30T12:37:27.507 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/34 file after the snapshot 2023-10-30T12:37:27.507 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/35 file after the snapshot 2023-10-30T12:37:27.507 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/36 file after the snapshot 2023-10-30T12:37:27.508 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/37 file after the snapshot 2023-10-30T12:37:27.508 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/38 file after the snapshot 2023-10-30T12:37:27.508 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/39 file after the snapshot 2023-10-30T12:37:27.508 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/.snap/snap-subdir-test 2023-10-30T12:37:27.531 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/30 file after the snapshot 2023-10-30T12:37:27.534 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/31 file after the snapshot 2023-10-30T12:37:27.534 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/32 file after the snapshot 2023-10-30T12:37:27.534 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/33 file after the snapshot 2023-10-30T12:37:27.534 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/34 file after the snapshot 2023-10-30T12:37:27.534 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/35 file after the snapshot 2023-10-30T12:37:27.535 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/36 file after the snapshot 2023-10-30T12:37:27.535 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/37 file after the snapshot 2023-10-30T12:37:27.535 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/38 file after the snapshot 2023-10-30T12:37:27.535 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/39 file after the snapshot 2023-10-30T12:37:27.535 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/.snap/snap-subdir-test 2023-10-30T12:37:27.558 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/30 file after the snapshot 2023-10-30T12:37:27.560 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/31 file after the snapshot 2023-10-30T12:37:27.560 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/32 file after the snapshot 2023-10-30T12:37:27.560 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/33 file after the snapshot 2023-10-30T12:37:27.561 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/34 file after the snapshot 2023-10-30T12:37:27.561 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/35 file after the snapshot 2023-10-30T12:37:27.561 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/36 file after the snapshot 2023-10-30T12:37:27.561 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/37 file after the snapshot 2023-10-30T12:37:27.561 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/38 file after the snapshot 2023-10-30T12:37:27.562 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/39 file after the snapshot 2023-10-30T12:37:27.562 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/.snap/snap-subdir-test 2023-10-30T12:37:27.588 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/30 file after the snapshot 2023-10-30T12:37:27.589 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/31 file after the snapshot 2023-10-30T12:37:27.589 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/32 file after the snapshot 2023-10-30T12:37:27.590 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/33 file after the snapshot 2023-10-30T12:37:27.590 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/34 file after the snapshot 2023-10-30T12:37:27.590 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/35 file after the snapshot 2023-10-30T12:37:27.590 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/36 file after the snapshot 2023-10-30T12:37:27.590 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/37 file after the snapshot 2023-10-30T12:37:27.591 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/38 file after the snapshot 2023-10-30T12:37:27.591 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/39 file after the snapshot 2023-10-30T12:37:27.591 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/.snap/snap-subdir-test 2023-10-30T12:37:27.615 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/30 file after the snapshot 2023-10-30T12:37:27.617 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/31 file after the snapshot 2023-10-30T12:37:27.617 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/32 file after the snapshot 2023-10-30T12:37:27.617 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/33 file after the snapshot 2023-10-30T12:37:27.618 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/34 file after the snapshot 2023-10-30T12:37:27.618 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/35 file after the snapshot 2023-10-30T12:37:27.618 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/36 file after the snapshot 2023-10-30T12:37:27.618 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/37 file after the snapshot 2023-10-30T12:37:27.618 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/38 file after the snapshot 2023-10-30T12:37:27.619 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/39 file after the snapshot 2023-10-30T12:37:27.619 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/.snap/snap-subdir-test 2023-10-30T12:37:27.644 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/30 file after the snapshot 2023-10-30T12:37:27.646 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/31 file after the snapshot 2023-10-30T12:37:27.646 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/32 file after the snapshot 2023-10-30T12:37:27.646 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/33 file after the snapshot 2023-10-30T12:37:27.646 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/34 file after the snapshot 2023-10-30T12:37:27.646 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/35 file after the snapshot 2023-10-30T12:37:27.647 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/36 file after the snapshot 2023-10-30T12:37:27.647 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/37 file after the snapshot 2023-10-30T12:37:27.647 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/38 file after the snapshot 2023-10-30T12:37:27.647 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/39 file after the snapshot 2023-10-30T12:37:27.647 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/.snap/snap-subdir-test 2023-10-30T12:37:28.328 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/30 file after the snapshot 2023-10-30T12:37:28.331 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/31 file after the snapshot 2023-10-30T12:37:28.331 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/32 file after the snapshot 2023-10-30T12:37:28.331 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/33 file after the snapshot 2023-10-30T12:37:28.331 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/34 file after the snapshot 2023-10-30T12:37:28.331 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/35 file after the snapshot 2023-10-30T12:37:28.332 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/36 file after the snapshot 2023-10-30T12:37:28.332 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/37 file after the snapshot 2023-10-30T12:37:28.332 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/38 file after the snapshot 2023-10-30T12:37:28.332 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/39 file after the snapshot 2023-10-30T12:37:28.332 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/.snap/snap-subdir-test 2023-10-30T12:37:28.355 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/30 file after the snapshot 2023-10-30T12:37:28.357 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/31 file after the snapshot 2023-10-30T12:37:28.357 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/32 file after the snapshot 2023-10-30T12:37:28.357 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/33 file after the snapshot 2023-10-30T12:37:28.357 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/34 file after the snapshot 2023-10-30T12:37:28.357 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/35 file after the snapshot 2023-10-30T12:37:28.358 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/36 file after the snapshot 2023-10-30T12:37:28.358 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/37 file after the snapshot 2023-10-30T12:37:28.358 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/38 file after the snapshot 2023-10-30T12:37:28.358 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/39 file after the snapshot 2023-10-30T12:37:28.358 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/.snap/snap-subdir-test 2023-10-30T12:37:28.386 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/30 file after the snapshot 2023-10-30T12:37:28.387 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/31 file after the snapshot 2023-10-30T12:37:28.387 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/32 file after the snapshot 2023-10-30T12:37:28.387 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/33 file after the snapshot 2023-10-30T12:37:28.388 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/34 file after the snapshot 2023-10-30T12:37:28.388 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/35 file after the snapshot 2023-10-30T12:37:28.388 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/36 file after the snapshot 2023-10-30T12:37:28.388 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/37 file after the snapshot 2023-10-30T12:37:28.388 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/38 file after the snapshot 2023-10-30T12:37:28.389 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/39 file after the snapshot 2023-10-30T12:37:28.389 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/.snap/snap-subdir-test 2023-10-30T12:37:28.407 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/30 file after the snapshot 2023-10-30T12:37:28.409 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/31 file after the snapshot 2023-10-30T12:37:28.409 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/32 file after the snapshot 2023-10-30T12:37:28.410 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/33 file after the snapshot 2023-10-30T12:37:28.410 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/34 file after the snapshot 2023-10-30T12:37:28.410 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/35 file after the snapshot 2023-10-30T12:37:28.410 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/36 file after the snapshot 2023-10-30T12:37:28.410 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/37 file after the snapshot 2023-10-30T12:37:28.411 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/38 file after the snapshot 2023-10-30T12:37:28.411 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/39 file after the snapshot 2023-10-30T12:37:28.411 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/.snap/snap-subdir-test 2023-10-30T12:37:28.430 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/30 file after the snapshot 2023-10-30T12:37:28.432 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/31 file after the snapshot 2023-10-30T12:37:28.432 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/32 file after the snapshot 2023-10-30T12:37:28.433 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/33 file after the snapshot 2023-10-30T12:37:28.433 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/34 file after the snapshot 2023-10-30T12:37:28.433 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/35 file after the snapshot 2023-10-30T12:37:28.433 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/36 file after the snapshot 2023-10-30T12:37:28.433 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/37 file after the snapshot 2023-10-30T12:37:28.433 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/38 file after the snapshot 2023-10-30T12:37:28.434 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/39 file after the snapshot 2023-10-30T12:37:28.434 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/.snap/snap-subdir-test 2023-10-30T12:37:29.224 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/30 file after the snapshot 2023-10-30T12:37:29.226 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/31 file after the snapshot 2023-10-30T12:37:29.226 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/32 file after the snapshot 2023-10-30T12:37:29.227 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/33 file after the snapshot 2023-10-30T12:37:29.227 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/34 file after the snapshot 2023-10-30T12:37:29.227 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/35 file after the snapshot 2023-10-30T12:37:29.227 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/36 file after the snapshot 2023-10-30T12:37:29.227 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/37 file after the snapshot 2023-10-30T12:37:29.227 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/38 file after the snapshot 2023-10-30T12:37:29.228 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/39 file after the snapshot 2023-10-30T12:37:29.228 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/.snap/snap-subdir-test 2023-10-30T12:37:29.258 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/30 file after the snapshot 2023-10-30T12:37:29.260 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/31 file after the snapshot 2023-10-30T12:37:29.260 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/32 file after the snapshot 2023-10-30T12:37:29.261 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/33 file after the snapshot 2023-10-30T12:37:29.261 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/34 file after the snapshot 2023-10-30T12:37:29.261 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/35 file after the snapshot 2023-10-30T12:37:29.261 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/36 file after the snapshot 2023-10-30T12:37:29.261 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/37 file after the snapshot 2023-10-30T12:37:29.262 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/38 file after the snapshot 2023-10-30T12:37:29.262 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/39 file after the snapshot 2023-10-30T12:37:29.262 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/.snap/snap-subdir-test 2023-10-30T12:37:29.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/30 file after the snapshot 2023-10-30T12:37:29.307 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/31 file after the snapshot 2023-10-30T12:37:29.307 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/32 file after the snapshot 2023-10-30T12:37:29.307 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/33 file after the snapshot 2023-10-30T12:37:29.307 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/34 file after the snapshot 2023-10-30T12:37:29.308 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/35 file after the snapshot 2023-10-30T12:37:29.308 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/36 file after the snapshot 2023-10-30T12:37:29.308 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/37 file after the snapshot 2023-10-30T12:37:29.308 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/38 file after the snapshot 2023-10-30T12:37:29.308 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/39 file after the snapshot 2023-10-30T12:37:29.309 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/.snap/snap-subdir-test 2023-10-30T12:37:29.331 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/30 file after the snapshot 2023-10-30T12:37:29.333 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/31 file after the snapshot 2023-10-30T12:37:29.333 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/32 file after the snapshot 2023-10-30T12:37:29.333 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/33 file after the snapshot 2023-10-30T12:37:29.333 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/34 file after the snapshot 2023-10-30T12:37:29.333 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/35 file after the snapshot 2023-10-30T12:37:29.334 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/36 file after the snapshot 2023-10-30T12:37:29.334 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/37 file after the snapshot 2023-10-30T12:37:29.334 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/38 file after the snapshot 2023-10-30T12:37:29.334 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/39 file after the snapshot 2023-10-30T12:37:29.334 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/.snap/snap-subdir-test 2023-10-30T12:37:29.816 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/30 file after the snapshot 2023-10-30T12:37:29.818 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/31 file after the snapshot 2023-10-30T12:37:29.818 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/32 file after the snapshot 2023-10-30T12:37:29.818 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/33 file after the snapshot 2023-10-30T12:37:29.819 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/34 file after the snapshot 2023-10-30T12:37:29.819 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/35 file after the snapshot 2023-10-30T12:37:29.819 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/36 file after the snapshot 2023-10-30T12:37:29.819 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/37 file after the snapshot 2023-10-30T12:37:29.819 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/38 file after the snapshot 2023-10-30T12:37:29.819 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/39 file after the snapshot 2023-10-30T12:37:29.820 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/.snap/snap-subdir-test 2023-10-30T12:37:29.861 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/30 file after the snapshot 2023-10-30T12:37:29.864 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/31 file after the snapshot 2023-10-30T12:37:29.864 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/32 file after the snapshot 2023-10-30T12:37:29.864 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/33 file after the snapshot 2023-10-30T12:37:29.864 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/34 file after the snapshot 2023-10-30T12:37:29.864 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/35 file after the snapshot 2023-10-30T12:37:29.865 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/36 file after the snapshot 2023-10-30T12:37:29.865 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/37 file after the snapshot 2023-10-30T12:37:29.865 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/38 file after the snapshot 2023-10-30T12:37:29.865 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/39 file after the snapshot 2023-10-30T12:37:29.865 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/.snap/snap-subdir-test 2023-10-30T12:37:29.892 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/30 file after the snapshot 2023-10-30T12:37:29.894 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/31 file after the snapshot 2023-10-30T12:37:29.894 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/32 file after the snapshot 2023-10-30T12:37:29.894 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/33 file after the snapshot 2023-10-30T12:37:29.895 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/34 file after the snapshot 2023-10-30T12:37:29.895 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/35 file after the snapshot 2023-10-30T12:37:29.895 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/36 file after the snapshot 2023-10-30T12:37:29.895 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/37 file after the snapshot 2023-10-30T12:37:29.895 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/38 file after the snapshot 2023-10-30T12:37:29.896 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/39 file after the snapshot 2023-10-30T12:37:29.896 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/.snap/snap-subdir-test 2023-10-30T12:37:29.924 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/30 file after the snapshot 2023-10-30T12:37:29.926 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/31 file after the snapshot 2023-10-30T12:37:29.926 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/32 file after the snapshot 2023-10-30T12:37:29.926 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/33 file after the snapshot 2023-10-30T12:37:29.927 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/34 file after the snapshot 2023-10-30T12:37:29.927 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/35 file after the snapshot 2023-10-30T12:37:29.927 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/36 file after the snapshot 2023-10-30T12:37:29.927 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/37 file after the snapshot 2023-10-30T12:37:29.927 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/38 file after the snapshot 2023-10-30T12:37:29.928 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/39 file after the snapshot 2023-10-30T12:37:29.928 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/.snap/snap-subdir-test 2023-10-30T12:37:29.948 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/30 file after the snapshot 2023-10-30T12:37:29.950 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/31 file after the snapshot 2023-10-30T12:37:29.950 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/32 file after the snapshot 2023-10-30T12:37:29.950 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/33 file after the snapshot 2023-10-30T12:37:29.950 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/34 file after the snapshot 2023-10-30T12:37:29.951 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/35 file after the snapshot 2023-10-30T12:37:29.951 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/36 file after the snapshot 2023-10-30T12:37:29.951 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/37 file after the snapshot 2023-10-30T12:37:29.951 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/38 file after the snapshot 2023-10-30T12:37:29.952 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/39 file after the snapshot 2023-10-30T12:37:29.952 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/.snap/snap-subdir-test 2023-10-30T12:37:29.978 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/30 file after the snapshot 2023-10-30T12:37:29.980 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/31 file after the snapshot 2023-10-30T12:37:29.980 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/32 file after the snapshot 2023-10-30T12:37:29.980 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/33 file after the snapshot 2023-10-30T12:37:29.981 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/34 file after the snapshot 2023-10-30T12:37:29.981 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/35 file after the snapshot 2023-10-30T12:37:29.981 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/36 file after the snapshot 2023-10-30T12:37:29.981 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/37 file after the snapshot 2023-10-30T12:37:29.981 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/38 file after the snapshot 2023-10-30T12:37:29.982 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/39 file after the snapshot 2023-10-30T12:37:29.982 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/.snap/snap-subdir-test 2023-10-30T12:37:30.071 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/30 file after the snapshot 2023-10-30T12:37:30.072 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/31 file after the snapshot 2023-10-30T12:37:30.072 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/32 file after the snapshot 2023-10-30T12:37:30.073 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/33 file after the snapshot 2023-10-30T12:37:30.073 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/34 file after the snapshot 2023-10-30T12:37:30.073 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/35 file after the snapshot 2023-10-30T12:37:30.073 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/36 file after the snapshot 2023-10-30T12:37:30.073 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/37 file after the snapshot 2023-10-30T12:37:30.074 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/38 file after the snapshot 2023-10-30T12:37:30.074 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/39 file after the snapshot 2023-10-30T12:37:30.074 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/.snap/snap-subdir-test 2023-10-30T12:37:30.096 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/30 file after the snapshot 2023-10-30T12:37:30.097 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/31 file after the snapshot 2023-10-30T12:37:30.097 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/32 file after the snapshot 2023-10-30T12:37:30.098 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/33 file after the snapshot 2023-10-30T12:37:30.098 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/34 file after the snapshot 2023-10-30T12:37:30.098 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/35 file after the snapshot 2023-10-30T12:37:30.098 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/36 file after the snapshot 2023-10-30T12:37:30.098 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/37 file after the snapshot 2023-10-30T12:37:30.099 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/38 file after the snapshot 2023-10-30T12:37:30.099 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/39 file after the snapshot 2023-10-30T12:37:30.099 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/.snap/snap-subdir-test 2023-10-30T12:37:30.130 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/30 file after the snapshot 2023-10-30T12:37:30.131 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/31 file after the snapshot 2023-10-30T12:37:30.132 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/32 file after the snapshot 2023-10-30T12:37:30.132 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/33 file after the snapshot 2023-10-30T12:37:30.132 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/34 file after the snapshot 2023-10-30T12:37:30.132 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/35 file after the snapshot 2023-10-30T12:37:30.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/36 file after the snapshot 2023-10-30T12:37:30.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/37 file after the snapshot 2023-10-30T12:37:30.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/38 file after the snapshot 2023-10-30T12:37:30.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/39 file after the snapshot 2023-10-30T12:37:30.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/.snap/snap-subdir-test 2023-10-30T12:37:30.162 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/30 file after the snapshot 2023-10-30T12:37:30.165 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/31 file after the snapshot 2023-10-30T12:37:30.165 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/32 file after the snapshot 2023-10-30T12:37:30.166 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/33 file after the snapshot 2023-10-30T12:37:30.166 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/34 file after the snapshot 2023-10-30T12:37:30.166 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/35 file after the snapshot 2023-10-30T12:37:30.166 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/36 file after the snapshot 2023-10-30T12:37:30.166 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/37 file after the snapshot 2023-10-30T12:37:30.167 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/38 file after the snapshot 2023-10-30T12:37:30.167 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/39 file after the snapshot 2023-10-30T12:37:30.167 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/.snap/snap-subdir-test 2023-10-30T12:37:30.189 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/30 file after the snapshot 2023-10-30T12:37:30.190 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/31 file after the snapshot 2023-10-30T12:37:30.190 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/32 file after the snapshot 2023-10-30T12:37:30.191 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/33 file after the snapshot 2023-10-30T12:37:30.191 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/34 file after the snapshot 2023-10-30T12:37:30.191 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/35 file after the snapshot 2023-10-30T12:37:30.191 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/36 file after the snapshot 2023-10-30T12:37:30.191 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/37 file after the snapshot 2023-10-30T12:37:30.192 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/38 file after the snapshot 2023-10-30T12:37:30.192 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/39 file after the snapshot 2023-10-30T12:37:30.192 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/.snap/snap-subdir-test 2023-10-30T12:37:30.217 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/30 file after the snapshot 2023-10-30T12:37:30.218 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/31 file after the snapshot 2023-10-30T12:37:30.218 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/32 file after the snapshot 2023-10-30T12:37:30.219 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/33 file after the snapshot 2023-10-30T12:37:30.219 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/34 file after the snapshot 2023-10-30T12:37:30.219 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/35 file after the snapshot 2023-10-30T12:37:30.219 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/36 file after the snapshot 2023-10-30T12:37:30.219 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/37 file after the snapshot 2023-10-30T12:37:30.219 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/38 file after the snapshot 2023-10-30T12:37:30.220 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/39 file after the snapshot 2023-10-30T12:37:30.220 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/.snap/snap-subdir-test 2023-10-30T12:37:30.242 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/30 file after the snapshot 2023-10-30T12:37:30.244 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/31 file after the snapshot 2023-10-30T12:37:30.244 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/32 file after the snapshot 2023-10-30T12:37:30.244 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/33 file after the snapshot 2023-10-30T12:37:30.244 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/34 file after the snapshot 2023-10-30T12:37:30.245 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/35 file after the snapshot 2023-10-30T12:37:30.245 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/36 file after the snapshot 2023-10-30T12:37:30.245 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/37 file after the snapshot 2023-10-30T12:37:30.245 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/38 file after the snapshot 2023-10-30T12:37:30.245 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/39 file after the snapshot 2023-10-30T12:37:30.246 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/.snap/snap-subdir-test 2023-10-30T12:37:31.390 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/30 file after the snapshot 2023-10-30T12:37:31.391 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/31 file after the snapshot 2023-10-30T12:37:31.392 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/32 file after the snapshot 2023-10-30T12:37:31.392 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/33 file after the snapshot 2023-10-30T12:37:31.392 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/34 file after the snapshot 2023-10-30T12:37:31.392 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/35 file after the snapshot 2023-10-30T12:37:31.392 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/36 file after the snapshot 2023-10-30T12:37:31.393 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/37 file after the snapshot 2023-10-30T12:37:31.393 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/38 file after the snapshot 2023-10-30T12:37:31.393 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/39 file after the snapshot 2023-10-30T12:37:31.393 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/.snap/snap-subdir-test 2023-10-30T12:37:32.319 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/30 file after the snapshot 2023-10-30T12:37:32.323 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/31 file after the snapshot 2023-10-30T12:37:32.323 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/32 file after the snapshot 2023-10-30T12:37:32.324 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/33 file after the snapshot 2023-10-30T12:37:32.324 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/34 file after the snapshot 2023-10-30T12:37:32.324 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/35 file after the snapshot 2023-10-30T12:37:32.325 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/36 file after the snapshot 2023-10-30T12:37:32.325 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/37 file after the snapshot 2023-10-30T12:37:32.325 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/38 file after the snapshot 2023-10-30T12:37:32.325 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/39 file after the snapshot 2023-10-30T12:37:32.326 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/.snap/snap-subdir-test 2023-10-30T12:37:32.358 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/30 file after the snapshot 2023-10-30T12:37:32.360 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/31 file after the snapshot 2023-10-30T12:37:32.360 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/32 file after the snapshot 2023-10-30T12:37:32.360 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/33 file after the snapshot 2023-10-30T12:37:32.360 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/34 file after the snapshot 2023-10-30T12:37:32.360 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/35 file after the snapshot 2023-10-30T12:37:32.361 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/36 file after the snapshot 2023-10-30T12:37:32.361 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/37 file after the snapshot 2023-10-30T12:37:32.361 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/38 file after the snapshot 2023-10-30T12:37:32.361 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/39 file after the snapshot 2023-10-30T12:37:32.361 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/.snap/snap-subdir-test 2023-10-30T12:37:32.380 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/30 file after the snapshot 2023-10-30T12:37:32.382 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/31 file after the snapshot 2023-10-30T12:37:32.382 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/32 file after the snapshot 2023-10-30T12:37:32.383 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/33 file after the snapshot 2023-10-30T12:37:32.383 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/34 file after the snapshot 2023-10-30T12:37:32.383 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/35 file after the snapshot 2023-10-30T12:37:32.383 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/36 file after the snapshot 2023-10-30T12:37:32.383 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/37 file after the snapshot 2023-10-30T12:37:32.384 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/38 file after the snapshot 2023-10-30T12:37:32.384 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/39 file after the snapshot 2023-10-30T12:37:32.384 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/.snap/snap-subdir-test 2023-10-30T12:37:32.408 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/30 file after the snapshot 2023-10-30T12:37:32.410 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/31 file after the snapshot 2023-10-30T12:37:32.410 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/32 file after the snapshot 2023-10-30T12:37:32.410 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/33 file after the snapshot 2023-10-30T12:37:32.410 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/34 file after the snapshot 2023-10-30T12:37:32.411 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/35 file after the snapshot 2023-10-30T12:37:32.411 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/36 file after the snapshot 2023-10-30T12:37:32.411 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/37 file after the snapshot 2023-10-30T12:37:32.411 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/38 file after the snapshot 2023-10-30T12:37:32.411 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/39 file after the snapshot 2023-10-30T12:37:32.412 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/.snap/snap-subdir-test 2023-10-30T12:37:32.434 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/30 file after the snapshot 2023-10-30T12:37:32.436 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/31 file after the snapshot 2023-10-30T12:37:32.436 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/32 file after the snapshot 2023-10-30T12:37:32.436 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/33 file after the snapshot 2023-10-30T12:37:32.436 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/34 file after the snapshot 2023-10-30T12:37:32.436 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/35 file after the snapshot 2023-10-30T12:37:32.437 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/36 file after the snapshot 2023-10-30T12:37:32.437 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/37 file after the snapshot 2023-10-30T12:37:32.437 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/38 file after the snapshot 2023-10-30T12:37:32.437 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/39 file after the snapshot 2023-10-30T12:37:32.437 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/.snap/snap-subdir-test 2023-10-30T12:37:32.467 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/30 file after the snapshot 2023-10-30T12:37:32.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/31 file after the snapshot 2023-10-30T12:37:32.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/32 file after the snapshot 2023-10-30T12:37:32.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/33 file after the snapshot 2023-10-30T12:37:32.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/34 file after the snapshot 2023-10-30T12:37:32.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/35 file after the snapshot 2023-10-30T12:37:32.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/36 file after the snapshot 2023-10-30T12:37:32.471 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/37 file after the snapshot 2023-10-30T12:37:32.471 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/38 file after the snapshot 2023-10-30T12:37:32.471 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/39 file after the snapshot 2023-10-30T12:37:32.471 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/.snap/snap-subdir-test 2023-10-30T12:37:32.503 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/30 file after the snapshot 2023-10-30T12:37:32.505 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/31 file after the snapshot 2023-10-30T12:37:32.506 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/32 file after the snapshot 2023-10-30T12:37:32.506 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/33 file after the snapshot 2023-10-30T12:37:32.506 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/34 file after the snapshot 2023-10-30T12:37:32.506 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/35 file after the snapshot 2023-10-30T12:37:32.506 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/36 file after the snapshot 2023-10-30T12:37:32.507 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/37 file after the snapshot 2023-10-30T12:37:32.507 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/38 file after the snapshot 2023-10-30T12:37:32.507 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/39 file after the snapshot 2023-10-30T12:37:32.507 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/.snap/snap-subdir-test 2023-10-30T12:37:32.786 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/30 file after the snapshot 2023-10-30T12:37:32.787 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/31 file after the snapshot 2023-10-30T12:37:32.787 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/32 file after the snapshot 2023-10-30T12:37:32.787 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/33 file after the snapshot 2023-10-30T12:37:32.788 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/34 file after the snapshot 2023-10-30T12:37:32.788 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/35 file after the snapshot 2023-10-30T12:37:32.788 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/36 file after the snapshot 2023-10-30T12:37:32.788 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/37 file after the snapshot 2023-10-30T12:37:32.788 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/38 file after the snapshot 2023-10-30T12:37:32.789 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/39 file after the snapshot 2023-10-30T12:37:32.789 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/.snap/snap-subdir-test 2023-10-30T12:37:32.813 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/30 file after the snapshot 2023-10-30T12:37:32.815 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/31 file after the snapshot 2023-10-30T12:37:32.815 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/32 file after the snapshot 2023-10-30T12:37:32.816 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/33 file after the snapshot 2023-10-30T12:37:32.816 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/34 file after the snapshot 2023-10-30T12:37:32.816 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/35 file after the snapshot 2023-10-30T12:37:32.816 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/36 file after the snapshot 2023-10-30T12:37:32.816 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/37 file after the snapshot 2023-10-30T12:37:32.817 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/38 file after the snapshot 2023-10-30T12:37:32.817 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/39 file after the snapshot 2023-10-30T12:37:32.817 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/.snap/snap-subdir-test 2023-10-30T12:37:32.845 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/30 file after the snapshot 2023-10-30T12:37:32.846 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/31 file after the snapshot 2023-10-30T12:37:32.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/32 file after the snapshot 2023-10-30T12:37:32.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/33 file after the snapshot 2023-10-30T12:37:32.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/34 file after the snapshot 2023-10-30T12:37:32.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/35 file after the snapshot 2023-10-30T12:37:32.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/36 file after the snapshot 2023-10-30T12:37:32.848 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/37 file after the snapshot 2023-10-30T12:37:32.848 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/38 file after the snapshot 2023-10-30T12:37:32.848 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/39 file after the snapshot 2023-10-30T12:37:32.848 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/.snap/snap-subdir-test 2023-10-30T12:37:32.877 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/30 file after the snapshot 2023-10-30T12:37:32.880 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/31 file after the snapshot 2023-10-30T12:37:32.880 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/32 file after the snapshot 2023-10-30T12:37:32.881 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/33 file after the snapshot 2023-10-30T12:37:32.881 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/34 file after the snapshot 2023-10-30T12:37:32.881 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/35 file after the snapshot 2023-10-30T12:37:32.881 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/36 file after the snapshot 2023-10-30T12:37:32.881 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/37 file after the snapshot 2023-10-30T12:37:32.882 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/38 file after the snapshot 2023-10-30T12:37:32.882 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/39 file after the snapshot 2023-10-30T12:37:32.882 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/.snap/snap-subdir-test 2023-10-30T12:37:32.903 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/30 file after the snapshot 2023-10-30T12:37:32.905 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/31 file after the snapshot 2023-10-30T12:37:32.905 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/32 file after the snapshot 2023-10-30T12:37:32.906 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/33 file after the snapshot 2023-10-30T12:37:32.906 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/34 file after the snapshot 2023-10-30T12:37:32.906 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/35 file after the snapshot 2023-10-30T12:37:32.906 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/36 file after the snapshot 2023-10-30T12:37:32.906 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/37 file after the snapshot 2023-10-30T12:37:32.907 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/38 file after the snapshot 2023-10-30T12:37:32.907 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/39 file after the snapshot 2023-10-30T12:37:32.907 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/.snap/snap-subdir-test 2023-10-30T12:37:32.928 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/30 file after the snapshot 2023-10-30T12:37:32.930 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/31 file after the snapshot 2023-10-30T12:37:32.930 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/32 file after the snapshot 2023-10-30T12:37:32.930 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/33 file after the snapshot 2023-10-30T12:37:32.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/34 file after the snapshot 2023-10-30T12:37:32.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/35 file after the snapshot 2023-10-30T12:37:32.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/36 file after the snapshot 2023-10-30T12:37:32.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/37 file after the snapshot 2023-10-30T12:37:32.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/38 file after the snapshot 2023-10-30T12:37:32.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/39 file after the snapshot 2023-10-30T12:37:32.932 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/.snap/snap-subdir-test 2023-10-30T12:37:32.950 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/30 file after the snapshot 2023-10-30T12:37:32.952 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/31 file after the snapshot 2023-10-30T12:37:32.952 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/32 file after the snapshot 2023-10-30T12:37:32.952 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/33 file after the snapshot 2023-10-30T12:37:32.953 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/34 file after the snapshot 2023-10-30T12:37:32.953 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/35 file after the snapshot 2023-10-30T12:37:32.953 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/36 file after the snapshot 2023-10-30T12:37:32.953 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/37 file after the snapshot 2023-10-30T12:37:32.953 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/38 file after the snapshot 2023-10-30T12:37:32.953 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/39 file after the snapshot 2023-10-30T12:37:32.954 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/.snap/snap-subdir-test 2023-10-30T12:37:32.987 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/30 file after the snapshot 2023-10-30T12:37:32.991 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/31 file after the snapshot 2023-10-30T12:37:32.991 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/32 file after the snapshot 2023-10-30T12:37:32.991 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/33 file after the snapshot 2023-10-30T12:37:32.992 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/34 file after the snapshot 2023-10-30T12:37:32.992 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/35 file after the snapshot 2023-10-30T12:37:32.992 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/36 file after the snapshot 2023-10-30T12:37:32.992 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/37 file after the snapshot 2023-10-30T12:37:32.992 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/38 file after the snapshot 2023-10-30T12:37:32.992 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/39 file after the snapshot 2023-10-30T12:37:32.993 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/.snap/snap-subdir-test 2023-10-30T12:37:33.043 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/30 file after the snapshot 2023-10-30T12:37:33.048 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/31 file after the snapshot 2023-10-30T12:37:33.048 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/32 file after the snapshot 2023-10-30T12:37:33.048 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/33 file after the snapshot 2023-10-30T12:37:33.048 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/34 file after the snapshot 2023-10-30T12:37:33.049 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/35 file after the snapshot 2023-10-30T12:37:33.049 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/36 file after the snapshot 2023-10-30T12:37:33.049 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/37 file after the snapshot 2023-10-30T12:37:33.049 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/38 file after the snapshot 2023-10-30T12:37:33.049 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/39 file after the snapshot 2023-10-30T12:37:33.050 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/.snap/snap-subdir-test 2023-10-30T12:37:33.074 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/30 file after the snapshot 2023-10-30T12:37:33.075 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/31 file after the snapshot 2023-10-30T12:37:33.076 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/32 file after the snapshot 2023-10-30T12:37:33.076 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/33 file after the snapshot 2023-10-30T12:37:33.076 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/34 file after the snapshot 2023-10-30T12:37:33.076 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/35 file after the snapshot 2023-10-30T12:37:33.076 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/36 file after the snapshot 2023-10-30T12:37:33.077 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/37 file after the snapshot 2023-10-30T12:37:33.077 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/38 file after the snapshot 2023-10-30T12:37:33.077 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/39 file after the snapshot 2023-10-30T12:37:33.077 INFO:tasks.workunit.client.0.smithi012.stdout:Take second snapshot .snap/test2 2023-10-30T12:37:33.398 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:37:33.402 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:37:33.760 INFO:tasks.workunit.client.0.smithi012.stdout:Copy content of .snap/test1 to copyofsnap1 ... 2023-10-30T12:37:35.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1' -> 'copyofsnap1/test1' 2023-10-30T12:37:35.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100' -> 'copyofsnap1/test1/100' 2023-10-30T12:37:35.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/11' -> 'copyofsnap1/test1/100/11' 2023-10-30T12:37:35.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/12' -> 'copyofsnap1/test1/100/12' 2023-10-30T12:37:35.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/13' -> 'copyofsnap1/test1/100/13' 2023-10-30T12:37:35.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/14' -> 'copyofsnap1/test1/100/14' 2023-10-30T12:37:35.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/15' -> 'copyofsnap1/test1/100/15' 2023-10-30T12:37:35.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/16' -> 'copyofsnap1/test1/100/16' 2023-10-30T12:37:35.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/17' -> 'copyofsnap1/test1/100/17' 2023-10-30T12:37:35.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/18' -> 'copyofsnap1/test1/100/18' 2023-10-30T12:37:35.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/19' -> 'copyofsnap1/test1/100/19' 2023-10-30T12:37:35.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/20' -> 'copyofsnap1/test1/100/20' 2023-10-30T12:37:35.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/10' -> 'copyofsnap1/test1/100/10' 2023-10-30T12:37:35.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101' -> 'copyofsnap1/test1/101' 2023-10-30T12:37:35.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/11' -> 'copyofsnap1/test1/101/11' 2023-10-30T12:37:35.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/12' -> 'copyofsnap1/test1/101/12' 2023-10-30T12:37:35.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/13' -> 'copyofsnap1/test1/101/13' 2023-10-30T12:37:35.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/14' -> 'copyofsnap1/test1/101/14' 2023-10-30T12:37:35.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/15' -> 'copyofsnap1/test1/101/15' 2023-10-30T12:37:35.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/16' -> 'copyofsnap1/test1/101/16' 2023-10-30T12:37:35.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/17' -> 'copyofsnap1/test1/101/17' 2023-10-30T12:37:35.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/18' -> 'copyofsnap1/test1/101/18' 2023-10-30T12:37:35.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/19' -> 'copyofsnap1/test1/101/19' 2023-10-30T12:37:35.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/20' -> 'copyofsnap1/test1/101/20' 2023-10-30T12:37:35.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/10' -> 'copyofsnap1/test1/101/10' 2023-10-30T12:37:35.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102' -> 'copyofsnap1/test1/102' 2023-10-30T12:37:35.368 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/11' -> 'copyofsnap1/test1/102/11' 2023-10-30T12:37:35.368 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/12' -> 'copyofsnap1/test1/102/12' 2023-10-30T12:37:35.368 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/13' -> 'copyofsnap1/test1/102/13' 2023-10-30T12:37:35.368 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/14' -> 'copyofsnap1/test1/102/14' 2023-10-30T12:37:35.368 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/15' -> 'copyofsnap1/test1/102/15' 2023-10-30T12:37:35.368 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/16' -> 'copyofsnap1/test1/102/16' 2023-10-30T12:37:35.369 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/17' -> 'copyofsnap1/test1/102/17' 2023-10-30T12:37:35.369 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/18' -> 'copyofsnap1/test1/102/18' 2023-10-30T12:37:35.369 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/19' -> 'copyofsnap1/test1/102/19' 2023-10-30T12:37:35.369 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/20' -> 'copyofsnap1/test1/102/20' 2023-10-30T12:37:35.369 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/10' -> 'copyofsnap1/test1/102/10' 2023-10-30T12:37:35.369 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103' -> 'copyofsnap1/test1/103' 2023-10-30T12:37:35.370 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/11' -> 'copyofsnap1/test1/103/11' 2023-10-30T12:37:35.370 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/12' -> 'copyofsnap1/test1/103/12' 2023-10-30T12:37:35.370 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/13' -> 'copyofsnap1/test1/103/13' 2023-10-30T12:37:35.370 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/14' -> 'copyofsnap1/test1/103/14' 2023-10-30T12:37:35.371 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/15' -> 'copyofsnap1/test1/103/15' 2023-10-30T12:37:35.371 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/16' -> 'copyofsnap1/test1/103/16' 2023-10-30T12:37:35.371 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/17' -> 'copyofsnap1/test1/103/17' 2023-10-30T12:37:35.371 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/18' -> 'copyofsnap1/test1/103/18' 2023-10-30T12:37:35.371 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/19' -> 'copyofsnap1/test1/103/19' 2023-10-30T12:37:35.372 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/20' -> 'copyofsnap1/test1/103/20' 2023-10-30T12:37:35.372 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/10' -> 'copyofsnap1/test1/103/10' 2023-10-30T12:37:35.372 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104' -> 'copyofsnap1/test1/104' 2023-10-30T12:37:35.372 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/11' -> 'copyofsnap1/test1/104/11' 2023-10-30T12:37:35.372 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/12' -> 'copyofsnap1/test1/104/12' 2023-10-30T12:37:35.373 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/13' -> 'copyofsnap1/test1/104/13' 2023-10-30T12:37:35.373 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/14' -> 'copyofsnap1/test1/104/14' 2023-10-30T12:37:35.373 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/15' -> 'copyofsnap1/test1/104/15' 2023-10-30T12:37:35.373 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/16' -> 'copyofsnap1/test1/104/16' 2023-10-30T12:37:35.373 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/17' -> 'copyofsnap1/test1/104/17' 2023-10-30T12:37:35.373 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/18' -> 'copyofsnap1/test1/104/18' 2023-10-30T12:37:35.374 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/19' -> 'copyofsnap1/test1/104/19' 2023-10-30T12:37:35.374 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/20' -> 'copyofsnap1/test1/104/20' 2023-10-30T12:37:35.374 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/10' -> 'copyofsnap1/test1/104/10' 2023-10-30T12:37:35.374 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105' -> 'copyofsnap1/test1/105' 2023-10-30T12:37:35.374 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/11' -> 'copyofsnap1/test1/105/11' 2023-10-30T12:37:35.375 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/12' -> 'copyofsnap1/test1/105/12' 2023-10-30T12:37:35.375 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/13' -> 'copyofsnap1/test1/105/13' 2023-10-30T12:37:35.375 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/14' -> 'copyofsnap1/test1/105/14' 2023-10-30T12:37:35.375 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/15' -> 'copyofsnap1/test1/105/15' 2023-10-30T12:37:35.375 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/16' -> 'copyofsnap1/test1/105/16' 2023-10-30T12:37:35.376 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/17' -> 'copyofsnap1/test1/105/17' 2023-10-30T12:37:35.376 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/18' -> 'copyofsnap1/test1/105/18' 2023-10-30T12:37:35.376 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/19' -> 'copyofsnap1/test1/105/19' 2023-10-30T12:37:35.376 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/20' -> 'copyofsnap1/test1/105/20' 2023-10-30T12:37:35.376 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/10' -> 'copyofsnap1/test1/105/10' 2023-10-30T12:37:35.377 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106' -> 'copyofsnap1/test1/106' 2023-10-30T12:37:35.377 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/11' -> 'copyofsnap1/test1/106/11' 2023-10-30T12:37:35.377 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/12' -> 'copyofsnap1/test1/106/12' 2023-10-30T12:37:35.377 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/13' -> 'copyofsnap1/test1/106/13' 2023-10-30T12:37:35.377 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/14' -> 'copyofsnap1/test1/106/14' 2023-10-30T12:37:35.377 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/15' -> 'copyofsnap1/test1/106/15' 2023-10-30T12:37:35.378 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/16' -> 'copyofsnap1/test1/106/16' 2023-10-30T12:37:35.378 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/17' -> 'copyofsnap1/test1/106/17' 2023-10-30T12:37:35.559 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/18' -> 'copyofsnap1/test1/106/18' 2023-10-30T12:37:35.560 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/19' -> 'copyofsnap1/test1/106/19' 2023-10-30T12:37:35.560 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/20' -> 'copyofsnap1/test1/106/20' 2023-10-30T12:37:35.560 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/10' -> 'copyofsnap1/test1/106/10' 2023-10-30T12:37:35.560 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107' -> 'copyofsnap1/test1/107' 2023-10-30T12:37:35.561 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/11' -> 'copyofsnap1/test1/107/11' 2023-10-30T12:37:35.561 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/12' -> 'copyofsnap1/test1/107/12' 2023-10-30T12:37:35.561 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/13' -> 'copyofsnap1/test1/107/13' 2023-10-30T12:37:35.561 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/14' -> 'copyofsnap1/test1/107/14' 2023-10-30T12:37:35.561 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/15' -> 'copyofsnap1/test1/107/15' 2023-10-30T12:37:35.561 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/16' -> 'copyofsnap1/test1/107/16' 2023-10-30T12:37:35.562 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/17' -> 'copyofsnap1/test1/107/17' 2023-10-30T12:37:35.562 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/18' -> 'copyofsnap1/test1/107/18' 2023-10-30T12:37:35.562 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/19' -> 'copyofsnap1/test1/107/19' 2023-10-30T12:37:35.562 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/20' -> 'copyofsnap1/test1/107/20' 2023-10-30T12:37:35.562 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/10' -> 'copyofsnap1/test1/107/10' 2023-10-30T12:37:35.563 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108' -> 'copyofsnap1/test1/108' 2023-10-30T12:37:35.563 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/11' -> 'copyofsnap1/test1/108/11' 2023-10-30T12:37:35.563 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/12' -> 'copyofsnap1/test1/108/12' 2023-10-30T12:37:35.563 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/13' -> 'copyofsnap1/test1/108/13' 2023-10-30T12:37:35.563 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/14' -> 'copyofsnap1/test1/108/14' 2023-10-30T12:37:35.563 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/15' -> 'copyofsnap1/test1/108/15' 2023-10-30T12:37:35.564 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/16' -> 'copyofsnap1/test1/108/16' 2023-10-30T12:37:35.564 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/17' -> 'copyofsnap1/test1/108/17' 2023-10-30T12:37:35.564 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/18' -> 'copyofsnap1/test1/108/18' 2023-10-30T12:37:35.564 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/19' -> 'copyofsnap1/test1/108/19' 2023-10-30T12:37:35.564 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/20' -> 'copyofsnap1/test1/108/20' 2023-10-30T12:37:35.565 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/10' -> 'copyofsnap1/test1/108/10' 2023-10-30T12:37:35.565 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109' -> 'copyofsnap1/test1/109' 2023-10-30T12:37:35.565 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/11' -> 'copyofsnap1/test1/109/11' 2023-10-30T12:37:35.565 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/12' -> 'copyofsnap1/test1/109/12' 2023-10-30T12:37:35.565 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/13' -> 'copyofsnap1/test1/109/13' 2023-10-30T12:37:35.565 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/14' -> 'copyofsnap1/test1/109/14' 2023-10-30T12:37:35.566 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/15' -> 'copyofsnap1/test1/109/15' 2023-10-30T12:37:35.566 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/16' -> 'copyofsnap1/test1/109/16' 2023-10-30T12:37:35.566 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/17' -> 'copyofsnap1/test1/109/17' 2023-10-30T12:37:35.566 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/18' -> 'copyofsnap1/test1/109/18' 2023-10-30T12:37:35.566 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/19' -> 'copyofsnap1/test1/109/19' 2023-10-30T12:37:35.567 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/20' -> 'copyofsnap1/test1/109/20' 2023-10-30T12:37:35.567 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/10' -> 'copyofsnap1/test1/109/10' 2023-10-30T12:37:35.567 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110' -> 'copyofsnap1/test1/110' 2023-10-30T12:37:35.567 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/11' -> 'copyofsnap1/test1/110/11' 2023-10-30T12:37:35.567 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/12' -> 'copyofsnap1/test1/110/12' 2023-10-30T12:37:35.568 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/13' -> 'copyofsnap1/test1/110/13' 2023-10-30T12:37:35.568 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/14' -> 'copyofsnap1/test1/110/14' 2023-10-30T12:37:35.568 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/15' -> 'copyofsnap1/test1/110/15' 2023-10-30T12:37:35.568 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/16' -> 'copyofsnap1/test1/110/16' 2023-10-30T12:37:35.568 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/17' -> 'copyofsnap1/test1/110/17' 2023-10-30T12:37:35.568 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/18' -> 'copyofsnap1/test1/110/18' 2023-10-30T12:37:35.569 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/19' -> 'copyofsnap1/test1/110/19' 2023-10-30T12:37:35.569 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/20' -> 'copyofsnap1/test1/110/20' 2023-10-30T12:37:35.569 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/10' -> 'copyofsnap1/test1/110/10' 2023-10-30T12:37:35.569 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111' -> 'copyofsnap1/test1/111' 2023-10-30T12:37:35.569 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/11' -> 'copyofsnap1/test1/111/11' 2023-10-30T12:37:35.570 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/12' -> 'copyofsnap1/test1/111/12' 2023-10-30T12:37:35.570 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/13' -> 'copyofsnap1/test1/111/13' 2023-10-30T12:37:35.570 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/14' -> 'copyofsnap1/test1/111/14' 2023-10-30T12:37:35.570 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/15' -> 'copyofsnap1/test1/111/15' 2023-10-30T12:37:35.570 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/16' -> 'copyofsnap1/test1/111/16' 2023-10-30T12:37:35.571 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/17' -> 'copyofsnap1/test1/111/17' 2023-10-30T12:37:35.571 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/18' -> 'copyofsnap1/test1/111/18' 2023-10-30T12:37:35.571 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/19' -> 'copyofsnap1/test1/111/19' 2023-10-30T12:37:35.571 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/20' -> 'copyofsnap1/test1/111/20' 2023-10-30T12:37:35.571 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/10' -> 'copyofsnap1/test1/111/10' 2023-10-30T12:37:35.572 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112' -> 'copyofsnap1/test1/112' 2023-10-30T12:37:35.572 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/11' -> 'copyofsnap1/test1/112/11' 2023-10-30T12:37:35.572 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/12' -> 'copyofsnap1/test1/112/12' 2023-10-30T12:37:35.572 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/13' -> 'copyofsnap1/test1/112/13' 2023-10-30T12:37:35.573 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/14' -> 'copyofsnap1/test1/112/14' 2023-10-30T12:37:35.573 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/15' -> 'copyofsnap1/test1/112/15' 2023-10-30T12:37:35.573 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/16' -> 'copyofsnap1/test1/112/16' 2023-10-30T12:37:35.573 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/17' -> 'copyofsnap1/test1/112/17' 2023-10-30T12:37:35.573 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/18' -> 'copyofsnap1/test1/112/18' 2023-10-30T12:37:35.574 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/19' -> 'copyofsnap1/test1/112/19' 2023-10-30T12:37:35.574 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/20' -> 'copyofsnap1/test1/112/20' 2023-10-30T12:37:35.574 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/10' -> 'copyofsnap1/test1/112/10' 2023-10-30T12:37:35.574 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113' -> 'copyofsnap1/test1/113' 2023-10-30T12:37:35.574 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/11' -> 'copyofsnap1/test1/113/11' 2023-10-30T12:37:35.575 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/12' -> 'copyofsnap1/test1/113/12' 2023-10-30T12:37:35.575 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/13' -> 'copyofsnap1/test1/113/13' 2023-10-30T12:37:35.575 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/14' -> 'copyofsnap1/test1/113/14' 2023-10-30T12:37:36.397 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/15' -> 'copyofsnap1/test1/113/15' 2023-10-30T12:37:36.398 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/16' -> 'copyofsnap1/test1/113/16' 2023-10-30T12:37:36.398 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/17' -> 'copyofsnap1/test1/113/17' 2023-10-30T12:37:36.398 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/18' -> 'copyofsnap1/test1/113/18' 2023-10-30T12:37:36.398 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/19' -> 'copyofsnap1/test1/113/19' 2023-10-30T12:37:36.398 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/20' -> 'copyofsnap1/test1/113/20' 2023-10-30T12:37:36.399 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/10' -> 'copyofsnap1/test1/113/10' 2023-10-30T12:37:36.399 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114' -> 'copyofsnap1/test1/114' 2023-10-30T12:37:36.399 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/11' -> 'copyofsnap1/test1/114/11' 2023-10-30T12:37:36.399 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/12' -> 'copyofsnap1/test1/114/12' 2023-10-30T12:37:36.399 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/13' -> 'copyofsnap1/test1/114/13' 2023-10-30T12:37:36.400 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/14' -> 'copyofsnap1/test1/114/14' 2023-10-30T12:37:36.400 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/15' -> 'copyofsnap1/test1/114/15' 2023-10-30T12:37:36.400 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/16' -> 'copyofsnap1/test1/114/16' 2023-10-30T12:37:36.400 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/17' -> 'copyofsnap1/test1/114/17' 2023-10-30T12:37:36.400 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/18' -> 'copyofsnap1/test1/114/18' 2023-10-30T12:37:36.401 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/19' -> 'copyofsnap1/test1/114/19' 2023-10-30T12:37:36.401 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/20' -> 'copyofsnap1/test1/114/20' 2023-10-30T12:37:36.401 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/10' -> 'copyofsnap1/test1/114/10' 2023-10-30T12:37:36.401 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115' -> 'copyofsnap1/test1/115' 2023-10-30T12:37:36.401 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/11' -> 'copyofsnap1/test1/115/11' 2023-10-30T12:37:36.402 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/12' -> 'copyofsnap1/test1/115/12' 2023-10-30T12:37:36.402 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/13' -> 'copyofsnap1/test1/115/13' 2023-10-30T12:37:36.402 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/14' -> 'copyofsnap1/test1/115/14' 2023-10-30T12:37:36.402 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/15' -> 'copyofsnap1/test1/115/15' 2023-10-30T12:37:36.402 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/16' -> 'copyofsnap1/test1/115/16' 2023-10-30T12:37:36.403 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/17' -> 'copyofsnap1/test1/115/17' 2023-10-30T12:37:36.403 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/18' -> 'copyofsnap1/test1/115/18' 2023-10-30T12:37:36.403 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/19' -> 'copyofsnap1/test1/115/19' 2023-10-30T12:37:36.403 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/20' -> 'copyofsnap1/test1/115/20' 2023-10-30T12:37:36.403 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/10' -> 'copyofsnap1/test1/115/10' 2023-10-30T12:37:36.403 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116' -> 'copyofsnap1/test1/116' 2023-10-30T12:37:36.404 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/11' -> 'copyofsnap1/test1/116/11' 2023-10-30T12:37:36.404 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/12' -> 'copyofsnap1/test1/116/12' 2023-10-30T12:37:36.404 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/13' -> 'copyofsnap1/test1/116/13' 2023-10-30T12:37:36.404 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/14' -> 'copyofsnap1/test1/116/14' 2023-10-30T12:37:36.404 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/15' -> 'copyofsnap1/test1/116/15' 2023-10-30T12:37:36.405 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/16' -> 'copyofsnap1/test1/116/16' 2023-10-30T12:37:36.405 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/17' -> 'copyofsnap1/test1/116/17' 2023-10-30T12:37:36.405 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/18' -> 'copyofsnap1/test1/116/18' 2023-10-30T12:37:36.405 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/19' -> 'copyofsnap1/test1/116/19' 2023-10-30T12:37:36.405 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/20' -> 'copyofsnap1/test1/116/20' 2023-10-30T12:37:36.406 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/10' -> 'copyofsnap1/test1/116/10' 2023-10-30T12:37:36.406 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117' -> 'copyofsnap1/test1/117' 2023-10-30T12:37:36.406 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/11' -> 'copyofsnap1/test1/117/11' 2023-10-30T12:37:36.406 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/12' -> 'copyofsnap1/test1/117/12' 2023-10-30T12:37:36.406 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/13' -> 'copyofsnap1/test1/117/13' 2023-10-30T12:37:36.407 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/14' -> 'copyofsnap1/test1/117/14' 2023-10-30T12:37:36.407 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/15' -> 'copyofsnap1/test1/117/15' 2023-10-30T12:37:36.407 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/16' -> 'copyofsnap1/test1/117/16' 2023-10-30T12:37:36.407 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/17' -> 'copyofsnap1/test1/117/17' 2023-10-30T12:37:36.407 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/18' -> 'copyofsnap1/test1/117/18' 2023-10-30T12:37:36.408 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/19' -> 'copyofsnap1/test1/117/19' 2023-10-30T12:37:36.408 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/20' -> 'copyofsnap1/test1/117/20' 2023-10-30T12:37:36.408 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/10' -> 'copyofsnap1/test1/117/10' 2023-10-30T12:37:36.408 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118' -> 'copyofsnap1/test1/118' 2023-10-30T12:37:36.408 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/11' -> 'copyofsnap1/test1/118/11' 2023-10-30T12:37:36.409 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/12' -> 'copyofsnap1/test1/118/12' 2023-10-30T12:37:36.409 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/13' -> 'copyofsnap1/test1/118/13' 2023-10-30T12:37:36.409 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/14' -> 'copyofsnap1/test1/118/14' 2023-10-30T12:37:36.409 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/15' -> 'copyofsnap1/test1/118/15' 2023-10-30T12:37:36.409 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/16' -> 'copyofsnap1/test1/118/16' 2023-10-30T12:37:36.409 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/17' -> 'copyofsnap1/test1/118/17' 2023-10-30T12:37:36.410 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/18' -> 'copyofsnap1/test1/118/18' 2023-10-30T12:37:36.410 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/19' -> 'copyofsnap1/test1/118/19' 2023-10-30T12:37:36.410 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/20' -> 'copyofsnap1/test1/118/20' 2023-10-30T12:37:36.410 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/10' -> 'copyofsnap1/test1/118/10' 2023-10-30T12:37:36.410 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119' -> 'copyofsnap1/test1/119' 2023-10-30T12:37:36.411 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/11' -> 'copyofsnap1/test1/119/11' 2023-10-30T12:37:36.411 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/12' -> 'copyofsnap1/test1/119/12' 2023-10-30T12:37:36.411 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/13' -> 'copyofsnap1/test1/119/13' 2023-10-30T12:37:36.411 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/14' -> 'copyofsnap1/test1/119/14' 2023-10-30T12:37:36.412 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/15' -> 'copyofsnap1/test1/119/15' 2023-10-30T12:37:36.412 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/16' -> 'copyofsnap1/test1/119/16' 2023-10-30T12:37:36.412 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/17' -> 'copyofsnap1/test1/119/17' 2023-10-30T12:37:36.412 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/18' -> 'copyofsnap1/test1/119/18' 2023-10-30T12:37:36.412 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/19' -> 'copyofsnap1/test1/119/19' 2023-10-30T12:37:36.412 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/20' -> 'copyofsnap1/test1/119/20' 2023-10-30T12:37:36.413 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/10' -> 'copyofsnap1/test1/119/10' 2023-10-30T12:37:36.413 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120' -> 'copyofsnap1/test1/120' 2023-10-30T12:37:36.413 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/11' -> 'copyofsnap1/test1/120/11' 2023-10-30T12:37:36.711 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:37:36.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.876+0000 7f951b1d3640 1 Processor -- start 2023-10-30T12:37:36.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.876+0000 7f951b1d3640 1 -- start start 2023-10-30T12:37:36.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.876+0000 7f951b1d3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f95141538a0 0x7f951415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:36.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.876+0000 7f951b1d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95140ca0e0 0x7f951415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:36.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.876+0000 7f951b1d3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95140cb800 con 0x7f95140cf160 2023-10-30T12:37:36.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.876+0000 7f951b1d3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951415dc30 con 0x7f95141538a0 2023-10-30T12:37:36.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.876+0000 7f951b1d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95140cb330 con 0x7f95140ca0e0 2023-10-30T12:37:36.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.876+0000 7f9519770640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f95140cf160 0x7f95140cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:57330/0 (socket says 172.21.15.12:57330) 2023-10-30T12:37:36.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.876+0000 7f9519770640 1 -- 172.21.15.12:0/1996156720 learned_addr learned my addr 172.21.15.12:0/1996156720 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:37:36.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.876+0000 7f9518f6f640 1 --2- 172.21.15.12:0/1996156720 >> v2:172.21.15.154:3300/0 conn(0x7f95141538a0 0x7f951415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:36.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.876+0000 7f9513fff640 1 --2- 172.21.15.12:0/1996156720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95140ca0e0 0x7f951415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:36.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.876+0000 7f95137fe640 1 -- 172.21.15.12:0/1996156720 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145908492 0 0) 0x7f95140cb800 con 0x7f95140cf160 2023-10-30T12:37:36.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.876+0000 7f95137fe640 1 -- 172.21.15.12:0/1996156720 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9504003530 con 0x7f95140cf160 2023-10-30T12:37:36.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f95137fe640 1 -- 172.21.15.12:0/1996156720 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4175125543 0 0) 0x7f9504003530 con 0x7f95140cf160 2023-10-30T12:37:36.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f95137fe640 1 -- 172.21.15.12:0/1996156720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95140ca0e0 msgr2=0x7f951415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:37:36.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f95137fe640 1 --2- 172.21.15.12:0/1996156720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95140ca0e0 0x7f951415a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:36.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f95137fe640 1 -- 172.21.15.12:0/1996156720 >> v2:172.21.15.154:3300/0 conn(0x7f95141538a0 msgr2=0x7f951415d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:37:36.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f95137fe640 1 --2- 172.21.15.12:0/1996156720 >> v2:172.21.15.154:3300/0 conn(0x7f95141538a0 0x7f951415d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:36.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f95137fe640 1 -- 172.21.15.12:0/1996156720 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95140cad70 con 0x7f95140cf160 2023-10-30T12:37:36.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f95137fe640 1 -- 172.21.15.12:0/1996156720 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f950c001a00 con 0x7f95140cf160 2023-10-30T12:37:36.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f9518f6f640 1 --2- 172.21.15.12:0/1996156720 >> v2:172.21.15.154:3300/0 conn(0x7f95141538a0 0x7f951415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:37:36.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f95137fe640 1 -- 172.21.15.12:0/1996156720 >> v1:172.21.15.12:6789/0 conn(0x7f95140cf160 legacy=0x7f95140cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:36.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f95137fe640 1 --2- 172.21.15.12:0/1996156720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9504003980 0x7f9504003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:36.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f95137fe640 1 --2- 172.21.15.12:0/1996156720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95040047d0 0x7f9504004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:36.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f95137fe640 1 --2- 172.21.15.12:0/1996156720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95140ca0e0 0x7f9504005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:36.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f95137fe640 1 -- 172.21.15.12:0/1996156720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951415dc30 con 0x7f9504003980 2023-10-30T12:37:36.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f95137fe640 1 -- 172.21.15.12:0/1996156720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95140cb330 con 0x7f95040047d0 2023-10-30T12:37:36.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f95137fe640 1 -- 172.21.15.12:0/1996156720 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f950400b9c0 con 0x7f95140ca0e0 2023-10-30T12:37:36.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f9518f6f640 1 --2- 172.21.15.12:0/1996156720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9504003980 0x7f9504003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:36.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f9519770640 1 --2- 172.21.15.12:0/1996156720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95140ca0e0 0x7f9504005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:36.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f9513fff640 1 --2- 172.21.15.12:0/1996156720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95040047d0 0x7f9504004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:36.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f9518f6f640 1 -- 172.21.15.12:0/1996156720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95140ca0e0 msgr2=0x7f9504005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:37:36.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f9518f6f640 1 --2- 172.21.15.12:0/1996156720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95140ca0e0 0x7f9504005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:36.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f9518f6f640 1 -- 172.21.15.12:0/1996156720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95040047d0 msgr2=0x7f9504004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:37:36.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f9518f6f640 1 --2- 172.21.15.12:0/1996156720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95040047d0 0x7f9504004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:36.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f9518f6f640 1 -- 172.21.15.12:0/1996156720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9504003530 con 0x7f9504003980 2023-10-30T12:37:36.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.880+0000 7f9518f6f640 1 --2- 172.21.15.12:0/1996156720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9504003980 0x7f9504003da0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9508009cb0 tx=0x7f950801cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9d50c639d20fa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:36.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f95137fe640 1 -- 172.21.15.12:0/1996156720 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95080a5740 con 0x7f9504003980 2023-10-30T12:37:36.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f95137fe640 1 -- 172.21.15.12:0/1996156720 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95080a58a0 con 0x7f9504003980 2023-10-30T12:37:36.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f95137fe640 1 -- 172.21.15.12:0/1996156720 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95080a5c10 con 0x7f9504003980 2023-10-30T12:37:36.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f951b1d3640 1 -- 172.21.15.12:0/1996156720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9504003980 msgr2=0x7f9504003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:36.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f951b1d3640 1 --2- 172.21.15.12:0/1996156720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9504003980 0x7f9504003da0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9508009cb0 tx=0x7f950801cc70 comp rx=0 tx=0).stop 2023-10-30T12:37:36.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f9519770640 1 -- 172.21.15.12:0/1996156720 reap_dead start 2023-10-30T12:37:36.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f951b1d3640 1 -- 172.21.15.12:0/1996156720 shutdown_connections 2023-10-30T12:37:36.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f951b1d3640 1 -- 172.21.15.12:0/1996156720 >> 172.21.15.12:0/1996156720 conn(0x7f951406bfe0 msgr2=0x7f951406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:36.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f951b1d3640 1 -- 172.21.15.12:0/1996156720 shutdown_connections 2023-10-30T12:37:36.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f951b1d3640 1 -- 172.21.15.12:0/1996156720 wait complete. 2023-10-30T12:37:36.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f951b1d3640 1 Processor -- start 2023-10-30T12:37:36.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f951b1d3640 1 -- start start 2023-10-30T12:37:36.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f951b1d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9504003980 0x7f95140ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:36.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f951b1d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95141f0000 0x7f9514157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:36.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f951b1d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95140cd550 0x7f95140b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:36.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f951b1d3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95140b4f00 con 0x7f95141f0000 2023-10-30T12:37:36.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f951b1d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95141efb70 con 0x7f9504003980 2023-10-30T12:37:36.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f951b1d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95140b3650 con 0x7f95140cd550 2023-10-30T12:37:36.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f9513fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95141f0000 0x7f9514157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:36.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f9513fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95141f0000 0x7f9514157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52572/0 (socket says 172.21.15.12:52572) 2023-10-30T12:37:36.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.884+0000 7f9513fff640 1 -- 172.21.15.12:0/2171809294 learned_addr learned my addr 172.21.15.12:0/2171809294 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:36.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f9518f6f640 1 --2- 172.21.15.12:0/2171809294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9504003980 0x7f95140ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:36.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f9519770640 1 --2- 172.21.15.12:0/2171809294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95140cd550 0x7f95140b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:36.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f9513fff640 1 -- 172.21.15.12:0/2171809294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95140cd550 msgr2=0x7f95140b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:36.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f9513fff640 1 --2- 172.21.15.12:0/2171809294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95140cd550 0x7f95140b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:36.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f9513fff640 1 -- 172.21.15.12:0/2171809294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9504003980 msgr2=0x7f95140ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:36.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f9513fff640 1 --2- 172.21.15.12:0/2171809294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9504003980 0x7f95140ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:36.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f9513fff640 1 -- 172.21.15.12:0/2171809294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95140b4980 con 0x7f95141f0000 2023-10-30T12:37:36.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f9513fff640 1 --2- 172.21.15.12:0/2171809294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95141f0000 0x7f9514157d90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9500011610 tx=0x7f9500011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:36.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f95117fa640 1 -- 172.21.15.12:0/2171809294 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500015870 con 0x7f95141f0000 2023-10-30T12:37:36.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f95117fa640 1 -- 172.21.15.12:0/2171809294 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95000159d0 con 0x7f95141f0000 2023-10-30T12:37:36.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f951b1d3640 1 -- 172.21.15.12:0/2171809294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95140b4c20 con 0x7f95141f0000 2023-10-30T12:37:36.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f95117fa640 1 -- 172.21.15.12:0/2171809294 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500015d40 con 0x7f95141f0000 2023-10-30T12:37:36.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f951b1d3640 1 -- 172.21.15.12:0/2171809294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95141ef800 con 0x7f95141f0000 2023-10-30T12:37:36.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f951b1d3640 1 -- 172.21.15.12:0/2171809294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95141ef380 con 0x7f95141f0000 2023-10-30T12:37:36.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f95117fa640 1 -- 172.21.15.12:0/2171809294 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f950000f480 con 0x7f95141f0000 2023-10-30T12:37:36.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f95117fa640 1 --2- 172.21.15.12:0/2171809294 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94f8066470 0x7f94f8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:36.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.888+0000 7f95117fa640 1 -- 172.21.15.12:0/2171809294 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9500084420 con 0x7f95141f0000 2023-10-30T12:37:36.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.892+0000 7f9518f6f640 1 --2- 172.21.15.12:0/2171809294 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94f8066470 0x7f94f8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:36.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.892+0000 7f9518f6f640 1 --2- 172.21.15.12:0/2171809294 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94f8066470 0x7f94f8068930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f95080af000 tx=0x7f95080046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:36.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:36.896+0000 7f95117fa640 1 -- 172.21.15.12:0/2171809294 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f95141ef380 con 0x7f95141f0000 2023-10-30T12:37:37.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.084+0000 7f951b1d3640 1 -- 172.21.15.12:0/2171809294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95141ef380 con 0x7f95141f0000 2023-10-30T12:37:37.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.084+0000 7f95117fa640 1 -- 172.21.15.12:0/2171809294 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6526 (secure 0 0 0) 0x7f95141ef380 con 0x7f95141f0000 2023-10-30T12:37:37.091 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:37:37.091 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:37:21.372155+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":9,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:37:37.091 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2023-10-30T12:37:37.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.084+0000 7f951b1d3640 1 -- 172.21.15.12:0/2171809294 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94f8066470 msgr2=0x7f94f8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:37.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.084+0000 7f951b1d3640 1 --2- 172.21.15.12:0/2171809294 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94f8066470 0x7f94f8068930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f95080af000 tx=0x7f95080046d0 comp rx=0 tx=0).stop 2023-10-30T12:37:37.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.084+0000 7f951b1d3640 1 -- 172.21.15.12:0/2171809294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95141f0000 msgr2=0x7f9514157d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:37.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.084+0000 7f951b1d3640 1 --2- 172.21.15.12:0/2171809294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95141f0000 0x7f9514157d90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9500011610 tx=0x7f9500011ad0 comp rx=0 tx=0).stop 2023-10-30T12:37:37.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.084+0000 7f951b1d3640 1 -- 172.21.15.12:0/2171809294 shutdown_connections 2023-10-30T12:37:37.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.084+0000 7f951b1d3640 1 --2- 172.21.15.12:0/2171809294 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94f8066470 0x7f94f8068930 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:37.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.084+0000 7f951b1d3640 1 --2- 172.21.15.12:0/2171809294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95140cd550 0x7f95140b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:37.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.084+0000 7f951b1d3640 1 --2- 172.21.15.12:0/2171809294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95141f0000 0x7f9514157d90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:37.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.084+0000 7f951b1d3640 1 --2- 172.21.15.12:0/2171809294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9504003980 0x7f95140ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:37.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.084+0000 7f951b1d3640 1 -- 172.21.15.12:0/2171809294 >> 172.21.15.12:0/2171809294 conn(0x7f951406bfe0 msgr2=0x7f95140c4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:37.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.084+0000 7f951b1d3640 1 -- 172.21.15.12:0/2171809294 shutdown_connections 2023-10-30T12:37:37.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.084+0000 7f951b1d3640 1 -- 172.21.15.12:0/2171809294 wait complete. 2023-10-30T12:37:37.111 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=2) 2023-10-30T12:37:37.112 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-30T12:37:37.112 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:37:37.128 INFO:tasks.ceph.mds.c:Stopped 2023-10-30T12:37:37.128 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:37:37.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.308+0000 7f0934846640 1 Processor -- start 2023-10-30T12:37:37.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.308+0000 7f0934846640 1 -- start start 2023-10-30T12:37:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f0934846640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09300c05d0 0x7f093013ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f0934846640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09300c6f60 0x7f093013f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f0934846640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09300c0a50 con 0x7f09300c6b80 2023-10-30T12:37:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f0934846640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093006f6e0 con 0x7f09300c05d0 2023-10-30T12:37:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f0934846640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093006f9c0 con 0x7f09300c6f60 2023-10-30T12:37:37.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f092e59c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f09300c6b80 0x7f09300bfec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:57344/0 (socket says 172.21.15.12:57344) 2023-10-30T12:37:37.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f092e59c640 1 -- 172.21.15.12:0/61318313 learned_addr learned my addr 172.21.15.12:0/61318313 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:37:37.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f092dd9b640 1 --2- 172.21.15.12:0/61318313 >> v2:172.21.15.154:3300/0 conn(0x7f09300c05d0 0x7f093013ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:37.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f092ed9d640 1 --2- 172.21.15.12:0/61318313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09300c6f60 0x7f093013f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:37.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f092d59a640 1 -- 172.21.15.12:0/61318313 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57698344 0 0) 0x7f09300c0a50 con 0x7f09300c6b80 2023-10-30T12:37:37.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f092d59a640 1 -- 172.21.15.12:0/61318313 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0918003530 con 0x7f09300c6b80 2023-10-30T12:37:37.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f092dd9b640 1 -- 172.21.15.12:0/61318313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09300c6f60 msgr2=0x7f093013f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:37:37.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f092dd9b640 1 --2- 172.21.15.12:0/61318313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09300c6f60 0x7f093013f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:37.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f092dd9b640 1 -- 172.21.15.12:0/61318313 >> v1:172.21.15.12:6789/0 conn(0x7f09300c6b80 legacy=0x7f09300bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:37.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f092dd9b640 1 -- 172.21.15.12:0/61318313 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093006fca0 con 0x7f09300c05d0 2023-10-30T12:37:37.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.312+0000 7f092ed9d640 1 --2- 172.21.15.12:0/61318313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09300c6f60 0x7f093013f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:37:37.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.316+0000 7f092dd9b640 1 --2- 172.21.15.12:0/61318313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09300c05d0 0x7f093013ed90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f09240097c0 tx=0x7f092401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a8cbaeeb056b2f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:37.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.316+0000 7f092d59a640 1 -- 172.21.15.12:0/61318313 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09240ad640 con 0x7f09300c05d0 2023-10-30T12:37:37.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.316+0000 7f092d59a640 1 -- 172.21.15.12:0/61318313 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09240ad7a0 con 0x7f09300c05d0 2023-10-30T12:37:37.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.316+0000 7f092d59a640 1 -- 172.21.15.12:0/61318313 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09240adb30 con 0x7f09300c05d0 2023-10-30T12:37:37.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.316+0000 7f0934846640 1 -- 172.21.15.12:0/61318313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09300c05d0 msgr2=0x7f093013ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:37.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.316+0000 7f0934846640 1 --2- 172.21.15.12:0/61318313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09300c05d0 0x7f093013ed90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f09240097c0 tx=0x7f092401c480 comp rx=0 tx=0).stop 2023-10-30T12:37:37.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.316+0000 7f0934846640 1 -- 172.21.15.12:0/61318313 shutdown_connections 2023-10-30T12:37:37.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.316+0000 7f0934846640 1 --2- 172.21.15.12:0/61318313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09300c6f60 0x7f093013f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:37.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.316+0000 7f0934846640 1 --2- 172.21.15.12:0/61318313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09300c05d0 0x7f093013ed90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:37.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.316+0000 7f0934846640 1 -- 172.21.15.12:0/61318313 >> 172.21.15.12:0/61318313 conn(0x7f09300c7910 msgr2=0x7f09300c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:37.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.316+0000 7f0934846640 1 -- 172.21.15.12:0/61318313 shutdown_connections 2023-10-30T12:37:37.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.316+0000 7f0934846640 1 -- 172.21.15.12:0/61318313 wait complete. 2023-10-30T12:37:37.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.316+0000 7f0934846640 1 Processor -- start 2023-10-30T12:37:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.316+0000 7f0934846640 1 -- start start 2023-10-30T12:37:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f0934846640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09300c05d0 0x7f09300c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:37.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f0934846640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09300c6b80 0x7f09300c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:37.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f0934846640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09300c6f60 0x7f093013ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:37.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f0934846640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093013df00 con 0x7f09300c6b80 2023-10-30T12:37:37.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f0934846640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09301384a0 con 0x7f09300c05d0 2023-10-30T12:37:37.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f0934846640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09301381c0 con 0x7f09300c6f60 2023-10-30T12:37:37.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f092dd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09300c6b80 0x7f09300c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:37.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f092dd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09300c6b80 0x7f09300c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52580/0 (socket says 172.21.15.12:52580) 2023-10-30T12:37:37.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f092dd9b640 1 -- 172.21.15.12:0/3341373994 learned_addr learned my addr 172.21.15.12:0/3341373994 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:37.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f092e59c640 1 --2- 172.21.15.12:0/3341373994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09300c05d0 0x7f09300c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:37.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f092ed9d640 1 --2- 172.21.15.12:0/3341373994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09300c6f60 0x7f093013ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:37.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f092dd9b640 1 -- 172.21.15.12:0/3341373994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09300c6f60 msgr2=0x7f093013ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:37.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f092dd9b640 1 --2- 172.21.15.12:0/3341373994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09300c6f60 0x7f093013ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:37.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f092dd9b640 1 -- 172.21.15.12:0/3341373994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09300c05d0 msgr2=0x7f09300c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:37.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f092dd9b640 1 --2- 172.21.15.12:0/3341373994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09300c05d0 0x7f09300c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:37.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f092dd9b640 1 -- 172.21.15.12:0/3341373994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0928009a70 con 0x7f09300c6b80 2023-10-30T12:37:37.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f092dd9b640 1 --2- 172.21.15.12:0/3341373994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09300c6b80 0x7f09300c5fb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f092401a0b0 tx=0x7f0924019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:37.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f091effd640 1 -- 172.21.15.12:0/3341373994 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09240b6840 con 0x7f09300c6b80 2023-10-30T12:37:37.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f091effd640 1 -- 172.21.15.12:0/3341373994 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09240a6040 con 0x7f09300c6b80 2023-10-30T12:37:37.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f0934846640 1 -- 172.21.15.12:0/3341373994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0924009420 con 0x7f09300c6b80 2023-10-30T12:37:37.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f091effd640 1 -- 172.21.15.12:0/3341373994 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09240b6c00 con 0x7f09300c6b80 2023-10-30T12:37:37.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f0934846640 1 -- 172.21.15.12:0/3341373994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09300c9930 con 0x7f09300c6b80 2023-10-30T12:37:37.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.320+0000 7f0934846640 1 -- 172.21.15.12:0/3341373994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09300cac00 con 0x7f09300c6b80 2023-10-30T12:37:37.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.324+0000 7f091effd640 1 -- 172.21.15.12:0/3341373994 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f09240b4040 con 0x7f09300c6b80 2023-10-30T12:37:37.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.324+0000 7f091effd640 1 --2- 172.21.15.12:0/3341373994 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f09100664d0 0x7f0910068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:37.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.324+0000 7f091effd640 1 -- 172.21.15.12:0/3341373994 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f092411ead0 con 0x7f09300c6b80 2023-10-30T12:37:37.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.324+0000 7f092e59c640 1 --2- 172.21.15.12:0/3341373994 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f09100664d0 0x7f0910068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:37.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.324+0000 7f092e59c640 1 --2- 172.21.15.12:0/3341373994 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f09100664d0 0x7f0910068990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f09200011f0 tx=0x7f0920007d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:37.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.328+0000 7f091effd640 1 -- 172.21.15.12:0/3341373994 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f09240ec0c0 con 0x7f09300c6b80 2023-10-30T12:37:37.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.520+0000 7f0934846640 1 -- 172.21.15.12:0/3341373994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f093013c720 con 0x7f09300c6b80 2023-10-30T12:37:37.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.524+0000 7f091effd640 1 -- 172.21.15.12:0/3341373994 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6526 (secure 0 0 0) 0x7f093013c720 con 0x7f09300c6b80 2023-10-30T12:37:37.529 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:37:37.530 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:37:21.372155+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":9,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:37:37.530 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2023-10-30T12:37:37.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.524+0000 7f0934846640 1 -- 172.21.15.12:0/3341373994 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f09100664d0 msgr2=0x7f0910068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:37.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.524+0000 7f0934846640 1 --2- 172.21.15.12:0/3341373994 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f09100664d0 0x7f0910068990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f09200011f0 tx=0x7f0920007d40 comp rx=0 tx=0).stop 2023-10-30T12:37:37.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.524+0000 7f0934846640 1 -- 172.21.15.12:0/3341373994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09300c6b80 msgr2=0x7f09300c5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:37.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.524+0000 7f0934846640 1 --2- 172.21.15.12:0/3341373994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09300c6b80 0x7f09300c5fb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f092401a0b0 tx=0x7f0924019220 comp rx=0 tx=0).stop 2023-10-30T12:37:37.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.524+0000 7f0934846640 1 -- 172.21.15.12:0/3341373994 shutdown_connections 2023-10-30T12:37:37.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.524+0000 7f0934846640 1 --2- 172.21.15.12:0/3341373994 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f09100664d0 0x7f0910068990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:37.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.524+0000 7f0934846640 1 --2- 172.21.15.12:0/3341373994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09300c6f60 0x7f093013ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:37.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.524+0000 7f0934846640 1 --2- 172.21.15.12:0/3341373994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09300c6b80 0x7f09300c5fb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.524+0000 7f0934846640 1 --2- 172.21.15.12:0/3341373994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09300c05d0 0x7f09300c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.524+0000 7f0934846640 1 -- 172.21.15.12:0/3341373994 >> 172.21.15.12:0/3341373994 conn(0x7f09300c7910 msgr2=0x7f09300bfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.524+0000 7f0934846640 1 -- 172.21.15.12:0/3341373994 shutdown_connections 2023-10-30T12:37:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:37.524+0000 7f0934846640 1 -- 172.21.15.12:0/3341373994 wait complete. 2023-10-30T12:37:37.550 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:37:39.551 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:37:39.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.696+0000 7f0d25ad6640 1 Processor -- start 2023-10-30T12:37:39.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d25ad6640 1 -- start start 2023-10-30T12:37:39.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d25ad6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d200cf160 0x7f0d200cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:39.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d25ad6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d200cb800 con 0x7f0d2015d6f0 2023-10-30T12:37:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d25ad6640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d200cb330 con 0x7f0d201538a0 2023-10-30T12:37:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d25ad6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200cad70 con 0x7f0d200cf160 2023-10-30T12:37:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1effd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0d2015d6f0 0x7f0d2015a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:57358/0 (socket says 172.21.15.12:57358) 2023-10-30T12:37:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1effd640 1 -- 172.21.15.12:0/1364772726 learned_addr learned my addr 172.21.15.12:0/1364772726 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:37:39.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1ffff640 1 --2- 172.21.15.12:0/1364772726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d200cf160 0x7f0d200cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:39.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480731394 0 0) 0x7f0d200cb800 con 0x7f0d2015d6f0 2023-10-30T12:37:39.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d08003530 con 0x7f0d2015d6f0 2023-10-30T12:37:39.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939792323 0 0) 0x7f0d200cb330 con 0x7f0d201538a0 2023-10-30T12:37:39.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d200cb800 con 0x7f0d201538a0 2023-10-30T12:37:39.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3399038138 0 0) 0x7f0d08003530 con 0x7f0d2015d6f0 2023-10-30T12:37:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d200cf160 msgr2=0x7f0d200cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:37:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 --2- 172.21.15.12:0/1364772726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d200cf160 0x7f0d200cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 >> v1:172.21.15.154:6789/0 conn(0x7f0d201538a0 legacy=0x7f0d200ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d200cb050 con 0x7f0d2015d6f0 2023-10-30T12:37:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f0d14001680 con 0x7f0d2015d6f0 2023-10-30T12:37:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 >> v1:172.21.15.12:6789/0 conn(0x7f0d2015d6f0 legacy=0x7f0d2015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 --2- 172.21.15.12:0/1364772726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d200cf160 0x7f0d08003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 --2- 172.21.15.12:0/1364772726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d080040a0 0x7f0d080047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 --2- 172.21.15.12:0/1364772726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d08004d10 0x7f0d08005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200cad70 con 0x7f0d08004d10 2023-10-30T12:37:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0800ba70 con 0x7f0d080040a0 2023-10-30T12:37:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0800bc30 con 0x7f0d200cf160 2023-10-30T12:37:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1ffff640 1 --2- 172.21.15.12:0/1364772726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d08004d10 0x7f0d08005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1effd640 1 --2- 172.21.15.12:0/1364772726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d080040a0 0x7f0d080047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1f7fe640 1 --2- 172.21.15.12:0/1364772726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d200cf160 0x7f0d08003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1ffff640 1 -- 172.21.15.12:0/1364772726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d200cf160 msgr2=0x7f0d08003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:37:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1ffff640 1 --2- 172.21.15.12:0/1364772726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d200cf160 0x7f0d08003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1ffff640 1 -- 172.21.15.12:0/1364772726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d080040a0 msgr2=0x7f0d080047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:37:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1ffff640 1 --2- 172.21.15.12:0/1364772726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d080040a0 0x7f0d080047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:39.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1ffff640 1 -- 172.21.15.12:0/1364772726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d08003530 con 0x7f0d08004d10 2023-10-30T12:37:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1ffff640 1 --2- 172.21.15.12:0/1364772726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d08004d10 0x7f0d08005190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009fc0 tx=0x7f0d1801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dafafc51bbb501b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d180a5c70 con 0x7f0d08004d10 2023-10-30T12:37:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1800f040 con 0x7f0d08004d10 2023-10-30T12:37:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1e7fc640 1 -- 172.21.15.12:0/1364772726 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d180ad440 con 0x7f0d08004d10 2023-10-30T12:37:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d25ad6640 1 -- 172.21.15.12:0/1364772726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d08004d10 msgr2=0x7f0d08005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d25ad6640 1 --2- 172.21.15.12:0/1364772726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d08004d10 0x7f0d08005190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009fc0 tx=0x7f0d1801cc60 comp rx=0 tx=0).stop 2023-10-30T12:37:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d1ffff640 1 -- 172.21.15.12:0/1364772726 reap_dead start 2023-10-30T12:37:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d25ad6640 1 -- 172.21.15.12:0/1364772726 shutdown_connections 2023-10-30T12:37:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d25ad6640 1 -- 172.21.15.12:0/1364772726 >> 172.21.15.12:0/1364772726 conn(0x7f0d2006bfe0 msgr2=0x7f0d2006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d25ad6640 1 -- 172.21.15.12:0/1364772726 shutdown_connections 2023-10-30T12:37:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.704+0000 7f0d25ad6640 1 -- 172.21.15.12:0/1364772726 wait complete. 2023-10-30T12:37:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d25ad6640 1 Processor -- start 2023-10-30T12:37:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d25ad6640 1 -- start start 2023-10-30T12:37:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d25ad6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d08004d10 0x7f0d200b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d25ad6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d200c64d0 0x7f0d2014c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d25ad6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d2014c780 0x7f0d200c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d25ad6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200c6c30 con 0x7f0d200c64d0 2023-10-30T12:37:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d25ad6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2014cc00 con 0x7f0d08004d10 2023-10-30T12:37:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d25ad6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200c7e00 con 0x7f0d2014c780 2023-10-30T12:37:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d1effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d200c64d0 0x7f0d2014c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d1effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d200c64d0 0x7f0d2014c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52604/0 (socket says 172.21.15.12:52604) 2023-10-30T12:37:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d1effd640 1 -- 172.21.15.12:0/3380562738 learned_addr learned my addr 172.21.15.12:0/3380562738 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d1f7fe640 1 --2- 172.21.15.12:0/3380562738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d08004d10 0x7f0d200b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d1ffff640 1 --2- 172.21.15.12:0/3380562738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d2014c780 0x7f0d200c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d1effd640 1 -- 172.21.15.12:0/3380562738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d2014c780 msgr2=0x7f0d200c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d1effd640 1 --2- 172.21.15.12:0/3380562738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d2014c780 0x7f0d200c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d1effd640 1 -- 172.21.15.12:0/3380562738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d08004d10 msgr2=0x7f0d200b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d1effd640 1 --2- 172.21.15.12:0/3380562738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d08004d10 0x7f0d200b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d1effd640 1 -- 172.21.15.12:0/3380562738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d20154300 con 0x7f0d200c64d0 2023-10-30T12:37:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d1effd640 1 --2- 172.21.15.12:0/3380562738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d200c64d0 0x7f0d2014c240 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0d14007340 tx=0x7f0d14007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d07fff640 1 -- 172.21.15.12:0/3380562738 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1400f9c0 con 0x7f0d200c64d0 2023-10-30T12:37:39.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d07fff640 1 -- 172.21.15.12:0/3380562738 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1400fb20 con 0x7f0d200c64d0 2023-10-30T12:37:39.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d07fff640 1 -- 172.21.15.12:0/3380562738 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1400fed0 con 0x7f0d200c64d0 2023-10-30T12:37:39.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d25ad6640 1 -- 172.21.15.12:0/3380562738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d201f2d80 con 0x7f0d200c64d0 2023-10-30T12:37:39.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.708+0000 7f0d25ad6640 1 -- 172.21.15.12:0/3380562738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d20154880 con 0x7f0d200c64d0 2023-10-30T12:37:39.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.712+0000 7f0d25ad6640 1 -- 172.21.15.12:0/3380562738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d20154a40 con 0x7f0d200c64d0 2023-10-30T12:37:39.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.712+0000 7f0d07fff640 1 -- 172.21.15.12:0/3380562738 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0d1401d460 con 0x7f0d200c64d0 2023-10-30T12:37:39.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.712+0000 7f0d07fff640 1 --2- 172.21.15.12:0/3380562738 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0d0c0664a0 0x7f0d0c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:39.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.712+0000 7f0d07fff640 1 -- 172.21.15.12:0/3380562738 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f0d14083fd0 con 0x7f0d200c64d0 2023-10-30T12:37:39.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.712+0000 7f0d1f7fe640 1 --2- 172.21.15.12:0/3380562738 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0d0c0664a0 0x7f0d0c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:39.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.712+0000 7f0d1f7fe640 1 --2- 172.21.15.12:0/3380562738 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0d0c0664a0 0x7f0d0c068960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0d10007fd0 tx=0x7f0d10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:39.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.720+0000 7f0d07fff640 1 -- 172.21.15.12:0/3380562738 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0d14051460 con 0x7f0d200c64d0 2023-10-30T12:37:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.904+0000 7f0d25ad6640 1 -- 172.21.15.12:0/3380562738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d20159830 con 0x7f0d200c64d0 2023-10-30T12:37:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.904+0000 7f0d07fff640 1 -- 172.21.15.12:0/3380562738 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6526 (secure 0 0 0) 0x7f0d14054a80 con 0x7f0d200c64d0 2023-10-30T12:37:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:37:39.911 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:37:21.372155+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":9,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:37:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2023-10-30T12:37:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.904+0000 7f0d25ad6640 1 -- 172.21.15.12:0/3380562738 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0d0c0664a0 msgr2=0x7f0d0c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.904+0000 7f0d25ad6640 1 --2- 172.21.15.12:0/3380562738 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0d0c0664a0 0x7f0d0c068960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0d10007fd0 tx=0x7f0d10009040 comp rx=0 tx=0).stop 2023-10-30T12:37:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.904+0000 7f0d25ad6640 1 -- 172.21.15.12:0/3380562738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d200c64d0 msgr2=0x7f0d2014c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.904+0000 7f0d25ad6640 1 --2- 172.21.15.12:0/3380562738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d200c64d0 0x7f0d2014c240 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0d14007340 tx=0x7f0d14007800 comp rx=0 tx=0).stop 2023-10-30T12:37:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.904+0000 7f0d25ad6640 1 -- 172.21.15.12:0/3380562738 shutdown_connections 2023-10-30T12:37:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.904+0000 7f0d25ad6640 1 --2- 172.21.15.12:0/3380562738 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0d0c0664a0 0x7f0d0c068960 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.904+0000 7f0d25ad6640 1 --2- 172.21.15.12:0/3380562738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d2014c780 0x7f0d200c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.904+0000 7f0d25ad6640 1 --2- 172.21.15.12:0/3380562738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d200c64d0 0x7f0d2014c240 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.904+0000 7f0d25ad6640 1 --2- 172.21.15.12:0/3380562738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d08004d10 0x7f0d200b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.904+0000 7f0d25ad6640 1 -- 172.21.15.12:0/3380562738 >> 172.21.15.12:0/3380562738 conn(0x7f0d2006bfe0 msgr2=0x7f0d2015b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.904+0000 7f0d25ad6640 1 -- 172.21.15.12:0/3380562738 shutdown_connections 2023-10-30T12:37:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:39.904+0000 7f0d25ad6640 1 -- 172.21.15.12:0/3380562738 wait complete. 2023-10-30T12:37:39.931 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:37:41.932 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:37:42.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.084+0000 7f6198f3e640 1 Processor -- start 2023-10-30T12:37:42.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6198f3e640 1 -- start start 2023-10-30T12:37:42.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6198f3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f619406f7a0 0x7f619406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6198f3e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6194066cf0 con 0x7f6194071110 2023-10-30T12:37:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6198f3e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61940c70d0 con 0x7f61940c6bd0 2023-10-30T12:37:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6198f3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61940700c0 con 0x7f619406f7a0 2023-10-30T12:37:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6192d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f6194071110 0x7f6194158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60846/0 (socket says 172.21.15.12:60846) 2023-10-30T12:37:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6192d9d640 1 -- 172.21.15.12:0/1738401099 learned_addr learned my addr 172.21.15.12:0/1738401099 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:37:42.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f619259c640 1 --2- 172.21.15.12:0/1738401099 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f619406f7a0 0x7f619406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:42.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6191d9b640 1 -- 172.21.15.12:0/1738401099 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009520980 0 0) 0x7f6194066cf0 con 0x7f6194071110 2023-10-30T12:37:42.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6191d9b640 1 -- 172.21.15.12:0/1738401099 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6180003530 con 0x7f6194071110 2023-10-30T12:37:42.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6191d9b640 1 -- 172.21.15.12:0/1738401099 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2663375379 0 0) 0x7f6180003530 con 0x7f6194071110 2023-10-30T12:37:42.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6191d9b640 1 -- 172.21.15.12:0/1738401099 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f619406f7a0 msgr2=0x7f619406fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:37:42.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6191d9b640 1 --2- 172.21.15.12:0/1738401099 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f619406f7a0 0x7f619406fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:42.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6191d9b640 1 -- 172.21.15.12:0/1738401099 >> v1:172.21.15.154:6789/0 conn(0x7f61940c6bd0 legacy=0x7f6194069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:42.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6191d9b640 1 -- 172.21.15.12:0/1738401099 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61940c7390 con 0x7f6194071110 2023-10-30T12:37:42.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6191d9b640 1 -- 172.21.15.12:0/1738401099 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f618c001a00 con 0x7f6194071110 2023-10-30T12:37:42.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6191d9b640 1 -- 172.21.15.12:0/1738401099 >> v1:172.21.15.12:6789/0 conn(0x7f6194071110 legacy=0x7f6194158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:42.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6191d9b640 1 --2- 172.21.15.12:0/1738401099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61800039b0 0x7f6180003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:42.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6191d9b640 1 --2- 172.21.15.12:0/1738401099 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f619406f7a0 0x7f61800047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:42.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6191d9b640 1 --2- 172.21.15.12:0/1738401099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6180004d10 0x7f6180005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:42.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6191d9b640 1 -- 172.21.15.12:0/1738401099 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61940700c0 con 0x7f61800039b0 2023-10-30T12:37:42.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6191d9b640 1 -- 172.21.15.12:0/1738401099 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618000b990 con 0x7f6180004d10 2023-10-30T12:37:42.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6191d9b640 1 -- 172.21.15.12:0/1738401099 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618000bb50 con 0x7f619406f7a0 2023-10-30T12:37:42.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f619259c640 1 --2- 172.21.15.12:0/1738401099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61800039b0 0x7f6180003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:42.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f618bfff640 1 --2- 172.21.15.12:0/1738401099 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f619406f7a0 0x7f61800047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:42.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f6192d9d640 1 --2- 172.21.15.12:0/1738401099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6180004d10 0x7f6180005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:42.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f619259c640 1 -- 172.21.15.12:0/1738401099 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f619406f7a0 msgr2=0x7f61800047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:37:42.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f619259c640 1 --2- 172.21.15.12:0/1738401099 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f619406f7a0 0x7f61800047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:42.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f619259c640 1 -- 172.21.15.12:0/1738401099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6180004d10 msgr2=0x7f6180005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:37:42.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f619259c640 1 --2- 172.21.15.12:0/1738401099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6180004d10 0x7f6180005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:42.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.088+0000 7f619259c640 1 -- 172.21.15.12:0/1738401099 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6180003530 con 0x7f61800039b0 2023-10-30T12:37:42.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.092+0000 7f619259c640 1 --2- 172.21.15.12:0/1738401099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61800039b0 0x7f6180003d90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6184009ae0 tx=0x7f618401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7a2942286d6506 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:42.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.092+0000 7f6191d9b640 1 -- 172.21.15.12:0/1738401099 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840a5580 con 0x7f61800039b0 2023-10-30T12:37:42.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.092+0000 7f6191d9b640 1 -- 172.21.15.12:0/1738401099 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61840a56e0 con 0x7f61800039b0 2023-10-30T12:37:42.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.092+0000 7f6191d9b640 1 -- 172.21.15.12:0/1738401099 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840a5a90 con 0x7f61800039b0 2023-10-30T12:37:42.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.092+0000 7f6198f3e640 1 -- 172.21.15.12:0/1738401099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61800039b0 msgr2=0x7f6180003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:42.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.092+0000 7f6198f3e640 1 --2- 172.21.15.12:0/1738401099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61800039b0 0x7f6180003d90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6184009ae0 tx=0x7f618401c990 comp rx=0 tx=0).stop 2023-10-30T12:37:42.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.092+0000 7f6192d9d640 1 -- 172.21.15.12:0/1738401099 reap_dead start 2023-10-30T12:37:42.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.092+0000 7f6198f3e640 1 -- 172.21.15.12:0/1738401099 shutdown_connections 2023-10-30T12:37:42.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.092+0000 7f6198f3e640 1 -- 172.21.15.12:0/1738401099 >> 172.21.15.12:0/1738401099 conn(0x7f61940c7910 msgr2=0x7f61940c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:42.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.092+0000 7f6198f3e640 1 -- 172.21.15.12:0/1738401099 shutdown_connections 2023-10-30T12:37:42.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.092+0000 7f6198f3e640 1 -- 172.21.15.12:0/1738401099 wait complete. 2023-10-30T12:37:42.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f6198f3e640 1 Processor -- start 2023-10-30T12:37:42.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f6198f3e640 1 -- start start 2023-10-30T12:37:42.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f6198f3e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61800039b0 0x7f619414ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:42.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f6198f3e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61941582f0 0x7f619414f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:42.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f6198f3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6194155d80 0x7f6194156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:42.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f6198f3e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6194150080 con 0x7f61941582f0 2023-10-30T12:37:42.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f6198f3e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6194156740 con 0x7f61800039b0 2023-10-30T12:37:42.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f6198f3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619414fac0 con 0x7f6194155d80 2023-10-30T12:37:42.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f618bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61941582f0 0x7f619414f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:42.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f618bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61941582f0 0x7f619414f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59228/0 (socket says 172.21.15.12:59228) 2023-10-30T12:37:42.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f618bfff640 1 -- 172.21.15.12:0/2534062972 learned_addr learned my addr 172.21.15.12:0/2534062972 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:42.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f6192d9d640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6194155d80 0x7f6194156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:42.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f619259c640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61800039b0 0x7f619414ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:42.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f618bfff640 1 -- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6194155d80 msgr2=0x7f6194156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:42.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f618bfff640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6194155d80 0x7f6194156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:42.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f618bfff640 1 -- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61800039b0 msgr2=0x7f619414ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:42.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f618bfff640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61800039b0 0x7f619414ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:42.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f618bfff640 1 -- 172.21.15.12:0/2534062972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619414fda0 con 0x7f61941582f0 2023-10-30T12:37:42.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f6192d9d640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6194155d80 0x7f6194156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:37:42.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f619259c640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61800039b0 0x7f619414ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:37:42.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f618bfff640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61941582f0 0x7f619414f440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f617c010f40 tx=0x7f617c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:42.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f618affd640 1 -- 172.21.15.12:0/2534062972 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c005270 con 0x7f61941582f0 2023-10-30T12:37:42.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f618affd640 1 -- 172.21.15.12:0/2534062972 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c00d030 con 0x7f61941582f0 2023-10-30T12:37:42.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f618affd640 1 -- 172.21.15.12:0/2534062972 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c00ee50 con 0x7f61941582f0 2023-10-30T12:37:42.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f6198f3e640 1 -- 172.21.15.12:0/2534062972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61940c4000 con 0x7f61941582f0 2023-10-30T12:37:42.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.096+0000 7f6198f3e640 1 -- 172.21.15.12:0/2534062972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61941ef540 con 0x7f61941582f0 2023-10-30T12:37:42.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.100+0000 7f6198f3e640 1 -- 172.21.15.12:0/2534062972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619404e170 con 0x7f61941582f0 2023-10-30T12:37:42.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.100+0000 7f618affd640 1 -- 172.21.15.12:0/2534062972 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f617c003600 con 0x7f61941582f0 2023-10-30T12:37:42.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.100+0000 7f618affd640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f61740664a0 0x7f6174068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:42.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.100+0000 7f618affd640 1 -- 172.21.15.12:0/2534062972 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f617c087980 con 0x7f61941582f0 2023-10-30T12:37:42.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.100+0000 7f619259c640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f61740664a0 0x7f6174068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:42.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.100+0000 7f619259c640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f61740664a0 0x7f6174068960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f61840af000 tx=0x7f61840046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:42.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.104+0000 7f618affd640 1 -- 172.21.15.12:0/2534062972 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f617c054cd0 con 0x7f61941582f0 2023-10-30T12:37:42.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.292+0000 7f6198f3e640 1 -- 172.21.15.12:0/2534062972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6194072150 con 0x7f61941582f0 2023-10-30T12:37:42.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.292+0000 7f618affd640 1 -- 172.21.15.12:0/2534062972 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6526 (secure 0 0 0) 0x7f617c018020 con 0x7f61941582f0 2023-10-30T12:37:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:37:42.296 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:37:21.372155+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":9,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:37:42.297 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2023-10-30T12:37:42.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.296+0000 7f6198f3e640 1 -- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f61740664a0 msgr2=0x7f6174068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:42.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.296+0000 7f6198f3e640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f61740664a0 0x7f6174068960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f61840af000 tx=0x7f61840046d0 comp rx=0 tx=0).stop 2023-10-30T12:37:42.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.296+0000 7f6198f3e640 1 -- 172.21.15.12:0/2534062972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61941582f0 msgr2=0x7f619414f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.296+0000 7f6198f3e640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61941582f0 0x7f619414f440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f617c010f40 tx=0x7f617c00e530 comp rx=0 tx=0).stop 2023-10-30T12:37:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.296+0000 7f6198f3e640 1 -- 172.21.15.12:0/2534062972 shutdown_connections 2023-10-30T12:37:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.296+0000 7f6198f3e640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f61740664a0 0x7f6174068960 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.296+0000 7f6198f3e640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6194155d80 0x7f6194156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.296+0000 7f6198f3e640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61941582f0 0x7f619414f440 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.296+0000 7f6198f3e640 1 --2- 172.21.15.12:0/2534062972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61800039b0 0x7f619414ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.296+0000 7f6198f3e640 1 -- 172.21.15.12:0/2534062972 >> 172.21.15.12:0/2534062972 conn(0x7f61940c7910 msgr2=0x7f61940c92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.296+0000 7f6198f3e640 1 -- 172.21.15.12:0/2534062972 shutdown_connections 2023-10-30T12:37:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:42.296+0000 7f6198f3e640 1 -- 172.21.15.12:0/2534062972 wait complete. 2023-10-30T12:37:42.317 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:37:44.318 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:37:44.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.472+0000 7fa49873e640 1 Processor -- start 2023-10-30T12:37:44.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.472+0000 7fa49873e640 1 -- start start 2023-10-30T12:37:44.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.472+0000 7fa49873e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4901538a0 0x7fa49015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:44.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.472+0000 7fa49873e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4900cb800 con 0x7fa4900cf160 2023-10-30T12:37:44.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.472+0000 7fa49873e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49015dc30 con 0x7fa4900ca0e0 2023-10-30T12:37:44.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.472+0000 7fa49873e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4900cb330 con 0x7fa4901538a0 2023-10-30T12:37:44.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.472+0000 7fa496cdb640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa4900cf160 0x7fa490164070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60858/0 (socket says 172.21.15.12:60858) 2023-10-30T12:37:44.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.472+0000 7fa496cdb640 1 -- 172.21.15.12:0/628353403 learned_addr learned my addr 172.21.15.12:0/628353403 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:37:44.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.472+0000 7fa4964da640 1 --2- 172.21.15.12:0/628353403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4901538a0 0x7fa49015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:44.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.472+0000 7fa4954d8640 1 -- 172.21.15.12:0/628353403 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477171018 0 0) 0x7fa4900cb800 con 0x7fa4900cf160 2023-10-30T12:37:44.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.472+0000 7fa4954d8640 1 -- 172.21.15.12:0/628353403 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa47c003530 con 0x7fa4900cf160 2023-10-30T12:37:44.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.472+0000 7fa4964da640 1 -- 172.21.15.12:0/628353403 >> v1:172.21.15.154:6789/0 conn(0x7fa4900ca0e0 legacy=0x7fa49015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:44.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.472+0000 7fa4964da640 1 -- 172.21.15.12:0/628353403 >> v1:172.21.15.12:6789/0 conn(0x7fa4900cf160 legacy=0x7fa490164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:44.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.472+0000 7fa4954d8640 1 -- 172.21.15.12:0/628353403 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2485240573 0 0) 0x7fa47c003530 con 0x7fa4900cf160 2023-10-30T12:37:44.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.472+0000 7fa4964da640 1 -- 172.21.15.12:0/628353403 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4900cad70 con 0x7fa4901538a0 2023-10-30T12:37:44.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.476+0000 7fa4964da640 1 --2- 172.21.15.12:0/628353403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4901538a0 0x7fa49015d6f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0097f0 tx=0x7fa48c01ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5caafdf84964f8fc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:44.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.476+0000 7fa4954d8640 1 -- 172.21.15.12:0/628353403 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa488001a00 con 0x7fa4901538a0 2023-10-30T12:37:44.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.476+0000 7fa4954d8640 1 -- 172.21.15.12:0/628353403 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c0a8650 con 0x7fa4901538a0 2023-10-30T12:37:44.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.476+0000 7fa4954d8640 1 -- 172.21.15.12:0/628353403 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c0a89c0 con 0x7fa4901538a0 2023-10-30T12:37:44.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.476+0000 7fa49873e640 1 -- 172.21.15.12:0/628353403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4901538a0 msgr2=0x7fa49015d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:44.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.476+0000 7fa49873e640 1 --2- 172.21.15.12:0/628353403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4901538a0 0x7fa49015d6f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0097f0 tx=0x7fa48c01ebd0 comp rx=0 tx=0).stop 2023-10-30T12:37:44.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.476+0000 7fa49873e640 1 -- 172.21.15.12:0/628353403 shutdown_connections 2023-10-30T12:37:44.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.476+0000 7fa49873e640 1 --2- 172.21.15.12:0/628353403 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4901538a0 0x7fa49015d6f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:44.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.476+0000 7fa49873e640 1 -- 172.21.15.12:0/628353403 >> 172.21.15.12:0/628353403 conn(0x7fa49006bfe0 msgr2=0x7fa49006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:44.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.476+0000 7fa49873e640 1 -- 172.21.15.12:0/628353403 shutdown_connections 2023-10-30T12:37:44.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.476+0000 7fa49873e640 1 -- 172.21.15.12:0/628353403 wait complete. 2023-10-30T12:37:44.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa49873e640 1 Processor -- start 2023-10-30T12:37:44.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa49873e640 1 -- start start 2023-10-30T12:37:44.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa49873e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4900ca0e0 0x7fa4901f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:44.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa49873e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900cf160 0x7fa49015a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:44.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa49873e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4901538a0 0x7fa4900b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:44.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa49873e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4901f3200 con 0x7fa4900cf160 2023-10-30T12:37:44.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa49873e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4900c7d10 con 0x7fa4900ca0e0 2023-10-30T12:37:44.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa49873e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4900ce800 con 0x7fa4901538a0 2023-10-30T12:37:44.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa495cd9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900cf160 0x7fa49015a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:44.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa495cd9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900cf160 0x7fa49015a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59242/0 (socket says 172.21.15.12:59242) 2023-10-30T12:37:44.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa495cd9640 1 -- 172.21.15.12:0/1654754779 learned_addr learned my addr 172.21.15.12:0/1654754779 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:44.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa4964da640 1 --2- 172.21.15.12:0/1654754779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4900ca0e0 0x7fa4901f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:44.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa496cdb640 1 --2- 172.21.15.12:0/1654754779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4901538a0 0x7fa4900b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:44.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa495cd9640 1 -- 172.21.15.12:0/1654754779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4901538a0 msgr2=0x7fa4900b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:44.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa495cd9640 1 --2- 172.21.15.12:0/1654754779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4901538a0 0x7fa4900b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:44.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa495cd9640 1 -- 172.21.15.12:0/1654754779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4900ca0e0 msgr2=0x7fa4901f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:44.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa495cd9640 1 --2- 172.21.15.12:0/1654754779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4900ca0e0 0x7fa4901f22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:44.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa495cd9640 1 -- 172.21.15.12:0/1654754779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c009420 con 0x7fa4900cf160 2023-10-30T12:37:44.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa495cd9640 1 --2- 172.21.15.12:0/1654754779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900cf160 0x7fa49015a8b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c420 tx=0x7fa48000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:44.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa486ffd640 1 -- 172.21.15.12:0/1654754779 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4800052d0 con 0x7fa4900cf160 2023-10-30T12:37:44.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa486ffd640 1 -- 172.21.15.12:0/1654754779 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa480005430 con 0x7fa4900cf160 2023-10-30T12:37:44.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa49873e640 1 -- 172.21.15.12:0/1654754779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49015b0f0 con 0x7fa4900cf160 2023-10-30T12:37:44.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa486ffd640 1 -- 172.21.15.12:0/1654754779 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480007260 con 0x7fa4900cf160 2023-10-30T12:37:44.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.480+0000 7fa49873e640 1 -- 172.21.15.12:0/1654754779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4901f2c40 con 0x7fa4900cf160 2023-10-30T12:37:44.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.484+0000 7fa49873e640 1 -- 172.21.15.12:0/1654754779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4900c7e80 con 0x7fa4900cf160 2023-10-30T12:37:44.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.484+0000 7fa486ffd640 1 -- 172.21.15.12:0/1654754779 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa4800073c0 con 0x7fa4900cf160 2023-10-30T12:37:44.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.484+0000 7fa486ffd640 1 --2- 172.21.15.12:0/1654754779 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4740664d0 0x7fa474068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:44.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.484+0000 7fa486ffd640 1 -- 172.21.15.12:0/1654754779 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa480083190 con 0x7fa4900cf160 2023-10-30T12:37:44.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.484+0000 7fa4964da640 1 --2- 172.21.15.12:0/1654754779 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4740664d0 0x7fa474068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:44.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.484+0000 7fa4964da640 1 --2- 172.21.15.12:0/1654754779 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4740664d0 0x7fa474068990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0b2000 tx=0x7fa48c0aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:44.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.488+0000 7fa486ffd640 1 -- 172.21.15.12:0/1654754779 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa480050500 con 0x7fa4900cf160 2023-10-30T12:37:44.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.676+0000 7fa49873e640 1 -- 172.21.15.12:0/1654754779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa49015dc30 con 0x7fa4900cf160 2023-10-30T12:37:44.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.676+0000 7fa486ffd640 1 -- 172.21.15.12:0/1654754779 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6526 (secure 0 0 0) 0x7fa49015dc30 con 0x7fa4900cf160 2023-10-30T12:37:44.680 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:37:44.681 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:37:21.372155+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":9,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:37:44.681 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2023-10-30T12:37:44.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.680+0000 7fa49873e640 1 -- 172.21.15.12:0/1654754779 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4740664d0 msgr2=0x7fa474068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:44.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.680+0000 7fa49873e640 1 --2- 172.21.15.12:0/1654754779 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4740664d0 0x7fa474068990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0b2000 tx=0x7fa48c0aa070 comp rx=0 tx=0).stop 2023-10-30T12:37:44.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.680+0000 7fa49873e640 1 -- 172.21.15.12:0/1654754779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900cf160 msgr2=0x7fa49015a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:44.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.680+0000 7fa49873e640 1 --2- 172.21.15.12:0/1654754779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900cf160 0x7fa49015a8b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c420 tx=0x7fa48000c8e0 comp rx=0 tx=0).stop 2023-10-30T12:37:44.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.680+0000 7fa49873e640 1 -- 172.21.15.12:0/1654754779 shutdown_connections 2023-10-30T12:37:44.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.680+0000 7fa49873e640 1 --2- 172.21.15.12:0/1654754779 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4740664d0 0x7fa474068990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:44.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.680+0000 7fa49873e640 1 --2- 172.21.15.12:0/1654754779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4901538a0 0x7fa4900b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:44.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.680+0000 7fa49873e640 1 --2- 172.21.15.12:0/1654754779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900cf160 0x7fa49015a8b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:44.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.680+0000 7fa49873e640 1 --2- 172.21.15.12:0/1654754779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4900ca0e0 0x7fa4901f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:44.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.680+0000 7fa49873e640 1 -- 172.21.15.12:0/1654754779 >> 172.21.15.12:0/1654754779 conn(0x7fa49006bfe0 msgr2=0x7fa49015a090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:44.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.680+0000 7fa49873e640 1 -- 172.21.15.12:0/1654754779 shutdown_connections 2023-10-30T12:37:44.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:44.680+0000 7fa49873e640 1 -- 172.21.15.12:0/1654754779 wait complete. 2023-10-30T12:37:44.702 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:37:46.703 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:37:46.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.856+0000 7f8f090a9640 1 Processor -- start 2023-10-30T12:37:46.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.856+0000 7f8f090a9640 1 -- start start 2023-10-30T12:37:46.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.856+0000 7f8f090a9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8f0415d6f0 0x7f8f0415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:46.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.856+0000 7f8f090a9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8f040cf160 0x7f8f040cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:46.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.856+0000 7f8f090a9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f040cb800 con 0x7f8f041538a0 2023-10-30T12:37:46.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.856+0000 7f8f090a9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f040cb330 con 0x7f8f040cf160 2023-10-30T12:37:46.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.856+0000 7f8f090a9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f040cad70 con 0x7f8f0415d6f0 2023-10-30T12:37:46.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.856+0000 7f8f0359e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8f040cf160 0x7f8f040cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:46.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.856+0000 7f8f0359e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8f040cf160 0x7f8f040cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59254/0 (socket says 172.21.15.12:59254) 2023-10-30T12:37:46.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.856+0000 7f8f0359e640 1 -- 172.21.15.12:0/1055016195 learned_addr learned my addr 172.21.15.12:0/1055016195 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:46.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.856+0000 7f8f0259c640 1 --2- 172.21.15.12:0/1055016195 >> v2:172.21.15.154:3300/0 conn(0x7f8f0415d6f0 0x7f8f0415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:46.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.856+0000 7f8f0359e640 1 -- 172.21.15.12:0/1055016195 >> v2:172.21.15.154:3300/0 conn(0x7f8f0415d6f0 msgr2=0x7f8f0415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:37:46.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.856+0000 7f8f0359e640 1 --2- 172.21.15.12:0/1055016195 >> v2:172.21.15.154:3300/0 conn(0x7f8f0415d6f0 0x7f8f0415a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:46.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.856+0000 7f8f0359e640 1 -- 172.21.15.12:0/1055016195 >> v1:172.21.15.12:6789/0 conn(0x7f8f041538a0 legacy=0x7f8f040ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:46.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.856+0000 7f8f0359e640 1 -- 172.21.15.12:0/1055016195 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f040cb050 con 0x7f8f040cf160 2023-10-30T12:37:46.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.860+0000 7f8f0359e640 1 --2- 172.21.15.12:0/1055016195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f040cf160 0x7f8f040cf590 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8f040b30a0 tx=0x7f8efc01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c188871ece9f0152 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:46.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.860+0000 7f8f01d9b640 1 -- 172.21.15.12:0/1055016195 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0af400 con 0x7f8f040cf160 2023-10-30T12:37:46.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.860+0000 7f8f01d9b640 1 -- 172.21.15.12:0/1055016195 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc0ab050 con 0x7f8f040cf160 2023-10-30T12:37:46.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.860+0000 7f8f01d9b640 1 -- 172.21.15.12:0/1055016195 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0af780 con 0x7f8f040cf160 2023-10-30T12:37:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.860+0000 7f8f090a9640 1 -- 172.21.15.12:0/1055016195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f040cf160 msgr2=0x7f8f040cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.860+0000 7f8f090a9640 1 --2- 172.21.15.12:0/1055016195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f040cf160 0x7f8f040cf590 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8f040b30a0 tx=0x7f8efc01f1c0 comp rx=0 tx=0).stop 2023-10-30T12:37:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.860+0000 7f8f090a9640 1 -- 172.21.15.12:0/1055016195 shutdown_connections 2023-10-30T12:37:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.860+0000 7f8f090a9640 1 --2- 172.21.15.12:0/1055016195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f040cf160 0x7f8f040cf590 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.860+0000 7f8f090a9640 1 --2- 172.21.15.12:0/1055016195 >> v2:172.21.15.154:3300/0 conn(0x7f8f0415d6f0 0x7f8f0415a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.860+0000 7f8f090a9640 1 -- 172.21.15.12:0/1055016195 >> 172.21.15.12:0/1055016195 conn(0x7f8f0406bfe0 msgr2=0x7f8f0406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:46.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.860+0000 7f8f090a9640 1 -- 172.21.15.12:0/1055016195 shutdown_connections 2023-10-30T12:37:46.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.860+0000 7f8f090a9640 1 -- 172.21.15.12:0/1055016195 wait complete. 2023-10-30T12:37:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f090a9640 1 Processor -- start 2023-10-30T12:37:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f090a9640 1 -- start start 2023-10-30T12:37:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f090a9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f040cf160 0x7f8f0414eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:46.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f090a9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f041538a0 0x7f8f0414f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:46.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f090a9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f0415d6f0 0x7f8f041f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:46.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f090a9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040b43c0 con 0x7f8f041538a0 2023-10-30T12:37:46.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f090a9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f041f33b0 con 0x7f8f040cf160 2023-10-30T12:37:46.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f090a9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040b4660 con 0x7f8f0415d6f0 2023-10-30T12:37:46.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f0259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f041538a0 0x7f8f0414f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:46.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f0259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f041538a0 0x7f8f0414f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59258/0 (socket says 172.21.15.12:59258) 2023-10-30T12:37:46.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f0259c640 1 -- 172.21.15.12:0/3076790802 learned_addr learned my addr 172.21.15.12:0/3076790802 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:46.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f02d9d640 1 --2- 172.21.15.12:0/3076790802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f040cf160 0x7f8f0414eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:46.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f0359e640 1 --2- 172.21.15.12:0/3076790802 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f0415d6f0 0x7f8f041f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:46.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f0259c640 1 -- 172.21.15.12:0/3076790802 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f0415d6f0 msgr2=0x7f8f041f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:46.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f0259c640 1 --2- 172.21.15.12:0/3076790802 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f0415d6f0 0x7f8f041f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:46.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f0259c640 1 -- 172.21.15.12:0/3076790802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f040cf160 msgr2=0x7f8f0414eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:46.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f0259c640 1 --2- 172.21.15.12:0/3076790802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f040cf160 0x7f8f0414eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:46.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f0259c640 1 -- 172.21.15.12:0/3076790802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc009b30 con 0x7f8f041538a0 2023-10-30T12:37:46.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f02d9d640 1 --2- 172.21.15.12:0/3076790802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f040cf160 0x7f8f0414eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:37:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.864+0000 7f8f0259c640 1 --2- 172.21.15.12:0/3076790802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f041538a0 0x7f8f0414f0d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800d5f0 tx=0x7f8ef800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.868+0000 7f8eeb7fe640 1 -- 172.21.15.12:0/3076790802 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80125f0 con 0x7f8f041538a0 2023-10-30T12:37:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.868+0000 7f8eeb7fe640 1 -- 172.21.15.12:0/3076790802 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef8012750 con 0x7f8f041538a0 2023-10-30T12:37:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.868+0000 7f8f090a9640 1 -- 172.21.15.12:0/3076790802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f040b4940 con 0x7f8f041538a0 2023-10-30T12:37:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.868+0000 7f8eeb7fe640 1 -- 172.21.15.12:0/3076790802 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8012ae0 con 0x7f8f041538a0 2023-10-30T12:37:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.868+0000 7f8f090a9640 1 -- 172.21.15.12:0/3076790802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f040b4f00 con 0x7f8f041538a0 2023-10-30T12:37:46.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.868+0000 7f8f090a9640 1 -- 172.21.15.12:0/3076790802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f040b2a50 con 0x7f8f041538a0 2023-10-30T12:37:46.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.868+0000 7f8eeb7fe640 1 -- 172.21.15.12:0/3076790802 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8ef8012c40 con 0x7f8f041538a0 2023-10-30T12:37:46.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.868+0000 7f8eeb7fe640 1 --2- 172.21.15.12:0/3076790802 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8ef00664a0 0x7f8ef0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:46.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.868+0000 7f8eeb7fe640 1 -- 172.21.15.12:0/3076790802 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f8ef80844b0 con 0x7f8f041538a0 2023-10-30T12:37:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.868+0000 7f8f02d9d640 1 --2- 172.21.15.12:0/3076790802 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8ef00664a0 0x7f8ef0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.872+0000 7f8f02d9d640 1 --2- 172.21.15.12:0/3076790802 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8ef00664a0 0x7f8ef0068960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4000f70 tx=0x7f8ef4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:46.872+0000 7f8eeb7fe640 1 -- 172.21.15.12:0/3076790802 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8ef8051800 con 0x7f8f041538a0 2023-10-30T12:37:47.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:47.060+0000 7f8f090a9640 1 -- 172.21.15.12:0/3076790802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f040b5090 con 0x7f8f041538a0 2023-10-30T12:37:47.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:47.064+0000 7f8eeb7fe640 1 -- 172.21.15.12:0/3076790802 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6526 (secure 0 0 0) 0x7f8ef8054e20 con 0x7f8f041538a0 2023-10-30T12:37:47.066 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:37:47.066 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:37:21.372155+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":9,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:37:47.066 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2023-10-30T12:37:47.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:47.064+0000 7f8f090a9640 1 -- 172.21.15.12:0/3076790802 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8ef00664a0 msgr2=0x7f8ef0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:47.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:47.064+0000 7f8f090a9640 1 --2- 172.21.15.12:0/3076790802 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8ef00664a0 0x7f8ef0068960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4000f70 tx=0x7f8ef4009040 comp rx=0 tx=0).stop 2023-10-30T12:37:47.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:47.064+0000 7f8f090a9640 1 -- 172.21.15.12:0/3076790802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f041538a0 msgr2=0x7f8f0414f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:47.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:47.064+0000 7f8f090a9640 1 --2- 172.21.15.12:0/3076790802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f041538a0 0x7f8f0414f0d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800d5f0 tx=0x7f8ef800dab0 comp rx=0 tx=0).stop 2023-10-30T12:37:47.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:47.064+0000 7f8f090a9640 1 -- 172.21.15.12:0/3076790802 shutdown_connections 2023-10-30T12:37:47.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:47.064+0000 7f8f090a9640 1 --2- 172.21.15.12:0/3076790802 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8ef00664a0 0x7f8ef0068960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:47.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:47.064+0000 7f8f090a9640 1 --2- 172.21.15.12:0/3076790802 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f0415d6f0 0x7f8f041f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:47.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:47.064+0000 7f8f090a9640 1 --2- 172.21.15.12:0/3076790802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f041538a0 0x7f8f0414f0d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:47.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:47.064+0000 7f8f090a9640 1 --2- 172.21.15.12:0/3076790802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f040cf160 0x7f8f0414eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:47.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:47.064+0000 7f8f090a9640 1 -- 172.21.15.12:0/3076790802 >> 172.21.15.12:0/3076790802 conn(0x7f8f0406bfe0 msgr2=0x7f8f0415b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:47.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:47.064+0000 7f8f090a9640 1 -- 172.21.15.12:0/3076790802 shutdown_connections 2023-10-30T12:37:47.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:47.064+0000 7f8f090a9640 1 -- 172.21.15.12:0/3076790802 wait complete. 2023-10-30T12:37:47.087 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:37:49.088 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:37:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.368+0000 7faf97d94640 1 Processor -- start 2023-10-30T12:37:49.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.368+0000 7faf97d94640 1 -- start start 2023-10-30T12:37:49.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.368+0000 7faf97d94640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf90153c30 0x7faf9006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:49.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.368+0000 7faf97d94640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf900cc0b0 0x7faf900cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:49.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.368+0000 7faf97d94640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9006ade0 con 0x7faf900b1ea0 2023-10-30T12:37:49.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.368+0000 7faf97d94640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf901596e0 con 0x7faf900cc0b0 2023-10-30T12:37:49.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.368+0000 7faf97d94640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf901598f0 con 0x7faf90153c30 2023-10-30T12:37:49.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.368+0000 7faf96331640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf900cc0b0 0x7faf900cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:49.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.368+0000 7faf96331640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf900cc0b0 0x7faf900cabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59262/0 (socket says 172.21.15.12:59262) 2023-10-30T12:37:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.368+0000 7faf96331640 1 -- 172.21.15.12:0/352194406 learned_addr learned my addr 172.21.15.12:0/352194406 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.368+0000 7faf95b30640 1 --2- 172.21.15.12:0/352194406 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf90153c30 0x7faf9006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.368+0000 7faf96331640 1 -- 172.21.15.12:0/352194406 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf90153c30 msgr2=0x7faf9006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:37:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.368+0000 7faf96331640 1 --2- 172.21.15.12:0/352194406 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf90153c30 0x7faf9006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.368+0000 7faf96331640 1 -- 172.21.15.12:0/352194406 >> v1:172.21.15.12:6789/0 conn(0x7faf900b1ea0 legacy=0x7faf900cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.368+0000 7faf96331640 1 -- 172.21.15.12:0/352194406 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf90159b90 con 0x7faf900cc0b0 2023-10-30T12:37:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.372+0000 7faf96331640 1 --2- 172.21.15.12:0/352194406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf900cc0b0 0x7faf900cabd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faf88009e50 tx=0x7faf8801f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ef35b60da571e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:49.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.372+0000 7faf94b2e640 1 -- 172.21.15.12:0/352194406 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf880af400 con 0x7faf900cc0b0 2023-10-30T12:37:49.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.372+0000 7faf94b2e640 1 -- 172.21.15.12:0/352194406 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf880ab050 con 0x7faf900cc0b0 2023-10-30T12:37:49.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.372+0000 7faf94b2e640 1 -- 172.21.15.12:0/352194406 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf880af780 con 0x7faf900cc0b0 2023-10-30T12:37:49.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.372+0000 7faf97d94640 1 -- 172.21.15.12:0/352194406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf900cc0b0 msgr2=0x7faf900cabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:49.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.372+0000 7faf97d94640 1 --2- 172.21.15.12:0/352194406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf900cc0b0 0x7faf900cabd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faf88009e50 tx=0x7faf8801f1f0 comp rx=0 tx=0).stop 2023-10-30T12:37:49.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.372+0000 7faf97d94640 1 -- 172.21.15.12:0/352194406 shutdown_connections 2023-10-30T12:37:49.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.372+0000 7faf97d94640 1 --2- 172.21.15.12:0/352194406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf900cc0b0 0x7faf900cabd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:49.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.372+0000 7faf97d94640 1 --2- 172.21.15.12:0/352194406 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf90153c30 0x7faf9006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:49.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.372+0000 7faf97d94640 1 -- 172.21.15.12:0/352194406 >> 172.21.15.12:0/352194406 conn(0x7faf9006bf40 msgr2=0x7faf9006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:49.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.372+0000 7faf97d94640 1 -- 172.21.15.12:0/352194406 shutdown_connections 2023-10-30T12:37:49.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.372+0000 7faf97d94640 1 -- 172.21.15.12:0/352194406 wait complete. 2023-10-30T12:37:49.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf97d94640 1 Processor -- start 2023-10-30T12:37:49.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf97d94640 1 -- start start 2023-10-30T12:37:49.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf97d94640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf900b1ea0 0x7faf900ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:49.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf97d94640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf900cc0b0 0x7faf9015ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:49.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf97d94640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf90153c30 0x7faf90143630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:49.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf97d94640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf900c6590 con 0x7faf90153c30 2023-10-30T12:37:49.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf97d94640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf901f2790 con 0x7faf900cc0b0 2023-10-30T12:37:49.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf97d94640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf900caf80 con 0x7faf900b1ea0 2023-10-30T12:37:49.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf96331640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf90153c30 0x7faf90143630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:49.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf96331640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf90153c30 0x7faf90143630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59264/0 (socket says 172.21.15.12:59264) 2023-10-30T12:37:49.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf96331640 1 -- 172.21.15.12:0/1697941950 learned_addr learned my addr 172.21.15.12:0/1697941950 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:49.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf95b30640 1 --2- 172.21.15.12:0/1697941950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf900b1ea0 0x7faf900ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:49.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf9532f640 1 --2- 172.21.15.12:0/1697941950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf900cc0b0 0x7faf9015ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:49.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf96331640 1 -- 172.21.15.12:0/1697941950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf900b1ea0 msgr2=0x7faf900ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:49.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf96331640 1 --2- 172.21.15.12:0/1697941950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf900b1ea0 0x7faf900ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:49.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf96331640 1 -- 172.21.15.12:0/1697941950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf900cc0b0 msgr2=0x7faf9015ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:49.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf96331640 1 --2- 172.21.15.12:0/1697941950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf900cc0b0 0x7faf9015ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:49.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf96331640 1 -- 172.21.15.12:0/1697941950 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf88009b30 con 0x7faf90153c30 2023-10-30T12:37:49.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf9532f640 1 --2- 172.21.15.12:0/1697941950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf900cc0b0 0x7faf9015ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:37:49.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf96331640 1 --2- 172.21.15.12:0/1697941950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf90153c30 0x7faf90143630 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faf8801ce20 tx=0x7faf8801bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:49.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf867fc640 1 -- 172.21.15.12:0/1697941950 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf880afd30 con 0x7faf90153c30 2023-10-30T12:37:49.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf867fc640 1 -- 172.21.15.12:0/1697941950 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf880afe90 con 0x7faf90153c30 2023-10-30T12:37:49.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf867fc640 1 -- 172.21.15.12:0/1697941950 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf880ba660 con 0x7faf90153c30 2023-10-30T12:37:49.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf97d94640 1 -- 172.21.15.12:0/1697941950 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf900c7380 con 0x7faf90153c30 2023-10-30T12:37:49.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.376+0000 7faf97d94640 1 -- 172.21.15.12:0/1697941950 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf901f2140 con 0x7faf90153c30 2023-10-30T12:37:49.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.380+0000 7faf97d94640 1 -- 172.21.15.12:0/1697941950 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf901f2940 con 0x7faf90153c30 2023-10-30T12:37:49.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.380+0000 7faf867fc640 1 -- 172.21.15.12:0/1697941950 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faf880b6040 con 0x7faf90153c30 2023-10-30T12:37:49.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.380+0000 7faf867fc640 1 --2- 172.21.15.12:0/1697941950 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faf740664d0 0x7faf74068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:49.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.380+0000 7faf867fc640 1 -- 172.21.15.12:0/1697941950 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5853+0+0 (secure 0 0 0) 0x7faf88121ff0 con 0x7faf90153c30 2023-10-30T12:37:49.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.380+0000 7faf95b30640 1 --2- 172.21.15.12:0/1697941950 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faf740664d0 0x7faf74068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:49.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.380+0000 7faf95b30640 1 --2- 172.21.15.12:0/1697941950 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faf740664d0 0x7faf74068990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009ac0 tx=0x7faf8c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:49.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.384+0000 7faf867fc640 1 -- 172.21.15.12:0/1697941950 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faf901f2940 con 0x7faf90153c30 2023-10-30T12:37:49.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.584+0000 7faf97d94640 1 -- 172.21.15.12:0/1697941950 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf901f2940 con 0x7faf90153c30 2023-10-30T12:37:49.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.584+0000 7faf867fc640 1 -- 172.21.15.12:0/1697941950 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6526 (secure 0 0 0) 0x7faf901f2940 con 0x7faf90153c30 2023-10-30T12:37:49.616 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:37:49.616 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:37:21.372155+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":9,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:37:49.617 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2023-10-30T12:37:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.584+0000 7faf97d94640 1 -- 172.21.15.12:0/1697941950 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faf740664d0 msgr2=0x7faf74068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.584+0000 7faf97d94640 1 --2- 172.21.15.12:0/1697941950 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faf740664d0 0x7faf74068990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009ac0 tx=0x7faf8c009190 comp rx=0 tx=0).stop 2023-10-30T12:37:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.584+0000 7faf97d94640 1 -- 172.21.15.12:0/1697941950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf90153c30 msgr2=0x7faf90143630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.584+0000 7faf97d94640 1 --2- 172.21.15.12:0/1697941950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf90153c30 0x7faf90143630 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faf8801ce20 tx=0x7faf8801bf90 comp rx=0 tx=0).stop 2023-10-30T12:37:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.584+0000 7faf97d94640 1 -- 172.21.15.12:0/1697941950 shutdown_connections 2023-10-30T12:37:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.584+0000 7faf97d94640 1 --2- 172.21.15.12:0/1697941950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf90153c30 0x7faf90143630 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.584+0000 7faf97d94640 1 --2- 172.21.15.12:0/1697941950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf900cc0b0 0x7faf9015ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.584+0000 7faf97d94640 1 --2- 172.21.15.12:0/1697941950 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faf740664d0 0x7faf74068990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.584+0000 7faf97d94640 1 --2- 172.21.15.12:0/1697941950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf900b1ea0 0x7faf900ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.584+0000 7faf97d94640 1 -- 172.21.15.12:0/1697941950 >> 172.21.15.12:0/1697941950 conn(0x7faf9006bf40 msgr2=0x7faf90073480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:49.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.584+0000 7faf97d94640 1 -- 172.21.15.12:0/1697941950 shutdown_connections 2023-10-30T12:37:49.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:49.588+0000 7faf97d94640 1 -- 172.21.15.12:0/1697941950 wait complete. 2023-10-30T12:37:49.620 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:37:51.623 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:37:51.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.776+0000 7f2fb1c47640 1 Processor -- start 2023-10-30T12:37:51.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.776+0000 7f2fb1c47640 1 -- start start 2023-10-30T12:37:51.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.776+0000 7f2fb1c47640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2fac15cd50 0x7f2fac0c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:51.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.776+0000 7f2fb1c47640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fac15a6c0 con 0x7f2fac0cefa0 2023-10-30T12:37:51.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.776+0000 7f2fb1c47640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fac15d300 con 0x7f2fac1538f0 2023-10-30T12:37:51.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.776+0000 7f2fb1c47640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fac15d5e0 con 0x7f2fac15cd50 2023-10-30T12:37:51.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.776+0000 7f2fabfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2fac15cd50 0x7f2fac0c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:51.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.776+0000 7f2fabfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2fac15cd50 0x7f2fac0c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56862/0 (socket says 172.21.15.12:56862) 2023-10-30T12:37:51.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.776+0000 7f2fabfff640 1 -- 172.21.15.12:0/517456772 learned_addr learned my addr 172.21.15.12:0/517456772 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:51.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.776+0000 7f2fabfff640 1 -- 172.21.15.12:0/517456772 >> v1:172.21.15.154:6789/0 conn(0x7f2fac1538f0 legacy=0x7f2fac159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:51.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.776+0000 7f2fabfff640 1 -- 172.21.15.12:0/517456772 >> v1:172.21.15.12:6789/0 conn(0x7f2fac0cefa0 legacy=0x7f2fac1640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:51.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.776+0000 7f2fabfff640 1 -- 172.21.15.12:0/517456772 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac15d8c0 con 0x7f2fac15cd50 2023-10-30T12:37:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.780+0000 7f2fabfff640 1 --2- 172.21.15.12:0/517456772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fac15cd50 0x7f2fac0c9f90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009870 tx=0x7f2f9801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e6398991ff476a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.780+0000 7f2fab7fe640 1 -- 172.21.15.12:0/517456772 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f980a8a30 con 0x7f2fac15cd50 2023-10-30T12:37:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.780+0000 7f2fab7fe640 1 -- 172.21.15.12:0/517456772 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f980a8b90 con 0x7f2fac15cd50 2023-10-30T12:37:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.780+0000 7f2fab7fe640 1 -- 172.21.15.12:0/517456772 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f980b0440 con 0x7f2fac15cd50 2023-10-30T12:37:51.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.780+0000 7f2fb1c47640 1 -- 172.21.15.12:0/517456772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fac15cd50 msgr2=0x7f2fac0c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:51.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.780+0000 7f2fb1c47640 1 --2- 172.21.15.12:0/517456772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fac15cd50 0x7f2fac0c9f90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009870 tx=0x7f2f9801ed30 comp rx=0 tx=0).stop 2023-10-30T12:37:51.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.780+0000 7f2fb1c47640 1 -- 172.21.15.12:0/517456772 shutdown_connections 2023-10-30T12:37:51.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.780+0000 7f2fb1c47640 1 --2- 172.21.15.12:0/517456772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fac15cd50 0x7f2fac0c9f90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:51.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.780+0000 7f2fb1c47640 1 -- 172.21.15.12:0/517456772 >> 172.21.15.12:0/517456772 conn(0x7f2fac06a640 msgr2=0x7f2fac065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:51.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.780+0000 7f2fb1c47640 1 -- 172.21.15.12:0/517456772 shutdown_connections 2023-10-30T12:37:51.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.780+0000 7f2fb1c47640 1 -- 172.21.15.12:0/517456772 wait complete. 2023-10-30T12:37:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fb1c47640 1 Processor -- start 2023-10-30T12:37:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fb1c47640 1 -- start start 2023-10-30T12:37:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fb1c47640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fac0cefa0 0x7f2fac0b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fb1c47640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fac1538f0 0x7f2fac0c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fb1c47640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fac15cd50 0x7f2fac145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fb1c47640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac14fba0 con 0x7f2fac1538f0 2023-10-30T12:37:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fb1c47640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac152a30 con 0x7f2fac0cefa0 2023-10-30T12:37:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fb1c47640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac0b3290 con 0x7f2fac15cd50 2023-10-30T12:37:51.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fac1538f0 0x7f2fac0c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:51.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fac1538f0 0x7f2fac0c4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56878/0 (socket says 172.21.15.12:56878) 2023-10-30T12:37:51.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fabfff640 1 -- 172.21.15.12:0/1507652265 learned_addr learned my addr 172.21.15.12:0/1507652265 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:51.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fb0c45640 1 --2- 172.21.15.12:0/1507652265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fac0cefa0 0x7f2fac0b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:51.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fb1446640 1 --2- 172.21.15.12:0/1507652265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fac15cd50 0x7f2fac145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fabfff640 1 -- 172.21.15.12:0/1507652265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fac15cd50 msgr2=0x7f2fac145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fabfff640 1 --2- 172.21.15.12:0/1507652265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fac15cd50 0x7f2fac145bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fabfff640 1 -- 172.21.15.12:0/1507652265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fac0cefa0 msgr2=0x7f2fac0b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fabfff640 1 --2- 172.21.15.12:0/1507652265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fac0cefa0 0x7f2fac0b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:51.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fabfff640 1 -- 172.21.15.12:0/1507652265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f980094d0 con 0x7f2fac1538f0 2023-10-30T12:37:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fabfff640 1 --2- 172.21.15.12:0/1507652265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fac1538f0 0x7f2fac0c4110 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2f980b3040 tx=0x7f2f9801bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fa97fa640 1 -- 172.21.15.12:0/1507652265 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f980be070 con 0x7f2fac1538f0 2023-10-30T12:37:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fa97fa640 1 -- 172.21.15.12:0/1507652265 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f980ba400 con 0x7f2fac1538f0 2023-10-30T12:37:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fb1c47640 1 -- 172.21.15.12:0/1507652265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fac0b48c0 con 0x7f2fac1538f0 2023-10-30T12:37:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fa97fa640 1 -- 172.21.15.12:0/1507652265 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f980ba7b0 con 0x7f2fac1538f0 2023-10-30T12:37:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fb1c47640 1 -- 172.21.15.12:0/1507652265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fac1f3160 con 0x7f2fac1538f0 2023-10-30T12:37:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.784+0000 7f2fb1c47640 1 -- 172.21.15.12:0/1507652265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fac04e170 con 0x7f2fac1538f0 2023-10-30T12:37:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.788+0000 7f2fa97fa640 1 -- 172.21.15.12:0/1507652265 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2f980ba910 con 0x7f2fac1538f0 2023-10-30T12:37:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.788+0000 7f2fa97fa640 1 --2- 172.21.15.12:0/1507652265 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f900664a0 0x7f2f90068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.788+0000 7f2fa97fa640 1 -- 172.21.15.12:0/1507652265 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2f98121d20 con 0x7f2fac1538f0 2023-10-30T12:37:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.788+0000 7f2fb0c45640 1 --2- 172.21.15.12:0/1507652265 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f900664a0 0x7f2f90068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:51.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.788+0000 7f2fb0c45640 1 --2- 172.21.15.12:0/1507652265 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f900664a0 0x7f2f90068960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0000f70 tx=0x7f2fa0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:51.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.792+0000 7f2fa97fa640 1 -- 172.21.15.12:0/1507652265 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2f980ef100 con 0x7f2fac1538f0 2023-10-30T12:37:51.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.980+0000 7f2fb1c47640 1 -- 172.21.15.12:0/1507652265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fac14fce0 con 0x7f2fac1538f0 2023-10-30T12:37:51.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.980+0000 7f2fa97fa640 1 -- 172.21.15.12:0/1507652265 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6526 (secure 0 0 0) 0x7f2f980f2720 con 0x7f2fac1538f0 2023-10-30T12:37:51.984 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:37:51.985 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:37:21.372155+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":9,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:37:51.985 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2023-10-30T12:37:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.984+0000 7f2fb1c47640 1 -- 172.21.15.12:0/1507652265 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f900664a0 msgr2=0x7f2f90068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.984+0000 7f2fb1c47640 1 --2- 172.21.15.12:0/1507652265 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f900664a0 0x7f2f90068960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0000f70 tx=0x7f2fa0009040 comp rx=0 tx=0).stop 2023-10-30T12:37:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.984+0000 7f2fb1c47640 1 -- 172.21.15.12:0/1507652265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fac1538f0 msgr2=0x7f2fac0c4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.984+0000 7f2fb1c47640 1 --2- 172.21.15.12:0/1507652265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fac1538f0 0x7f2fac0c4110 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2f980b3040 tx=0x7f2f9801bad0 comp rx=0 tx=0).stop 2023-10-30T12:37:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.984+0000 7f2fb1c47640 1 -- 172.21.15.12:0/1507652265 shutdown_connections 2023-10-30T12:37:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.984+0000 7f2fb1c47640 1 --2- 172.21.15.12:0/1507652265 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f900664a0 0x7f2f90068960 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.984+0000 7f2fb1c47640 1 --2- 172.21.15.12:0/1507652265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fac15cd50 0x7f2fac145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.984+0000 7f2fb1c47640 1 --2- 172.21.15.12:0/1507652265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fac1538f0 0x7f2fac0c4110 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.984+0000 7f2fb1c47640 1 --2- 172.21.15.12:0/1507652265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fac0cefa0 0x7f2fac0b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.984+0000 7f2fb1c47640 1 -- 172.21.15.12:0/1507652265 >> 172.21.15.12:0/1507652265 conn(0x7f2fac06a640 msgr2=0x7f2fac0c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.984+0000 7f2fb1c47640 1 -- 172.21.15.12:0/1507652265 shutdown_connections 2023-10-30T12:37:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:51.984+0000 7f2fb1c47640 1 -- 172.21.15.12:0/1507652265 wait complete. 2023-10-30T12:37:52.006 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:37:54.007 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:37:54.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.160+0000 7f98cc854640 1 Processor -- start 2023-10-30T12:37:54.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.160+0000 7f98cc854640 1 -- start start 2023-10-30T12:37:54.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.160+0000 7f98cc854640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f98c80c93d0 0x7f98c806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:54.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.160+0000 7f98cc854640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c806afa0 con 0x7f98c806e470 2023-10-30T12:37:54.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.160+0000 7f98cc854640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c8154150 con 0x7f98c8153d70 2023-10-30T12:37:54.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.160+0000 7f98cc854640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c806d780 con 0x7f98c80c93d0 2023-10-30T12:37:54.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.160+0000 7f98bffff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f98c806e470 0x7f98c80c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:47544/0 (socket says 172.21.15.12:47544) 2023-10-30T12:37:54.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.160+0000 7f98bffff640 1 -- 172.21.15.12:0/559074073 learned_addr learned my addr 172.21.15.12:0/559074073 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:37:54.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.160+0000 7f98c6d9d640 1 --2- 172.21.15.12:0/559074073 >> v2:172.21.15.12:3300/0 conn(0x7f98c80c93d0 0x7f98c806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:54.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.164+0000 7f98c6d9d640 1 -- 172.21.15.12:0/559074073 >> v1:172.21.15.154:6789/0 conn(0x7f98c8153d70 legacy=0x7f98c80cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:54.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.164+0000 7f98c6d9d640 1 -- 172.21.15.12:0/559074073 >> v1:172.21.15.12:6789/0 conn(0x7f98c806e470 legacy=0x7f98c80c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:54.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.164+0000 7f98c6d9d640 1 -- 172.21.15.12:0/559074073 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c806e030 con 0x7f98c80c93d0 2023-10-30T12:37:54.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.164+0000 7f98c6d9d640 1 --2- 172.21.15.12:0/559074073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c80c93d0 0x7f98c806ca80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009d90 tx=0x7f98c001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f9055b09bcd842 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:54.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.164+0000 7f98c5d9b640 1 -- 172.21.15.12:0/559074073 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00a59e0 con 0x7f98c80c93d0 2023-10-30T12:37:54.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.164+0000 7f98c5d9b640 1 -- 172.21.15.12:0/559074073 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c00a5b40 con 0x7f98c80c93d0 2023-10-30T12:37:54.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.164+0000 7f98c5d9b640 1 -- 172.21.15.12:0/559074073 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00ad440 con 0x7f98c80c93d0 2023-10-30T12:37:54.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.164+0000 7f98cc854640 1 -- 172.21.15.12:0/559074073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c80c93d0 msgr2=0x7f98c806ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:54.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.164+0000 7f98cc854640 1 --2- 172.21.15.12:0/559074073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c80c93d0 0x7f98c806ca80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009d90 tx=0x7f98c001c920 comp rx=0 tx=0).stop 2023-10-30T12:37:54.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.164+0000 7f98cc854640 1 -- 172.21.15.12:0/559074073 shutdown_connections 2023-10-30T12:37:54.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.164+0000 7f98cc854640 1 --2- 172.21.15.12:0/559074073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c80c93d0 0x7f98c806ca80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:54.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.164+0000 7f98cc854640 1 -- 172.21.15.12:0/559074073 >> 172.21.15.12:0/559074073 conn(0x7f98c8071ea0 msgr2=0x7f98c8065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:54.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.164+0000 7f98cc854640 1 -- 172.21.15.12:0/559074073 shutdown_connections 2023-10-30T12:37:54.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.164+0000 7f98cc854640 1 -- 172.21.15.12:0/559074073 wait complete. 2023-10-30T12:37:54.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98cc854640 1 Processor -- start 2023-10-30T12:37:54.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98cc854640 1 -- start start 2023-10-30T12:37:54.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98cc854640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98c806e470 0x7f98c8149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:54.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98cc854640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c80c93d0 0x7f98c814d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:54.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98cc854640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98c8153d70 0x7f98c8156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:54.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98cc854640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c814bd80 con 0x7f98c80c93d0 2023-10-30T12:37:54.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98cc854640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c814b080 con 0x7f98c8153d70 2023-10-30T12:37:54.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98cc854640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c8157050 con 0x7f98c806e470 2023-10-30T12:37:54.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98bffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c80c93d0 0x7f98c814d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:54.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98bffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c80c93d0 0x7f98c814d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56890/0 (socket says 172.21.15.12:56890) 2023-10-30T12:37:54.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98bffff640 1 -- 172.21.15.12:0/2801283342 learned_addr learned my addr 172.21.15.12:0/2801283342 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:54.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98c6d9d640 1 --2- 172.21.15.12:0/2801283342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98c8153d70 0x7f98c8156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:54.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98c659c640 1 --2- 172.21.15.12:0/2801283342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98c806e470 0x7f98c8149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:54.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98bffff640 1 -- 172.21.15.12:0/2801283342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98c806e470 msgr2=0x7f98c8149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:54.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98bffff640 1 --2- 172.21.15.12:0/2801283342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98c806e470 0x7f98c8149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:54.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98bffff640 1 -- 172.21.15.12:0/2801283342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98c8153d70 msgr2=0x7f98c8156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:54.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98bffff640 1 --2- 172.21.15.12:0/2801283342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98c8153d70 0x7f98c8156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:54.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98bffff640 1 -- 172.21.15.12:0/2801283342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0009a70 con 0x7f98c80c93d0 2023-10-30T12:37:54.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98c659c640 1 --2- 172.21.15.12:0/2801283342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98c806e470 0x7f98c8149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:37:54.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.168+0000 7f98bffff640 1 --2- 172.21.15.12:0/2801283342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c80c93d0 0x7f98c814d350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f98b000c400 tx=0x7f98b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:54.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.172+0000 7f98beffd640 1 -- 172.21.15.12:0/2801283342 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0005cf0 con 0x7f98c80c93d0 2023-10-30T12:37:54.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.172+0000 7f98beffd640 1 -- 172.21.15.12:0/2801283342 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b0005e50 con 0x7f98c80c93d0 2023-10-30T12:37:54.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.172+0000 7f98beffd640 1 -- 172.21.15.12:0/2801283342 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b00072d0 con 0x7f98c80c93d0 2023-10-30T12:37:54.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.172+0000 7f98cc854640 1 -- 172.21.15.12:0/2801283342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c814ada0 con 0x7f98c80c93d0 2023-10-30T12:37:54.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.172+0000 7f98cc854640 1 -- 172.21.15.12:0/2801283342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c8152480 con 0x7f98c80c93d0 2023-10-30T12:37:54.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.172+0000 7f98cc854640 1 -- 172.21.15.12:0/2801283342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c81f2790 con 0x7f98c80c93d0 2023-10-30T12:37:54.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.172+0000 7f98beffd640 1 -- 172.21.15.12:0/2801283342 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f98b0007430 con 0x7f98c80c93d0 2023-10-30T12:37:54.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.172+0000 7f98beffd640 1 --2- 172.21.15.12:0/2801283342 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f98a80664a0 0x7f98a8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:54.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.172+0000 7f98beffd640 1 -- 172.21.15.12:0/2801283342 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f98b00834d0 con 0x7f98c80c93d0 2023-10-30T12:37:54.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.172+0000 7f98c659c640 1 --2- 172.21.15.12:0/2801283342 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f98a80664a0 0x7f98a8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:54.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.176+0000 7f98c659c640 1 --2- 172.21.15.12:0/2801283342 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f98a80664a0 0x7f98a8068960 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98b8000f70 tx=0x7f98b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:54.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.176+0000 7f98beffd640 1 -- 172.21.15.12:0/2801283342 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f98c81f2790 con 0x7f98c80c93d0 2023-10-30T12:37:54.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.368+0000 7f98cc854640 1 -- 172.21.15.12:0/2801283342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98c81f2790 con 0x7f98c80c93d0 2023-10-30T12:37:54.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.368+0000 7f98beffd640 1 -- 172.21.15.12:0/2801283342 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6526 (secure 0 0 0) 0x7f98c81f2790 con 0x7f98c80c93d0 2023-10-30T12:37:54.372 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:37:54.372 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:37:21.372155+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":9,"addr":"172.21.15.12:6837/3836803994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3836803994},{"type":"v1","addr":"172.21.15.12:6837","nonce":3836803994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:37:54.373 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2023-10-30T12:37:54.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.372+0000 7f98cc854640 1 -- 172.21.15.12:0/2801283342 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f98a80664a0 msgr2=0x7f98a8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:54.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.372+0000 7f98cc854640 1 --2- 172.21.15.12:0/2801283342 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f98a80664a0 0x7f98a8068960 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98b8000f70 tx=0x7f98b8009040 comp rx=0 tx=0).stop 2023-10-30T12:37:54.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.372+0000 7f98cc854640 1 -- 172.21.15.12:0/2801283342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c80c93d0 msgr2=0x7f98c814d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:54.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.372+0000 7f98cc854640 1 --2- 172.21.15.12:0/2801283342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c80c93d0 0x7f98c814d350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f98b000c400 tx=0x7f98b000c8c0 comp rx=0 tx=0).stop 2023-10-30T12:37:54.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.372+0000 7f98cc854640 1 -- 172.21.15.12:0/2801283342 shutdown_connections 2023-10-30T12:37:54.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.372+0000 7f98cc854640 1 --2- 172.21.15.12:0/2801283342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98c8153d70 0x7f98c8156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:54.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.372+0000 7f98cc854640 1 --2- 172.21.15.12:0/2801283342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c80c93d0 0x7f98c814d350 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:54.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.372+0000 7f98cc854640 1 --2- 172.21.15.12:0/2801283342 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f98a80664a0 0x7f98a8068960 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:54.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.372+0000 7f98cc854640 1 --2- 172.21.15.12:0/2801283342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98c806e470 0x7f98c8149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:54.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.372+0000 7f98cc854640 1 -- 172.21.15.12:0/2801283342 >> 172.21.15.12:0/2801283342 conn(0x7f98c8071ea0 msgr2=0x7f98c81426d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:54.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.372+0000 7f98cc854640 1 -- 172.21.15.12:0/2801283342 shutdown_connections 2023-10-30T12:37:54.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:54.372+0000 7f98cc854640 1 -- 172.21.15.12:0/2801283342 wait complete. 2023-10-30T12:37:54.394 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:37:56.395 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:37:56.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.548+0000 7f697a3d3640 1 Processor -- start 2023-10-30T12:37:56.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.548+0000 7f697a3d3640 1 -- start start 2023-10-30T12:37:56.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.548+0000 7f697a3d3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f69741538a0 0x7f697415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:56.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.548+0000 7f697a3d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69740ca0e0 0x7f697415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:56.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.548+0000 7f697a3d3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69740cb800 con 0x7f69740cf160 2023-10-30T12:37:56.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.548+0000 7f697a3d3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697415dc30 con 0x7f69741538a0 2023-10-30T12:37:56.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.548+0000 7f697a3d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69740cb330 con 0x7f69740ca0e0 2023-10-30T12:37:56.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.548+0000 7f6973fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f69741538a0 0x7f697415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:56.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.548+0000 7f6973fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f69741538a0 0x7f697415d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56896/0 (socket says 172.21.15.12:56896) 2023-10-30T12:37:56.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.548+0000 7f6973fff640 1 -- 172.21.15.12:0/59910858 learned_addr learned my addr 172.21.15.12:0/59910858 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:56.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.548+0000 7f69737fe640 1 --2- 172.21.15.12:0/59910858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69740ca0e0 0x7f697415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:56.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.548+0000 7f6973fff640 1 -- 172.21.15.12:0/59910858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69740ca0e0 msgr2=0x7f697415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:37:56.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.548+0000 7f6973fff640 1 --2- 172.21.15.12:0/59910858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69740ca0e0 0x7f697415a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:56.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.548+0000 7f6973fff640 1 -- 172.21.15.12:0/59910858 >> v1:172.21.15.154:6789/0 conn(0x7f69740cf160 legacy=0x7f69740cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:56.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.548+0000 7f6973fff640 1 -- 172.21.15.12:0/59910858 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69740cad70 con 0x7f69741538a0 2023-10-30T12:37:56.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.552+0000 7f6973fff640 1 --2- 172.21.15.12:0/59910858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69741538a0 0x7f697415d6f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6964009870 tx=0x7f696401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e153c4d70006d98 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:56.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.552+0000 7f6972ffd640 1 -- 172.21.15.12:0/59910858 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69640a8a30 con 0x7f69741538a0 2023-10-30T12:37:56.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.552+0000 7f6972ffd640 1 -- 172.21.15.12:0/59910858 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69640a8b90 con 0x7f69741538a0 2023-10-30T12:37:56.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.552+0000 7f6972ffd640 1 -- 172.21.15.12:0/59910858 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69640b0440 con 0x7f69741538a0 2023-10-30T12:37:56.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.552+0000 7f697a3d3640 1 -- 172.21.15.12:0/59910858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69741538a0 msgr2=0x7f697415d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:56.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.552+0000 7f697a3d3640 1 --2- 172.21.15.12:0/59910858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69741538a0 0x7f697415d6f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6964009870 tx=0x7f696401ed30 comp rx=0 tx=0).stop 2023-10-30T12:37:56.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.552+0000 7f697a3d3640 1 -- 172.21.15.12:0/59910858 shutdown_connections 2023-10-30T12:37:56.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.552+0000 7f697a3d3640 1 --2- 172.21.15.12:0/59910858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69740ca0e0 0x7f697415a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:56.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.552+0000 7f697a3d3640 1 --2- 172.21.15.12:0/59910858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69741538a0 0x7f697415d6f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:56.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.552+0000 7f697a3d3640 1 -- 172.21.15.12:0/59910858 >> 172.21.15.12:0/59910858 conn(0x7f697406bfe0 msgr2=0x7f697406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:56.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.552+0000 7f697a3d3640 1 -- 172.21.15.12:0/59910858 shutdown_connections 2023-10-30T12:37:56.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.552+0000 7f697a3d3640 1 -- 172.21.15.12:0/59910858 wait complete. 2023-10-30T12:37:56.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f697a3d3640 1 Processor -- start 2023-10-30T12:37:56.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f697a3d3640 1 -- start start 2023-10-30T12:37:56.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f697a3d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69740ca0e0 0x7f69740ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:56.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f697a3d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69740cf160 0x7f69741e3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:56.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f697a3d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69741538a0 0x7f69741e4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:56.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f697a3d3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6974159340 con 0x7f69740cf160 2023-10-30T12:37:56.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f697a3d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6974159620 con 0x7f69740ca0e0 2023-10-30T12:37:56.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f697a3d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69741e2b80 con 0x7f69741538a0 2023-10-30T12:37:56.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f69737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69740cf160 0x7f69741e3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:56.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f69737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69740cf160 0x7f69741e3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56900/0 (socket says 172.21.15.12:56900) 2023-10-30T12:37:56.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f69737fe640 1 -- 172.21.15.12:0/3448896677 learned_addr learned my addr 172.21.15.12:0/3448896677 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:56.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f6973fff640 1 --2- 172.21.15.12:0/3448896677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69740ca0e0 0x7f69740ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:56.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f6978970640 1 --2- 172.21.15.12:0/3448896677 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69741538a0 0x7f69741e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:56.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f69737fe640 1 -- 172.21.15.12:0/3448896677 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69741538a0 msgr2=0x7f69741e4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:56.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f69737fe640 1 --2- 172.21.15.12:0/3448896677 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69741538a0 0x7f69741e4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:56.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f69737fe640 1 -- 172.21.15.12:0/3448896677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69740ca0e0 msgr2=0x7f69740ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:56.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f69737fe640 1 --2- 172.21.15.12:0/3448896677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69740ca0e0 0x7f69740ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:56.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f69737fe640 1 -- 172.21.15.12:0/3448896677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69640094d0 con 0x7f69740cf160 2023-10-30T12:37:56.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f6978970640 1 --2- 172.21.15.12:0/3448896677 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69741538a0 0x7f69741e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:37:56.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f69737fe640 1 --2- 172.21.15.12:0/3448896677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69740cf160 0x7f69741e3e10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f696800d800 tx=0x7f696800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:56.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f6970ff9640 1 -- 172.21.15.12:0/3448896677 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696800b6c0 con 0x7f69740cf160 2023-10-30T12:37:56.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f6970ff9640 1 -- 172.21.15.12:0/3448896677 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696800b860 con 0x7f69740cf160 2023-10-30T12:37:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f697a3d3640 1 -- 172.21.15.12:0/3448896677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69741e2e60 con 0x7f69740cf160 2023-10-30T12:37:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f6970ff9640 1 -- 172.21.15.12:0/3448896677 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696800bc10 con 0x7f69740cf160 2023-10-30T12:37:56.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.556+0000 7f697a3d3640 1 -- 172.21.15.12:0/3448896677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69741e38f0 con 0x7f69740cf160 2023-10-30T12:37:56.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.560+0000 7f697a3d3640 1 -- 172.21.15.12:0/3448896677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69740b2a50 con 0x7f69740cf160 2023-10-30T12:37:56.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.560+0000 7f6970ff9640 1 -- 172.21.15.12:0/3448896677 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f696801e460 con 0x7f69740cf160 2023-10-30T12:37:56.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.560+0000 7f6970ff9640 1 --2- 172.21.15.12:0/3448896677 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f69600664a0 0x7f6960068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:56.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.560+0000 7f6970ff9640 1 -- 172.21.15.12:0/3448896677 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f6968084670 con 0x7f69740cf160 2023-10-30T12:37:56.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.560+0000 7f6973fff640 1 --2- 172.21.15.12:0/3448896677 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f69600664a0 0x7f6960068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:56.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.560+0000 7f6973fff640 1 --2- 172.21.15.12:0/3448896677 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f69600664a0 0x7f6960068960 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f69640b2000 tx=0x7f6964008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:56.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.564+0000 7f6970ff9640 1 -- 172.21.15.12:0/3448896677 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f69680518f0 con 0x7f69740cf160 2023-10-30T12:37:56.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.752+0000 7f697a3d3640 1 -- 172.21.15.12:0/3448896677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69741e3b10 con 0x7f69740cf160 2023-10-30T12:37:56.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.756+0000 7f6970ff9640 1 -- 172.21.15.12:0/3448896677 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5737 (secure 0 0 0) 0x7f6968054f10 con 0x7f69740cf160 2023-10-30T12:37:56.757 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:37:56.758 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:37:55.139912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:37:56.758 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 16 2023-10-30T12:37:56.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.756+0000 7f697a3d3640 1 -- 172.21.15.12:0/3448896677 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f69600664a0 msgr2=0x7f6960068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:56.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.756+0000 7f697a3d3640 1 --2- 172.21.15.12:0/3448896677 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f69600664a0 0x7f6960068960 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f69640b2000 tx=0x7f6964008040 comp rx=0 tx=0).stop 2023-10-30T12:37:56.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.756+0000 7f697a3d3640 1 -- 172.21.15.12:0/3448896677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69740cf160 msgr2=0x7f69741e3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:56.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.756+0000 7f697a3d3640 1 --2- 172.21.15.12:0/3448896677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69740cf160 0x7f69741e3e10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f696800d800 tx=0x7f696800dcc0 comp rx=0 tx=0).stop 2023-10-30T12:37:56.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.756+0000 7f697a3d3640 1 -- 172.21.15.12:0/3448896677 shutdown_connections 2023-10-30T12:37:56.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.756+0000 7f697a3d3640 1 --2- 172.21.15.12:0/3448896677 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f69600664a0 0x7f6960068960 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:56.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.756+0000 7f697a3d3640 1 --2- 172.21.15.12:0/3448896677 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f69741538a0 0x7f69741e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:56.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.756+0000 7f697a3d3640 1 --2- 172.21.15.12:0/3448896677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69740cf160 0x7f69741e3e10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:56.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.756+0000 7f697a3d3640 1 --2- 172.21.15.12:0/3448896677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69740ca0e0 0x7f69740ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:56.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.756+0000 7f697a3d3640 1 -- 172.21.15.12:0/3448896677 >> 172.21.15.12:0/3448896677 conn(0x7f697406bfe0 msgr2=0x7f6974065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:56.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.756+0000 7f697a3d3640 1 -- 172.21.15.12:0/3448896677 shutdown_connections 2023-10-30T12:37:56.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.756+0000 7f697a3d3640 1 -- 172.21.15.12:0/3448896677 wait complete. 2023-10-30T12:37:56.778 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2023-10-30T12:37:56.778 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:37:56.779 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:37:56.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.920+0000 7f8f69e54640 1 Processor -- start 2023-10-30T12:37:56.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.920+0000 7f8f69e54640 1 -- start start 2023-10-30T12:37:56.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.920+0000 7f8f69e54640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8f640c6b80 0x7f8f640bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:56.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.920+0000 7f8f69e54640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f640c0a50 con 0x7f8f6413f4a0 2023-10-30T12:37:56.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.920+0000 7f8f69e54640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6406f6e0 con 0x7f8f640c6f60 2023-10-30T12:37:56.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.920+0000 7f8f69e54640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6406f9c0 con 0x7f8f640c6b80 2023-10-30T12:37:56.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.920+0000 7f8f63fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8f6413f4a0 0x7f8f641566a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:47550/0 (socket says 172.21.15.12:47550) 2023-10-30T12:37:56.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.920+0000 7f8f63fff640 1 -- 172.21.15.12:0/2879409668 learned_addr learned my addr 172.21.15.12:0/2879409668 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:37:56.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.920+0000 7f8f637fe640 1 --2- 172.21.15.12:0/2879409668 >> v2:172.21.15.154:3300/0 conn(0x7f8f640c6b80 0x7f8f640bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:56.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.924+0000 7f8f627fc640 1 -- 172.21.15.12:0/2879409668 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589658011 0 0) 0x7f8f640c0a50 con 0x7f8f6413f4a0 2023-10-30T12:37:56.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.924+0000 7f8f627fc640 1 -- 172.21.15.12:0/2879409668 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f4c005740 con 0x7f8f6413f4a0 2023-10-30T12:37:56.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.924+0000 7f8f637fe640 1 -- 172.21.15.12:0/2879409668 >> v1:172.21.15.154:6789/0 conn(0x7f8f640c6f60 legacy=0x7f8f6413ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:56.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.924+0000 7f8f637fe640 1 -- 172.21.15.12:0/2879409668 >> v1:172.21.15.12:6789/0 conn(0x7f8f6413f4a0 legacy=0x7f8f641566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:56.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.924+0000 7f8f627fc640 1 -- 172.21.15.12:0/2879409668 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 247562248 0 0) 0x7f8f4c005740 con 0x7f8f6413f4a0 2023-10-30T12:37:56.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.924+0000 7f8f637fe640 1 -- 172.21.15.12:0/2879409668 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6406fca0 con 0x7f8f640c6b80 2023-10-30T12:37:56.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.924+0000 7f8f637fe640 1 --2- 172.21.15.12:0/2879409668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f640c6b80 0x7f8f640bfec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8f540097c0 tx=0x7f8f5401c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfd61bdf842a80b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:56.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.924+0000 7f8f627fc640 1 -- 172.21.15.12:0/2879409668 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c001a00 con 0x7f8f640c6b80 2023-10-30T12:37:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.924+0000 7f8f627fc640 1 -- 172.21.15.12:0/2879409668 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f540a7050 con 0x7f8f640c6b80 2023-10-30T12:37:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.924+0000 7f8f627fc640 1 -- 172.21.15.12:0/2879409668 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f540ab600 con 0x7f8f640c6b80 2023-10-30T12:37:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f69e54640 1 -- 172.21.15.12:0/2879409668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f640c6b80 msgr2=0x7f8f640bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:56.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f69e54640 1 --2- 172.21.15.12:0/2879409668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f640c6b80 0x7f8f640bfec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8f540097c0 tx=0x7f8f5401c3a0 comp rx=0 tx=0).stop 2023-10-30T12:37:56.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f69e54640 1 -- 172.21.15.12:0/2879409668 shutdown_connections 2023-10-30T12:37:56.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f69e54640 1 --2- 172.21.15.12:0/2879409668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f640c6b80 0x7f8f640bfec0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:56.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f69e54640 1 -- 172.21.15.12:0/2879409668 >> 172.21.15.12:0/2879409668 conn(0x7f8f640c7910 msgr2=0x7f8f640c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:56.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f69e54640 1 -- 172.21.15.12:0/2879409668 shutdown_connections 2023-10-30T12:37:56.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f69e54640 1 -- 172.21.15.12:0/2879409668 wait complete. 2023-10-30T12:37:56.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f69e54640 1 Processor -- start 2023-10-30T12:37:56.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f69e54640 1 -- start start 2023-10-30T12:37:56.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f69e54640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f640c6b80 0x7f8f6413aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:56.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f69e54640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f640c6f60 0x7f8f641551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:56.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f69e54640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6413f4a0 0x7f8f64155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:56.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f69e54640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6413aaf0 con 0x7f8f6413f4a0 2023-10-30T12:37:56.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f69e54640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f64154d40 con 0x7f8f640c6f60 2023-10-30T12:37:56.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f69e54640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f641f2cb0 con 0x7f8f640c6b80 2023-10-30T12:37:56.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f63fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6413f4a0 0x7f8f64155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:56.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f63fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6413f4a0 0x7f8f64155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56908/0 (socket says 172.21.15.12:56908) 2023-10-30T12:37:56.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f63fff640 1 -- 172.21.15.12:0/3455247756 learned_addr learned my addr 172.21.15.12:0/3455247756 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:56.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f637fe640 1 --2- 172.21.15.12:0/3455247756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f640c6b80 0x7f8f6413aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:56.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f62ffd640 1 --2- 172.21.15.12:0/3455247756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f640c6f60 0x7f8f641551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:56.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f63fff640 1 -- 172.21.15.12:0/3455247756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f640c6b80 msgr2=0x7f8f6413aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:56.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f63fff640 1 --2- 172.21.15.12:0/3455247756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f640c6b80 0x7f8f6413aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:56.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f63fff640 1 -- 172.21.15.12:0/3455247756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f640c6f60 msgr2=0x7f8f641551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:56.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f63fff640 1 --2- 172.21.15.12:0/3455247756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f640c6f60 0x7f8f641551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:56.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.928+0000 7f8f63fff640 1 -- 172.21.15.12:0/3455247756 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f54009420 con 0x7f8f6413f4a0 2023-10-30T12:37:56.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.932+0000 7f8f63fff640 1 --2- 172.21.15.12:0/3455247756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6413f4a0 0x7f8f64155710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00cc30 tx=0x7f8f5c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:56.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.932+0000 7f8f4bfff640 1 -- 172.21.15.12:0/3455247756 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c00ce40 con 0x7f8f6413f4a0 2023-10-30T12:37:56.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.932+0000 7f8f4bfff640 1 -- 172.21.15.12:0/3455247756 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5c012400 con 0x7f8f6413f4a0 2023-10-30T12:37:56.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.932+0000 7f8f4bfff640 1 -- 172.21.15.12:0/3455247756 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c0127b0 con 0x7f8f6413f4a0 2023-10-30T12:37:56.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.932+0000 7f8f69e54640 1 -- 172.21.15.12:0/3455247756 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6413a270 con 0x7f8f6413f4a0 2023-10-30T12:37:56.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.932+0000 7f8f69e54640 1 -- 172.21.15.12:0/3455247756 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6413a810 con 0x7f8f6413f4a0 2023-10-30T12:37:56.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.932+0000 7f8f69e54640 1 -- 172.21.15.12:0/3455247756 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6404e170 con 0x7f8f6413f4a0 2023-10-30T12:37:56.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.932+0000 7f8f4bfff640 1 -- 172.21.15.12:0/3455247756 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8f5c012910 con 0x7f8f6413f4a0 2023-10-30T12:37:56.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.932+0000 7f8f4bfff640 1 --2- 172.21.15.12:0/3455247756 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f500664a0 0x7f8f50068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:56.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.932+0000 7f8f4bfff640 1 -- 172.21.15.12:0/3455247756 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f8f5c0845b0 con 0x7f8f6413f4a0 2023-10-30T12:37:56.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.932+0000 7f8f637fe640 1 --2- 172.21.15.12:0/3455247756 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f500664a0 0x7f8f50068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:56.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.936+0000 7f8f637fe640 1 --2- 172.21.15.12:0/3455247756 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f500664a0 0x7f8f50068960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f540ad000 tx=0x7f8f540a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:56.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:56.936+0000 7f8f4bfff640 1 -- 172.21.15.12:0/3455247756 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8f5c0519f0 con 0x7f8f6413f4a0 2023-10-30T12:37:57.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:57.128+0000 7f8f69e54640 1 -- 172.21.15.12:0/3455247756 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f64154e80 con 0x7f8f6413f4a0 2023-10-30T12:37:57.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:57.128+0000 7f8f4bfff640 1 -- 172.21.15.12:0/3455247756 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5737 (secure 0 0 0) 0x7f8f5c055010 con 0x7f8f6413f4a0 2023-10-30T12:37:57.131 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:37:57.131 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:37:55.139912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:37:57.132 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 16 2023-10-30T12:37:57.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:57.128+0000 7f8f69e54640 1 -- 172.21.15.12:0/3455247756 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f500664a0 msgr2=0x7f8f50068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:57.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:57.128+0000 7f8f69e54640 1 --2- 172.21.15.12:0/3455247756 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f500664a0 0x7f8f50068960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f540ad000 tx=0x7f8f540a6000 comp rx=0 tx=0).stop 2023-10-30T12:37:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:57.128+0000 7f8f69e54640 1 -- 172.21.15.12:0/3455247756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6413f4a0 msgr2=0x7f8f64155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:57.128+0000 7f8f69e54640 1 --2- 172.21.15.12:0/3455247756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6413f4a0 0x7f8f64155710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00cc30 tx=0x7f8f5c00a430 comp rx=0 tx=0).stop 2023-10-30T12:37:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:57.128+0000 7f8f69e54640 1 -- 172.21.15.12:0/3455247756 shutdown_connections 2023-10-30T12:37:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:57.128+0000 7f8f69e54640 1 --2- 172.21.15.12:0/3455247756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6413f4a0 0x7f8f64155710 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:57.128+0000 7f8f69e54640 1 --2- 172.21.15.12:0/3455247756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f640c6f60 0x7f8f641551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:57.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:57.128+0000 7f8f69e54640 1 --2- 172.21.15.12:0/3455247756 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f500664a0 0x7f8f50068960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:57.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:57.128+0000 7f8f69e54640 1 --2- 172.21.15.12:0/3455247756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f640c6b80 0x7f8f6413aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:57.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:57.128+0000 7f8f69e54640 1 -- 172.21.15.12:0/3455247756 >> 172.21.15.12:0/3455247756 conn(0x7f8f640c7910 msgr2=0x7f8f64153670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:57.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:57.128+0000 7f8f69e54640 1 -- 172.21.15.12:0/3455247756 shutdown_connections 2023-10-30T12:37:57.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:57.128+0000 7f8f69e54640 1 -- 172.21.15.12:0/3455247756 wait complete. 2023-10-30T12:37:57.152 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 16, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 16, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4262": { "addr": "172.21.15.12:6835/2676506149", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 2676506149, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 2676506149, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4262, "incarnation": 16, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:replay", "state_seq": 1 }, "gid_4282": { "addr": "172.21.15.154:6839/1517672481", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1517672481, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 1517672481, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4282, "incarnation": 10, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 7 }, "gid_4285": { "addr": "172.21.15.12:6836/291392212", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 291392212, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 291392212, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 4285, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 6 } }, "last_failure": 0, "last_failure_osd_epoch": 28, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:37:55.139912+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4285, "mds_1": 4282, "mds_2": 4262 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:37:59.153 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:37:59.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.304+0000 7fd1ec246640 1 Processor -- start 2023-10-30T12:37:59.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.304+0000 7fd1ec246640 1 -- start start 2023-10-30T12:37:59.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.304+0000 7fd1ec246640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd1e40c6bd0 0x7fd1e406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:59.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.304+0000 7fd1ec246640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1e4071110 0x7fd1e4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:59.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.304+0000 7fd1ec246640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e4066cf0 con 0x7fd1e406f7a0 2023-10-30T12:37:59.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.304+0000 7fd1ec246640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1e40700c0 con 0x7fd1e40c6bd0 2023-10-30T12:37:59.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.304+0000 7fd1ec246640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e40c70d0 con 0x7fd1e4071110 2023-10-30T12:37:59.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.304+0000 7fd1e9fe2640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd1e406f7a0 0x7fd1e4069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:47562/0 (socket says 172.21.15.12:47562) 2023-10-30T12:37:59.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.304+0000 7fd1e9fe2640 1 -- 172.21.15.12:0/3130739297 learned_addr learned my addr 172.21.15.12:0/3130739297 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:37:59.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.308+0000 7fd1ea7e3640 1 --2- 172.21.15.12:0/3130739297 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1e4071110 0x7fd1e4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:59.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.308+0000 7fd1e97e1640 1 --2- 172.21.15.12:0/3130739297 >> v2:172.21.15.154:3300/0 conn(0x7fd1e40c6bd0 0x7fd1e406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:59.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.308+0000 7fd1e8fe0640 1 -- 172.21.15.12:0/3130739297 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746484142 0 0) 0x7fd1e4066cf0 con 0x7fd1e406f7a0 2023-10-30T12:37:59.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.308+0000 7fd1e8fe0640 1 -- 172.21.15.12:0/3130739297 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1d0003530 con 0x7fd1e406f7a0 2023-10-30T12:37:59.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.308+0000 7fd1e8fe0640 1 -- 172.21.15.12:0/3130739297 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1151992725 0 0) 0x7fd1d0003530 con 0x7fd1e406f7a0 2023-10-30T12:37:59.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.308+0000 7fd1ea7e3640 1 -- 172.21.15.12:0/3130739297 >> v2:172.21.15.154:3300/0 conn(0x7fd1e40c6bd0 msgr2=0x7fd1e406fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:37:59.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.308+0000 7fd1ea7e3640 1 --2- 172.21.15.12:0/3130739297 >> v2:172.21.15.154:3300/0 conn(0x7fd1e40c6bd0 0x7fd1e406fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:59.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.308+0000 7fd1ea7e3640 1 -- 172.21.15.12:0/3130739297 >> v1:172.21.15.12:6789/0 conn(0x7fd1e406f7a0 legacy=0x7fd1e4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:59.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.308+0000 7fd1ea7e3640 1 -- 172.21.15.12:0/3130739297 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e40c7390 con 0x7fd1e4071110 2023-10-30T12:37:59.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.308+0000 7fd1e97e1640 1 --2- 172.21.15.12:0/3130739297 >> v2:172.21.15.154:3300/0 conn(0x7fd1e40c6bd0 0x7fd1e406fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:37:59.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.308+0000 7fd1ea7e3640 1 --2- 172.21.15.12:0/3130739297 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1e4071110 0x7fd1e4158110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009d90 tx=0x7fd1dc01f060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ddb64b1a5f3facc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:59.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1e8fe0640 1 -- 172.21.15.12:0/3130739297 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0001680 con 0x7fd1e4071110 2023-10-30T12:37:59.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1e8fe0640 1 -- 172.21.15.12:0/3130739297 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1dc0a8bb0 con 0x7fd1e4071110 2023-10-30T12:37:59.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1e8fe0640 1 -- 172.21.15.12:0/3130739297 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0b0440 con 0x7fd1e4071110 2023-10-30T12:37:59.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 -- 172.21.15.12:0/3130739297 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1e4071110 msgr2=0x7fd1e4158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:59.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 --2- 172.21.15.12:0/3130739297 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1e4071110 0x7fd1e4158110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009d90 tx=0x7fd1dc01f060 comp rx=0 tx=0).stop 2023-10-30T12:37:59.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 -- 172.21.15.12:0/3130739297 shutdown_connections 2023-10-30T12:37:59.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 --2- 172.21.15.12:0/3130739297 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1e4071110 0x7fd1e4158110 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:59.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 --2- 172.21.15.12:0/3130739297 >> v2:172.21.15.154:3300/0 conn(0x7fd1e40c6bd0 0x7fd1e406fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:59.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 -- 172.21.15.12:0/3130739297 >> 172.21.15.12:0/3130739297 conn(0x7fd1e40c7910 msgr2=0x7fd1e40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:59.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 -- 172.21.15.12:0/3130739297 shutdown_connections 2023-10-30T12:37:59.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 -- 172.21.15.12:0/3130739297 wait complete. 2023-10-30T12:37:59.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 Processor -- start 2023-10-30T12:37:59.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 -- start start 2023-10-30T12:37:59.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1e406f7a0 0x7fd1e41eca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:59.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1e4071110 0x7fd1e40ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:59.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1e40c6bd0 0x7fd1e414fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:59.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e40a9a00 con 0x7fd1e4071110 2023-10-30T12:37:59.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e41f03e0 con 0x7fd1e406f7a0 2023-10-30T12:37:59.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1ec246640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e41f06a0 con 0x7fd1e40c6bd0 2023-10-30T12:37:59.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1e97e1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1e4071110 0x7fd1e40ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:59.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1e97e1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1e4071110 0x7fd1e40ab6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56918/0 (socket says 172.21.15.12:56918) 2023-10-30T12:37:59.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.312+0000 7fd1e97e1640 1 -- 172.21.15.12:0/4142709790 learned_addr learned my addr 172.21.15.12:0/4142709790 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:37:59.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1ea7e3640 1 --2- 172.21.15.12:0/4142709790 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1e40c6bd0 0x7fd1e414fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:59.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1e9fe2640 1 --2- 172.21.15.12:0/4142709790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1e406f7a0 0x7fd1e41eca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:59.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1e97e1640 1 -- 172.21.15.12:0/4142709790 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1e40c6bd0 msgr2=0x7fd1e414fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:59.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1e97e1640 1 --2- 172.21.15.12:0/4142709790 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1e40c6bd0 0x7fd1e414fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:59.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1e97e1640 1 -- 172.21.15.12:0/4142709790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1e406f7a0 msgr2=0x7fd1e41eca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:59.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1e97e1640 1 --2- 172.21.15.12:0/4142709790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1e406f7a0 0x7fd1e41eca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:59.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1e97e1640 1 -- 172.21.15.12:0/4142709790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d4009420 con 0x7fd1e4071110 2023-10-30T12:37:59.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1e97e1640 1 --2- 172.21.15.12:0/4142709790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1e4071110 0x7fd1e40ab6e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400ebb0 tx=0x7fd1d400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:59.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1da7fc640 1 -- 172.21.15.12:0/4142709790 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400eeb0 con 0x7fd1e4071110 2023-10-30T12:37:59.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1da7fc640 1 -- 172.21.15.12:0/4142709790 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d400cc30 con 0x7fd1e4071110 2023-10-30T12:37:59.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1da7fc640 1 -- 172.21.15.12:0/4142709790 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4013470 con 0x7fd1e4071110 2023-10-30T12:37:59.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1ec246640 1 -- 172.21.15.12:0/4142709790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1dc009a70 con 0x7fd1e4071110 2023-10-30T12:37:59.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1ec246640 1 -- 172.21.15.12:0/4142709790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e41f0c60 con 0x7fd1e4071110 2023-10-30T12:37:59.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1ec246640 1 -- 172.21.15.12:0/4142709790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e40a9b40 con 0x7fd1e4071110 2023-10-30T12:37:59.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1da7fc640 1 -- 172.21.15.12:0/4142709790 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd1d40135d0 con 0x7fd1e4071110 2023-10-30T12:37:59.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.316+0000 7fd1da7fc640 1 --2- 172.21.15.12:0/4142709790 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1c80664a0 0x7fd1c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:37:59.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.320+0000 7fd1da7fc640 1 -- 172.21.15.12:0/4142709790 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fd1d4084210 con 0x7fd1e4071110 2023-10-30T12:37:59.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.320+0000 7fd1e9fe2640 1 --2- 172.21.15.12:0/4142709790 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1c80664a0 0x7fd1c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:37:59.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.320+0000 7fd1e9fe2640 1 --2- 172.21.15.12:0/4142709790 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1c80664a0 0x7fd1c8068960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0007a10 tx=0x7fd1e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:37:59.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.320+0000 7fd1da7fc640 1 -- 172.21.15.12:0/4142709790 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd1e40a9b40 con 0x7fd1e4071110 2023-10-30T12:37:59.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.512+0000 7fd1ec246640 1 -- 172.21.15.12:0/4142709790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1e40a9b40 con 0x7fd1e4071110 2023-10-30T12:37:59.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.512+0000 7fd1da7fc640 1 -- 172.21.15.12:0/4142709790 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5737 (secure 0 0 0) 0x7fd1e40a9b40 con 0x7fd1e4071110 2023-10-30T12:37:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:37:59.516 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:37:55.139912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:37:59.516 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 16 2023-10-30T12:37:59.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.512+0000 7fd1ec246640 1 -- 172.21.15.12:0/4142709790 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1c80664a0 msgr2=0x7fd1c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:59.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.512+0000 7fd1ec246640 1 --2- 172.21.15.12:0/4142709790 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1c80664a0 0x7fd1c8068960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0007a10 tx=0x7fd1e0009040 comp rx=0 tx=0).stop 2023-10-30T12:37:59.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.512+0000 7fd1ec246640 1 -- 172.21.15.12:0/4142709790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1e4071110 msgr2=0x7fd1e40ab6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:37:59.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.512+0000 7fd1ec246640 1 --2- 172.21.15.12:0/4142709790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1e4071110 0x7fd1e40ab6e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400ebb0 tx=0x7fd1d400c520 comp rx=0 tx=0).stop 2023-10-30T12:37:59.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.512+0000 7fd1ec246640 1 -- 172.21.15.12:0/4142709790 shutdown_connections 2023-10-30T12:37:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.512+0000 7fd1ec246640 1 --2- 172.21.15.12:0/4142709790 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1c80664a0 0x7fd1c8068960 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.512+0000 7fd1ec246640 1 --2- 172.21.15.12:0/4142709790 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1e40c6bd0 0x7fd1e414fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.512+0000 7fd1ec246640 1 --2- 172.21.15.12:0/4142709790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1e4071110 0x7fd1e40ab6e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.512+0000 7fd1ec246640 1 --2- 172.21.15.12:0/4142709790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1e406f7a0 0x7fd1e41eca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:37:59.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.512+0000 7fd1ec246640 1 -- 172.21.15.12:0/4142709790 >> 172.21.15.12:0/4142709790 conn(0x7fd1e40c7910 msgr2=0x7fd1e40c5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:37:59.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.516+0000 7fd1ec246640 1 -- 172.21.15.12:0/4142709790 shutdown_connections 2023-10-30T12:37:59.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:37:59.516+0000 7fd1ec246640 1 -- 172.21.15.12:0/4142709790 wait complete. 2023-10-30T12:37:59.536 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:38:01.537 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:38:01.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.688+0000 7fdc67247640 1 Processor -- start 2023-10-30T12:38:01.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.692+0000 7fdc67247640 1 -- start start 2023-10-30T12:38:01.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.692+0000 7fdc67247640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdc6015cd50 0x7fdc600c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:01.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.692+0000 7fdc67247640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6015a6c0 con 0x7fdc600cefa0 2023-10-30T12:38:01.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.692+0000 7fdc67247640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6015d300 con 0x7fdc601538f0 2023-10-30T12:38:01.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.692+0000 7fdc67247640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6015d5e0 con 0x7fdc6015cd50 2023-10-30T12:38:01.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.692+0000 7fdc66a46640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdc600cefa0 0x7fdc601640e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45010/0 (socket says 172.21.15.12:45010) 2023-10-30T12:38:01.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.692+0000 7fdc66a46640 1 -- 172.21.15.12:0/2263578198 learned_addr learned my addr 172.21.15.12:0/2263578198 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:38:01.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.692+0000 7fdc65a44640 1 --2- 172.21.15.12:0/2263578198 >> v2:172.21.15.154:3300/0 conn(0x7fdc6015cd50 0x7fdc600c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:01.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.692+0000 7fdc65243640 1 -- 172.21.15.12:0/2263578198 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022024590 0 0) 0x7fdc6015a6c0 con 0x7fdc600cefa0 2023-10-30T12:38:01.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.692+0000 7fdc65243640 1 -- 172.21.15.12:0/2263578198 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc4c003530 con 0x7fdc600cefa0 2023-10-30T12:38:01.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.692+0000 7fdc65a44640 1 -- 172.21.15.12:0/2263578198 >> v1:172.21.15.154:6789/0 conn(0x7fdc601538f0 legacy=0x7fdc60159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:01.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.692+0000 7fdc65a44640 1 -- 172.21.15.12:0/2263578198 >> v1:172.21.15.12:6789/0 conn(0x7fdc600cefa0 legacy=0x7fdc601640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:01.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.692+0000 7fdc65a44640 1 -- 172.21.15.12:0/2263578198 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6015d8c0 con 0x7fdc6015cd50 2023-10-30T12:38:01.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.696+0000 7fdc65a44640 1 --2- 172.21.15.12:0/2263578198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc6015cd50 0x7fdc600c9f90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdc500097c0 tx=0x7fdc5001ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba8c0ba79675231b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:01.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.696+0000 7fdc65243640 1 -- 172.21.15.12:0/2263578198 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500a8970 con 0x7fdc6015cd50 2023-10-30T12:38:01.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.696+0000 7fdc65243640 1 -- 172.21.15.12:0/2263578198 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc500a8ad0 con 0x7fdc6015cd50 2023-10-30T12:38:01.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.696+0000 7fdc65243640 1 -- 172.21.15.12:0/2263578198 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500a8e60 con 0x7fdc6015cd50 2023-10-30T12:38:01.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.696+0000 7fdc67247640 1 -- 172.21.15.12:0/2263578198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc6015cd50 msgr2=0x7fdc600c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:01.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.696+0000 7fdc67247640 1 --2- 172.21.15.12:0/2263578198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc6015cd50 0x7fdc600c9f90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdc500097c0 tx=0x7fdc5001ec70 comp rx=0 tx=0).stop 2023-10-30T12:38:01.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.696+0000 7fdc67247640 1 -- 172.21.15.12:0/2263578198 shutdown_connections 2023-10-30T12:38:01.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.696+0000 7fdc67247640 1 --2- 172.21.15.12:0/2263578198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc6015cd50 0x7fdc600c9f90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:01.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.696+0000 7fdc67247640 1 -- 172.21.15.12:0/2263578198 >> 172.21.15.12:0/2263578198 conn(0x7fdc6006a640 msgr2=0x7fdc60065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:01.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.696+0000 7fdc67247640 1 -- 172.21.15.12:0/2263578198 shutdown_connections 2023-10-30T12:38:01.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.696+0000 7fdc67247640 1 -- 172.21.15.12:0/2263578198 wait complete. 2023-10-30T12:38:01.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc67247640 1 Processor -- start 2023-10-30T12:38:01.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc67247640 1 -- start start 2023-10-30T12:38:01.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc67247640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc600cefa0 0x7fdc601f3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:01.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc67247640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc601538f0 0x7fdc601453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:01.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc67247640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc6015cd50 0x7fdc600cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:01.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc67247640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6015bbe0 con 0x7fdc601538f0 2023-10-30T12:38:01.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc67247640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6014db50 con 0x7fdc600cefa0 2023-10-30T12:38:01.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc67247640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6014e080 con 0x7fdc6015cd50 2023-10-30T12:38:01.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc65a44640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc601538f0 0x7fdc601453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:01.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc65a44640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc601538f0 0x7fdc601453c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58262/0 (socket says 172.21.15.12:58262) 2023-10-30T12:38:01.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc65a44640 1 -- 172.21.15.12:0/744375284 learned_addr learned my addr 172.21.15.12:0/744375284 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:38:01.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc66a46640 1 --2- 172.21.15.12:0/744375284 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc6015cd50 0x7fdc600cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:01.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc66245640 1 --2- 172.21.15.12:0/744375284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc600cefa0 0x7fdc601f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:01.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc65a44640 1 -- 172.21.15.12:0/744375284 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc6015cd50 msgr2=0x7fdc600cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:01.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc65a44640 1 --2- 172.21.15.12:0/744375284 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc6015cd50 0x7fdc600cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:01.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc65a44640 1 -- 172.21.15.12:0/744375284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc600cefa0 msgr2=0x7fdc601f3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:01.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc65a44640 1 --2- 172.21.15.12:0/744375284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc600cefa0 0x7fdc601f3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:01.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc65a44640 1 -- 172.21.15.12:0/744375284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc50009420 con 0x7fdc601538f0 2023-10-30T12:38:01.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc65a44640 1 --2- 172.21.15.12:0/744375284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc601538f0 0x7fdc601453c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdc500b3040 tx=0x7fdc5001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:01.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc56ffd640 1 -- 172.21.15.12:0/744375284 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500b9630 con 0x7fdc601538f0 2023-10-30T12:38:01.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc56ffd640 1 -- 172.21.15.12:0/744375284 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc500b97b0 con 0x7fdc601538f0 2023-10-30T12:38:01.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc67247640 1 -- 172.21.15.12:0/744375284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc601f2bc0 con 0x7fdc601538f0 2023-10-30T12:38:01.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc56ffd640 1 -- 172.21.15.12:0/744375284 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500b9b60 con 0x7fdc601538f0 2023-10-30T12:38:01.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.700+0000 7fdc67247640 1 -- 172.21.15.12:0/744375284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc600b4480 con 0x7fdc601538f0 2023-10-30T12:38:01.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.704+0000 7fdc67247640 1 -- 172.21.15.12:0/744375284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc601f2d50 con 0x7fdc601538f0 2023-10-30T12:38:01.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.704+0000 7fdc56ffd640 1 -- 172.21.15.12:0/744375284 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdc500b7040 con 0x7fdc601538f0 2023-10-30T12:38:01.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.704+0000 7fdc56ffd640 1 --2- 172.21.15.12:0/744375284 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc440664d0 0x7fdc44068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:01.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.704+0000 7fdc56ffd640 1 -- 172.21.15.12:0/744375284 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fdc50121d90 con 0x7fdc601538f0 2023-10-30T12:38:01.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.704+0000 7fdc66245640 1 --2- 172.21.15.12:0/744375284 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc440664d0 0x7fdc44068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:01.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.704+0000 7fdc66245640 1 --2- 172.21.15.12:0/744375284 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc440664d0 0x7fdc44068990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c000f70 tx=0x7fdc5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:01.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.708+0000 7fdc56ffd640 1 -- 172.21.15.12:0/744375284 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdc500ef250 con 0x7fdc601538f0 2023-10-30T12:38:01.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.900+0000 7fdc67247640 1 -- 172.21.15.12:0/744375284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc6004e170 con 0x7fdc601538f0 2023-10-30T12:38:01.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.900+0000 7fdc56ffd640 1 -- 172.21.15.12:0/744375284 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+5739 (secure 0 0 0) 0x7fdc500f2870 con 0x7fdc601538f0 2023-10-30T12:38:01.902 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:38:01.903 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:38:00.229976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:resolve","state_seq":47,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:38:01.903 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-10-30T12:38:01.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.900+0000 7fdc67247640 1 -- 172.21.15.12:0/744375284 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc440664d0 msgr2=0x7fdc44068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:01.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.900+0000 7fdc67247640 1 --2- 172.21.15.12:0/744375284 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc440664d0 0x7fdc44068990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c000f70 tx=0x7fdc5c009040 comp rx=0 tx=0).stop 2023-10-30T12:38:01.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.900+0000 7fdc67247640 1 -- 172.21.15.12:0/744375284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc601538f0 msgr2=0x7fdc601453c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:01.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.900+0000 7fdc67247640 1 --2- 172.21.15.12:0/744375284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc601538f0 0x7fdc601453c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdc500b3040 tx=0x7fdc5001ba10 comp rx=0 tx=0).stop 2023-10-30T12:38:01.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.900+0000 7fdc67247640 1 -- 172.21.15.12:0/744375284 shutdown_connections 2023-10-30T12:38:01.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.900+0000 7fdc67247640 1 --2- 172.21.15.12:0/744375284 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc440664d0 0x7fdc44068990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:01.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.900+0000 7fdc67247640 1 --2- 172.21.15.12:0/744375284 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc6015cd50 0x7fdc600cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.900+0000 7fdc67247640 1 --2- 172.21.15.12:0/744375284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc601538f0 0x7fdc601453c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.900+0000 7fdc67247640 1 --2- 172.21.15.12:0/744375284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc600cefa0 0x7fdc601f3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.900+0000 7fdc67247640 1 -- 172.21.15.12:0/744375284 >> 172.21.15.12:0/744375284 conn(0x7fdc6006a640 msgr2=0x7fdc600c48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.900+0000 7fdc67247640 1 -- 172.21.15.12:0/744375284 shutdown_connections 2023-10-30T12:38:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:01.900+0000 7fdc67247640 1 -- 172.21.15.12:0/744375284 wait complete. 2023-10-30T12:38:01.923 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:38:03.508 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:38:03.512 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:38:03.924 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:38:04.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6cab72640 1 Processor -- start 2023-10-30T12:38:04.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6cab72640 1 -- start start 2023-10-30T12:38:04.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6cab72640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6c41538a0 0x7ff6c415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:04.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6cab72640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff6c40ca0e0 0x7ff6c415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:04.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6cab72640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff6c40cf160 0x7ff6c40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:04.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6cab72640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c40cb800 con 0x7ff6c40cf160 2023-10-30T12:38:04.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6cab72640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c415dc30 con 0x7ff6c40ca0e0 2023-10-30T12:38:04.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6cab72640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c40cb330 con 0x7ff6c41538a0 2023-10-30T12:38:04.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6c910f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff6c40cf160 0x7ff6c40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:04.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6c910f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff6c40cf160 0x7ff6c40cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58272/0 (socket says 172.21.15.12:58272) 2023-10-30T12:38:04.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6c910f640 1 -- 172.21.15.12:0/3561648715 learned_addr learned my addr 172.21.15.12:0/3561648715 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:38:04.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6c890e640 1 --2- 172.21.15.12:0/3561648715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6c41538a0 0x7ff6c415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:04.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6c3fff640 1 --2- 172.21.15.12:0/3561648715 >> v2:172.21.15.154:3300/0 conn(0x7ff6c40ca0e0 0x7ff6c415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:04.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6c910f640 1 -- 172.21.15.12:0/3561648715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6c41538a0 msgr2=0x7ff6c415d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:38:04.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6c910f640 1 --2- 172.21.15.12:0/3561648715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6c41538a0 0x7ff6c415d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:04.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6c910f640 1 -- 172.21.15.12:0/3561648715 >> v2:172.21.15.154:3300/0 conn(0x7ff6c40ca0e0 msgr2=0x7ff6c415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:38:04.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6c910f640 1 --2- 172.21.15.12:0/3561648715 >> v2:172.21.15.154:3300/0 conn(0x7ff6c40ca0e0 0x7ff6c415a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:04.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6c910f640 1 -- 172.21.15.12:0/3561648715 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c40cad70 con 0x7ff6c40cf160 2023-10-30T12:38:04.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.076+0000 7ff6c3fff640 1 --2- 172.21.15.12:0/3561648715 >> v2:172.21.15.154:3300/0 conn(0x7ff6c40ca0e0 0x7ff6c415a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:38:04.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.080+0000 7ff6c910f640 1 --2- 172.21.15.12:0/3561648715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c40cf160 0x7ff6c40cf590 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40ccd10 tx=0x7ff6bc01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc056c81401ac20d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:04.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.080+0000 7ff6c37fe640 1 -- 172.21.15.12:0/3561648715 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0af400 con 0x7ff6c40cf160 2023-10-30T12:38:04.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.080+0000 7ff6c37fe640 1 -- 172.21.15.12:0/3561648715 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6bc0ab050 con 0x7ff6c40cf160 2023-10-30T12:38:04.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.080+0000 7ff6c37fe640 1 -- 172.21.15.12:0/3561648715 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0af780 con 0x7ff6c40cf160 2023-10-30T12:38:04.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.080+0000 7ff6cab72640 1 -- 172.21.15.12:0/3561648715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c40cf160 msgr2=0x7ff6c40cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:04.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.080+0000 7ff6cab72640 1 --2- 172.21.15.12:0/3561648715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c40cf160 0x7ff6c40cf590 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40ccd10 tx=0x7ff6bc01f1c0 comp rx=0 tx=0).stop 2023-10-30T12:38:04.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.080+0000 7ff6cab72640 1 -- 172.21.15.12:0/3561648715 shutdown_connections 2023-10-30T12:38:04.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.080+0000 7ff6cab72640 1 --2- 172.21.15.12:0/3561648715 >> v2:172.21.15.154:3300/0 conn(0x7ff6c40ca0e0 0x7ff6c415a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:04.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.080+0000 7ff6cab72640 1 --2- 172.21.15.12:0/3561648715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c40cf160 0x7ff6c40cf590 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:04.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.080+0000 7ff6cab72640 1 --2- 172.21.15.12:0/3561648715 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6c41538a0 0x7ff6c415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:04.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.080+0000 7ff6cab72640 1 -- 172.21.15.12:0/3561648715 >> 172.21.15.12:0/3561648715 conn(0x7ff6c406bfe0 msgr2=0x7ff6c406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:04.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.080+0000 7ff6cab72640 1 -- 172.21.15.12:0/3561648715 shutdown_connections 2023-10-30T12:38:04.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.080+0000 7ff6cab72640 1 -- 172.21.15.12:0/3561648715 wait complete. 2023-10-30T12:38:04.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6cab72640 1 Processor -- start 2023-10-30T12:38:04.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6cab72640 1 -- start start 2023-10-30T12:38:04.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6cab72640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c40ca0e0 0x7ff6c41e3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:04.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6cab72640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c40cf160 0x7ff6c41e43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:04.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6cab72640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6c41538a0 0x7ff6c40b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:04.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6cab72640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c41e3a20 con 0x7ff6c40ca0e0 2023-10-30T12:38:04.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6cab72640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c40b50e0 con 0x7ff6c40cf160 2023-10-30T12:38:04.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6cab72640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c4158430 con 0x7ff6c41538a0 2023-10-30T12:38:04.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6c890e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c40ca0e0 0x7ff6c41e3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:04.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6c890e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c40ca0e0 0x7ff6c41e3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58284/0 (socket says 172.21.15.12:58284) 2023-10-30T12:38:04.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6c890e640 1 -- 172.21.15.12:0/3813581367 learned_addr learned my addr 172.21.15.12:0/3813581367 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:38:04.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6c910f640 1 --2- 172.21.15.12:0/3813581367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6c41538a0 0x7ff6c40b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:04.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6c3fff640 1 --2- 172.21.15.12:0/3813581367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c40cf160 0x7ff6c41e43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:04.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6c890e640 1 -- 172.21.15.12:0/3813581367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6c41538a0 msgr2=0x7ff6c40b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:04.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6c890e640 1 --2- 172.21.15.12:0/3813581367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6c41538a0 0x7ff6c40b41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:04.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6c890e640 1 -- 172.21.15.12:0/3813581367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c40cf160 msgr2=0x7ff6c41e43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:04.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6c890e640 1 --2- 172.21.15.12:0/3813581367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c40cf160 0x7ff6c41e43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:04.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6c890e640 1 -- 172.21.15.12:0/3813581367 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc009b30 con 0x7ff6c40ca0e0 2023-10-30T12:38:04.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6c910f640 1 --2- 172.21.15.12:0/3813581367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6c41538a0 0x7ff6c40b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:38:04.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6c890e640 1 --2- 172.21.15.12:0/3813581367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c40ca0e0 0x7ff6c41e3eb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800e870 tx=0x7ff6b800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:04.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6c17fa640 1 -- 172.21.15.12:0/3813581367 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8009510 con 0x7ff6c40ca0e0 2023-10-30T12:38:04.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6c17fa640 1 -- 172.21.15.12:0/3813581367 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b800eec0 con 0x7ff6c40ca0e0 2023-10-30T12:38:04.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6cab72640 1 -- 172.21.15.12:0/3813581367 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c41efaf0 con 0x7ff6c40ca0e0 2023-10-30T12:38:04.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6c17fa640 1 -- 172.21.15.12:0/3813581367 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b800cb40 con 0x7ff6c40ca0e0 2023-10-30T12:38:04.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6cab72640 1 -- 172.21.15.12:0/3813581367 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c41e2c60 con 0x7ff6c40ca0e0 2023-10-30T12:38:04.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.084+0000 7ff6cab72640 1 -- 172.21.15.12:0/3813581367 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c40b2a50 con 0x7ff6c40ca0e0 2023-10-30T12:38:04.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.088+0000 7ff6c17fa640 1 -- 172.21.15.12:0/3813581367 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff6b800cca0 con 0x7ff6c40ca0e0 2023-10-30T12:38:04.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.088+0000 7ff6c17fa640 1 --2- 172.21.15.12:0/3813581367 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff6a80664d0 0x7ff6a8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:04.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.088+0000 7ff6c17fa640 1 -- 172.21.15.12:0/3813581367 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5911+0+0 (secure 0 0 0) 0x7ff6b8084150 con 0x7ff6c40ca0e0 2023-10-30T12:38:04.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.088+0000 7ff6c3fff640 1 --2- 172.21.15.12:0/3813581367 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff6a80664d0 0x7ff6a8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:04.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.088+0000 7ff6c3fff640 1 --2- 172.21.15.12:0/3813581367 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff6a80664d0 0x7ff6a8068990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4152930 tx=0x7ff6b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:04.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.092+0000 7ff6c17fa640 1 -- 172.21.15.12:0/3813581367 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff6b80513d0 con 0x7ff6c40ca0e0 2023-10-30T12:38:04.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.284+0000 7ff6cab72640 1 -- 172.21.15.12:0/3813581367 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6c404e170 con 0x7ff6c40ca0e0 2023-10-30T12:38:04.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.284+0000 7ff6c17fa640 1 -- 172.21.15.12:0/3813581367 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+5739 (secure 0 0 0) 0x7ff6b80549f0 con 0x7ff6c40ca0e0 2023-10-30T12:38:04.288 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:38:04.288 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:38:00.229976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:resolve","state_seq":47,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:38:04.289 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-10-30T12:38:04.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.288+0000 7ff6cab72640 1 -- 172.21.15.12:0/3813581367 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff6a80664d0 msgr2=0x7ff6a8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:04.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.288+0000 7ff6cab72640 1 --2- 172.21.15.12:0/3813581367 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff6a80664d0 0x7ff6a8068990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4152930 tx=0x7ff6b0008040 comp rx=0 tx=0).stop 2023-10-30T12:38:04.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.288+0000 7ff6cab72640 1 -- 172.21.15.12:0/3813581367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c40ca0e0 msgr2=0x7ff6c41e3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:04.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.288+0000 7ff6cab72640 1 --2- 172.21.15.12:0/3813581367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c40ca0e0 0x7ff6c41e3eb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800e870 tx=0x7ff6b800ec70 comp rx=0 tx=0).stop 2023-10-30T12:38:04.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.288+0000 7ff6cab72640 1 -- 172.21.15.12:0/3813581367 shutdown_connections 2023-10-30T12:38:04.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.288+0000 7ff6cab72640 1 --2- 172.21.15.12:0/3813581367 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff6a80664d0 0x7ff6a8068990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:04.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.288+0000 7ff6cab72640 1 --2- 172.21.15.12:0/3813581367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6c41538a0 0x7ff6c40b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:04.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.288+0000 7ff6cab72640 1 --2- 172.21.15.12:0/3813581367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c40cf160 0x7ff6c41e43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:04.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.288+0000 7ff6cab72640 1 --2- 172.21.15.12:0/3813581367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c40ca0e0 0x7ff6c41e3eb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:04.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.288+0000 7ff6cab72640 1 -- 172.21.15.12:0/3813581367 >> 172.21.15.12:0/3813581367 conn(0x7ff6c406bfe0 msgr2=0x7ff6c406c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:04.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.288+0000 7ff6cab72640 1 -- 172.21.15.12:0/3813581367 shutdown_connections 2023-10-30T12:38:04.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:04.288+0000 7ff6cab72640 1 -- 172.21.15.12:0/3813581367 wait complete. 2023-10-30T12:38:04.309 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:38:06.310 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:38:06.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.463+0000 7f669950f640 1 Processor -- start 2023-10-30T12:38:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.463+0000 7f669950f640 1 -- start start 2023-10-30T12:38:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.463+0000 7f669950f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66940c6bd0 0x7f669406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.463+0000 7f669950f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6694071110 0x7f6694158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.463+0000 7f669950f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6694066cf0 con 0x7f669406f7a0 2023-10-30T12:38:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.463+0000 7f669950f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66940700c0 con 0x7f6694071110 2023-10-30T12:38:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.463+0000 7f669950f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66940c70d0 con 0x7f66940c6bd0 2023-10-30T12:38:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.467+0000 7f66937fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6694071110 0x7f6694158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.467+0000 7f66937fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6694071110 0x7f6694158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58296/0 (socket says 172.21.15.12:58296) 2023-10-30T12:38:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.467+0000 7f66937fe640 1 -- 172.21.15.12:0/802625707 learned_addr learned my addr 172.21.15.12:0/802625707 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:38:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.467+0000 7f668bfff640 1 --2- 172.21.15.12:0/802625707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66940c6bd0 0x7f669406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.467+0000 7f66937fe640 1 -- 172.21.15.12:0/802625707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66940c6bd0 msgr2=0x7f669406fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:38:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.467+0000 7f66937fe640 1 --2- 172.21.15.12:0/802625707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66940c6bd0 0x7f669406fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:06.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.467+0000 7f66937fe640 1 -- 172.21.15.12:0/802625707 >> v1:172.21.15.154:6789/0 conn(0x7f669406f7a0 legacy=0x7f6694069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:06.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.467+0000 7f66937fe640 1 -- 172.21.15.12:0/802625707 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66940c7390 con 0x7f6694071110 2023-10-30T12:38:06.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.467+0000 7f66937fe640 1 --2- 172.21.15.12:0/802625707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6694071110 0x7f6694158110 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f668c009e50 tx=0x7f668c01f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95a014b82b0bd3a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:06.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.467+0000 7f66927fc640 1 -- 172.21.15.12:0/802625707 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c0af400 con 0x7f6694071110 2023-10-30T12:38:06.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.467+0000 7f66927fc640 1 -- 172.21.15.12:0/802625707 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c0ab050 con 0x7f6694071110 2023-10-30T12:38:06.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.467+0000 7f66927fc640 1 -- 172.21.15.12:0/802625707 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c0af780 con 0x7f6694071110 2023-10-30T12:38:06.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 -- 172.21.15.12:0/802625707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6694071110 msgr2=0x7f6694158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:06.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 --2- 172.21.15.12:0/802625707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6694071110 0x7f6694158110 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f668c009e50 tx=0x7f668c01f1f0 comp rx=0 tx=0).stop 2023-10-30T12:38:06.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 -- 172.21.15.12:0/802625707 shutdown_connections 2023-10-30T12:38:06.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 --2- 172.21.15.12:0/802625707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6694071110 0x7f6694158110 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:06.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 --2- 172.21.15.12:0/802625707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66940c6bd0 0x7f669406fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:06.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 -- 172.21.15.12:0/802625707 >> 172.21.15.12:0/802625707 conn(0x7f66940c7910 msgr2=0x7f66940c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:06.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 -- 172.21.15.12:0/802625707 shutdown_connections 2023-10-30T12:38:06.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 -- 172.21.15.12:0/802625707 wait complete. 2023-10-30T12:38:06.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 Processor -- start 2023-10-30T12:38:06.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 -- start start 2023-10-30T12:38:06.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f669406f7a0 0x7f6694154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:06.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6694071110 0x7f66940c3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:06.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66940c6bd0 0x7f6694153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:06.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669414ee90 con 0x7f669406f7a0 2023-10-30T12:38:06.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669414f6a0 con 0x7f6694071110 2023-10-30T12:38:06.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f669950f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6694151f20 con 0x7f66940c6bd0 2023-10-30T12:38:06.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f6692ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f669406f7a0 0x7f6694154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:06.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f6692ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f669406f7a0 0x7f6694154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58298/0 (socket says 172.21.15.12:58298) 2023-10-30T12:38:06.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f6692ffd640 1 -- 172.21.15.12:0/3496302653 learned_addr learned my addr 172.21.15.12:0/3496302653 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:38:06.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f66937fe640 1 --2- 172.21.15.12:0/3496302653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66940c6bd0 0x7f6694153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:06.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f668bfff640 1 --2- 172.21.15.12:0/3496302653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6694071110 0x7f66940c3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:06.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f6692ffd640 1 -- 172.21.15.12:0/3496302653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66940c6bd0 msgr2=0x7f6694153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:06.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f6692ffd640 1 --2- 172.21.15.12:0/3496302653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66940c6bd0 0x7f6694153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:06.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f6692ffd640 1 -- 172.21.15.12:0/3496302653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6694071110 msgr2=0x7f66940c3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:06.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f6692ffd640 1 --2- 172.21.15.12:0/3496302653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6694071110 0x7f66940c3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:06.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.471+0000 7f6692ffd640 1 -- 172.21.15.12:0/3496302653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c009b30 con 0x7f669406f7a0 2023-10-30T12:38:06.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.475+0000 7f6692ffd640 1 --2- 172.21.15.12:0/3496302653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f669406f7a0 0x7f6694154a30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f668400c3c0 tx=0x7f668400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:06.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.475+0000 7f668b7fe640 1 -- 172.21.15.12:0/3496302653 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66840052d0 con 0x7f669406f7a0 2023-10-30T12:38:06.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.475+0000 7f668b7fe640 1 -- 172.21.15.12:0/3496302653 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6684005430 con 0x7f669406f7a0 2023-10-30T12:38:06.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.475+0000 7f668b7fe640 1 -- 172.21.15.12:0/3496302653 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684003840 con 0x7f669406f7a0 2023-10-30T12:38:06.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.475+0000 7f669950f640 1 -- 172.21.15.12:0/3496302653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6694152200 con 0x7f669406f7a0 2023-10-30T12:38:06.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.475+0000 7f669950f640 1 -- 172.21.15.12:0/3496302653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669414f3c0 con 0x7f669406f7a0 2023-10-30T12:38:06.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.475+0000 7f669950f640 1 -- 172.21.15.12:0/3496302653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669404e170 con 0x7f669406f7a0 2023-10-30T12:38:06.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.475+0000 7f668b7fe640 1 -- 172.21.15.12:0/3496302653 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f66840039a0 con 0x7f669406f7a0 2023-10-30T12:38:06.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.475+0000 7f668b7fe640 1 --2- 172.21.15.12:0/3496302653 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66740664a0 0x7f6674068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:06.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.475+0000 7f668b7fe640 1 -- 172.21.15.12:0/3496302653 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f6684083600 con 0x7f669406f7a0 2023-10-30T12:38:06.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.475+0000 7f668bfff640 1 --2- 172.21.15.12:0/3496302653 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66740664a0 0x7f6674068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:06.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.479+0000 7f668bfff640 1 --2- 172.21.15.12:0/3496302653 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66740664a0 0x7f6674068960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f66941e7010 tx=0x7f667c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:06.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.479+0000 7f668b7fe640 1 -- 172.21.15.12:0/3496302653 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6684050880 con 0x7f669406f7a0 2023-10-30T12:38:06.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.671+0000 7f669950f640 1 -- 172.21.15.12:0/3496302653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66940c6160 con 0x7f669406f7a0 2023-10-30T12:38:06.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.671+0000 7f668b7fe640 1 -- 172.21.15.12:0/3496302653 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5741 (secure 0 0 0) 0x7f6684053ea0 con 0x7f669406f7a0 2023-10-30T12:38:06.676 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:38:06.676 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:38:06.168020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:reconnect","state_seq":49,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:38:06.677 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 18 2023-10-30T12:38:06.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.675+0000 7f669950f640 1 -- 172.21.15.12:0/3496302653 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66740664a0 msgr2=0x7f6674068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:06.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.675+0000 7f669950f640 1 --2- 172.21.15.12:0/3496302653 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66740664a0 0x7f6674068960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f66941e7010 tx=0x7f667c008040 comp rx=0 tx=0).stop 2023-10-30T12:38:06.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.675+0000 7f669950f640 1 -- 172.21.15.12:0/3496302653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f669406f7a0 msgr2=0x7f6694154a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:06.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.675+0000 7f669950f640 1 --2- 172.21.15.12:0/3496302653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f669406f7a0 0x7f6694154a30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f668400c3c0 tx=0x7f668400c880 comp rx=0 tx=0).stop 2023-10-30T12:38:06.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.675+0000 7f669950f640 1 -- 172.21.15.12:0/3496302653 shutdown_connections 2023-10-30T12:38:06.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.675+0000 7f669950f640 1 --2- 172.21.15.12:0/3496302653 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66740664a0 0x7f6674068960 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:06.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.675+0000 7f669950f640 1 --2- 172.21.15.12:0/3496302653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66940c6bd0 0x7f6694153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:06.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.675+0000 7f669950f640 1 --2- 172.21.15.12:0/3496302653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6694071110 0x7f66940c3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:06.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.675+0000 7f669950f640 1 --2- 172.21.15.12:0/3496302653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f669406f7a0 0x7f6694154a30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:06.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.675+0000 7f669950f640 1 -- 172.21.15.12:0/3496302653 >> 172.21.15.12:0/3496302653 conn(0x7f66940c7910 msgr2=0x7f66940c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:06.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.675+0000 7f669950f640 1 -- 172.21.15.12:0/3496302653 shutdown_connections 2023-10-30T12:38:06.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:06.675+0000 7f669950f640 1 -- 172.21.15.12:0/3496302653 wait complete. 2023-10-30T12:38:06.697 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:38:08.698 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:38:08.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f029cd71640 1 Processor -- start 2023-10-30T12:38:08.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f029cd71640 1 -- start start 2023-10-30T12:38:08.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f029cd71640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f02980717b0 0x7f029815ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:08.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f029cd71640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0298071b90 0x7f02981513c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:08.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f029cd71640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029815b060 con 0x7f02980c6290 2023-10-30T12:38:08.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f029cd71640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0298157530 con 0x7f02980717b0 2023-10-30T12:38:08.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f029cd71640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02980c4ab0 con 0x7f0298071b90 2023-10-30T12:38:08.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f0297fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f02980c6290 0x7f0298069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45016/0 (socket says 172.21.15.12:45016) 2023-10-30T12:38:08.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f0297fff640 1 -- 172.21.15.12:0/3282014054 learned_addr learned my addr 172.21.15.12:0/3282014054 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:38:08.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f02977fe640 1 --2- 172.21.15.12:0/3282014054 >> v2:172.21.15.154:3300/0 conn(0x7f02980717b0 0x7f029815ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:08.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f0296ffd640 1 --2- 172.21.15.12:0/3282014054 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0298071b90 0x7f02981513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:08.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f02967fc640 1 -- 172.21.15.12:0/3282014054 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478895588 0 0) 0x7f029815b060 con 0x7f02980c6290 2023-10-30T12:38:08.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f02967fc640 1 -- 172.21.15.12:0/3282014054 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0280003530 con 0x7f02980c6290 2023-10-30T12:38:08.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f02977fe640 1 -- 172.21.15.12:0/3282014054 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0298071b90 msgr2=0x7f02981513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:38:08.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f02977fe640 1 --2- 172.21.15.12:0/3282014054 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0298071b90 0x7f02981513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:08.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f02977fe640 1 -- 172.21.15.12:0/3282014054 >> v1:172.21.15.12:6789/0 conn(0x7f02980c6290 legacy=0x7f0298069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:08.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f02967fc640 1 -- 172.21.15.12:0/3282014054 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2518425247 0 0) 0x7f0280003530 con 0x7f02980c6290 2023-10-30T12:38:08.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f02977fe640 1 -- 172.21.15.12:0/3282014054 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02980c4d90 con 0x7f02980717b0 2023-10-30T12:38:08.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.851+0000 7f0296ffd640 1 --2- 172.21.15.12:0/3282014054 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0298071b90 0x7f02981513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:38:08.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.855+0000 7f02977fe640 1 --2- 172.21.15.12:0/3282014054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02980717b0 0x7f029815ab20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f02880097a0 tx=0x7f028801eb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1d2f1b358ac8293 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:08.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.855+0000 7f02967fc640 1 -- 172.21.15.12:0/3282014054 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290001a00 con 0x7f02980717b0 2023-10-30T12:38:08.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.855+0000 7f02967fc640 1 -- 172.21.15.12:0/3282014054 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02880a85b0 con 0x7f02980717b0 2023-10-30T12:38:08.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.855+0000 7f02967fc640 1 -- 172.21.15.12:0/3282014054 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880a8920 con 0x7f02980717b0 2023-10-30T12:38:08.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.855+0000 7f029cd71640 1 -- 172.21.15.12:0/3282014054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02980717b0 msgr2=0x7f029815ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:08.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.855+0000 7f029cd71640 1 --2- 172.21.15.12:0/3282014054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02980717b0 0x7f029815ab20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f02880097a0 tx=0x7f028801eb70 comp rx=0 tx=0).stop 2023-10-30T12:38:08.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.855+0000 7f029cd71640 1 -- 172.21.15.12:0/3282014054 shutdown_connections 2023-10-30T12:38:08.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.855+0000 7f029cd71640 1 --2- 172.21.15.12:0/3282014054 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0298071b90 0x7f02981513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:08.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.855+0000 7f029cd71640 1 --2- 172.21.15.12:0/3282014054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02980717b0 0x7f029815ab20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:08.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.855+0000 7f029cd71640 1 -- 172.21.15.12:0/3282014054 >> 172.21.15.12:0/3282014054 conn(0x7f02980c75d0 msgr2=0x7f02980c79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:08.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.855+0000 7f029cd71640 1 -- 172.21.15.12:0/3282014054 shutdown_connections 2023-10-30T12:38:08.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.855+0000 7f029cd71640 1 -- 172.21.15.12:0/3282014054 wait complete. 2023-10-30T12:38:08.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f029cd71640 1 Processor -- start 2023-10-30T12:38:08.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f029cd71640 1 -- start start 2023-10-30T12:38:08.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f029cd71640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02980717b0 0x7f02980c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:08.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f029cd71640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0298071b90 0x7f029814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:08.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f029cd71640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02980c6290 0x7f0298158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:08.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f029cd71640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029814cf60 con 0x7f02980c6290 2023-10-30T12:38:08.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f029cd71640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02981e6d20 con 0x7f0298071b90 2023-10-30T12:38:08.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f029cd71640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0298159800 con 0x7f02980717b0 2023-10-30T12:38:08.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f0297fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02980c6290 0x7f0298158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f0297fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02980c6290 0x7f0298158cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58310/0 (socket says 172.21.15.12:58310) 2023-10-30T12:38:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f0296ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0298071b90 0x7f029814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f02977fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02980717b0 0x7f02980c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f0297fff640 1 -- 172.21.15.12:0/879692274 learned_addr learned my addr 172.21.15.12:0/879692274 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:38:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f0297fff640 1 -- 172.21.15.12:0/879692274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02980717b0 msgr2=0x7f02980c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f0297fff640 1 --2- 172.21.15.12:0/879692274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02980717b0 0x7f02980c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f0297fff640 1 -- 172.21.15.12:0/879692274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0298071b90 msgr2=0x7f029814ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f0297fff640 1 --2- 172.21.15.12:0/879692274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0298071b90 0x7f029814ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f0297fff640 1 -- 172.21.15.12:0/879692274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c009420 con 0x7f02980c6290 2023-10-30T12:38:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f0297fff640 1 --2- 172.21.15.12:0/879692274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02980c6290 0x7f0298158cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f029000cc00 tx=0x7f029000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f027ffff640 1 -- 172.21.15.12:0/879692274 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029000ce10 con 0x7f02980c6290 2023-10-30T12:38:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f027ffff640 1 -- 172.21.15.12:0/879692274 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0290012400 con 0x7f02980c6290 2023-10-30T12:38:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f029cd71640 1 -- 172.21.15.12:0/879692274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02880093b0 con 0x7f02980c6290 2023-10-30T12:38:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f027ffff640 1 -- 172.21.15.12:0/879692274 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02900127b0 con 0x7f02980c6290 2023-10-30T12:38:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.859+0000 7f029cd71640 1 -- 172.21.15.12:0/879692274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02981ef370 con 0x7f02980c6290 2023-10-30T12:38:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.863+0000 7f029cd71640 1 -- 172.21.15.12:0/879692274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029804e170 con 0x7f02980c6290 2023-10-30T12:38:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.863+0000 7f027ffff640 1 -- 172.21.15.12:0/879692274 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0290012910 con 0x7f02980c6290 2023-10-30T12:38:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.863+0000 7f027ffff640 1 --2- 172.21.15.12:0/879692274 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f02840664a0 0x7f0284068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.863+0000 7f027ffff640 1 -- 172.21.15.12:0/879692274 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f02900846c0 con 0x7f02980c6290 2023-10-30T12:38:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.863+0000 7f02977fe640 1 --2- 172.21.15.12:0/879692274 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f02840664a0 0x7f0284068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.863+0000 7f02977fe640 1 --2- 172.21.15.12:0/879692274 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f02840664a0 0x7f0284068960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f02880b2000 tx=0x7f02880aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:08.867+0000 7f027ffff640 1 -- 172.21.15.12:0/879692274 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0290051970 con 0x7f02980c6290 2023-10-30T12:38:09.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:09.055+0000 7f029cd71640 1 -- 172.21.15.12:0/879692274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0298156330 con 0x7f02980c6290 2023-10-30T12:38:09.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:09.059+0000 7f027ffff640 1 -- 172.21.15.12:0/879692274 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5741 (secure 0 0 0) 0x7f0290054f90 con 0x7f02980c6290 2023-10-30T12:38:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:38:09.062 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:38:06.168020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:reconnect","state_seq":49,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:38:09.063 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 18 2023-10-30T12:38:09.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:09.059+0000 7f029cd71640 1 -- 172.21.15.12:0/879692274 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f02840664a0 msgr2=0x7f0284068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:09.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:09.059+0000 7f029cd71640 1 --2- 172.21.15.12:0/879692274 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f02840664a0 0x7f0284068960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f02880b2000 tx=0x7f02880aa070 comp rx=0 tx=0).stop 2023-10-30T12:38:09.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:09.059+0000 7f029cd71640 1 -- 172.21.15.12:0/879692274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02980c6290 msgr2=0x7f0298158cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:09.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:09.059+0000 7f029cd71640 1 --2- 172.21.15.12:0/879692274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02980c6290 0x7f0298158cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f029000cc00 tx=0x7f029000a430 comp rx=0 tx=0).stop 2023-10-30T12:38:09.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:09.059+0000 7f029cd71640 1 -- 172.21.15.12:0/879692274 shutdown_connections 2023-10-30T12:38:09.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:09.059+0000 7f029cd71640 1 --2- 172.21.15.12:0/879692274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02980c6290 0x7f0298158cc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:09.059+0000 7f029cd71640 1 --2- 172.21.15.12:0/879692274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0298071b90 0x7f029814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:09.059+0000 7f029cd71640 1 --2- 172.21.15.12:0/879692274 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f02840664a0 0x7f0284068960 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:09.059+0000 7f029cd71640 1 --2- 172.21.15.12:0/879692274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02980717b0 0x7f02980c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:09.059+0000 7f029cd71640 1 -- 172.21.15.12:0/879692274 >> 172.21.15.12:0/879692274 conn(0x7f02980c75d0 msgr2=0x7f02980c1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:09.059+0000 7f029cd71640 1 -- 172.21.15.12:0/879692274 shutdown_connections 2023-10-30T12:38:09.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:09.059+0000 7f029cd71640 1 -- 172.21.15.12:0/879692274 wait complete. 2023-10-30T12:38:09.084 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 18, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 18, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4262": { "addr": "172.21.15.12:6835/2676506149", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 2676506149, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 2676506149, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4262, "incarnation": 16, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:reconnect", "state_seq": 49 }, "gid_4282": { "addr": "172.21.15.154:6839/1517672481", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1517672481, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 1517672481, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4282, "incarnation": 10, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 7 }, "gid_4285": { "addr": "172.21.15.12:6836/291392212", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 291392212, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 291392212, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 4285, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 6 } }, "last_failure": 0, "last_failure_osd_epoch": 28, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:38:06.168020+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4285, "mds_1": 4282, "mds_2": 4262 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:38:11.085 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:38:11.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.235+0000 7f2def765640 1 Processor -- start 2023-10-30T12:38:11.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.235+0000 7f2def765640 1 -- start start 2023-10-30T12:38:11.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.239+0000 7f2def765640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2de806f7a0 0x7f2de806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:11.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.239+0000 7f2def765640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de8066cf0 con 0x7f2de8071110 2023-10-30T12:38:11.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.239+0000 7f2def765640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de80c70d0 con 0x7f2de80c6bd0 2023-10-30T12:38:11.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.239+0000 7f2def765640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de80700c0 con 0x7f2de806f7a0 2023-10-30T12:38:11.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.239+0000 7f2dedd02640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2de8071110 0x7f2de8158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41462/0 (socket says 172.21.15.12:41462) 2023-10-30T12:38:11.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.239+0000 7f2dedd02640 1 -- 172.21.15.12:0/2940565250 learned_addr learned my addr 172.21.15.12:0/2940565250 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:38:11.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.239+0000 7f2ded501640 1 --2- 172.21.15.12:0/2940565250 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2de806f7a0 0x7f2de806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:11.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.239+0000 7f2ddffff640 1 -- 172.21.15.12:0/2940565250 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916313633 0 0) 0x7f2de8066cf0 con 0x7f2de8071110 2023-10-30T12:38:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.239+0000 7f2ddffff640 1 -- 172.21.15.12:0/2940565250 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dd4003530 con 0x7f2de8071110 2023-10-30T12:38:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.239+0000 7f2ddffff640 1 -- 172.21.15.12:0/2940565250 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024334084 0 0) 0x7f2de80c70d0 con 0x7f2de80c6bd0 2023-10-30T12:38:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.239+0000 7f2ddffff640 1 -- 172.21.15.12:0/2940565250 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2de8066cf0 con 0x7f2de80c6bd0 2023-10-30T12:38:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.239+0000 7f2ded501640 1 -- 172.21.15.12:0/2940565250 >> v1:172.21.15.154:6789/0 conn(0x7f2de80c6bd0 legacy=0x7f2de8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.239+0000 7f2ded501640 1 -- 172.21.15.12:0/2940565250 >> v1:172.21.15.12:6789/0 conn(0x7f2de8071110 legacy=0x7f2de8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.239+0000 7f2ddffff640 1 -- 172.21.15.12:0/2940565250 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2731204624 0 0) 0x7f2dd4003530 con 0x7f2de8071110 2023-10-30T12:38:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.239+0000 7f2ded501640 1 -- 172.21.15.12:0/2940565250 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de80c7390 con 0x7f2de806f7a0 2023-10-30T12:38:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.243+0000 7f2ded501640 1 --2- 172.21.15.12:0/2940565250 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2de806f7a0 0x7f2de806fb80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2de40097f0 tx=0x7f2de401ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea97be63f64df7ad server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.243+0000 7f2ddffff640 1 -- 172.21.15.12:0/2940565250 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0001a00 con 0x7f2de806f7a0 2023-10-30T12:38:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.243+0000 7f2ddffff640 1 -- 172.21.15.12:0/2940565250 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de40a8650 con 0x7f2de806f7a0 2023-10-30T12:38:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.243+0000 7f2ddffff640 1 -- 172.21.15.12:0/2940565250 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de40a89c0 con 0x7f2de806f7a0 2023-10-30T12:38:11.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.243+0000 7f2def765640 1 -- 172.21.15.12:0/2940565250 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2de806f7a0 msgr2=0x7f2de806fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:11.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.243+0000 7f2def765640 1 --2- 172.21.15.12:0/2940565250 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2de806f7a0 0x7f2de806fb80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2de40097f0 tx=0x7f2de401ebd0 comp rx=0 tx=0).stop 2023-10-30T12:38:11.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.243+0000 7f2def765640 1 -- 172.21.15.12:0/2940565250 shutdown_connections 2023-10-30T12:38:11.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.243+0000 7f2def765640 1 --2- 172.21.15.12:0/2940565250 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2de806f7a0 0x7f2de806fb80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:11.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.243+0000 7f2def765640 1 -- 172.21.15.12:0/2940565250 >> 172.21.15.12:0/2940565250 conn(0x7f2de80c7910 msgr2=0x7f2de80c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:11.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.243+0000 7f2def765640 1 -- 172.21.15.12:0/2940565250 shutdown_connections 2023-10-30T12:38:11.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.243+0000 7f2def765640 1 -- 172.21.15.12:0/2940565250 wait complete. 2023-10-30T12:38:11.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2def765640 1 Processor -- start 2023-10-30T12:38:11.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2def765640 1 -- start start 2023-10-30T12:38:11.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2def765640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2de806f7a0 0x7f2de815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:11.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2def765640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2de8071110 0x7f2de80c06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:11.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2def765640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2de80c6bd0 0x7f2de80c0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:11.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2def765640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de815b8a0 con 0x7f2de8071110 2023-10-30T12:38:11.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2def765640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de815a760 con 0x7f2de806f7a0 2023-10-30T12:38:11.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2def765640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de80c1580 con 0x7f2de80c6bd0 2023-10-30T12:38:11.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2decd00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2de8071110 0x7f2de80c06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:11.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2decd00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2de8071110 0x7f2de80c06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48700/0 (socket says 172.21.15.12:48700) 2023-10-30T12:38:11.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2decd00640 1 -- 172.21.15.12:0/1873746440 learned_addr learned my addr 172.21.15.12:0/1873746440 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:38:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2ded501640 1 --2- 172.21.15.12:0/1873746440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2de806f7a0 0x7f2de815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2dedd02640 1 --2- 172.21.15.12:0/1873746440 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2de80c6bd0 0x7f2de80c0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2decd00640 1 -- 172.21.15.12:0/1873746440 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2de80c6bd0 msgr2=0x7f2de80c0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2decd00640 1 --2- 172.21.15.12:0/1873746440 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2de80c6bd0 0x7f2de80c0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2decd00640 1 -- 172.21.15.12:0/1873746440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2de806f7a0 msgr2=0x7f2de815a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2decd00640 1 --2- 172.21.15.12:0/1873746440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2de806f7a0 0x7f2de815a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2decd00640 1 -- 172.21.15.12:0/1873746440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de4009420 con 0x7f2de8071110 2023-10-30T12:38:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2decd00640 1 --2- 172.21.15.12:0/1873746440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2de8071110 0x7f2de80c06c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800c490 tx=0x7f2dd800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2dddffb640 1 -- 172.21.15.12:0/1873746440 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80052d0 con 0x7f2de8071110 2023-10-30T12:38:11.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2dddffb640 1 -- 172.21.15.12:0/1873746440 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd8005430 con 0x7f2de8071110 2023-10-30T12:38:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2dddffb640 1 -- 172.21.15.12:0/1873746440 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8012870 con 0x7f2de8071110 2023-10-30T12:38:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2def765640 1 -- 172.21.15.12:0/1873746440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de80c1820 con 0x7f2de8071110 2023-10-30T12:38:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2def765640 1 -- 172.21.15.12:0/1873746440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de81ef9f0 con 0x7f2de8071110 2023-10-30T12:38:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.247+0000 7f2def765640 1 -- 172.21.15.12:0/1873746440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de804e170 con 0x7f2de8071110 2023-10-30T12:38:11.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.251+0000 7f2dddffb640 1 -- 172.21.15.12:0/1873746440 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2dd80129d0 con 0x7f2de8071110 2023-10-30T12:38:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.251+0000 7f2dddffb640 1 --2- 172.21.15.12:0/1873746440 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2dcc0664d0 0x7f2dcc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.251+0000 7f2dddffb640 1 -- 172.21.15.12:0/1873746440 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f2dd8083a00 con 0x7f2de8071110 2023-10-30T12:38:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.251+0000 7f2ded501640 1 --2- 172.21.15.12:0/1873746440 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2dcc0664d0 0x7f2dcc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:11.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.251+0000 7f2ded501640 1 --2- 172.21.15.12:0/1873746440 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2dcc0664d0 0x7f2dcc068990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2de40b2000 tx=0x7f2de4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:11.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.255+0000 7f2dddffb640 1 -- 172.21.15.12:0/1873746440 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2dd8050dc0 con 0x7f2de8071110 2023-10-30T12:38:11.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.443+0000 7f2def765640 1 -- 172.21.15.12:0/1873746440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2de8150d90 con 0x7f2de8071110 2023-10-30T12:38:11.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.447+0000 7f2dddffb640 1 -- 172.21.15.12:0/1873746440 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5738 (secure 0 0 0) 0x7f2dd80543e0 con 0x7f2de8071110 2023-10-30T12:38:11.450 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:38:11.450 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:38:11.183869+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:rejoin","state_seq":51,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:38:11.450 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 19 2023-10-30T12:38:11.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.447+0000 7f2def765640 1 -- 172.21.15.12:0/1873746440 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2dcc0664d0 msgr2=0x7f2dcc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:11.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.447+0000 7f2def765640 1 --2- 172.21.15.12:0/1873746440 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2dcc0664d0 0x7f2dcc068990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2de40b2000 tx=0x7f2de4008040 comp rx=0 tx=0).stop 2023-10-30T12:38:11.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.447+0000 7f2def765640 1 -- 172.21.15.12:0/1873746440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2de8071110 msgr2=0x7f2de80c06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:11.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.447+0000 7f2def765640 1 --2- 172.21.15.12:0/1873746440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2de8071110 0x7f2de80c06c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800c490 tx=0x7f2dd800c950 comp rx=0 tx=0).stop 2023-10-30T12:38:11.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.447+0000 7f2def765640 1 -- 172.21.15.12:0/1873746440 shutdown_connections 2023-10-30T12:38:11.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.447+0000 7f2def765640 1 --2- 172.21.15.12:0/1873746440 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2dcc0664d0 0x7f2dcc068990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.447+0000 7f2def765640 1 --2- 172.21.15.12:0/1873746440 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2de80c6bd0 0x7f2de80c0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.447+0000 7f2def765640 1 --2- 172.21.15.12:0/1873746440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2de8071110 0x7f2de80c06c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.447+0000 7f2def765640 1 --2- 172.21.15.12:0/1873746440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2de806f7a0 0x7f2de815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.447+0000 7f2def765640 1 -- 172.21.15.12:0/1873746440 >> 172.21.15.12:0/1873746440 conn(0x7f2de80c7910 msgr2=0x7f2de806a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.447+0000 7f2def765640 1 -- 172.21.15.12:0/1873746440 shutdown_connections 2023-10-30T12:38:11.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:11.447+0000 7f2def765640 1 -- 172.21.15.12:0/1873746440 wait complete. 2023-10-30T12:38:11.470 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:38:13.471 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:38:13.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8e5a3640 1 Processor -- start 2023-10-30T12:38:13.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8e5a3640 1 -- start start 2023-10-30T12:38:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8e5a3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a880cefa0 0x7f8a880cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8e5a3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8815a6c0 con 0x7f8a881538f0 2023-10-30T12:38:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8e5a3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a880ca6a0 con 0x7f8a8815cd50 2023-10-30T12:38:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8e5a3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8815d300 con 0x7f8a880cefa0 2023-10-30T12:38:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8d5a1640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8a881538f0 0x7f8a88159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41466/0 (socket says 172.21.15.12:41466) 2023-10-30T12:38:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8d5a1640 1 -- 172.21.15.12:0/717098656 learned_addr learned my addr 172.21.15.12:0/717098656 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:38:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8dda2640 1 --2- 172.21.15.12:0/717098656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a880cefa0 0x7f8a880cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:13.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475662596 0 0) 0x7f8a8815a6c0 con 0x7f8a881538f0 2023-10-30T12:38:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a74003530 con 0x7f8a881538f0 2023-10-30T12:38:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317077445 0 0) 0x7f8a880ca6a0 con 0x7f8a8815cd50 2023-10-30T12:38:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a8815a6c0 con 0x7f8a8815cd50 2023-10-30T12:38:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2342029910 0 0) 0x7f8a74003530 con 0x7f8a881538f0 2023-10-30T12:38:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a880cefa0 msgr2=0x7f8a880cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:38:13.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 --2- 172.21.15.12:0/717098656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a880cefa0 0x7f8a880cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:13.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 >> v1:172.21.15.154:6789/0 conn(0x7f8a8815cd50 legacy=0x7f8a880c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:13.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8815d5e0 con 0x7f8a881538f0 2023-10-30T12:38:13.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f8a84001680 con 0x7f8a881538f0 2023-10-30T12:38:13.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 >> v1:172.21.15.12:6789/0 conn(0x7f8a881538f0 legacy=0x7f8a88159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:13.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 --2- 172.21.15.12:0/717098656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a74003b40 0x7f8a74003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:13.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 --2- 172.21.15.12:0/717098656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a880cefa0 0x7f8a740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:13.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 --2- 172.21.15.12:0/717098656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a74004d10 0x7f8a74005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:13.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8815d300 con 0x7f8a74003b40 2023-10-30T12:38:13.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7400ba40 con 0x7f8a74004d10 2023-10-30T12:38:13.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7400bbb0 con 0x7f8a880cefa0 2023-10-30T12:38:13.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8d5a1640 1 --2- 172.21.15.12:0/717098656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a74003b40 0x7f8a74003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:13.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8cda0640 1 --2- 172.21.15.12:0/717098656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a880cefa0 0x7f8a740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:13.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8dda2640 1 --2- 172.21.15.12:0/717098656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a74004d10 0x7f8a74005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:13.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8d5a1640 1 -- 172.21.15.12:0/717098656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a880cefa0 msgr2=0x7f8a740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:38:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8d5a1640 1 --2- 172.21.15.12:0/717098656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a880cefa0 0x7f8a740047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8d5a1640 1 -- 172.21.15.12:0/717098656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a74004d10 msgr2=0x7f8a74005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:38:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8d5a1640 1 --2- 172.21.15.12:0/717098656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a74004d10 0x7f8a74005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8d5a1640 1 -- 172.21.15.12:0/717098656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a74003530 con 0x7f8a74003b40 2023-10-30T12:38:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.623+0000 7f8a8dda2640 1 --2- 172.21.15.12:0/717098656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a74004d10 0x7f8a74005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:38:13.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.627+0000 7f8a8d5a1640 1 --2- 172.21.15.12:0/717098656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a74003b40 0x7f8a74003f40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400c160 tx=0x7f8a8401cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd72de514afb6bb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:13.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.627+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a840058e0 con 0x7f8a74003b40 2023-10-30T12:38:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.627+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a84005a60 con 0x7f8a74003b40 2023-10-30T12:38:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.627+0000 7f8a7ffff640 1 -- 172.21.15.12:0/717098656 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84005e10 con 0x7f8a74003b40 2023-10-30T12:38:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.627+0000 7f8a8e5a3640 1 -- 172.21.15.12:0/717098656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a74003b40 msgr2=0x7f8a74003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.627+0000 7f8a8e5a3640 1 --2- 172.21.15.12:0/717098656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a74003b40 0x7f8a74003f40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400c160 tx=0x7f8a8401cbc0 comp rx=0 tx=0).stop 2023-10-30T12:38:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.627+0000 7f8a8dda2640 1 -- 172.21.15.12:0/717098656 reap_dead start 2023-10-30T12:38:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.627+0000 7f8a8e5a3640 1 -- 172.21.15.12:0/717098656 shutdown_connections 2023-10-30T12:38:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.627+0000 7f8a8e5a3640 1 -- 172.21.15.12:0/717098656 >> 172.21.15.12:0/717098656 conn(0x7f8a8806a640 msgr2=0x7f8a88065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.627+0000 7f8a8e5a3640 1 -- 172.21.15.12:0/717098656 shutdown_connections 2023-10-30T12:38:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.627+0000 7f8a8e5a3640 1 -- 172.21.15.12:0/717098656 wait complete. 2023-10-30T12:38:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8e5a3640 1 Processor -- start 2023-10-30T12:38:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8e5a3640 1 -- start start 2023-10-30T12:38:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8e5a3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a74003b40 0x7f8a880c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8e5a3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a88157d80 0x7f8a881581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8e5a3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a881f2540 0x7f8a881f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8e5a3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a88159000 con 0x7f8a74003b40 2023-10-30T12:38:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8e5a3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a881573a0 con 0x7f8a881f2540 2023-10-30T12:38:13.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8e5a3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a881578f0 con 0x7f8a88157d80 2023-10-30T12:38:13.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8d5a1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a74003b40 0x7f8a880c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:13.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8d5a1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a74003b40 0x7f8a880c83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48722/0 (socket says 172.21.15.12:48722) 2023-10-30T12:38:13.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8d5a1640 1 -- 172.21.15.12:0/1811542039 learned_addr learned my addr 172.21.15.12:0/1811542039 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:38:13.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8dda2640 1 --2- 172.21.15.12:0/1811542039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a881f2540 0x7f8a881f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:13.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8cda0640 1 --2- 172.21.15.12:0/1811542039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a88157d80 0x7f8a881581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:13.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8d5a1640 1 -- 172.21.15.12:0/1811542039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a88157d80 msgr2=0x7f8a881581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:13.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8d5a1640 1 --2- 172.21.15.12:0/1811542039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a88157d80 0x7f8a881581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:13.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8d5a1640 1 -- 172.21.15.12:0/1811542039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a881f2540 msgr2=0x7f8a881f29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:13.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8d5a1640 1 --2- 172.21.15.12:0/1811542039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a881f2540 0x7f8a881f29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:13.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8d5a1640 1 -- 172.21.15.12:0/1811542039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8815bc00 con 0x7f8a74003b40 2023-10-30T12:38:13.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8d5a1640 1 --2- 172.21.15.12:0/1811542039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a74003b40 0x7f8a880c83e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8a840ab040 tx=0x7f8a74005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:13.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a7dffb640 1 -- 172.21.15.12:0/1811542039 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a840a5cd0 con 0x7f8a74003b40 2023-10-30T12:38:13.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a7dffb640 1 -- 172.21.15.12:0/1811542039 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a840a5e30 con 0x7f8a74003b40 2023-10-30T12:38:13.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a8e5a3640 1 -- 172.21.15.12:0/1811542039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a88158700 con 0x7f8a74003b40 2023-10-30T12:38:13.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.631+0000 7f8a7dffb640 1 -- 172.21.15.12:0/1811542039 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84004a10 con 0x7f8a74003b40 2023-10-30T12:38:13.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.635+0000 7f8a8e5a3640 1 -- 172.21.15.12:0/1811542039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8815c180 con 0x7f8a74003b40 2023-10-30T12:38:13.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.635+0000 7f8a8e5a3640 1 -- 172.21.15.12:0/1811542039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8815c370 con 0x7f8a74003b40 2023-10-30T12:38:13.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.635+0000 7f8a7dffb640 1 -- 172.21.15.12:0/1811542039 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8a84004b70 con 0x7f8a74003b40 2023-10-30T12:38:13.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.635+0000 7f8a7dffb640 1 --2- 172.21.15.12:0/1811542039 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8a6c0664a0 0x7f8a6c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:13.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.635+0000 7f8a7dffb640 1 -- 172.21.15.12:0/1811542039 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f8a78001d30 con 0x7f8a74003b40 2023-10-30T12:38:13.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.635+0000 7f8a8cda0640 1 --2- 172.21.15.12:0/1811542039 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8a6c0664a0 0x7f8a6c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:13.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.639+0000 7f8a8cda0640 1 --2- 172.21.15.12:0/1811542039 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8a6c0664a0 0x7f8a6c068960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a88157040 tx=0x7f8a78001490 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:13.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.639+0000 7f8a7dffb640 1 -- 172.21.15.12:0/1811542039 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8a840eada0 con 0x7f8a74003b40 2023-10-30T12:38:13.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.831+0000 7f8a8e5a3640 1 -- 172.21.15.12:0/1811542039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a8804e170 con 0x7f8a74003b40 2023-10-30T12:38:13.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.831+0000 7f8a7dffb640 1 -- 172.21.15.12:0/1811542039 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5738 (secure 0 0 0) 0x7f8a840ee3c0 con 0x7f8a74003b40 2023-10-30T12:38:13.835 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:38:13.835 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:38:11.183869+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:rejoin","state_seq":51,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:38:13.835 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 19 2023-10-30T12:38:13.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.831+0000 7f8a8e5a3640 1 -- 172.21.15.12:0/1811542039 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8a6c0664a0 msgr2=0x7f8a6c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:13.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.831+0000 7f8a8e5a3640 1 --2- 172.21.15.12:0/1811542039 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8a6c0664a0 0x7f8a6c068960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a88157040 tx=0x7f8a78001490 comp rx=0 tx=0).stop 2023-10-30T12:38:13.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.831+0000 7f8a8e5a3640 1 -- 172.21.15.12:0/1811542039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a74003b40 msgr2=0x7f8a880c83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:13.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.831+0000 7f8a8e5a3640 1 --2- 172.21.15.12:0/1811542039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a74003b40 0x7f8a880c83e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8a840ab040 tx=0x7f8a74005800 comp rx=0 tx=0).stop 2023-10-30T12:38:13.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.831+0000 7f8a8e5a3640 1 -- 172.21.15.12:0/1811542039 shutdown_connections 2023-10-30T12:38:13.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.831+0000 7f8a8e5a3640 1 --2- 172.21.15.12:0/1811542039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a881f2540 0x7f8a881f29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:13.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.831+0000 7f8a8e5a3640 1 --2- 172.21.15.12:0/1811542039 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8a6c0664a0 0x7f8a6c068960 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:13.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.831+0000 7f8a8e5a3640 1 --2- 172.21.15.12:0/1811542039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a88157d80 0x7f8a881581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:13.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.831+0000 7f8a8e5a3640 1 --2- 172.21.15.12:0/1811542039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a74003b40 0x7f8a880c83e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:13.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.831+0000 7f8a8e5a3640 1 -- 172.21.15.12:0/1811542039 >> 172.21.15.12:0/1811542039 conn(0x7f8a8806a640 msgr2=0x7f8a880c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:13.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.831+0000 7f8a8e5a3640 1 -- 172.21.15.12:0/1811542039 shutdown_connections 2023-10-30T12:38:13.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:13.831+0000 7f8a8e5a3640 1 -- 172.21.15.12:0/1811542039 wait complete. 2023-10-30T12:38:13.856 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:38:15.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:38:16.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.011+0000 7f4d5592a640 1 Processor -- start 2023-10-30T12:38:16.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.011+0000 7f4d5592a640 1 -- start start 2023-10-30T12:38:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.011+0000 7f4d5592a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d50153d70 0x7f4d5006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.011+0000 7f4d5592a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4d5006e470 0x7f4d500cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.011+0000 7f4d5592a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d5006afa0 con 0x7f4d500c8490 2023-10-30T12:38:16.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.011+0000 7f4d5592a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5006d780 con 0x7f4d5006e470 2023-10-30T12:38:16.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.011+0000 7f4d5592a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d50154150 con 0x7f4d50153d70 2023-10-30T12:38:16.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.011+0000 7f4d47fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4d5006e470 0x7f4d500cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:16.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.011+0000 7f4d47fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4d5006e470 0x7f4d500cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48728/0 (socket says 172.21.15.12:48728) 2023-10-30T12:38:16.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.011+0000 7f4d47fff640 1 -- 172.21.15.12:0/3287946100 learned_addr learned my addr 172.21.15.12:0/3287946100 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:38:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.011+0000 7f4d4effd640 1 --2- 172.21.15.12:0/3287946100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d50153d70 0x7f4d5006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.011+0000 7f4d47fff640 1 -- 172.21.15.12:0/3287946100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d50153d70 msgr2=0x7f4d5006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:38:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.011+0000 7f4d47fff640 1 --2- 172.21.15.12:0/3287946100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d50153d70 0x7f4d5006ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.011+0000 7f4d47fff640 1 -- 172.21.15.12:0/3287946100 >> v1:172.21.15.154:6789/0 conn(0x7f4d500c8490 legacy=0x7f4d500c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.011+0000 7f4d47fff640 1 -- 172.21.15.12:0/3287946100 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5006e030 con 0x7f4d5006e470 2023-10-30T12:38:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.015+0000 7f4d47fff640 1 --2- 172.21.15.12:0/3287946100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d5006e470 0x7f4d500cce00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009870 tx=0x7f4d3801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b019e8462965fa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.015+0000 7f4d4e7fc640 1 -- 172.21.15.12:0/3287946100 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d380aa670 con 0x7f4d5006e470 2023-10-30T12:38:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.015+0000 7f4d4e7fc640 1 -- 172.21.15.12:0/3287946100 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d380aa7d0 con 0x7f4d5006e470 2023-10-30T12:38:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.015+0000 7f4d4e7fc640 1 -- 172.21.15.12:0/3287946100 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d380aab60 con 0x7f4d5006e470 2023-10-30T12:38:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.015+0000 7f4d5592a640 1 -- 172.21.15.12:0/3287946100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d5006e470 msgr2=0x7f4d500cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.015+0000 7f4d5592a640 1 --2- 172.21.15.12:0/3287946100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d5006e470 0x7f4d500cce00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009870 tx=0x7f4d3801c510 comp rx=0 tx=0).stop 2023-10-30T12:38:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.015+0000 7f4d5592a640 1 -- 172.21.15.12:0/3287946100 shutdown_connections 2023-10-30T12:38:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.015+0000 7f4d5592a640 1 --2- 172.21.15.12:0/3287946100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d5006e470 0x7f4d500cce00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:16.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.015+0000 7f4d5592a640 1 --2- 172.21.15.12:0/3287946100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d50153d70 0x7f4d5006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:16.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.015+0000 7f4d5592a640 1 -- 172.21.15.12:0/3287946100 >> 172.21.15.12:0/3287946100 conn(0x7f4d50071ea0 msgr2=0x7f4d50065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:16.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.015+0000 7f4d5592a640 1 -- 172.21.15.12:0/3287946100 shutdown_connections 2023-10-30T12:38:16.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.015+0000 7f4d5592a640 1 -- 172.21.15.12:0/3287946100 wait complete. 2023-10-30T12:38:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.015+0000 7f4d5592a640 1 Processor -- start 2023-10-30T12:38:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.015+0000 7f4d5592a640 1 -- start start 2023-10-30T12:38:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d5592a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d5006e470 0x7f4d50149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d5592a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d500c8490 0x7f4d501f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d5592a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d50153d70 0x7f4d50155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d5592a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d50156910 con 0x7f4d50153d70 2023-10-30T12:38:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d5592a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d50156b80 con 0x7f4d5006e470 2023-10-30T12:38:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d5592a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d50156e60 con 0x7f4d500c8490 2023-10-30T12:38:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d4f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d50153d70 0x7f4d50155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d4f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d50153d70 0x7f4d50155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48736/0 (socket says 172.21.15.12:48736) 2023-10-30T12:38:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d4f7fe640 1 -- 172.21.15.12:0/1967788896 learned_addr learned my addr 172.21.15.12:0/1967788896 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:38:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d4effd640 1 --2- 172.21.15.12:0/1967788896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d5006e470 0x7f4d50149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d47fff640 1 --2- 172.21.15.12:0/1967788896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d500c8490 0x7f4d501f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d4f7fe640 1 -- 172.21.15.12:0/1967788896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d500c8490 msgr2=0x7f4d501f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d4f7fe640 1 --2- 172.21.15.12:0/1967788896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d500c8490 0x7f4d501f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d4f7fe640 1 -- 172.21.15.12:0/1967788896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d5006e470 msgr2=0x7f4d50149960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d4f7fe640 1 --2- 172.21.15.12:0/1967788896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d5006e470 0x7f4d50149960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d4f7fe640 1 -- 172.21.15.12:0/1967788896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d380094d0 con 0x7f4d50153d70 2023-10-30T12:38:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d47fff640 1 --2- 172.21.15.12:0/1967788896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d500c8490 0x7f4d501f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:38:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d4f7fe640 1 --2- 172.21.15.12:0/1967788896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d50153d70 0x7f4d50155340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800c770 tx=0x7f4d4800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d4cff9640 1 -- 172.21.15.12:0/1967788896 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48003b80 con 0x7f4d50153d70 2023-10-30T12:38:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d4cff9640 1 -- 172.21.15.12:0/1967788896 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d48003ce0 con 0x7f4d50153d70 2023-10-30T12:38:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d4cff9640 1 -- 172.21.15.12:0/1967788896 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4800aa90 con 0x7f4d50153d70 2023-10-30T12:38:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d5592a640 1 -- 172.21.15.12:0/1967788896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d50157140 con 0x7f4d50153d70 2023-10-30T12:38:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d5592a640 1 -- 172.21.15.12:0/1967788896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d501f6370 con 0x7f4d50153d70 2023-10-30T12:38:16.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.019+0000 7f4d5592a640 1 -- 172.21.15.12:0/1967788896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5004e170 con 0x7f4d50153d70 2023-10-30T12:38:16.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.023+0000 7f4d4cff9640 1 -- 172.21.15.12:0/1967788896 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4d4800abf0 con 0x7f4d50153d70 2023-10-30T12:38:16.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.023+0000 7f4d4cff9640 1 --2- 172.21.15.12:0/1967788896 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4d300664a0 0x7f4d30068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:16.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.023+0000 7f4d4cff9640 1 -- 172.21.15.12:0/1967788896 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f4d48083d80 con 0x7f4d50153d70 2023-10-30T12:38:16.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.023+0000 7f4d4effd640 1 --2- 172.21.15.12:0/1967788896 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4d300664a0 0x7f4d30068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:16.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.023+0000 7f4d4effd640 1 --2- 172.21.15.12:0/1967788896 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4d300664a0 0x7f4d30068960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4d400079c0 tx=0x7f4d40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:16.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.027+0000 7f4d4cff9640 1 -- 172.21.15.12:0/1967788896 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4d480510a0 con 0x7f4d50153d70 2023-10-30T12:38:16.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.219+0000 7f4d5592a640 1 -- 172.21.15.12:0/1967788896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d5006c2a0 con 0x7f4d50153d70 2023-10-30T12:38:16.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.219+0000 7f4d4cff9640 1 -- 172.21.15.12:0/1967788896 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5738 (secure 0 0 0) 0x7f4d480546c0 con 0x7f4d50153d70 2023-10-30T12:38:16.225 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:38:16.225 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:38:11.183869+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:rejoin","state_seq":51,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:38:16.225 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 19 2023-10-30T12:38:16.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.223+0000 7f4d5592a640 1 -- 172.21.15.12:0/1967788896 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4d300664a0 msgr2=0x7f4d30068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:16.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.223+0000 7f4d5592a640 1 --2- 172.21.15.12:0/1967788896 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4d300664a0 0x7f4d30068960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4d400079c0 tx=0x7f4d40008040 comp rx=0 tx=0).stop 2023-10-30T12:38:16.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.223+0000 7f4d5592a640 1 -- 172.21.15.12:0/1967788896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d50153d70 msgr2=0x7f4d50155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:16.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.223+0000 7f4d5592a640 1 --2- 172.21.15.12:0/1967788896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d50153d70 0x7f4d50155340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800c770 tx=0x7f4d4800cc30 comp rx=0 tx=0).stop 2023-10-30T12:38:16.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.223+0000 7f4d5592a640 1 -- 172.21.15.12:0/1967788896 shutdown_connections 2023-10-30T12:38:16.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.223+0000 7f4d5592a640 1 --2- 172.21.15.12:0/1967788896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d50153d70 0x7f4d50155340 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:16.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.223+0000 7f4d5592a640 1 --2- 172.21.15.12:0/1967788896 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4d300664a0 0x7f4d30068960 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:16.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.223+0000 7f4d5592a640 1 --2- 172.21.15.12:0/1967788896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d500c8490 0x7f4d501f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:16.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.223+0000 7f4d5592a640 1 --2- 172.21.15.12:0/1967788896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d5006e470 0x7f4d50149960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:16.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.223+0000 7f4d5592a640 1 -- 172.21.15.12:0/1967788896 >> 172.21.15.12:0/1967788896 conn(0x7f4d50071ea0 msgr2=0x7f4d50151e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:16.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.223+0000 7f4d5592a640 1 -- 172.21.15.12:0/1967788896 shutdown_connections 2023-10-30T12:38:16.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:16.223+0000 7f4d5592a640 1 -- 172.21.15.12:0/1967788896 wait complete. 2023-10-30T12:38:16.248 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:38:18.249 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:38:18.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.399+0000 7f31fa6d6640 1 Processor -- start 2023-10-30T12:38:18.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.399+0000 7f31fa6d6640 1 -- start start 2023-10-30T12:38:18.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.399+0000 7f31fa6d6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f31f40c6f60 0x7f31f41566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:18.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.399+0000 7f31fa6d6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f40c0a50 con 0x7f31f40c6b80 2023-10-30T12:38:18.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.399+0000 7f31fa6d6640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f406f6e0 con 0x7f31f40c05d0 2023-10-30T12:38:18.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.399+0000 7f31fa6d6640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f406f9c0 con 0x7f31f40c6f60 2023-10-30T12:38:18.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.399+0000 7f31f3fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f31f40c6b80 0x7f31f40bfec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41470/0 (socket says 172.21.15.12:41470) 2023-10-30T12:38:18.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.399+0000 7f31f3fff640 1 -- 172.21.15.12:0/781019472 learned_addr learned my addr 172.21.15.12:0/781019472 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:38:18.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f8c73640 1 --2- 172.21.15.12:0/781019472 >> v2:172.21.15.154:3300/0 conn(0x7f31f40c6f60 0x7f31f41566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:18.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f2ffd640 1 -- 172.21.15.12:0/781019472 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588850714 0 0) 0x7f31f40c0a50 con 0x7f31f40c6b80 2023-10-30T12:38:18.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f2ffd640 1 -- 172.21.15.12:0/781019472 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31dc003530 con 0x7f31f40c6b80 2023-10-30T12:38:18.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f2ffd640 1 -- 172.21.15.12:0/781019472 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3203127423 0 0) 0x7f31dc003530 con 0x7f31f40c6b80 2023-10-30T12:38:18.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f2ffd640 1 -- 172.21.15.12:0/781019472 >> v2:172.21.15.154:3300/0 conn(0x7f31f40c6f60 msgr2=0x7f31f41566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:38:18.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f2ffd640 1 --2- 172.21.15.12:0/781019472 >> v2:172.21.15.154:3300/0 conn(0x7f31f40c6f60 0x7f31f41566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:18.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f2ffd640 1 -- 172.21.15.12:0/781019472 >> v1:172.21.15.154:6789/0 conn(0x7f31f40c05d0 legacy=0x7f31f413ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:18.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f2ffd640 1 -- 172.21.15.12:0/781019472 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f406fca0 con 0x7f31f40c6b80 2023-10-30T12:38:18.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f2ffd640 1 -- 172.21.15.12:0/781019472 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f31e4001680 con 0x7f31f40c6b80 2023-10-30T12:38:18.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f2ffd640 1 -- 172.21.15.12:0/781019472 >> v1:172.21.15.12:6789/0 conn(0x7f31f40c6b80 legacy=0x7f31f40bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:18.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f2ffd640 1 --2- 172.21.15.12:0/781019472 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31dc003980 0x7f31dc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:18.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f2ffd640 1 --2- 172.21.15.12:0/781019472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31dc0047d0 0x7f31dc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:18.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f2ffd640 1 --2- 172.21.15.12:0/781019472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31dc005190 0x7f31dc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:18.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f2ffd640 1 -- 172.21.15.12:0/781019472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f406f9c0 con 0x7f31dc0047d0 2023-10-30T12:38:18.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f2ffd640 1 -- 172.21.15.12:0/781019472 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc00c030 con 0x7f31dc005190 2023-10-30T12:38:18.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f2ffd640 1 -- 172.21.15.12:0/781019472 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc00c1f0 con 0x7f31dc003980 2023-10-30T12:38:18.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f37fe640 1 --2- 172.21.15.12:0/781019472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31dc0047d0 0x7f31dc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:18.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f3fff640 1 --2- 172.21.15.12:0/781019472 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31dc003980 0x7f31dc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:18.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f8c73640 1 --2- 172.21.15.12:0/781019472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31dc005190 0x7f31dc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:18.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f37fe640 1 -- 172.21.15.12:0/781019472 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31dc003980 msgr2=0x7f31dc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:38:18.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f37fe640 1 --2- 172.21.15.12:0/781019472 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31dc003980 0x7f31dc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:18.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f37fe640 1 -- 172.21.15.12:0/781019472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31dc005190 msgr2=0x7f31dc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:38:18.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f37fe640 1 --2- 172.21.15.12:0/781019472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31dc005190 0x7f31dc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:18.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f37fe640 1 -- 172.21.15.12:0/781019472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc003530 con 0x7f31dc0047d0 2023-10-30T12:38:18.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f8c73640 1 --2- 172.21.15.12:0/781019472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31dc005190 0x7f31dc00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:38:18.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.403+0000 7f31f8c73640 1 -- 172.21.15.12:0/781019472 reap_dead start 2023-10-30T12:38:18.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.407+0000 7f31f37fe640 1 --2- 172.21.15.12:0/781019472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31dc0047d0 0x7f31dc004c50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f31e8001750 tx=0x7f31e801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f854114ccf54e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:18.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.407+0000 7f31f2ffd640 1 -- 172.21.15.12:0/781019472 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e80052d0 con 0x7f31dc0047d0 2023-10-30T12:38:18.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.407+0000 7f31f2ffd640 1 -- 172.21.15.12:0/781019472 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e80a7050 con 0x7f31dc0047d0 2023-10-30T12:38:18.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.407+0000 7f31f2ffd640 1 -- 172.21.15.12:0/781019472 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8003600 con 0x7f31dc0047d0 2023-10-30T12:38:18.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.407+0000 7f31fa6d6640 1 -- 172.21.15.12:0/781019472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31dc0047d0 msgr2=0x7f31dc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:18.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.407+0000 7f31fa6d6640 1 --2- 172.21.15.12:0/781019472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31dc0047d0 0x7f31dc004c50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f31e8001750 tx=0x7f31e801c3a0 comp rx=0 tx=0).stop 2023-10-30T12:38:18.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.407+0000 7f31fa6d6640 1 -- 172.21.15.12:0/781019472 shutdown_connections 2023-10-30T12:38:18.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.407+0000 7f31fa6d6640 1 --2- 172.21.15.12:0/781019472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31dc0047d0 0x7f31dc004c50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:18.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.407+0000 7f31fa6d6640 1 -- 172.21.15.12:0/781019472 >> 172.21.15.12:0/781019472 conn(0x7f31f40c7910 msgr2=0x7f31f40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:18.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.407+0000 7f31fa6d6640 1 -- 172.21.15.12:0/781019472 shutdown_connections 2023-10-30T12:38:18.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.407+0000 7f31fa6d6640 1 -- 172.21.15.12:0/781019472 wait complete. 2023-10-30T12:38:18.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.407+0000 7f31fa6d6640 1 Processor -- start 2023-10-30T12:38:18.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.407+0000 7f31fa6d6640 1 -- start start 2023-10-30T12:38:18.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31fa6d6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31dc0047d0 0x7f31f40c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:18.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31fa6d6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31f4153b70 0x7f31f40ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:18.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31fa6d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f4158930 0x7f31f4158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:18.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31fa6d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f41efea0 con 0x7f31f4158930 2023-10-30T12:38:18.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31fa6d6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40c4b00 con 0x7f31f4153b70 2023-10-30T12:38:18.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31fa6d6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f41592f0 con 0x7f31dc0047d0 2023-10-30T12:38:18.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f8c73640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f4158930 0x7f31f4158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:18.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f8c73640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f4158930 0x7f31f4158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48756/0 (socket says 172.21.15.12:48756) 2023-10-30T12:38:18.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f8c73640 1 -- 172.21.15.12:0/531551174 learned_addr learned my addr 172.21.15.12:0/531551174 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:38:18.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f3fff640 1 --2- 172.21.15.12:0/531551174 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31dc0047d0 0x7f31f40c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:18.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f37fe640 1 --2- 172.21.15.12:0/531551174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31f4153b70 0x7f31f40ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:18.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f8c73640 1 -- 172.21.15.12:0/531551174 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31dc0047d0 msgr2=0x7f31f40c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:18.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f8c73640 1 --2- 172.21.15.12:0/531551174 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31dc0047d0 0x7f31f40c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:18.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f8c73640 1 -- 172.21.15.12:0/531551174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31f4153b70 msgr2=0x7f31f40ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:18.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f8c73640 1 --2- 172.21.15.12:0/531551174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31f4153b70 0x7f31f40ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:18.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f8c73640 1 -- 172.21.15.12:0/531551174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f40c4da0 con 0x7f31f4158930 2023-10-30T12:38:18.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f37fe640 1 --2- 172.21.15.12:0/531551174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31f4153b70 0x7f31f40ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:38:18.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f8c73640 1 --2- 172.21.15.12:0/531551174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f4158930 0x7f31f4158db0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f31f40b1190 tx=0x7f31ec010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f0ff9640 1 -- 172.21.15.12:0/531551174 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec01b070 con 0x7f31f4158930 2023-10-30T12:38:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f0ff9640 1 -- 172.21.15.12:0/531551174 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec00f040 con 0x7f31f4158930 2023-10-30T12:38:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f0ff9640 1 -- 172.21.15.12:0/531551174 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec00e050 con 0x7f31f4158930 2023-10-30T12:38:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31fa6d6640 1 -- 172.21.15.12:0/531551174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f40c5060 con 0x7f31f4158930 2023-10-30T12:38:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31fa6d6640 1 -- 172.21.15.12:0/531551174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f41ef850 con 0x7f31f4158930 2023-10-30T12:38:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31fa6d6640 1 -- 172.21.15.12:0/531551174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f404e170 con 0x7f31f4158930 2023-10-30T12:38:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.411+0000 7f31f0ff9640 1 -- 172.21.15.12:0/531551174 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f31ec0039a0 con 0x7f31f4158930 2023-10-30T12:38:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.415+0000 7f31f0ff9640 1 --2- 172.21.15.12:0/531551174 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31e00664a0 0x7f31e0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.415+0000 7f31f0ff9640 1 -- 172.21.15.12:0/531551174 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f31ec084500 con 0x7f31f4158930 2023-10-30T12:38:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.415+0000 7f31f3fff640 1 --2- 172.21.15.12:0/531551174 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31e00664a0 0x7f31e0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.415+0000 7f31f3fff640 1 --2- 172.21.15.12:0/531551174 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31e00664a0 0x7f31e0068960 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f31f4069010 tx=0x7f31e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.419+0000 7f31f0ff9640 1 -- 172.21.15.12:0/531551174 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f31ec051780 con 0x7f31f4158930 2023-10-30T12:38:18.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.607+0000 7f31fa6d6640 1 -- 172.21.15.12:0/531551174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31f406a510 con 0x7f31f4158930 2023-10-30T12:38:18.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.607+0000 7f31f0ff9640 1 -- 172.21.15.12:0/531551174 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5738 (secure 0 0 0) 0x7f31ec054da0 con 0x7f31f4158930 2023-10-30T12:38:18.612 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:38:18.612 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:38:16.238773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:active","state_seq":53,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:38:18.613 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 20 2023-10-30T12:38:18.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.611+0000 7f31fa6d6640 1 -- 172.21.15.12:0/531551174 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31e00664a0 msgr2=0x7f31e0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:18.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.611+0000 7f31fa6d6640 1 --2- 172.21.15.12:0/531551174 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31e00664a0 0x7f31e0068960 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f31f4069010 tx=0x7f31e4009040 comp rx=0 tx=0).stop 2023-10-30T12:38:18.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.611+0000 7f31fa6d6640 1 -- 172.21.15.12:0/531551174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f4158930 msgr2=0x7f31f4158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:18.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.611+0000 7f31fa6d6640 1 --2- 172.21.15.12:0/531551174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f4158930 0x7f31f4158db0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f31f40b1190 tx=0x7f31ec010590 comp rx=0 tx=0).stop 2023-10-30T12:38:18.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.611+0000 7f31fa6d6640 1 -- 172.21.15.12:0/531551174 shutdown_connections 2023-10-30T12:38:18.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.611+0000 7f31fa6d6640 1 --2- 172.21.15.12:0/531551174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f4158930 0x7f31f4158db0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:18.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.611+0000 7f31fa6d6640 1 --2- 172.21.15.12:0/531551174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31f4153b70 0x7f31f40ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:18.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.611+0000 7f31fa6d6640 1 --2- 172.21.15.12:0/531551174 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31e00664a0 0x7f31e0068960 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:18.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.611+0000 7f31fa6d6640 1 --2- 172.21.15.12:0/531551174 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31dc0047d0 0x7f31f40c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:18.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.611+0000 7f31fa6d6640 1 -- 172.21.15.12:0/531551174 >> 172.21.15.12:0/531551174 conn(0x7f31f40c7910 msgr2=0x7f31f406a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:18.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.611+0000 7f31fa6d6640 1 -- 172.21.15.12:0/531551174 shutdown_connections 2023-10-30T12:38:18.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:18.611+0000 7f31fa6d6640 1 -- 172.21.15.12:0/531551174 wait complete. 2023-10-30T12:38:18.633 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=2, replacing gid=4269 2023-10-30T12:38:18.633 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.c 2023-10-30T12:38:19.481 INFO:tasks.ceph.mds.e.smithi012.stderr:2023-10-30T12:38:19.475+0000 7fbd9da0c640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-30T12:38:20.634 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2023-10-30T12:38:20.634 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-30T12:38:20.635 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-30T12:38:20.639 INFO:tasks.ceph.mds.c:Started 2023-10-30T12:38:20.640 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:38:20.737 INFO:tasks.ceph.mds.c.smithi012.stdout:starting mds.c at 2023-10-30T12:38:20.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.791+0000 7fee76e32640 1 Processor -- start 2023-10-30T12:38:20.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.791+0000 7fee76e32640 1 -- start start 2023-10-30T12:38:20.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.791+0000 7fee76e32640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee70156f80 0x7fee70159880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:20.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.791+0000 7fee76e32640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fee70070650 0x7fee70071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:20.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.791+0000 7fee76e32640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee700c2f20 con 0x7fee70070c70 2023-10-30T12:38:20.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.791+0000 7fee76e32640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee70157360 con 0x7fee70070650 2023-10-30T12:38:20.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.791+0000 7fee76e32640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee700c85c0 con 0x7fee70156f80 2023-10-30T12:38:20.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.795+0000 7fee6ffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fee70070650 0x7fee70071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:20.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.795+0000 7fee6ffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fee70070650 0x7fee70071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48784/0 (socket says 172.21.15.12:48784) 2023-10-30T12:38:20.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.795+0000 7fee6ffff640 1 -- 172.21.15.12:0/4157724532 learned_addr learned my addr 172.21.15.12:0/4157724532 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:38:20.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.795+0000 7fee74bce640 1 --2- 172.21.15.12:0/4157724532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee70156f80 0x7fee70159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:20.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.795+0000 7fee6ffff640 1 -- 172.21.15.12:0/4157724532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee70156f80 msgr2=0x7fee70159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:38:20.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.795+0000 7fee6ffff640 1 --2- 172.21.15.12:0/4157724532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee70156f80 0x7fee70159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:20.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.795+0000 7fee6ffff640 1 -- 172.21.15.12:0/4157724532 >> v1:172.21.15.154:6789/0 conn(0x7fee70070c70 legacy=0x7fee7015cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:20.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.795+0000 7fee6ffff640 1 -- 172.21.15.12:0/4157724532 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee70071230 con 0x7fee70070650 2023-10-30T12:38:20.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.795+0000 7fee6ffff640 1 --2- 172.21.15.12:0/4157724532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee70070650 0x7fee70071bb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009870 tx=0x7fee5c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747430e49e9ccf3f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:20.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.795+0000 7fee6f7fe640 1 -- 172.21.15.12:0/4157724532 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0aa660 con 0x7fee70070650 2023-10-30T12:38:20.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.795+0000 7fee6f7fe640 1 -- 172.21.15.12:0/4157724532 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c0aa7c0 con 0x7fee70070650 2023-10-30T12:38:20.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.795+0000 7fee6f7fe640 1 -- 172.21.15.12:0/4157724532 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0aab50 con 0x7fee70070650 2023-10-30T12:38:20.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 -- 172.21.15.12:0/4157724532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee70070650 msgr2=0x7fee70071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:20.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 --2- 172.21.15.12:0/4157724532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee70070650 0x7fee70071bb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009870 tx=0x7fee5c01c540 comp rx=0 tx=0).stop 2023-10-30T12:38:20.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 -- 172.21.15.12:0/4157724532 shutdown_connections 2023-10-30T12:38:20.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 --2- 172.21.15.12:0/4157724532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee70070650 0x7fee70071bb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:20.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 --2- 172.21.15.12:0/4157724532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee70156f80 0x7fee70159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:20.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 -- 172.21.15.12:0/4157724532 >> 172.21.15.12:0/4157724532 conn(0x7fee700cbeb0 msgr2=0x7fee700cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:20.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 -- 172.21.15.12:0/4157724532 shutdown_connections 2023-10-30T12:38:20.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 -- 172.21.15.12:0/4157724532 wait complete. 2023-10-30T12:38:20.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 Processor -- start 2023-10-30T12:38:20.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 -- start start 2023-10-30T12:38:20.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee70070650 0x7fee7015cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:20.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee70070c70 0x7fee700afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:20.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee70156f80 0x7fee700b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:20.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee701f3280 con 0x7fee70156f80 2023-10-30T12:38:20.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee701f2420 con 0x7fee70070c70 2023-10-30T12:38:20.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee76e32640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee701f2700 con 0x7fee70070650 2023-10-30T12:38:20.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee753cf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee70156f80 0x7fee700b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:20.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee753cf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee70156f80 0x7fee700b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48786/0 (socket says 172.21.15.12:48786) 2023-10-30T12:38:20.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee753cf640 1 -- 172.21.15.12:0/1192306896 learned_addr learned my addr 172.21.15.12:0/1192306896 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:38:20.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee74bce640 1 --2- 172.21.15.12:0/1192306896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee70070650 0x7fee7015cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:20.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee6ffff640 1 --2- 172.21.15.12:0/1192306896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee70070c70 0x7fee700afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:20.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee753cf640 1 -- 172.21.15.12:0/1192306896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee70070650 msgr2=0x7fee7015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee753cf640 1 --2- 172.21.15.12:0/1192306896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee70070650 0x7fee7015cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee753cf640 1 -- 172.21.15.12:0/1192306896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee70070c70 msgr2=0x7fee700afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee753cf640 1 --2- 172.21.15.12:0/1192306896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee70070c70 0x7fee700afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.799+0000 7fee753cf640 1 -- 172.21.15.12:0/1192306896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c0094d0 con 0x7fee70156f80 2023-10-30T12:38:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.803+0000 7fee753cf640 1 --2- 172.21.15.12:0/1192306896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee70156f80 0x7fee700b00c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fee6800c770 tx=0x7fee6800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.803+0000 7fee6d7fa640 1 -- 172.21.15.12:0/1192306896 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee68003b80 con 0x7fee70156f80 2023-10-30T12:38:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.803+0000 7fee6d7fa640 1 -- 172.21.15.12:0/1192306896 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee68003ce0 con 0x7fee70156f80 2023-10-30T12:38:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.803+0000 7fee76e32640 1 -- 172.21.15.12:0/1192306896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee701f29e0 con 0x7fee70156f80 2023-10-30T12:38:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.803+0000 7fee6d7fa640 1 -- 172.21.15.12:0/1192306896 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6800aa90 con 0x7fee70156f80 2023-10-30T12:38:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.803+0000 7fee76e32640 1 -- 172.21.15.12:0/1192306896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee701f2fa0 con 0x7fee70156f80 2023-10-30T12:38:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.803+0000 7fee76e32640 1 -- 172.21.15.12:0/1192306896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7004e170 con 0x7fee70156f80 2023-10-30T12:38:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.803+0000 7fee6d7fa640 1 -- 172.21.15.12:0/1192306896 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fee6800abf0 con 0x7fee70156f80 2023-10-30T12:38:20.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.803+0000 7fee6d7fa640 1 --2- 172.21.15.12:0/1192306896 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fee540664d0 0x7fee54068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.803+0000 7fee6d7fa640 1 -- 172.21.15.12:0/1192306896 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fee68083d30 con 0x7fee70156f80 2023-10-30T12:38:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.807+0000 7fee74bce640 1 --2- 172.21.15.12:0/1192306896 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fee540664d0 0x7fee54068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.807+0000 7fee74bce640 1 --2- 172.21.15.12:0/1192306896 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fee540664d0 0x7fee54068990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fee64009ac0 tx=0x7fee64009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.811+0000 7fee6d7fa640 1 -- 172.21.15.12:0/1192306896 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fee68051060 con 0x7fee70156f80 2023-10-30T12:38:21.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.999+0000 7fee76e32640 1 -- 172.21.15.12:0/1192306896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee700af8e0 con 0x7fee70156f80 2023-10-30T12:38:21.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.999+0000 7fee6d7fa640 1 -- 172.21.15.12:0/1192306896 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5738 (secure 0 0 0) 0x7fee68054680 con 0x7fee70156f80 2023-10-30T12:38:21.003 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:38:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:38:16.238773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:active","state_seq":53,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:38:21.003 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 20 2023-10-30T12:38:21.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.999+0000 7fee76e32640 1 -- 172.21.15.12:0/1192306896 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fee540664d0 msgr2=0x7fee54068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:21.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.999+0000 7fee76e32640 1 --2- 172.21.15.12:0/1192306896 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fee540664d0 0x7fee54068990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fee64009ac0 tx=0x7fee64009190 comp rx=0 tx=0).stop 2023-10-30T12:38:21.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.999+0000 7fee76e32640 1 -- 172.21.15.12:0/1192306896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee70156f80 msgr2=0x7fee700b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:21.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.999+0000 7fee76e32640 1 --2- 172.21.15.12:0/1192306896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee70156f80 0x7fee700b00c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fee6800c770 tx=0x7fee6800cc30 comp rx=0 tx=0).stop 2023-10-30T12:38:21.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.999+0000 7fee76e32640 1 -- 172.21.15.12:0/1192306896 shutdown_connections 2023-10-30T12:38:21.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.999+0000 7fee76e32640 1 --2- 172.21.15.12:0/1192306896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee70156f80 0x7fee700b00c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:21.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.999+0000 7fee76e32640 1 --2- 172.21.15.12:0/1192306896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee70070c70 0x7fee700afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:21.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.999+0000 7fee76e32640 1 --2- 172.21.15.12:0/1192306896 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fee540664d0 0x7fee54068990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:21.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.999+0000 7fee76e32640 1 --2- 172.21.15.12:0/1192306896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee70070650 0x7fee7015cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.999+0000 7fee76e32640 1 -- 172.21.15.12:0/1192306896 >> 172.21.15.12:0/1192306896 conn(0x7fee700cbeb0 msgr2=0x7fee700b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.999+0000 7fee76e32640 1 -- 172.21.15.12:0/1192306896 shutdown_connections 2023-10-30T12:38:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:20.999+0000 7fee76e32640 1 -- 172.21.15.12:0/1192306896 wait complete. 2023-10-30T12:38:21.024 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2023-10-30T12:38:23.025 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:38:23.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad9179d640 1 Processor -- start 2023-10-30T12:38:23.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad9179d640 1 -- start start 2023-10-30T12:38:23.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad9179d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fad8c0c6b80 0x7fad8c0bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:23.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad9179d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fad8c13f4a0 0x7fad8c0c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:23.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad9179d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8c0c0a50 con 0x7fad8c0c6f60 2023-10-30T12:38:23.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad9179d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8c06f6e0 con 0x7fad8c0c6b80 2023-10-30T12:38:23.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad9179d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8c06f9c0 con 0x7fad8c13f4a0 2023-10-30T12:38:23.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad8affd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fad8c0c6b80 0x7fad8c0bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:23.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad83fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fad8c0c6f60 0x7fad8c13ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:47206/0 (socket says 172.21.15.12:47206) 2023-10-30T12:38:23.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad83fff640 1 -- 172.21.15.12:0/1015721609 learned_addr learned my addr 172.21.15.12:0/1015721609 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:38:23.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad8b7fe640 1 --2- 172.21.15.12:0/1015721609 >> v2:172.21.15.154:3300/0 conn(0x7fad8c13f4a0 0x7fad8c0c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:23.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad8affd640 1 -- 172.21.15.12:0/1015721609 >> v2:172.21.15.154:3300/0 conn(0x7fad8c13f4a0 msgr2=0x7fad8c0c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:38:23.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad8affd640 1 --2- 172.21.15.12:0/1015721609 >> v2:172.21.15.154:3300/0 conn(0x7fad8c13f4a0 0x7fad8c0c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:23.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad8affd640 1 -- 172.21.15.12:0/1015721609 >> v1:172.21.15.12:6789/0 conn(0x7fad8c0c6f60 legacy=0x7fad8c13ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:23.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad8affd640 1 -- 172.21.15.12:0/1015721609 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c06fca0 con 0x7fad8c0c6b80 2023-10-30T12:38:23.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.167+0000 7fad8a7fc640 1 -- 172.21.15.12:0/1015721609 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1610428875 0 0) 0x7fad8c0c0a50 con 0x7fad8c0c6f60 2023-10-30T12:38:23.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.171+0000 7fad8affd640 1 --2- 172.21.15.12:0/1015721609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad8c0c6b80 0x7fad8c0bfec0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0c8980 tx=0x7fad7c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5428142055e3cfd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:23.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.171+0000 7fad8a7fc640 1 -- 172.21.15.12:0/1015721609 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0ab600 con 0x7fad8c0c6b80 2023-10-30T12:38:23.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.171+0000 7fad8a7fc640 1 -- 172.21.15.12:0/1015721609 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c0a7050 con 0x7fad8c0c6b80 2023-10-30T12:38:23.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.171+0000 7fad8a7fc640 1 -- 172.21.15.12:0/1015721609 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0ab980 con 0x7fad8c0c6b80 2023-10-30T12:38:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.171+0000 7fad9179d640 1 -- 172.21.15.12:0/1015721609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad8c0c6b80 msgr2=0x7fad8c0bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.171+0000 7fad9179d640 1 --2- 172.21.15.12:0/1015721609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad8c0c6b80 0x7fad8c0bfec0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0c8980 tx=0x7fad7c01c460 comp rx=0 tx=0).stop 2023-10-30T12:38:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.171+0000 7fad9179d640 1 -- 172.21.15.12:0/1015721609 shutdown_connections 2023-10-30T12:38:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.171+0000 7fad9179d640 1 --2- 172.21.15.12:0/1015721609 >> v2:172.21.15.154:3300/0 conn(0x7fad8c13f4a0 0x7fad8c0c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.171+0000 7fad9179d640 1 --2- 172.21.15.12:0/1015721609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad8c0c6b80 0x7fad8c0bfec0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:23.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.171+0000 7fad9179d640 1 -- 172.21.15.12:0/1015721609 >> 172.21.15.12:0/1015721609 conn(0x7fad8c0c7910 msgr2=0x7fad8c0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:23.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.171+0000 7fad9179d640 1 -- 172.21.15.12:0/1015721609 shutdown_connections 2023-10-30T12:38:23.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.171+0000 7fad9179d640 1 -- 172.21.15.12:0/1015721609 wait complete. 2023-10-30T12:38:23.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad9179d640 1 Processor -- start 2023-10-30T12:38:23.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad9179d640 1 -- start start 2023-10-30T12:38:23.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad9179d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad8c0c6b80 0x7fad8c158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad9179d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad8c0c6f60 0x7fad8c154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad9179d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad8c13f4a0 0x7fad8c154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad9179d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0cac90 con 0x7fad8c0c6f60 2023-10-30T12:38:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad9179d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c151080 con 0x7fad8c13f4a0 2023-10-30T12:38:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad9179d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0ca9b0 con 0x7fad8c0c6b80 2023-10-30T12:38:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad83fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad8c0c6f60 0x7fad8c154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad83fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad8c0c6f60 0x7fad8c154710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50130/0 (socket says 172.21.15.12:50130) 2023-10-30T12:38:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad83fff640 1 -- 172.21.15.12:0/777575051 learned_addr learned my addr 172.21.15.12:0/777575051 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:38:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad8b7fe640 1 --2- 172.21.15.12:0/777575051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad8c13f4a0 0x7fad8c154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad8affd640 1 --2- 172.21.15.12:0/777575051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad8c0c6b80 0x7fad8c158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad83fff640 1 -- 172.21.15.12:0/777575051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad8c0c6b80 msgr2=0x7fad8c158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad83fff640 1 --2- 172.21.15.12:0/777575051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad8c0c6b80 0x7fad8c158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad83fff640 1 -- 172.21.15.12:0/777575051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad8c13f4a0 msgr2=0x7fad8c154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad83fff640 1 --2- 172.21.15.12:0/777575051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad8c13f4a0 0x7fad8c154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad83fff640 1 -- 172.21.15.12:0/777575051 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c009420 con 0x7fad8c0c6f60 2023-10-30T12:38:23.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad83fff640 1 --2- 172.21.15.12:0/777575051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad8c0c6f60 0x7fad8c154710 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fad740011f0 tx=0x7fad7400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:23.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad837fe640 1 -- 172.21.15.12:0/777575051 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad740074e0 con 0x7fad8c0c6f60 2023-10-30T12:38:23.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad837fe640 1 -- 172.21.15.12:0/777575051 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad74007640 con 0x7fad8c0c6f60 2023-10-30T12:38:23.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad837fe640 1 -- 172.21.15.12:0/777575051 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74004a10 con 0x7fad8c0c6f60 2023-10-30T12:38:23.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad9179d640 1 -- 172.21.15.12:0/777575051 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8c151320 con 0x7fad8c0c6f60 2023-10-30T12:38:23.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad9179d640 1 -- 172.21.15.12:0/777575051 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c1518e0 con 0x7fad8c0c6f60 2023-10-30T12:38:23.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.175+0000 7fad9179d640 1 -- 172.21.15.12:0/777575051 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8c1544a0 con 0x7fad8c0c6f60 2023-10-30T12:38:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.179+0000 7fad837fe640 1 -- 172.21.15.12:0/777575051 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fad74004b70 con 0x7fad8c0c6f60 2023-10-30T12:38:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.179+0000 7fad837fe640 1 --2- 172.21.15.12:0/777575051 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fad6c0664a0 0x7fad6c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:38:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.179+0000 7fad837fe640 1 -- 172.21.15.12:0/777575051 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fad74083190 con 0x7fad8c0c6f60 2023-10-30T12:38:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.179+0000 7fad8affd640 1 --2- 172.21.15.12:0/777575051 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fad6c0664a0 0x7fad6c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:38:23.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.179+0000 7fad8affd640 1 --2- 172.21.15.12:0/777575051 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fad6c0664a0 0x7fad6c068960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0ad000 tx=0x7fad7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:38:23.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.183+0000 7fad837fe640 1 -- 172.21.15.12:0/777575051 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fad740504a0 con 0x7fad8c0c6f60 2023-10-30T12:38:23.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.375+0000 7fad9179d640 1 -- 172.21.15.12:0/777575051 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad8c04e170 con 0x7fad8c0c6f60 2023-10-30T12:38:23.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.375+0000 7fad837fe640 1 -- 172.21.15.12:0/777575051 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+6529 (secure 0 0 0) 0x7fad74053ac0 con 0x7fad8c0c6f60 2023-10-30T12:38:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:38:23.380 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4971,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:38:16.238773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:active","state_seq":53,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:38:23.380 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2023-10-30T12:38:23.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.375+0000 7fad9179d640 1 -- 172.21.15.12:0/777575051 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fad6c0664a0 msgr2=0x7fad6c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:23.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.375+0000 7fad9179d640 1 --2- 172.21.15.12:0/777575051 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fad6c0664a0 0x7fad6c068960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0ad000 tx=0x7fad7c008040 comp rx=0 tx=0).stop 2023-10-30T12:38:23.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.375+0000 7fad9179d640 1 -- 172.21.15.12:0/777575051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad8c0c6f60 msgr2=0x7fad8c154710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:38:23.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.375+0000 7fad9179d640 1 --2- 172.21.15.12:0/777575051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad8c0c6f60 0x7fad8c154710 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fad740011f0 tx=0x7fad7400c9b0 comp rx=0 tx=0).stop 2023-10-30T12:38:23.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.375+0000 7fad9179d640 1 -- 172.21.15.12:0/777575051 shutdown_connections 2023-10-30T12:38:23.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.375+0000 7fad9179d640 1 --2- 172.21.15.12:0/777575051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad8c13f4a0 0x7fad8c154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:23.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.375+0000 7fad9179d640 1 --2- 172.21.15.12:0/777575051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad8c0c6f60 0x7fad8c154710 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:23.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.375+0000 7fad9179d640 1 --2- 172.21.15.12:0/777575051 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fad6c0664a0 0x7fad6c068960 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:23.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.375+0000 7fad9179d640 1 --2- 172.21.15.12:0/777575051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fad8c0c6b80 0x7fad8c158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:38:23.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.375+0000 7fad9179d640 1 -- 172.21.15.12:0/777575051 >> 172.21.15.12:0/777575051 conn(0x7fad8c0c7910 msgr2=0x7fad8c13ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:38:23.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.375+0000 7fad9179d640 1 -- 172.21.15.12:0/777575051 shutdown_connections 2023-10-30T12:38:23.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:38:23.375+0000 7fad9179d640 1 -- 172.21.15.12:0/777575051 wait complete. 2023-10-30T12:38:23.400 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2023-10-30T12:38:23.400 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 119 secs before thrashing 2023-10-30T12:38:23.492 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/12' -> 'copyofsnap1/test1/120/12' 2023-10-30T12:38:23.492 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/13' -> 'copyofsnap1/test1/120/13' 2023-10-30T12:38:23.493 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/14' -> 'copyofsnap1/test1/120/14' 2023-10-30T12:38:23.493 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/15' -> 'copyofsnap1/test1/120/15' 2023-10-30T12:38:23.493 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/16' -> 'copyofsnap1/test1/120/16' 2023-10-30T12:38:23.493 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/17' -> 'copyofsnap1/test1/120/17' 2023-10-30T12:38:23.493 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/18' -> 'copyofsnap1/test1/120/18' 2023-10-30T12:38:23.494 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/19' -> 'copyofsnap1/test1/120/19' 2023-10-30T12:38:23.494 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/20' -> 'copyofsnap1/test1/120/20' 2023-10-30T12:38:23.494 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/10' -> 'copyofsnap1/test1/120/10' 2023-10-30T12:38:23.494 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121' -> 'copyofsnap1/test1/121' 2023-10-30T12:38:23.494 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/11' -> 'copyofsnap1/test1/121/11' 2023-10-30T12:38:23.495 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/12' -> 'copyofsnap1/test1/121/12' 2023-10-30T12:38:23.495 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/13' -> 'copyofsnap1/test1/121/13' 2023-10-30T12:38:23.495 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/14' -> 'copyofsnap1/test1/121/14' 2023-10-30T12:38:23.495 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/15' -> 'copyofsnap1/test1/121/15' 2023-10-30T12:38:23.495 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/16' -> 'copyofsnap1/test1/121/16' 2023-10-30T12:38:23.496 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/17' -> 'copyofsnap1/test1/121/17' 2023-10-30T12:38:23.496 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/18' -> 'copyofsnap1/test1/121/18' 2023-10-30T12:38:23.496 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/19' -> 'copyofsnap1/test1/121/19' 2023-10-30T12:38:23.496 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/20' -> 'copyofsnap1/test1/121/20' 2023-10-30T12:38:23.496 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/10' -> 'copyofsnap1/test1/121/10' 2023-10-30T12:38:23.496 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122' -> 'copyofsnap1/test1/122' 2023-10-30T12:38:23.497 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/11' -> 'copyofsnap1/test1/122/11' 2023-10-30T12:38:23.497 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/12' -> 'copyofsnap1/test1/122/12' 2023-10-30T12:38:23.497 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/13' -> 'copyofsnap1/test1/122/13' 2023-10-30T12:38:23.497 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/14' -> 'copyofsnap1/test1/122/14' 2023-10-30T12:38:23.497 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/15' -> 'copyofsnap1/test1/122/15' 2023-10-30T12:38:23.498 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/16' -> 'copyofsnap1/test1/122/16' 2023-10-30T12:38:23.498 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/17' -> 'copyofsnap1/test1/122/17' 2023-10-30T12:38:23.498 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/18' -> 'copyofsnap1/test1/122/18' 2023-10-30T12:38:23.498 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/19' -> 'copyofsnap1/test1/122/19' 2023-10-30T12:38:23.498 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/20' -> 'copyofsnap1/test1/122/20' 2023-10-30T12:38:23.499 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/10' -> 'copyofsnap1/test1/122/10' 2023-10-30T12:38:23.499 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123' -> 'copyofsnap1/test1/123' 2023-10-30T12:38:23.499 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/11' -> 'copyofsnap1/test1/123/11' 2023-10-30T12:38:23.499 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/12' -> 'copyofsnap1/test1/123/12' 2023-10-30T12:38:23.499 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/13' -> 'copyofsnap1/test1/123/13' 2023-10-30T12:38:23.499 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/14' -> 'copyofsnap1/test1/123/14' 2023-10-30T12:38:23.500 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/15' -> 'copyofsnap1/test1/123/15' 2023-10-30T12:38:23.500 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/16' -> 'copyofsnap1/test1/123/16' 2023-10-30T12:38:23.500 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/17' -> 'copyofsnap1/test1/123/17' 2023-10-30T12:38:23.500 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/18' -> 'copyofsnap1/test1/123/18' 2023-10-30T12:38:23.500 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/19' -> 'copyofsnap1/test1/123/19' 2023-10-30T12:38:23.501 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/20' -> 'copyofsnap1/test1/123/20' 2023-10-30T12:38:23.501 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/10' -> 'copyofsnap1/test1/123/10' 2023-10-30T12:38:23.501 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124' -> 'copyofsnap1/test1/124' 2023-10-30T12:38:23.501 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/11' -> 'copyofsnap1/test1/124/11' 2023-10-30T12:38:23.501 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/12' -> 'copyofsnap1/test1/124/12' 2023-10-30T12:38:23.502 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/13' -> 'copyofsnap1/test1/124/13' 2023-10-30T12:38:23.502 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/14' -> 'copyofsnap1/test1/124/14' 2023-10-30T12:38:23.502 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/15' -> 'copyofsnap1/test1/124/15' 2023-10-30T12:38:23.502 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/16' -> 'copyofsnap1/test1/124/16' 2023-10-30T12:38:23.502 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/17' -> 'copyofsnap1/test1/124/17' 2023-10-30T12:38:23.503 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/18' -> 'copyofsnap1/test1/124/18' 2023-10-30T12:38:23.503 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/19' -> 'copyofsnap1/test1/124/19' 2023-10-30T12:38:23.503 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/20' -> 'copyofsnap1/test1/124/20' 2023-10-30T12:38:23.503 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/10' -> 'copyofsnap1/test1/124/10' 2023-10-30T12:38:23.503 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125' -> 'copyofsnap1/test1/125' 2023-10-30T12:38:23.503 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/11' -> 'copyofsnap1/test1/125/11' 2023-10-30T12:38:23.504 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/12' -> 'copyofsnap1/test1/125/12' 2023-10-30T12:38:23.504 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/13' -> 'copyofsnap1/test1/125/13' 2023-10-30T12:38:23.504 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/14' -> 'copyofsnap1/test1/125/14' 2023-10-30T12:38:23.504 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/15' -> 'copyofsnap1/test1/125/15' 2023-10-30T12:38:23.504 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/16' -> 'copyofsnap1/test1/125/16' 2023-10-30T12:38:23.504 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/17' -> 'copyofsnap1/test1/125/17' 2023-10-30T12:38:23.505 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/18' -> 'copyofsnap1/test1/125/18' 2023-10-30T12:38:23.505 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/19' -> 'copyofsnap1/test1/125/19' 2023-10-30T12:38:23.505 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/20' -> 'copyofsnap1/test1/125/20' 2023-10-30T12:38:23.505 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/10' -> 'copyofsnap1/test1/125/10' 2023-10-30T12:38:23.505 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126' -> 'copyofsnap1/test1/126' 2023-10-30T12:38:23.505 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/11' -> 'copyofsnap1/test1/126/11' 2023-10-30T12:38:23.506 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/12' -> 'copyofsnap1/test1/126/12' 2023-10-30T12:38:23.506 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/13' -> 'copyofsnap1/test1/126/13' 2023-10-30T12:38:23.506 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/14' -> 'copyofsnap1/test1/126/14' 2023-10-30T12:38:23.506 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/15' -> 'copyofsnap1/test1/126/15' 2023-10-30T12:38:23.506 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/16' -> 'copyofsnap1/test1/126/16' 2023-10-30T12:38:23.507 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/17' -> 'copyofsnap1/test1/126/17' 2023-10-30T12:38:23.507 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/18' -> 'copyofsnap1/test1/126/18' 2023-10-30T12:38:23.507 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/19' -> 'copyofsnap1/test1/126/19' 2023-10-30T12:38:23.507 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/20' -> 'copyofsnap1/test1/126/20' 2023-10-30T12:38:24.009 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/10' -> 'copyofsnap1/test1/126/10' 2023-10-30T12:38:24.010 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127' -> 'copyofsnap1/test1/127' 2023-10-30T12:38:24.010 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/11' -> 'copyofsnap1/test1/127/11' 2023-10-30T12:38:24.010 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/12' -> 'copyofsnap1/test1/127/12' 2023-10-30T12:38:24.010 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/13' -> 'copyofsnap1/test1/127/13' 2023-10-30T12:38:24.010 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/14' -> 'copyofsnap1/test1/127/14' 2023-10-30T12:38:24.011 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/15' -> 'copyofsnap1/test1/127/15' 2023-10-30T12:38:24.011 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/16' -> 'copyofsnap1/test1/127/16' 2023-10-30T12:38:24.011 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/17' -> 'copyofsnap1/test1/127/17' 2023-10-30T12:38:24.011 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/18' -> 'copyofsnap1/test1/127/18' 2023-10-30T12:38:24.011 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/19' -> 'copyofsnap1/test1/127/19' 2023-10-30T12:38:24.012 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/20' -> 'copyofsnap1/test1/127/20' 2023-10-30T12:38:24.012 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/10' -> 'copyofsnap1/test1/127/10' 2023-10-30T12:38:24.012 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128' -> 'copyofsnap1/test1/128' 2023-10-30T12:38:24.012 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/11' -> 'copyofsnap1/test1/128/11' 2023-10-30T12:38:24.012 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/12' -> 'copyofsnap1/test1/128/12' 2023-10-30T12:38:24.013 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/13' -> 'copyofsnap1/test1/128/13' 2023-10-30T12:38:24.013 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/14' -> 'copyofsnap1/test1/128/14' 2023-10-30T12:38:24.013 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/15' -> 'copyofsnap1/test1/128/15' 2023-10-30T12:38:24.013 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/16' -> 'copyofsnap1/test1/128/16' 2023-10-30T12:38:24.013 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/17' -> 'copyofsnap1/test1/128/17' 2023-10-30T12:38:24.013 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/18' -> 'copyofsnap1/test1/128/18' 2023-10-30T12:38:24.014 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/19' -> 'copyofsnap1/test1/128/19' 2023-10-30T12:38:24.014 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/20' -> 'copyofsnap1/test1/128/20' 2023-10-30T12:38:24.014 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/10' -> 'copyofsnap1/test1/128/10' 2023-10-30T12:38:24.014 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129' -> 'copyofsnap1/test1/129' 2023-10-30T12:38:24.014 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/11' -> 'copyofsnap1/test1/129/11' 2023-10-30T12:38:24.015 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/12' -> 'copyofsnap1/test1/129/12' 2023-10-30T12:38:24.015 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/13' -> 'copyofsnap1/test1/129/13' 2023-10-30T12:38:24.015 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/14' -> 'copyofsnap1/test1/129/14' 2023-10-30T12:38:24.015 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/15' -> 'copyofsnap1/test1/129/15' 2023-10-30T12:38:24.015 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/16' -> 'copyofsnap1/test1/129/16' 2023-10-30T12:38:24.016 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/17' -> 'copyofsnap1/test1/129/17' 2023-10-30T12:38:24.016 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/18' -> 'copyofsnap1/test1/129/18' 2023-10-30T12:38:24.016 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/19' -> 'copyofsnap1/test1/129/19' 2023-10-30T12:38:24.016 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/20' -> 'copyofsnap1/test1/129/20' 2023-10-30T12:38:24.016 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/10' -> 'copyofsnap1/test1/129/10' 2023-10-30T12:38:24.016 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130' -> 'copyofsnap1/test1/130' 2023-10-30T12:38:24.017 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/11' -> 'copyofsnap1/test1/130/11' 2023-10-30T12:38:24.017 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/12' -> 'copyofsnap1/test1/130/12' 2023-10-30T12:38:24.017 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/13' -> 'copyofsnap1/test1/130/13' 2023-10-30T12:38:24.017 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/14' -> 'copyofsnap1/test1/130/14' 2023-10-30T12:38:24.017 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/15' -> 'copyofsnap1/test1/130/15' 2023-10-30T12:38:24.018 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/16' -> 'copyofsnap1/test1/130/16' 2023-10-30T12:38:24.018 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/17' -> 'copyofsnap1/test1/130/17' 2023-10-30T12:38:24.018 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/18' -> 'copyofsnap1/test1/130/18' 2023-10-30T12:38:24.018 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/19' -> 'copyofsnap1/test1/130/19' 2023-10-30T12:38:24.018 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/20' -> 'copyofsnap1/test1/130/20' 2023-10-30T12:38:24.019 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/10' -> 'copyofsnap1/test1/130/10' 2023-10-30T12:38:24.019 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131' -> 'copyofsnap1/test1/131' 2023-10-30T12:38:24.019 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/11' -> 'copyofsnap1/test1/131/11' 2023-10-30T12:38:24.019 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/12' -> 'copyofsnap1/test1/131/12' 2023-10-30T12:38:24.019 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/13' -> 'copyofsnap1/test1/131/13' 2023-10-30T12:38:24.020 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/14' -> 'copyofsnap1/test1/131/14' 2023-10-30T12:38:24.020 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/15' -> 'copyofsnap1/test1/131/15' 2023-10-30T12:38:24.020 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/16' -> 'copyofsnap1/test1/131/16' 2023-10-30T12:38:24.020 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/17' -> 'copyofsnap1/test1/131/17' 2023-10-30T12:38:24.020 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/18' -> 'copyofsnap1/test1/131/18' 2023-10-30T12:38:24.020 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/19' -> 'copyofsnap1/test1/131/19' 2023-10-30T12:38:24.021 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/20' -> 'copyofsnap1/test1/131/20' 2023-10-30T12:38:24.021 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/10' -> 'copyofsnap1/test1/131/10' 2023-10-30T12:38:24.021 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132' -> 'copyofsnap1/test1/132' 2023-10-30T12:38:24.021 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/11' -> 'copyofsnap1/test1/132/11' 2023-10-30T12:38:24.021 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/12' -> 'copyofsnap1/test1/132/12' 2023-10-30T12:38:24.022 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/13' -> 'copyofsnap1/test1/132/13' 2023-10-30T12:38:24.022 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/14' -> 'copyofsnap1/test1/132/14' 2023-10-30T12:38:24.022 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/15' -> 'copyofsnap1/test1/132/15' 2023-10-30T12:38:24.022 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/16' -> 'copyofsnap1/test1/132/16' 2023-10-30T12:38:24.022 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/17' -> 'copyofsnap1/test1/132/17' 2023-10-30T12:38:24.023 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/18' -> 'copyofsnap1/test1/132/18' 2023-10-30T12:38:24.023 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/19' -> 'copyofsnap1/test1/132/19' 2023-10-30T12:38:24.023 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/20' -> 'copyofsnap1/test1/132/20' 2023-10-30T12:38:24.023 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/10' -> 'copyofsnap1/test1/132/10' 2023-10-30T12:38:24.023 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133' -> 'copyofsnap1/test1/133' 2023-10-30T12:38:24.024 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/11' -> 'copyofsnap1/test1/133/11' 2023-10-30T12:38:24.024 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/12' -> 'copyofsnap1/test1/133/12' 2023-10-30T12:38:24.024 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/13' -> 'copyofsnap1/test1/133/13' 2023-10-30T12:38:24.024 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/14' -> 'copyofsnap1/test1/133/14' 2023-10-30T12:38:24.024 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/15' -> 'copyofsnap1/test1/133/15' 2023-10-30T12:38:24.024 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/16' -> 'copyofsnap1/test1/133/16' 2023-10-30T12:38:24.025 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/17' -> 'copyofsnap1/test1/133/17' 2023-10-30T12:38:25.487 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/18' -> 'copyofsnap1/test1/133/18' 2023-10-30T12:38:25.488 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/19' -> 'copyofsnap1/test1/133/19' 2023-10-30T12:38:25.488 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/20' -> 'copyofsnap1/test1/133/20' 2023-10-30T12:38:25.488 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/10' -> 'copyofsnap1/test1/133/10' 2023-10-30T12:38:25.488 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134' -> 'copyofsnap1/test1/134' 2023-10-30T12:38:25.489 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/11' -> 'copyofsnap1/test1/134/11' 2023-10-30T12:38:25.489 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/12' -> 'copyofsnap1/test1/134/12' 2023-10-30T12:38:25.489 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/13' -> 'copyofsnap1/test1/134/13' 2023-10-30T12:38:25.489 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/14' -> 'copyofsnap1/test1/134/14' 2023-10-30T12:38:25.489 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/15' -> 'copyofsnap1/test1/134/15' 2023-10-30T12:38:25.489 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/16' -> 'copyofsnap1/test1/134/16' 2023-10-30T12:38:25.490 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/17' -> 'copyofsnap1/test1/134/17' 2023-10-30T12:38:25.490 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/18' -> 'copyofsnap1/test1/134/18' 2023-10-30T12:38:25.490 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/19' -> 'copyofsnap1/test1/134/19' 2023-10-30T12:38:25.490 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/20' -> 'copyofsnap1/test1/134/20' 2023-10-30T12:38:25.491 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/10' -> 'copyofsnap1/test1/134/10' 2023-10-30T12:38:25.491 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135' -> 'copyofsnap1/test1/135' 2023-10-30T12:38:25.491 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/11' -> 'copyofsnap1/test1/135/11' 2023-10-30T12:38:25.491 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/12' -> 'copyofsnap1/test1/135/12' 2023-10-30T12:38:25.491 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/13' -> 'copyofsnap1/test1/135/13' 2023-10-30T12:38:25.492 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/14' -> 'copyofsnap1/test1/135/14' 2023-10-30T12:38:25.492 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/15' -> 'copyofsnap1/test1/135/15' 2023-10-30T12:38:25.492 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/16' -> 'copyofsnap1/test1/135/16' 2023-10-30T12:38:25.492 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/17' -> 'copyofsnap1/test1/135/17' 2023-10-30T12:38:25.492 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/18' -> 'copyofsnap1/test1/135/18' 2023-10-30T12:38:25.492 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/19' -> 'copyofsnap1/test1/135/19' 2023-10-30T12:38:25.493 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/20' -> 'copyofsnap1/test1/135/20' 2023-10-30T12:38:25.493 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/10' -> 'copyofsnap1/test1/135/10' 2023-10-30T12:38:25.493 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136' -> 'copyofsnap1/test1/136' 2023-10-30T12:38:25.493 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/11' -> 'copyofsnap1/test1/136/11' 2023-10-30T12:38:25.493 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/12' -> 'copyofsnap1/test1/136/12' 2023-10-30T12:38:25.494 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/13' -> 'copyofsnap1/test1/136/13' 2023-10-30T12:38:25.494 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/14' -> 'copyofsnap1/test1/136/14' 2023-10-30T12:38:25.494 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/15' -> 'copyofsnap1/test1/136/15' 2023-10-30T12:38:25.494 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/16' -> 'copyofsnap1/test1/136/16' 2023-10-30T12:38:25.494 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/17' -> 'copyofsnap1/test1/136/17' 2023-10-30T12:38:25.495 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/18' -> 'copyofsnap1/test1/136/18' 2023-10-30T12:38:25.495 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/19' -> 'copyofsnap1/test1/136/19' 2023-10-30T12:38:25.495 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/20' -> 'copyofsnap1/test1/136/20' 2023-10-30T12:38:25.495 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/10' -> 'copyofsnap1/test1/136/10' 2023-10-30T12:38:25.495 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137' -> 'copyofsnap1/test1/137' 2023-10-30T12:38:25.496 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/11' -> 'copyofsnap1/test1/137/11' 2023-10-30T12:38:25.496 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/12' -> 'copyofsnap1/test1/137/12' 2023-10-30T12:38:25.496 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/13' -> 'copyofsnap1/test1/137/13' 2023-10-30T12:38:25.496 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/14' -> 'copyofsnap1/test1/137/14' 2023-10-30T12:38:25.496 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/15' -> 'copyofsnap1/test1/137/15' 2023-10-30T12:38:25.497 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/16' -> 'copyofsnap1/test1/137/16' 2023-10-30T12:38:25.497 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/17' -> 'copyofsnap1/test1/137/17' 2023-10-30T12:38:25.497 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/18' -> 'copyofsnap1/test1/137/18' 2023-10-30T12:38:25.497 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/19' -> 'copyofsnap1/test1/137/19' 2023-10-30T12:38:25.497 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/20' -> 'copyofsnap1/test1/137/20' 2023-10-30T12:38:25.497 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/10' -> 'copyofsnap1/test1/137/10' 2023-10-30T12:38:25.498 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138' -> 'copyofsnap1/test1/138' 2023-10-30T12:38:25.498 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/11' -> 'copyofsnap1/test1/138/11' 2023-10-30T12:38:25.498 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/12' -> 'copyofsnap1/test1/138/12' 2023-10-30T12:38:25.498 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/13' -> 'copyofsnap1/test1/138/13' 2023-10-30T12:38:25.498 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/14' -> 'copyofsnap1/test1/138/14' 2023-10-30T12:38:25.499 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/15' -> 'copyofsnap1/test1/138/15' 2023-10-30T12:38:25.499 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/16' -> 'copyofsnap1/test1/138/16' 2023-10-30T12:38:25.499 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/17' -> 'copyofsnap1/test1/138/17' 2023-10-30T12:38:25.499 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/18' -> 'copyofsnap1/test1/138/18' 2023-10-30T12:38:25.499 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/19' -> 'copyofsnap1/test1/138/19' 2023-10-30T12:38:25.500 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/20' -> 'copyofsnap1/test1/138/20' 2023-10-30T12:38:25.500 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/10' -> 'copyofsnap1/test1/138/10' 2023-10-30T12:38:25.500 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139' -> 'copyofsnap1/test1/139' 2023-10-30T12:38:25.500 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/11' -> 'copyofsnap1/test1/139/11' 2023-10-30T12:38:25.500 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/12' -> 'copyofsnap1/test1/139/12' 2023-10-30T12:38:25.501 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/13' -> 'copyofsnap1/test1/139/13' 2023-10-30T12:38:25.501 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/14' -> 'copyofsnap1/test1/139/14' 2023-10-30T12:38:25.501 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/15' -> 'copyofsnap1/test1/139/15' 2023-10-30T12:38:25.501 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/16' -> 'copyofsnap1/test1/139/16' 2023-10-30T12:38:25.501 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/17' -> 'copyofsnap1/test1/139/17' 2023-10-30T12:38:25.502 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/18' -> 'copyofsnap1/test1/139/18' 2023-10-30T12:38:25.502 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/19' -> 'copyofsnap1/test1/139/19' 2023-10-30T12:38:25.502 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/20' -> 'copyofsnap1/test1/139/20' 2023-10-30T12:38:25.502 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/10' -> 'copyofsnap1/test1/139/10' 2023-10-30T12:38:25.502 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140' -> 'copyofsnap1/test1/140' 2023-10-30T12:38:25.502 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/11' -> 'copyofsnap1/test1/140/11' 2023-10-30T12:38:25.503 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/12' -> 'copyofsnap1/test1/140/12' 2023-10-30T12:38:25.503 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/13' -> 'copyofsnap1/test1/140/13' 2023-10-30T12:38:25.503 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/14' -> 'copyofsnap1/test1/140/14' 2023-10-30T12:38:26.108 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/15' -> 'copyofsnap1/test1/140/15' 2023-10-30T12:38:26.109 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/16' -> 'copyofsnap1/test1/140/16' 2023-10-30T12:38:26.109 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/17' -> 'copyofsnap1/test1/140/17' 2023-10-30T12:38:26.109 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/18' -> 'copyofsnap1/test1/140/18' 2023-10-30T12:38:26.109 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/19' -> 'copyofsnap1/test1/140/19' 2023-10-30T12:38:26.109 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/20' -> 'copyofsnap1/test1/140/20' 2023-10-30T12:38:26.110 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/10' -> 'copyofsnap1/test1/140/10' 2023-10-30T12:38:26.110 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141' -> 'copyofsnap1/test1/141' 2023-10-30T12:38:26.110 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/11' -> 'copyofsnap1/test1/141/11' 2023-10-30T12:38:26.110 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/12' -> 'copyofsnap1/test1/141/12' 2023-10-30T12:38:26.111 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/13' -> 'copyofsnap1/test1/141/13' 2023-10-30T12:38:26.111 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/14' -> 'copyofsnap1/test1/141/14' 2023-10-30T12:38:26.111 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/15' -> 'copyofsnap1/test1/141/15' 2023-10-30T12:38:26.111 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/16' -> 'copyofsnap1/test1/141/16' 2023-10-30T12:38:26.111 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/17' -> 'copyofsnap1/test1/141/17' 2023-10-30T12:38:26.111 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/18' -> 'copyofsnap1/test1/141/18' 2023-10-30T12:38:26.112 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/19' -> 'copyofsnap1/test1/141/19' 2023-10-30T12:38:26.112 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/20' -> 'copyofsnap1/test1/141/20' 2023-10-30T12:38:26.112 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/10' -> 'copyofsnap1/test1/141/10' 2023-10-30T12:38:26.112 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142' -> 'copyofsnap1/test1/142' 2023-10-30T12:38:26.112 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/11' -> 'copyofsnap1/test1/142/11' 2023-10-30T12:38:26.113 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/12' -> 'copyofsnap1/test1/142/12' 2023-10-30T12:38:26.113 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/13' -> 'copyofsnap1/test1/142/13' 2023-10-30T12:38:26.113 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/14' -> 'copyofsnap1/test1/142/14' 2023-10-30T12:38:26.113 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/15' -> 'copyofsnap1/test1/142/15' 2023-10-30T12:38:26.113 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/16' -> 'copyofsnap1/test1/142/16' 2023-10-30T12:38:26.114 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/17' -> 'copyofsnap1/test1/142/17' 2023-10-30T12:38:26.114 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/18' -> 'copyofsnap1/test1/142/18' 2023-10-30T12:38:26.114 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/19' -> 'copyofsnap1/test1/142/19' 2023-10-30T12:38:26.114 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/20' -> 'copyofsnap1/test1/142/20' 2023-10-30T12:38:26.114 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/10' -> 'copyofsnap1/test1/142/10' 2023-10-30T12:38:26.114 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143' -> 'copyofsnap1/test1/143' 2023-10-30T12:38:26.115 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/11' -> 'copyofsnap1/test1/143/11' 2023-10-30T12:38:26.115 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/12' -> 'copyofsnap1/test1/143/12' 2023-10-30T12:38:26.115 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/13' -> 'copyofsnap1/test1/143/13' 2023-10-30T12:38:26.115 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/14' -> 'copyofsnap1/test1/143/14' 2023-10-30T12:38:26.116 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/15' -> 'copyofsnap1/test1/143/15' 2023-10-30T12:38:26.116 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/16' -> 'copyofsnap1/test1/143/16' 2023-10-30T12:38:26.116 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/17' -> 'copyofsnap1/test1/143/17' 2023-10-30T12:38:26.116 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/18' -> 'copyofsnap1/test1/143/18' 2023-10-30T12:38:26.116 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/19' -> 'copyofsnap1/test1/143/19' 2023-10-30T12:38:26.116 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/20' -> 'copyofsnap1/test1/143/20' 2023-10-30T12:38:26.117 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/10' -> 'copyofsnap1/test1/143/10' 2023-10-30T12:38:26.117 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144' -> 'copyofsnap1/test1/144' 2023-10-30T12:38:26.117 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/11' -> 'copyofsnap1/test1/144/11' 2023-10-30T12:38:26.117 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/12' -> 'copyofsnap1/test1/144/12' 2023-10-30T12:38:26.117 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/13' -> 'copyofsnap1/test1/144/13' 2023-10-30T12:38:26.118 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/14' -> 'copyofsnap1/test1/144/14' 2023-10-30T12:38:26.118 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/15' -> 'copyofsnap1/test1/144/15' 2023-10-30T12:38:26.118 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/16' -> 'copyofsnap1/test1/144/16' 2023-10-30T12:38:26.118 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/17' -> 'copyofsnap1/test1/144/17' 2023-10-30T12:38:26.118 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/18' -> 'copyofsnap1/test1/144/18' 2023-10-30T12:38:26.119 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/19' -> 'copyofsnap1/test1/144/19' 2023-10-30T12:38:26.119 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/20' -> 'copyofsnap1/test1/144/20' 2023-10-30T12:38:26.119 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/10' -> 'copyofsnap1/test1/144/10' 2023-10-30T12:38:26.119 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145' -> 'copyofsnap1/test1/145' 2023-10-30T12:38:26.119 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/11' -> 'copyofsnap1/test1/145/11' 2023-10-30T12:38:26.119 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/12' -> 'copyofsnap1/test1/145/12' 2023-10-30T12:38:26.120 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/13' -> 'copyofsnap1/test1/145/13' 2023-10-30T12:38:26.120 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/14' -> 'copyofsnap1/test1/145/14' 2023-10-30T12:38:26.120 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/15' -> 'copyofsnap1/test1/145/15' 2023-10-30T12:38:26.120 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/16' -> 'copyofsnap1/test1/145/16' 2023-10-30T12:38:26.120 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/17' -> 'copyofsnap1/test1/145/17' 2023-10-30T12:38:26.121 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/18' -> 'copyofsnap1/test1/145/18' 2023-10-30T12:38:26.121 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/19' -> 'copyofsnap1/test1/145/19' 2023-10-30T12:38:26.121 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/20' -> 'copyofsnap1/test1/145/20' 2023-10-30T12:38:26.121 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/10' -> 'copyofsnap1/test1/145/10' 2023-10-30T12:38:26.121 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146' -> 'copyofsnap1/test1/146' 2023-10-30T12:38:26.121 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/11' -> 'copyofsnap1/test1/146/11' 2023-10-30T12:38:26.122 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/12' -> 'copyofsnap1/test1/146/12' 2023-10-30T12:38:26.122 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/13' -> 'copyofsnap1/test1/146/13' 2023-10-30T12:38:26.122 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/14' -> 'copyofsnap1/test1/146/14' 2023-10-30T12:38:26.122 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/15' -> 'copyofsnap1/test1/146/15' 2023-10-30T12:38:26.122 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/16' -> 'copyofsnap1/test1/146/16' 2023-10-30T12:38:26.123 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/17' -> 'copyofsnap1/test1/146/17' 2023-10-30T12:38:26.123 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/18' -> 'copyofsnap1/test1/146/18' 2023-10-30T12:38:26.123 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/19' -> 'copyofsnap1/test1/146/19' 2023-10-30T12:38:26.123 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/20' -> 'copyofsnap1/test1/146/20' 2023-10-30T12:38:26.123 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/10' -> 'copyofsnap1/test1/146/10' 2023-10-30T12:38:26.124 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147' -> 'copyofsnap1/test1/147' 2023-10-30T12:38:26.124 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/11' -> 'copyofsnap1/test1/147/11' 2023-10-30T12:38:26.124 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/12' -> 'copyofsnap1/test1/147/12' 2023-10-30T12:38:28.807 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/13' -> 'copyofsnap1/test1/147/13' 2023-10-30T12:38:28.807 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/14' -> 'copyofsnap1/test1/147/14' 2023-10-30T12:38:28.808 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/15' -> 'copyofsnap1/test1/147/15' 2023-10-30T12:38:28.808 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/16' -> 'copyofsnap1/test1/147/16' 2023-10-30T12:38:28.808 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/17' -> 'copyofsnap1/test1/147/17' 2023-10-30T12:38:28.808 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/18' -> 'copyofsnap1/test1/147/18' 2023-10-30T12:38:28.808 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/19' -> 'copyofsnap1/test1/147/19' 2023-10-30T12:38:28.809 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/20' -> 'copyofsnap1/test1/147/20' 2023-10-30T12:38:28.809 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/10' -> 'copyofsnap1/test1/147/10' 2023-10-30T12:38:28.809 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148' -> 'copyofsnap1/test1/148' 2023-10-30T12:38:28.809 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/11' -> 'copyofsnap1/test1/148/11' 2023-10-30T12:38:28.810 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/12' -> 'copyofsnap1/test1/148/12' 2023-10-30T12:38:28.810 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/13' -> 'copyofsnap1/test1/148/13' 2023-10-30T12:38:28.810 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/14' -> 'copyofsnap1/test1/148/14' 2023-10-30T12:38:28.810 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/15' -> 'copyofsnap1/test1/148/15' 2023-10-30T12:38:28.810 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/16' -> 'copyofsnap1/test1/148/16' 2023-10-30T12:38:28.811 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/17' -> 'copyofsnap1/test1/148/17' 2023-10-30T12:38:28.811 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/18' -> 'copyofsnap1/test1/148/18' 2023-10-30T12:38:28.811 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/19' -> 'copyofsnap1/test1/148/19' 2023-10-30T12:38:28.811 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/20' -> 'copyofsnap1/test1/148/20' 2023-10-30T12:38:28.812 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/10' -> 'copyofsnap1/test1/148/10' 2023-10-30T12:38:28.812 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149' -> 'copyofsnap1/test1/149' 2023-10-30T12:38:28.812 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/11' -> 'copyofsnap1/test1/149/11' 2023-10-30T12:38:28.812 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/12' -> 'copyofsnap1/test1/149/12' 2023-10-30T12:38:28.812 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/13' -> 'copyofsnap1/test1/149/13' 2023-10-30T12:38:28.813 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/14' -> 'copyofsnap1/test1/149/14' 2023-10-30T12:38:28.813 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/15' -> 'copyofsnap1/test1/149/15' 2023-10-30T12:38:28.813 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/16' -> 'copyofsnap1/test1/149/16' 2023-10-30T12:38:28.813 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/17' -> 'copyofsnap1/test1/149/17' 2023-10-30T12:38:28.813 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/18' -> 'copyofsnap1/test1/149/18' 2023-10-30T12:38:28.813 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/19' -> 'copyofsnap1/test1/149/19' 2023-10-30T12:38:28.814 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/20' -> 'copyofsnap1/test1/149/20' 2023-10-30T12:38:28.814 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/10' -> 'copyofsnap1/test1/149/10' 2023-10-30T12:38:28.814 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150' -> 'copyofsnap1/test1/150' 2023-10-30T12:38:28.814 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/11' -> 'copyofsnap1/test1/150/11' 2023-10-30T12:38:28.814 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/12' -> 'copyofsnap1/test1/150/12' 2023-10-30T12:38:28.815 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/13' -> 'copyofsnap1/test1/150/13' 2023-10-30T12:38:28.815 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/14' -> 'copyofsnap1/test1/150/14' 2023-10-30T12:38:28.815 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/15' -> 'copyofsnap1/test1/150/15' 2023-10-30T12:38:28.815 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/16' -> 'copyofsnap1/test1/150/16' 2023-10-30T12:38:28.815 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/17' -> 'copyofsnap1/test1/150/17' 2023-10-30T12:38:28.816 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/18' -> 'copyofsnap1/test1/150/18' 2023-10-30T12:38:28.816 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/19' -> 'copyofsnap1/test1/150/19' 2023-10-30T12:38:28.816 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/20' -> 'copyofsnap1/test1/150/20' 2023-10-30T12:38:28.816 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/10' -> 'copyofsnap1/test1/150/10' 2023-10-30T12:38:28.816 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151' -> 'copyofsnap1/test1/151' 2023-10-30T12:38:28.817 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/11' -> 'copyofsnap1/test1/151/11' 2023-10-30T12:38:28.817 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/12' -> 'copyofsnap1/test1/151/12' 2023-10-30T12:38:28.817 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/13' -> 'copyofsnap1/test1/151/13' 2023-10-30T12:38:28.817 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/14' -> 'copyofsnap1/test1/151/14' 2023-10-30T12:38:28.817 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/15' -> 'copyofsnap1/test1/151/15' 2023-10-30T12:38:28.818 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/16' -> 'copyofsnap1/test1/151/16' 2023-10-30T12:38:28.818 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/17' -> 'copyofsnap1/test1/151/17' 2023-10-30T12:38:28.818 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/18' -> 'copyofsnap1/test1/151/18' 2023-10-30T12:38:28.818 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/19' -> 'copyofsnap1/test1/151/19' 2023-10-30T12:38:28.818 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/20' -> 'copyofsnap1/test1/151/20' 2023-10-30T12:38:28.819 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/10' -> 'copyofsnap1/test1/151/10' 2023-10-30T12:38:28.819 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152' -> 'copyofsnap1/test1/152' 2023-10-30T12:38:28.819 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/11' -> 'copyofsnap1/test1/152/11' 2023-10-30T12:38:28.819 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/12' -> 'copyofsnap1/test1/152/12' 2023-10-30T12:38:28.819 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/13' -> 'copyofsnap1/test1/152/13' 2023-10-30T12:38:28.819 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/14' -> 'copyofsnap1/test1/152/14' 2023-10-30T12:38:28.820 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/15' -> 'copyofsnap1/test1/152/15' 2023-10-30T12:38:28.820 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/16' -> 'copyofsnap1/test1/152/16' 2023-10-30T12:38:28.820 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/17' -> 'copyofsnap1/test1/152/17' 2023-10-30T12:38:28.820 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/18' -> 'copyofsnap1/test1/152/18' 2023-10-30T12:38:28.820 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/19' -> 'copyofsnap1/test1/152/19' 2023-10-30T12:38:28.821 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/20' -> 'copyofsnap1/test1/152/20' 2023-10-30T12:38:28.821 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/10' -> 'copyofsnap1/test1/152/10' 2023-10-30T12:38:28.821 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153' -> 'copyofsnap1/test1/153' 2023-10-30T12:38:28.821 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/11' -> 'copyofsnap1/test1/153/11' 2023-10-30T12:38:28.821 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/12' -> 'copyofsnap1/test1/153/12' 2023-10-30T12:38:28.821 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/13' -> 'copyofsnap1/test1/153/13' 2023-10-30T12:38:28.822 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/14' -> 'copyofsnap1/test1/153/14' 2023-10-30T12:38:28.822 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/15' -> 'copyofsnap1/test1/153/15' 2023-10-30T12:38:28.822 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/16' -> 'copyofsnap1/test1/153/16' 2023-10-30T12:38:28.822 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/17' -> 'copyofsnap1/test1/153/17' 2023-10-30T12:38:28.823 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/18' -> 'copyofsnap1/test1/153/18' 2023-10-30T12:38:28.823 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/19' -> 'copyofsnap1/test1/153/19' 2023-10-30T12:38:28.823 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/20' -> 'copyofsnap1/test1/153/20' 2023-10-30T12:38:28.823 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/10' -> 'copyofsnap1/test1/153/10' 2023-10-30T12:38:29.833 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154' -> 'copyofsnap1/test1/154' 2023-10-30T12:38:29.834 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/11' -> 'copyofsnap1/test1/154/11' 2023-10-30T12:38:29.834 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/12' -> 'copyofsnap1/test1/154/12' 2023-10-30T12:38:29.834 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/13' -> 'copyofsnap1/test1/154/13' 2023-10-30T12:38:29.834 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/14' -> 'copyofsnap1/test1/154/14' 2023-10-30T12:38:29.834 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/15' -> 'copyofsnap1/test1/154/15' 2023-10-30T12:38:29.835 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/16' -> 'copyofsnap1/test1/154/16' 2023-10-30T12:38:29.835 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/17' -> 'copyofsnap1/test1/154/17' 2023-10-30T12:38:29.835 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/18' -> 'copyofsnap1/test1/154/18' 2023-10-30T12:38:29.835 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/19' -> 'copyofsnap1/test1/154/19' 2023-10-30T12:38:29.835 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/20' -> 'copyofsnap1/test1/154/20' 2023-10-30T12:38:29.836 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/10' -> 'copyofsnap1/test1/154/10' 2023-10-30T12:38:29.836 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155' -> 'copyofsnap1/test1/155' 2023-10-30T12:38:29.836 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/11' -> 'copyofsnap1/test1/155/11' 2023-10-30T12:38:29.836 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/12' -> 'copyofsnap1/test1/155/12' 2023-10-30T12:38:29.836 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/13' -> 'copyofsnap1/test1/155/13' 2023-10-30T12:38:29.836 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/14' -> 'copyofsnap1/test1/155/14' 2023-10-30T12:38:29.837 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/15' -> 'copyofsnap1/test1/155/15' 2023-10-30T12:38:29.837 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/16' -> 'copyofsnap1/test1/155/16' 2023-10-30T12:38:29.837 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/17' -> 'copyofsnap1/test1/155/17' 2023-10-30T12:38:29.837 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/18' -> 'copyofsnap1/test1/155/18' 2023-10-30T12:38:29.837 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/19' -> 'copyofsnap1/test1/155/19' 2023-10-30T12:38:29.838 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/20' -> 'copyofsnap1/test1/155/20' 2023-10-30T12:38:29.838 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/10' -> 'copyofsnap1/test1/155/10' 2023-10-30T12:38:29.838 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156' -> 'copyofsnap1/test1/156' 2023-10-30T12:38:29.838 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/11' -> 'copyofsnap1/test1/156/11' 2023-10-30T12:38:29.838 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/12' -> 'copyofsnap1/test1/156/12' 2023-10-30T12:38:29.839 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/13' -> 'copyofsnap1/test1/156/13' 2023-10-30T12:38:29.839 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/14' -> 'copyofsnap1/test1/156/14' 2023-10-30T12:38:29.839 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/15' -> 'copyofsnap1/test1/156/15' 2023-10-30T12:38:29.839 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/16' -> 'copyofsnap1/test1/156/16' 2023-10-30T12:38:29.839 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/17' -> 'copyofsnap1/test1/156/17' 2023-10-30T12:38:29.839 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/18' -> 'copyofsnap1/test1/156/18' 2023-10-30T12:38:29.840 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/19' -> 'copyofsnap1/test1/156/19' 2023-10-30T12:38:29.840 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/20' -> 'copyofsnap1/test1/156/20' 2023-10-30T12:38:29.840 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/10' -> 'copyofsnap1/test1/156/10' 2023-10-30T12:38:29.840 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157' -> 'copyofsnap1/test1/157' 2023-10-30T12:38:29.840 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/11' -> 'copyofsnap1/test1/157/11' 2023-10-30T12:38:29.841 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/12' -> 'copyofsnap1/test1/157/12' 2023-10-30T12:38:29.841 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/13' -> 'copyofsnap1/test1/157/13' 2023-10-30T12:38:29.841 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/14' -> 'copyofsnap1/test1/157/14' 2023-10-30T12:38:29.841 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/15' -> 'copyofsnap1/test1/157/15' 2023-10-30T12:38:29.841 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/16' -> 'copyofsnap1/test1/157/16' 2023-10-30T12:38:29.841 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/17' -> 'copyofsnap1/test1/157/17' 2023-10-30T12:38:29.842 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/18' -> 'copyofsnap1/test1/157/18' 2023-10-30T12:38:29.842 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/19' -> 'copyofsnap1/test1/157/19' 2023-10-30T12:38:29.842 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/20' -> 'copyofsnap1/test1/157/20' 2023-10-30T12:38:29.842 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/10' -> 'copyofsnap1/test1/157/10' 2023-10-30T12:38:29.843 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158' -> 'copyofsnap1/test1/158' 2023-10-30T12:38:29.843 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/11' -> 'copyofsnap1/test1/158/11' 2023-10-30T12:38:29.843 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/12' -> 'copyofsnap1/test1/158/12' 2023-10-30T12:38:29.843 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/13' -> 'copyofsnap1/test1/158/13' 2023-10-30T12:38:29.843 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/14' -> 'copyofsnap1/test1/158/14' 2023-10-30T12:38:29.844 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/15' -> 'copyofsnap1/test1/158/15' 2023-10-30T12:38:29.844 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/16' -> 'copyofsnap1/test1/158/16' 2023-10-30T12:38:29.844 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/17' -> 'copyofsnap1/test1/158/17' 2023-10-30T12:38:29.844 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/18' -> 'copyofsnap1/test1/158/18' 2023-10-30T12:38:29.844 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/19' -> 'copyofsnap1/test1/158/19' 2023-10-30T12:38:29.845 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/20' -> 'copyofsnap1/test1/158/20' 2023-10-30T12:38:29.845 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/10' -> 'copyofsnap1/test1/158/10' 2023-10-30T12:38:29.845 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159' -> 'copyofsnap1/test1/159' 2023-10-30T12:38:29.845 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/11' -> 'copyofsnap1/test1/159/11' 2023-10-30T12:38:29.845 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/12' -> 'copyofsnap1/test1/159/12' 2023-10-30T12:38:29.845 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/13' -> 'copyofsnap1/test1/159/13' 2023-10-30T12:38:29.846 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/14' -> 'copyofsnap1/test1/159/14' 2023-10-30T12:38:29.846 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/15' -> 'copyofsnap1/test1/159/15' 2023-10-30T12:38:29.846 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/16' -> 'copyofsnap1/test1/159/16' 2023-10-30T12:38:29.846 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/17' -> 'copyofsnap1/test1/159/17' 2023-10-30T12:38:29.846 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/18' -> 'copyofsnap1/test1/159/18' 2023-10-30T12:38:29.847 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/19' -> 'copyofsnap1/test1/159/19' 2023-10-30T12:38:29.847 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/20' -> 'copyofsnap1/test1/159/20' 2023-10-30T12:38:29.847 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/10' -> 'copyofsnap1/test1/159/10' 2023-10-30T12:38:29.847 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160' -> 'copyofsnap1/test1/160' 2023-10-30T12:38:29.847 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/11' -> 'copyofsnap1/test1/160/11' 2023-10-30T12:38:29.848 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/12' -> 'copyofsnap1/test1/160/12' 2023-10-30T12:38:29.848 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/13' -> 'copyofsnap1/test1/160/13' 2023-10-30T12:38:29.848 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/14' -> 'copyofsnap1/test1/160/14' 2023-10-30T12:38:29.848 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/15' -> 'copyofsnap1/test1/160/15' 2023-10-30T12:38:29.848 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/16' -> 'copyofsnap1/test1/160/16' 2023-10-30T12:38:29.849 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/17' -> 'copyofsnap1/test1/160/17' 2023-10-30T12:38:29.849 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/18' -> 'copyofsnap1/test1/160/18' 2023-10-30T12:38:30.275 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/19' -> 'copyofsnap1/test1/160/19' 2023-10-30T12:38:30.276 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/20' -> 'copyofsnap1/test1/160/20' 2023-10-30T12:38:30.276 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/10' -> 'copyofsnap1/test1/160/10' 2023-10-30T12:38:30.276 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161' -> 'copyofsnap1/test1/161' 2023-10-30T12:38:30.276 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/11' -> 'copyofsnap1/test1/161/11' 2023-10-30T12:38:30.276 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/12' -> 'copyofsnap1/test1/161/12' 2023-10-30T12:38:30.277 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/13' -> 'copyofsnap1/test1/161/13' 2023-10-30T12:38:30.277 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/14' -> 'copyofsnap1/test1/161/14' 2023-10-30T12:38:30.277 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/15' -> 'copyofsnap1/test1/161/15' 2023-10-30T12:38:30.277 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/16' -> 'copyofsnap1/test1/161/16' 2023-10-30T12:38:30.277 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/17' -> 'copyofsnap1/test1/161/17' 2023-10-30T12:38:30.278 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/18' -> 'copyofsnap1/test1/161/18' 2023-10-30T12:38:30.278 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/19' -> 'copyofsnap1/test1/161/19' 2023-10-30T12:38:30.278 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/20' -> 'copyofsnap1/test1/161/20' 2023-10-30T12:38:30.278 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/10' -> 'copyofsnap1/test1/161/10' 2023-10-30T12:38:30.278 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162' -> 'copyofsnap1/test1/162' 2023-10-30T12:38:30.279 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/11' -> 'copyofsnap1/test1/162/11' 2023-10-30T12:38:30.279 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/12' -> 'copyofsnap1/test1/162/12' 2023-10-30T12:38:30.279 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/13' -> 'copyofsnap1/test1/162/13' 2023-10-30T12:38:30.279 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/14' -> 'copyofsnap1/test1/162/14' 2023-10-30T12:38:30.279 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/15' -> 'copyofsnap1/test1/162/15' 2023-10-30T12:38:30.280 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/16' -> 'copyofsnap1/test1/162/16' 2023-10-30T12:38:30.280 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/17' -> 'copyofsnap1/test1/162/17' 2023-10-30T12:38:30.280 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/18' -> 'copyofsnap1/test1/162/18' 2023-10-30T12:38:30.280 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/19' -> 'copyofsnap1/test1/162/19' 2023-10-30T12:38:30.280 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/20' -> 'copyofsnap1/test1/162/20' 2023-10-30T12:38:30.280 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/10' -> 'copyofsnap1/test1/162/10' 2023-10-30T12:38:30.281 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163' -> 'copyofsnap1/test1/163' 2023-10-30T12:38:30.281 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/11' -> 'copyofsnap1/test1/163/11' 2023-10-30T12:38:30.281 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/12' -> 'copyofsnap1/test1/163/12' 2023-10-30T12:38:30.281 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/13' -> 'copyofsnap1/test1/163/13' 2023-10-30T12:38:30.281 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/14' -> 'copyofsnap1/test1/163/14' 2023-10-30T12:38:30.282 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/15' -> 'copyofsnap1/test1/163/15' 2023-10-30T12:38:30.282 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/16' -> 'copyofsnap1/test1/163/16' 2023-10-30T12:38:30.282 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/17' -> 'copyofsnap1/test1/163/17' 2023-10-30T12:38:30.282 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/18' -> 'copyofsnap1/test1/163/18' 2023-10-30T12:38:30.282 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/19' -> 'copyofsnap1/test1/163/19' 2023-10-30T12:38:30.283 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/20' -> 'copyofsnap1/test1/163/20' 2023-10-30T12:38:30.283 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/10' -> 'copyofsnap1/test1/163/10' 2023-10-30T12:38:30.283 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164' -> 'copyofsnap1/test1/164' 2023-10-30T12:38:30.283 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/11' -> 'copyofsnap1/test1/164/11' 2023-10-30T12:38:30.283 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/12' -> 'copyofsnap1/test1/164/12' 2023-10-30T12:38:30.283 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/13' -> 'copyofsnap1/test1/164/13' 2023-10-30T12:38:30.284 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/14' -> 'copyofsnap1/test1/164/14' 2023-10-30T12:38:30.284 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/15' -> 'copyofsnap1/test1/164/15' 2023-10-30T12:38:30.284 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/16' -> 'copyofsnap1/test1/164/16' 2023-10-30T12:38:30.284 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/17' -> 'copyofsnap1/test1/164/17' 2023-10-30T12:38:30.284 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/18' -> 'copyofsnap1/test1/164/18' 2023-10-30T12:38:30.285 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/19' -> 'copyofsnap1/test1/164/19' 2023-10-30T12:38:30.285 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/20' -> 'copyofsnap1/test1/164/20' 2023-10-30T12:38:30.285 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/10' -> 'copyofsnap1/test1/164/10' 2023-10-30T12:38:30.285 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165' -> 'copyofsnap1/test1/165' 2023-10-30T12:38:30.286 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/11' -> 'copyofsnap1/test1/165/11' 2023-10-30T12:38:30.286 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/12' -> 'copyofsnap1/test1/165/12' 2023-10-30T12:38:30.286 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/13' -> 'copyofsnap1/test1/165/13' 2023-10-30T12:38:30.286 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/14' -> 'copyofsnap1/test1/165/14' 2023-10-30T12:38:30.286 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/15' -> 'copyofsnap1/test1/165/15' 2023-10-30T12:38:30.287 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/16' -> 'copyofsnap1/test1/165/16' 2023-10-30T12:38:30.287 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/17' -> 'copyofsnap1/test1/165/17' 2023-10-30T12:38:30.287 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/18' -> 'copyofsnap1/test1/165/18' 2023-10-30T12:38:30.287 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/19' -> 'copyofsnap1/test1/165/19' 2023-10-30T12:38:30.287 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/20' -> 'copyofsnap1/test1/165/20' 2023-10-30T12:38:30.288 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/10' -> 'copyofsnap1/test1/165/10' 2023-10-30T12:38:30.288 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166' -> 'copyofsnap1/test1/166' 2023-10-30T12:38:30.288 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/11' -> 'copyofsnap1/test1/166/11' 2023-10-30T12:38:30.288 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/12' -> 'copyofsnap1/test1/166/12' 2023-10-30T12:38:30.288 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/13' -> 'copyofsnap1/test1/166/13' 2023-10-30T12:38:30.288 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/14' -> 'copyofsnap1/test1/166/14' 2023-10-30T12:38:30.289 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/15' -> 'copyofsnap1/test1/166/15' 2023-10-30T12:38:30.289 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/16' -> 'copyofsnap1/test1/166/16' 2023-10-30T12:38:30.289 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/17' -> 'copyofsnap1/test1/166/17' 2023-10-30T12:38:30.289 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/18' -> 'copyofsnap1/test1/166/18' 2023-10-30T12:38:30.289 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/19' -> 'copyofsnap1/test1/166/19' 2023-10-30T12:38:30.290 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/20' -> 'copyofsnap1/test1/166/20' 2023-10-30T12:38:30.290 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/10' -> 'copyofsnap1/test1/166/10' 2023-10-30T12:38:30.290 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167' -> 'copyofsnap1/test1/167' 2023-10-30T12:38:30.290 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/11' -> 'copyofsnap1/test1/167/11' 2023-10-30T12:38:30.290 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/12' -> 'copyofsnap1/test1/167/12' 2023-10-30T12:38:30.291 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/13' -> 'copyofsnap1/test1/167/13' 2023-10-30T12:38:30.291 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/14' -> 'copyofsnap1/test1/167/14' 2023-10-30T12:38:30.291 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/15' -> 'copyofsnap1/test1/167/15' 2023-10-30T12:38:31.855 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/16' -> 'copyofsnap1/test1/167/16' 2023-10-30T12:38:31.855 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/17' -> 'copyofsnap1/test1/167/17' 2023-10-30T12:38:31.856 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/18' -> 'copyofsnap1/test1/167/18' 2023-10-30T12:38:31.856 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/19' -> 'copyofsnap1/test1/167/19' 2023-10-30T12:38:31.856 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/20' -> 'copyofsnap1/test1/167/20' 2023-10-30T12:38:31.856 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/10' -> 'copyofsnap1/test1/167/10' 2023-10-30T12:38:31.856 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168' -> 'copyofsnap1/test1/168' 2023-10-30T12:38:31.857 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/11' -> 'copyofsnap1/test1/168/11' 2023-10-30T12:38:31.857 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/12' -> 'copyofsnap1/test1/168/12' 2023-10-30T12:38:31.857 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/13' -> 'copyofsnap1/test1/168/13' 2023-10-30T12:38:31.857 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/14' -> 'copyofsnap1/test1/168/14' 2023-10-30T12:38:31.857 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/15' -> 'copyofsnap1/test1/168/15' 2023-10-30T12:38:31.858 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/16' -> 'copyofsnap1/test1/168/16' 2023-10-30T12:38:31.858 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/17' -> 'copyofsnap1/test1/168/17' 2023-10-30T12:38:31.858 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/18' -> 'copyofsnap1/test1/168/18' 2023-10-30T12:38:31.858 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/19' -> 'copyofsnap1/test1/168/19' 2023-10-30T12:38:31.858 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/20' -> 'copyofsnap1/test1/168/20' 2023-10-30T12:38:31.859 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/10' -> 'copyofsnap1/test1/168/10' 2023-10-30T12:38:31.859 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169' -> 'copyofsnap1/test1/169' 2023-10-30T12:38:31.859 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/11' -> 'copyofsnap1/test1/169/11' 2023-10-30T12:38:31.859 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/12' -> 'copyofsnap1/test1/169/12' 2023-10-30T12:38:31.859 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/13' -> 'copyofsnap1/test1/169/13' 2023-10-30T12:38:31.860 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/14' -> 'copyofsnap1/test1/169/14' 2023-10-30T12:38:31.860 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/15' -> 'copyofsnap1/test1/169/15' 2023-10-30T12:38:31.860 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/16' -> 'copyofsnap1/test1/169/16' 2023-10-30T12:38:31.860 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/17' -> 'copyofsnap1/test1/169/17' 2023-10-30T12:38:31.860 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/18' -> 'copyofsnap1/test1/169/18' 2023-10-30T12:38:31.861 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/19' -> 'copyofsnap1/test1/169/19' 2023-10-30T12:38:31.861 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/20' -> 'copyofsnap1/test1/169/20' 2023-10-30T12:38:31.861 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/10' -> 'copyofsnap1/test1/169/10' 2023-10-30T12:38:31.861 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170' -> 'copyofsnap1/test1/170' 2023-10-30T12:38:31.861 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/11' -> 'copyofsnap1/test1/170/11' 2023-10-30T12:38:31.862 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/12' -> 'copyofsnap1/test1/170/12' 2023-10-30T12:38:31.862 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/13' -> 'copyofsnap1/test1/170/13' 2023-10-30T12:38:31.862 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/14' -> 'copyofsnap1/test1/170/14' 2023-10-30T12:38:31.862 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/15' -> 'copyofsnap1/test1/170/15' 2023-10-30T12:38:31.862 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/16' -> 'copyofsnap1/test1/170/16' 2023-10-30T12:38:31.863 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/17' -> 'copyofsnap1/test1/170/17' 2023-10-30T12:38:31.863 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/18' -> 'copyofsnap1/test1/170/18' 2023-10-30T12:38:31.863 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/19' -> 'copyofsnap1/test1/170/19' 2023-10-30T12:38:31.863 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/20' -> 'copyofsnap1/test1/170/20' 2023-10-30T12:38:31.863 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/10' -> 'copyofsnap1/test1/170/10' 2023-10-30T12:38:31.863 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171' -> 'copyofsnap1/test1/171' 2023-10-30T12:38:31.864 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/11' -> 'copyofsnap1/test1/171/11' 2023-10-30T12:38:31.864 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/12' -> 'copyofsnap1/test1/171/12' 2023-10-30T12:38:31.864 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/13' -> 'copyofsnap1/test1/171/13' 2023-10-30T12:38:31.864 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/14' -> 'copyofsnap1/test1/171/14' 2023-10-30T12:38:31.864 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/15' -> 'copyofsnap1/test1/171/15' 2023-10-30T12:38:31.865 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/16' -> 'copyofsnap1/test1/171/16' 2023-10-30T12:38:31.865 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/17' -> 'copyofsnap1/test1/171/17' 2023-10-30T12:38:31.865 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/18' -> 'copyofsnap1/test1/171/18' 2023-10-30T12:38:31.865 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/19' -> 'copyofsnap1/test1/171/19' 2023-10-30T12:38:31.866 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/20' -> 'copyofsnap1/test1/171/20' 2023-10-30T12:38:31.866 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/10' -> 'copyofsnap1/test1/171/10' 2023-10-30T12:38:31.866 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172' -> 'copyofsnap1/test1/172' 2023-10-30T12:38:31.866 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/11' -> 'copyofsnap1/test1/172/11' 2023-10-30T12:38:31.867 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/12' -> 'copyofsnap1/test1/172/12' 2023-10-30T12:38:31.867 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/13' -> 'copyofsnap1/test1/172/13' 2023-10-30T12:38:31.867 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/14' -> 'copyofsnap1/test1/172/14' 2023-10-30T12:38:31.867 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/15' -> 'copyofsnap1/test1/172/15' 2023-10-30T12:38:31.867 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/16' -> 'copyofsnap1/test1/172/16' 2023-10-30T12:38:31.868 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/17' -> 'copyofsnap1/test1/172/17' 2023-10-30T12:38:31.868 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/18' -> 'copyofsnap1/test1/172/18' 2023-10-30T12:38:31.868 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/19' -> 'copyofsnap1/test1/172/19' 2023-10-30T12:38:31.868 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/20' -> 'copyofsnap1/test1/172/20' 2023-10-30T12:38:31.868 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/10' -> 'copyofsnap1/test1/172/10' 2023-10-30T12:38:31.868 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173' -> 'copyofsnap1/test1/173' 2023-10-30T12:38:31.869 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/11' -> 'copyofsnap1/test1/173/11' 2023-10-30T12:38:31.869 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/12' -> 'copyofsnap1/test1/173/12' 2023-10-30T12:38:31.869 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/13' -> 'copyofsnap1/test1/173/13' 2023-10-30T12:38:31.869 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/14' -> 'copyofsnap1/test1/173/14' 2023-10-30T12:38:31.869 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/15' -> 'copyofsnap1/test1/173/15' 2023-10-30T12:38:31.870 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/16' -> 'copyofsnap1/test1/173/16' 2023-10-30T12:38:31.870 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/17' -> 'copyofsnap1/test1/173/17' 2023-10-30T12:38:31.870 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/18' -> 'copyofsnap1/test1/173/18' 2023-10-30T12:38:31.870 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/19' -> 'copyofsnap1/test1/173/19' 2023-10-30T12:38:31.870 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/20' -> 'copyofsnap1/test1/173/20' 2023-10-30T12:38:31.871 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/10' -> 'copyofsnap1/test1/173/10' 2023-10-30T12:38:31.871 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174' -> 'copyofsnap1/test1/174' 2023-10-30T12:38:31.871 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/11' -> 'copyofsnap1/test1/174/11' 2023-10-30T12:38:31.871 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/12' -> 'copyofsnap1/test1/174/12' 2023-10-30T12:38:33.705 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:38:33.709 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:38:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/13' -> 'copyofsnap1/test1/174/13' 2023-10-30T12:38:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/14' -> 'copyofsnap1/test1/174/14' 2023-10-30T12:38:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/15' -> 'copyofsnap1/test1/174/15' 2023-10-30T12:38:34.536 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/16' -> 'copyofsnap1/test1/174/16' 2023-10-30T12:38:34.536 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/17' -> 'copyofsnap1/test1/174/17' 2023-10-30T12:38:34.536 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/18' -> 'copyofsnap1/test1/174/18' 2023-10-30T12:38:34.536 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/19' -> 'copyofsnap1/test1/174/19' 2023-10-30T12:38:34.536 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/20' -> 'copyofsnap1/test1/174/20' 2023-10-30T12:38:34.537 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/10' -> 'copyofsnap1/test1/174/10' 2023-10-30T12:38:34.537 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175' -> 'copyofsnap1/test1/175' 2023-10-30T12:38:34.537 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/11' -> 'copyofsnap1/test1/175/11' 2023-10-30T12:38:34.537 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/12' -> 'copyofsnap1/test1/175/12' 2023-10-30T12:38:34.537 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/13' -> 'copyofsnap1/test1/175/13' 2023-10-30T12:38:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/14' -> 'copyofsnap1/test1/175/14' 2023-10-30T12:38:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/15' -> 'copyofsnap1/test1/175/15' 2023-10-30T12:38:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/16' -> 'copyofsnap1/test1/175/16' 2023-10-30T12:38:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/17' -> 'copyofsnap1/test1/175/17' 2023-10-30T12:38:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/18' -> 'copyofsnap1/test1/175/18' 2023-10-30T12:38:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/19' -> 'copyofsnap1/test1/175/19' 2023-10-30T12:38:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/20' -> 'copyofsnap1/test1/175/20' 2023-10-30T12:38:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/10' -> 'copyofsnap1/test1/175/10' 2023-10-30T12:38:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176' -> 'copyofsnap1/test1/176' 2023-10-30T12:38:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/11' -> 'copyofsnap1/test1/176/11' 2023-10-30T12:38:34.540 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/12' -> 'copyofsnap1/test1/176/12' 2023-10-30T12:38:34.540 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/13' -> 'copyofsnap1/test1/176/13' 2023-10-30T12:38:34.540 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/14' -> 'copyofsnap1/test1/176/14' 2023-10-30T12:38:34.540 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/15' -> 'copyofsnap1/test1/176/15' 2023-10-30T12:38:34.540 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/16' -> 'copyofsnap1/test1/176/16' 2023-10-30T12:38:34.541 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/17' -> 'copyofsnap1/test1/176/17' 2023-10-30T12:38:34.541 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/18' -> 'copyofsnap1/test1/176/18' 2023-10-30T12:38:34.541 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/19' -> 'copyofsnap1/test1/176/19' 2023-10-30T12:38:34.541 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/20' -> 'copyofsnap1/test1/176/20' 2023-10-30T12:38:34.541 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/10' -> 'copyofsnap1/test1/176/10' 2023-10-30T12:38:34.541 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177' -> 'copyofsnap1/test1/177' 2023-10-30T12:38:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/11' -> 'copyofsnap1/test1/177/11' 2023-10-30T12:38:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/12' -> 'copyofsnap1/test1/177/12' 2023-10-30T12:38:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/13' -> 'copyofsnap1/test1/177/13' 2023-10-30T12:38:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/14' -> 'copyofsnap1/test1/177/14' 2023-10-30T12:38:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/15' -> 'copyofsnap1/test1/177/15' 2023-10-30T12:38:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/16' -> 'copyofsnap1/test1/177/16' 2023-10-30T12:38:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/17' -> 'copyofsnap1/test1/177/17' 2023-10-30T12:38:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/18' -> 'copyofsnap1/test1/177/18' 2023-10-30T12:38:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/19' -> 'copyofsnap1/test1/177/19' 2023-10-30T12:38:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/20' -> 'copyofsnap1/test1/177/20' 2023-10-30T12:38:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/10' -> 'copyofsnap1/test1/177/10' 2023-10-30T12:38:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178' -> 'copyofsnap1/test1/178' 2023-10-30T12:38:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/11' -> 'copyofsnap1/test1/178/11' 2023-10-30T12:38:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/12' -> 'copyofsnap1/test1/178/12' 2023-10-30T12:38:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/13' -> 'copyofsnap1/test1/178/13' 2023-10-30T12:38:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/14' -> 'copyofsnap1/test1/178/14' 2023-10-30T12:38:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/15' -> 'copyofsnap1/test1/178/15' 2023-10-30T12:38:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/16' -> 'copyofsnap1/test1/178/16' 2023-10-30T12:38:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/17' -> 'copyofsnap1/test1/178/17' 2023-10-30T12:38:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/18' -> 'copyofsnap1/test1/178/18' 2023-10-30T12:38:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/19' -> 'copyofsnap1/test1/178/19' 2023-10-30T12:38:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/20' -> 'copyofsnap1/test1/178/20' 2023-10-30T12:38:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/10' -> 'copyofsnap1/test1/178/10' 2023-10-30T12:38:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179' -> 'copyofsnap1/test1/179' 2023-10-30T12:38:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/11' -> 'copyofsnap1/test1/179/11' 2023-10-30T12:38:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/12' -> 'copyofsnap1/test1/179/12' 2023-10-30T12:38:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/13' -> 'copyofsnap1/test1/179/13' 2023-10-30T12:38:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/14' -> 'copyofsnap1/test1/179/14' 2023-10-30T12:38:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/15' -> 'copyofsnap1/test1/179/15' 2023-10-30T12:38:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/16' -> 'copyofsnap1/test1/179/16' 2023-10-30T12:38:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/17' -> 'copyofsnap1/test1/179/17' 2023-10-30T12:38:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/18' -> 'copyofsnap1/test1/179/18' 2023-10-30T12:38:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/19' -> 'copyofsnap1/test1/179/19' 2023-10-30T12:38:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/20' -> 'copyofsnap1/test1/179/20' 2023-10-30T12:38:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/10' -> 'copyofsnap1/test1/179/10' 2023-10-30T12:38:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180' -> 'copyofsnap1/test1/180' 2023-10-30T12:38:34.549 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/11' -> 'copyofsnap1/test1/180/11' 2023-10-30T12:38:34.549 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/12' -> 'copyofsnap1/test1/180/12' 2023-10-30T12:38:34.549 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/13' -> 'copyofsnap1/test1/180/13' 2023-10-30T12:38:34.549 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/14' -> 'copyofsnap1/test1/180/14' 2023-10-30T12:38:34.549 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/15' -> 'copyofsnap1/test1/180/15' 2023-10-30T12:38:34.550 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/16' -> 'copyofsnap1/test1/180/16' 2023-10-30T12:38:34.550 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/17' -> 'copyofsnap1/test1/180/17' 2023-10-30T12:38:34.550 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/18' -> 'copyofsnap1/test1/180/18' 2023-10-30T12:38:34.550 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/19' -> 'copyofsnap1/test1/180/19' 2023-10-30T12:38:34.550 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/20' -> 'copyofsnap1/test1/180/20' 2023-10-30T12:38:34.551 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/10' -> 'copyofsnap1/test1/180/10' 2023-10-30T12:38:35.738 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181' -> 'copyofsnap1/test1/181' 2023-10-30T12:38:35.738 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/11' -> 'copyofsnap1/test1/181/11' 2023-10-30T12:38:35.738 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/12' -> 'copyofsnap1/test1/181/12' 2023-10-30T12:38:35.739 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/13' -> 'copyofsnap1/test1/181/13' 2023-10-30T12:38:35.739 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/14' -> 'copyofsnap1/test1/181/14' 2023-10-30T12:38:35.739 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/15' -> 'copyofsnap1/test1/181/15' 2023-10-30T12:38:35.739 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/16' -> 'copyofsnap1/test1/181/16' 2023-10-30T12:38:35.739 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/17' -> 'copyofsnap1/test1/181/17' 2023-10-30T12:38:35.740 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/18' -> 'copyofsnap1/test1/181/18' 2023-10-30T12:38:35.740 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/19' -> 'copyofsnap1/test1/181/19' 2023-10-30T12:38:35.740 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/20' -> 'copyofsnap1/test1/181/20' 2023-10-30T12:38:35.740 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/10' -> 'copyofsnap1/test1/181/10' 2023-10-30T12:38:35.740 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182' -> 'copyofsnap1/test1/182' 2023-10-30T12:38:35.741 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/11' -> 'copyofsnap1/test1/182/11' 2023-10-30T12:38:35.741 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/12' -> 'copyofsnap1/test1/182/12' 2023-10-30T12:38:35.741 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/13' -> 'copyofsnap1/test1/182/13' 2023-10-30T12:38:35.741 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/14' -> 'copyofsnap1/test1/182/14' 2023-10-30T12:38:35.741 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/15' -> 'copyofsnap1/test1/182/15' 2023-10-30T12:38:35.742 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/16' -> 'copyofsnap1/test1/182/16' 2023-10-30T12:38:35.742 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/17' -> 'copyofsnap1/test1/182/17' 2023-10-30T12:38:35.742 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/18' -> 'copyofsnap1/test1/182/18' 2023-10-30T12:38:35.742 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/19' -> 'copyofsnap1/test1/182/19' 2023-10-30T12:38:35.742 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/20' -> 'copyofsnap1/test1/182/20' 2023-10-30T12:38:35.743 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/10' -> 'copyofsnap1/test1/182/10' 2023-10-30T12:38:35.743 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183' -> 'copyofsnap1/test1/183' 2023-10-30T12:38:35.743 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/11' -> 'copyofsnap1/test1/183/11' 2023-10-30T12:38:35.743 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/12' -> 'copyofsnap1/test1/183/12' 2023-10-30T12:38:35.743 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/13' -> 'copyofsnap1/test1/183/13' 2023-10-30T12:38:35.744 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/14' -> 'copyofsnap1/test1/183/14' 2023-10-30T12:38:35.744 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/15' -> 'copyofsnap1/test1/183/15' 2023-10-30T12:38:35.744 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/16' -> 'copyofsnap1/test1/183/16' 2023-10-30T12:38:35.744 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/17' -> 'copyofsnap1/test1/183/17' 2023-10-30T12:38:35.744 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/18' -> 'copyofsnap1/test1/183/18' 2023-10-30T12:38:35.745 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/19' -> 'copyofsnap1/test1/183/19' 2023-10-30T12:38:35.745 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/20' -> 'copyofsnap1/test1/183/20' 2023-10-30T12:38:35.745 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/10' -> 'copyofsnap1/test1/183/10' 2023-10-30T12:38:35.745 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184' -> 'copyofsnap1/test1/184' 2023-10-30T12:38:35.745 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/11' -> 'copyofsnap1/test1/184/11' 2023-10-30T12:38:35.745 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/12' -> 'copyofsnap1/test1/184/12' 2023-10-30T12:38:35.746 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/13' -> 'copyofsnap1/test1/184/13' 2023-10-30T12:38:35.746 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/14' -> 'copyofsnap1/test1/184/14' 2023-10-30T12:38:35.746 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/15' -> 'copyofsnap1/test1/184/15' 2023-10-30T12:38:35.746 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/16' -> 'copyofsnap1/test1/184/16' 2023-10-30T12:38:35.747 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/17' -> 'copyofsnap1/test1/184/17' 2023-10-30T12:38:35.747 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/18' -> 'copyofsnap1/test1/184/18' 2023-10-30T12:38:35.747 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/19' -> 'copyofsnap1/test1/184/19' 2023-10-30T12:38:35.747 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/20' -> 'copyofsnap1/test1/184/20' 2023-10-30T12:38:35.747 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/10' -> 'copyofsnap1/test1/184/10' 2023-10-30T12:38:35.747 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185' -> 'copyofsnap1/test1/185' 2023-10-30T12:38:35.748 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/11' -> 'copyofsnap1/test1/185/11' 2023-10-30T12:38:35.748 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/12' -> 'copyofsnap1/test1/185/12' 2023-10-30T12:38:35.748 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/13' -> 'copyofsnap1/test1/185/13' 2023-10-30T12:38:35.748 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/14' -> 'copyofsnap1/test1/185/14' 2023-10-30T12:38:35.748 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/15' -> 'copyofsnap1/test1/185/15' 2023-10-30T12:38:35.749 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/16' -> 'copyofsnap1/test1/185/16' 2023-10-30T12:38:35.749 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/17' -> 'copyofsnap1/test1/185/17' 2023-10-30T12:38:35.749 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/18' -> 'copyofsnap1/test1/185/18' 2023-10-30T12:38:35.749 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/19' -> 'copyofsnap1/test1/185/19' 2023-10-30T12:38:35.749 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/20' -> 'copyofsnap1/test1/185/20' 2023-10-30T12:38:35.749 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/10' -> 'copyofsnap1/test1/185/10' 2023-10-30T12:38:35.750 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186' -> 'copyofsnap1/test1/186' 2023-10-30T12:38:35.750 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/11' -> 'copyofsnap1/test1/186/11' 2023-10-30T12:38:35.750 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/12' -> 'copyofsnap1/test1/186/12' 2023-10-30T12:38:35.750 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/13' -> 'copyofsnap1/test1/186/13' 2023-10-30T12:38:35.750 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/14' -> 'copyofsnap1/test1/186/14' 2023-10-30T12:38:35.751 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/15' -> 'copyofsnap1/test1/186/15' 2023-10-30T12:38:35.751 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/16' -> 'copyofsnap1/test1/186/16' 2023-10-30T12:38:35.751 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/17' -> 'copyofsnap1/test1/186/17' 2023-10-30T12:38:35.751 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/18' -> 'copyofsnap1/test1/186/18' 2023-10-30T12:38:35.751 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/19' -> 'copyofsnap1/test1/186/19' 2023-10-30T12:38:35.751 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/20' -> 'copyofsnap1/test1/186/20' 2023-10-30T12:38:35.752 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/10' -> 'copyofsnap1/test1/186/10' 2023-10-30T12:38:35.752 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187' -> 'copyofsnap1/test1/187' 2023-10-30T12:38:35.752 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/11' -> 'copyofsnap1/test1/187/11' 2023-10-30T12:38:35.752 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/12' -> 'copyofsnap1/test1/187/12' 2023-10-30T12:38:35.752 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/13' -> 'copyofsnap1/test1/187/13' 2023-10-30T12:38:35.753 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/14' -> 'copyofsnap1/test1/187/14' 2023-10-30T12:38:35.753 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/15' -> 'copyofsnap1/test1/187/15' 2023-10-30T12:38:35.753 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/16' -> 'copyofsnap1/test1/187/16' 2023-10-30T12:38:35.753 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/17' -> 'copyofsnap1/test1/187/17' 2023-10-30T12:38:35.753 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/18' -> 'copyofsnap1/test1/187/18' 2023-10-30T12:38:36.920 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/19' -> 'copyofsnap1/test1/187/19' 2023-10-30T12:38:36.920 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/20' -> 'copyofsnap1/test1/187/20' 2023-10-30T12:38:36.921 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/10' -> 'copyofsnap1/test1/187/10' 2023-10-30T12:38:36.921 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188' -> 'copyofsnap1/test1/188' 2023-10-30T12:38:36.921 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/11' -> 'copyofsnap1/test1/188/11' 2023-10-30T12:38:36.921 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/12' -> 'copyofsnap1/test1/188/12' 2023-10-30T12:38:36.921 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/13' -> 'copyofsnap1/test1/188/13' 2023-10-30T12:38:36.922 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/14' -> 'copyofsnap1/test1/188/14' 2023-10-30T12:38:36.922 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/15' -> 'copyofsnap1/test1/188/15' 2023-10-30T12:38:36.922 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/16' -> 'copyofsnap1/test1/188/16' 2023-10-30T12:38:36.922 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/17' -> 'copyofsnap1/test1/188/17' 2023-10-30T12:38:36.923 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/18' -> 'copyofsnap1/test1/188/18' 2023-10-30T12:38:36.923 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/19' -> 'copyofsnap1/test1/188/19' 2023-10-30T12:38:36.923 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/20' -> 'copyofsnap1/test1/188/20' 2023-10-30T12:38:36.923 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/10' -> 'copyofsnap1/test1/188/10' 2023-10-30T12:38:36.924 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189' -> 'copyofsnap1/test1/189' 2023-10-30T12:38:36.924 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/11' -> 'copyofsnap1/test1/189/11' 2023-10-30T12:38:36.924 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/12' -> 'copyofsnap1/test1/189/12' 2023-10-30T12:38:36.924 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/13' -> 'copyofsnap1/test1/189/13' 2023-10-30T12:38:36.924 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/14' -> 'copyofsnap1/test1/189/14' 2023-10-30T12:38:36.924 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/15' -> 'copyofsnap1/test1/189/15' 2023-10-30T12:38:36.925 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/16' -> 'copyofsnap1/test1/189/16' 2023-10-30T12:38:36.925 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/17' -> 'copyofsnap1/test1/189/17' 2023-10-30T12:38:36.925 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/18' -> 'copyofsnap1/test1/189/18' 2023-10-30T12:38:36.925 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/19' -> 'copyofsnap1/test1/189/19' 2023-10-30T12:38:36.925 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/20' -> 'copyofsnap1/test1/189/20' 2023-10-30T12:38:36.926 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/10' -> 'copyofsnap1/test1/189/10' 2023-10-30T12:38:36.926 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190' -> 'copyofsnap1/test1/190' 2023-10-30T12:38:36.926 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/11' -> 'copyofsnap1/test1/190/11' 2023-10-30T12:38:36.926 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/12' -> 'copyofsnap1/test1/190/12' 2023-10-30T12:38:36.926 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/13' -> 'copyofsnap1/test1/190/13' 2023-10-30T12:38:36.927 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/14' -> 'copyofsnap1/test1/190/14' 2023-10-30T12:38:36.927 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/15' -> 'copyofsnap1/test1/190/15' 2023-10-30T12:38:36.927 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/16' -> 'copyofsnap1/test1/190/16' 2023-10-30T12:38:36.927 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/17' -> 'copyofsnap1/test1/190/17' 2023-10-30T12:38:36.927 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/18' -> 'copyofsnap1/test1/190/18' 2023-10-30T12:38:36.928 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/19' -> 'copyofsnap1/test1/190/19' 2023-10-30T12:38:36.928 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/20' -> 'copyofsnap1/test1/190/20' 2023-10-30T12:38:36.928 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/10' -> 'copyofsnap1/test1/190/10' 2023-10-30T12:38:36.928 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191' -> 'copyofsnap1/test1/191' 2023-10-30T12:38:36.928 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/11' -> 'copyofsnap1/test1/191/11' 2023-10-30T12:38:36.929 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/12' -> 'copyofsnap1/test1/191/12' 2023-10-30T12:38:36.929 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/13' -> 'copyofsnap1/test1/191/13' 2023-10-30T12:38:36.929 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/14' -> 'copyofsnap1/test1/191/14' 2023-10-30T12:38:36.929 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/15' -> 'copyofsnap1/test1/191/15' 2023-10-30T12:38:36.929 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/16' -> 'copyofsnap1/test1/191/16' 2023-10-30T12:38:36.929 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/17' -> 'copyofsnap1/test1/191/17' 2023-10-30T12:38:36.930 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/18' -> 'copyofsnap1/test1/191/18' 2023-10-30T12:38:36.930 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/19' -> 'copyofsnap1/test1/191/19' 2023-10-30T12:38:36.930 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/20' -> 'copyofsnap1/test1/191/20' 2023-10-30T12:38:36.930 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/10' -> 'copyofsnap1/test1/191/10' 2023-10-30T12:38:36.930 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192' -> 'copyofsnap1/test1/192' 2023-10-30T12:38:36.930 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/11' -> 'copyofsnap1/test1/192/11' 2023-10-30T12:38:36.931 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/12' -> 'copyofsnap1/test1/192/12' 2023-10-30T12:38:36.931 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/13' -> 'copyofsnap1/test1/192/13' 2023-10-30T12:38:36.931 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/14' -> 'copyofsnap1/test1/192/14' 2023-10-30T12:38:36.931 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/15' -> 'copyofsnap1/test1/192/15' 2023-10-30T12:38:36.931 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/16' -> 'copyofsnap1/test1/192/16' 2023-10-30T12:38:36.931 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/17' -> 'copyofsnap1/test1/192/17' 2023-10-30T12:38:36.932 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/18' -> 'copyofsnap1/test1/192/18' 2023-10-30T12:38:36.932 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/19' -> 'copyofsnap1/test1/192/19' 2023-10-30T12:38:36.932 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/20' -> 'copyofsnap1/test1/192/20' 2023-10-30T12:38:36.932 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/10' -> 'copyofsnap1/test1/192/10' 2023-10-30T12:38:36.932 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193' -> 'copyofsnap1/test1/193' 2023-10-30T12:38:36.932 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/11' -> 'copyofsnap1/test1/193/11' 2023-10-30T12:38:36.933 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/12' -> 'copyofsnap1/test1/193/12' 2023-10-30T12:38:36.933 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/13' -> 'copyofsnap1/test1/193/13' 2023-10-30T12:38:36.933 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/14' -> 'copyofsnap1/test1/193/14' 2023-10-30T12:38:36.933 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/15' -> 'copyofsnap1/test1/193/15' 2023-10-30T12:38:36.933 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/16' -> 'copyofsnap1/test1/193/16' 2023-10-30T12:38:36.933 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/17' -> 'copyofsnap1/test1/193/17' 2023-10-30T12:38:36.934 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/18' -> 'copyofsnap1/test1/193/18' 2023-10-30T12:38:36.934 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/19' -> 'copyofsnap1/test1/193/19' 2023-10-30T12:38:36.934 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/20' -> 'copyofsnap1/test1/193/20' 2023-10-30T12:38:36.934 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/10' -> 'copyofsnap1/test1/193/10' 2023-10-30T12:38:36.934 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194' -> 'copyofsnap1/test1/194' 2023-10-30T12:38:36.934 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/11' -> 'copyofsnap1/test1/194/11' 2023-10-30T12:38:36.935 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/12' -> 'copyofsnap1/test1/194/12' 2023-10-30T12:38:36.935 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/13' -> 'copyofsnap1/test1/194/13' 2023-10-30T12:38:36.935 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/14' -> 'copyofsnap1/test1/194/14' 2023-10-30T12:38:36.935 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/15' -> 'copyofsnap1/test1/194/15' 2023-10-30T12:38:37.316 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/16' -> 'copyofsnap1/test1/194/16' 2023-10-30T12:38:37.317 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/17' -> 'copyofsnap1/test1/194/17' 2023-10-30T12:38:37.317 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/18' -> 'copyofsnap1/test1/194/18' 2023-10-30T12:38:37.317 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/19' -> 'copyofsnap1/test1/194/19' 2023-10-30T12:38:37.317 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/20' -> 'copyofsnap1/test1/194/20' 2023-10-30T12:38:37.317 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/10' -> 'copyofsnap1/test1/194/10' 2023-10-30T12:38:37.318 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195' -> 'copyofsnap1/test1/195' 2023-10-30T12:38:37.318 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/11' -> 'copyofsnap1/test1/195/11' 2023-10-30T12:38:37.318 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/12' -> 'copyofsnap1/test1/195/12' 2023-10-30T12:38:37.318 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/13' -> 'copyofsnap1/test1/195/13' 2023-10-30T12:38:37.318 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/14' -> 'copyofsnap1/test1/195/14' 2023-10-30T12:38:37.319 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/15' -> 'copyofsnap1/test1/195/15' 2023-10-30T12:38:37.319 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/16' -> 'copyofsnap1/test1/195/16' 2023-10-30T12:38:37.319 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/17' -> 'copyofsnap1/test1/195/17' 2023-10-30T12:38:37.319 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/18' -> 'copyofsnap1/test1/195/18' 2023-10-30T12:38:37.319 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/19' -> 'copyofsnap1/test1/195/19' 2023-10-30T12:38:37.319 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/20' -> 'copyofsnap1/test1/195/20' 2023-10-30T12:38:37.320 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/10' -> 'copyofsnap1/test1/195/10' 2023-10-30T12:38:37.320 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196' -> 'copyofsnap1/test1/196' 2023-10-30T12:38:37.320 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/11' -> 'copyofsnap1/test1/196/11' 2023-10-30T12:38:37.320 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/12' -> 'copyofsnap1/test1/196/12' 2023-10-30T12:38:37.320 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/13' -> 'copyofsnap1/test1/196/13' 2023-10-30T12:38:37.321 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/14' -> 'copyofsnap1/test1/196/14' 2023-10-30T12:38:37.321 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/15' -> 'copyofsnap1/test1/196/15' 2023-10-30T12:38:37.321 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/16' -> 'copyofsnap1/test1/196/16' 2023-10-30T12:38:37.321 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/17' -> 'copyofsnap1/test1/196/17' 2023-10-30T12:38:37.321 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/18' -> 'copyofsnap1/test1/196/18' 2023-10-30T12:38:37.321 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/19' -> 'copyofsnap1/test1/196/19' 2023-10-30T12:38:37.322 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/20' -> 'copyofsnap1/test1/196/20' 2023-10-30T12:38:37.322 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/10' -> 'copyofsnap1/test1/196/10' 2023-10-30T12:38:37.322 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197' -> 'copyofsnap1/test1/197' 2023-10-30T12:38:37.322 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/11' -> 'copyofsnap1/test1/197/11' 2023-10-30T12:38:37.322 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/12' -> 'copyofsnap1/test1/197/12' 2023-10-30T12:38:37.322 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/13' -> 'copyofsnap1/test1/197/13' 2023-10-30T12:38:37.323 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/14' -> 'copyofsnap1/test1/197/14' 2023-10-30T12:38:37.323 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/15' -> 'copyofsnap1/test1/197/15' 2023-10-30T12:38:37.323 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/16' -> 'copyofsnap1/test1/197/16' 2023-10-30T12:38:37.323 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/17' -> 'copyofsnap1/test1/197/17' 2023-10-30T12:38:37.323 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/18' -> 'copyofsnap1/test1/197/18' 2023-10-30T12:38:37.323 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/19' -> 'copyofsnap1/test1/197/19' 2023-10-30T12:38:37.324 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/20' -> 'copyofsnap1/test1/197/20' 2023-10-30T12:38:37.324 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/10' -> 'copyofsnap1/test1/197/10' 2023-10-30T12:38:37.324 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198' -> 'copyofsnap1/test1/198' 2023-10-30T12:38:37.324 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/11' -> 'copyofsnap1/test1/198/11' 2023-10-30T12:38:37.324 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/12' -> 'copyofsnap1/test1/198/12' 2023-10-30T12:38:37.324 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/13' -> 'copyofsnap1/test1/198/13' 2023-10-30T12:38:37.325 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/14' -> 'copyofsnap1/test1/198/14' 2023-10-30T12:38:37.325 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/15' -> 'copyofsnap1/test1/198/15' 2023-10-30T12:38:37.325 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/16' -> 'copyofsnap1/test1/198/16' 2023-10-30T12:38:37.325 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/17' -> 'copyofsnap1/test1/198/17' 2023-10-30T12:38:37.325 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/18' -> 'copyofsnap1/test1/198/18' 2023-10-30T12:38:37.326 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/19' -> 'copyofsnap1/test1/198/19' 2023-10-30T12:38:37.326 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/20' -> 'copyofsnap1/test1/198/20' 2023-10-30T12:38:37.326 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/10' -> 'copyofsnap1/test1/198/10' 2023-10-30T12:38:37.326 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199' -> 'copyofsnap1/test1/199' 2023-10-30T12:38:37.326 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/11' -> 'copyofsnap1/test1/199/11' 2023-10-30T12:38:37.326 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/12' -> 'copyofsnap1/test1/199/12' 2023-10-30T12:38:37.327 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/13' -> 'copyofsnap1/test1/199/13' 2023-10-30T12:38:37.327 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/14' -> 'copyofsnap1/test1/199/14' 2023-10-30T12:38:37.327 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/15' -> 'copyofsnap1/test1/199/15' 2023-10-30T12:38:37.327 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/16' -> 'copyofsnap1/test1/199/16' 2023-10-30T12:38:37.327 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/17' -> 'copyofsnap1/test1/199/17' 2023-10-30T12:38:37.327 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/18' -> 'copyofsnap1/test1/199/18' 2023-10-30T12:38:37.328 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/19' -> 'copyofsnap1/test1/199/19' 2023-10-30T12:38:37.328 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/20' -> 'copyofsnap1/test1/199/20' 2023-10-30T12:38:37.328 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/10' -> 'copyofsnap1/test1/199/10' 2023-10-30T12:38:37.328 INFO:tasks.workunit.client.0.smithi012.stdout:Take third snapshot .snap/test3 2023-10-30T12:38:37.445 INFO:tasks.workunit.client.0.smithi012.stdout:Delete the snapshots... 2023-10-30T12:38:39.605 INFO:tasks.workunit.client.0.smithi012.stdout:.//.snap/test1 2023-10-30T12:38:39.605 INFO:tasks.workunit.client.0.smithi012.stdout:.//.snap/test2 2023-10-30T12:38:39.606 INFO:tasks.workunit.client.0.smithi012.stdout:.//.snap/test3 2023-10-30T12:38:39.614 INFO:tasks.workunit.client.0.smithi012.stdout:./274/.snap/snap-subdir-test 2023-10-30T12:38:39.619 INFO:tasks.workunit.client.0.smithi012.stdout:./166/.snap/snap-subdir-test 2023-10-30T12:38:39.627 INFO:tasks.workunit.client.0.smithi012.stdout:./256/.snap/snap-subdir-test 2023-10-30T12:38:39.636 INFO:tasks.workunit.client.0.smithi012.stdout:./247/.snap/snap-subdir-test 2023-10-30T12:38:39.642 INFO:tasks.workunit.client.0.smithi012.stdout:./248/.snap/snap-subdir-test 2023-10-30T12:38:39.650 INFO:tasks.workunit.client.0.smithi012.stdout:./147/.snap/snap-subdir-test 2023-10-30T12:38:39.655 INFO:tasks.workunit.client.0.smithi012.stdout:./151/.snap/snap-subdir-test 2023-10-30T12:38:39.659 INFO:tasks.workunit.client.0.smithi012.stdout:./170/.snap/snap-subdir-test 2023-10-30T12:38:39.664 INFO:tasks.workunit.client.0.smithi012.stdout:./167/.snap/snap-subdir-test 2023-10-30T12:38:40.216 INFO:tasks.workunit.client.0.smithi012.stdout:./140/.snap/snap-subdir-test 2023-10-30T12:38:40.225 INFO:tasks.workunit.client.0.smithi012.stdout:./142/.snap/snap-subdir-test 2023-10-30T12:38:40.229 INFO:tasks.workunit.client.0.smithi012.stdout:./150/.snap/snap-subdir-test 2023-10-30T12:38:40.397 INFO:tasks.workunit.client.0.smithi012.stdout:./273/.snap/snap-subdir-test 2023-10-30T12:38:40.406 INFO:tasks.workunit.client.0.smithi012.stdout:./269/.snap/snap-subdir-test 2023-10-30T12:38:40.415 INFO:tasks.workunit.client.0.smithi012.stdout:./105/.snap/snap-subdir-test 2023-10-30T12:38:40.423 INFO:tasks.workunit.client.0.smithi012.stdout:./210/.snap/snap-subdir-test 2023-10-30T12:38:40.429 INFO:tasks.workunit.client.0.smithi012.stdout:./265/.snap/snap-subdir-test 2023-10-30T12:38:40.433 INFO:tasks.workunit.client.0.smithi012.stdout:./218/.snap/snap-subdir-test 2023-10-30T12:38:40.440 INFO:tasks.workunit.client.0.smithi012.stdout:./290/.snap/snap-subdir-test 2023-10-30T12:38:40.445 INFO:tasks.workunit.client.0.smithi012.stdout:./149/.snap/snap-subdir-test 2023-10-30T12:38:40.454 INFO:tasks.workunit.client.0.smithi012.stdout:./109/.snap/snap-subdir-test 2023-10-30T12:38:40.461 INFO:tasks.workunit.client.0.smithi012.stdout:./292/.snap/snap-subdir-test 2023-10-30T12:38:40.470 INFO:tasks.workunit.client.0.smithi012.stdout:./242/.snap/snap-subdir-test 2023-10-30T12:38:40.475 INFO:tasks.workunit.client.0.smithi012.stdout:./158/.snap/snap-subdir-test 2023-10-30T12:38:40.485 INFO:tasks.workunit.client.0.smithi012.stdout:./243/.snap/snap-subdir-test 2023-10-30T12:38:40.495 INFO:tasks.workunit.client.0.smithi012.stdout:./106/.snap/snap-subdir-test 2023-10-30T12:38:40.506 INFO:tasks.workunit.client.0.smithi012.stdout:./250/.snap/snap-subdir-test 2023-10-30T12:38:40.518 INFO:tasks.workunit.client.0.smithi012.stdout:./221/.snap/snap-subdir-test 2023-10-30T12:38:40.529 INFO:tasks.workunit.client.0.smithi012.stdout:./244/.snap/snap-subdir-test 2023-10-30T12:38:40.541 INFO:tasks.workunit.client.0.smithi012.stdout:./258/.snap/snap-subdir-test 2023-10-30T12:38:40.551 INFO:tasks.workunit.client.0.smithi012.stdout:./268/.snap/snap-subdir-test 2023-10-30T12:38:40.561 INFO:tasks.workunit.client.0.smithi012.stdout:./189/.snap/snap-subdir-test 2023-10-30T12:38:41.380 INFO:tasks.workunit.client.0.smithi012.stdout:./285/.snap/snap-subdir-test 2023-10-30T12:38:42.042 INFO:tasks.workunit.client.0.smithi012.stdout:./208/.snap/snap-subdir-test 2023-10-30T12:38:42.056 INFO:tasks.workunit.client.0.smithi012.stdout:./288/.snap/snap-subdir-test 2023-10-30T12:38:42.844 INFO:tasks.workunit.client.0.smithi012.stdout:./209/.snap/snap-subdir-test 2023-10-30T12:38:42.855 INFO:tasks.workunit.client.0.smithi012.stdout:./253/.snap/snap-subdir-test 2023-10-30T12:38:42.865 INFO:tasks.workunit.client.0.smithi012.stdout:./249/.snap/snap-subdir-test 2023-10-30T12:38:42.878 INFO:tasks.workunit.client.0.smithi012.stdout:./133/.snap/snap-subdir-test 2023-10-30T12:38:42.889 INFO:tasks.workunit.client.0.smithi012.stdout:./291/.snap/snap-subdir-test 2023-10-30T12:38:42.901 INFO:tasks.workunit.client.0.smithi012.stdout:./226/.snap/snap-subdir-test 2023-10-30T12:38:42.906 INFO:tasks.workunit.client.0.smithi012.stdout:./173/.snap/snap-subdir-test 2023-10-30T12:38:42.919 INFO:tasks.workunit.client.0.smithi012.stdout:./174/.snap/snap-subdir-test 2023-10-30T12:38:42.930 INFO:tasks.workunit.client.0.smithi012.stdout:./299/.snap/snap-subdir-test 2023-10-30T12:38:42.938 INFO:tasks.workunit.client.0.smithi012.stdout:./262/.snap/snap-subdir-test 2023-10-30T12:38:42.950 INFO:tasks.workunit.client.0.smithi012.stdout:./231/.snap/snap-subdir-test 2023-10-30T12:38:42.962 INFO:tasks.workunit.client.0.smithi012.stdout:./214/.snap/snap-subdir-test 2023-10-30T12:38:42.973 INFO:tasks.workunit.client.0.smithi012.stdout:./281/.snap/snap-subdir-test 2023-10-30T12:38:42.984 INFO:tasks.workunit.client.0.smithi012.stdout:./289/.snap/snap-subdir-test 2023-10-30T12:38:43.194 INFO:tasks.workunit.client.0.smithi012.stdout:./283/.snap/snap-subdir-test 2023-10-30T12:38:43.204 INFO:tasks.workunit.client.0.smithi012.stdout:./232/.snap/snap-subdir-test 2023-10-30T12:38:43.216 INFO:tasks.workunit.client.0.smithi012.stdout:./298/.snap/snap-subdir-test 2023-10-30T12:38:43.229 INFO:tasks.workunit.client.0.smithi012.stdout:./233/.snap/snap-subdir-test 2023-10-30T12:38:43.242 INFO:tasks.workunit.client.0.smithi012.stdout:./155/.snap/snap-subdir-test 2023-10-30T12:38:43.256 INFO:tasks.workunit.client.0.smithi012.stdout:./114/.snap/snap-subdir-test 2023-10-30T12:38:43.267 INFO:tasks.workunit.client.0.smithi012.stdout:./159/.snap/snap-subdir-test 2023-10-30T12:38:43.278 INFO:tasks.workunit.client.0.smithi012.stdout:./187/.snap/snap-subdir-test 2023-10-30T12:38:43.291 INFO:tasks.workunit.client.0.smithi012.stdout:./185/.snap/snap-subdir-test 2023-10-30T12:38:43.303 INFO:tasks.workunit.client.0.smithi012.stdout:./293/.snap/snap-subdir-test 2023-10-30T12:38:43.314 INFO:tasks.workunit.client.0.smithi012.stdout:./228/.snap/snap-subdir-test 2023-10-30T12:38:43.327 INFO:tasks.workunit.client.0.smithi012.stdout:./120/.snap/snap-subdir-test 2023-10-30T12:38:43.339 INFO:tasks.workunit.client.0.smithi012.stdout:./110/.snap/snap-subdir-test 2023-10-30T12:38:43.349 INFO:tasks.workunit.client.0.smithi012.stdout:./267/.snap/snap-subdir-test 2023-10-30T12:38:43.360 INFO:tasks.workunit.client.0.smithi012.stdout:./202/.snap/snap-subdir-test 2023-10-30T12:38:43.370 INFO:tasks.workunit.client.0.smithi012.stdout:./137/.snap/snap-subdir-test 2023-10-30T12:38:43.382 INFO:tasks.workunit.client.0.smithi012.stdout:./246/.snap/snap-subdir-test 2023-10-30T12:38:43.392 INFO:tasks.workunit.client.0.smithi012.stdout:./219/.snap/snap-subdir-test 2023-10-30T12:38:43.403 INFO:tasks.workunit.client.0.smithi012.stdout:./198/.snap/snap-subdir-test 2023-10-30T12:38:43.415 INFO:tasks.workunit.client.0.smithi012.stdout:./136/.snap/snap-subdir-test 2023-10-30T12:38:43.425 INFO:tasks.workunit.client.0.smithi012.stdout:./270/.snap/snap-subdir-test 2023-10-30T12:38:43.437 INFO:tasks.workunit.client.0.smithi012.stdout:./125/.snap/snap-subdir-test 2023-10-30T12:38:43.451 INFO:tasks.workunit.client.0.smithi012.stdout:./194/.snap/snap-subdir-test 2023-10-30T12:38:43.467 INFO:tasks.workunit.client.0.smithi012.stdout:./119/.snap/snap-subdir-test 2023-10-30T12:38:43.895 INFO:tasks.workunit.client.0.smithi012.stdout:./297/.snap/snap-subdir-test 2023-10-30T12:38:43.905 INFO:tasks.workunit.client.0.smithi012.stdout:./230/.snap/snap-subdir-test 2023-10-30T12:38:43.915 INFO:tasks.workunit.client.0.smithi012.stdout:./260/.snap/snap-subdir-test 2023-10-30T12:38:43.923 INFO:tasks.workunit.client.0.smithi012.stdout:./157/.snap/snap-subdir-test 2023-10-30T12:38:43.927 INFO:tasks.workunit.client.0.smithi012.stdout:./172/.snap/snap-subdir-test 2023-10-30T12:38:43.934 INFO:tasks.workunit.client.0.smithi012.stdout:./277/.snap/snap-subdir-test 2023-10-30T12:38:43.944 INFO:tasks.workunit.client.0.smithi012.stdout:./186/.snap/snap-subdir-test 2023-10-30T12:38:43.951 INFO:tasks.workunit.client.0.smithi012.stdout:./237/.snap/snap-subdir-test 2023-10-30T12:38:44.211 INFO:tasks.workunit.client.0.smithi012.stdout:./190/.snap/snap-subdir-test 2023-10-30T12:38:44.219 INFO:tasks.workunit.client.0.smithi012.stdout:./213/.snap/snap-subdir-test 2023-10-30T12:38:44.230 INFO:tasks.workunit.client.0.smithi012.stdout:./104/.snap/snap-subdir-test 2023-10-30T12:38:44.238 INFO:tasks.workunit.client.0.smithi012.stdout:./164/.snap/snap-subdir-test 2023-10-30T12:38:44.249 INFO:tasks.workunit.client.0.smithi012.stdout:./112/.snap/snap-subdir-test 2023-10-30T12:38:44.259 INFO:tasks.workunit.client.0.smithi012.stdout:./295/.snap/snap-subdir-test 2023-10-30T12:38:44.518 INFO:tasks.workunit.client.0.smithi012.stdout:./272/.snap/snap-subdir-test 2023-10-30T12:38:45.268 INFO:tasks.workunit.client.0.smithi012.stdout:./100/.snap/snap-subdir-test 2023-10-30T12:38:45.278 INFO:tasks.workunit.client.0.smithi012.stdout:./134/.snap/snap-subdir-test 2023-10-30T12:38:45.288 INFO:tasks.workunit.client.0.smithi012.stdout:./200/.snap/snap-subdir-test 2023-10-30T12:38:45.299 INFO:tasks.workunit.client.0.smithi012.stdout:./115/.snap/snap-subdir-test 2023-10-30T12:38:45.765 INFO:tasks.workunit.client.0.smithi012.stdout:./284/.snap/snap-subdir-test 2023-10-30T12:38:45.776 INFO:tasks.workunit.client.0.smithi012.stdout:./280/.snap/snap-subdir-test 2023-10-30T12:38:45.786 INFO:tasks.workunit.client.0.smithi012.stdout:./116/.snap/snap-subdir-test 2023-10-30T12:38:45.796 INFO:tasks.workunit.client.0.smithi012.stdout:./245/.snap/snap-subdir-test 2023-10-30T12:38:45.804 INFO:tasks.workunit.client.0.smithi012.stdout:./195/.snap/snap-subdir-test 2023-10-30T12:38:45.811 INFO:tasks.workunit.client.0.smithi012.stdout:./138/.snap/snap-subdir-test 2023-10-30T12:38:45.823 INFO:tasks.workunit.client.0.smithi012.stdout:./135/.snap/snap-subdir-test 2023-10-30T12:38:45.837 INFO:tasks.workunit.client.0.smithi012.stdout:./117/.snap/snap-subdir-test 2023-10-30T12:38:46.377 INFO:tasks.workunit.client.0.smithi012.stdout:./241/.snap/snap-subdir-test 2023-10-30T12:38:46.391 INFO:tasks.workunit.client.0.smithi012.stdout:./143/.snap/snap-subdir-test 2023-10-30T12:38:46.404 INFO:tasks.workunit.client.0.smithi012.stdout:./180/.snap/snap-subdir-test 2023-10-30T12:38:46.428 INFO:tasks.workunit.client.0.smithi012.stdout:./126/.snap/snap-subdir-test 2023-10-30T12:38:46.747 INFO:tasks.workunit.client.0.smithi012.stdout:./161/.snap/snap-subdir-test 2023-10-30T12:38:46.759 INFO:tasks.workunit.client.0.smithi012.stdout:./204/.snap/snap-subdir-test 2023-10-30T12:38:46.770 INFO:tasks.workunit.client.0.smithi012.stdout:./176/.snap/snap-subdir-test 2023-10-30T12:38:46.781 INFO:tasks.workunit.client.0.smithi012.stdout:./255/.snap/snap-subdir-test 2023-10-30T12:38:46.792 INFO:tasks.workunit.client.0.smithi012.stdout:./215/.snap/snap-subdir-test 2023-10-30T12:38:46.807 INFO:tasks.workunit.client.0.smithi012.stdout:./111/.snap/snap-subdir-test 2023-10-30T12:38:46.816 INFO:tasks.workunit.client.0.smithi012.stdout:./238/.snap/snap-subdir-test 2023-10-30T12:38:47.768 INFO:tasks.workunit.client.0.smithi012.stdout:./191/.snap/snap-subdir-test 2023-10-30T12:38:48.032 INFO:tasks.workunit.client.0.smithi012.stdout:./184/.snap/snap-subdir-test 2023-10-30T12:38:48.044 INFO:tasks.workunit.client.0.smithi012.stdout:./192/.snap/snap-subdir-test 2023-10-30T12:38:48.059 INFO:tasks.workunit.client.0.smithi012.stdout:./103/.snap/snap-subdir-test 2023-10-30T12:38:48.072 INFO:tasks.workunit.client.0.smithi012.stdout:./128/.snap/snap-subdir-test 2023-10-30T12:38:48.512 INFO:tasks.workunit.client.0.smithi012.stdout:./229/.snap/snap-subdir-test 2023-10-30T12:38:48.525 INFO:tasks.workunit.client.0.smithi012.stdout:./144/.snap/snap-subdir-test 2023-10-30T12:38:48.538 INFO:tasks.workunit.client.0.smithi012.stdout:./146/.snap/snap-subdir-test 2023-10-30T12:38:48.550 INFO:tasks.workunit.client.0.smithi012.stdout:./227/.snap/snap-subdir-test 2023-10-30T12:38:48.562 INFO:tasks.workunit.client.0.smithi012.stdout:./278/.snap/snap-subdir-test 2023-10-30T12:38:48.576 INFO:tasks.workunit.client.0.smithi012.stdout:./141/.snap/snap-subdir-test 2023-10-30T12:38:48.589 INFO:tasks.workunit.client.0.smithi012.stdout:./234/.snap/snap-subdir-test 2023-10-30T12:38:48.602 INFO:tasks.workunit.client.0.smithi012.stdout:./225/.snap/snap-subdir-test 2023-10-30T12:38:48.614 INFO:tasks.workunit.client.0.smithi012.stdout:./182/.snap/snap-subdir-test 2023-10-30T12:38:48.629 INFO:tasks.workunit.client.0.smithi012.stdout:./107/.snap/snap-subdir-test 2023-10-30T12:38:48.640 INFO:tasks.workunit.client.0.smithi012.stdout:./178/.snap/snap-subdir-test 2023-10-30T12:38:48.648 INFO:tasks.workunit.client.0.smithi012.stdout:./266/.snap/snap-subdir-test 2023-10-30T12:38:48.658 INFO:tasks.workunit.client.0.smithi012.stdout:./240/.snap/snap-subdir-test 2023-10-30T12:38:48.668 INFO:tasks.workunit.client.0.smithi012.stdout:./156/.snap/snap-subdir-test 2023-10-30T12:38:48.679 INFO:tasks.workunit.client.0.smithi012.stdout:./131/.snap/snap-subdir-test 2023-10-30T12:38:48.689 INFO:tasks.workunit.client.0.smithi012.stdout:./123/.snap/snap-subdir-test 2023-10-30T12:38:48.698 INFO:tasks.workunit.client.0.smithi012.stdout:./160/.snap/snap-subdir-test 2023-10-30T12:38:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:./165/.snap/snap-subdir-test 2023-10-30T12:38:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:./101/.snap/snap-subdir-test 2023-10-30T12:38:48.977 INFO:tasks.workunit.client.0.smithi012.stdout:./148/.snap/snap-subdir-test 2023-10-30T12:38:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:./168/.snap/snap-subdir-test 2023-10-30T12:38:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:./179/.snap/snap-subdir-test 2023-10-30T12:38:49.095 INFO:tasks.workunit.client.0.smithi012.stdout:./257/.snap/snap-subdir-test 2023-10-30T12:38:49.108 INFO:tasks.workunit.client.0.smithi012.stdout:./252/.snap/snap-subdir-test 2023-10-30T12:38:49.119 INFO:tasks.workunit.client.0.smithi012.stdout:./132/.snap/snap-subdir-test 2023-10-30T12:38:49.129 INFO:tasks.workunit.client.0.smithi012.stdout:./236/.snap/snap-subdir-test 2023-10-30T12:38:49.139 INFO:tasks.workunit.client.0.smithi012.stdout:./211/.snap/snap-subdir-test 2023-10-30T12:38:49.150 INFO:tasks.workunit.client.0.smithi012.stdout:./199/.snap/snap-subdir-test 2023-10-30T12:38:49.159 INFO:tasks.workunit.client.0.smithi012.stdout:./205/.snap/snap-subdir-test 2023-10-30T12:38:49.167 INFO:tasks.workunit.client.0.smithi012.stdout:./163/.snap/snap-subdir-test 2023-10-30T12:38:49.171 INFO:tasks.workunit.client.0.smithi012.stdout:./261/.snap/snap-subdir-test 2023-10-30T12:38:49.182 INFO:tasks.workunit.client.0.smithi012.stdout:./201/.snap/snap-subdir-test 2023-10-30T12:38:49.189 INFO:tasks.workunit.client.0.smithi012.stdout:./282/.snap/snap-subdir-test 2023-10-30T12:38:49.199 INFO:tasks.workunit.client.0.smithi012.stdout:./296/.snap/snap-subdir-test 2023-10-30T12:38:49.207 INFO:tasks.workunit.client.0.smithi012.stdout:./217/.snap/snap-subdir-test 2023-10-30T12:38:49.469 INFO:tasks.workunit.client.0.smithi012.stdout:./153/.snap/snap-subdir-test 2023-10-30T12:38:49.478 INFO:tasks.workunit.client.0.smithi012.stdout:./206/.snap/snap-subdir-test 2023-10-30T12:38:49.488 INFO:tasks.workunit.client.0.smithi012.stdout:./122/.snap/snap-subdir-test 2023-10-30T12:38:49.497 INFO:tasks.workunit.client.0.smithi012.stdout:./162/.snap/snap-subdir-test 2023-10-30T12:38:49.504 INFO:tasks.workunit.client.0.smithi012.stdout:./197/.snap/snap-subdir-test 2023-10-30T12:38:49.513 INFO:tasks.workunit.client.0.smithi012.stdout:./216/.snap/snap-subdir-test 2023-10-30T12:38:51.356 INFO:tasks.workunit.client.0.smithi012.stdout:./254/.snap/snap-subdir-test 2023-10-30T12:38:51.366 INFO:tasks.workunit.client.0.smithi012.stdout:./113/.snap/snap-subdir-test 2023-10-30T12:38:51.376 INFO:tasks.workunit.client.0.smithi012.stdout:./152/.snap/snap-subdir-test 2023-10-30T12:38:51.380 INFO:tasks.workunit.client.0.smithi012.stdout:./121/.snap/snap-subdir-test 2023-10-30T12:38:51.389 INFO:tasks.workunit.client.0.smithi012.stdout:./118/.snap/snap-subdir-test 2023-10-30T12:38:51.396 INFO:tasks.workunit.client.0.smithi012.stdout:./196/.snap/snap-subdir-test 2023-10-30T12:38:51.400 INFO:tasks.workunit.client.0.smithi012.stdout:./124/.snap/snap-subdir-test 2023-10-30T12:38:51.406 INFO:tasks.workunit.client.0.smithi012.stdout:./239/.snap/snap-subdir-test 2023-10-30T12:38:51.413 INFO:tasks.workunit.client.0.smithi012.stdout:./188/.snap/snap-subdir-test 2023-10-30T12:38:51.421 INFO:tasks.workunit.client.0.smithi012.stdout:./287/.snap/snap-subdir-test 2023-10-30T12:38:51.428 INFO:tasks.workunit.client.0.smithi012.stdout:./275/.snap/snap-subdir-test 2023-10-30T12:38:51.434 INFO:tasks.workunit.client.0.smithi012.stdout:./259/.snap/snap-subdir-test 2023-10-30T12:38:51.441 INFO:tasks.workunit.client.0.smithi012.stdout:./224/.snap/snap-subdir-test 2023-10-30T12:38:51.450 INFO:tasks.workunit.client.0.smithi012.stdout:./264/.snap/snap-subdir-test 2023-10-30T12:38:51.457 INFO:tasks.workunit.client.0.smithi012.stdout:./145/.snap/snap-subdir-test 2023-10-30T12:38:51.466 INFO:tasks.workunit.client.0.smithi012.stdout:./279/.snap/snap-subdir-test 2023-10-30T12:38:51.473 INFO:tasks.workunit.client.0.smithi012.stdout:./263/.snap/snap-subdir-test 2023-10-30T12:38:51.819 INFO:tasks.workunit.client.0.smithi012.stdout:./130/.snap/snap-subdir-test 2023-10-30T12:38:51.826 INFO:tasks.workunit.client.0.smithi012.stdout:./193/.snap/snap-subdir-test 2023-10-30T12:38:51.836 INFO:tasks.workunit.client.0.smithi012.stdout:./223/.snap/snap-subdir-test 2023-10-30T12:38:51.846 INFO:tasks.workunit.client.0.smithi012.stdout:./207/.snap/snap-subdir-test 2023-10-30T12:38:51.855 INFO:tasks.workunit.client.0.smithi012.stdout:./276/.snap/snap-subdir-test 2023-10-30T12:38:52.346 INFO:tasks.workunit.client.0.smithi012.stdout:./271/.snap/snap-subdir-test 2023-10-30T12:38:52.354 INFO:tasks.workunit.client.0.smithi012.stdout:./251/.snap/snap-subdir-test 2023-10-30T12:38:52.364 INFO:tasks.workunit.client.0.smithi012.stdout:./175/.snap/snap-subdir-test 2023-10-30T12:38:52.375 INFO:tasks.workunit.client.0.smithi012.stdout:./222/.snap/snap-subdir-test 2023-10-30T12:38:52.384 INFO:tasks.workunit.client.0.smithi012.stdout:./220/.snap/snap-subdir-test 2023-10-30T12:38:52.392 INFO:tasks.workunit.client.0.smithi012.stdout:./203/.snap/snap-subdir-test 2023-10-30T12:38:52.402 INFO:tasks.workunit.client.0.smithi012.stdout:./181/.snap/snap-subdir-test 2023-10-30T12:38:52.413 INFO:tasks.workunit.client.0.smithi012.stdout:./286/.snap/snap-subdir-test 2023-10-30T12:38:52.422 INFO:tasks.workunit.client.0.smithi012.stdout:./127/.snap/snap-subdir-test 2023-10-30T12:38:52.433 INFO:tasks.workunit.client.0.smithi012.stdout:./108/.snap/snap-subdir-test 2023-10-30T12:38:52.443 INFO:tasks.workunit.client.0.smithi012.stdout:./183/.snap/snap-subdir-test 2023-10-30T12:38:52.451 INFO:tasks.workunit.client.0.smithi012.stdout:./294/.snap/snap-subdir-test 2023-10-30T12:38:52.461 INFO:tasks.workunit.client.0.smithi012.stdout:./235/.snap/snap-subdir-test 2023-10-30T12:38:52.726 INFO:tasks.workunit.client.0.smithi012.stdout:./177/.snap/snap-subdir-test 2023-10-30T12:38:52.737 INFO:tasks.workunit.client.0.smithi012.stdout:./169/.snap/snap-subdir-test 2023-10-30T12:38:52.745 INFO:tasks.workunit.client.0.smithi012.stdout:./212/.snap/snap-subdir-test 2023-10-30T12:38:52.756 INFO:tasks.workunit.client.0.smithi012.stdout:./154/.snap/snap-subdir-test 2023-10-30T12:38:52.765 INFO:tasks.workunit.client.0.smithi012.stdout:./171/.snap/snap-subdir-test 2023-10-30T12:38:52.775 INFO:tasks.workunit.client.0.smithi012.stdout:./139/.snap/snap-subdir-test 2023-10-30T12:38:52.786 INFO:tasks.workunit.client.0.smithi012.stdout:./102/.snap/snap-subdir-test 2023-10-30T12:38:52.795 INFO:tasks.workunit.client.0.smithi012.stdout:./129/.snap/snap-subdir-test 2023-10-30T12:39:03.755 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:39:03.761 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:39:03.987 INFO:tasks.workunit.client.0.smithi012.stdout:Delete all the files and directories ... 2023-10-30T12:39:05.607 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/19' 2023-10-30T12:39:05.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/15' 2023-10-30T12:39:05.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/31' 2023-10-30T12:39:05.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/36' 2023-10-30T12:39:05.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/20' 2023-10-30T12:39:05.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/12' 2023-10-30T12:39:05.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/39' 2023-10-30T12:39:05.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/10' 2023-10-30T12:39:05.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/34' 2023-10-30T12:39:05.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/38' 2023-10-30T12:39:05.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/11' 2023-10-30T12:39:05.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/16' 2023-10-30T12:39:05.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/32' 2023-10-30T12:39:05.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/30' 2023-10-30T12:39:05.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/14' 2023-10-30T12:39:05.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/18' 2023-10-30T12:39:05.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/13' 2023-10-30T12:39:05.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/33' 2023-10-30T12:39:05.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/17' 2023-10-30T12:39:05.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/35' 2023-10-30T12:39:05.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/37' 2023-10-30T12:39:05.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './100' 2023-10-30T12:39:05.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/19' 2023-10-30T12:39:05.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/15' 2023-10-30T12:39:05.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/31' 2023-10-30T12:39:05.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/36' 2023-10-30T12:39:05.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/20' 2023-10-30T12:39:05.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/12' 2023-10-30T12:39:05.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/39' 2023-10-30T12:39:05.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/10' 2023-10-30T12:39:05.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/34' 2023-10-30T12:39:05.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/38' 2023-10-30T12:39:05.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/11' 2023-10-30T12:39:05.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/16' 2023-10-30T12:39:05.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/32' 2023-10-30T12:39:05.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/30' 2023-10-30T12:39:05.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/14' 2023-10-30T12:39:05.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/18' 2023-10-30T12:39:05.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/13' 2023-10-30T12:39:05.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/33' 2023-10-30T12:39:05.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/17' 2023-10-30T12:39:05.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/35' 2023-10-30T12:39:05.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/37' 2023-10-30T12:39:05.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './101' 2023-10-30T12:39:05.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/19' 2023-10-30T12:39:05.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/15' 2023-10-30T12:39:05.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/31' 2023-10-30T12:39:05.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/36' 2023-10-30T12:39:05.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/20' 2023-10-30T12:39:05.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/12' 2023-10-30T12:39:05.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/39' 2023-10-30T12:39:05.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/10' 2023-10-30T12:39:05.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/34' 2023-10-30T12:39:05.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/38' 2023-10-30T12:39:05.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/11' 2023-10-30T12:39:05.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/16' 2023-10-30T12:39:05.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/32' 2023-10-30T12:39:05.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/30' 2023-10-30T12:39:05.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/14' 2023-10-30T12:39:05.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/18' 2023-10-30T12:39:05.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/13' 2023-10-30T12:39:05.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/33' 2023-10-30T12:39:05.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/17' 2023-10-30T12:39:05.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/35' 2023-10-30T12:39:05.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/37' 2023-10-30T12:39:05.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './102' 2023-10-30T12:39:05.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/19' 2023-10-30T12:39:05.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/15' 2023-10-30T12:39:05.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/31' 2023-10-30T12:39:05.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/36' 2023-10-30T12:39:05.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/20' 2023-10-30T12:39:05.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/12' 2023-10-30T12:39:05.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/39' 2023-10-30T12:39:05.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/10' 2023-10-30T12:39:05.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/34' 2023-10-30T12:39:05.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/38' 2023-10-30T12:39:05.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/11' 2023-10-30T12:39:05.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/16' 2023-10-30T12:39:05.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/32' 2023-10-30T12:39:05.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/30' 2023-10-30T12:39:05.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/14' 2023-10-30T12:39:05.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/18' 2023-10-30T12:39:05.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/13' 2023-10-30T12:39:05.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/33' 2023-10-30T12:39:05.625 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/17' 2023-10-30T12:39:05.625 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/35' 2023-10-30T12:39:05.625 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/37' 2023-10-30T12:39:05.625 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './103' 2023-10-30T12:39:05.625 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/19' 2023-10-30T12:39:05.626 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/15' 2023-10-30T12:39:05.626 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/31' 2023-10-30T12:39:05.626 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/36' 2023-10-30T12:39:05.626 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/20' 2023-10-30T12:39:05.626 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/12' 2023-10-30T12:39:05.626 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/39' 2023-10-30T12:39:05.627 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/10' 2023-10-30T12:39:05.627 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/34' 2023-10-30T12:39:05.627 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/38' 2023-10-30T12:39:05.627 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/11' 2023-10-30T12:39:05.627 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/16' 2023-10-30T12:39:05.628 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/32' 2023-10-30T12:39:05.628 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/30' 2023-10-30T12:39:05.628 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/14' 2023-10-30T12:39:05.628 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/18' 2023-10-30T12:39:05.628 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/13' 2023-10-30T12:39:05.629 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/33' 2023-10-30T12:39:05.629 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/17' 2023-10-30T12:39:05.629 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/35' 2023-10-30T12:39:05.629 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/37' 2023-10-30T12:39:05.629 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './104' 2023-10-30T12:39:05.630 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/19' 2023-10-30T12:39:05.630 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/15' 2023-10-30T12:39:05.630 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/31' 2023-10-30T12:39:05.630 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/36' 2023-10-30T12:39:05.631 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/20' 2023-10-30T12:39:05.631 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/12' 2023-10-30T12:39:05.631 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/39' 2023-10-30T12:39:05.631 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/10' 2023-10-30T12:39:05.631 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/34' 2023-10-30T12:39:05.632 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/38' 2023-10-30T12:39:05.632 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/11' 2023-10-30T12:39:05.632 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/16' 2023-10-30T12:39:05.632 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/32' 2023-10-30T12:39:05.632 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/30' 2023-10-30T12:39:05.633 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/14' 2023-10-30T12:39:05.633 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/18' 2023-10-30T12:39:05.633 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/13' 2023-10-30T12:39:05.633 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/33' 2023-10-30T12:39:05.633 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/17' 2023-10-30T12:39:05.633 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/35' 2023-10-30T12:39:05.634 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/37' 2023-10-30T12:39:05.634 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './105' 2023-10-30T12:39:05.634 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/19' 2023-10-30T12:39:05.634 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/15' 2023-10-30T12:39:05.635 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/31' 2023-10-30T12:39:05.635 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/36' 2023-10-30T12:39:05.635 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/20' 2023-10-30T12:39:05.635 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/12' 2023-10-30T12:39:05.635 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/39' 2023-10-30T12:39:05.635 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/10' 2023-10-30T12:39:05.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/34' 2023-10-30T12:39:05.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/38' 2023-10-30T12:39:05.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/11' 2023-10-30T12:39:05.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/16' 2023-10-30T12:39:05.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/32' 2023-10-30T12:39:05.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/30' 2023-10-30T12:39:05.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/14' 2023-10-30T12:39:05.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/18' 2023-10-30T12:39:05.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/13' 2023-10-30T12:39:05.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/33' 2023-10-30T12:39:05.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/17' 2023-10-30T12:39:05.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/35' 2023-10-30T12:39:05.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/37' 2023-10-30T12:39:05.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './106' 2023-10-30T12:39:05.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/19' 2023-10-30T12:39:05.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/15' 2023-10-30T12:39:05.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/31' 2023-10-30T12:39:05.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/36' 2023-10-30T12:39:05.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/20' 2023-10-30T12:39:05.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/12' 2023-10-30T12:39:05.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/39' 2023-10-30T12:39:05.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/10' 2023-10-30T12:39:05.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/34' 2023-10-30T12:39:05.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/38' 2023-10-30T12:39:05.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/11' 2023-10-30T12:39:05.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/16' 2023-10-30T12:39:05.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/32' 2023-10-30T12:39:05.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/30' 2023-10-30T12:39:05.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/14' 2023-10-30T12:39:05.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/18' 2023-10-30T12:39:05.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/13' 2023-10-30T12:39:05.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/33' 2023-10-30T12:39:05.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/17' 2023-10-30T12:39:05.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/35' 2023-10-30T12:39:05.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/37' 2023-10-30T12:39:05.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './107' 2023-10-30T12:39:05.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/19' 2023-10-30T12:39:05.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/15' 2023-10-30T12:39:05.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/31' 2023-10-30T12:39:05.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/36' 2023-10-30T12:39:05.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/20' 2023-10-30T12:39:05.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/12' 2023-10-30T12:39:05.644 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/39' 2023-10-30T12:39:05.644 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/10' 2023-10-30T12:39:05.644 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/34' 2023-10-30T12:39:05.644 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/38' 2023-10-30T12:39:05.644 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/11' 2023-10-30T12:39:05.644 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/16' 2023-10-30T12:39:05.645 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/32' 2023-10-30T12:39:05.645 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/30' 2023-10-30T12:39:05.645 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/14' 2023-10-30T12:39:05.645 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/18' 2023-10-30T12:39:05.645 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/13' 2023-10-30T12:39:05.646 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/33' 2023-10-30T12:39:05.646 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/17' 2023-10-30T12:39:05.646 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/35' 2023-10-30T12:39:05.646 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/37' 2023-10-30T12:39:05.646 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './108' 2023-10-30T12:39:05.647 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/19' 2023-10-30T12:39:05.647 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/15' 2023-10-30T12:39:05.647 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/31' 2023-10-30T12:39:05.647 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/36' 2023-10-30T12:39:05.647 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/20' 2023-10-30T12:39:05.648 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/12' 2023-10-30T12:39:05.648 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/39' 2023-10-30T12:39:05.648 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/10' 2023-10-30T12:39:05.648 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/34' 2023-10-30T12:39:05.648 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/38' 2023-10-30T12:39:05.649 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/11' 2023-10-30T12:39:05.649 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/16' 2023-10-30T12:39:05.649 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/32' 2023-10-30T12:39:05.649 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/30' 2023-10-30T12:39:09.147 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/14' 2023-10-30T12:39:09.147 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/18' 2023-10-30T12:39:09.147 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/13' 2023-10-30T12:39:09.148 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/33' 2023-10-30T12:39:09.148 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/17' 2023-10-30T12:39:09.148 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/35' 2023-10-30T12:39:09.148 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/37' 2023-10-30T12:39:09.148 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './109' 2023-10-30T12:39:09.149 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/19' 2023-10-30T12:39:09.149 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/15' 2023-10-30T12:39:09.149 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/31' 2023-10-30T12:39:09.149 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/36' 2023-10-30T12:39:09.149 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/20' 2023-10-30T12:39:09.150 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/12' 2023-10-30T12:39:09.150 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/39' 2023-10-30T12:39:09.150 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/10' 2023-10-30T12:39:09.150 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/34' 2023-10-30T12:39:09.150 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/38' 2023-10-30T12:39:09.151 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/11' 2023-10-30T12:39:09.151 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/16' 2023-10-30T12:39:09.151 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/32' 2023-10-30T12:39:09.151 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/30' 2023-10-30T12:39:09.153 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/14' 2023-10-30T12:39:09.153 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/18' 2023-10-30T12:39:09.153 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/13' 2023-10-30T12:39:09.153 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/33' 2023-10-30T12:39:09.154 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/17' 2023-10-30T12:39:09.154 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/35' 2023-10-30T12:39:09.154 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/37' 2023-10-30T12:39:09.154 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './110' 2023-10-30T12:39:09.155 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/19' 2023-10-30T12:39:09.155 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/15' 2023-10-30T12:39:09.155 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/31' 2023-10-30T12:39:09.155 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/36' 2023-10-30T12:39:09.155 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/20' 2023-10-30T12:39:09.156 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/12' 2023-10-30T12:39:09.156 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/39' 2023-10-30T12:39:09.156 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/10' 2023-10-30T12:39:09.156 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/34' 2023-10-30T12:39:09.156 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/38' 2023-10-30T12:39:09.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/11' 2023-10-30T12:39:09.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/16' 2023-10-30T12:39:09.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/32' 2023-10-30T12:39:09.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/30' 2023-10-30T12:39:09.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/14' 2023-10-30T12:39:09.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/18' 2023-10-30T12:39:09.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/13' 2023-10-30T12:39:09.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/33' 2023-10-30T12:39:09.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/17' 2023-10-30T12:39:09.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/35' 2023-10-30T12:39:09.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/37' 2023-10-30T12:39:09.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './111' 2023-10-30T12:39:09.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/19' 2023-10-30T12:39:09.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/15' 2023-10-30T12:39:09.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/31' 2023-10-30T12:39:09.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/36' 2023-10-30T12:39:09.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/20' 2023-10-30T12:39:09.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/12' 2023-10-30T12:39:09.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/39' 2023-10-30T12:39:09.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/10' 2023-10-30T12:39:09.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/34' 2023-10-30T12:39:09.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/38' 2023-10-30T12:39:09.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/11' 2023-10-30T12:39:09.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/16' 2023-10-30T12:39:09.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/32' 2023-10-30T12:39:09.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/30' 2023-10-30T12:39:09.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/14' 2023-10-30T12:39:09.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/18' 2023-10-30T12:39:09.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/13' 2023-10-30T12:39:09.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/33' 2023-10-30T12:39:09.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/17' 2023-10-30T12:39:09.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/35' 2023-10-30T12:39:09.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/37' 2023-10-30T12:39:09.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './112' 2023-10-30T12:39:09.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/19' 2023-10-30T12:39:09.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/15' 2023-10-30T12:39:09.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/31' 2023-10-30T12:39:09.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/36' 2023-10-30T12:39:09.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/20' 2023-10-30T12:39:09.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/12' 2023-10-30T12:39:09.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/39' 2023-10-30T12:39:09.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/10' 2023-10-30T12:39:09.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/34' 2023-10-30T12:39:09.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/38' 2023-10-30T12:39:09.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/11' 2023-10-30T12:39:09.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/16' 2023-10-30T12:39:09.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/32' 2023-10-30T12:39:09.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/30' 2023-10-30T12:39:09.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/14' 2023-10-30T12:39:09.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/18' 2023-10-30T12:39:09.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/13' 2023-10-30T12:39:09.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/33' 2023-10-30T12:39:09.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/17' 2023-10-30T12:39:09.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/35' 2023-10-30T12:39:09.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/37' 2023-10-30T12:39:09.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './113' 2023-10-30T12:39:09.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/19' 2023-10-30T12:39:09.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/15' 2023-10-30T12:39:09.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/31' 2023-10-30T12:39:09.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/36' 2023-10-30T12:39:09.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/20' 2023-10-30T12:39:09.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/12' 2023-10-30T12:39:09.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/39' 2023-10-30T12:39:09.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/10' 2023-10-30T12:39:09.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/34' 2023-10-30T12:39:09.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/38' 2023-10-30T12:39:09.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/11' 2023-10-30T12:39:09.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/16' 2023-10-30T12:39:09.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/32' 2023-10-30T12:39:09.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/30' 2023-10-30T12:39:09.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/14' 2023-10-30T12:39:09.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/18' 2023-10-30T12:39:09.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/13' 2023-10-30T12:39:09.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/33' 2023-10-30T12:39:09.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/17' 2023-10-30T12:39:09.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/35' 2023-10-30T12:39:09.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/37' 2023-10-30T12:39:09.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './114' 2023-10-30T12:39:09.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/19' 2023-10-30T12:39:09.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/15' 2023-10-30T12:39:09.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/31' 2023-10-30T12:39:09.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/36' 2023-10-30T12:39:09.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/20' 2023-10-30T12:39:09.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/12' 2023-10-30T12:39:09.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/39' 2023-10-30T12:39:09.174 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/10' 2023-10-30T12:39:09.174 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/34' 2023-10-30T12:39:09.174 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/38' 2023-10-30T12:39:09.174 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/11' 2023-10-30T12:39:09.174 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/16' 2023-10-30T12:39:09.175 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/32' 2023-10-30T12:39:09.175 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/30' 2023-10-30T12:39:09.175 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/14' 2023-10-30T12:39:09.175 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/18' 2023-10-30T12:39:09.175 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/13' 2023-10-30T12:39:09.176 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/33' 2023-10-30T12:39:09.176 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/17' 2023-10-30T12:39:09.176 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/35' 2023-10-30T12:39:09.176 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/37' 2023-10-30T12:39:09.176 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './115' 2023-10-30T12:39:09.176 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/19' 2023-10-30T12:39:09.177 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/15' 2023-10-30T12:39:09.177 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/31' 2023-10-30T12:39:09.177 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/36' 2023-10-30T12:39:09.177 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/20' 2023-10-30T12:39:09.178 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/12' 2023-10-30T12:39:09.178 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/39' 2023-10-30T12:39:09.178 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/10' 2023-10-30T12:39:09.178 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/34' 2023-10-30T12:39:09.178 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/38' 2023-10-30T12:39:09.179 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/11' 2023-10-30T12:39:09.179 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/16' 2023-10-30T12:39:09.179 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/32' 2023-10-30T12:39:09.179 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/30' 2023-10-30T12:39:09.179 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/14' 2023-10-30T12:39:09.179 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/18' 2023-10-30T12:39:09.180 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/13' 2023-10-30T12:39:09.180 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/33' 2023-10-30T12:39:09.180 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/17' 2023-10-30T12:39:09.180 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/35' 2023-10-30T12:39:09.180 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/37' 2023-10-30T12:39:09.181 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './116' 2023-10-30T12:39:09.181 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/19' 2023-10-30T12:39:09.181 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/15' 2023-10-30T12:39:09.181 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/31' 2023-10-30T12:39:09.182 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/36' 2023-10-30T12:39:09.182 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/20' 2023-10-30T12:39:09.182 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/12' 2023-10-30T12:39:09.182 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/39' 2023-10-30T12:39:09.182 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/10' 2023-10-30T12:39:09.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/34' 2023-10-30T12:39:09.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/38' 2023-10-30T12:39:09.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/11' 2023-10-30T12:39:09.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/16' 2023-10-30T12:39:09.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/32' 2023-10-30T12:39:09.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/30' 2023-10-30T12:39:09.184 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/14' 2023-10-30T12:39:09.184 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/18' 2023-10-30T12:39:09.184 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/13' 2023-10-30T12:39:09.184 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/33' 2023-10-30T12:39:09.184 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/17' 2023-10-30T12:39:09.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/35' 2023-10-30T12:39:09.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/37' 2023-10-30T12:39:09.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './117' 2023-10-30T12:39:09.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/19' 2023-10-30T12:39:09.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/15' 2023-10-30T12:39:09.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/31' 2023-10-30T12:39:09.186 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/36' 2023-10-30T12:39:09.186 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/20' 2023-10-30T12:39:09.186 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/12' 2023-10-30T12:39:09.186 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/39' 2023-10-30T12:39:09.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/10' 2023-10-30T12:39:09.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/34' 2023-10-30T12:39:09.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/38' 2023-10-30T12:39:09.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/11' 2023-10-30T12:39:09.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/16' 2023-10-30T12:39:09.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/32' 2023-10-30T12:39:09.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/30' 2023-10-30T12:39:09.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/14' 2023-10-30T12:39:09.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/18' 2023-10-30T12:39:09.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/13' 2023-10-30T12:39:09.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/33' 2023-10-30T12:39:09.189 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/17' 2023-10-30T12:39:09.189 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/35' 2023-10-30T12:39:09.189 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/37' 2023-10-30T12:39:09.189 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './118' 2023-10-30T12:39:09.189 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/19' 2023-10-30T12:39:09.190 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/15' 2023-10-30T12:39:09.190 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/31' 2023-10-30T12:39:09.190 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/36' 2023-10-30T12:39:09.190 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/20' 2023-10-30T12:39:09.190 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/12' 2023-10-30T12:39:11.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/39' 2023-10-30T12:39:11.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/10' 2023-10-30T12:39:11.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/34' 2023-10-30T12:39:11.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/38' 2023-10-30T12:39:11.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/11' 2023-10-30T12:39:11.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/16' 2023-10-30T12:39:11.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/32' 2023-10-30T12:39:11.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/30' 2023-10-30T12:39:11.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/14' 2023-10-30T12:39:11.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/18' 2023-10-30T12:39:11.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/13' 2023-10-30T12:39:11.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/33' 2023-10-30T12:39:11.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/17' 2023-10-30T12:39:11.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/35' 2023-10-30T12:39:11.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/37' 2023-10-30T12:39:11.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './119' 2023-10-30T12:39:11.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/19' 2023-10-30T12:39:11.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/15' 2023-10-30T12:39:11.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/31' 2023-10-30T12:39:11.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/36' 2023-10-30T12:39:11.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/20' 2023-10-30T12:39:11.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/12' 2023-10-30T12:39:11.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/39' 2023-10-30T12:39:11.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/10' 2023-10-30T12:39:11.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/34' 2023-10-30T12:39:11.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/38' 2023-10-30T12:39:11.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/11' 2023-10-30T12:39:11.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/16' 2023-10-30T12:39:11.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/32' 2023-10-30T12:39:11.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/30' 2023-10-30T12:39:11.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/14' 2023-10-30T12:39:11.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/18' 2023-10-30T12:39:11.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/13' 2023-10-30T12:39:11.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/33' 2023-10-30T12:39:11.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/17' 2023-10-30T12:39:11.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/35' 2023-10-30T12:39:11.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/37' 2023-10-30T12:39:11.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './120' 2023-10-30T12:39:11.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/19' 2023-10-30T12:39:11.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/15' 2023-10-30T12:39:11.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/31' 2023-10-30T12:39:11.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/36' 2023-10-30T12:39:11.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/20' 2023-10-30T12:39:11.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/12' 2023-10-30T12:39:11.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/39' 2023-10-30T12:39:11.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/10' 2023-10-30T12:39:11.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/34' 2023-10-30T12:39:11.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/38' 2023-10-30T12:39:11.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/11' 2023-10-30T12:39:11.566 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/16' 2023-10-30T12:39:11.566 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/32' 2023-10-30T12:39:11.566 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/30' 2023-10-30T12:39:11.566 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/14' 2023-10-30T12:39:11.566 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/18' 2023-10-30T12:39:11.567 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/13' 2023-10-30T12:39:11.567 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/33' 2023-10-30T12:39:11.567 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/17' 2023-10-30T12:39:11.567 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/35' 2023-10-30T12:39:11.567 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/37' 2023-10-30T12:39:11.567 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './121' 2023-10-30T12:39:11.568 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/19' 2023-10-30T12:39:11.568 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/15' 2023-10-30T12:39:11.568 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/31' 2023-10-30T12:39:11.568 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/36' 2023-10-30T12:39:11.569 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/20' 2023-10-30T12:39:11.569 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/12' 2023-10-30T12:39:11.569 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/39' 2023-10-30T12:39:11.569 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/10' 2023-10-30T12:39:11.569 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/34' 2023-10-30T12:39:11.570 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/38' 2023-10-30T12:39:11.570 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/11' 2023-10-30T12:39:11.570 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/16' 2023-10-30T12:39:11.570 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/32' 2023-10-30T12:39:11.570 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/30' 2023-10-30T12:39:11.570 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/14' 2023-10-30T12:39:11.571 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/18' 2023-10-30T12:39:11.571 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/13' 2023-10-30T12:39:11.571 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/33' 2023-10-30T12:39:11.571 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/17' 2023-10-30T12:39:11.571 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/35' 2023-10-30T12:39:11.571 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/37' 2023-10-30T12:39:11.572 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './122' 2023-10-30T12:39:11.572 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/19' 2023-10-30T12:39:11.572 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/15' 2023-10-30T12:39:11.572 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/31' 2023-10-30T12:39:11.572 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/36' 2023-10-30T12:39:11.573 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/20' 2023-10-30T12:39:11.573 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/12' 2023-10-30T12:39:11.573 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/39' 2023-10-30T12:39:11.573 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/10' 2023-10-30T12:39:11.573 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/34' 2023-10-30T12:39:11.573 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/38' 2023-10-30T12:39:11.574 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/11' 2023-10-30T12:39:11.574 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/16' 2023-10-30T12:39:11.574 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/32' 2023-10-30T12:39:11.574 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/30' 2023-10-30T12:39:11.574 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/14' 2023-10-30T12:39:11.574 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/18' 2023-10-30T12:39:11.575 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/13' 2023-10-30T12:39:11.575 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/33' 2023-10-30T12:39:11.575 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/17' 2023-10-30T12:39:11.575 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/35' 2023-10-30T12:39:11.575 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/37' 2023-10-30T12:39:11.575 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './123' 2023-10-30T12:39:11.576 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/19' 2023-10-30T12:39:11.576 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/15' 2023-10-30T12:39:11.576 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/31' 2023-10-30T12:39:11.576 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/36' 2023-10-30T12:39:11.576 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/20' 2023-10-30T12:39:11.576 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/12' 2023-10-30T12:39:11.577 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/39' 2023-10-30T12:39:11.577 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/10' 2023-10-30T12:39:11.577 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/34' 2023-10-30T12:39:11.577 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/38' 2023-10-30T12:39:11.577 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/11' 2023-10-30T12:39:11.577 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/16' 2023-10-30T12:39:11.578 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/32' 2023-10-30T12:39:11.578 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/30' 2023-10-30T12:39:11.578 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/14' 2023-10-30T12:39:11.578 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/18' 2023-10-30T12:39:11.578 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/13' 2023-10-30T12:39:11.579 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/33' 2023-10-30T12:39:11.579 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/17' 2023-10-30T12:39:11.579 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/35' 2023-10-30T12:39:11.579 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/37' 2023-10-30T12:39:11.579 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './124' 2023-10-30T12:39:11.579 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/19' 2023-10-30T12:39:11.580 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/15' 2023-10-30T12:39:11.580 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/31' 2023-10-30T12:39:11.580 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/36' 2023-10-30T12:39:11.580 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/20' 2023-10-30T12:39:11.580 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/12' 2023-10-30T12:39:11.581 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/39' 2023-10-30T12:39:11.581 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/10' 2023-10-30T12:39:11.581 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/34' 2023-10-30T12:39:11.581 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/38' 2023-10-30T12:39:11.582 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/11' 2023-10-30T12:39:11.582 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/16' 2023-10-30T12:39:11.582 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/32' 2023-10-30T12:39:11.582 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/30' 2023-10-30T12:39:11.582 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/14' 2023-10-30T12:39:11.583 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/18' 2023-10-30T12:39:11.583 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/13' 2023-10-30T12:39:11.583 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/33' 2023-10-30T12:39:11.583 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/17' 2023-10-30T12:39:11.583 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/35' 2023-10-30T12:39:11.584 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/37' 2023-10-30T12:39:11.584 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './125' 2023-10-30T12:39:11.584 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/19' 2023-10-30T12:39:11.584 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/15' 2023-10-30T12:39:11.584 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/31' 2023-10-30T12:39:11.585 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/36' 2023-10-30T12:39:11.585 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/20' 2023-10-30T12:39:11.585 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/12' 2023-10-30T12:39:11.585 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/39' 2023-10-30T12:39:11.585 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/10' 2023-10-30T12:39:11.585 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/34' 2023-10-30T12:39:11.586 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/38' 2023-10-30T12:39:11.586 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/11' 2023-10-30T12:39:11.586 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/16' 2023-10-30T12:39:11.586 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/32' 2023-10-30T12:39:11.586 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/30' 2023-10-30T12:39:11.587 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/14' 2023-10-30T12:39:11.587 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/18' 2023-10-30T12:39:11.587 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/13' 2023-10-30T12:39:11.587 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/33' 2023-10-30T12:39:11.587 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/17' 2023-10-30T12:39:11.588 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/35' 2023-10-30T12:39:11.588 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/37' 2023-10-30T12:39:11.588 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './126' 2023-10-30T12:39:11.588 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/19' 2023-10-30T12:39:11.588 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/15' 2023-10-30T12:39:11.589 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/31' 2023-10-30T12:39:11.589 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/36' 2023-10-30T12:39:11.589 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/20' 2023-10-30T12:39:11.589 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/12' 2023-10-30T12:39:11.589 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/39' 2023-10-30T12:39:11.589 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/10' 2023-10-30T12:39:11.590 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/34' 2023-10-30T12:39:11.590 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/38' 2023-10-30T12:39:11.590 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/11' 2023-10-30T12:39:11.590 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/16' 2023-10-30T12:39:11.590 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/32' 2023-10-30T12:39:11.590 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/30' 2023-10-30T12:39:11.591 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/14' 2023-10-30T12:39:11.591 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/18' 2023-10-30T12:39:11.591 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/13' 2023-10-30T12:39:11.591 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/33' 2023-10-30T12:39:11.591 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/17' 2023-10-30T12:39:11.592 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/35' 2023-10-30T12:39:11.592 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/37' 2023-10-30T12:39:11.592 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './127' 2023-10-30T12:39:11.592 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/19' 2023-10-30T12:39:11.592 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/15' 2023-10-30T12:39:11.592 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/31' 2023-10-30T12:39:11.593 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/36' 2023-10-30T12:39:11.593 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/20' 2023-10-30T12:39:11.593 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/12' 2023-10-30T12:39:11.593 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/39' 2023-10-30T12:39:11.593 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/10' 2023-10-30T12:39:11.593 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/34' 2023-10-30T12:39:11.594 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/38' 2023-10-30T12:39:11.594 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/11' 2023-10-30T12:39:11.594 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/16' 2023-10-30T12:39:11.594 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/32' 2023-10-30T12:39:11.594 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/30' 2023-10-30T12:39:11.594 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/14' 2023-10-30T12:39:11.595 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/18' 2023-10-30T12:39:11.595 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/13' 2023-10-30T12:39:11.595 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/33' 2023-10-30T12:39:11.595 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/17' 2023-10-30T12:39:11.595 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/35' 2023-10-30T12:39:13.975 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/37' 2023-10-30T12:39:13.975 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './128' 2023-10-30T12:39:13.976 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/19' 2023-10-30T12:39:13.976 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/15' 2023-10-30T12:39:13.976 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/31' 2023-10-30T12:39:13.976 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/36' 2023-10-30T12:39:13.977 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/20' 2023-10-30T12:39:13.977 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/12' 2023-10-30T12:39:13.977 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/39' 2023-10-30T12:39:13.977 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/10' 2023-10-30T12:39:13.977 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/34' 2023-10-30T12:39:13.978 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/38' 2023-10-30T12:39:13.978 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/11' 2023-10-30T12:39:13.978 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/16' 2023-10-30T12:39:13.978 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/32' 2023-10-30T12:39:13.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/30' 2023-10-30T12:39:13.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/14' 2023-10-30T12:39:13.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/18' 2023-10-30T12:39:13.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/13' 2023-10-30T12:39:13.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/33' 2023-10-30T12:39:13.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/17' 2023-10-30T12:39:13.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/35' 2023-10-30T12:39:13.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/37' 2023-10-30T12:39:13.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './129' 2023-10-30T12:39:13.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/19' 2023-10-30T12:39:13.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/15' 2023-10-30T12:39:13.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/31' 2023-10-30T12:39:13.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/36' 2023-10-30T12:39:13.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/20' 2023-10-30T12:39:13.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/12' 2023-10-30T12:39:13.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/39' 2023-10-30T12:39:13.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/10' 2023-10-30T12:39:13.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/34' 2023-10-30T12:39:13.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/38' 2023-10-30T12:39:13.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/11' 2023-10-30T12:39:13.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/16' 2023-10-30T12:39:13.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/32' 2023-10-30T12:39:13.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/30' 2023-10-30T12:39:13.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/14' 2023-10-30T12:39:13.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/18' 2023-10-30T12:39:13.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/13' 2023-10-30T12:39:13.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/33' 2023-10-30T12:39:13.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/17' 2023-10-30T12:39:13.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/35' 2023-10-30T12:39:13.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/37' 2023-10-30T12:39:13.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './130' 2023-10-30T12:39:13.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/19' 2023-10-30T12:39:13.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/15' 2023-10-30T12:39:13.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/31' 2023-10-30T12:39:13.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/36' 2023-10-30T12:39:13.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/20' 2023-10-30T12:39:13.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/12' 2023-10-30T12:39:13.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/39' 2023-10-30T12:39:13.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/10' 2023-10-30T12:39:13.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/34' 2023-10-30T12:39:13.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/38' 2023-10-30T12:39:13.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/11' 2023-10-30T12:39:13.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/16' 2023-10-30T12:39:13.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/32' 2023-10-30T12:39:13.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/30' 2023-10-30T12:39:13.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/14' 2023-10-30T12:39:13.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/18' 2023-10-30T12:39:13.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/13' 2023-10-30T12:39:13.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/33' 2023-10-30T12:39:13.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/17' 2023-10-30T12:39:13.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/35' 2023-10-30T12:39:13.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/37' 2023-10-30T12:39:13.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './131' 2023-10-30T12:39:13.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/19' 2023-10-30T12:39:13.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/15' 2023-10-30T12:39:13.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/31' 2023-10-30T12:39:13.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/36' 2023-10-30T12:39:13.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/20' 2023-10-30T12:39:13.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/12' 2023-10-30T12:39:13.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/39' 2023-10-30T12:39:13.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/10' 2023-10-30T12:39:13.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/34' 2023-10-30T12:39:13.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/38' 2023-10-30T12:39:13.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/11' 2023-10-30T12:39:13.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/16' 2023-10-30T12:39:13.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/32' 2023-10-30T12:39:13.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/30' 2023-10-30T12:39:13.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/14' 2023-10-30T12:39:13.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/18' 2023-10-30T12:39:13.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/13' 2023-10-30T12:39:13.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/33' 2023-10-30T12:39:13.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/17' 2023-10-30T12:39:13.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/35' 2023-10-30T12:39:13.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/37' 2023-10-30T12:39:13.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './132' 2023-10-30T12:39:13.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/19' 2023-10-30T12:39:13.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/15' 2023-10-30T12:39:13.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/31' 2023-10-30T12:39:13.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/36' 2023-10-30T12:39:13.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/20' 2023-10-30T12:39:13.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/12' 2023-10-30T12:39:13.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/39' 2023-10-30T12:39:13.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/10' 2023-10-30T12:39:13.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/34' 2023-10-30T12:39:13.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/38' 2023-10-30T12:39:13.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/11' 2023-10-30T12:39:13.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/16' 2023-10-30T12:39:13.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/32' 2023-10-30T12:39:13.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/30' 2023-10-30T12:39:13.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/14' 2023-10-30T12:39:13.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/18' 2023-10-30T12:39:13.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/13' 2023-10-30T12:39:13.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/33' 2023-10-30T12:39:13.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/17' 2023-10-30T12:39:13.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/35' 2023-10-30T12:39:13.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/37' 2023-10-30T12:39:13.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './133' 2023-10-30T12:39:13.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/19' 2023-10-30T12:39:13.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/15' 2023-10-30T12:39:13.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/31' 2023-10-30T12:39:13.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/36' 2023-10-30T12:39:13.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/20' 2023-10-30T12:39:13.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/12' 2023-10-30T12:39:13.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/39' 2023-10-30T12:39:13.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/10' 2023-10-30T12:39:13.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/34' 2023-10-30T12:39:13.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/38' 2023-10-30T12:39:13.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/11' 2023-10-30T12:39:14.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/16' 2023-10-30T12:39:14.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/32' 2023-10-30T12:39:14.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/30' 2023-10-30T12:39:14.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/14' 2023-10-30T12:39:14.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/18' 2023-10-30T12:39:14.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/13' 2023-10-30T12:39:14.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/33' 2023-10-30T12:39:14.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/17' 2023-10-30T12:39:14.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/35' 2023-10-30T12:39:14.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/37' 2023-10-30T12:39:14.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './134' 2023-10-30T12:39:14.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/19' 2023-10-30T12:39:14.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/15' 2023-10-30T12:39:14.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/31' 2023-10-30T12:39:14.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/36' 2023-10-30T12:39:14.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/20' 2023-10-30T12:39:14.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/12' 2023-10-30T12:39:14.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/39' 2023-10-30T12:39:14.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/10' 2023-10-30T12:39:14.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/34' 2023-10-30T12:39:14.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/38' 2023-10-30T12:39:14.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/11' 2023-10-30T12:39:14.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/16' 2023-10-30T12:39:14.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/32' 2023-10-30T12:39:14.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/30' 2023-10-30T12:39:14.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/14' 2023-10-30T12:39:14.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/18' 2023-10-30T12:39:14.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/13' 2023-10-30T12:39:14.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/33' 2023-10-30T12:39:14.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/17' 2023-10-30T12:39:14.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/35' 2023-10-30T12:39:14.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/37' 2023-10-30T12:39:14.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './135' 2023-10-30T12:39:14.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/19' 2023-10-30T12:39:14.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/15' 2023-10-30T12:39:14.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/31' 2023-10-30T12:39:14.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/36' 2023-10-30T12:39:14.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/20' 2023-10-30T12:39:14.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/12' 2023-10-30T12:39:14.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/39' 2023-10-30T12:39:14.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/10' 2023-10-30T12:39:14.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/34' 2023-10-30T12:39:14.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/38' 2023-10-30T12:39:14.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/11' 2023-10-30T12:39:14.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/16' 2023-10-30T12:39:14.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/32' 2023-10-30T12:39:14.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/30' 2023-10-30T12:39:14.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/14' 2023-10-30T12:39:14.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/18' 2023-10-30T12:39:14.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/13' 2023-10-30T12:39:14.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/33' 2023-10-30T12:39:14.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/17' 2023-10-30T12:39:14.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/35' 2023-10-30T12:39:14.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/37' 2023-10-30T12:39:14.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './136' 2023-10-30T12:39:14.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/19' 2023-10-30T12:39:14.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/15' 2023-10-30T12:39:14.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/31' 2023-10-30T12:39:14.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/36' 2023-10-30T12:39:14.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/20' 2023-10-30T12:39:14.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/12' 2023-10-30T12:39:14.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/39' 2023-10-30T12:39:14.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/10' 2023-10-30T12:39:14.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/34' 2023-10-30T12:39:14.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/38' 2023-10-30T12:39:14.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/11' 2023-10-30T12:39:14.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/16' 2023-10-30T12:39:14.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/32' 2023-10-30T12:39:14.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/30' 2023-10-30T12:39:14.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/14' 2023-10-30T12:39:14.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/18' 2023-10-30T12:39:14.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/13' 2023-10-30T12:39:14.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/33' 2023-10-30T12:39:14.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/17' 2023-10-30T12:39:14.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/35' 2023-10-30T12:39:14.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/37' 2023-10-30T12:39:14.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './137' 2023-10-30T12:39:14.015 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/19' 2023-10-30T12:39:14.015 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/15' 2023-10-30T12:39:14.015 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/31' 2023-10-30T12:39:14.015 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/36' 2023-10-30T12:39:14.015 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/20' 2023-10-30T12:39:14.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/12' 2023-10-30T12:39:14.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/39' 2023-10-30T12:39:14.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/10' 2023-10-30T12:39:14.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/34' 2023-10-30T12:39:14.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/38' 2023-10-30T12:39:14.017 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/11' 2023-10-30T12:39:14.017 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/16' 2023-10-30T12:39:17.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/32' 2023-10-30T12:39:17.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/30' 2023-10-30T12:39:17.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/14' 2023-10-30T12:39:17.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/18' 2023-10-30T12:39:17.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/13' 2023-10-30T12:39:17.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/33' 2023-10-30T12:39:17.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/17' 2023-10-30T12:39:17.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/35' 2023-10-30T12:39:17.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/37' 2023-10-30T12:39:17.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './138' 2023-10-30T12:39:17.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/19' 2023-10-30T12:39:17.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/15' 2023-10-30T12:39:17.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/31' 2023-10-30T12:39:17.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/36' 2023-10-30T12:39:17.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/20' 2023-10-30T12:39:17.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/12' 2023-10-30T12:39:17.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/39' 2023-10-30T12:39:17.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/10' 2023-10-30T12:39:17.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/34' 2023-10-30T12:39:17.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/38' 2023-10-30T12:39:17.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/11' 2023-10-30T12:39:17.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/16' 2023-10-30T12:39:17.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/32' 2023-10-30T12:39:17.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/30' 2023-10-30T12:39:17.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/14' 2023-10-30T12:39:17.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/18' 2023-10-30T12:39:17.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/13' 2023-10-30T12:39:17.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/33' 2023-10-30T12:39:17.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/17' 2023-10-30T12:39:17.174 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/35' 2023-10-30T12:39:17.174 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/37' 2023-10-30T12:39:17.174 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './139' 2023-10-30T12:39:17.174 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/19' 2023-10-30T12:39:17.175 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/15' 2023-10-30T12:39:17.175 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/31' 2023-10-30T12:39:17.175 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/36' 2023-10-30T12:39:17.175 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/20' 2023-10-30T12:39:17.175 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/12' 2023-10-30T12:39:17.176 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/39' 2023-10-30T12:39:17.176 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/10' 2023-10-30T12:39:17.176 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/34' 2023-10-30T12:39:17.176 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/38' 2023-10-30T12:39:17.176 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/11' 2023-10-30T12:39:17.177 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/16' 2023-10-30T12:39:17.177 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/32' 2023-10-30T12:39:17.177 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/30' 2023-10-30T12:39:17.177 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/14' 2023-10-30T12:39:17.177 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/18' 2023-10-30T12:39:17.178 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/13' 2023-10-30T12:39:17.178 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/33' 2023-10-30T12:39:17.178 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/17' 2023-10-30T12:39:17.178 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/35' 2023-10-30T12:39:17.178 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/37' 2023-10-30T12:39:17.179 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './140' 2023-10-30T12:39:17.179 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/19' 2023-10-30T12:39:17.179 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/15' 2023-10-30T12:39:17.179 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/31' 2023-10-30T12:39:17.180 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/36' 2023-10-30T12:39:17.180 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/20' 2023-10-30T12:39:17.180 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/12' 2023-10-30T12:39:17.180 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/39' 2023-10-30T12:39:17.180 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/10' 2023-10-30T12:39:17.181 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/34' 2023-10-30T12:39:17.181 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/38' 2023-10-30T12:39:17.181 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/11' 2023-10-30T12:39:17.181 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/16' 2023-10-30T12:39:17.181 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/32' 2023-10-30T12:39:17.182 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/30' 2023-10-30T12:39:17.182 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/14' 2023-10-30T12:39:17.182 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/18' 2023-10-30T12:39:17.182 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/13' 2023-10-30T12:39:17.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/33' 2023-10-30T12:39:17.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/17' 2023-10-30T12:39:17.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/35' 2023-10-30T12:39:17.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/37' 2023-10-30T12:39:17.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './141' 2023-10-30T12:39:17.184 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/19' 2023-10-30T12:39:17.184 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/15' 2023-10-30T12:39:17.184 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/31' 2023-10-30T12:39:17.184 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/36' 2023-10-30T12:39:17.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/20' 2023-10-30T12:39:17.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/12' 2023-10-30T12:39:17.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/39' 2023-10-30T12:39:17.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/10' 2023-10-30T12:39:17.186 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/34' 2023-10-30T12:39:17.186 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/38' 2023-10-30T12:39:17.186 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/11' 2023-10-30T12:39:17.186 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/16' 2023-10-30T12:39:17.186 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/32' 2023-10-30T12:39:17.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/30' 2023-10-30T12:39:17.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/14' 2023-10-30T12:39:17.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/18' 2023-10-30T12:39:17.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/13' 2023-10-30T12:39:17.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/33' 2023-10-30T12:39:17.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/17' 2023-10-30T12:39:17.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/35' 2023-10-30T12:39:17.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/37' 2023-10-30T12:39:17.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './142' 2023-10-30T12:39:17.189 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/19' 2023-10-30T12:39:17.189 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/15' 2023-10-30T12:39:17.189 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/31' 2023-10-30T12:39:17.189 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/36' 2023-10-30T12:39:17.189 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/20' 2023-10-30T12:39:17.190 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/12' 2023-10-30T12:39:17.190 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/39' 2023-10-30T12:39:17.190 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/10' 2023-10-30T12:39:17.190 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/34' 2023-10-30T12:39:17.191 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/38' 2023-10-30T12:39:17.191 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/11' 2023-10-30T12:39:17.191 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/16' 2023-10-30T12:39:17.191 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/32' 2023-10-30T12:39:17.191 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/30' 2023-10-30T12:39:17.192 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/14' 2023-10-30T12:39:17.192 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/18' 2023-10-30T12:39:17.192 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/13' 2023-10-30T12:39:17.192 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/33' 2023-10-30T12:39:17.192 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/17' 2023-10-30T12:39:17.193 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/35' 2023-10-30T12:39:17.193 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/37' 2023-10-30T12:39:17.193 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './143' 2023-10-30T12:39:17.193 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/19' 2023-10-30T12:39:17.193 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/15' 2023-10-30T12:39:17.193 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/31' 2023-10-30T12:39:17.194 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/36' 2023-10-30T12:39:17.194 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/20' 2023-10-30T12:39:17.194 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/12' 2023-10-30T12:39:17.194 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/39' 2023-10-30T12:39:17.195 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/10' 2023-10-30T12:39:17.195 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/34' 2023-10-30T12:39:17.195 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/38' 2023-10-30T12:39:17.195 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/11' 2023-10-30T12:39:17.195 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/16' 2023-10-30T12:39:17.196 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/32' 2023-10-30T12:39:17.196 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/30' 2023-10-30T12:39:17.196 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/14' 2023-10-30T12:39:17.196 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/18' 2023-10-30T12:39:17.196 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/13' 2023-10-30T12:39:17.197 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/33' 2023-10-30T12:39:17.197 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/17' 2023-10-30T12:39:17.197 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/35' 2023-10-30T12:39:17.197 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/37' 2023-10-30T12:39:17.197 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './144' 2023-10-30T12:39:17.198 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/19' 2023-10-30T12:39:17.198 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/15' 2023-10-30T12:39:17.198 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/31' 2023-10-30T12:39:17.198 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/36' 2023-10-30T12:39:17.198 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/20' 2023-10-30T12:39:17.199 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/12' 2023-10-30T12:39:17.199 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/39' 2023-10-30T12:39:17.199 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/10' 2023-10-30T12:39:17.199 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/34' 2023-10-30T12:39:17.199 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/38' 2023-10-30T12:39:17.200 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/11' 2023-10-30T12:39:17.200 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/16' 2023-10-30T12:39:17.200 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/32' 2023-10-30T12:39:17.200 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/30' 2023-10-30T12:39:17.201 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/14' 2023-10-30T12:39:17.201 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/18' 2023-10-30T12:39:17.201 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/13' 2023-10-30T12:39:17.201 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/33' 2023-10-30T12:39:17.201 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/17' 2023-10-30T12:39:17.202 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/35' 2023-10-30T12:39:17.202 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/37' 2023-10-30T12:39:17.202 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './145' 2023-10-30T12:39:17.202 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/19' 2023-10-30T12:39:17.203 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/15' 2023-10-30T12:39:17.203 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/31' 2023-10-30T12:39:17.203 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/36' 2023-10-30T12:39:17.203 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/20' 2023-10-30T12:39:17.204 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/12' 2023-10-30T12:39:17.204 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/39' 2023-10-30T12:39:17.204 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/10' 2023-10-30T12:39:17.204 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/34' 2023-10-30T12:39:17.205 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/38' 2023-10-30T12:39:17.205 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/11' 2023-10-30T12:39:17.205 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/16' 2023-10-30T12:39:17.205 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/32' 2023-10-30T12:39:17.205 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/30' 2023-10-30T12:39:17.206 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/14' 2023-10-30T12:39:17.206 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/18' 2023-10-30T12:39:17.206 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/13' 2023-10-30T12:39:17.206 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/33' 2023-10-30T12:39:17.206 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/17' 2023-10-30T12:39:17.207 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/35' 2023-10-30T12:39:17.207 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/37' 2023-10-30T12:39:17.207 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './146' 2023-10-30T12:39:17.207 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/19' 2023-10-30T12:39:17.207 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/15' 2023-10-30T12:39:17.208 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/31' 2023-10-30T12:39:17.208 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/36' 2023-10-30T12:39:17.208 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/20' 2023-10-30T12:39:17.208 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/12' 2023-10-30T12:39:17.208 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/39' 2023-10-30T12:39:17.209 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/10' 2023-10-30T12:39:17.209 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/34' 2023-10-30T12:39:17.209 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/38' 2023-10-30T12:39:17.209 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/11' 2023-10-30T12:39:17.210 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/16' 2023-10-30T12:39:17.210 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/32' 2023-10-30T12:39:17.210 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/30' 2023-10-30T12:39:17.210 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/14' 2023-10-30T12:39:17.210 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/18' 2023-10-30T12:39:17.211 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/13' 2023-10-30T12:39:17.211 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/33' 2023-10-30T12:39:17.211 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/17' 2023-10-30T12:39:17.211 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/35' 2023-10-30T12:39:17.211 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/37' 2023-10-30T12:39:17.212 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './147' 2023-10-30T12:39:17.212 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/19' 2023-10-30T12:39:17.212 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/15' 2023-10-30T12:39:17.212 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/31' 2023-10-30T12:39:17.212 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/36' 2023-10-30T12:39:21.364 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/20' 2023-10-30T12:39:21.364 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/12' 2023-10-30T12:39:21.365 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/39' 2023-10-30T12:39:21.365 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/10' 2023-10-30T12:39:21.365 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/34' 2023-10-30T12:39:21.365 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/38' 2023-10-30T12:39:21.365 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/11' 2023-10-30T12:39:21.366 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/16' 2023-10-30T12:39:21.366 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/32' 2023-10-30T12:39:21.366 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/30' 2023-10-30T12:39:21.366 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/14' 2023-10-30T12:39:21.366 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/18' 2023-10-30T12:39:21.367 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/13' 2023-10-30T12:39:21.367 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/33' 2023-10-30T12:39:21.367 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/17' 2023-10-30T12:39:21.367 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/35' 2023-10-30T12:39:21.367 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/37' 2023-10-30T12:39:21.367 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './148' 2023-10-30T12:39:21.368 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/19' 2023-10-30T12:39:21.368 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/15' 2023-10-30T12:39:21.368 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/31' 2023-10-30T12:39:21.368 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/36' 2023-10-30T12:39:21.368 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/20' 2023-10-30T12:39:21.369 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/12' 2023-10-30T12:39:21.369 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/39' 2023-10-30T12:39:21.369 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/10' 2023-10-30T12:39:21.369 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/34' 2023-10-30T12:39:21.369 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/38' 2023-10-30T12:39:21.370 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/11' 2023-10-30T12:39:21.370 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/16' 2023-10-30T12:39:21.370 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/32' 2023-10-30T12:39:21.370 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/30' 2023-10-30T12:39:21.370 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/14' 2023-10-30T12:39:21.371 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/18' 2023-10-30T12:39:21.371 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/13' 2023-10-30T12:39:21.371 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/33' 2023-10-30T12:39:21.371 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/17' 2023-10-30T12:39:21.371 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/35' 2023-10-30T12:39:21.371 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/37' 2023-10-30T12:39:21.372 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './149' 2023-10-30T12:39:21.372 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/19' 2023-10-30T12:39:21.372 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/15' 2023-10-30T12:39:21.372 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/31' 2023-10-30T12:39:21.372 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/36' 2023-10-30T12:39:21.372 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/20' 2023-10-30T12:39:21.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/12' 2023-10-30T12:39:21.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/39' 2023-10-30T12:39:21.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/10' 2023-10-30T12:39:21.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/34' 2023-10-30T12:39:21.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/38' 2023-10-30T12:39:21.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/11' 2023-10-30T12:39:21.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/16' 2023-10-30T12:39:21.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/32' 2023-10-30T12:39:21.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/30' 2023-10-30T12:39:21.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/14' 2023-10-30T12:39:21.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/18' 2023-10-30T12:39:21.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/13' 2023-10-30T12:39:21.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/33' 2023-10-30T12:39:21.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/17' 2023-10-30T12:39:21.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/35' 2023-10-30T12:39:21.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/37' 2023-10-30T12:39:21.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './150' 2023-10-30T12:39:21.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/19' 2023-10-30T12:39:21.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/15' 2023-10-30T12:39:21.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/31' 2023-10-30T12:39:21.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/36' 2023-10-30T12:39:21.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/20' 2023-10-30T12:39:21.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/12' 2023-10-30T12:39:21.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/39' 2023-10-30T12:39:21.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/10' 2023-10-30T12:39:21.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/34' 2023-10-30T12:39:21.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/38' 2023-10-30T12:39:21.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/11' 2023-10-30T12:39:21.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/16' 2023-10-30T12:39:21.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/32' 2023-10-30T12:39:21.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/30' 2023-10-30T12:39:21.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/14' 2023-10-30T12:39:21.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/18' 2023-10-30T12:39:21.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/13' 2023-10-30T12:39:21.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/33' 2023-10-30T12:39:21.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/17' 2023-10-30T12:39:21.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/35' 2023-10-30T12:39:21.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/37' 2023-10-30T12:39:21.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './151' 2023-10-30T12:39:21.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/19' 2023-10-30T12:39:21.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/15' 2023-10-30T12:39:21.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/31' 2023-10-30T12:39:21.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/36' 2023-10-30T12:39:21.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/20' 2023-10-30T12:39:21.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/12' 2023-10-30T12:39:21.381 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/39' 2023-10-30T12:39:21.381 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/10' 2023-10-30T12:39:21.381 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/34' 2023-10-30T12:39:21.381 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/38' 2023-10-30T12:39:21.381 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/11' 2023-10-30T12:39:21.381 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/16' 2023-10-30T12:39:21.382 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/32' 2023-10-30T12:39:21.382 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/30' 2023-10-30T12:39:21.382 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/14' 2023-10-30T12:39:21.382 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/18' 2023-10-30T12:39:21.382 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/13' 2023-10-30T12:39:21.382 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/33' 2023-10-30T12:39:21.383 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/17' 2023-10-30T12:39:21.383 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/35' 2023-10-30T12:39:21.383 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/37' 2023-10-30T12:39:21.383 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './152' 2023-10-30T12:39:21.383 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/19' 2023-10-30T12:39:21.384 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/15' 2023-10-30T12:39:21.384 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/31' 2023-10-30T12:39:21.384 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/36' 2023-10-30T12:39:21.384 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/20' 2023-10-30T12:39:21.384 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/12' 2023-10-30T12:39:21.384 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/39' 2023-10-30T12:39:21.385 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/10' 2023-10-30T12:39:21.385 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/34' 2023-10-30T12:39:21.385 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/38' 2023-10-30T12:39:21.385 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/11' 2023-10-30T12:39:21.385 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/16' 2023-10-30T12:39:21.386 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/32' 2023-10-30T12:39:21.386 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/30' 2023-10-30T12:39:21.386 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/14' 2023-10-30T12:39:21.386 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/18' 2023-10-30T12:39:21.386 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/13' 2023-10-30T12:39:21.386 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/33' 2023-10-30T12:39:21.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/17' 2023-10-30T12:39:21.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/35' 2023-10-30T12:39:21.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/37' 2023-10-30T12:39:21.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './153' 2023-10-30T12:39:21.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/19' 2023-10-30T12:39:21.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/15' 2023-10-30T12:39:21.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/31' 2023-10-30T12:39:21.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/36' 2023-10-30T12:39:21.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/20' 2023-10-30T12:39:21.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/12' 2023-10-30T12:39:21.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/39' 2023-10-30T12:39:21.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/10' 2023-10-30T12:39:21.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/34' 2023-10-30T12:39:21.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/38' 2023-10-30T12:39:21.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/11' 2023-10-30T12:39:21.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/16' 2023-10-30T12:39:21.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/32' 2023-10-30T12:39:21.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/30' 2023-10-30T12:39:21.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/14' 2023-10-30T12:39:21.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/18' 2023-10-30T12:39:21.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/13' 2023-10-30T12:39:21.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/33' 2023-10-30T12:39:21.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/17' 2023-10-30T12:39:21.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/35' 2023-10-30T12:39:21.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/37' 2023-10-30T12:39:21.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './154' 2023-10-30T12:39:21.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/19' 2023-10-30T12:39:21.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/15' 2023-10-30T12:39:21.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/31' 2023-10-30T12:39:21.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/36' 2023-10-30T12:39:21.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/20' 2023-10-30T12:39:21.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/12' 2023-10-30T12:39:21.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/39' 2023-10-30T12:39:21.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/10' 2023-10-30T12:39:21.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/34' 2023-10-30T12:39:21.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/38' 2023-10-30T12:39:21.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/11' 2023-10-30T12:39:21.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/16' 2023-10-30T12:39:21.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/32' 2023-10-30T12:39:21.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/30' 2023-10-30T12:39:21.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/14' 2023-10-30T12:39:21.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/18' 2023-10-30T12:39:21.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/13' 2023-10-30T12:39:21.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/33' 2023-10-30T12:39:21.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/17' 2023-10-30T12:39:21.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/35' 2023-10-30T12:39:21.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/37' 2023-10-30T12:39:21.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './155' 2023-10-30T12:39:21.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/19' 2023-10-30T12:39:21.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/15' 2023-10-30T12:39:21.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/31' 2023-10-30T12:39:21.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/36' 2023-10-30T12:39:21.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/20' 2023-10-30T12:39:21.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/12' 2023-10-30T12:39:21.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/39' 2023-10-30T12:39:21.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/10' 2023-10-30T12:39:21.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/34' 2023-10-30T12:39:21.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/38' 2023-10-30T12:39:21.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/11' 2023-10-30T12:39:21.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/16' 2023-10-30T12:39:21.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/32' 2023-10-30T12:39:21.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/30' 2023-10-30T12:39:21.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/14' 2023-10-30T12:39:21.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/18' 2023-10-30T12:39:21.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/13' 2023-10-30T12:39:21.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/33' 2023-10-30T12:39:21.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/17' 2023-10-30T12:39:21.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/35' 2023-10-30T12:39:21.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/37' 2023-10-30T12:39:21.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './156' 2023-10-30T12:39:21.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/19' 2023-10-30T12:39:21.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/15' 2023-10-30T12:39:21.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/31' 2023-10-30T12:39:21.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/36' 2023-10-30T12:39:21.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/20' 2023-10-30T12:39:21.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/12' 2023-10-30T12:39:21.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/39' 2023-10-30T12:39:21.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/10' 2023-10-30T12:39:21.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/34' 2023-10-30T12:39:21.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/38' 2023-10-30T12:39:21.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/11' 2023-10-30T12:39:21.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/16' 2023-10-30T12:39:21.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/32' 2023-10-30T12:39:21.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/30' 2023-10-30T12:39:21.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/14' 2023-10-30T12:39:21.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/18' 2023-10-30T12:39:21.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/13' 2023-10-30T12:39:21.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/33' 2023-10-30T12:39:23.511 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/17' 2023-10-30T12:39:23.512 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/35' 2023-10-30T12:39:23.512 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/37' 2023-10-30T12:39:23.512 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './157' 2023-10-30T12:39:23.512 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/19' 2023-10-30T12:39:23.512 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/15' 2023-10-30T12:39:23.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/31' 2023-10-30T12:39:23.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/36' 2023-10-30T12:39:23.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/20' 2023-10-30T12:39:23.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/12' 2023-10-30T12:39:23.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/39' 2023-10-30T12:39:23.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/10' 2023-10-30T12:39:23.514 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/34' 2023-10-30T12:39:23.514 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/38' 2023-10-30T12:39:23.515 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/11' 2023-10-30T12:39:23.515 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/16' 2023-10-30T12:39:23.515 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/32' 2023-10-30T12:39:23.515 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/30' 2023-10-30T12:39:23.516 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/14' 2023-10-30T12:39:23.516 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/18' 2023-10-30T12:39:23.516 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/13' 2023-10-30T12:39:23.516 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/33' 2023-10-30T12:39:23.516 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/17' 2023-10-30T12:39:23.517 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/35' 2023-10-30T12:39:23.517 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/37' 2023-10-30T12:39:23.517 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './158' 2023-10-30T12:39:23.517 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/19' 2023-10-30T12:39:23.517 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/15' 2023-10-30T12:39:23.518 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/31' 2023-10-30T12:39:23.518 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/36' 2023-10-30T12:39:23.518 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/20' 2023-10-30T12:39:23.518 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/12' 2023-10-30T12:39:23.518 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/39' 2023-10-30T12:39:23.519 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/10' 2023-10-30T12:39:23.519 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/34' 2023-10-30T12:39:23.519 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/38' 2023-10-30T12:39:23.519 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/11' 2023-10-30T12:39:23.519 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/16' 2023-10-30T12:39:23.519 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/32' 2023-10-30T12:39:23.520 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/30' 2023-10-30T12:39:23.520 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/14' 2023-10-30T12:39:23.520 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/18' 2023-10-30T12:39:23.520 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/13' 2023-10-30T12:39:23.520 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/33' 2023-10-30T12:39:23.520 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/17' 2023-10-30T12:39:23.521 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/35' 2023-10-30T12:39:23.521 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/37' 2023-10-30T12:39:23.521 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './159' 2023-10-30T12:39:23.521 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/19' 2023-10-30T12:39:23.521 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/15' 2023-10-30T12:39:23.521 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/31' 2023-10-30T12:39:23.522 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/36' 2023-10-30T12:39:23.522 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/20' 2023-10-30T12:39:23.522 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/12' 2023-10-30T12:39:23.522 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/39' 2023-10-30T12:39:23.522 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/10' 2023-10-30T12:39:23.523 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/34' 2023-10-30T12:39:23.523 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/38' 2023-10-30T12:39:23.523 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/11' 2023-10-30T12:39:23.523 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/16' 2023-10-30T12:39:23.523 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/32' 2023-10-30T12:39:23.523 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/30' 2023-10-30T12:39:23.524 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/14' 2023-10-30T12:39:23.524 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/18' 2023-10-30T12:39:23.524 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/13' 2023-10-30T12:39:23.524 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/33' 2023-10-30T12:39:23.524 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/17' 2023-10-30T12:39:23.524 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/35' 2023-10-30T12:39:23.525 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/37' 2023-10-30T12:39:23.525 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './160' 2023-10-30T12:39:23.525 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/19' 2023-10-30T12:39:23.525 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/15' 2023-10-30T12:39:23.525 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/31' 2023-10-30T12:39:23.525 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/36' 2023-10-30T12:39:23.526 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/20' 2023-10-30T12:39:23.526 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/12' 2023-10-30T12:39:23.526 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/39' 2023-10-30T12:39:23.526 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/10' 2023-10-30T12:39:23.526 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/34' 2023-10-30T12:39:23.527 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/38' 2023-10-30T12:39:23.527 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/11' 2023-10-30T12:39:23.527 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/16' 2023-10-30T12:39:23.527 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/32' 2023-10-30T12:39:23.527 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/30' 2023-10-30T12:39:23.527 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/14' 2023-10-30T12:39:23.528 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/18' 2023-10-30T12:39:23.528 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/13' 2023-10-30T12:39:23.528 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/33' 2023-10-30T12:39:23.528 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/17' 2023-10-30T12:39:23.528 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/35' 2023-10-30T12:39:23.529 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/37' 2023-10-30T12:39:23.529 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './161' 2023-10-30T12:39:23.529 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/19' 2023-10-30T12:39:23.529 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/15' 2023-10-30T12:39:23.529 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/31' 2023-10-30T12:39:23.529 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/36' 2023-10-30T12:39:23.530 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/20' 2023-10-30T12:39:23.530 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/12' 2023-10-30T12:39:23.530 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/39' 2023-10-30T12:39:23.530 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/10' 2023-10-30T12:39:23.530 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/34' 2023-10-30T12:39:23.531 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/38' 2023-10-30T12:39:23.531 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/11' 2023-10-30T12:39:23.531 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/16' 2023-10-30T12:39:23.531 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/32' 2023-10-30T12:39:23.531 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/30' 2023-10-30T12:39:23.531 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/14' 2023-10-30T12:39:23.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/18' 2023-10-30T12:39:23.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/13' 2023-10-30T12:39:23.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/33' 2023-10-30T12:39:23.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/17' 2023-10-30T12:39:23.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/35' 2023-10-30T12:39:23.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/37' 2023-10-30T12:39:23.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './162' 2023-10-30T12:39:23.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/19' 2023-10-30T12:39:23.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/15' 2023-10-30T12:39:23.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/31' 2023-10-30T12:39:23.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/36' 2023-10-30T12:39:23.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/20' 2023-10-30T12:39:23.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/12' 2023-10-30T12:39:23.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/39' 2023-10-30T12:39:23.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/10' 2023-10-30T12:39:23.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/34' 2023-10-30T12:39:23.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/38' 2023-10-30T12:39:23.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/11' 2023-10-30T12:39:23.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/16' 2023-10-30T12:39:23.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/32' 2023-10-30T12:39:23.536 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/30' 2023-10-30T12:39:23.536 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/14' 2023-10-30T12:39:23.536 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/18' 2023-10-30T12:39:23.536 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/13' 2023-10-30T12:39:23.536 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/33' 2023-10-30T12:39:23.536 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/17' 2023-10-30T12:39:23.537 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/35' 2023-10-30T12:39:23.537 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/37' 2023-10-30T12:39:23.537 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './163' 2023-10-30T12:39:23.537 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/19' 2023-10-30T12:39:23.537 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/15' 2023-10-30T12:39:23.537 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/31' 2023-10-30T12:39:23.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/36' 2023-10-30T12:39:23.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/20' 2023-10-30T12:39:23.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/12' 2023-10-30T12:39:23.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/39' 2023-10-30T12:39:23.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/10' 2023-10-30T12:39:23.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/34' 2023-10-30T12:39:23.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/38' 2023-10-30T12:39:23.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/11' 2023-10-30T12:39:23.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/16' 2023-10-30T12:39:23.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/32' 2023-10-30T12:39:23.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/30' 2023-10-30T12:39:23.540 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/14' 2023-10-30T12:39:23.540 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/18' 2023-10-30T12:39:23.540 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/13' 2023-10-30T12:39:23.540 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/33' 2023-10-30T12:39:23.540 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/17' 2023-10-30T12:39:23.540 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/35' 2023-10-30T12:39:23.541 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/37' 2023-10-30T12:39:23.541 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './164' 2023-10-30T12:39:23.541 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/19' 2023-10-30T12:39:23.541 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/15' 2023-10-30T12:39:23.541 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/31' 2023-10-30T12:39:23.541 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/36' 2023-10-30T12:39:23.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/20' 2023-10-30T12:39:23.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/12' 2023-10-30T12:39:23.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/39' 2023-10-30T12:39:23.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/10' 2023-10-30T12:39:23.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/34' 2023-10-30T12:39:23.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/38' 2023-10-30T12:39:23.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/11' 2023-10-30T12:39:23.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/16' 2023-10-30T12:39:23.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/32' 2023-10-30T12:39:23.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/30' 2023-10-30T12:39:23.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/14' 2023-10-30T12:39:23.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/18' 2023-10-30T12:39:23.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/13' 2023-10-30T12:39:23.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/33' 2023-10-30T12:39:23.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/17' 2023-10-30T12:39:23.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/35' 2023-10-30T12:39:23.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/37' 2023-10-30T12:39:23.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './165' 2023-10-30T12:39:23.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/19' 2023-10-30T12:39:23.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/15' 2023-10-30T12:39:23.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/31' 2023-10-30T12:39:23.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/36' 2023-10-30T12:39:23.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/20' 2023-10-30T12:39:23.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/12' 2023-10-30T12:39:23.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/39' 2023-10-30T12:39:23.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/10' 2023-10-30T12:39:23.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/34' 2023-10-30T12:39:23.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/38' 2023-10-30T12:39:23.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/11' 2023-10-30T12:39:23.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/16' 2023-10-30T12:39:23.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/32' 2023-10-30T12:39:23.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/30' 2023-10-30T12:39:23.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/14' 2023-10-30T12:39:23.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/18' 2023-10-30T12:39:23.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/13' 2023-10-30T12:39:23.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/33' 2023-10-30T12:39:23.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/17' 2023-10-30T12:39:23.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/35' 2023-10-30T12:39:23.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/37' 2023-10-30T12:39:23.549 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './166' 2023-10-30T12:39:23.549 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/19' 2023-10-30T12:39:23.549 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/15' 2023-10-30T12:39:23.549 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/31' 2023-10-30T12:39:23.549 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/36' 2023-10-30T12:39:23.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/20' 2023-10-30T12:39:23.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/12' 2023-10-30T12:39:23.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/39' 2023-10-30T12:39:23.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/10' 2023-10-30T12:39:23.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/34' 2023-10-30T12:39:23.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/38' 2023-10-30T12:39:26.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/11' 2023-10-30T12:39:26.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/16' 2023-10-30T12:39:26.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/32' 2023-10-30T12:39:26.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/30' 2023-10-30T12:39:26.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/14' 2023-10-30T12:39:26.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/18' 2023-10-30T12:39:26.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/13' 2023-10-30T12:39:26.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/33' 2023-10-30T12:39:26.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/17' 2023-10-30T12:39:26.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/35' 2023-10-30T12:39:26.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/37' 2023-10-30T12:39:26.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './167' 2023-10-30T12:39:26.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/19' 2023-10-30T12:39:26.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/15' 2023-10-30T12:39:26.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/31' 2023-10-30T12:39:26.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/36' 2023-10-30T12:39:26.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/20' 2023-10-30T12:39:26.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/12' 2023-10-30T12:39:26.301 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/39' 2023-10-30T12:39:26.301 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/10' 2023-10-30T12:39:26.301 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/34' 2023-10-30T12:39:26.301 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/38' 2023-10-30T12:39:26.301 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/11' 2023-10-30T12:39:26.302 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/16' 2023-10-30T12:39:26.302 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/32' 2023-10-30T12:39:26.302 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/30' 2023-10-30T12:39:26.302 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/14' 2023-10-30T12:39:26.303 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/18' 2023-10-30T12:39:26.303 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/13' 2023-10-30T12:39:26.303 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/33' 2023-10-30T12:39:26.303 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/17' 2023-10-30T12:39:26.303 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/35' 2023-10-30T12:39:26.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/37' 2023-10-30T12:39:26.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './168' 2023-10-30T12:39:26.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/19' 2023-10-30T12:39:26.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/15' 2023-10-30T12:39:26.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/31' 2023-10-30T12:39:26.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/36' 2023-10-30T12:39:26.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/20' 2023-10-30T12:39:26.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/12' 2023-10-30T12:39:26.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/39' 2023-10-30T12:39:26.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/10' 2023-10-30T12:39:26.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/34' 2023-10-30T12:39:26.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/38' 2023-10-30T12:39:26.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/11' 2023-10-30T12:39:26.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/16' 2023-10-30T12:39:26.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/32' 2023-10-30T12:39:26.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/30' 2023-10-30T12:39:26.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/14' 2023-10-30T12:39:26.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/18' 2023-10-30T12:39:26.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/13' 2023-10-30T12:39:26.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/33' 2023-10-30T12:39:26.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/17' 2023-10-30T12:39:26.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/35' 2023-10-30T12:39:26.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/37' 2023-10-30T12:39:26.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './169' 2023-10-30T12:39:26.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/19' 2023-10-30T12:39:26.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/15' 2023-10-30T12:39:26.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/31' 2023-10-30T12:39:26.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/36' 2023-10-30T12:39:26.310 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/20' 2023-10-30T12:39:26.310 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/12' 2023-10-30T12:39:26.310 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/39' 2023-10-30T12:39:26.310 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/10' 2023-10-30T12:39:26.310 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/34' 2023-10-30T12:39:26.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/38' 2023-10-30T12:39:26.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/11' 2023-10-30T12:39:26.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/16' 2023-10-30T12:39:26.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/32' 2023-10-30T12:39:26.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/30' 2023-10-30T12:39:26.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/14' 2023-10-30T12:39:26.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/18' 2023-10-30T12:39:26.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/13' 2023-10-30T12:39:26.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/33' 2023-10-30T12:39:26.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/17' 2023-10-30T12:39:26.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/35' 2023-10-30T12:39:26.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/37' 2023-10-30T12:39:26.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './170' 2023-10-30T12:39:26.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/19' 2023-10-30T12:39:26.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/15' 2023-10-30T12:39:26.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/31' 2023-10-30T12:39:26.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/36' 2023-10-30T12:39:26.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/20' 2023-10-30T12:39:26.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/12' 2023-10-30T12:39:26.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/39' 2023-10-30T12:39:26.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/10' 2023-10-30T12:39:26.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/34' 2023-10-30T12:39:26.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/38' 2023-10-30T12:39:26.315 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/11' 2023-10-30T12:39:26.315 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/16' 2023-10-30T12:39:26.315 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/32' 2023-10-30T12:39:26.315 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/30' 2023-10-30T12:39:26.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/14' 2023-10-30T12:39:26.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/18' 2023-10-30T12:39:26.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/13' 2023-10-30T12:39:26.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/33' 2023-10-30T12:39:26.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/17' 2023-10-30T12:39:26.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/35' 2023-10-30T12:39:26.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/37' 2023-10-30T12:39:26.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './171' 2023-10-30T12:39:26.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/19' 2023-10-30T12:39:26.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/15' 2023-10-30T12:39:26.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/31' 2023-10-30T12:39:26.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/36' 2023-10-30T12:39:26.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/20' 2023-10-30T12:39:26.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/12' 2023-10-30T12:39:26.319 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/39' 2023-10-30T12:39:26.319 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/10' 2023-10-30T12:39:26.319 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/34' 2023-10-30T12:39:26.319 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/38' 2023-10-30T12:39:26.320 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/11' 2023-10-30T12:39:26.320 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/16' 2023-10-30T12:39:26.320 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/32' 2023-10-30T12:39:26.320 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/30' 2023-10-30T12:39:26.321 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/14' 2023-10-30T12:39:26.321 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/18' 2023-10-30T12:39:26.321 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/13' 2023-10-30T12:39:26.321 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/33' 2023-10-30T12:39:26.321 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/17' 2023-10-30T12:39:26.322 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/35' 2023-10-30T12:39:26.322 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/37' 2023-10-30T12:39:26.322 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './172' 2023-10-30T12:39:26.322 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/19' 2023-10-30T12:39:26.322 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/15' 2023-10-30T12:39:26.323 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/31' 2023-10-30T12:39:26.323 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/36' 2023-10-30T12:39:26.323 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/20' 2023-10-30T12:39:26.323 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/12' 2023-10-30T12:39:26.323 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/39' 2023-10-30T12:39:26.324 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/10' 2023-10-30T12:39:26.324 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/34' 2023-10-30T12:39:26.324 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/38' 2023-10-30T12:39:26.324 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/11' 2023-10-30T12:39:26.324 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/16' 2023-10-30T12:39:26.325 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/32' 2023-10-30T12:39:26.325 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/30' 2023-10-30T12:39:26.325 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/14' 2023-10-30T12:39:26.325 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/18' 2023-10-30T12:39:26.325 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/13' 2023-10-30T12:39:26.326 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/33' 2023-10-30T12:39:26.326 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/17' 2023-10-30T12:39:26.326 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/35' 2023-10-30T12:39:26.326 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/37' 2023-10-30T12:39:26.326 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './173' 2023-10-30T12:39:26.327 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/19' 2023-10-30T12:39:26.327 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/15' 2023-10-30T12:39:26.327 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/31' 2023-10-30T12:39:26.327 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/36' 2023-10-30T12:39:26.327 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/20' 2023-10-30T12:39:26.328 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/12' 2023-10-30T12:39:26.328 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/39' 2023-10-30T12:39:26.328 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/10' 2023-10-30T12:39:26.328 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/34' 2023-10-30T12:39:26.328 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/38' 2023-10-30T12:39:26.329 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/11' 2023-10-30T12:39:26.329 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/16' 2023-10-30T12:39:26.329 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/32' 2023-10-30T12:39:26.329 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/30' 2023-10-30T12:39:26.329 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/14' 2023-10-30T12:39:26.330 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/18' 2023-10-30T12:39:26.330 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/13' 2023-10-30T12:39:26.330 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/33' 2023-10-30T12:39:26.330 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/17' 2023-10-30T12:39:26.331 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/35' 2023-10-30T12:39:26.331 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/37' 2023-10-30T12:39:26.331 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './174' 2023-10-30T12:39:26.331 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/19' 2023-10-30T12:39:26.331 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/15' 2023-10-30T12:39:26.331 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/31' 2023-10-30T12:39:26.332 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/36' 2023-10-30T12:39:26.332 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/20' 2023-10-30T12:39:26.332 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/12' 2023-10-30T12:39:26.332 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/39' 2023-10-30T12:39:26.333 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/10' 2023-10-30T12:39:26.333 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/34' 2023-10-30T12:39:26.333 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/38' 2023-10-30T12:39:26.333 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/11' 2023-10-30T12:39:26.333 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/16' 2023-10-30T12:39:26.334 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/32' 2023-10-30T12:39:26.334 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/30' 2023-10-30T12:39:26.334 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/14' 2023-10-30T12:39:26.334 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/18' 2023-10-30T12:39:26.334 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/13' 2023-10-30T12:39:26.335 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/33' 2023-10-30T12:39:26.335 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/17' 2023-10-30T12:39:26.335 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/35' 2023-10-30T12:39:26.335 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/37' 2023-10-30T12:39:26.335 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './175' 2023-10-30T12:39:26.336 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/19' 2023-10-30T12:39:26.336 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/15' 2023-10-30T12:39:26.336 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/31' 2023-10-30T12:39:26.336 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/36' 2023-10-30T12:39:26.336 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/20' 2023-10-30T12:39:26.337 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/12' 2023-10-30T12:39:26.337 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/39' 2023-10-30T12:39:26.337 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/10' 2023-10-30T12:39:26.337 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/34' 2023-10-30T12:39:26.337 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/38' 2023-10-30T12:39:26.338 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/11' 2023-10-30T12:39:26.338 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/16' 2023-10-30T12:39:26.338 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/32' 2023-10-30T12:39:26.338 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/30' 2023-10-30T12:39:26.339 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/14' 2023-10-30T12:39:26.339 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/18' 2023-10-30T12:39:26.339 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/13' 2023-10-30T12:39:26.339 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/33' 2023-10-30T12:39:26.339 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/17' 2023-10-30T12:39:26.340 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/35' 2023-10-30T12:39:26.340 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/37' 2023-10-30T12:39:26.340 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './176' 2023-10-30T12:39:26.340 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/19' 2023-10-30T12:39:26.340 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/15' 2023-10-30T12:39:30.969 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/31' 2023-10-30T12:39:30.969 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/36' 2023-10-30T12:39:30.970 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/20' 2023-10-30T12:39:30.970 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/12' 2023-10-30T12:39:30.970 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/39' 2023-10-30T12:39:30.970 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/10' 2023-10-30T12:39:30.971 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/34' 2023-10-30T12:39:30.971 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/38' 2023-10-30T12:39:30.971 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/11' 2023-10-30T12:39:30.971 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/16' 2023-10-30T12:39:30.971 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/32' 2023-10-30T12:39:30.972 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/30' 2023-10-30T12:39:30.972 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/14' 2023-10-30T12:39:30.972 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/18' 2023-10-30T12:39:30.972 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/13' 2023-10-30T12:39:30.972 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/33' 2023-10-30T12:39:30.973 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/17' 2023-10-30T12:39:30.973 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/35' 2023-10-30T12:39:30.973 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/37' 2023-10-30T12:39:30.973 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './177' 2023-10-30T12:39:30.973 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/19' 2023-10-30T12:39:30.973 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/15' 2023-10-30T12:39:30.974 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/31' 2023-10-30T12:39:30.974 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/36' 2023-10-30T12:39:30.974 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/20' 2023-10-30T12:39:30.974 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/12' 2023-10-30T12:39:30.975 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/39' 2023-10-30T12:39:30.975 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/10' 2023-10-30T12:39:30.975 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/34' 2023-10-30T12:39:30.975 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/38' 2023-10-30T12:39:30.975 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/11' 2023-10-30T12:39:30.975 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/16' 2023-10-30T12:39:30.976 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/32' 2023-10-30T12:39:30.976 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/30' 2023-10-30T12:39:30.976 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/14' 2023-10-30T12:39:30.976 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/18' 2023-10-30T12:39:30.976 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/13' 2023-10-30T12:39:30.977 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/33' 2023-10-30T12:39:30.977 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/17' 2023-10-30T12:39:30.977 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/35' 2023-10-30T12:39:30.977 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/37' 2023-10-30T12:39:30.977 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './178' 2023-10-30T12:39:30.977 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/19' 2023-10-30T12:39:30.978 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/15' 2023-10-30T12:39:30.978 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/31' 2023-10-30T12:39:30.978 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/36' 2023-10-30T12:39:30.978 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/20' 2023-10-30T12:39:30.978 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/12' 2023-10-30T12:39:30.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/39' 2023-10-30T12:39:30.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/10' 2023-10-30T12:39:30.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/34' 2023-10-30T12:39:30.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/38' 2023-10-30T12:39:30.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/11' 2023-10-30T12:39:30.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/16' 2023-10-30T12:39:30.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/32' 2023-10-30T12:39:30.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/30' 2023-10-30T12:39:30.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/14' 2023-10-30T12:39:30.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/18' 2023-10-30T12:39:30.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/13' 2023-10-30T12:39:30.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/33' 2023-10-30T12:39:30.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/17' 2023-10-30T12:39:30.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/35' 2023-10-30T12:39:30.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/37' 2023-10-30T12:39:30.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './179' 2023-10-30T12:39:30.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/19' 2023-10-30T12:39:30.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/15' 2023-10-30T12:39:30.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/31' 2023-10-30T12:39:30.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/36' 2023-10-30T12:39:30.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/20' 2023-10-30T12:39:30.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/12' 2023-10-30T12:39:30.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/39' 2023-10-30T12:39:30.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/10' 2023-10-30T12:39:30.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/34' 2023-10-30T12:39:30.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/38' 2023-10-30T12:39:30.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/11' 2023-10-30T12:39:30.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/16' 2023-10-30T12:39:30.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/32' 2023-10-30T12:39:30.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/30' 2023-10-30T12:39:30.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/14' 2023-10-30T12:39:30.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/18' 2023-10-30T12:39:30.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/13' 2023-10-30T12:39:30.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/33' 2023-10-30T12:39:30.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/17' 2023-10-30T12:39:30.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/35' 2023-10-30T12:39:30.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/37' 2023-10-30T12:39:30.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './180' 2023-10-30T12:39:30.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/19' 2023-10-30T12:39:30.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/15' 2023-10-30T12:39:30.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/31' 2023-10-30T12:39:30.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/36' 2023-10-30T12:39:30.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/20' 2023-10-30T12:39:30.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/12' 2023-10-30T12:39:30.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/39' 2023-10-30T12:39:30.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/10' 2023-10-30T12:39:30.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/34' 2023-10-30T12:39:30.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/38' 2023-10-30T12:39:30.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/11' 2023-10-30T12:39:30.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/16' 2023-10-30T12:39:30.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/32' 2023-10-30T12:39:30.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/30' 2023-10-30T12:39:30.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/14' 2023-10-30T12:39:30.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/18' 2023-10-30T12:39:30.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/13' 2023-10-30T12:39:30.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/33' 2023-10-30T12:39:30.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/17' 2023-10-30T12:39:30.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/35' 2023-10-30T12:39:30.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/37' 2023-10-30T12:39:30.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './181' 2023-10-30T12:39:30.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/19' 2023-10-30T12:39:30.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/15' 2023-10-30T12:39:30.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/31' 2023-10-30T12:39:30.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/36' 2023-10-30T12:39:30.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/20' 2023-10-30T12:39:30.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/12' 2023-10-30T12:39:30.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/39' 2023-10-30T12:39:30.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/10' 2023-10-30T12:39:30.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/34' 2023-10-30T12:39:30.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/38' 2023-10-30T12:39:30.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/11' 2023-10-30T12:39:30.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/16' 2023-10-30T12:39:30.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/32' 2023-10-30T12:39:30.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/30' 2023-10-30T12:39:30.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/14' 2023-10-30T12:39:30.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/18' 2023-10-30T12:39:30.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/13' 2023-10-30T12:39:30.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/33' 2023-10-30T12:39:30.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/17' 2023-10-30T12:39:30.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/35' 2023-10-30T12:39:30.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/37' 2023-10-30T12:39:30.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './182' 2023-10-30T12:39:30.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/19' 2023-10-30T12:39:30.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/15' 2023-10-30T12:39:30.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/31' 2023-10-30T12:39:30.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/36' 2023-10-30T12:39:30.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/20' 2023-10-30T12:39:30.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/12' 2023-10-30T12:39:30.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/39' 2023-10-30T12:39:30.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/10' 2023-10-30T12:39:30.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/34' 2023-10-30T12:39:30.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/38' 2023-10-30T12:39:30.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/11' 2023-10-30T12:39:30.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/16' 2023-10-30T12:39:30.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/32' 2023-10-30T12:39:30.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/30' 2023-10-30T12:39:30.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/14' 2023-10-30T12:39:30.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/18' 2023-10-30T12:39:30.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/13' 2023-10-30T12:39:30.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/33' 2023-10-30T12:39:30.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/17' 2023-10-30T12:39:30.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/35' 2023-10-30T12:39:30.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/37' 2023-10-30T12:39:30.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './183' 2023-10-30T12:39:30.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/19' 2023-10-30T12:39:30.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/15' 2023-10-30T12:39:30.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/31' 2023-10-30T12:39:30.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/36' 2023-10-30T12:39:30.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/20' 2023-10-30T12:39:30.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/12' 2023-10-30T12:39:30.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/39' 2023-10-30T12:39:30.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/10' 2023-10-30T12:39:30.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/34' 2023-10-30T12:39:30.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/38' 2023-10-30T12:39:30.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/11' 2023-10-30T12:39:31.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/16' 2023-10-30T12:39:31.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/32' 2023-10-30T12:39:31.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/30' 2023-10-30T12:39:31.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/14' 2023-10-30T12:39:31.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/18' 2023-10-30T12:39:31.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/13' 2023-10-30T12:39:31.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/33' 2023-10-30T12:39:31.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/17' 2023-10-30T12:39:31.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/35' 2023-10-30T12:39:31.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/37' 2023-10-30T12:39:31.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './184' 2023-10-30T12:39:31.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/19' 2023-10-30T12:39:31.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/15' 2023-10-30T12:39:31.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/31' 2023-10-30T12:39:31.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/36' 2023-10-30T12:39:31.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/20' 2023-10-30T12:39:31.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/12' 2023-10-30T12:39:31.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/39' 2023-10-30T12:39:31.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/10' 2023-10-30T12:39:31.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/34' 2023-10-30T12:39:31.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/38' 2023-10-30T12:39:31.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/11' 2023-10-30T12:39:31.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/16' 2023-10-30T12:39:31.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/32' 2023-10-30T12:39:31.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/30' 2023-10-30T12:39:31.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/14' 2023-10-30T12:39:31.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/18' 2023-10-30T12:39:31.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/13' 2023-10-30T12:39:31.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/33' 2023-10-30T12:39:31.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/17' 2023-10-30T12:39:31.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/35' 2023-10-30T12:39:31.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/37' 2023-10-30T12:39:31.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './185' 2023-10-30T12:39:31.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/19' 2023-10-30T12:39:31.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/15' 2023-10-30T12:39:31.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/31' 2023-10-30T12:39:31.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/36' 2023-10-30T12:39:31.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/20' 2023-10-30T12:39:31.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/12' 2023-10-30T12:39:31.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/39' 2023-10-30T12:39:31.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/10' 2023-10-30T12:39:31.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/34' 2023-10-30T12:39:31.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/38' 2023-10-30T12:39:31.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/11' 2023-10-30T12:39:31.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/16' 2023-10-30T12:39:31.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/32' 2023-10-30T12:39:31.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/30' 2023-10-30T12:39:31.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/14' 2023-10-30T12:39:31.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/18' 2023-10-30T12:39:32.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/13' 2023-10-30T12:39:32.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/33' 2023-10-30T12:39:32.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/17' 2023-10-30T12:39:32.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/35' 2023-10-30T12:39:32.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/37' 2023-10-30T12:39:32.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './186' 2023-10-30T12:39:32.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/19' 2023-10-30T12:39:32.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/15' 2023-10-30T12:39:32.772 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/31' 2023-10-30T12:39:32.772 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/36' 2023-10-30T12:39:32.772 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/20' 2023-10-30T12:39:32.772 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/12' 2023-10-30T12:39:32.772 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/39' 2023-10-30T12:39:32.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/10' 2023-10-30T12:39:32.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/34' 2023-10-30T12:39:32.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/38' 2023-10-30T12:39:32.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/11' 2023-10-30T12:39:32.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/16' 2023-10-30T12:39:32.774 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/32' 2023-10-30T12:39:32.774 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/30' 2023-10-30T12:39:32.774 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/14' 2023-10-30T12:39:32.774 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/18' 2023-10-30T12:39:32.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/13' 2023-10-30T12:39:32.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/33' 2023-10-30T12:39:32.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/17' 2023-10-30T12:39:32.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/35' 2023-10-30T12:39:32.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/37' 2023-10-30T12:39:32.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './187' 2023-10-30T12:39:32.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/19' 2023-10-30T12:39:32.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/15' 2023-10-30T12:39:32.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/31' 2023-10-30T12:39:32.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/36' 2023-10-30T12:39:32.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/20' 2023-10-30T12:39:32.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/12' 2023-10-30T12:39:32.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/39' 2023-10-30T12:39:32.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/10' 2023-10-30T12:39:32.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/34' 2023-10-30T12:39:32.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/38' 2023-10-30T12:39:32.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/11' 2023-10-30T12:39:32.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/16' 2023-10-30T12:39:32.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/32' 2023-10-30T12:39:32.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/30' 2023-10-30T12:39:32.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/14' 2023-10-30T12:39:32.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/18' 2023-10-30T12:39:32.779 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/13' 2023-10-30T12:39:32.779 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/33' 2023-10-30T12:39:32.779 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/17' 2023-10-30T12:39:32.779 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/35' 2023-10-30T12:39:32.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/37' 2023-10-30T12:39:32.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './188' 2023-10-30T12:39:32.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/19' 2023-10-30T12:39:32.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/15' 2023-10-30T12:39:32.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/31' 2023-10-30T12:39:32.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/36' 2023-10-30T12:39:32.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/20' 2023-10-30T12:39:32.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/12' 2023-10-30T12:39:32.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/39' 2023-10-30T12:39:32.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/10' 2023-10-30T12:39:32.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/34' 2023-10-30T12:39:32.782 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/38' 2023-10-30T12:39:32.782 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/11' 2023-10-30T12:39:32.782 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/16' 2023-10-30T12:39:32.782 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/32' 2023-10-30T12:39:32.782 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/30' 2023-10-30T12:39:32.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/14' 2023-10-30T12:39:32.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/18' 2023-10-30T12:39:32.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/13' 2023-10-30T12:39:32.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/33' 2023-10-30T12:39:32.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/17' 2023-10-30T12:39:32.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/35' 2023-10-30T12:39:32.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/37' 2023-10-30T12:39:32.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './189' 2023-10-30T12:39:32.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/19' 2023-10-30T12:39:32.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/15' 2023-10-30T12:39:32.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/31' 2023-10-30T12:39:32.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/36' 2023-10-30T12:39:32.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/20' 2023-10-30T12:39:32.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/12' 2023-10-30T12:39:32.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/39' 2023-10-30T12:39:32.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/10' 2023-10-30T12:39:32.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/34' 2023-10-30T12:39:32.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/38' 2023-10-30T12:39:32.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/11' 2023-10-30T12:39:32.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/16' 2023-10-30T12:39:32.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/32' 2023-10-30T12:39:32.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/30' 2023-10-30T12:39:32.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/14' 2023-10-30T12:39:32.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/18' 2023-10-30T12:39:32.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/13' 2023-10-30T12:39:32.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/33' 2023-10-30T12:39:32.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/17' 2023-10-30T12:39:32.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/35' 2023-10-30T12:39:32.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/37' 2023-10-30T12:39:32.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './190' 2023-10-30T12:39:32.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/19' 2023-10-30T12:39:32.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/15' 2023-10-30T12:39:32.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/31' 2023-10-30T12:39:32.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/36' 2023-10-30T12:39:32.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/20' 2023-10-30T12:39:32.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/12' 2023-10-30T12:39:32.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/39' 2023-10-30T12:39:32.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/10' 2023-10-30T12:39:32.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/34' 2023-10-30T12:39:32.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/38' 2023-10-30T12:39:32.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/11' 2023-10-30T12:39:32.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/16' 2023-10-30T12:39:32.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/32' 2023-10-30T12:39:32.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/30' 2023-10-30T12:39:32.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/14' 2023-10-30T12:39:32.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/18' 2023-10-30T12:39:32.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/13' 2023-10-30T12:39:32.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/33' 2023-10-30T12:39:32.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/17' 2023-10-30T12:39:32.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/35' 2023-10-30T12:39:32.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/37' 2023-10-30T12:39:32.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './191' 2023-10-30T12:39:32.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/19' 2023-10-30T12:39:32.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/15' 2023-10-30T12:39:32.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/31' 2023-10-30T12:39:32.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/36' 2023-10-30T12:39:32.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/20' 2023-10-30T12:39:32.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/12' 2023-10-30T12:39:32.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/39' 2023-10-30T12:39:32.794 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/10' 2023-10-30T12:39:32.794 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/34' 2023-10-30T12:39:32.794 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/38' 2023-10-30T12:39:32.794 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/11' 2023-10-30T12:39:32.794 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/16' 2023-10-30T12:39:32.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/32' 2023-10-30T12:39:32.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/30' 2023-10-30T12:39:32.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/14' 2023-10-30T12:39:32.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/18' 2023-10-30T12:39:32.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/13' 2023-10-30T12:39:32.796 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/33' 2023-10-30T12:39:32.796 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/17' 2023-10-30T12:39:32.796 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/35' 2023-10-30T12:39:32.796 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/37' 2023-10-30T12:39:32.796 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './192' 2023-10-30T12:39:32.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/19' 2023-10-30T12:39:32.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/15' 2023-10-30T12:39:32.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/31' 2023-10-30T12:39:32.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/36' 2023-10-30T12:39:32.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/20' 2023-10-30T12:39:32.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/12' 2023-10-30T12:39:32.798 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/39' 2023-10-30T12:39:32.798 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/10' 2023-10-30T12:39:32.798 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/34' 2023-10-30T12:39:32.798 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/38' 2023-10-30T12:39:32.799 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/11' 2023-10-30T12:39:32.799 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/16' 2023-10-30T12:39:32.799 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/32' 2023-10-30T12:39:32.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/30' 2023-10-30T12:39:32.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/14' 2023-10-30T12:39:32.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/18' 2023-10-30T12:39:32.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/13' 2023-10-30T12:39:32.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/33' 2023-10-30T12:39:32.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/17' 2023-10-30T12:39:32.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/35' 2023-10-30T12:39:32.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/37' 2023-10-30T12:39:32.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './193' 2023-10-30T12:39:32.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/19' 2023-10-30T12:39:32.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/15' 2023-10-30T12:39:32.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/31' 2023-10-30T12:39:32.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/36' 2023-10-30T12:39:32.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/20' 2023-10-30T12:39:32.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/12' 2023-10-30T12:39:32.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/39' 2023-10-30T12:39:32.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/10' 2023-10-30T12:39:32.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/34' 2023-10-30T12:39:32.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/38' 2023-10-30T12:39:32.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/11' 2023-10-30T12:39:32.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/16' 2023-10-30T12:39:32.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/32' 2023-10-30T12:39:32.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/30' 2023-10-30T12:39:32.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/14' 2023-10-30T12:39:32.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/18' 2023-10-30T12:39:32.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/13' 2023-10-30T12:39:32.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/33' 2023-10-30T12:39:32.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/17' 2023-10-30T12:39:32.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/35' 2023-10-30T12:39:32.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/37' 2023-10-30T12:39:32.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './194' 2023-10-30T12:39:32.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/19' 2023-10-30T12:39:32.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/15' 2023-10-30T12:39:32.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/31' 2023-10-30T12:39:32.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/36' 2023-10-30T12:39:32.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/20' 2023-10-30T12:39:32.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/12' 2023-10-30T12:39:32.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/39' 2023-10-30T12:39:32.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/10' 2023-10-30T12:39:32.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/34' 2023-10-30T12:39:32.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/38' 2023-10-30T12:39:32.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/11' 2023-10-30T12:39:32.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/16' 2023-10-30T12:39:32.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/32' 2023-10-30T12:39:32.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/30' 2023-10-30T12:39:32.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/14' 2023-10-30T12:39:32.810 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/18' 2023-10-30T12:39:32.810 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/13' 2023-10-30T12:39:32.810 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/33' 2023-10-30T12:39:32.810 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/17' 2023-10-30T12:39:32.811 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/35' 2023-10-30T12:39:32.811 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/37' 2023-10-30T12:39:32.811 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './195' 2023-10-30T12:39:32.811 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/19' 2023-10-30T12:39:32.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/15' 2023-10-30T12:39:32.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/31' 2023-10-30T12:39:32.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/36' 2023-10-30T12:39:32.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/20' 2023-10-30T12:39:32.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/12' 2023-10-30T12:39:32.813 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/39' 2023-10-30T12:39:32.813 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/10' 2023-10-30T12:39:33.899 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:39:33.904 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:39:34.335 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/34' 2023-10-30T12:39:34.335 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/38' 2023-10-30T12:39:34.336 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/11' 2023-10-30T12:39:34.336 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/16' 2023-10-30T12:39:34.336 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/32' 2023-10-30T12:39:34.336 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/30' 2023-10-30T12:39:34.336 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/14' 2023-10-30T12:39:34.337 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/18' 2023-10-30T12:39:34.337 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/13' 2023-10-30T12:39:34.337 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/33' 2023-10-30T12:39:34.337 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/17' 2023-10-30T12:39:34.337 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/35' 2023-10-30T12:39:34.338 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/37' 2023-10-30T12:39:34.338 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './196' 2023-10-30T12:39:34.338 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/19' 2023-10-30T12:39:34.338 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/15' 2023-10-30T12:39:34.338 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/31' 2023-10-30T12:39:34.339 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/36' 2023-10-30T12:39:34.339 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/20' 2023-10-30T12:39:34.339 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/12' 2023-10-30T12:39:34.339 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/39' 2023-10-30T12:39:34.339 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/10' 2023-10-30T12:39:34.339 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/34' 2023-10-30T12:39:34.340 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/38' 2023-10-30T12:39:34.340 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/11' 2023-10-30T12:39:34.340 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/16' 2023-10-30T12:39:34.340 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/32' 2023-10-30T12:39:34.340 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/30' 2023-10-30T12:39:34.341 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/14' 2023-10-30T12:39:34.341 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/18' 2023-10-30T12:39:34.341 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/13' 2023-10-30T12:39:34.341 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/33' 2023-10-30T12:39:34.341 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/17' 2023-10-30T12:39:34.341 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/35' 2023-10-30T12:39:34.342 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/37' 2023-10-30T12:39:34.342 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './197' 2023-10-30T12:39:34.342 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/19' 2023-10-30T12:39:34.342 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/15' 2023-10-30T12:39:34.342 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/31' 2023-10-30T12:39:34.343 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/36' 2023-10-30T12:39:34.343 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/20' 2023-10-30T12:39:34.343 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/12' 2023-10-30T12:39:34.343 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/39' 2023-10-30T12:39:34.343 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/10' 2023-10-30T12:39:34.344 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/34' 2023-10-30T12:39:34.344 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/38' 2023-10-30T12:39:34.344 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/11' 2023-10-30T12:39:34.344 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/16' 2023-10-30T12:39:34.344 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/32' 2023-10-30T12:39:34.345 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/30' 2023-10-30T12:39:34.345 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/14' 2023-10-30T12:39:34.345 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/18' 2023-10-30T12:39:34.345 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/13' 2023-10-30T12:39:34.345 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/33' 2023-10-30T12:39:34.345 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/17' 2023-10-30T12:39:34.346 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/35' 2023-10-30T12:39:34.346 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/37' 2023-10-30T12:39:34.346 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './198' 2023-10-30T12:39:34.346 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/19' 2023-10-30T12:39:34.347 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/15' 2023-10-30T12:39:34.347 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/31' 2023-10-30T12:39:34.347 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/36' 2023-10-30T12:39:34.347 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/20' 2023-10-30T12:39:34.347 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/12' 2023-10-30T12:39:34.347 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/39' 2023-10-30T12:39:34.348 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/10' 2023-10-30T12:39:34.348 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/34' 2023-10-30T12:39:34.348 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/38' 2023-10-30T12:39:34.348 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/11' 2023-10-30T12:39:34.348 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/16' 2023-10-30T12:39:34.349 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/32' 2023-10-30T12:39:34.349 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/30' 2023-10-30T12:39:34.349 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/14' 2023-10-30T12:39:34.349 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/18' 2023-10-30T12:39:34.349 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/13' 2023-10-30T12:39:34.350 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/33' 2023-10-30T12:39:34.350 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/17' 2023-10-30T12:39:34.350 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/35' 2023-10-30T12:39:34.350 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/37' 2023-10-30T12:39:34.350 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './199' 2023-10-30T12:39:34.351 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/21' 2023-10-30T12:39:34.351 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/27' 2023-10-30T12:39:34.351 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/31' 2023-10-30T12:39:34.351 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/36' 2023-10-30T12:39:34.351 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/20' 2023-10-30T12:39:34.351 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/39' 2023-10-30T12:39:34.352 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/34' 2023-10-30T12:39:34.352 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/38' 2023-10-30T12:39:34.352 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/28' 2023-10-30T12:39:34.352 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/24' 2023-10-30T12:39:34.352 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/29' 2023-10-30T12:39:34.352 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/32' 2023-10-30T12:39:34.353 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/22' 2023-10-30T12:39:34.353 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/25' 2023-10-30T12:39:34.353 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/30' 2023-10-30T12:39:34.353 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/33' 2023-10-30T12:39:34.353 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/23' 2023-10-30T12:39:34.354 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/35' 2023-10-30T12:39:34.354 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/26' 2023-10-30T12:39:34.354 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/37' 2023-10-30T12:39:34.354 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './200' 2023-10-30T12:39:34.354 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/21' 2023-10-30T12:39:34.354 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/27' 2023-10-30T12:39:34.355 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/31' 2023-10-30T12:39:34.355 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/36' 2023-10-30T12:39:34.355 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/20' 2023-10-30T12:39:34.355 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/39' 2023-10-30T12:39:34.355 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/34' 2023-10-30T12:39:34.355 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/38' 2023-10-30T12:39:34.356 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/28' 2023-10-30T12:39:34.356 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/24' 2023-10-30T12:39:34.356 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/29' 2023-10-30T12:39:34.356 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/32' 2023-10-30T12:39:34.356 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/22' 2023-10-30T12:39:34.356 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/25' 2023-10-30T12:39:34.357 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/30' 2023-10-30T12:39:34.357 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/33' 2023-10-30T12:39:34.357 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/23' 2023-10-30T12:39:34.357 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/35' 2023-10-30T12:39:34.357 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/26' 2023-10-30T12:39:34.357 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/37' 2023-10-30T12:39:34.358 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './201' 2023-10-30T12:39:34.358 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/21' 2023-10-30T12:39:34.358 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/27' 2023-10-30T12:39:34.358 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/31' 2023-10-30T12:39:34.359 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/36' 2023-10-30T12:39:34.359 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/20' 2023-10-30T12:39:34.359 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/39' 2023-10-30T12:39:34.359 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/34' 2023-10-30T12:39:34.360 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/38' 2023-10-30T12:39:34.360 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/28' 2023-10-30T12:39:34.360 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/24' 2023-10-30T12:39:34.360 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/29' 2023-10-30T12:39:34.360 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/32' 2023-10-30T12:39:34.361 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/22' 2023-10-30T12:39:34.361 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/25' 2023-10-30T12:39:34.361 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/30' 2023-10-30T12:39:34.361 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/33' 2023-10-30T12:39:34.361 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/23' 2023-10-30T12:39:34.362 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/35' 2023-10-30T12:39:34.362 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/26' 2023-10-30T12:39:34.362 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/37' 2023-10-30T12:39:34.362 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './202' 2023-10-30T12:39:34.362 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/21' 2023-10-30T12:39:34.362 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/27' 2023-10-30T12:39:34.363 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/31' 2023-10-30T12:39:34.363 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/36' 2023-10-30T12:39:34.363 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/20' 2023-10-30T12:39:34.363 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/39' 2023-10-30T12:39:34.363 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/34' 2023-10-30T12:39:34.364 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/38' 2023-10-30T12:39:34.364 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/28' 2023-10-30T12:39:34.364 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/24' 2023-10-30T12:39:34.364 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/29' 2023-10-30T12:39:34.364 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/32' 2023-10-30T12:39:34.365 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/22' 2023-10-30T12:39:34.365 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/25' 2023-10-30T12:39:34.365 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/30' 2023-10-30T12:39:34.365 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/33' 2023-10-30T12:39:34.365 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/23' 2023-10-30T12:39:34.366 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/35' 2023-10-30T12:39:34.366 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/26' 2023-10-30T12:39:34.366 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/37' 2023-10-30T12:39:34.366 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './203' 2023-10-30T12:39:34.366 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/21' 2023-10-30T12:39:34.367 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/27' 2023-10-30T12:39:34.367 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/31' 2023-10-30T12:39:34.367 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/36' 2023-10-30T12:39:34.367 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/20' 2023-10-30T12:39:34.367 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/39' 2023-10-30T12:39:34.367 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/34' 2023-10-30T12:39:34.368 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/38' 2023-10-30T12:39:34.368 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/28' 2023-10-30T12:39:34.368 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/24' 2023-10-30T12:39:34.368 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/29' 2023-10-30T12:39:34.368 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/32' 2023-10-30T12:39:34.369 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/22' 2023-10-30T12:39:34.369 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/25' 2023-10-30T12:39:34.369 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/30' 2023-10-30T12:39:34.369 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/33' 2023-10-30T12:39:34.369 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/23' 2023-10-30T12:39:34.369 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/35' 2023-10-30T12:39:34.370 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/26' 2023-10-30T12:39:34.370 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/37' 2023-10-30T12:39:34.370 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './204' 2023-10-30T12:39:34.370 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/21' 2023-10-30T12:39:34.370 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/27' 2023-10-30T12:39:34.371 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/31' 2023-10-30T12:39:34.371 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/36' 2023-10-30T12:39:34.371 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/20' 2023-10-30T12:39:34.371 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/39' 2023-10-30T12:39:34.371 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/34' 2023-10-30T12:39:34.371 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/38' 2023-10-30T12:39:34.372 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/28' 2023-10-30T12:39:34.372 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/24' 2023-10-30T12:39:34.372 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/29' 2023-10-30T12:39:34.372 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/32' 2023-10-30T12:39:34.372 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/22' 2023-10-30T12:39:34.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/25' 2023-10-30T12:39:34.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/30' 2023-10-30T12:39:34.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/33' 2023-10-30T12:39:34.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/23' 2023-10-30T12:39:34.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/35' 2023-10-30T12:39:34.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/26' 2023-10-30T12:39:34.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/37' 2023-10-30T12:39:34.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './205' 2023-10-30T12:39:34.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/21' 2023-10-30T12:39:34.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/27' 2023-10-30T12:39:34.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/31' 2023-10-30T12:39:34.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/36' 2023-10-30T12:39:34.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/20' 2023-10-30T12:39:34.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/39' 2023-10-30T12:39:35.681 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/34' 2023-10-30T12:39:35.681 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/38' 2023-10-30T12:39:35.681 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/28' 2023-10-30T12:39:35.682 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/24' 2023-10-30T12:39:35.682 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/29' 2023-10-30T12:39:35.682 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/32' 2023-10-30T12:39:35.682 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/22' 2023-10-30T12:39:35.682 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/25' 2023-10-30T12:39:35.683 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/30' 2023-10-30T12:39:35.683 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/33' 2023-10-30T12:39:35.683 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/23' 2023-10-30T12:39:35.683 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/35' 2023-10-30T12:39:35.683 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/26' 2023-10-30T12:39:35.683 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/37' 2023-10-30T12:39:35.684 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './206' 2023-10-30T12:39:35.684 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/21' 2023-10-30T12:39:35.684 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/27' 2023-10-30T12:39:35.684 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/31' 2023-10-30T12:39:35.684 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/36' 2023-10-30T12:39:35.685 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/20' 2023-10-30T12:39:35.685 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/39' 2023-10-30T12:39:35.685 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/34' 2023-10-30T12:39:35.685 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/38' 2023-10-30T12:39:35.685 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/28' 2023-10-30T12:39:35.686 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/24' 2023-10-30T12:39:35.686 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/29' 2023-10-30T12:39:35.686 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/32' 2023-10-30T12:39:35.686 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/22' 2023-10-30T12:39:35.686 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/25' 2023-10-30T12:39:35.686 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/30' 2023-10-30T12:39:35.687 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/33' 2023-10-30T12:39:35.687 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/23' 2023-10-30T12:39:35.687 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/35' 2023-10-30T12:39:35.687 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/26' 2023-10-30T12:39:35.687 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/37' 2023-10-30T12:39:35.687 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './207' 2023-10-30T12:39:35.688 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/21' 2023-10-30T12:39:35.688 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/27' 2023-10-30T12:39:35.688 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/31' 2023-10-30T12:39:35.688 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/36' 2023-10-30T12:39:35.688 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/20' 2023-10-30T12:39:35.689 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/39' 2023-10-30T12:39:35.689 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/34' 2023-10-30T12:39:35.689 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/38' 2023-10-30T12:39:35.689 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/28' 2023-10-30T12:39:35.689 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/24' 2023-10-30T12:39:35.689 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/29' 2023-10-30T12:39:35.690 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/32' 2023-10-30T12:39:35.690 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/22' 2023-10-30T12:39:35.690 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/25' 2023-10-30T12:39:35.690 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/30' 2023-10-30T12:39:35.690 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/33' 2023-10-30T12:39:35.691 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/23' 2023-10-30T12:39:35.691 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/35' 2023-10-30T12:39:35.691 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/26' 2023-10-30T12:39:35.691 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/37' 2023-10-30T12:39:35.691 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './208' 2023-10-30T12:39:35.692 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/21' 2023-10-30T12:39:35.692 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/27' 2023-10-30T12:39:35.692 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/31' 2023-10-30T12:39:35.692 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/36' 2023-10-30T12:39:35.692 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/20' 2023-10-30T12:39:35.693 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/39' 2023-10-30T12:39:35.693 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/34' 2023-10-30T12:39:35.693 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/38' 2023-10-30T12:39:35.693 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/28' 2023-10-30T12:39:35.693 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/24' 2023-10-30T12:39:35.693 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/29' 2023-10-30T12:39:35.694 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/32' 2023-10-30T12:39:35.694 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/22' 2023-10-30T12:39:35.694 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/25' 2023-10-30T12:39:35.694 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/30' 2023-10-30T12:39:35.694 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/33' 2023-10-30T12:39:35.695 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/23' 2023-10-30T12:39:35.695 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/35' 2023-10-30T12:39:35.695 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/26' 2023-10-30T12:39:35.695 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/37' 2023-10-30T12:39:35.695 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './209' 2023-10-30T12:39:35.695 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/21' 2023-10-30T12:39:35.696 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/27' 2023-10-30T12:39:35.696 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/31' 2023-10-30T12:39:35.696 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/36' 2023-10-30T12:39:35.696 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/20' 2023-10-30T12:39:35.696 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/39' 2023-10-30T12:39:35.696 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/34' 2023-10-30T12:39:35.697 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/38' 2023-10-30T12:39:35.697 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/28' 2023-10-30T12:39:35.697 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/24' 2023-10-30T12:39:35.697 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/29' 2023-10-30T12:39:35.697 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/32' 2023-10-30T12:39:35.697 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/22' 2023-10-30T12:39:35.698 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/25' 2023-10-30T12:39:35.698 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/30' 2023-10-30T12:39:35.698 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/33' 2023-10-30T12:39:35.698 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/23' 2023-10-30T12:39:35.698 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/35' 2023-10-30T12:39:35.699 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/26' 2023-10-30T12:39:35.699 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/37' 2023-10-30T12:39:35.699 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './210' 2023-10-30T12:39:35.699 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/21' 2023-10-30T12:39:35.699 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/27' 2023-10-30T12:39:35.699 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/31' 2023-10-30T12:39:35.700 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/36' 2023-10-30T12:39:35.700 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/20' 2023-10-30T12:39:35.700 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/39' 2023-10-30T12:39:35.700 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/34' 2023-10-30T12:39:35.700 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/38' 2023-10-30T12:39:35.700 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/28' 2023-10-30T12:39:35.701 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/24' 2023-10-30T12:39:35.701 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/29' 2023-10-30T12:39:35.701 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/32' 2023-10-30T12:39:35.701 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/22' 2023-10-30T12:39:35.701 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/25' 2023-10-30T12:39:35.702 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/30' 2023-10-30T12:39:35.702 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/33' 2023-10-30T12:39:35.702 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/23' 2023-10-30T12:39:35.702 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/35' 2023-10-30T12:39:35.702 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/26' 2023-10-30T12:39:35.702 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/37' 2023-10-30T12:39:35.703 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './211' 2023-10-30T12:39:35.703 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/21' 2023-10-30T12:39:35.703 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/27' 2023-10-30T12:39:35.703 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/31' 2023-10-30T12:39:35.703 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/36' 2023-10-30T12:39:35.703 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/20' 2023-10-30T12:39:35.704 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/39' 2023-10-30T12:39:35.704 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/34' 2023-10-30T12:39:35.704 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/38' 2023-10-30T12:39:35.704 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/28' 2023-10-30T12:39:35.704 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/24' 2023-10-30T12:39:35.704 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/29' 2023-10-30T12:39:35.705 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/32' 2023-10-30T12:39:35.705 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/22' 2023-10-30T12:39:35.705 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/25' 2023-10-30T12:39:35.705 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/30' 2023-10-30T12:39:35.705 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/33' 2023-10-30T12:39:35.706 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/23' 2023-10-30T12:39:35.706 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/35' 2023-10-30T12:39:35.706 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/26' 2023-10-30T12:39:35.706 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/37' 2023-10-30T12:39:35.706 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './212' 2023-10-30T12:39:35.706 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/21' 2023-10-30T12:39:35.707 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/27' 2023-10-30T12:39:35.707 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/31' 2023-10-30T12:39:35.707 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/36' 2023-10-30T12:39:35.707 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/20' 2023-10-30T12:39:35.707 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/39' 2023-10-30T12:39:35.707 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/34' 2023-10-30T12:39:35.708 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/38' 2023-10-30T12:39:35.708 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/28' 2023-10-30T12:39:35.708 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/24' 2023-10-30T12:39:35.708 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/29' 2023-10-30T12:39:35.708 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/32' 2023-10-30T12:39:35.709 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/22' 2023-10-30T12:39:35.709 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/25' 2023-10-30T12:39:35.709 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/30' 2023-10-30T12:39:35.709 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/33' 2023-10-30T12:39:35.709 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/23' 2023-10-30T12:39:35.709 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/35' 2023-10-30T12:39:35.710 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/26' 2023-10-30T12:39:35.710 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/37' 2023-10-30T12:39:35.710 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './213' 2023-10-30T12:39:35.710 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/21' 2023-10-30T12:39:35.710 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/27' 2023-10-30T12:39:35.711 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/31' 2023-10-30T12:39:35.711 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/36' 2023-10-30T12:39:35.711 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/20' 2023-10-30T12:39:35.711 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/39' 2023-10-30T12:39:35.711 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/34' 2023-10-30T12:39:35.711 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/38' 2023-10-30T12:39:35.712 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/28' 2023-10-30T12:39:35.712 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/24' 2023-10-30T12:39:35.712 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/29' 2023-10-30T12:39:35.712 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/32' 2023-10-30T12:39:35.712 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/22' 2023-10-30T12:39:35.713 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/25' 2023-10-30T12:39:35.713 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/30' 2023-10-30T12:39:35.713 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/33' 2023-10-30T12:39:35.713 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/23' 2023-10-30T12:39:35.713 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/35' 2023-10-30T12:39:35.713 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/26' 2023-10-30T12:39:35.714 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/37' 2023-10-30T12:39:35.714 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './214' 2023-10-30T12:39:35.714 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/21' 2023-10-30T12:39:35.714 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/27' 2023-10-30T12:39:35.714 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/31' 2023-10-30T12:39:35.714 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/36' 2023-10-30T12:39:35.715 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/20' 2023-10-30T12:39:35.715 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/39' 2023-10-30T12:39:35.715 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/34' 2023-10-30T12:39:35.715 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/38' 2023-10-30T12:39:35.715 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/28' 2023-10-30T12:39:35.715 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/24' 2023-10-30T12:39:35.716 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/29' 2023-10-30T12:39:35.716 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/32' 2023-10-30T12:39:35.716 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/22' 2023-10-30T12:39:35.716 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/25' 2023-10-30T12:39:35.716 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/30' 2023-10-30T12:39:35.717 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/33' 2023-10-30T12:39:35.717 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/23' 2023-10-30T12:39:35.717 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/35' 2023-10-30T12:39:35.717 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/26' 2023-10-30T12:39:35.717 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/37' 2023-10-30T12:39:35.717 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './215' 2023-10-30T12:39:35.718 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/21' 2023-10-30T12:39:35.718 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/27' 2023-10-30T12:39:35.718 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/31' 2023-10-30T12:39:35.718 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/36' 2023-10-30T12:39:35.718 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/20' 2023-10-30T12:39:35.718 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/39' 2023-10-30T12:39:35.719 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/34' 2023-10-30T12:39:35.719 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/38' 2023-10-30T12:39:37.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/28' 2023-10-30T12:39:37.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/24' 2023-10-30T12:39:37.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/29' 2023-10-30T12:39:37.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/32' 2023-10-30T12:39:37.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/22' 2023-10-30T12:39:37.772 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/25' 2023-10-30T12:39:37.772 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/30' 2023-10-30T12:39:37.772 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/33' 2023-10-30T12:39:37.772 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/23' 2023-10-30T12:39:37.772 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/35' 2023-10-30T12:39:37.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/26' 2023-10-30T12:39:37.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/37' 2023-10-30T12:39:37.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './216' 2023-10-30T12:39:37.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/21' 2023-10-30T12:39:37.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/27' 2023-10-30T12:39:37.774 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/31' 2023-10-30T12:39:37.774 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/36' 2023-10-30T12:39:37.774 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/20' 2023-10-30T12:39:37.774 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/39' 2023-10-30T12:39:37.774 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/34' 2023-10-30T12:39:37.774 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/38' 2023-10-30T12:39:37.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/28' 2023-10-30T12:39:37.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/24' 2023-10-30T12:39:37.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/29' 2023-10-30T12:39:37.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/32' 2023-10-30T12:39:37.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/22' 2023-10-30T12:39:37.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/25' 2023-10-30T12:39:37.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/30' 2023-10-30T12:39:37.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/33' 2023-10-30T12:39:37.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/23' 2023-10-30T12:39:37.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/35' 2023-10-30T12:39:37.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/26' 2023-10-30T12:39:37.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/37' 2023-10-30T12:39:37.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './217' 2023-10-30T12:39:37.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/21' 2023-10-30T12:39:37.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/27' 2023-10-30T12:39:37.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/31' 2023-10-30T12:39:37.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/36' 2023-10-30T12:39:37.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/20' 2023-10-30T12:39:37.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/39' 2023-10-30T12:39:37.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/34' 2023-10-30T12:39:37.779 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/38' 2023-10-30T12:39:37.779 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/28' 2023-10-30T12:39:37.779 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/24' 2023-10-30T12:39:37.779 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/29' 2023-10-30T12:39:37.779 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/32' 2023-10-30T12:39:37.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/22' 2023-10-30T12:39:37.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/25' 2023-10-30T12:39:37.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/30' 2023-10-30T12:39:37.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/33' 2023-10-30T12:39:37.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/23' 2023-10-30T12:39:37.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/35' 2023-10-30T12:39:37.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/26' 2023-10-30T12:39:37.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/37' 2023-10-30T12:39:37.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './218' 2023-10-30T12:39:37.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/21' 2023-10-30T12:39:37.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/27' 2023-10-30T12:39:37.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/31' 2023-10-30T12:39:37.782 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/36' 2023-10-30T12:39:37.782 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/20' 2023-10-30T12:39:37.782 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/39' 2023-10-30T12:39:37.782 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/34' 2023-10-30T12:39:37.782 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/38' 2023-10-30T12:39:37.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/28' 2023-10-30T12:39:37.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/24' 2023-10-30T12:39:37.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/29' 2023-10-30T12:39:37.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/32' 2023-10-30T12:39:37.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/22' 2023-10-30T12:39:37.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/25' 2023-10-30T12:39:37.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/30' 2023-10-30T12:39:37.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/33' 2023-10-30T12:39:37.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/23' 2023-10-30T12:39:37.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/35' 2023-10-30T12:39:37.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/26' 2023-10-30T12:39:37.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/37' 2023-10-30T12:39:37.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './219' 2023-10-30T12:39:37.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/21' 2023-10-30T12:39:37.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/27' 2023-10-30T12:39:37.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/31' 2023-10-30T12:39:37.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/36' 2023-10-30T12:39:37.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/20' 2023-10-30T12:39:37.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/39' 2023-10-30T12:39:37.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/34' 2023-10-30T12:39:37.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/38' 2023-10-30T12:39:37.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/28' 2023-10-30T12:39:37.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/24' 2023-10-30T12:39:37.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/29' 2023-10-30T12:39:37.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/32' 2023-10-30T12:39:37.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/22' 2023-10-30T12:39:37.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/25' 2023-10-30T12:39:37.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/30' 2023-10-30T12:39:37.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/33' 2023-10-30T12:39:37.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/23' 2023-10-30T12:39:37.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/35' 2023-10-30T12:39:37.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/26' 2023-10-30T12:39:37.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/37' 2023-10-30T12:39:37.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './220' 2023-10-30T12:39:37.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/21' 2023-10-30T12:39:37.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/27' 2023-10-30T12:39:37.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/31' 2023-10-30T12:39:37.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/36' 2023-10-30T12:39:37.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/20' 2023-10-30T12:39:37.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/39' 2023-10-30T12:39:37.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/34' 2023-10-30T12:39:37.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/38' 2023-10-30T12:39:37.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/28' 2023-10-30T12:39:37.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/24' 2023-10-30T12:39:37.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/29' 2023-10-30T12:39:37.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/32' 2023-10-30T12:39:37.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/22' 2023-10-30T12:39:37.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/25' 2023-10-30T12:39:37.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/30' 2023-10-30T12:39:37.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/33' 2023-10-30T12:39:37.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/23' 2023-10-30T12:39:37.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/35' 2023-10-30T12:39:37.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/26' 2023-10-30T12:39:37.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/37' 2023-10-30T12:39:37.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './221' 2023-10-30T12:39:37.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/21' 2023-10-30T12:39:37.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/27' 2023-10-30T12:39:37.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/31' 2023-10-30T12:39:37.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/36' 2023-10-30T12:39:37.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/20' 2023-10-30T12:39:37.794 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/39' 2023-10-30T12:39:37.794 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/34' 2023-10-30T12:39:37.794 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/38' 2023-10-30T12:39:37.794 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/28' 2023-10-30T12:39:37.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/24' 2023-10-30T12:39:37.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/29' 2023-10-30T12:39:37.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/32' 2023-10-30T12:39:37.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/22' 2023-10-30T12:39:37.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/25' 2023-10-30T12:39:37.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/30' 2023-10-30T12:39:37.796 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/33' 2023-10-30T12:39:37.796 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/23' 2023-10-30T12:39:37.796 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/35' 2023-10-30T12:39:37.796 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/26' 2023-10-30T12:39:37.796 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/37' 2023-10-30T12:39:37.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './222' 2023-10-30T12:39:37.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/21' 2023-10-30T12:39:37.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/27' 2023-10-30T12:39:37.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/31' 2023-10-30T12:39:37.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/36' 2023-10-30T12:39:37.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/20' 2023-10-30T12:39:37.798 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/39' 2023-10-30T12:39:37.798 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/34' 2023-10-30T12:39:37.798 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/38' 2023-10-30T12:39:37.798 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/28' 2023-10-30T12:39:37.798 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/24' 2023-10-30T12:39:37.799 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/29' 2023-10-30T12:39:37.799 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/32' 2023-10-30T12:39:37.799 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/22' 2023-10-30T12:39:37.799 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/25' 2023-10-30T12:39:37.799 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/30' 2023-10-30T12:39:37.799 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/33' 2023-10-30T12:39:37.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/23' 2023-10-30T12:39:37.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/35' 2023-10-30T12:39:37.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/26' 2023-10-30T12:39:37.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/37' 2023-10-30T12:39:37.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './223' 2023-10-30T12:39:37.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/21' 2023-10-30T12:39:37.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/27' 2023-10-30T12:39:37.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/31' 2023-10-30T12:39:37.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/36' 2023-10-30T12:39:37.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/20' 2023-10-30T12:39:37.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/39' 2023-10-30T12:39:37.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/34' 2023-10-30T12:39:37.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/38' 2023-10-30T12:39:37.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/28' 2023-10-30T12:39:37.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/24' 2023-10-30T12:39:37.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/29' 2023-10-30T12:39:37.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/32' 2023-10-30T12:39:37.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/22' 2023-10-30T12:39:37.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/25' 2023-10-30T12:39:37.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/30' 2023-10-30T12:39:37.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/33' 2023-10-30T12:39:37.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/23' 2023-10-30T12:39:37.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/35' 2023-10-30T12:39:37.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/26' 2023-10-30T12:39:37.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/37' 2023-10-30T12:39:37.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './224' 2023-10-30T12:39:37.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/21' 2023-10-30T12:39:37.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/27' 2023-10-30T12:39:37.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/31' 2023-10-30T12:39:37.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/36' 2023-10-30T12:39:37.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/20' 2023-10-30T12:39:37.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/39' 2023-10-30T12:39:37.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/34' 2023-10-30T12:39:37.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/38' 2023-10-30T12:39:37.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/28' 2023-10-30T12:39:37.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/24' 2023-10-30T12:39:37.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/29' 2023-10-30T12:39:37.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/32' 2023-10-30T12:39:37.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/22' 2023-10-30T12:39:37.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/25' 2023-10-30T12:39:37.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/30' 2023-10-30T12:39:37.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/33' 2023-10-30T12:39:37.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/23' 2023-10-30T12:39:37.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/35' 2023-10-30T12:39:37.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/26' 2023-10-30T12:39:37.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/37' 2023-10-30T12:39:37.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './225' 2023-10-30T12:39:37.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/21' 2023-10-30T12:39:37.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/27' 2023-10-30T12:39:37.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/31' 2023-10-30T12:39:37.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/36' 2023-10-30T12:39:37.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/20' 2023-10-30T12:39:37.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/39' 2023-10-30T12:39:37.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/34' 2023-10-30T12:39:37.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/38' 2023-10-30T12:39:37.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/28' 2023-10-30T12:39:37.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/24' 2023-10-30T12:39:41.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/29' 2023-10-30T12:39:41.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/32' 2023-10-30T12:39:41.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/22' 2023-10-30T12:39:41.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/25' 2023-10-30T12:39:41.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/30' 2023-10-30T12:39:41.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/33' 2023-10-30T12:39:41.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/23' 2023-10-30T12:39:41.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/35' 2023-10-30T12:39:41.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/26' 2023-10-30T12:39:41.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/37' 2023-10-30T12:39:41.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './226' 2023-10-30T12:39:41.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/21' 2023-10-30T12:39:41.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/27' 2023-10-30T12:39:41.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/31' 2023-10-30T12:39:41.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/36' 2023-10-30T12:39:41.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/20' 2023-10-30T12:39:41.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/39' 2023-10-30T12:39:41.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/34' 2023-10-30T12:39:41.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/38' 2023-10-30T12:39:41.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/28' 2023-10-30T12:39:41.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/24' 2023-10-30T12:39:41.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/29' 2023-10-30T12:39:41.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/32' 2023-10-30T12:39:41.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/22' 2023-10-30T12:39:41.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/25' 2023-10-30T12:39:41.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/30' 2023-10-30T12:39:41.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/33' 2023-10-30T12:39:41.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/23' 2023-10-30T12:39:41.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/35' 2023-10-30T12:39:41.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/26' 2023-10-30T12:39:41.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/37' 2023-10-30T12:39:41.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './227' 2023-10-30T12:39:41.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/21' 2023-10-30T12:39:41.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/27' 2023-10-30T12:39:41.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/31' 2023-10-30T12:39:41.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/36' 2023-10-30T12:39:41.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/20' 2023-10-30T12:39:41.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/39' 2023-10-30T12:39:41.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/34' 2023-10-30T12:39:41.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/38' 2023-10-30T12:39:41.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/28' 2023-10-30T12:39:41.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/24' 2023-10-30T12:39:41.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/29' 2023-10-30T12:39:41.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/32' 2023-10-30T12:39:41.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/22' 2023-10-30T12:39:41.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/25' 2023-10-30T12:39:41.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/30' 2023-10-30T12:39:41.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/33' 2023-10-30T12:39:41.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/23' 2023-10-30T12:39:41.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/35' 2023-10-30T12:39:41.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/26' 2023-10-30T12:39:41.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/37' 2023-10-30T12:39:41.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './228' 2023-10-30T12:39:41.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/21' 2023-10-30T12:39:41.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/27' 2023-10-30T12:39:41.625 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/31' 2023-10-30T12:39:41.625 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/36' 2023-10-30T12:39:41.625 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/20' 2023-10-30T12:39:41.625 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/39' 2023-10-30T12:39:41.625 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/34' 2023-10-30T12:39:41.626 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/38' 2023-10-30T12:39:41.626 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/28' 2023-10-30T12:39:41.626 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/24' 2023-10-30T12:39:41.626 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/29' 2023-10-30T12:39:41.626 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/32' 2023-10-30T12:39:41.626 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/22' 2023-10-30T12:39:41.627 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/25' 2023-10-30T12:39:41.627 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/30' 2023-10-30T12:39:41.627 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/33' 2023-10-30T12:39:41.627 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/23' 2023-10-30T12:39:41.627 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/35' 2023-10-30T12:39:41.628 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/26' 2023-10-30T12:39:41.628 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/37' 2023-10-30T12:39:41.628 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './229' 2023-10-30T12:39:41.628 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/21' 2023-10-30T12:39:41.628 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/27' 2023-10-30T12:39:41.629 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/31' 2023-10-30T12:39:41.629 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/36' 2023-10-30T12:39:41.629 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/20' 2023-10-30T12:39:41.629 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/39' 2023-10-30T12:39:41.629 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/34' 2023-10-30T12:39:41.629 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/38' 2023-10-30T12:39:41.630 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/28' 2023-10-30T12:39:41.630 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/24' 2023-10-30T12:39:41.630 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/29' 2023-10-30T12:39:41.630 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/32' 2023-10-30T12:39:41.630 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/22' 2023-10-30T12:39:41.631 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/25' 2023-10-30T12:39:41.631 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/30' 2023-10-30T12:39:41.631 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/33' 2023-10-30T12:39:41.631 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/23' 2023-10-30T12:39:41.631 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/35' 2023-10-30T12:39:41.631 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/26' 2023-10-30T12:39:41.632 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/37' 2023-10-30T12:39:41.632 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './230' 2023-10-30T12:39:41.632 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/21' 2023-10-30T12:39:41.632 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/27' 2023-10-30T12:39:41.632 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/31' 2023-10-30T12:39:41.633 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/36' 2023-10-30T12:39:41.633 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/20' 2023-10-30T12:39:41.633 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/39' 2023-10-30T12:39:41.633 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/34' 2023-10-30T12:39:41.633 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/38' 2023-10-30T12:39:41.633 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/28' 2023-10-30T12:39:41.634 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/24' 2023-10-30T12:39:41.634 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/29' 2023-10-30T12:39:41.634 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/32' 2023-10-30T12:39:41.634 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/22' 2023-10-30T12:39:41.634 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/25' 2023-10-30T12:39:41.635 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/30' 2023-10-30T12:39:41.635 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/33' 2023-10-30T12:39:41.635 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/23' 2023-10-30T12:39:41.635 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/35' 2023-10-30T12:39:41.635 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/26' 2023-10-30T12:39:41.635 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/37' 2023-10-30T12:39:41.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './231' 2023-10-30T12:39:41.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/21' 2023-10-30T12:39:41.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/27' 2023-10-30T12:39:41.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/31' 2023-10-30T12:39:41.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/36' 2023-10-30T12:39:41.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/20' 2023-10-30T12:39:41.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/39' 2023-10-30T12:39:41.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/34' 2023-10-30T12:39:41.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/38' 2023-10-30T12:39:41.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/28' 2023-10-30T12:39:41.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/24' 2023-10-30T12:39:41.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/29' 2023-10-30T12:39:41.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/32' 2023-10-30T12:39:41.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/22' 2023-10-30T12:39:41.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/25' 2023-10-30T12:39:41.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/30' 2023-10-30T12:39:41.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/33' 2023-10-30T12:39:41.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/23' 2023-10-30T12:39:41.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/35' 2023-10-30T12:39:41.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/26' 2023-10-30T12:39:41.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/37' 2023-10-30T12:39:41.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './232' 2023-10-30T12:39:41.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/21' 2023-10-30T12:39:41.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/27' 2023-10-30T12:39:41.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/31' 2023-10-30T12:39:41.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/36' 2023-10-30T12:39:41.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/20' 2023-10-30T12:39:41.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/39' 2023-10-30T12:39:41.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/34' 2023-10-30T12:39:41.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/38' 2023-10-30T12:39:41.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/28' 2023-10-30T12:39:41.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/24' 2023-10-30T12:39:41.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/29' 2023-10-30T12:39:41.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/32' 2023-10-30T12:39:41.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/22' 2023-10-30T12:39:41.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/25' 2023-10-30T12:39:41.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/30' 2023-10-30T12:39:41.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/33' 2023-10-30T12:39:41.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/23' 2023-10-30T12:39:41.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/35' 2023-10-30T12:39:41.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/26' 2023-10-30T12:39:41.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/37' 2023-10-30T12:39:41.644 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './233' 2023-10-30T12:39:41.644 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/21' 2023-10-30T12:39:41.644 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/27' 2023-10-30T12:39:41.644 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/31' 2023-10-30T12:39:41.644 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/36' 2023-10-30T12:39:41.644 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/20' 2023-10-30T12:39:41.645 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/39' 2023-10-30T12:39:41.645 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/34' 2023-10-30T12:39:41.645 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/38' 2023-10-30T12:39:41.645 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/28' 2023-10-30T12:39:41.645 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/24' 2023-10-30T12:39:41.646 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/29' 2023-10-30T12:39:41.646 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/32' 2023-10-30T12:39:41.646 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/22' 2023-10-30T12:39:41.646 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/25' 2023-10-30T12:39:41.646 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/30' 2023-10-30T12:39:41.647 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/33' 2023-10-30T12:39:41.647 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/23' 2023-10-30T12:39:41.647 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/35' 2023-10-30T12:39:41.647 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/26' 2023-10-30T12:39:41.647 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/37' 2023-10-30T12:39:41.647 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './234' 2023-10-30T12:39:41.648 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/21' 2023-10-30T12:39:41.648 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/27' 2023-10-30T12:39:41.648 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/31' 2023-10-30T12:39:41.648 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/36' 2023-10-30T12:39:41.648 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/20' 2023-10-30T12:39:41.649 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/39' 2023-10-30T12:39:41.649 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/34' 2023-10-30T12:39:41.649 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/38' 2023-10-30T12:39:41.649 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/28' 2023-10-30T12:39:41.649 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/24' 2023-10-30T12:39:41.650 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/29' 2023-10-30T12:39:41.650 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/32' 2023-10-30T12:39:41.650 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/22' 2023-10-30T12:39:41.650 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/25' 2023-10-30T12:39:41.650 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/30' 2023-10-30T12:39:41.651 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/33' 2023-10-30T12:39:41.651 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/23' 2023-10-30T12:39:41.651 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/35' 2023-10-30T12:39:41.651 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/26' 2023-10-30T12:39:41.651 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/37' 2023-10-30T12:39:41.651 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './235' 2023-10-30T12:39:41.652 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/21' 2023-10-30T12:39:41.652 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/27' 2023-10-30T12:39:41.652 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/31' 2023-10-30T12:39:41.652 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/36' 2023-10-30T12:39:41.652 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/20' 2023-10-30T12:39:41.652 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/39' 2023-10-30T12:39:41.653 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/34' 2023-10-30T12:39:41.653 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/38' 2023-10-30T12:39:41.653 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/28' 2023-10-30T12:39:41.653 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/24' 2023-10-30T12:39:41.654 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/29' 2023-10-30T12:39:41.654 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/32' 2023-10-30T12:39:45.486 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/22' 2023-10-30T12:39:45.486 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/25' 2023-10-30T12:39:45.486 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/30' 2023-10-30T12:39:45.486 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/33' 2023-10-30T12:39:45.487 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/23' 2023-10-30T12:39:45.487 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/35' 2023-10-30T12:39:45.487 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/26' 2023-10-30T12:39:45.487 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/37' 2023-10-30T12:39:45.487 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './236' 2023-10-30T12:39:45.488 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/21' 2023-10-30T12:39:45.488 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/27' 2023-10-30T12:39:45.488 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/31' 2023-10-30T12:39:45.488 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/36' 2023-10-30T12:39:45.488 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/20' 2023-10-30T12:39:45.488 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/39' 2023-10-30T12:39:45.489 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/34' 2023-10-30T12:39:45.489 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/38' 2023-10-30T12:39:45.489 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/28' 2023-10-30T12:39:45.489 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/24' 2023-10-30T12:39:45.489 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/29' 2023-10-30T12:39:45.490 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/32' 2023-10-30T12:39:45.490 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/22' 2023-10-30T12:39:45.490 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/25' 2023-10-30T12:39:45.490 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/30' 2023-10-30T12:39:45.490 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/33' 2023-10-30T12:39:45.491 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/23' 2023-10-30T12:39:45.491 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/35' 2023-10-30T12:39:45.491 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/26' 2023-10-30T12:39:45.491 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/37' 2023-10-30T12:39:45.491 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './237' 2023-10-30T12:39:45.491 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/21' 2023-10-30T12:39:45.492 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/27' 2023-10-30T12:39:45.492 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/31' 2023-10-30T12:39:45.492 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/36' 2023-10-30T12:39:45.492 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/20' 2023-10-30T12:39:45.492 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/39' 2023-10-30T12:39:45.492 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/34' 2023-10-30T12:39:45.493 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/38' 2023-10-30T12:39:45.493 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/28' 2023-10-30T12:39:45.493 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/24' 2023-10-30T12:39:45.493 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/29' 2023-10-30T12:39:45.493 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/32' 2023-10-30T12:39:45.494 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/22' 2023-10-30T12:39:45.494 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/25' 2023-10-30T12:39:45.494 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/30' 2023-10-30T12:39:45.494 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/33' 2023-10-30T12:39:45.494 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/23' 2023-10-30T12:39:45.494 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/35' 2023-10-30T12:39:45.495 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/26' 2023-10-30T12:39:45.495 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/37' 2023-10-30T12:39:45.495 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './238' 2023-10-30T12:39:45.495 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/21' 2023-10-30T12:39:45.495 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/27' 2023-10-30T12:39:45.495 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/31' 2023-10-30T12:39:45.496 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/36' 2023-10-30T12:39:45.496 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/20' 2023-10-30T12:39:45.496 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/39' 2023-10-30T12:39:45.496 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/34' 2023-10-30T12:39:45.496 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/38' 2023-10-30T12:39:45.497 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/28' 2023-10-30T12:39:45.497 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/24' 2023-10-30T12:39:45.497 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/29' 2023-10-30T12:39:45.497 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/32' 2023-10-30T12:39:45.497 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/22' 2023-10-30T12:39:45.498 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/25' 2023-10-30T12:39:45.498 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/30' 2023-10-30T12:39:45.498 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/33' 2023-10-30T12:39:45.498 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/23' 2023-10-30T12:39:45.499 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/35' 2023-10-30T12:39:45.499 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/26' 2023-10-30T12:39:45.499 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/37' 2023-10-30T12:39:45.499 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './239' 2023-10-30T12:39:45.499 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/21' 2023-10-30T12:39:45.500 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/27' 2023-10-30T12:39:45.500 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/31' 2023-10-30T12:39:45.500 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/36' 2023-10-30T12:39:45.500 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/20' 2023-10-30T12:39:45.500 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/39' 2023-10-30T12:39:45.501 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/34' 2023-10-30T12:39:45.501 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/38' 2023-10-30T12:39:45.501 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/28' 2023-10-30T12:39:45.501 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/24' 2023-10-30T12:39:45.501 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/29' 2023-10-30T12:39:45.501 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/32' 2023-10-30T12:39:45.502 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/22' 2023-10-30T12:39:45.502 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/25' 2023-10-30T12:39:45.502 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/30' 2023-10-30T12:39:45.502 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/33' 2023-10-30T12:39:45.502 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/23' 2023-10-30T12:39:45.503 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/35' 2023-10-30T12:39:45.503 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/26' 2023-10-30T12:39:45.503 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/37' 2023-10-30T12:39:45.503 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './240' 2023-10-30T12:39:45.503 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/21' 2023-10-30T12:39:45.504 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/27' 2023-10-30T12:39:45.504 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/31' 2023-10-30T12:39:45.504 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/36' 2023-10-30T12:39:45.504 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/20' 2023-10-30T12:39:45.504 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/39' 2023-10-30T12:39:45.505 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/34' 2023-10-30T12:39:45.505 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/38' 2023-10-30T12:39:45.505 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/28' 2023-10-30T12:39:45.505 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/24' 2023-10-30T12:39:45.505 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/29' 2023-10-30T12:39:45.505 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/32' 2023-10-30T12:39:45.506 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/22' 2023-10-30T12:39:45.506 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/25' 2023-10-30T12:39:45.506 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/30' 2023-10-30T12:39:45.506 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/33' 2023-10-30T12:39:45.506 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/23' 2023-10-30T12:39:45.507 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/35' 2023-10-30T12:39:45.507 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/26' 2023-10-30T12:39:45.507 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/37' 2023-10-30T12:39:45.507 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './241' 2023-10-30T12:39:45.507 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/21' 2023-10-30T12:39:45.508 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/27' 2023-10-30T12:39:45.508 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/31' 2023-10-30T12:39:45.508 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/36' 2023-10-30T12:39:45.508 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/20' 2023-10-30T12:39:45.508 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/39' 2023-10-30T12:39:45.508 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/34' 2023-10-30T12:39:45.509 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/38' 2023-10-30T12:39:45.509 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/28' 2023-10-30T12:39:45.509 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/24' 2023-10-30T12:39:45.509 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/29' 2023-10-30T12:39:45.509 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/32' 2023-10-30T12:39:45.510 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/22' 2023-10-30T12:39:45.510 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/25' 2023-10-30T12:39:45.510 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/30' 2023-10-30T12:39:45.510 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/33' 2023-10-30T12:39:45.510 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/23' 2023-10-30T12:39:45.511 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/35' 2023-10-30T12:39:45.511 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/26' 2023-10-30T12:39:45.511 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/37' 2023-10-30T12:39:45.511 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './242' 2023-10-30T12:39:45.511 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/21' 2023-10-30T12:39:45.512 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/27' 2023-10-30T12:39:45.512 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/31' 2023-10-30T12:39:45.512 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/36' 2023-10-30T12:39:45.512 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/20' 2023-10-30T12:39:45.512 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/39' 2023-10-30T12:39:45.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/34' 2023-10-30T12:39:45.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/38' 2023-10-30T12:39:45.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/28' 2023-10-30T12:39:45.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/24' 2023-10-30T12:39:45.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/29' 2023-10-30T12:39:45.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/32' 2023-10-30T12:39:45.514 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/22' 2023-10-30T12:39:45.514 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/25' 2023-10-30T12:39:45.514 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/30' 2023-10-30T12:39:45.514 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/33' 2023-10-30T12:39:45.514 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/23' 2023-10-30T12:39:45.515 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/35' 2023-10-30T12:39:45.515 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/26' 2023-10-30T12:39:45.515 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/37' 2023-10-30T12:39:45.515 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './243' 2023-10-30T12:39:45.515 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/21' 2023-10-30T12:39:45.516 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/27' 2023-10-30T12:39:45.516 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/31' 2023-10-30T12:39:45.516 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/36' 2023-10-30T12:39:45.516 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/20' 2023-10-30T12:39:45.516 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/39' 2023-10-30T12:39:45.517 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/34' 2023-10-30T12:39:45.517 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/38' 2023-10-30T12:39:45.517 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/28' 2023-10-30T12:39:45.517 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/24' 2023-10-30T12:39:45.517 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/29' 2023-10-30T12:39:45.518 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/32' 2023-10-30T12:39:45.518 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/22' 2023-10-30T12:39:45.518 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/25' 2023-10-30T12:39:45.518 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/30' 2023-10-30T12:39:45.518 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/33' 2023-10-30T12:39:45.519 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/23' 2023-10-30T12:39:45.519 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/35' 2023-10-30T12:39:45.519 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/26' 2023-10-30T12:39:45.519 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/37' 2023-10-30T12:39:45.519 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './244' 2023-10-30T12:39:45.520 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/21' 2023-10-30T12:39:45.520 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/27' 2023-10-30T12:39:45.520 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/31' 2023-10-30T12:39:45.520 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/36' 2023-10-30T12:39:45.520 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/20' 2023-10-30T12:39:45.520 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/39' 2023-10-30T12:39:45.521 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/34' 2023-10-30T12:39:45.521 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/38' 2023-10-30T12:39:45.521 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/28' 2023-10-30T12:39:45.521 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/24' 2023-10-30T12:39:45.521 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/29' 2023-10-30T12:39:45.521 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/32' 2023-10-30T12:39:45.522 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/22' 2023-10-30T12:39:45.522 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/25' 2023-10-30T12:39:45.522 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/30' 2023-10-30T12:39:45.522 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/33' 2023-10-30T12:39:45.522 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/23' 2023-10-30T12:39:45.523 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/35' 2023-10-30T12:39:45.523 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/26' 2023-10-30T12:39:45.523 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/37' 2023-10-30T12:39:45.523 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './245' 2023-10-30T12:39:45.523 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/21' 2023-10-30T12:39:45.523 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/27' 2023-10-30T12:39:45.524 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/31' 2023-10-30T12:39:45.524 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/36' 2023-10-30T12:39:45.524 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/20' 2023-10-30T12:39:45.524 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/39' 2023-10-30T12:39:45.524 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/34' 2023-10-30T12:39:45.524 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/38' 2023-10-30T12:39:45.525 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/28' 2023-10-30T12:39:45.525 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/24' 2023-10-30T12:39:45.525 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/29' 2023-10-30T12:39:45.525 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/32' 2023-10-30T12:39:45.525 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/22' 2023-10-30T12:39:53.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/25' 2023-10-30T12:39:53.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/30' 2023-10-30T12:39:53.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/33' 2023-10-30T12:39:53.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/23' 2023-10-30T12:39:53.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/35' 2023-10-30T12:39:53.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/26' 2023-10-30T12:39:53.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/37' 2023-10-30T12:39:53.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './246' 2023-10-30T12:39:53.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/21' 2023-10-30T12:39:53.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/27' 2023-10-30T12:39:53.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/31' 2023-10-30T12:39:53.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/36' 2023-10-30T12:39:53.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/20' 2023-10-30T12:39:53.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/39' 2023-10-30T12:39:53.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/34' 2023-10-30T12:39:53.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/38' 2023-10-30T12:39:53.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/28' 2023-10-30T12:39:53.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/24' 2023-10-30T12:39:53.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/29' 2023-10-30T12:39:53.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/32' 2023-10-30T12:39:53.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/22' 2023-10-30T12:39:53.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/25' 2023-10-30T12:39:53.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/30' 2023-10-30T12:39:53.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/33' 2023-10-30T12:39:53.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/23' 2023-10-30T12:39:53.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/35' 2023-10-30T12:39:53.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/26' 2023-10-30T12:39:53.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/37' 2023-10-30T12:39:53.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './247' 2023-10-30T12:39:53.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/21' 2023-10-30T12:39:53.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/27' 2023-10-30T12:39:53.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/31' 2023-10-30T12:39:53.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/36' 2023-10-30T12:39:53.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/20' 2023-10-30T12:39:53.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/39' 2023-10-30T12:39:53.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/34' 2023-10-30T12:39:53.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/38' 2023-10-30T12:39:53.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/28' 2023-10-30T12:39:53.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/24' 2023-10-30T12:39:53.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/29' 2023-10-30T12:39:53.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/32' 2023-10-30T12:39:53.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/22' 2023-10-30T12:39:53.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/25' 2023-10-30T12:39:53.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/30' 2023-10-30T12:39:53.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/33' 2023-10-30T12:39:53.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/23' 2023-10-30T12:39:53.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/35' 2023-10-30T12:39:53.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/26' 2023-10-30T12:39:53.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/37' 2023-10-30T12:39:53.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './248' 2023-10-30T12:39:53.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/21' 2023-10-30T12:39:53.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/27' 2023-10-30T12:39:53.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/31' 2023-10-30T12:39:53.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/36' 2023-10-30T12:39:53.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/20' 2023-10-30T12:39:53.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/39' 2023-10-30T12:39:53.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/34' 2023-10-30T12:39:53.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/38' 2023-10-30T12:39:53.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/28' 2023-10-30T12:39:53.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/24' 2023-10-30T12:39:53.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/29' 2023-10-30T12:39:53.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/32' 2023-10-30T12:39:53.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/22' 2023-10-30T12:39:53.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/25' 2023-10-30T12:39:53.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/30' 2023-10-30T12:39:53.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/33' 2023-10-30T12:39:53.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/23' 2023-10-30T12:39:53.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/35' 2023-10-30T12:39:53.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/26' 2023-10-30T12:39:53.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/37' 2023-10-30T12:39:53.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './249' 2023-10-30T12:39:53.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/21' 2023-10-30T12:39:53.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/27' 2023-10-30T12:39:53.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/31' 2023-10-30T12:39:53.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/36' 2023-10-30T12:39:53.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/20' 2023-10-30T12:39:53.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/39' 2023-10-30T12:39:53.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/34' 2023-10-30T12:39:53.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/38' 2023-10-30T12:39:53.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/28' 2023-10-30T12:39:53.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/24' 2023-10-30T12:39:53.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/29' 2023-10-30T12:39:53.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/32' 2023-10-30T12:39:53.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/22' 2023-10-30T12:39:53.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/25' 2023-10-30T12:39:53.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/30' 2023-10-30T12:39:53.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/33' 2023-10-30T12:39:53.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/23' 2023-10-30T12:39:53.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/35' 2023-10-30T12:39:53.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/26' 2023-10-30T12:39:53.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/37' 2023-10-30T12:39:53.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './250' 2023-10-30T12:39:53.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/21' 2023-10-30T12:39:53.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/27' 2023-10-30T12:39:53.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/31' 2023-10-30T12:39:54.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/36' 2023-10-30T12:39:54.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/20' 2023-10-30T12:39:54.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/39' 2023-10-30T12:39:54.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/34' 2023-10-30T12:39:54.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/38' 2023-10-30T12:39:54.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/28' 2023-10-30T12:39:54.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/24' 2023-10-30T12:39:54.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/29' 2023-10-30T12:39:54.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/32' 2023-10-30T12:39:54.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/22' 2023-10-30T12:39:54.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/25' 2023-10-30T12:39:54.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/30' 2023-10-30T12:39:54.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/33' 2023-10-30T12:39:54.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/23' 2023-10-30T12:39:54.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/35' 2023-10-30T12:39:54.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/26' 2023-10-30T12:39:54.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/37' 2023-10-30T12:39:54.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './251' 2023-10-30T12:39:54.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/21' 2023-10-30T12:39:54.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/27' 2023-10-30T12:39:54.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/31' 2023-10-30T12:39:54.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/36' 2023-10-30T12:39:54.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/20' 2023-10-30T12:39:54.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/39' 2023-10-30T12:39:54.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/34' 2023-10-30T12:39:54.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/38' 2023-10-30T12:39:54.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/28' 2023-10-30T12:39:54.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/24' 2023-10-30T12:39:54.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/29' 2023-10-30T12:39:54.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/32' 2023-10-30T12:39:54.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/22' 2023-10-30T12:39:54.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/25' 2023-10-30T12:39:54.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/30' 2023-10-30T12:39:54.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/33' 2023-10-30T12:39:54.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/23' 2023-10-30T12:39:54.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/35' 2023-10-30T12:39:54.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/26' 2023-10-30T12:39:54.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/37' 2023-10-30T12:39:54.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './252' 2023-10-30T12:39:54.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/21' 2023-10-30T12:39:54.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/27' 2023-10-30T12:39:54.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/31' 2023-10-30T12:39:54.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/36' 2023-10-30T12:39:54.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/20' 2023-10-30T12:39:54.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/39' 2023-10-30T12:39:54.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/34' 2023-10-30T12:39:54.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/38' 2023-10-30T12:39:54.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/28' 2023-10-30T12:39:54.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/24' 2023-10-30T12:39:54.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/29' 2023-10-30T12:39:54.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/32' 2023-10-30T12:39:54.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/22' 2023-10-30T12:39:54.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/25' 2023-10-30T12:39:54.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/30' 2023-10-30T12:39:54.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/33' 2023-10-30T12:39:54.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/23' 2023-10-30T12:39:54.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/35' 2023-10-30T12:39:54.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/26' 2023-10-30T12:39:54.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/37' 2023-10-30T12:39:54.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './253' 2023-10-30T12:39:54.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/21' 2023-10-30T12:39:54.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/27' 2023-10-30T12:39:54.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/31' 2023-10-30T12:39:54.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/36' 2023-10-30T12:39:54.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/20' 2023-10-30T12:39:54.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/39' 2023-10-30T12:39:54.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/34' 2023-10-30T12:39:54.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/38' 2023-10-30T12:39:54.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/28' 2023-10-30T12:39:54.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/24' 2023-10-30T12:39:54.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/29' 2023-10-30T12:39:54.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/32' 2023-10-30T12:39:54.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/22' 2023-10-30T12:39:54.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/25' 2023-10-30T12:39:54.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/30' 2023-10-30T12:39:54.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/33' 2023-10-30T12:39:54.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/23' 2023-10-30T12:39:54.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/35' 2023-10-30T12:39:54.015 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/26' 2023-10-30T12:39:54.015 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/37' 2023-10-30T12:39:54.015 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './254' 2023-10-30T12:39:54.015 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/21' 2023-10-30T12:39:54.015 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/27' 2023-10-30T12:39:54.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/31' 2023-10-30T12:39:54.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/36' 2023-10-30T12:39:54.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/20' 2023-10-30T12:39:54.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/39' 2023-10-30T12:39:54.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/34' 2023-10-30T12:39:54.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/38' 2023-10-30T12:39:54.017 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/28' 2023-10-30T12:39:54.017 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/24' 2023-10-30T12:39:54.017 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/29' 2023-10-30T12:39:54.017 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/32' 2023-10-30T12:39:54.017 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/22' 2023-10-30T12:39:54.018 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/25' 2023-10-30T12:39:54.018 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/30' 2023-10-30T12:39:54.018 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/33' 2023-10-30T12:39:54.018 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/23' 2023-10-30T12:39:54.018 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/35' 2023-10-30T12:39:54.019 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/26' 2023-10-30T12:39:54.019 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/37' 2023-10-30T12:39:54.019 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './255' 2023-10-30T12:39:54.019 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/21' 2023-10-30T12:39:54.019 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/27' 2023-10-30T12:39:54.020 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/31' 2023-10-30T12:39:54.020 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/36' 2023-10-30T12:39:54.020 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/20' 2023-10-30T12:39:54.020 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/39' 2023-10-30T12:39:54.020 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/34' 2023-10-30T12:39:54.020 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/38' 2023-10-30T12:39:54.021 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/28' 2023-10-30T12:39:54.021 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/24' 2023-10-30T12:39:54.021 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/29' 2023-10-30T12:39:54.021 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/32' 2023-10-30T12:39:54.021 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/22' 2023-10-30T12:39:54.022 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/25' 2023-10-30T12:39:54.022 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/30' 2023-10-30T12:39:56.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/33' 2023-10-30T12:39:56.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/23' 2023-10-30T12:39:56.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/35' 2023-10-30T12:39:56.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/26' 2023-10-30T12:39:56.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/37' 2023-10-30T12:39:56.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './256' 2023-10-30T12:39:56.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/21' 2023-10-30T12:39:56.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/27' 2023-10-30T12:39:56.381 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/31' 2023-10-30T12:39:56.381 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/36' 2023-10-30T12:39:56.381 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/20' 2023-10-30T12:39:56.381 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/39' 2023-10-30T12:39:56.381 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/34' 2023-10-30T12:39:56.382 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/38' 2023-10-30T12:39:56.382 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/28' 2023-10-30T12:39:56.382 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/24' 2023-10-30T12:39:56.382 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/29' 2023-10-30T12:39:56.382 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/32' 2023-10-30T12:39:56.383 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/22' 2023-10-30T12:39:56.383 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/25' 2023-10-30T12:39:56.383 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/30' 2023-10-30T12:39:56.383 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/33' 2023-10-30T12:39:56.383 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/23' 2023-10-30T12:39:56.384 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/35' 2023-10-30T12:39:56.384 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/26' 2023-10-30T12:39:56.384 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/37' 2023-10-30T12:39:56.384 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './257' 2023-10-30T12:39:56.384 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/21' 2023-10-30T12:39:56.385 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/27' 2023-10-30T12:39:56.385 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/31' 2023-10-30T12:39:56.385 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/36' 2023-10-30T12:39:56.385 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/20' 2023-10-30T12:39:56.385 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/39' 2023-10-30T12:39:56.385 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/34' 2023-10-30T12:39:56.386 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/38' 2023-10-30T12:39:56.386 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/28' 2023-10-30T12:39:56.386 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/24' 2023-10-30T12:39:56.386 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/29' 2023-10-30T12:39:56.386 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/32' 2023-10-30T12:39:56.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/22' 2023-10-30T12:39:56.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/25' 2023-10-30T12:39:56.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/30' 2023-10-30T12:39:56.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/33' 2023-10-30T12:39:56.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/23' 2023-10-30T12:39:56.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/35' 2023-10-30T12:39:56.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/26' 2023-10-30T12:39:56.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/37' 2023-10-30T12:39:56.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './258' 2023-10-30T12:39:56.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/21' 2023-10-30T12:39:56.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/27' 2023-10-30T12:39:56.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/31' 2023-10-30T12:39:56.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/36' 2023-10-30T12:39:56.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/20' 2023-10-30T12:39:56.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/39' 2023-10-30T12:39:56.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/34' 2023-10-30T12:39:56.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/38' 2023-10-30T12:39:56.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/28' 2023-10-30T12:39:56.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/24' 2023-10-30T12:39:56.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/29' 2023-10-30T12:39:56.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/32' 2023-10-30T12:39:56.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/22' 2023-10-30T12:39:56.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/25' 2023-10-30T12:39:56.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/30' 2023-10-30T12:39:56.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/33' 2023-10-30T12:39:56.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/23' 2023-10-30T12:39:56.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/35' 2023-10-30T12:39:56.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/26' 2023-10-30T12:39:56.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/37' 2023-10-30T12:39:56.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './259' 2023-10-30T12:39:56.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/21' 2023-10-30T12:39:56.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/27' 2023-10-30T12:39:56.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/31' 2023-10-30T12:39:56.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/36' 2023-10-30T12:39:56.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/20' 2023-10-30T12:39:56.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/39' 2023-10-30T12:39:56.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/34' 2023-10-30T12:39:56.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/38' 2023-10-30T12:39:56.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/28' 2023-10-30T12:39:56.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/24' 2023-10-30T12:39:56.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/29' 2023-10-30T12:39:56.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/32' 2023-10-30T12:39:56.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/22' 2023-10-30T12:39:56.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/25' 2023-10-30T12:39:56.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/30' 2023-10-30T12:39:56.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/33' 2023-10-30T12:39:56.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/23' 2023-10-30T12:39:56.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/35' 2023-10-30T12:39:56.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/26' 2023-10-30T12:39:56.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/37' 2023-10-30T12:39:56.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './260' 2023-10-30T12:39:56.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/21' 2023-10-30T12:39:56.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/27' 2023-10-30T12:39:56.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/31' 2023-10-30T12:39:56.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/36' 2023-10-30T12:39:56.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/20' 2023-10-30T12:39:56.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/39' 2023-10-30T12:39:56.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/34' 2023-10-30T12:39:56.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/38' 2023-10-30T12:39:56.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/28' 2023-10-30T12:39:56.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/24' 2023-10-30T12:39:56.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/29' 2023-10-30T12:39:56.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/32' 2023-10-30T12:39:56.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/22' 2023-10-30T12:39:56.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/25' 2023-10-30T12:39:56.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/30' 2023-10-30T12:39:56.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/33' 2023-10-30T12:39:56.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/23' 2023-10-30T12:39:56.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/35' 2023-10-30T12:39:56.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/26' 2023-10-30T12:39:56.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/37' 2023-10-30T12:39:56.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './261' 2023-10-30T12:39:56.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/21' 2023-10-30T12:39:56.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/27' 2023-10-30T12:39:56.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/31' 2023-10-30T12:39:56.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/36' 2023-10-30T12:39:56.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/20' 2023-10-30T12:39:56.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/39' 2023-10-30T12:39:56.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/34' 2023-10-30T12:39:56.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/38' 2023-10-30T12:39:56.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/28' 2023-10-30T12:39:56.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/24' 2023-10-30T12:39:56.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/29' 2023-10-30T12:39:56.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/32' 2023-10-30T12:39:56.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/22' 2023-10-30T12:39:56.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/25' 2023-10-30T12:39:56.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/30' 2023-10-30T12:39:56.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/33' 2023-10-30T12:39:56.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/23' 2023-10-30T12:39:56.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/35' 2023-10-30T12:39:56.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/26' 2023-10-30T12:39:56.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/37' 2023-10-30T12:39:56.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './262' 2023-10-30T12:39:56.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/21' 2023-10-30T12:39:56.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/27' 2023-10-30T12:39:56.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/31' 2023-10-30T12:39:56.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/36' 2023-10-30T12:39:56.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/20' 2023-10-30T12:39:56.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/39' 2023-10-30T12:39:56.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/34' 2023-10-30T12:39:56.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/38' 2023-10-30T12:39:56.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/28' 2023-10-30T12:39:56.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/24' 2023-10-30T12:39:56.406 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/29' 2023-10-30T12:39:56.406 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/32' 2023-10-30T12:39:56.406 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/22' 2023-10-30T12:39:56.406 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/25' 2023-10-30T12:39:56.406 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/30' 2023-10-30T12:39:56.407 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/33' 2023-10-30T12:39:56.407 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/23' 2023-10-30T12:39:56.407 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/35' 2023-10-30T12:39:56.407 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/26' 2023-10-30T12:39:56.407 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/37' 2023-10-30T12:39:56.407 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './263' 2023-10-30T12:39:56.408 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/21' 2023-10-30T12:39:56.408 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/27' 2023-10-30T12:39:56.408 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/31' 2023-10-30T12:39:56.408 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/36' 2023-10-30T12:39:56.408 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/20' 2023-10-30T12:39:56.408 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/39' 2023-10-30T12:39:56.409 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/34' 2023-10-30T12:39:56.409 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/38' 2023-10-30T12:39:56.409 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/28' 2023-10-30T12:39:56.409 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/24' 2023-10-30T12:39:56.409 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/29' 2023-10-30T12:39:56.409 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/32' 2023-10-30T12:39:56.410 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/22' 2023-10-30T12:39:56.410 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/25' 2023-10-30T12:39:56.410 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/30' 2023-10-30T12:39:56.410 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/33' 2023-10-30T12:39:56.410 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/23' 2023-10-30T12:39:56.411 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/35' 2023-10-30T12:39:56.411 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/26' 2023-10-30T12:39:56.411 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/37' 2023-10-30T12:39:56.411 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './264' 2023-10-30T12:39:56.411 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/21' 2023-10-30T12:39:56.411 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/27' 2023-10-30T12:39:56.412 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/31' 2023-10-30T12:39:56.412 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/36' 2023-10-30T12:39:56.412 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/20' 2023-10-30T12:39:56.412 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/39' 2023-10-30T12:39:56.412 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/34' 2023-10-30T12:39:56.413 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/38' 2023-10-30T12:39:56.413 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/28' 2023-10-30T12:39:56.413 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/24' 2023-10-30T12:39:56.413 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/29' 2023-10-30T12:39:56.413 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/32' 2023-10-30T12:39:56.413 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/22' 2023-10-30T12:39:56.414 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/25' 2023-10-30T12:39:56.414 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/30' 2023-10-30T12:39:56.414 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/33' 2023-10-30T12:39:56.414 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/23' 2023-10-30T12:39:56.414 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/35' 2023-10-30T12:39:56.415 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/26' 2023-10-30T12:39:56.415 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/37' 2023-10-30T12:39:56.415 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './265' 2023-10-30T12:39:56.415 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/21' 2023-10-30T12:39:56.415 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/27' 2023-10-30T12:39:56.416 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/31' 2023-10-30T12:39:56.416 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/36' 2023-10-30T12:39:56.416 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/20' 2023-10-30T12:39:56.416 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/39' 2023-10-30T12:39:56.416 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/34' 2023-10-30T12:39:56.416 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/38' 2023-10-30T12:39:56.417 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/28' 2023-10-30T12:39:56.417 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/24' 2023-10-30T12:39:56.417 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/29' 2023-10-30T12:39:56.417 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/32' 2023-10-30T12:39:56.417 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/22' 2023-10-30T12:39:56.417 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/25' 2023-10-30T12:39:56.418 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/30' 2023-10-30T12:39:56.418 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/33' 2023-10-30T12:39:56.418 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/23' 2023-10-30T12:40:03.956 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:40:03.961 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:40:07.935 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/35' 2023-10-30T12:40:07.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/26' 2023-10-30T12:40:07.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/37' 2023-10-30T12:40:07.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './266' 2023-10-30T12:40:07.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/21' 2023-10-30T12:40:07.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/27' 2023-10-30T12:40:07.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/31' 2023-10-30T12:40:07.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/36' 2023-10-30T12:40:07.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/20' 2023-10-30T12:40:07.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/39' 2023-10-30T12:40:07.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/34' 2023-10-30T12:40:07.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/38' 2023-10-30T12:40:07.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/28' 2023-10-30T12:40:07.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/24' 2023-10-30T12:40:07.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/29' 2023-10-30T12:40:07.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/32' 2023-10-30T12:40:07.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/22' 2023-10-30T12:40:07.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/25' 2023-10-30T12:40:07.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/30' 2023-10-30T12:40:07.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/33' 2023-10-30T12:40:07.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/23' 2023-10-30T12:40:07.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/35' 2023-10-30T12:40:07.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/26' 2023-10-30T12:40:07.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/37' 2023-10-30T12:40:07.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './267' 2023-10-30T12:40:07.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/21' 2023-10-30T12:40:07.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/27' 2023-10-30T12:40:07.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/31' 2023-10-30T12:40:07.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/36' 2023-10-30T12:40:07.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/20' 2023-10-30T12:40:07.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/39' 2023-10-30T12:40:07.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/34' 2023-10-30T12:40:07.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/38' 2023-10-30T12:40:07.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/28' 2023-10-30T12:40:07.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/24' 2023-10-30T12:40:07.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/29' 2023-10-30T12:40:07.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/32' 2023-10-30T12:40:07.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/22' 2023-10-30T12:40:07.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/25' 2023-10-30T12:40:07.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/30' 2023-10-30T12:40:07.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/33' 2023-10-30T12:40:07.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/23' 2023-10-30T12:40:07.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/35' 2023-10-30T12:40:07.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/26' 2023-10-30T12:40:07.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/37' 2023-10-30T12:40:07.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './268' 2023-10-30T12:40:07.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/21' 2023-10-30T12:40:07.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/27' 2023-10-30T12:40:07.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/31' 2023-10-30T12:40:07.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/36' 2023-10-30T12:40:07.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/20' 2023-10-30T12:40:07.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/39' 2023-10-30T12:40:07.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/34' 2023-10-30T12:40:07.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/38' 2023-10-30T12:40:07.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/28' 2023-10-30T12:40:07.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/24' 2023-10-30T12:40:07.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/29' 2023-10-30T12:40:07.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/32' 2023-10-30T12:40:07.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/22' 2023-10-30T12:40:07.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/25' 2023-10-30T12:40:07.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/30' 2023-10-30T12:40:07.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/33' 2023-10-30T12:40:07.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/23' 2023-10-30T12:40:07.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/35' 2023-10-30T12:40:07.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/26' 2023-10-30T12:40:07.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/37' 2023-10-30T12:40:07.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './269' 2023-10-30T12:40:07.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/21' 2023-10-30T12:40:07.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/27' 2023-10-30T12:40:07.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/31' 2023-10-30T12:40:07.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/36' 2023-10-30T12:40:07.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/20' 2023-10-30T12:40:07.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/39' 2023-10-30T12:40:07.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/34' 2023-10-30T12:40:07.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/38' 2023-10-30T12:40:07.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/28' 2023-10-30T12:40:07.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/24' 2023-10-30T12:40:07.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/29' 2023-10-30T12:40:07.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/32' 2023-10-30T12:40:07.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/22' 2023-10-30T12:40:07.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/25' 2023-10-30T12:40:07.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/30' 2023-10-30T12:40:07.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/33' 2023-10-30T12:40:07.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/23' 2023-10-30T12:40:07.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/35' 2023-10-30T12:40:07.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/26' 2023-10-30T12:40:07.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/37' 2023-10-30T12:40:07.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './270' 2023-10-30T12:40:07.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/21' 2023-10-30T12:40:07.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/27' 2023-10-30T12:40:07.951 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/31' 2023-10-30T12:40:07.951 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/36' 2023-10-30T12:40:07.951 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/20' 2023-10-30T12:40:07.951 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/39' 2023-10-30T12:40:07.951 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/34' 2023-10-30T12:40:07.951 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/38' 2023-10-30T12:40:07.952 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/28' 2023-10-30T12:40:07.952 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/24' 2023-10-30T12:40:07.952 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/29' 2023-10-30T12:40:07.952 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/32' 2023-10-30T12:40:07.952 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/22' 2023-10-30T12:40:07.952 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/25' 2023-10-30T12:40:07.953 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/30' 2023-10-30T12:40:07.953 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/33' 2023-10-30T12:40:07.953 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/23' 2023-10-30T12:40:07.953 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/35' 2023-10-30T12:40:07.953 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/26' 2023-10-30T12:40:07.953 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/37' 2023-10-30T12:40:07.953 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './271' 2023-10-30T12:40:07.954 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/21' 2023-10-30T12:40:07.954 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/27' 2023-10-30T12:40:07.954 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/31' 2023-10-30T12:40:07.954 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/36' 2023-10-30T12:40:07.954 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/20' 2023-10-30T12:40:07.955 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/39' 2023-10-30T12:40:07.955 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/34' 2023-10-30T12:40:07.955 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/38' 2023-10-30T12:40:07.955 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/28' 2023-10-30T12:40:07.955 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/24' 2023-10-30T12:40:07.955 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/29' 2023-10-30T12:40:07.956 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/32' 2023-10-30T12:40:07.956 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/22' 2023-10-30T12:40:07.956 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/25' 2023-10-30T12:40:07.956 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/30' 2023-10-30T12:40:07.956 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/33' 2023-10-30T12:40:07.956 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/23' 2023-10-30T12:40:07.957 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/35' 2023-10-30T12:40:07.957 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/26' 2023-10-30T12:40:07.957 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/37' 2023-10-30T12:40:07.957 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './272' 2023-10-30T12:40:07.957 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/21' 2023-10-30T12:40:07.957 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/27' 2023-10-30T12:40:07.958 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/31' 2023-10-30T12:40:07.958 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/36' 2023-10-30T12:40:07.958 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/20' 2023-10-30T12:40:07.958 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/39' 2023-10-30T12:40:07.958 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/34' 2023-10-30T12:40:07.958 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/38' 2023-10-30T12:40:07.959 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/28' 2023-10-30T12:40:07.959 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/24' 2023-10-30T12:40:07.959 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/29' 2023-10-30T12:40:07.959 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/32' 2023-10-30T12:40:07.959 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/22' 2023-10-30T12:40:07.959 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/25' 2023-10-30T12:40:07.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/30' 2023-10-30T12:40:07.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/33' 2023-10-30T12:40:07.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/23' 2023-10-30T12:40:07.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/35' 2023-10-30T12:40:07.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/26' 2023-10-30T12:40:07.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/37' 2023-10-30T12:40:07.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './273' 2023-10-30T12:40:07.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/21' 2023-10-30T12:40:07.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/27' 2023-10-30T12:40:07.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/31' 2023-10-30T12:40:07.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/36' 2023-10-30T12:40:07.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/20' 2023-10-30T12:40:07.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/39' 2023-10-30T12:40:07.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/34' 2023-10-30T12:40:07.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/38' 2023-10-30T12:40:07.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/28' 2023-10-30T12:40:07.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/24' 2023-10-30T12:40:07.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/29' 2023-10-30T12:40:07.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/32' 2023-10-30T12:40:07.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/22' 2023-10-30T12:40:07.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/25' 2023-10-30T12:40:07.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/30' 2023-10-30T12:40:07.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/33' 2023-10-30T12:40:07.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/23' 2023-10-30T12:40:07.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/35' 2023-10-30T12:40:07.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/26' 2023-10-30T12:40:07.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/37' 2023-10-30T12:40:07.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './274' 2023-10-30T12:40:07.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/21' 2023-10-30T12:40:07.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/27' 2023-10-30T12:40:07.965 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/31' 2023-10-30T12:40:07.965 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/36' 2023-10-30T12:40:07.965 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/20' 2023-10-30T12:40:07.965 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/39' 2023-10-30T12:40:07.965 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/34' 2023-10-30T12:40:07.966 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/38' 2023-10-30T12:40:07.966 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/28' 2023-10-30T12:40:07.966 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/24' 2023-10-30T12:40:07.966 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/29' 2023-10-30T12:40:07.966 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/32' 2023-10-30T12:40:07.966 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/22' 2023-10-30T12:40:07.967 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/25' 2023-10-30T12:40:07.967 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/30' 2023-10-30T12:40:07.967 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/33' 2023-10-30T12:40:07.967 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/23' 2023-10-30T12:40:07.967 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/35' 2023-10-30T12:40:07.967 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/26' 2023-10-30T12:40:07.968 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/37' 2023-10-30T12:40:07.968 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './275' 2023-10-30T12:40:07.968 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/21' 2023-10-30T12:40:07.968 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/27' 2023-10-30T12:40:07.968 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/31' 2023-10-30T12:40:07.968 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/36' 2023-10-30T12:40:07.969 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/20' 2023-10-30T12:40:07.969 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/39' 2023-10-30T12:40:07.969 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/34' 2023-10-30T12:40:07.969 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/38' 2023-10-30T12:40:07.969 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/28' 2023-10-30T12:40:07.969 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/24' 2023-10-30T12:40:07.970 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/29' 2023-10-30T12:40:07.970 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/32' 2023-10-30T12:40:07.970 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/22' 2023-10-30T12:40:07.970 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/25' 2023-10-30T12:40:07.970 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/30' 2023-10-30T12:40:07.970 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/33' 2023-10-30T12:40:07.971 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/23' 2023-10-30T12:40:07.971 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/35' 2023-10-30T12:40:07.971 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/26' 2023-10-30T12:40:20.766 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/37' 2023-10-30T12:40:20.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './276' 2023-10-30T12:40:20.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/21' 2023-10-30T12:40:20.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/27' 2023-10-30T12:40:20.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/31' 2023-10-30T12:40:20.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/36' 2023-10-30T12:40:20.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/20' 2023-10-30T12:40:20.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/39' 2023-10-30T12:40:20.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/34' 2023-10-30T12:40:20.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/38' 2023-10-30T12:40:20.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/28' 2023-10-30T12:40:20.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/24' 2023-10-30T12:40:20.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/29' 2023-10-30T12:40:20.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/32' 2023-10-30T12:40:20.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/22' 2023-10-30T12:40:20.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/25' 2023-10-30T12:40:20.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/30' 2023-10-30T12:40:20.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/33' 2023-10-30T12:40:20.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/23' 2023-10-30T12:40:20.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/35' 2023-10-30T12:40:20.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/26' 2023-10-30T12:40:20.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/37' 2023-10-30T12:40:20.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './277' 2023-10-30T12:40:20.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/21' 2023-10-30T12:40:20.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/27' 2023-10-30T12:40:20.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/31' 2023-10-30T12:40:20.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/36' 2023-10-30T12:40:20.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/20' 2023-10-30T12:40:20.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/39' 2023-10-30T12:40:20.772 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/34' 2023-10-30T12:40:20.772 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/38' 2023-10-30T12:40:20.772 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/28' 2023-10-30T12:40:20.772 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/24' 2023-10-30T12:40:20.772 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/29' 2023-10-30T12:40:20.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/32' 2023-10-30T12:40:20.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/22' 2023-10-30T12:40:20.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/25' 2023-10-30T12:40:20.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/30' 2023-10-30T12:40:20.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/33' 2023-10-30T12:40:20.773 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/23' 2023-10-30T12:40:20.774 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/35' 2023-10-30T12:40:20.774 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/26' 2023-10-30T12:40:20.774 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/37' 2023-10-30T12:40:20.774 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './278' 2023-10-30T12:40:20.774 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/21' 2023-10-30T12:40:20.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/27' 2023-10-30T12:40:20.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/31' 2023-10-30T12:40:20.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/36' 2023-10-30T12:40:20.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/20' 2023-10-30T12:40:20.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/39' 2023-10-30T12:40:20.775 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/34' 2023-10-30T12:40:20.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/38' 2023-10-30T12:40:20.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/28' 2023-10-30T12:40:20.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/24' 2023-10-30T12:40:20.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/29' 2023-10-30T12:40:20.776 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/32' 2023-10-30T12:40:20.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/22' 2023-10-30T12:40:20.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/25' 2023-10-30T12:40:20.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/30' 2023-10-30T12:40:20.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/33' 2023-10-30T12:40:20.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/23' 2023-10-30T12:40:20.777 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/35' 2023-10-30T12:40:20.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/26' 2023-10-30T12:40:20.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/37' 2023-10-30T12:40:20.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './279' 2023-10-30T12:40:20.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/21' 2023-10-30T12:40:20.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/27' 2023-10-30T12:40:20.778 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/31' 2023-10-30T12:40:20.779 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/36' 2023-10-30T12:40:20.779 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/20' 2023-10-30T12:40:20.779 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/39' 2023-10-30T12:40:20.779 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/34' 2023-10-30T12:40:20.779 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/38' 2023-10-30T12:40:20.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/28' 2023-10-30T12:40:20.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/24' 2023-10-30T12:40:20.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/29' 2023-10-30T12:40:20.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/32' 2023-10-30T12:40:20.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/22' 2023-10-30T12:40:20.780 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/25' 2023-10-30T12:40:20.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/30' 2023-10-30T12:40:20.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/33' 2023-10-30T12:40:20.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/23' 2023-10-30T12:40:20.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/35' 2023-10-30T12:40:20.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/26' 2023-10-30T12:40:20.781 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/37' 2023-10-30T12:40:20.782 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './280' 2023-10-30T12:40:20.782 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/21' 2023-10-30T12:40:20.782 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/27' 2023-10-30T12:40:20.782 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/31' 2023-10-30T12:40:20.782 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/36' 2023-10-30T12:40:20.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/20' 2023-10-30T12:40:20.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/39' 2023-10-30T12:40:20.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/34' 2023-10-30T12:40:20.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/38' 2023-10-30T12:40:20.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/28' 2023-10-30T12:40:20.783 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/24' 2023-10-30T12:40:20.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/29' 2023-10-30T12:40:20.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/32' 2023-10-30T12:40:20.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/22' 2023-10-30T12:40:20.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/25' 2023-10-30T12:40:20.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/30' 2023-10-30T12:40:20.784 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/33' 2023-10-30T12:40:20.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/23' 2023-10-30T12:40:20.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/35' 2023-10-30T12:40:20.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/26' 2023-10-30T12:40:20.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/37' 2023-10-30T12:40:20.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './281' 2023-10-30T12:40:20.785 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/21' 2023-10-30T12:40:20.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/27' 2023-10-30T12:40:20.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/31' 2023-10-30T12:40:20.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/36' 2023-10-30T12:40:20.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/20' 2023-10-30T12:40:20.786 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/39' 2023-10-30T12:40:20.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/34' 2023-10-30T12:40:20.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/38' 2023-10-30T12:40:20.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/28' 2023-10-30T12:40:20.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/24' 2023-10-30T12:40:20.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/29' 2023-10-30T12:40:20.787 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/32' 2023-10-30T12:40:20.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/22' 2023-10-30T12:40:20.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/25' 2023-10-30T12:40:20.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/30' 2023-10-30T12:40:20.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/33' 2023-10-30T12:40:20.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/23' 2023-10-30T12:40:20.788 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/35' 2023-10-30T12:40:20.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/26' 2023-10-30T12:40:20.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/37' 2023-10-30T12:40:20.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './282' 2023-10-30T12:40:20.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/21' 2023-10-30T12:40:20.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/27' 2023-10-30T12:40:20.789 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/31' 2023-10-30T12:40:20.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/36' 2023-10-30T12:40:20.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/20' 2023-10-30T12:40:20.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/39' 2023-10-30T12:40:20.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/34' 2023-10-30T12:40:20.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/38' 2023-10-30T12:40:20.790 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/28' 2023-10-30T12:40:20.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/24' 2023-10-30T12:40:20.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/29' 2023-10-30T12:40:20.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/32' 2023-10-30T12:40:20.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/22' 2023-10-30T12:40:20.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/25' 2023-10-30T12:40:20.791 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/30' 2023-10-30T12:40:20.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/33' 2023-10-30T12:40:20.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/23' 2023-10-30T12:40:20.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/35' 2023-10-30T12:40:20.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/26' 2023-10-30T12:40:20.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/37' 2023-10-30T12:40:20.792 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './283' 2023-10-30T12:40:20.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/21' 2023-10-30T12:40:20.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/27' 2023-10-30T12:40:20.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/31' 2023-10-30T12:40:20.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/36' 2023-10-30T12:40:20.793 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/20' 2023-10-30T12:40:20.794 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/39' 2023-10-30T12:40:20.794 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/34' 2023-10-30T12:40:20.794 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/38' 2023-10-30T12:40:20.794 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/28' 2023-10-30T12:40:20.794 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/24' 2023-10-30T12:40:20.794 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/29' 2023-10-30T12:40:20.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/32' 2023-10-30T12:40:20.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/22' 2023-10-30T12:40:20.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/25' 2023-10-30T12:40:20.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/30' 2023-10-30T12:40:20.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/33' 2023-10-30T12:40:20.795 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/23' 2023-10-30T12:40:20.796 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/35' 2023-10-30T12:40:20.796 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/26' 2023-10-30T12:40:20.796 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/37' 2023-10-30T12:40:20.796 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './284' 2023-10-30T12:40:20.796 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/21' 2023-10-30T12:40:20.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/27' 2023-10-30T12:40:20.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/31' 2023-10-30T12:40:20.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/36' 2023-10-30T12:40:20.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/20' 2023-10-30T12:40:20.797 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/39' 2023-10-30T12:40:20.798 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/34' 2023-10-30T12:40:20.798 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/38' 2023-10-30T12:40:20.798 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/28' 2023-10-30T12:40:20.798 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/24' 2023-10-30T12:40:20.798 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/29' 2023-10-30T12:40:20.799 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/32' 2023-10-30T12:40:20.799 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/22' 2023-10-30T12:40:20.799 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/25' 2023-10-30T12:40:20.799 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/30' 2023-10-30T12:40:20.799 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/33' 2023-10-30T12:40:20.799 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/23' 2023-10-30T12:40:20.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/35' 2023-10-30T12:40:20.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/26' 2023-10-30T12:40:20.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/37' 2023-10-30T12:40:20.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './285' 2023-10-30T12:40:20.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/21' 2023-10-30T12:40:20.800 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/27' 2023-10-30T12:40:20.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/31' 2023-10-30T12:40:20.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/36' 2023-10-30T12:40:20.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/20' 2023-10-30T12:40:20.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/39' 2023-10-30T12:40:20.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/34' 2023-10-30T12:40:20.801 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/38' 2023-10-30T12:40:20.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/28' 2023-10-30T12:40:20.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/24' 2023-10-30T12:40:20.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/29' 2023-10-30T12:40:20.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/32' 2023-10-30T12:40:20.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/22' 2023-10-30T12:40:20.802 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/25' 2023-10-30T12:40:20.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/30' 2023-10-30T12:40:20.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/33' 2023-10-30T12:40:20.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/23' 2023-10-30T12:40:20.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/35' 2023-10-30T12:40:20.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/26' 2023-10-30T12:40:20.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/37' 2023-10-30T12:40:20.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './286' 2023-10-30T12:40:22.401 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:22.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.554+0000 7f1280194640 1 Processor -- start 2023-10-30T12:40:22.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.554+0000 7f1280194640 1 -- start start 2023-10-30T12:40:22.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.558+0000 7f1280194640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f12781538a0 0x7f127815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:22.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.558+0000 7f1280194640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f12780ca0e0 0x7f127815a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:22.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.558+0000 7f1280194640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12780cb800 con 0x7f12780cf160 2023-10-30T12:40:22.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.558+0000 7f1280194640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f127815dc30 con 0x7f12780ca0e0 2023-10-30T12:40:22.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.558+0000 7f1280194640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12780cb330 con 0x7f12781538a0 2023-10-30T12:40:22.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.558+0000 7f127d72f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f12780ca0e0 0x7f127815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:22.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.558+0000 7f127d72f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f12780ca0e0 0x7f127815a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46354/0 (socket says 172.21.15.12:46354) 2023-10-30T12:40:22.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.558+0000 7f127d72f640 1 -- 172.21.15.12:0/4290124775 learned_addr learned my addr 172.21.15.12:0/4290124775 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:22.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.558+0000 7f127df30640 1 --2- 172.21.15.12:0/4290124775 >> v2:172.21.15.154:3300/0 conn(0x7f12781538a0 0x7f127815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:22.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.558+0000 7f127d72f640 1 -- 172.21.15.12:0/4290124775 >> v2:172.21.15.154:3300/0 conn(0x7f12781538a0 msgr2=0x7f127815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:22.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.558+0000 7f127d72f640 1 --2- 172.21.15.12:0/4290124775 >> v2:172.21.15.154:3300/0 conn(0x7f12781538a0 0x7f127815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:22.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.558+0000 7f127d72f640 1 -- 172.21.15.12:0/4290124775 >> v1:172.21.15.154:6789/0 conn(0x7f12780cf160 legacy=0x7f12780cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:22.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.558+0000 7f127d72f640 1 -- 172.21.15.12:0/4290124775 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12780cad70 con 0x7f12780ca0e0 2023-10-30T12:40:22.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.562+0000 7f127d72f640 1 --2- 172.21.15.12:0/4290124775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12780ca0e0 0x7f127815a090 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f12780cfd10 tx=0x7f126801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a144a745bf739d33 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:22.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.562+0000 7f127cf2e640 1 -- 172.21.15.12:0/4290124775 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12680a89f0 con 0x7f12780ca0e0 2023-10-30T12:40:22.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.562+0000 7f127cf2e640 1 -- 172.21.15.12:0/4290124775 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12680a8b50 con 0x7f12780ca0e0 2023-10-30T12:40:22.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.562+0000 7f127cf2e640 1 -- 172.21.15.12:0/4290124775 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12680b0440 con 0x7f12780ca0e0 2023-10-30T12:40:22.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.562+0000 7f1280194640 1 -- 172.21.15.12:0/4290124775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12780ca0e0 msgr2=0x7f127815a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:22.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.562+0000 7f1280194640 1 --2- 172.21.15.12:0/4290124775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12780ca0e0 0x7f127815a090 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f12780cfd10 tx=0x7f126801ecf0 comp rx=0 tx=0).stop 2023-10-30T12:40:22.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.562+0000 7f1280194640 1 -- 172.21.15.12:0/4290124775 shutdown_connections 2023-10-30T12:40:22.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.562+0000 7f1280194640 1 --2- 172.21.15.12:0/4290124775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12780ca0e0 0x7f127815a090 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:22.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.562+0000 7f1280194640 1 --2- 172.21.15.12:0/4290124775 >> v2:172.21.15.154:3300/0 conn(0x7f12781538a0 0x7f127815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:22.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.562+0000 7f1280194640 1 -- 172.21.15.12:0/4290124775 >> 172.21.15.12:0/4290124775 conn(0x7f127806bfe0 msgr2=0x7f127806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:22.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.562+0000 7f1280194640 1 -- 172.21.15.12:0/4290124775 shutdown_connections 2023-10-30T12:40:22.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.562+0000 7f1280194640 1 -- 172.21.15.12:0/4290124775 wait complete. 2023-10-30T12:40:22.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.562+0000 7f1280194640 1 Processor -- start 2023-10-30T12:40:22.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.562+0000 7f1280194640 1 -- start start 2023-10-30T12:40:22.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f1280194640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12780ca0e0 0x7f12780ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:22.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f1280194640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12780cf160 0x7f1278142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:22.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f1280194640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12781538a0 0x7f12781ef7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:22.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f1280194640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12781584d0 con 0x7f12780ca0e0 2023-10-30T12:40:22.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f1280194640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12781ef390 con 0x7f12781538a0 2023-10-30T12:40:22.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f1280194640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12780cec20 con 0x7f12780cf160 2023-10-30T12:40:22.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f127df30640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12780ca0e0 0x7f12780ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:22.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f127df30640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12780ca0e0 0x7f12780ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46356/0 (socket says 172.21.15.12:46356) 2023-10-30T12:40:22.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f127df30640 1 -- 172.21.15.12:0/2723258732 learned_addr learned my addr 172.21.15.12:0/2723258732 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:22.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f127e731640 1 --2- 172.21.15.12:0/2723258732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12781538a0 0x7f12781ef7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:22.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f127d72f640 1 --2- 172.21.15.12:0/2723258732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12780cf160 0x7f1278142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:22.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f127df30640 1 -- 172.21.15.12:0/2723258732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12780cf160 msgr2=0x7f1278142550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:22.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f127df30640 1 --2- 172.21.15.12:0/2723258732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12780cf160 0x7f1278142550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:22.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f127df30640 1 -- 172.21.15.12:0/2723258732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12781538a0 msgr2=0x7f12781ef7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:22.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f127df30640 1 --2- 172.21.15.12:0/2723258732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12781538a0 0x7f12781ef7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:22.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f127df30640 1 -- 172.21.15.12:0/2723258732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12680094d0 con 0x7f12780ca0e0 2023-10-30T12:40:22.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f127df30640 1 --2- 172.21.15.12:0/2723258732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12780ca0e0 0x7f12780ce630 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f127400d5f0 tx=0x7f127400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:22.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f126e7fc640 1 -- 172.21.15.12:0/2723258732 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12740125f0 con 0x7f12780ca0e0 2023-10-30T12:40:22.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f126e7fc640 1 -- 172.21.15.12:0/2723258732 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1274012750 con 0x7f12780ca0e0 2023-10-30T12:40:22.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f126e7fc640 1 -- 172.21.15.12:0/2723258732 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274012b00 con 0x7f12780ca0e0 2023-10-30T12:40:22.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f1280194640 1 -- 172.21.15.12:0/2723258732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12780ceec0 con 0x7f12780ca0e0 2023-10-30T12:40:22.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f1280194640 1 -- 172.21.15.12:0/2723258732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1278149080 con 0x7f12780ca0e0 2023-10-30T12:40:22.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f1280194640 1 -- 172.21.15.12:0/2723258732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12780b2a50 con 0x7f12780ca0e0 2023-10-30T12:40:22.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.566+0000 7f126e7fc640 1 -- 172.21.15.12:0/2723258732 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1274012c60 con 0x7f12780ca0e0 2023-10-30T12:40:22.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.570+0000 7f126e7fc640 1 --2- 172.21.15.12:0/2723258732 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f125c0664f0 0x7f125c0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:22.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.570+0000 7f126e7fc640 1 -- 172.21.15.12:0/2723258732 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f1274084530 con 0x7f12780ca0e0 2023-10-30T12:40:22.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.570+0000 7f127d72f640 1 --2- 172.21.15.12:0/2723258732 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f125c0664f0 0x7f125c0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:22.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.570+0000 7f127d72f640 1 --2- 172.21.15.12:0/2723258732 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f125c0664f0 0x7f125c0689b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f12680b2000 tx=0x7f1268008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:22.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.574+0000 7f126e7fc640 1 -- 172.21.15.12:0/2723258732 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1274051820 con 0x7f12780ca0e0 2023-10-30T12:40:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.774+0000 7f1280194640 1 -- 172.21.15.12:0/2723258732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f127804e170 con 0x7f12780ca0e0 2023-10-30T12:40:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.778+0000 7f126e7fc640 1 -- 172.21.15.12:0/2723258732 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6535 (secure 0 0 0) 0x7f1274054e40 con 0x7f12780ca0e0 2023-10-30T12:40:22.782 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:22.782 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4971,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:12.331151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:active","state_seq":53,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 25 2023-10-30T12:40:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.778+0000 7f1280194640 1 -- 172.21.15.12:0/2723258732 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f125c0664f0 msgr2=0x7f125c0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.778+0000 7f1280194640 1 --2- 172.21.15.12:0/2723258732 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f125c0664f0 0x7f125c0689b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f12680b2000 tx=0x7f1268008040 comp rx=0 tx=0).stop 2023-10-30T12:40:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.778+0000 7f1280194640 1 -- 172.21.15.12:0/2723258732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12780ca0e0 msgr2=0x7f12780ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.778+0000 7f1280194640 1 --2- 172.21.15.12:0/2723258732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12780ca0e0 0x7f12780ce630 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f127400d5f0 tx=0x7f127400dab0 comp rx=0 tx=0).stop 2023-10-30T12:40:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.778+0000 7f1280194640 1 -- 172.21.15.12:0/2723258732 shutdown_connections 2023-10-30T12:40:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.778+0000 7f1280194640 1 --2- 172.21.15.12:0/2723258732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12781538a0 0x7f12781ef7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.778+0000 7f1280194640 1 --2- 172.21.15.12:0/2723258732 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f125c0664f0 0x7f125c0689b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.778+0000 7f1280194640 1 --2- 172.21.15.12:0/2723258732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12780cf160 0x7f1278142550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.778+0000 7f1280194640 1 --2- 172.21.15.12:0/2723258732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12780ca0e0 0x7f12780ce630 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:22.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.778+0000 7f1280194640 1 -- 172.21.15.12:0/2723258732 >> 172.21.15.12:0/2723258732 conn(0x7f127806bfe0 msgr2=0x7f1278152310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:22.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.778+0000 7f1280194640 1 -- 172.21.15.12:0/2723258732 shutdown_connections 2023-10-30T12:40:22.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:22.778+0000 7f1280194640 1 -- 172.21.15.12:0/2723258732 wait complete. 2023-10-30T12:40:22.803 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=2) 2023-10-30T12:40:22.804 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-30T12:40:22.804 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:40:22.822 INFO:tasks.ceph.mds.a:Stopped 2023-10-30T12:40:22.822 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:23.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a7ad6640 1 Processor -- start 2023-10-30T12:40:23.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a7ad6640 1 -- start start 2023-10-30T12:40:23.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a7ad6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa2a00c6b80 0x7fa2a00bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:23.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a7ad6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2a00c6f60 0x7fa2a00c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:23.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a7ad6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa2a00c8430 0x7fa2a013ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:23.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a7ad6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a00c0a50 con 0x7fa2a00c6b80 2023-10-30T12:40:23.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a7ad6640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a006f6e0 con 0x7fa2a00c8430 2023-10-30T12:40:23.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a7ad6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a006f9c0 con 0x7fa2a00c6f60 2023-10-30T12:40:23.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a5872640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa2a00c6b80 0x7fa2a00bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:23.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a5872640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa2a00c6b80 0x7fa2a00bfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46370/0 (socket says 172.21.15.12:46370) 2023-10-30T12:40:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a5872640 1 -- 172.21.15.12:0/1777074870 learned_addr learned my addr 172.21.15.12:0/1777074870 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a6073640 1 --2- 172.21.15.12:0/1777074870 >> v2:172.21.15.154:3300/0 conn(0x7fa2a00c8430 0x7fa2a013ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a5071640 1 --2- 172.21.15.12:0/1777074870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2a00c6f60 0x7fa2a00c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a5872640 1 -- 172.21.15.12:0/1777074870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2a00c6f60 msgr2=0x7fa2a00c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a5872640 1 --2- 172.21.15.12:0/1777074870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2a00c6f60 0x7fa2a00c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a5872640 1 -- 172.21.15.12:0/1777074870 >> v2:172.21.15.154:3300/0 conn(0x7fa2a00c8430 msgr2=0x7fa2a013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a5872640 1 --2- 172.21.15.12:0/1777074870 >> v2:172.21.15.154:3300/0 conn(0x7fa2a00c8430 0x7fa2a013ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a5872640 1 -- 172.21.15.12:0/1777074870 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a006fca0 con 0x7fa2a00c6b80 2023-10-30T12:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.002+0000 7fa2a6073640 1 --2- 172.21.15.12:0/1777074870 >> v2:172.21.15.154:3300/0 conn(0x7fa2a00c8430 0x7fa2a013ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.006+0000 7fa2a5872640 1 --2- 172.21.15.12:0/1777074870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2a00c6b80 0x7fa2a00bfec0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00b0860 tx=0x7fa29c01eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606142f5a162a6a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.006+0000 7fa2a4870640 1 -- 172.21.15.12:0/1777074870 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c0a7ae0 con 0x7fa2a00c6b80 2023-10-30T12:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.006+0000 7fa2a4870640 1 -- 172.21.15.12:0/1777074870 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29c0a7c40 con 0x7fa2a00c6b80 2023-10-30T12:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.006+0000 7fa2a4870640 1 -- 172.21.15.12:0/1777074870 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c0af440 con 0x7fa2a00c6b80 2023-10-30T12:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.006+0000 7fa2a7ad6640 1 -- 172.21.15.12:0/1777074870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2a00c6b80 msgr2=0x7fa2a00bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.006+0000 7fa2a7ad6640 1 --2- 172.21.15.12:0/1777074870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2a00c6b80 0x7fa2a00bfec0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00b0860 tx=0x7fa29c01eb40 comp rx=0 tx=0).stop 2023-10-30T12:40:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.006+0000 7fa2a7ad6640 1 -- 172.21.15.12:0/1777074870 shutdown_connections 2023-10-30T12:40:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.006+0000 7fa2a7ad6640 1 --2- 172.21.15.12:0/1777074870 >> v2:172.21.15.154:3300/0 conn(0x7fa2a00c8430 0x7fa2a013ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.006+0000 7fa2a7ad6640 1 --2- 172.21.15.12:0/1777074870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2a00c6f60 0x7fa2a00c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.006+0000 7fa2a7ad6640 1 --2- 172.21.15.12:0/1777074870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2a00c6b80 0x7fa2a00bfec0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.006+0000 7fa2a7ad6640 1 -- 172.21.15.12:0/1777074870 >> 172.21.15.12:0/1777074870 conn(0x7fa2a00c7910 msgr2=0x7fa2a00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.006+0000 7fa2a7ad6640 1 -- 172.21.15.12:0/1777074870 shutdown_connections 2023-10-30T12:40:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.006+0000 7fa2a7ad6640 1 -- 172.21.15.12:0/1777074870 wait complete. 2023-10-30T12:40:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a7ad6640 1 Processor -- start 2023-10-30T12:40:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a7ad6640 1 -- start start 2023-10-30T12:40:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a7ad6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2a00c6b80 0x7fa2a012f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a7ad6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2a00c6f60 0x7fa2a0133570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a7ad6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2a00c8430 0x7fa2a0133ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:23.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a7ad6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a01330e0 con 0x7fa2a00c6f60 2023-10-30T12:40:23.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a7ad6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a01327f0 con 0x7fa2a00c6b80 2023-10-30T12:40:23.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a7ad6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a00b0630 con 0x7fa2a00c8430 2023-10-30T12:40:23.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a5071640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2a00c6f60 0x7fa2a0133570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:23.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a5071640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2a00c6f60 0x7fa2a0133570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46380/0 (socket says 172.21.15.12:46380) 2023-10-30T12:40:23.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a5071640 1 -- 172.21.15.12:0/2576628501 learned_addr learned my addr 172.21.15.12:0/2576628501 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:23.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a5872640 1 --2- 172.21.15.12:0/2576628501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2a00c6b80 0x7fa2a012f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:23.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a6073640 1 --2- 172.21.15.12:0/2576628501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2a00c8430 0x7fa2a0133ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:23.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a5071640 1 -- 172.21.15.12:0/2576628501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2a00c8430 msgr2=0x7fa2a0133ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:23.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a5071640 1 --2- 172.21.15.12:0/2576628501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2a00c8430 0x7fa2a0133ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:23.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a5071640 1 -- 172.21.15.12:0/2576628501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2a00c6b80 msgr2=0x7fa2a012f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:23.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a5071640 1 --2- 172.21.15.12:0/2576628501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2a00c6b80 0x7fa2a012f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:23.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a5071640 1 -- 172.21.15.12:0/2576628501 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c00b770 con 0x7fa2a00c6f60 2023-10-30T12:40:23.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a5071640 1 --2- 172.21.15.12:0/2576628501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2a00c6f60 0x7fa2a0133570 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa29000d800 tx=0x7fa29000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:23.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2967fc640 1 -- 172.21.15.12:0/2576628501 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2900096a0 con 0x7fa2a00c6f60 2023-10-30T12:40:23.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2967fc640 1 -- 172.21.15.12:0/2576628501 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa290009800 con 0x7fa2a00c6f60 2023-10-30T12:40:23.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a7ad6640 1 -- 172.21.15.12:0/2576628501 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a0130e20 con 0x7fa2a00c6f60 2023-10-30T12:40:23.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2967fc640 1 -- 172.21.15.12:0/2576628501 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa290009b70 con 0x7fa2a00c6f60 2023-10-30T12:40:23.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.010+0000 7fa2a7ad6640 1 -- 172.21.15.12:0/2576628501 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a0132a90 con 0x7fa2a00c6f60 2023-10-30T12:40:23.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.014+0000 7fa2a7ad6640 1 -- 172.21.15.12:0/2576628501 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a0133290 con 0x7fa2a00c6f60 2023-10-30T12:40:23.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.014+0000 7fa2967fc640 1 -- 172.21.15.12:0/2576628501 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa290009cd0 con 0x7fa2a00c6f60 2023-10-30T12:40:23.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.014+0000 7fa2967fc640 1 --2- 172.21.15.12:0/2576628501 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa2840664a0 0x7fa284068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:23.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.014+0000 7fa2967fc640 1 -- 172.21.15.12:0/2576628501 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fa290084820 con 0x7fa2a00c6f60 2023-10-30T12:40:23.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.014+0000 7fa2a5872640 1 --2- 172.21.15.12:0/2576628501 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa2840664a0 0x7fa284068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:23.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.018+0000 7fa2a5872640 1 --2- 172.21.15.12:0/2576628501 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa2840664a0 0x7fa284068960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0b1000 tx=0x7fa29c0a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:23.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.018+0000 7fa2967fc640 1 -- 172.21.15.12:0/2576628501 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa290051a90 con 0x7fa2a00c6f60 2023-10-30T12:40:23.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.210+0000 7fa2a7ad6640 1 -- 172.21.15.12:0/2576628501 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2a004e170 con 0x7fa2a00c6f60 2023-10-30T12:40:23.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.210+0000 7fa2967fc640 1 -- 172.21.15.12:0/2576628501 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6535 (secure 0 0 0) 0x7fa2900550b0 con 0x7fa2a00c6f60 2023-10-30T12:40:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:23.215 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4971,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:12.331151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:active","state_seq":53,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:23.215 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 25 2023-10-30T12:40:23.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.210+0000 7fa2a7ad6640 1 -- 172.21.15.12:0/2576628501 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa2840664a0 msgr2=0x7fa284068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:23.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.210+0000 7fa2a7ad6640 1 --2- 172.21.15.12:0/2576628501 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa2840664a0 0x7fa284068960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0b1000 tx=0x7fa29c0a9070 comp rx=0 tx=0).stop 2023-10-30T12:40:23.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.210+0000 7fa2a7ad6640 1 -- 172.21.15.12:0/2576628501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2a00c6f60 msgr2=0x7fa2a0133570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:23.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.210+0000 7fa2a7ad6640 1 --2- 172.21.15.12:0/2576628501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2a00c6f60 0x7fa2a0133570 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa29000d800 tx=0x7fa29000dcc0 comp rx=0 tx=0).stop 2023-10-30T12:40:23.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.210+0000 7fa2a7ad6640 1 -- 172.21.15.12:0/2576628501 shutdown_connections 2023-10-30T12:40:23.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.210+0000 7fa2a7ad6640 1 --2- 172.21.15.12:0/2576628501 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa2840664a0 0x7fa284068960 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:23.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.210+0000 7fa2a7ad6640 1 --2- 172.21.15.12:0/2576628501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2a00c8430 0x7fa2a0133ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:23.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.210+0000 7fa2a7ad6640 1 --2- 172.21.15.12:0/2576628501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2a00c6f60 0x7fa2a0133570 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:23.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.210+0000 7fa2a7ad6640 1 --2- 172.21.15.12:0/2576628501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2a00c6b80 0x7fa2a012f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:23.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.210+0000 7fa2a7ad6640 1 -- 172.21.15.12:0/2576628501 >> 172.21.15.12:0/2576628501 conn(0x7fa2a00c7910 msgr2=0x7fa2a00c0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:23.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.210+0000 7fa2a7ad6640 1 -- 172.21.15.12:0/2576628501 shutdown_connections 2023-10-30T12:40:23.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:23.210+0000 7fa2a7ad6640 1 -- 172.21.15.12:0/2576628501 wait complete. 2023-10-30T12:40:23.235 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:40:25.236 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:25.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.386+0000 7f5166e79640 1 Processor -- start 2023-10-30T12:40:25.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.386+0000 7f5166e79640 1 -- start start 2023-10-30T12:40:25.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.390+0000 7f5166e79640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51600c6bd0 0x7f516006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:25.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.390+0000 7f5166e79640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5160071110 0x7f5160158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:25.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.390+0000 7f5166e79640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5160066cf0 con 0x7f516006f7a0 2023-10-30T12:40:25.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.390+0000 7f5166e79640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51600700c0 con 0x7f5160071110 2023-10-30T12:40:25.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.390+0000 7f5166e79640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51600c70d0 con 0x7f51600c6bd0 2023-10-30T12:40:25.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.390+0000 7f5165416640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5160071110 0x7f5160158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:25.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.390+0000 7f5165416640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5160071110 0x7f5160158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46392/0 (socket says 172.21.15.12:46392) 2023-10-30T12:40:25.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.390+0000 7f5165416640 1 -- 172.21.15.12:0/1107655336 learned_addr learned my addr 172.21.15.12:0/1107655336 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:25.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.390+0000 7f515ffff640 1 --2- 172.21.15.12:0/1107655336 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51600c6bd0 0x7f516006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:25.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.390+0000 7f5165416640 1 -- 172.21.15.12:0/1107655336 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51600c6bd0 msgr2=0x7f516006fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:25.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.390+0000 7f5165416640 1 --2- 172.21.15.12:0/1107655336 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51600c6bd0 0x7f516006fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:25.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.390+0000 7f5165416640 1 -- 172.21.15.12:0/1107655336 >> v1:172.21.15.154:6789/0 conn(0x7f516006f7a0 legacy=0x7f5160069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:25.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.390+0000 7f5165416640 1 -- 172.21.15.12:0/1107655336 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51600c7390 con 0x7f5160071110 2023-10-30T12:40:25.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5165416640 1 --2- 172.21.15.12:0/1107655336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5160071110 0x7f5160158110 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5158009e50 tx=0x7f515801f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1094b2e6e2b49ba server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:25.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f515f7fe640 1 -- 172.21.15.12:0/1107655336 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51580af400 con 0x7f5160071110 2023-10-30T12:40:25.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f515f7fe640 1 -- 172.21.15.12:0/1107655336 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51580ab050 con 0x7f5160071110 2023-10-30T12:40:25.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f515f7fe640 1 -- 172.21.15.12:0/1107655336 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51580af780 con 0x7f5160071110 2023-10-30T12:40:25.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 -- 172.21.15.12:0/1107655336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5160071110 msgr2=0x7f5160158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:25.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 --2- 172.21.15.12:0/1107655336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5160071110 0x7f5160158110 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5158009e50 tx=0x7f515801f1f0 comp rx=0 tx=0).stop 2023-10-30T12:40:25.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 -- 172.21.15.12:0/1107655336 shutdown_connections 2023-10-30T12:40:25.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 --2- 172.21.15.12:0/1107655336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5160071110 0x7f5160158110 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:25.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 --2- 172.21.15.12:0/1107655336 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51600c6bd0 0x7f516006fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:25.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 -- 172.21.15.12:0/1107655336 >> 172.21.15.12:0/1107655336 conn(0x7f51600c7910 msgr2=0x7f51600c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:25.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 -- 172.21.15.12:0/1107655336 shutdown_connections 2023-10-30T12:40:25.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 -- 172.21.15.12:0/1107655336 wait complete. 2023-10-30T12:40:25.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 Processor -- start 2023-10-30T12:40:25.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 -- start start 2023-10-30T12:40:25.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f516006f7a0 0x7f51600ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:25.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5160071110 0x7f51600b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:25.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51600c6bd0 0x7f51601461d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:25.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51601f27a0 con 0x7f5160071110 2023-10-30T12:40:25.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5160157850 con 0x7f516006f7a0 2023-10-30T12:40:25.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.394+0000 7f5166e79640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51601f2c90 con 0x7f51600c6bd0 2023-10-30T12:40:25.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f515ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5160071110 0x7f51600b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:25.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f515ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5160071110 0x7f51600b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46396/0 (socket says 172.21.15.12:46396) 2023-10-30T12:40:25.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f515ffff640 1 -- 172.21.15.12:0/1947395608 learned_addr learned my addr 172.21.15.12:0/1947395608 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:25.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f5164c15640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f516006f7a0 0x7f51600ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:25.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f5165416640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51600c6bd0 0x7f51601461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:25.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f515ffff640 1 -- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51600c6bd0 msgr2=0x7f51601461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:25.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f515ffff640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51600c6bd0 0x7f51601461d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:25.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f515ffff640 1 -- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f516006f7a0 msgr2=0x7f51600ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:25.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f515ffff640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f516006f7a0 0x7f51600ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:25.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f515ffff640 1 -- 172.21.15.12:0/1947395608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5158009b30 con 0x7f5160071110 2023-10-30T12:40:25.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f5164c15640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f516006f7a0 0x7f51600ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:40:25.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f5165416640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51600c6bd0 0x7f51601461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:40:25.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f515ffff640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5160071110 0x7f51600b0030 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f514c00d800 tx=0x7f514c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:25.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f515d7fa640 1 -- 172.21.15.12:0/1947395608 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c00b6c0 con 0x7f5160071110 2023-10-30T12:40:25.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f515d7fa640 1 -- 172.21.15.12:0/1947395608 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c00b820 con 0x7f5160071110 2023-10-30T12:40:25.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f5166e79640 1 -- 172.21.15.12:0/1947395608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5160157520 con 0x7f5160071110 2023-10-30T12:40:25.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f515d7fa640 1 -- 172.21.15.12:0/1947395608 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c00bb90 con 0x7f5160071110 2023-10-30T12:40:25.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f5166e79640 1 -- 172.21.15.12:0/1947395608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51601f29b0 con 0x7f5160071110 2023-10-30T12:40:25.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f5166e79640 1 -- 172.21.15.12:0/1947395608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516004e170 con 0x7f5160071110 2023-10-30T12:40:25.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.398+0000 7f515d7fa640 1 -- 172.21.15.12:0/1947395608 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f514c00bcf0 con 0x7f5160071110 2023-10-30T12:40:25.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.402+0000 7f515d7fa640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51440664d0 0x7f5144068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:25.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.402+0000 7f515d7fa640 1 -- 172.21.15.12:0/1947395608 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f514c0845b0 con 0x7f5160071110 2023-10-30T12:40:25.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.402+0000 7f5164c15640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51440664d0 0x7f5144068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:25.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.402+0000 7f5164c15640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51440664d0 0x7f5144068990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5154000f70 tx=0x7f5154009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:25.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.406+0000 7f515d7fa640 1 -- 172.21.15.12:0/1947395608 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f514c051820 con 0x7f5160071110 2023-10-30T12:40:25.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.598+0000 7f5166e79640 1 -- 172.21.15.12:0/1947395608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51600c0f60 con 0x7f5160071110 2023-10-30T12:40:25.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.598+0000 7f515d7fa640 1 -- 172.21.15.12:0/1947395608 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6535 (secure 0 0 0) 0x7f514c054e40 con 0x7f5160071110 2023-10-30T12:40:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:25.603 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4971,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:12.331151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:active","state_seq":53,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:25.604 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 25 2023-10-30T12:40:25.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.598+0000 7f5166e79640 1 -- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51440664d0 msgr2=0x7f5144068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:25.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.598+0000 7f5166e79640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51440664d0 0x7f5144068990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5154000f70 tx=0x7f5154009040 comp rx=0 tx=0).stop 2023-10-30T12:40:25.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.598+0000 7f5166e79640 1 -- 172.21.15.12:0/1947395608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5160071110 msgr2=0x7f51600b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:25.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.598+0000 7f5166e79640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5160071110 0x7f51600b0030 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f514c00d800 tx=0x7f514c00dcc0 comp rx=0 tx=0).stop 2023-10-30T12:40:25.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.598+0000 7f5166e79640 1 -- 172.21.15.12:0/1947395608 shutdown_connections 2023-10-30T12:40:25.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.598+0000 7f5166e79640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51440664d0 0x7f5144068990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:25.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.598+0000 7f5166e79640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51600c6bd0 0x7f51601461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:25.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.598+0000 7f5166e79640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5160071110 0x7f51600b0030 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:25.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.598+0000 7f5166e79640 1 --2- 172.21.15.12:0/1947395608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f516006f7a0 0x7f51600ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:25.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.598+0000 7f5166e79640 1 -- 172.21.15.12:0/1947395608 >> 172.21.15.12:0/1947395608 conn(0x7f51600c7910 msgr2=0x7f51600c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:25.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.598+0000 7f5166e79640 1 -- 172.21.15.12:0/1947395608 shutdown_connections 2023-10-30T12:40:25.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:25.598+0000 7f5166e79640 1 -- 172.21.15.12:0/1947395608 wait complete. 2023-10-30T12:40:25.625 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:40:27.625 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:27.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.774+0000 7fa98c291640 1 Processor -- start 2023-10-30T12:40:27.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.774+0000 7fa98c291640 1 -- start start 2023-10-30T12:40:27.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98c291640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa98406f7a0 0x7fa98406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98c291640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9840c6bd0 0x7fa984069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98c291640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa984071110 0x7fa984158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98c291640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9840700c0 con 0x7fa984071110 2023-10-30T12:40:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98c291640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98406a3d0 con 0x7fa98406f7a0 2023-10-30T12:40:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98c291640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa984066cf0 con 0x7fa9840c6bd0 2023-10-30T12:40:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98a82e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa984071110 0x7fa984158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98a82e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa984071110 0x7fa984158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46412/0 (socket says 172.21.15.12:46412) 2023-10-30T12:40:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98a82e640 1 -- 172.21.15.12:0/1763532509 learned_addr learned my addr 172.21.15.12:0/1763532509 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98982c640 1 --2- 172.21.15.12:0/1763532509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9840c6bd0 0x7fa984069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98a02d640 1 --2- 172.21.15.12:0/1763532509 >> v2:172.21.15.154:3300/0 conn(0x7fa98406f7a0 0x7fa98406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98a82e640 1 -- 172.21.15.12:0/1763532509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9840c6bd0 msgr2=0x7fa984069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98a82e640 1 --2- 172.21.15.12:0/1763532509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9840c6bd0 0x7fa984069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98a82e640 1 -- 172.21.15.12:0/1763532509 >> v2:172.21.15.154:3300/0 conn(0x7fa98406f7a0 msgr2=0x7fa98406fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98a82e640 1 --2- 172.21.15.12:0/1763532509 >> v2:172.21.15.154:3300/0 conn(0x7fa98406f7a0 0x7fa98406fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.778+0000 7fa98a82e640 1 -- 172.21.15.12:0/1763532509 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9840c70d0 con 0x7fa984071110 2023-10-30T12:40:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.782+0000 7fa98a82e640 1 --2- 172.21.15.12:0/1763532509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa984071110 0x7fa984158110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa98406c830 tx=0x7fa97c01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a74102da9d5ab17 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.782+0000 7fa98902b640 1 -- 172.21.15.12:0/1763532509 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0af400 con 0x7fa984071110 2023-10-30T12:40:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.782+0000 7fa98902b640 1 -- 172.21.15.12:0/1763532509 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c0ab050 con 0x7fa984071110 2023-10-30T12:40:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.782+0000 7fa98902b640 1 -- 172.21.15.12:0/1763532509 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0af780 con 0x7fa984071110 2023-10-30T12:40:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.782+0000 7fa98c291640 1 -- 172.21.15.12:0/1763532509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa984071110 msgr2=0x7fa984158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.782+0000 7fa98c291640 1 --2- 172.21.15.12:0/1763532509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa984071110 0x7fa984158110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa98406c830 tx=0x7fa97c01f1c0 comp rx=0 tx=0).stop 2023-10-30T12:40:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.782+0000 7fa98c291640 1 -- 172.21.15.12:0/1763532509 shutdown_connections 2023-10-30T12:40:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.782+0000 7fa98c291640 1 --2- 172.21.15.12:0/1763532509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa984071110 0x7fa984158110 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.782+0000 7fa98c291640 1 --2- 172.21.15.12:0/1763532509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9840c6bd0 0x7fa984069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.782+0000 7fa98c291640 1 --2- 172.21.15.12:0/1763532509 >> v2:172.21.15.154:3300/0 conn(0x7fa98406f7a0 0x7fa98406fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.782+0000 7fa98c291640 1 -- 172.21.15.12:0/1763532509 >> 172.21.15.12:0/1763532509 conn(0x7fa9840c7910 msgr2=0x7fa9840c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.782+0000 7fa98c291640 1 -- 172.21.15.12:0/1763532509 shutdown_connections 2023-10-30T12:40:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.782+0000 7fa98c291640 1 -- 172.21.15.12:0/1763532509 wait complete. 2023-10-30T12:40:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.782+0000 7fa98c291640 1 Processor -- start 2023-10-30T12:40:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.782+0000 7fa98c291640 1 -- start start 2023-10-30T12:40:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98c291640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa98406f7a0 0x7fa9840c1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98c291640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa984071110 0x7fa9840aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98c291640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9840c6bd0 0x7fa9840c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98c291640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa984149800 con 0x7fa984071110 2023-10-30T12:40:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98c291640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9840c1ec0 con 0x7fa9840c6bd0 2023-10-30T12:40:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98c291640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9841f25a0 con 0x7fa98406f7a0 2023-10-30T12:40:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98982c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa984071110 0x7fa9840aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98982c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa984071110 0x7fa9840aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46424/0 (socket says 172.21.15.12:46424) 2023-10-30T12:40:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98982c640 1 -- 172.21.15.12:0/709271979 learned_addr learned my addr 172.21.15.12:0/709271979 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98a02d640 1 --2- 172.21.15.12:0/709271979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa98406f7a0 0x7fa9840c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98a82e640 1 --2- 172.21.15.12:0/709271979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9840c6bd0 0x7fa9840c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98982c640 1 -- 172.21.15.12:0/709271979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa98406f7a0 msgr2=0x7fa9840c1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98982c640 1 --2- 172.21.15.12:0/709271979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa98406f7a0 0x7fa9840c1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98982c640 1 -- 172.21.15.12:0/709271979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9840c6bd0 msgr2=0x7fa9840c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98982c640 1 --2- 172.21.15.12:0/709271979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9840c6bd0 0x7fa9840c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98982c640 1 -- 172.21.15.12:0/709271979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c009b30 con 0x7fa984071110 2023-10-30T12:40:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98a82e640 1 --2- 172.21.15.12:0/709271979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9840c6bd0 0x7fa9840c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:40:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98982c640 1 --2- 172.21.15.12:0/709271979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa984071110 0x7fa9840aee30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa97400d5f0 tx=0x7fa97400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa97affd640 1 -- 172.21.15.12:0/709271979 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9740125f0 con 0x7fa984071110 2023-10-30T12:40:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa97affd640 1 -- 172.21.15.12:0/709271979 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa974012750 con 0x7fa984071110 2023-10-30T12:40:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa97affd640 1 -- 172.21.15.12:0/709271979 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974012ae0 con 0x7fa984071110 2023-10-30T12:40:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98c291640 1 -- 172.21.15.12:0/709271979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9841f2880 con 0x7fa984071110 2023-10-30T12:40:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98c291640 1 -- 172.21.15.12:0/709271979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9841f2e40 con 0x7fa984071110 2023-10-30T12:40:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa98c291640 1 -- 172.21.15.12:0/709271979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98406ad80 con 0x7fa984071110 2023-10-30T12:40:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.786+0000 7fa97affd640 1 -- 172.21.15.12:0/709271979 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa974012c40 con 0x7fa984071110 2023-10-30T12:40:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.790+0000 7fa97affd640 1 --2- 172.21.15.12:0/709271979 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa9680664a0 0x7fa968068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.790+0000 7fa97affd640 1 -- 172.21.15.12:0/709271979 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fa9740844d0 con 0x7fa984071110 2023-10-30T12:40:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.790+0000 7fa98a02d640 1 --2- 172.21.15.12:0/709271979 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa9680664a0 0x7fa968068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:27.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.790+0000 7fa98a02d640 1 --2- 172.21.15.12:0/709271979 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa9680664a0 0x7fa968068960 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa9800045f0 tx=0x7fa980008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:27.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.794+0000 7fa97affd640 1 -- 172.21.15.12:0/709271979 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa974051740 con 0x7fa984071110 2023-10-30T12:40:27.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.982+0000 7fa98c291640 1 -- 172.21.15.12:0/709271979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa98404e170 con 0x7fa984071110 2023-10-30T12:40:27.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.982+0000 7fa97affd640 1 -- 172.21.15.12:0/709271979 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6535 (secure 0 0 0) 0x7fa974007380 con 0x7fa984071110 2023-10-30T12:40:27.987 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:27.987 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4971,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:12.331151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:active","state_seq":53,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:27.988 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 25 2023-10-30T12:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.982+0000 7fa98c291640 1 -- 172.21.15.12:0/709271979 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa9680664a0 msgr2=0x7fa968068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.982+0000 7fa98c291640 1 --2- 172.21.15.12:0/709271979 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa9680664a0 0x7fa968068960 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa9800045f0 tx=0x7fa980008040 comp rx=0 tx=0).stop 2023-10-30T12:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.982+0000 7fa98c291640 1 -- 172.21.15.12:0/709271979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa984071110 msgr2=0x7fa9840aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.982+0000 7fa98c291640 1 --2- 172.21.15.12:0/709271979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa984071110 0x7fa9840aee30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa97400d5f0 tx=0x7fa97400dab0 comp rx=0 tx=0).stop 2023-10-30T12:40:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.982+0000 7fa98c291640 1 -- 172.21.15.12:0/709271979 shutdown_connections 2023-10-30T12:40:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.982+0000 7fa98c291640 1 --2- 172.21.15.12:0/709271979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9840c6bd0 0x7fa9840c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.982+0000 7fa98c291640 1 --2- 172.21.15.12:0/709271979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa984071110 0x7fa9840aee30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.982+0000 7fa98c291640 1 --2- 172.21.15.12:0/709271979 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa9680664a0 0x7fa968068960 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.982+0000 7fa98c291640 1 --2- 172.21.15.12:0/709271979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa98406f7a0 0x7fa9840c1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:27.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.982+0000 7fa98c291640 1 -- 172.21.15.12:0/709271979 >> 172.21.15.12:0/709271979 conn(0x7fa9840c7910 msgr2=0x7fa9840c5500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:27.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.982+0000 7fa98c291640 1 -- 172.21.15.12:0/709271979 shutdown_connections 2023-10-30T12:40:27.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:27.982+0000 7fa98c291640 1 -- 172.21.15.12:0/709271979 wait complete. 2023-10-30T12:40:28.008 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:40:30.009 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:30.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.158+0000 7f4eb2fd3640 1 Processor -- start 2023-10-30T12:40:30.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.158+0000 7f4eb2fd3640 1 -- start start 2023-10-30T12:40:30.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.158+0000 7f4eb2fd3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4eac0c6fe0 0x7f4eac071260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:30.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.158+0000 7f4eb2fd3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eac0700b0 con 0x7f4eac153bf0 2023-10-30T12:40:30.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.158+0000 7f4eb2fd3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eac0c6530 con 0x7f4eac0c5f60 2023-10-30T12:40:30.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.158+0000 7f4eb2fd3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eac0c6810 con 0x7f4eac0c6fe0 2023-10-30T12:40:30.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.162+0000 7f4eb0d6f640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4eac153bf0 0x7f4eac0c74a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60312/0 (socket says 172.21.15.12:60312) 2023-10-30T12:40:30.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.162+0000 7f4eb0d6f640 1 -- 172.21.15.12:0/3303798339 learned_addr learned my addr 172.21.15.12:0/3303798339 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:40:30.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.162+0000 7f4eabfff640 1 --2- 172.21.15.12:0/3303798339 >> v2:172.21.15.12:3300/0 conn(0x7f4eac0c6fe0 0x7f4eac071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:30.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.162+0000 7f4eabfff640 1 -- 172.21.15.12:0/3303798339 >> v1:172.21.15.154:6789/0 conn(0x7f4eac0c5f60 legacy=0x7f4eac0cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:30.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.162+0000 7f4eabfff640 1 -- 172.21.15.12:0/3303798339 >> v1:172.21.15.12:6789/0 conn(0x7f4eac153bf0 legacy=0x7f4eac0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:30.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.162+0000 7f4eabfff640 1 -- 172.21.15.12:0/3303798339 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac159b10 con 0x7f4eac0c6fe0 2023-10-30T12:40:30.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.162+0000 7f4eab7fe640 1 -- 172.21.15.12:0/3303798339 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596636257 0 0) 0x7f4eac0700b0 con 0x7f4eac153bf0 2023-10-30T12:40:30.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.162+0000 7f4eabfff640 1 --2- 172.21.15.12:0/3303798339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eac0c6fe0 0x7f4eac071260 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4e980097c0 tx=0x7f4e9801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6350d8f31464bd5c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:30.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.162+0000 7f4eab7fe640 1 -- 172.21.15.12:0/3303798339 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e980a8970 con 0x7f4eac0c6fe0 2023-10-30T12:40:30.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.162+0000 7f4eab7fe640 1 -- 172.21.15.12:0/3303798339 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e980a8ad0 con 0x7f4eac0c6fe0 2023-10-30T12:40:30.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.162+0000 7f4eab7fe640 1 -- 172.21.15.12:0/3303798339 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e980a8e60 con 0x7f4eac0c6fe0 2023-10-30T12:40:30.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb2fd3640 1 -- 172.21.15.12:0/3303798339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eac0c6fe0 msgr2=0x7f4eac071260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:30.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb2fd3640 1 --2- 172.21.15.12:0/3303798339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eac0c6fe0 0x7f4eac071260 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4e980097c0 tx=0x7f4e9801ec70 comp rx=0 tx=0).stop 2023-10-30T12:40:30.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb2fd3640 1 -- 172.21.15.12:0/3303798339 shutdown_connections 2023-10-30T12:40:30.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb2fd3640 1 --2- 172.21.15.12:0/3303798339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eac0c6fe0 0x7f4eac071260 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:30.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb2fd3640 1 -- 172.21.15.12:0/3303798339 >> 172.21.15.12:0/3303798339 conn(0x7f4eac06c6e0 msgr2=0x7f4eac069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:30.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb2fd3640 1 -- 172.21.15.12:0/3303798339 shutdown_connections 2023-10-30T12:40:30.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb2fd3640 1 -- 172.21.15.12:0/3303798339 wait complete. 2023-10-30T12:40:30.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb2fd3640 1 Processor -- start 2023-10-30T12:40:30.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb2fd3640 1 -- start start 2023-10-30T12:40:30.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb2fd3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4eac0c5f60 0x7f4eac0c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:30.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb2fd3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4eac0c6fe0 0x7f4eac0c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb2fd3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eac153bf0 0x7f4eac1544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb2fd3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac1efd80 con 0x7f4eac153bf0 2023-10-30T12:40:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb2fd3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac0c4950 con 0x7f4eac0c6fe0 2023-10-30T12:40:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb2fd3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac1ef820 con 0x7f4eac0c5f60 2023-10-30T12:40:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb1570640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eac153bf0 0x7f4eac1544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:30.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb1570640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eac153bf0 0x7f4eac1544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46432/0 (socket says 172.21.15.12:46432) 2023-10-30T12:40:30.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb1570640 1 -- 172.21.15.12:0/2885737710 learned_addr learned my addr 172.21.15.12:0/2885737710 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:30.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eabfff640 1 --2- 172.21.15.12:0/2885737710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4eac0c6fe0 0x7f4eac0c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:30.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.166+0000 7f4eb0d6f640 1 --2- 172.21.15.12:0/2885737710 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4eac0c5f60 0x7f4eac0c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.170+0000 7f4eb1570640 1 -- 172.21.15.12:0/2885737710 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4eac0c5f60 msgr2=0x7f4eac0c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.170+0000 7f4eb1570640 1 --2- 172.21.15.12:0/2885737710 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4eac0c5f60 0x7f4eac0c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.170+0000 7f4eb1570640 1 -- 172.21.15.12:0/2885737710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4eac0c6fe0 msgr2=0x7f4eac0c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.170+0000 7f4eb1570640 1 --2- 172.21.15.12:0/2885737710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4eac0c6fe0 0x7f4eac0c5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.170+0000 7f4eb1570640 1 -- 172.21.15.12:0/2885737710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e98009420 con 0x7f4eac153bf0 2023-10-30T12:40:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.170+0000 7f4eb0d6f640 1 --2- 172.21.15.12:0/2885737710 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4eac0c5f60 0x7f4eac0c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:40:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.170+0000 7f4eb1570640 1 --2- 172.21.15.12:0/2885737710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eac153bf0 0x7f4eac1544d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400c7d0 tx=0x7f4ea400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.170+0000 7f4ea97fa640 1 -- 172.21.15.12:0/2885737710 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4003b80 con 0x7f4eac153bf0 2023-10-30T12:40:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.170+0000 7f4ea97fa640 1 -- 172.21.15.12:0/2885737710 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea4003ce0 con 0x7f4eac153bf0 2023-10-30T12:40:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.170+0000 7f4eb2fd3640 1 -- 172.21.15.12:0/2885737710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eac1efaa0 con 0x7f4eac153bf0 2023-10-30T12:40:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.170+0000 7f4ea97fa640 1 -- 172.21.15.12:0/2885737710 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea400aa90 con 0x7f4eac153bf0 2023-10-30T12:40:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.170+0000 7f4eb2fd3640 1 -- 172.21.15.12:0/2885737710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eac0c45e0 con 0x7f4eac153bf0 2023-10-30T12:40:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.170+0000 7f4eb2fd3640 1 -- 172.21.15.12:0/2885737710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eac04e170 con 0x7f4eac153bf0 2023-10-30T12:40:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.170+0000 7f4ea97fa640 1 -- 172.21.15.12:0/2885737710 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4ea400abf0 con 0x7f4eac153bf0 2023-10-30T12:40:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.174+0000 7f4ea97fa640 1 --2- 172.21.15.12:0/2885737710 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4e900664a0 0x7f4e90068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.174+0000 7f4ea97fa640 1 -- 172.21.15.12:0/2885737710 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f4ea4083d10 con 0x7f4eac153bf0 2023-10-30T12:40:30.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.174+0000 7f4eb0d6f640 1 --2- 172.21.15.12:0/2885737710 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4e900664a0 0x7f4e90068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:30.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.174+0000 7f4eb0d6f640 1 --2- 172.21.15.12:0/2885737710 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4e900664a0 0x7f4e90068960 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0000f70 tx=0x7f4ea0007dd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:30.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.178+0000 7f4ea97fa640 1 -- 172.21.15.12:0/2885737710 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4ea4051090 con 0x7f4eac153bf0 2023-10-30T12:40:30.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.370+0000 7f4eb2fd3640 1 -- 172.21.15.12:0/2885737710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4eac0c4a90 con 0x7f4eac153bf0 2023-10-30T12:40:30.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.370+0000 7f4ea97fa640 1 -- 172.21.15.12:0/2885737710 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6535 (secure 0 0 0) 0x7f4ea40546b0 con 0x7f4eac153bf0 2023-10-30T12:40:30.375 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:30.375 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4971,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:12.331151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:active","state_seq":53,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 25 2023-10-30T12:40:30.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.370+0000 7f4eb2fd3640 1 -- 172.21.15.12:0/2885737710 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4e900664a0 msgr2=0x7f4e90068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:30.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.370+0000 7f4eb2fd3640 1 --2- 172.21.15.12:0/2885737710 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4e900664a0 0x7f4e90068960 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0000f70 tx=0x7f4ea0007dd0 comp rx=0 tx=0).stop 2023-10-30T12:40:30.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.370+0000 7f4eb2fd3640 1 -- 172.21.15.12:0/2885737710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eac153bf0 msgr2=0x7f4eac1544d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:30.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.370+0000 7f4eb2fd3640 1 --2- 172.21.15.12:0/2885737710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eac153bf0 0x7f4eac1544d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400c7d0 tx=0x7f4ea400cc90 comp rx=0 tx=0).stop 2023-10-30T12:40:30.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.370+0000 7f4eb2fd3640 1 -- 172.21.15.12:0/2885737710 shutdown_connections 2023-10-30T12:40:30.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.370+0000 7f4eb2fd3640 1 --2- 172.21.15.12:0/2885737710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eac153bf0 0x7f4eac1544d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:30.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.370+0000 7f4eb2fd3640 1 --2- 172.21.15.12:0/2885737710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4eac0c6fe0 0x7f4eac0c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:30.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.370+0000 7f4eb2fd3640 1 --2- 172.21.15.12:0/2885737710 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4e900664a0 0x7f4e90068960 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:30.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.370+0000 7f4eb2fd3640 1 --2- 172.21.15.12:0/2885737710 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4eac0c5f60 0x7f4eac0c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:30.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.370+0000 7f4eb2fd3640 1 -- 172.21.15.12:0/2885737710 >> 172.21.15.12:0/2885737710 conn(0x7f4eac06c6e0 msgr2=0x7f4eac152be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:30.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.370+0000 7f4eb2fd3640 1 -- 172.21.15.12:0/2885737710 shutdown_connections 2023-10-30T12:40:30.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:30.370+0000 7f4eb2fd3640 1 -- 172.21.15.12:0/2885737710 wait complete. 2023-10-30T12:40:30.397 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:40:32.398 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:32.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cea06640 1 Processor -- start 2023-10-30T12:40:32.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cea06640 1 -- start start 2023-10-30T12:40:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cea06640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f80c815cd50 0x7f80c80c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cea06640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c815a6c0 con 0x7f80c81538f0 2023-10-30T12:40:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cea06640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c815d300 con 0x7f80c80cefa0 2023-10-30T12:40:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cea06640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c815d5e0 con 0x7f80c815cd50 2023-10-30T12:40:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cda04640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f80c81538f0 0x7f80c8159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35642/0 (socket says 172.21.15.12:35642) 2023-10-30T12:40:32.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cda04640 1 -- 172.21.15.12:0/3234875887 learned_addr learned my addr 172.21.15.12:0/3234875887 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:40:32.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cd203640 1 --2- 172.21.15.12:0/3234875887 >> v2:172.21.15.154:3300/0 conn(0x7f80c815cd50 0x7f80c80c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:32.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271344298 0 0) 0x7f80c815a6c0 con 0x7f80c81538f0 2023-10-30T12:40:32.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80b4003530 con 0x7f80c81538f0 2023-10-30T12:40:32.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537389940 0 0) 0x7f80c815d300 con 0x7f80c80cefa0 2023-10-30T12:40:32.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80c815a6c0 con 0x7f80c80cefa0 2023-10-30T12:40:32.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4045856670 0 0) 0x7f80b4003530 con 0x7f80c81538f0 2023-10-30T12:40:32.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 >> v2:172.21.15.154:3300/0 conn(0x7f80c815cd50 msgr2=0x7f80c80c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:32.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 --2- 172.21.15.12:0/3234875887 >> v2:172.21.15.154:3300/0 conn(0x7f80c815cd50 0x7f80c80c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:32.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 >> v1:172.21.15.154:6789/0 conn(0x7f80c80cefa0 legacy=0x7f80c81640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:32.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c815d8c0 con 0x7f80c81538f0 2023-10-30T12:40:32.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f80c4001680 con 0x7f80c81538f0 2023-10-30T12:40:32.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 >> v1:172.21.15.12:6789/0 conn(0x7f80c81538f0 legacy=0x7f80c8159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:32.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 --2- 172.21.15.12:0/3234875887 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80b4003b10 0x7f80b4003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:32.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 --2- 172.21.15.12:0/3234875887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80b40047d0 0x7f80b4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:32.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 --2- 172.21.15.12:0/3234875887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b4005190 0x7f80b400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:32.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c815d5e0 con 0x7f80b40047d0 2023-10-30T12:40:32.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b400bf80 con 0x7f80b4005190 2023-10-30T12:40:32.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b400c140 con 0x7f80b4003b10 2023-10-30T12:40:32.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cd203640 1 --2- 172.21.15.12:0/3234875887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80b40047d0 0x7f80b4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:32.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cda04640 1 --2- 172.21.15.12:0/3234875887 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80b4003b10 0x7f80b4003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:32.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80ce205640 1 --2- 172.21.15.12:0/3234875887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b4005190 0x7f80b400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:32.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cd203640 1 -- 172.21.15.12:0/3234875887 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80b4003b10 msgr2=0x7f80b4003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:32.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cd203640 1 --2- 172.21.15.12:0/3234875887 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80b4003b10 0x7f80b4003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:32.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cd203640 1 -- 172.21.15.12:0/3234875887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b4005190 msgr2=0x7f80b400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.550+0000 7f80cd203640 1 --2- 172.21.15.12:0/3234875887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b4005190 0x7f80b400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.554+0000 7f80cd203640 1 -- 172.21.15.12:0/3234875887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b4003530 con 0x7f80b40047d0 2023-10-30T12:40:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.554+0000 7f80ce205640 1 -- 172.21.15.12:0/3234875887 reap_dead start 2023-10-30T12:40:32.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.554+0000 7f80cd203640 1 --2- 172.21.15.12:0/3234875887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80b40047d0 0x7f80b4004c50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009ab0 tx=0x7f80b801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4969e3de6837ca2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:32.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.554+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b80a5510 con 0x7f80b40047d0 2023-10-30T12:40:32.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.554+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b80a5670 con 0x7f80b40047d0 2023-10-30T12:40:32.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.554+0000 7f80cca02640 1 -- 172.21.15.12:0/3234875887 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b80a5a20 con 0x7f80b40047d0 2023-10-30T12:40:32.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.554+0000 7f80cea06640 1 -- 172.21.15.12:0/3234875887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80b40047d0 msgr2=0x7f80b4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:32.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.554+0000 7f80cea06640 1 --2- 172.21.15.12:0/3234875887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80b40047d0 0x7f80b4004c50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009ab0 tx=0x7f80b801ca30 comp rx=0 tx=0).stop 2023-10-30T12:40:32.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.554+0000 7f80cea06640 1 -- 172.21.15.12:0/3234875887 shutdown_connections 2023-10-30T12:40:32.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.554+0000 7f80cea06640 1 --2- 172.21.15.12:0/3234875887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80b40047d0 0x7f80b4004c50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:32.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.554+0000 7f80cea06640 1 -- 172.21.15.12:0/3234875887 >> 172.21.15.12:0/3234875887 conn(0x7f80c806a640 msgr2=0x7f80c8065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:32.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cea06640 1 -- 172.21.15.12:0/3234875887 shutdown_connections 2023-10-30T12:40:32.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cea06640 1 -- 172.21.15.12:0/3234875887 wait complete. 2023-10-30T12:40:32.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cea06640 1 Processor -- start 2023-10-30T12:40:32.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cea06640 1 -- start start 2023-10-30T12:40:32.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cea06640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80b40047d0 0x7f80c806bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:32.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cea06640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80c813f050 0x7f80c806c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cea06640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80c81f2720 0x7f80c81f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cea06640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c80b3f50 con 0x7f80b40047d0 2023-10-30T12:40:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cea06640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c8152520 con 0x7f80c813f050 2023-10-30T12:40:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cea06640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c81f3110 con 0x7f80c81f2720 2023-10-30T12:40:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cda04640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80b40047d0 0x7f80c806bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cda04640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80b40047d0 0x7f80c806bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42646/0 (socket says 172.21.15.12:42646) 2023-10-30T12:40:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cda04640 1 -- 172.21.15.12:0/3367988041 learned_addr learned my addr 172.21.15.12:0/3367988041 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cd203640 1 --2- 172.21.15.12:0/3367988041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80c813f050 0x7f80c806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80ce205640 1 --2- 172.21.15.12:0/3367988041 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80c81f2720 0x7f80c81f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cda04640 1 -- 172.21.15.12:0/3367988041 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80c81f2720 msgr2=0x7f80c81f2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:32.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cda04640 1 --2- 172.21.15.12:0/3367988041 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80c81f2720 0x7f80c81f2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:32.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cda04640 1 -- 172.21.15.12:0/3367988041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80c813f050 msgr2=0x7f80c806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:32.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cda04640 1 --2- 172.21.15.12:0/3367988041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80c813f050 0x7f80c806c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:32.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cda04640 1 -- 172.21.15.12:0/3367988041 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c81530a0 con 0x7f80b40047d0 2023-10-30T12:40:32.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80ce205640 1 --2- 172.21.15.12:0/3367988041 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80c81f2720 0x7f80c81f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:40:32.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80cda04640 1 --2- 172.21.15.12:0/3367988041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80b40047d0 0x7f80c806bd00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f80c4001060 tx=0x7f80c4011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:32.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80be7fc640 1 -- 172.21.15.12:0/3367988041 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c400fb60 con 0x7f80b40047d0 2023-10-30T12:40:32.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.558+0000 7f80be7fc640 1 -- 172.21.15.12:0/3367988041 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c400fd00 con 0x7f80b40047d0 2023-10-30T12:40:32.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.562+0000 7f80be7fc640 1 -- 172.21.15.12:0/3367988041 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4015650 con 0x7f80b40047d0 2023-10-30T12:40:32.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.562+0000 7f80cea06640 1 -- 172.21.15.12:0/3367988041 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c8151ef0 con 0x7f80b40047d0 2023-10-30T12:40:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.562+0000 7f80cea06640 1 -- 172.21.15.12:0/3367988041 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c8152a50 con 0x7f80b40047d0 2023-10-30T12:40:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.562+0000 7f80cea06640 1 -- 172.21.15.12:0/3367988041 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c804e170 con 0x7f80b40047d0 2023-10-30T12:40:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.562+0000 7f80be7fc640 1 -- 172.21.15.12:0/3367988041 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f80c40157b0 con 0x7f80b40047d0 2023-10-30T12:40:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.562+0000 7f80be7fc640 1 --2- 172.21.15.12:0/3367988041 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80ac0664a0 0x7f80ac068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.562+0000 7f80be7fc640 1 -- 172.21.15.12:0/3367988041 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f80c4087360 con 0x7f80b40047d0 2023-10-30T12:40:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.562+0000 7f80cd203640 1 --2- 172.21.15.12:0/3367988041 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80ac0664a0 0x7f80ac068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:32.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.566+0000 7f80cd203640 1 --2- 172.21.15.12:0/3367988041 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80ac0664a0 0x7f80ac068960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009ab0 tx=0x7f80b80a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:32.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.566+0000 7f80be7fc640 1 -- 172.21.15.12:0/3367988041 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f80c4054660 con 0x7f80b40047d0 2023-10-30T12:40:32.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.758+0000 7f80cea06640 1 -- 172.21.15.12:0/3367988041 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80c8159a60 con 0x7f80b40047d0 2023-10-30T12:40:32.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.758+0000 7f80be7fc640 1 -- 172.21.15.12:0/3367988041 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6535 (secure 0 0 0) 0x7f80c4057c80 con 0x7f80b40047d0 2023-10-30T12:40:32.765 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:32.765 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4971,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:12.331151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:active","state_seq":53,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:32.765 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 25 2023-10-30T12:40:32.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.762+0000 7f80cea06640 1 -- 172.21.15.12:0/3367988041 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80ac0664a0 msgr2=0x7f80ac068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:32.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.762+0000 7f80cea06640 1 --2- 172.21.15.12:0/3367988041 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80ac0664a0 0x7f80ac068960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009ab0 tx=0x7f80b80a7040 comp rx=0 tx=0).stop 2023-10-30T12:40:32.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.762+0000 7f80cea06640 1 -- 172.21.15.12:0/3367988041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80b40047d0 msgr2=0x7f80c806bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:32.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.762+0000 7f80cea06640 1 --2- 172.21.15.12:0/3367988041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80b40047d0 0x7f80c806bd00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f80c4001060 tx=0x7f80c4011de0 comp rx=0 tx=0).stop 2023-10-30T12:40:32.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.762+0000 7f80cea06640 1 -- 172.21.15.12:0/3367988041 shutdown_connections 2023-10-30T12:40:32.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.762+0000 7f80cea06640 1 --2- 172.21.15.12:0/3367988041 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80ac0664a0 0x7f80ac068960 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:32.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.762+0000 7f80cea06640 1 --2- 172.21.15.12:0/3367988041 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80c81f2720 0x7f80c81f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:32.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.762+0000 7f80cea06640 1 --2- 172.21.15.12:0/3367988041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80c813f050 0x7f80c806c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:32.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.762+0000 7f80cea06640 1 --2- 172.21.15.12:0/3367988041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80b40047d0 0x7f80c806bd00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:32.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.762+0000 7f80cea06640 1 -- 172.21.15.12:0/3367988041 >> 172.21.15.12:0/3367988041 conn(0x7f80c806a640 msgr2=0x7f80c80c5750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:32.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.762+0000 7f80cea06640 1 -- 172.21.15.12:0/3367988041 shutdown_connections 2023-10-30T12:40:32.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:32.762+0000 7f80cea06640 1 -- 172.21.15.12:0/3367988041 wait complete. 2023-10-30T12:40:32.786 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:40:34.217 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:40:34.221 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:40:34.787 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:34.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba7a446640 1 Processor -- start 2023-10-30T12:40:34.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba7a446640 1 -- start start 2023-10-30T12:40:34.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba7a446640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba7415d6f0 0x7fba7415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:34.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba7a446640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba740cb800 con 0x7fba740cf160 2023-10-30T12:40:34.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba7a446640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba740cb330 con 0x7fba741538a0 2023-10-30T12:40:34.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba7a446640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba740cad70 con 0x7fba7415d6f0 2023-10-30T12:40:34.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba789e3640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fba740cf160 0x7fba74164070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35652/0 (socket says 172.21.15.12:35652) 2023-10-30T12:40:34.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba789e3640 1 -- 172.21.15.12:0/1619515380 learned_addr learned my addr 172.21.15.12:0/1619515380 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:40:34.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba737fe640 1 --2- 172.21.15.12:0/1619515380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba7415d6f0 0x7fba7415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:34.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776196946 0 0) 0x7fba740cb800 con 0x7fba740cf160 2023-10-30T12:40:34.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba5c003530 con 0x7fba740cf160 2023-10-30T12:40:34.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121449306 0 0) 0x7fba740cb330 con 0x7fba741538a0 2023-10-30T12:40:34.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba740cb800 con 0x7fba741538a0 2023-10-30T12:40:34.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2203560813 0 0) 0x7fba5c003530 con 0x7fba740cf160 2023-10-30T12:40:34.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba7415d6f0 msgr2=0x7fba7415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:34.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 --2- 172.21.15.12:0/1619515380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba7415d6f0 0x7fba7415a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:34.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 >> v1:172.21.15.154:6789/0 conn(0x7fba741538a0 legacy=0x7fba740ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba740cb050 con 0x7fba740cf160 2023-10-30T12:40:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fba6c001a00 con 0x7fba740cf160 2023-10-30T12:40:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 >> v1:172.21.15.12:6789/0 conn(0x7fba740cf160 legacy=0x7fba74164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 --2- 172.21.15.12:0/1619515380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba5c003b40 0x7fba5c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 --2- 172.21.15.12:0/1619515380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba5c0047d0 0x7fba5c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 --2- 172.21.15.12:0/1619515380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba7415d6f0 0x7fba5c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba740cad70 con 0x7fba5c0047d0 2023-10-30T12:40:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c00ba70 con 0x7fba5c003b40 2023-10-30T12:40:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c00bc30 con 0x7fba7415d6f0 2023-10-30T12:40:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba737fe640 1 --2- 172.21.15.12:0/1619515380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba5c0047d0 0x7fba5c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:34.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba73fff640 1 --2- 172.21.15.12:0/1619515380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba5c003b40 0x7fba5c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:34.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba789e3640 1 --2- 172.21.15.12:0/1619515380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba7415d6f0 0x7fba5c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:34.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba737fe640 1 -- 172.21.15.12:0/1619515380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba7415d6f0 msgr2=0x7fba5c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:34.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba737fe640 1 --2- 172.21.15.12:0/1619515380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba7415d6f0 0x7fba5c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:34.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba737fe640 1 -- 172.21.15.12:0/1619515380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba5c003b40 msgr2=0x7fba5c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:34.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba737fe640 1 --2- 172.21.15.12:0/1619515380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba5c003b40 0x7fba5c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:34.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.938+0000 7fba737fe640 1 -- 172.21.15.12:0/1619515380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c003530 con 0x7fba5c0047d0 2023-10-30T12:40:34.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.942+0000 7fba737fe640 1 --2- 172.21.15.12:0/1619515380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba5c0047d0 0x7fba5c004c50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fba68009ae0 tx=0x7fba6801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfb84ce3fab243 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:34.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.942+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba680a5580 con 0x7fba5c0047d0 2023-10-30T12:40:34.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.942+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba680a56e0 con 0x7fba5c0047d0 2023-10-30T12:40:34.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.942+0000 7fba72ffd640 1 -- 172.21.15.12:0/1619515380 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba680a5a90 con 0x7fba5c0047d0 2023-10-30T12:40:34.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.942+0000 7fba7a446640 1 -- 172.21.15.12:0/1619515380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba5c0047d0 msgr2=0x7fba5c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:34.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.942+0000 7fba7a446640 1 --2- 172.21.15.12:0/1619515380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba5c0047d0 0x7fba5c004c50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fba68009ae0 tx=0x7fba6801c990 comp rx=0 tx=0).stop 2023-10-30T12:40:34.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.942+0000 7fba789e3640 1 -- 172.21.15.12:0/1619515380 reap_dead start 2023-10-30T12:40:34.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.942+0000 7fba7a446640 1 -- 172.21.15.12:0/1619515380 shutdown_connections 2023-10-30T12:40:34.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.942+0000 7fba7a446640 1 -- 172.21.15.12:0/1619515380 >> 172.21.15.12:0/1619515380 conn(0x7fba7406bfe0 msgr2=0x7fba7406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:34.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.942+0000 7fba7a446640 1 -- 172.21.15.12:0/1619515380 shutdown_connections 2023-10-30T12:40:34.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.942+0000 7fba7a446640 1 -- 172.21.15.12:0/1619515380 wait complete. 2023-10-30T12:40:34.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba7a446640 1 Processor -- start 2023-10-30T12:40:34.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba7a446640 1 -- start start 2023-10-30T12:40:34.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba7a446640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba5c0047d0 0x7fba740c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:34.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba7a446640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba741f2870 0x7fba741f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:34.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba7a446640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba741e38c0 0x7fba741e3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:34.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba7a446640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba741f31d0 con 0x7fba741f2870 2023-10-30T12:40:34.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba7a446640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba741e3430 con 0x7fba5c0047d0 2023-10-30T12:40:34.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba7a446640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba741e4280 con 0x7fba741e38c0 2023-10-30T12:40:34.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba741f2870 0x7fba741f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:34.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba741f2870 0x7fba741f2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42674/0 (socket says 172.21.15.12:42674) 2023-10-30T12:40:34.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba737fe640 1 -- 172.21.15.12:0/2133527222 learned_addr learned my addr 172.21.15.12:0/2133527222 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:34.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba789e3640 1 --2- 172.21.15.12:0/2133527222 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba741e38c0 0x7fba741e3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:34.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba73fff640 1 --2- 172.21.15.12:0/2133527222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba5c0047d0 0x7fba740c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:34.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba737fe640 1 -- 172.21.15.12:0/2133527222 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba741e38c0 msgr2=0x7fba741e3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:34.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba737fe640 1 --2- 172.21.15.12:0/2133527222 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba741e38c0 0x7fba741e3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:34.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba737fe640 1 -- 172.21.15.12:0/2133527222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba5c0047d0 msgr2=0x7fba740c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:34.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba737fe640 1 --2- 172.21.15.12:0/2133527222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba5c0047d0 0x7fba740c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:34.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba737fe640 1 -- 172.21.15.12:0/2133527222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba740c4fb0 con 0x7fba741f2870 2023-10-30T12:40:34.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba737fe640 1 --2- 172.21.15.12:0/2133527222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba741f2870 0x7fba741f2c90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fba68009ae0 tx=0x7fba5c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:34.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba70ff9640 1 -- 172.21.15.12:0/2133527222 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba680bc070 con 0x7fba741f2870 2023-10-30T12:40:34.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba70ff9640 1 -- 172.21.15.12:0/2133527222 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba68004020 con 0x7fba741f2870 2023-10-30T12:40:34.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba70ff9640 1 -- 172.21.15.12:0/2133527222 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba680a5580 con 0x7fba741f2870 2023-10-30T12:40:34.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba7a446640 1 -- 172.21.15.12:0/2133527222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba740b4ca0 con 0x7fba741f2870 2023-10-30T12:40:34.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.946+0000 7fba7a446640 1 -- 172.21.15.12:0/2133527222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba740b4f60 con 0x7fba741f2870 2023-10-30T12:40:34.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.950+0000 7fba7a446640 1 -- 172.21.15.12:0/2133527222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba740b5150 con 0x7fba741f2870 2023-10-30T12:40:34.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.950+0000 7fba70ff9640 1 -- 172.21.15.12:0/2133527222 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fba680b4040 con 0x7fba741f2870 2023-10-30T12:40:34.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.950+0000 7fba70ff9640 1 --2- 172.21.15.12:0/2133527222 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba600664d0 0x7fba60068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:34.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.950+0000 7fba70ff9640 1 -- 172.21.15.12:0/2133527222 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fba6811daa0 con 0x7fba741f2870 2023-10-30T12:40:34.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.950+0000 7fba73fff640 1 --2- 172.21.15.12:0/2133527222 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba600664d0 0x7fba60068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:34.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.950+0000 7fba73fff640 1 --2- 172.21.15.12:0/2133527222 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba600664d0 0x7fba60068990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fba64007af0 tx=0x7fba64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:34.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:34.954+0000 7fba70ff9640 1 -- 172.21.15.12:0/2133527222 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fba680ead10 con 0x7fba741f2870 2023-10-30T12:40:35.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:35.146+0000 7fba7a446640 1 -- 172.21.15.12:0/2133527222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba74159810 con 0x7fba741f2870 2023-10-30T12:40:35.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:35.146+0000 7fba70ff9640 1 -- 172.21.15.12:0/2133527222 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6535 (secure 0 0 0) 0x7fba680ee330 con 0x7fba741f2870 2023-10-30T12:40:35.151 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:35.151 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4971,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:12.331151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:active","state_seq":53,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:35.152 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 25 2023-10-30T12:40:35.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:35.146+0000 7fba7a446640 1 -- 172.21.15.12:0/2133527222 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba600664d0 msgr2=0x7fba60068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:35.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:35.146+0000 7fba7a446640 1 --2- 172.21.15.12:0/2133527222 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba600664d0 0x7fba60068990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fba64007af0 tx=0x7fba64009040 comp rx=0 tx=0).stop 2023-10-30T12:40:35.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:35.146+0000 7fba7a446640 1 -- 172.21.15.12:0/2133527222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba741f2870 msgr2=0x7fba741f2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:35.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:35.146+0000 7fba7a446640 1 --2- 172.21.15.12:0/2133527222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba741f2870 0x7fba741f2c90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fba68009ae0 tx=0x7fba5c005800 comp rx=0 tx=0).stop 2023-10-30T12:40:35.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:35.146+0000 7fba7a446640 1 -- 172.21.15.12:0/2133527222 shutdown_connections 2023-10-30T12:40:35.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:35.146+0000 7fba7a446640 1 --2- 172.21.15.12:0/2133527222 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba600664d0 0x7fba60068990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:35.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:35.146+0000 7fba7a446640 1 --2- 172.21.15.12:0/2133527222 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba741e38c0 0x7fba741e3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:35.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:35.146+0000 7fba7a446640 1 --2- 172.21.15.12:0/2133527222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba741f2870 0x7fba741f2c90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:35.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:35.146+0000 7fba7a446640 1 --2- 172.21.15.12:0/2133527222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba5c0047d0 0x7fba740c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:35.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:35.146+0000 7fba7a446640 1 -- 172.21.15.12:0/2133527222 >> 172.21.15.12:0/2133527222 conn(0x7fba7406bfe0 msgr2=0x7fba7415b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:35.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:35.146+0000 7fba7a446640 1 -- 172.21.15.12:0/2133527222 shutdown_connections 2023-10-30T12:40:35.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:35.146+0000 7fba7a446640 1 -- 172.21.15.12:0/2133527222 wait complete. 2023-10-30T12:40:35.172 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:40:37.173 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:37.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.322+0000 7f074a1d3640 1 Processor -- start 2023-10-30T12:40:37.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.322+0000 7f074a1d3640 1 -- start start 2023-10-30T12:40:37.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.326+0000 7f074a1d3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f07440c6120 0x7f07441574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:37.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.326+0000 7f074a1d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0744070650 0x7f0744070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:37.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.326+0000 7f074a1d3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f074406bf70 con 0x7f0744071a70 2023-10-30T12:40:37.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.326+0000 7f074a1d3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074406a4f0 con 0x7f07440c6120 2023-10-30T12:40:37.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.326+0000 7f074a1d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07440c8460 con 0x7f0744070650 2023-10-30T12:40:37.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.326+0000 7f0742ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f07440c6120 0x7f07441574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:37.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.326+0000 7f0742ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f07440c6120 0x7f07441574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42684/0 (socket says 172.21.15.12:42684) 2023-10-30T12:40:37.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.326+0000 7f0742ffd640 1 -- 172.21.15.12:0/3832610681 learned_addr learned my addr 172.21.15.12:0/3832610681 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:37.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.326+0000 7f07437fe640 1 --1- 172.21.15.12:0/3832610681 >> v1:172.21.15.12:6789/0 conn(0x7f0744071a70 0x7f074415aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35666/0 (socket says 172.21.15.12:35666) 2023-10-30T12:40:37.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.326+0000 7f0743fff640 1 --2- 172.21.15.12:0/3832610681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0744070650 0x7f0744070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:37.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.326+0000 7f0742ffd640 1 -- 172.21.15.12:0/3832610681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0744070650 msgr2=0x7f0744070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:37.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.326+0000 7f0742ffd640 1 --2- 172.21.15.12:0/3832610681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0744070650 0x7f0744070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:37.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.326+0000 7f0742ffd640 1 -- 172.21.15.12:0/3832610681 >> v1:172.21.15.12:6789/0 conn(0x7f0744071a70 legacy=0x7f074415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:37.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.326+0000 7f0742ffd640 1 -- 172.21.15.12:0/3832610681 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07440c8720 con 0x7f07440c6120 2023-10-30T12:40:37.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.326+0000 7f0742ffd640 1 --2- 172.21.15.12:0/3832610681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07440c6120 0x7f07441574f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0738009870 tx=0x7f073801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f47bb1bbf84424 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:37.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f07427fc640 1 -- 172.21.15.12:0/3832610681 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07380ac620 con 0x7f07440c6120 2023-10-30T12:40:37.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f07427fc640 1 -- 172.21.15.12:0/3832610681 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07380ac780 con 0x7f07440c6120 2023-10-30T12:40:37.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f07427fc640 1 -- 172.21.15.12:0/3832610681 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07380acb10 con 0x7f07440c6120 2023-10-30T12:40:37.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 -- 172.21.15.12:0/3832610681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07440c6120 msgr2=0x7f07441574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:37.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 --2- 172.21.15.12:0/3832610681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07440c6120 0x7f07441574f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0738009870 tx=0x7f073801c4b0 comp rx=0 tx=0).stop 2023-10-30T12:40:37.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 -- 172.21.15.12:0/3832610681 shutdown_connections 2023-10-30T12:40:37.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 --2- 172.21.15.12:0/3832610681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0744070650 0x7f0744070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:37.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 --2- 172.21.15.12:0/3832610681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07440c6120 0x7f07441574f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:37.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 -- 172.21.15.12:0/3832610681 >> 172.21.15.12:0/3832610681 conn(0x7f07440cbeb0 msgr2=0x7f07440cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:37.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 -- 172.21.15.12:0/3832610681 shutdown_connections 2023-10-30T12:40:37.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 -- 172.21.15.12:0/3832610681 wait complete. 2023-10-30T12:40:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 Processor -- start 2023-10-30T12:40:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 -- start start 2023-10-30T12:40:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0744070650 0x7f074413fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0744071a70 0x7f07440c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07440c6120 0x7f07441471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07440c4c10 con 0x7f0744070650 2023-10-30T12:40:37.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07441f0d50 con 0x7f0744071a70 2023-10-30T12:40:37.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.330+0000 7f074a1d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07441402c0 con 0x7f07440c6120 2023-10-30T12:40:37.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f07437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0744070650 0x7f074413fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:37.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f07437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0744070650 0x7f074413fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42698/0 (socket says 172.21.15.12:42698) 2023-10-30T12:40:37.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f07437fe640 1 -- 172.21.15.12:0/3001896194 learned_addr learned my addr 172.21.15.12:0/3001896194 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:37.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f0742ffd640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0744071a70 0x7f07440c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:37.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f0743fff640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07440c6120 0x7f07441471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:37.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f07437fe640 1 -- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07440c6120 msgr2=0x7f07441471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:37.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f07437fe640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07440c6120 0x7f07441471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:37.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f07437fe640 1 -- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0744071a70 msgr2=0x7f07440c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:37.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f07437fe640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0744071a70 0x7f07440c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:37.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f07437fe640 1 -- 172.21.15.12:0/3001896194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07380094d0 con 0x7f0744070650 2023-10-30T12:40:37.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f0742ffd640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0744071a70 0x7f07440c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:40:37.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f0743fff640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07440c6120 0x7f07441471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:40:37.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f07437fe640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0744070650 0x7f074413fd80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f073400c4c0 tx=0x7f073400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:37.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f072bfff640 1 -- 172.21.15.12:0/3001896194 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0734007480 con 0x7f0744070650 2023-10-30T12:40:37.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f072bfff640 1 -- 172.21.15.12:0/3001896194 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07340075e0 con 0x7f0744070650 2023-10-30T12:40:37.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f074a1d3640 1 -- 172.21.15.12:0/3001896194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07440c3fd0 con 0x7f0744070650 2023-10-30T12:40:37.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f072bfff640 1 -- 172.21.15.12:0/3001896194 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0734004a10 con 0x7f0744070650 2023-10-30T12:40:37.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f074a1d3640 1 -- 172.21.15.12:0/3001896194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07440c4270 con 0x7f0744070650 2023-10-30T12:40:37.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f074a1d3640 1 -- 172.21.15.12:0/3001896194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074404e170 con 0x7f0744070650 2023-10-30T12:40:37.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f072bfff640 1 -- 172.21.15.12:0/3001896194 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0734004b70 con 0x7f0744070650 2023-10-30T12:40:37.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.334+0000 7f072bfff640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f07300664a0 0x7f0730068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:37.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.338+0000 7f072bfff640 1 -- 172.21.15.12:0/3001896194 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f0734083530 con 0x7f0744070650 2023-10-30T12:40:37.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.338+0000 7f0742ffd640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f07300664a0 0x7f0730068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:37.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.338+0000 7f0742ffd640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f07300664a0 0x7f0730068960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f07380ae000 tx=0x7f07380a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:37.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.342+0000 7f072bfff640 1 -- 172.21.15.12:0/3001896194 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f07340507a0 con 0x7f0744070650 2023-10-30T12:40:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.534+0000 7f074a1d3640 1 -- 172.21.15.12:0/3001896194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f074406bbd0 con 0x7f0744070650 2023-10-30T12:40:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.534+0000 7f072bfff640 1 -- 172.21.15.12:0/3001896194 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6535 (secure 0 0 0) 0x7f0734053dc0 con 0x7f0744070650 2023-10-30T12:40:37.539 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:37.539 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4971,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:12.331151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:active","state_seq":53,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 25 2023-10-30T12:40:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.534+0000 7f074a1d3640 1 -- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f07300664a0 msgr2=0x7f0730068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.534+0000 7f074a1d3640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f07300664a0 0x7f0730068960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f07380ae000 tx=0x7f07380a7000 comp rx=0 tx=0).stop 2023-10-30T12:40:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.534+0000 7f074a1d3640 1 -- 172.21.15.12:0/3001896194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0744070650 msgr2=0x7f074413fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.534+0000 7f074a1d3640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0744070650 0x7f074413fd80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f073400c4c0 tx=0x7f073400c980 comp rx=0 tx=0).stop 2023-10-30T12:40:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.534+0000 7f074a1d3640 1 -- 172.21.15.12:0/3001896194 shutdown_connections 2023-10-30T12:40:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.534+0000 7f074a1d3640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f07300664a0 0x7f0730068960 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.534+0000 7f074a1d3640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07440c6120 0x7f07441471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.534+0000 7f074a1d3640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0744071a70 0x7f07440c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.534+0000 7f074a1d3640 1 --2- 172.21.15.12:0/3001896194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0744070650 0x7f074413fd80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.534+0000 7f074a1d3640 1 -- 172.21.15.12:0/3001896194 >> 172.21.15.12:0/3001896194 conn(0x7f07440cbeb0 msgr2=0x7f074414fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.534+0000 7f074a1d3640 1 -- 172.21.15.12:0/3001896194 shutdown_connections 2023-10-30T12:40:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:37.534+0000 7f074a1d3640 1 -- 172.21.15.12:0/3001896194 wait complete. 2023-10-30T12:40:37.560 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:40:39.561 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.710+0000 7f66f4b94640 1 Processor -- start 2023-10-30T12:40:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.710+0000 7f66f4b94640 1 -- start start 2023-10-30T12:40:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66f4b94640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f66f00c6fe0 0x7f66f0071260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66f4b94640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f00700b0 con 0x7f66f0153bf0 2023-10-30T12:40:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66f4b94640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f00c6530 con 0x7f66f00c5f60 2023-10-30T12:40:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66f4b94640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f00c6810 con 0x7f66f00c6fe0 2023-10-30T12:40:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66ee59c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f66f0153bf0 0x7f66f00c74a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35672/0 (socket says 172.21.15.12:35672) 2023-10-30T12:40:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66ee59c640 1 -- 172.21.15.12:0/2319481090 learned_addr learned my addr 172.21.15.12:0/2319481090 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:40:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66e7fff640 1 --2- 172.21.15.12:0/2319481090 >> v2:172.21.15.154:3300/0 conn(0x7f66f00c6fe0 0x7f66f0071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66edd9b640 1 -- 172.21.15.12:0/2319481090 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287037016 0 0) 0x7f66f00700b0 con 0x7f66f0153bf0 2023-10-30T12:40:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66edd9b640 1 -- 172.21.15.12:0/2319481090 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66dc003530 con 0x7f66f0153bf0 2023-10-30T12:40:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66edd9b640 1 -- 172.21.15.12:0/2319481090 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3191117117 0 0) 0x7f66dc003530 con 0x7f66f0153bf0 2023-10-30T12:40:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66edd9b640 1 -- 172.21.15.12:0/2319481090 >> v2:172.21.15.154:3300/0 conn(0x7f66f00c6fe0 msgr2=0x7f66f0071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66edd9b640 1 --2- 172.21.15.12:0/2319481090 >> v2:172.21.15.154:3300/0 conn(0x7f66f00c6fe0 0x7f66f0071260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66edd9b640 1 -- 172.21.15.12:0/2319481090 >> v1:172.21.15.154:6789/0 conn(0x7f66f00c5f60 legacy=0x7f66f00cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66edd9b640 1 -- 172.21.15.12:0/2319481090 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f0159b10 con 0x7f66f0153bf0 2023-10-30T12:40:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66edd9b640 1 -- 172.21.15.12:0/2319481090 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f66e0001680 con 0x7f66f0153bf0 2023-10-30T12:40:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66edd9b640 1 -- 172.21.15.12:0/2319481090 >> v1:172.21.15.12:6789/0 conn(0x7f66f0153bf0 legacy=0x7f66f00c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66edd9b640 1 --2- 172.21.15.12:0/2319481090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66dc003980 0x7f66dc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66edd9b640 1 --2- 172.21.15.12:0/2319481090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66dc0047d0 0x7f66dc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66edd9b640 1 --2- 172.21.15.12:0/2319481090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66dc005190 0x7f66dc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66edd9b640 1 -- 172.21.15.12:0/2319481090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f00c6810 con 0x7f66dc005190 2023-10-30T12:40:39.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66edd9b640 1 -- 172.21.15.12:0/2319481090 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc00c040 con 0x7f66dc003980 2023-10-30T12:40:39.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66edd9b640 1 -- 172.21.15.12:0/2319481090 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc00c200 con 0x7f66dc0047d0 2023-10-30T12:40:39.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66eed9d640 1 --2- 172.21.15.12:0/2319481090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66dc005190 0x7f66dc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:39.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66e7fff640 1 --2- 172.21.15.12:0/2319481090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66dc0047d0 0x7f66dc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:39.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66ee59c640 1 --2- 172.21.15.12:0/2319481090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66dc003980 0x7f66dc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:39.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66eed9d640 1 -- 172.21.15.12:0/2319481090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66dc0047d0 msgr2=0x7f66dc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:39.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66eed9d640 1 --2- 172.21.15.12:0/2319481090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66dc0047d0 0x7f66dc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:39.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66eed9d640 1 -- 172.21.15.12:0/2319481090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66dc003980 msgr2=0x7f66dc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:39.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66eed9d640 1 --2- 172.21.15.12:0/2319481090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66dc003980 0x7f66dc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:39.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.714+0000 7f66eed9d640 1 -- 172.21.15.12:0/2319481090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc003530 con 0x7f66dc005190 2023-10-30T12:40:39.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.718+0000 7f66eed9d640 1 -- 172.21.15.12:0/2319481090 reap_dead start 2023-10-30T12:40:39.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.718+0000 7f66eed9d640 1 --2- 172.21.15.12:0/2319481090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66dc005190 0x7f66dc00b990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f66e8001e70 tx=0x7f66e801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83bb2f04dd59530 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:39.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.718+0000 7f66edd9b640 1 -- 172.21.15.12:0/2319481090 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8003b80 con 0x7f66dc005190 2023-10-30T12:40:39.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.718+0000 7f66edd9b640 1 -- 172.21.15.12:0/2319481090 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e8003d20 con 0x7f66dc005190 2023-10-30T12:40:39.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.718+0000 7f66edd9b640 1 -- 172.21.15.12:0/2319481090 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8005260 con 0x7f66dc005190 2023-10-30T12:40:39.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.718+0000 7f66f4b94640 1 -- 172.21.15.12:0/2319481090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66dc005190 msgr2=0x7f66dc00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:39.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.718+0000 7f66f4b94640 1 --2- 172.21.15.12:0/2319481090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66dc005190 0x7f66dc00b990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f66e8001e70 tx=0x7f66e801c760 comp rx=0 tx=0).stop 2023-10-30T12:40:39.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.718+0000 7f66f4b94640 1 -- 172.21.15.12:0/2319481090 shutdown_connections 2023-10-30T12:40:39.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.718+0000 7f66f4b94640 1 --2- 172.21.15.12:0/2319481090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66dc005190 0x7f66dc00b990 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:39.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.718+0000 7f66f4b94640 1 -- 172.21.15.12:0/2319481090 >> 172.21.15.12:0/2319481090 conn(0x7f66f006c6e0 msgr2=0x7f66f0069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:39.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.718+0000 7f66f4b94640 1 -- 172.21.15.12:0/2319481090 shutdown_connections 2023-10-30T12:40:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.718+0000 7f66f4b94640 1 -- 172.21.15.12:0/2319481090 wait complete. 2023-10-30T12:40:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66f4b94640 1 Processor -- start 2023-10-30T12:40:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66f4b94640 1 -- start start 2023-10-30T12:40:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66f4b94640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66dc005190 0x7f66f0159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66f4b94640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66f0157770 0x7f66f00c4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:39.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66f4b94640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66f01568b0 0x7f66f0156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:39.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66f4b94640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01ef570 con 0x7f66dc005190 2023-10-30T12:40:39.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66f4b94640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01597c0 con 0x7f66f01568b0 2023-10-30T12:40:39.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66f4b94640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01ee960 con 0x7f66f0157770 2023-10-30T12:40:39.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66ee59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66dc005190 0x7f66f0159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:39.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66ee59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66dc005190 0x7f66f0159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42722/0 (socket says 172.21.15.12:42722) 2023-10-30T12:40:39.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66ee59c640 1 -- 172.21.15.12:0/3915992745 learned_addr learned my addr 172.21.15.12:0/3915992745 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:39.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66e7fff640 1 --2- 172.21.15.12:0/3915992745 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66f0157770 0x7f66f00c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:39.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66eed9d640 1 --2- 172.21.15.12:0/3915992745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66f01568b0 0x7f66f0156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:39.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66ee59c640 1 -- 172.21.15.12:0/3915992745 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66f0157770 msgr2=0x7f66f00c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:39.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66ee59c640 1 --2- 172.21.15.12:0/3915992745 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66f0157770 0x7f66f00c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:39.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66ee59c640 1 -- 172.21.15.12:0/3915992745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66f01568b0 msgr2=0x7f66f0156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:39.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66ee59c640 1 --2- 172.21.15.12:0/3915992745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66f01568b0 0x7f66f0156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:39.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66ee59c640 1 -- 172.21.15.12:0/3915992745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f01eec40 con 0x7f66dc005190 2023-10-30T12:40:39.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66ee59c640 1 --2- 172.21.15.12:0/3915992745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66dc005190 0x7f66f0159140 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f66e00072c0 tx=0x7f66e00076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66e6ffd640 1 -- 172.21.15.12:0/3915992745 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e000f940 con 0x7f66dc005190 2023-10-30T12:40:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66e6ffd640 1 -- 172.21.15.12:0/3915992745 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e000faa0 con 0x7f66dc005190 2023-10-30T12:40:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66e6ffd640 1 -- 172.21.15.12:0/3915992745 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e000fe50 con 0x7f66dc005190 2023-10-30T12:40:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66f4b94640 1 -- 172.21.15.12:0/3915992745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f01eef20 con 0x7f66dc005190 2023-10-30T12:40:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.722+0000 7f66f4b94640 1 -- 172.21.15.12:0/3915992745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f0151ea0 con 0x7f66dc005190 2023-10-30T12:40:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.726+0000 7f66f4b94640 1 -- 172.21.15.12:0/3915992745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f004e170 con 0x7f66dc005190 2023-10-30T12:40:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.726+0000 7f66e6ffd640 1 -- 172.21.15.12:0/3915992745 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f66e001d460 con 0x7f66dc005190 2023-10-30T12:40:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.726+0000 7f66e6ffd640 1 --2- 172.21.15.12:0/3915992745 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66d00664a0 0x7f66d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.726+0000 7f66e6ffd640 1 -- 172.21.15.12:0/3915992745 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f66e0084150 con 0x7f66dc005190 2023-10-30T12:40:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.726+0000 7f66e7fff640 1 --2- 172.21.15.12:0/3915992745 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66d00664a0 0x7f66d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.726+0000 7f66e7fff640 1 --2- 172.21.15.12:0/3915992745 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66d00664a0 0x7f66d0068960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f66f0152d60 tx=0x7f66d8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:39.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.730+0000 7f66e6ffd640 1 -- 172.21.15.12:0/3915992745 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f66e0051450 con 0x7f66dc005190 2023-10-30T12:40:39.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.918+0000 7f66f4b94640 1 -- 172.21.15.12:0/3915992745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66f0159dc0 con 0x7f66dc005190 2023-10-30T12:40:39.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.918+0000 7f66e6ffd640 1 -- 172.21.15.12:0/3915992745 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6535 (secure 0 0 0) 0x7f66e0054a70 con 0x7f66dc005190 2023-10-30T12:40:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:39.925 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4971,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:12.331151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"a","rank":2,"incarnation":16,"state":"up:active","state_seq":53,"addr":"172.21.15.12:6835/2676506149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2676506149},{"type":"v1","addr":"172.21.15.12:6835","nonce":2676506149}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:39.925 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 25 2023-10-30T12:40:39.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.922+0000 7f66f4b94640 1 -- 172.21.15.12:0/3915992745 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66d00664a0 msgr2=0x7f66d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:39.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.922+0000 7f66f4b94640 1 --2- 172.21.15.12:0/3915992745 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66d00664a0 0x7f66d0068960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f66f0152d60 tx=0x7f66d8009190 comp rx=0 tx=0).stop 2023-10-30T12:40:39.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.922+0000 7f66f4b94640 1 -- 172.21.15.12:0/3915992745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66dc005190 msgr2=0x7f66f0159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:39.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.922+0000 7f66f4b94640 1 --2- 172.21.15.12:0/3915992745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66dc005190 0x7f66f0159140 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f66e00072c0 tx=0x7f66e00076c0 comp rx=0 tx=0).stop 2023-10-30T12:40:39.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.922+0000 7f66f4b94640 1 -- 172.21.15.12:0/3915992745 shutdown_connections 2023-10-30T12:40:39.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.922+0000 7f66f4b94640 1 --2- 172.21.15.12:0/3915992745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66f01568b0 0x7f66f0156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:39.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.922+0000 7f66f4b94640 1 --2- 172.21.15.12:0/3915992745 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f66d00664a0 0x7f66d0068960 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:39.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.922+0000 7f66f4b94640 1 --2- 172.21.15.12:0/3915992745 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66f0157770 0x7f66f00c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:39.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.922+0000 7f66f4b94640 1 --2- 172.21.15.12:0/3915992745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66dc005190 0x7f66f0159140 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:39.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.922+0000 7f66f4b94640 1 -- 172.21.15.12:0/3915992745 >> 172.21.15.12:0/3915992745 conn(0x7f66f006c6e0 msgr2=0x7f66f0152d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:39.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.922+0000 7f66f4b94640 1 -- 172.21.15.12:0/3915992745 shutdown_connections 2023-10-30T12:40:39.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:39.922+0000 7f66f4b94640 1 -- 172.21.15.12:0/3915992745 wait complete. 2023-10-30T12:40:39.946 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:40:41.947 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:42.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.098+0000 7efe18279640 1 Processor -- start 2023-10-30T12:40:42.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.098+0000 7efe18279640 1 -- start start 2023-10-30T12:40:42.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe18279640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe10151800 0x7efe1006e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:42.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe18279640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efe100c1ee0 0x7efe100c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:42.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe18279640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1006e090 con 0x7efe100c9230 2023-10-30T12:40:42.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe18279640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe1006dc70 con 0x7efe100c1ee0 2023-10-30T12:40:42.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe18279640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe100c8a50 con 0x7efe10151800 2023-10-30T12:40:42.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe16816640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7efe100c9230 0x7efe100cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39230/0 (socket says 172.21.15.12:39230) 2023-10-30T12:40:42.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe16816640 1 -- 172.21.15.12:0/3183724993 learned_addr learned my addr 172.21.15.12:0/3183724993 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:40:42.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15814640 1 --2- 172.21.15.12:0/3183724993 >> v2:172.21.15.154:3300/0 conn(0x7efe100c1ee0 0x7efe100c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:42.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe16015640 1 --2- 172.21.15.12:0/3183724993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe10151800 0x7efe1006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:42.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 -- 172.21.15.12:0/3183724993 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179936097 0 0) 0x7efe1006e090 con 0x7efe100c9230 2023-10-30T12:40:42.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 -- 172.21.15.12:0/3183724993 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdfc003530 con 0x7efe100c9230 2023-10-30T12:40:42.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 -- 172.21.15.12:0/3183724993 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 688531798 0 0) 0x7efdfc003530 con 0x7efe100c9230 2023-10-30T12:40:42.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 -- 172.21.15.12:0/3183724993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe10151800 msgr2=0x7efe1006e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:42.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 --2- 172.21.15.12:0/3183724993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe10151800 0x7efe1006e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 -- 172.21.15.12:0/3183724993 >> v2:172.21.15.154:3300/0 conn(0x7efe100c1ee0 msgr2=0x7efe100c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:42.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 --2- 172.21.15.12:0/3183724993 >> v2:172.21.15.154:3300/0 conn(0x7efe100c1ee0 0x7efe100c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 -- 172.21.15.12:0/3183724993 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe100c3b10 con 0x7efe100c9230 2023-10-30T12:40:42.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 -- 172.21.15.12:0/3183724993 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7efe08001a20 con 0x7efe100c9230 2023-10-30T12:40:42.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 -- 172.21.15.12:0/3183724993 >> v1:172.21.15.12:6789/0 conn(0x7efe100c9230 legacy=0x7efe100cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:42.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 --2- 172.21.15.12:0/3183724993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdfc003980 0x7efdfc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:42.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 --2- 172.21.15.12:0/3183724993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe10151800 0x7efdfc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:42.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 --2- 172.21.15.12:0/3183724993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdfc004d10 0x7efdfc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:42.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 -- 172.21.15.12:0/3183724993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1006dc70 con 0x7efdfc004d10 2023-10-30T12:40:42.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 -- 172.21.15.12:0/3183724993 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe100c8a50 con 0x7efdfc003980 2023-10-30T12:40:42.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15013640 1 -- 172.21.15.12:0/3183724993 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc00b990 con 0x7efe10151800 2023-10-30T12:40:42.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15814640 1 --2- 172.21.15.12:0/3183724993 >> v2:172.21.15.154:3300/0 conn(0x7efe100c1ee0 0x7efe100c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:40:42.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe16816640 1 --2- 172.21.15.12:0/3183724993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdfc004d10 0x7efdfc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:42.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe16015640 1 --2- 172.21.15.12:0/3183724993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdfc003980 0x7efdfc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:42.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe15814640 1 --2- 172.21.15.12:0/3183724993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe10151800 0x7efdfc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:42.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe16816640 1 -- 172.21.15.12:0/3183724993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe10151800 msgr2=0x7efdfc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:42.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe16816640 1 --2- 172.21.15.12:0/3183724993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe10151800 0x7efdfc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe16816640 1 -- 172.21.15.12:0/3183724993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdfc003980 msgr2=0x7efdfc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:42.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe16816640 1 --2- 172.21.15.12:0/3183724993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdfc003980 0x7efdfc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.102+0000 7efe16816640 1 -- 172.21.15.12:0/3183724993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc003530 con 0x7efdfc004d10 2023-10-30T12:40:42.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.106+0000 7efe16816640 1 --2- 172.21.15.12:0/3183724993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdfc004d10 0x7efdfc005190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efe0800a850 tx=0x7efe0801d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d22389ceede1a83 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:42.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.106+0000 7efe15013640 1 -- 172.21.15.12:0/3183724993 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe080ad630 con 0x7efdfc004d10 2023-10-30T12:40:42.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.106+0000 7efe15013640 1 -- 172.21.15.12:0/3183724993 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe080a8070 con 0x7efdfc004d10 2023-10-30T12:40:42.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.106+0000 7efe15013640 1 -- 172.21.15.12:0/3183724993 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe080ad9d0 con 0x7efdfc004d10 2023-10-30T12:40:42.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.106+0000 7efe18279640 1 -- 172.21.15.12:0/3183724993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdfc004d10 msgr2=0x7efdfc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:42.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.106+0000 7efe18279640 1 --2- 172.21.15.12:0/3183724993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdfc004d10 0x7efdfc005190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efe0800a850 tx=0x7efe0801d340 comp rx=0 tx=0).stop 2023-10-30T12:40:42.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.106+0000 7efe16816640 1 -- 172.21.15.12:0/3183724993 reap_dead start 2023-10-30T12:40:42.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.106+0000 7efe18279640 1 -- 172.21.15.12:0/3183724993 shutdown_connections 2023-10-30T12:40:42.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.106+0000 7efe18279640 1 -- 172.21.15.12:0/3183724993 >> 172.21.15.12:0/3183724993 conn(0x7efe10071ea0 msgr2=0x7efe10065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:42.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.106+0000 7efe18279640 1 -- 172.21.15.12:0/3183724993 shutdown_connections 2023-10-30T12:40:42.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.106+0000 7efe18279640 1 -- 172.21.15.12:0/3183724993 wait complete. 2023-10-30T12:40:42.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe18279640 1 Processor -- start 2023-10-30T12:40:42.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe18279640 1 -- start start 2023-10-30T12:40:42.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe18279640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdfc004d10 0x7efe101ef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:42.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe18279640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe10158610 0x7efe100cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:42.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe18279640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe101f37e0 0x7efe101476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:42.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe18279640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe101f33c0 con 0x7efdfc004d10 2023-10-30T12:40:42.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe18279640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe100c5ff0 con 0x7efe10158610 2023-10-30T12:40:42.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe18279640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe100c62d0 con 0x7efe101f37e0 2023-10-30T12:40:42.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe16015640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdfc004d10 0x7efe101ef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:42.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe16015640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdfc004d10 0x7efe101ef5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60150/0 (socket says 172.21.15.12:60150) 2023-10-30T12:40:42.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe16015640 1 -- 172.21.15.12:0/1293614256 learned_addr learned my addr 172.21.15.12:0/1293614256 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:42.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe15814640 1 --2- 172.21.15.12:0/1293614256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe10158610 0x7efe100cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:42.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe16816640 1 --2- 172.21.15.12:0/1293614256 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe101f37e0 0x7efe101476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:42.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe16015640 1 -- 172.21.15.12:0/1293614256 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe101f37e0 msgr2=0x7efe101476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:42.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe16015640 1 --2- 172.21.15.12:0/1293614256 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe101f37e0 0x7efe101476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe16015640 1 -- 172.21.15.12:0/1293614256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe10158610 msgr2=0x7efe100cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:42.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe16015640 1 --2- 172.21.15.12:0/1293614256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe10158610 0x7efe100cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe16015640 1 -- 172.21.15.12:0/1293614256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe101f2d00 con 0x7efdfc004d10 2023-10-30T12:40:42.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe16816640 1 --2- 172.21.15.12:0/1293614256 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe101f37e0 0x7efe101476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:40:42.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe16015640 1 --2- 172.21.15.12:0/1293614256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdfc004d10 0x7efe101ef5f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efe0c011960 tx=0x7efe0c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:42.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe06ffd640 1 -- 172.21.15.12:0/1293614256 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c00faf0 con 0x7efdfc004d10 2023-10-30T12:40:42.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe06ffd640 1 -- 172.21.15.12:0/1293614256 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0c00fc50 con 0x7efdfc004d10 2023-10-30T12:40:42.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe06ffd640 1 -- 172.21.15.12:0/1293614256 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c014440 con 0x7efdfc004d10 2023-10-30T12:40:42.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe18279640 1 -- 172.21.15.12:0/1293614256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe101f2fe0 con 0x7efdfc004d10 2023-10-30T12:40:42.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe18279640 1 -- 172.21.15.12:0/1293614256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe100cc690 con 0x7efdfc004d10 2023-10-30T12:40:42.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.110+0000 7efe18279640 1 -- 172.21.15.12:0/1293614256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe101f2b60 con 0x7efdfc004d10 2023-10-30T12:40:42.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.114+0000 7efe06ffd640 1 -- 172.21.15.12:0/1293614256 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efe0c0145a0 con 0x7efdfc004d10 2023-10-30T12:40:42.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.114+0000 7efe06ffd640 1 --2- 172.21.15.12:0/1293614256 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efdf40664a0 0x7efdf4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:42.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.114+0000 7efe06ffd640 1 -- 172.21.15.12:0/1293614256 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5997+0+0 (secure 0 0 0) 0x7efe0c0848e0 con 0x7efdfc004d10 2023-10-30T12:40:42.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.114+0000 7efe15814640 1 --2- 172.21.15.12:0/1293614256 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efdf40664a0 0x7efdf4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:42.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.114+0000 7efe15814640 1 --2- 172.21.15.12:0/1293614256 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efdf40664a0 0x7efdf4068960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe100c7480 tx=0x7efe00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:42.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.118+0000 7efe06ffd640 1 -- 172.21.15.12:0/1293614256 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efe101f2b60 con 0x7efdfc004d10 2023-10-30T12:40:42.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.314+0000 7efe18279640 1 -- 172.21.15.12:0/1293614256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe101f2b60 con 0x7efdfc004d10 2023-10-30T12:40:42.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.314+0000 7efe06ffd640 1 -- 172.21.15.12:0/1293614256 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5740 (secure 0 0 0) 0x7efe101f2b60 con 0x7efdfc004d10 2023-10-30T12:40:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:42.319 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:40.195798+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 26 2023-10-30T12:40:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.314+0000 7efe18279640 1 -- 172.21.15.12:0/1293614256 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efdf40664a0 msgr2=0x7efdf4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.314+0000 7efe18279640 1 --2- 172.21.15.12:0/1293614256 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efdf40664a0 0x7efdf4068960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe100c7480 tx=0x7efe00008040 comp rx=0 tx=0).stop 2023-10-30T12:40:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.314+0000 7efe18279640 1 -- 172.21.15.12:0/1293614256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdfc004d10 msgr2=0x7efe101ef5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.314+0000 7efe18279640 1 --2- 172.21.15.12:0/1293614256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdfc004d10 0x7efe101ef5f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efe0c011960 tx=0x7efe0c011e20 comp rx=0 tx=0).stop 2023-10-30T12:40:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.314+0000 7efe18279640 1 -- 172.21.15.12:0/1293614256 shutdown_connections 2023-10-30T12:40:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.314+0000 7efe18279640 1 --2- 172.21.15.12:0/1293614256 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efdf40664a0 0x7efdf4068960 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.314+0000 7efe18279640 1 --2- 172.21.15.12:0/1293614256 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe101f37e0 0x7efe101476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.314+0000 7efe18279640 1 --2- 172.21.15.12:0/1293614256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe10158610 0x7efe100cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.314+0000 7efe18279640 1 --2- 172.21.15.12:0/1293614256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdfc004d10 0x7efe101ef5f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.314+0000 7efe18279640 1 -- 172.21.15.12:0/1293614256 >> 172.21.15.12:0/1293614256 conn(0x7efe10071ea0 msgr2=0x7efe1006ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.314+0000 7efe18279640 1 -- 172.21.15.12:0/1293614256 shutdown_connections 2023-10-30T12:40:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.314+0000 7efe18279640 1 -- 172.21.15.12:0/1293614256 wait complete. 2023-10-30T12:40:42.339 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2023-10-30T12:40:42.340 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:40:42.340 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:42.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f7701bc5640 1 Processor -- start 2023-10-30T12:40:42.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f7701bc5640 1 -- start start 2023-10-30T12:40:42.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f7701bc5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f76fc15d6f0 0x7f76fc15a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:42.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f7701bc5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76fc0cf160 0x7f76fc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:42.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f7701bc5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76fc0cb800 con 0x7f76fc1538a0 2023-10-30T12:40:42.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f7701bc5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76fc0cb330 con 0x7f76fc15d6f0 2023-10-30T12:40:42.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f7701bc5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc0cad70 con 0x7f76fc0cf160 2023-10-30T12:40:42.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f76fbfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76fc0cf160 0x7f76fc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:42.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f76faffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f76fc15d6f0 0x7f76fc15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:42.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f76faffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f76fc15d6f0 0x7f76fc15a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:48592/0 (socket says 172.21.15.12:48592) 2023-10-30T12:40:42.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f76fbfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76fc0cf160 0x7f76fc0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:50868/0 (socket says 172.21.15.12:50868) 2023-10-30T12:40:42.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f76fb7fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f76fc1538a0 0x7f76fc0ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:38942/0 (socket says 172.21.15.12:38942) 2023-10-30T12:40:42.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f76fbfff640 1 -- 172.21.15.12:0/2833505328 learned_addr learned my addr 172.21.15.12:0/2833505328 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:42.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f76fbfff640 1 -- 172.21.15.12:0/2833505328 >> v2:172.21.15.154:3300/0 conn(0x7f76fc15d6f0 msgr2=0x7f76fc15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f76fbfff640 1 --2- 172.21.15.12:0/2833505328 >> v2:172.21.15.154:3300/0 conn(0x7f76fc15d6f0 0x7f76fc15a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f76fbfff640 1 -- 172.21.15.12:0/2833505328 >> v1:172.21.15.154:6789/0 conn(0x7f76fc1538a0 legacy=0x7f76fc0ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f76fbfff640 1 -- 172.21.15.12:0/2833505328 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc0cb050 con 0x7f76fc0cf160 2023-10-30T12:40:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.478+0000 7f76faffd640 1 --2- 172.21.15.12:0/2833505328 >> v2:172.21.15.154:3300/0 conn(0x7f76fc15d6f0 0x7f76fc15a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:40:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.482+0000 7f76fbfff640 1 --2- 172.21.15.12:0/2833505328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76fc0cf160 0x7f76fc0cf590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009e50 tx=0x7f76f401f1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48b9d4ff9db7a18d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.482+0000 7f76fa7fc640 1 -- 172.21.15.12:0/2833505328 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f40af400 con 0x7f76fc0cf160 2023-10-30T12:40:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.482+0000 7f76fa7fc640 1 -- 172.21.15.12:0/2833505328 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f40ab050 con 0x7f76fc0cf160 2023-10-30T12:40:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.482+0000 7f76fa7fc640 1 -- 172.21.15.12:0/2833505328 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f40af780 con 0x7f76fc0cf160 2023-10-30T12:40:42.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.482+0000 7f7701bc5640 1 -- 172.21.15.12:0/2833505328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76fc0cf160 msgr2=0x7f76fc0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.482+0000 7f7701bc5640 1 --2- 172.21.15.12:0/2833505328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76fc0cf160 0x7f76fc0cf590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009e50 tx=0x7f76f401f1f0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.482+0000 7f7701bc5640 1 -- 172.21.15.12:0/2833505328 shutdown_connections 2023-10-30T12:40:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.482+0000 7f7701bc5640 1 --2- 172.21.15.12:0/2833505328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76fc0cf160 0x7f76fc0cf590 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.482+0000 7f7701bc5640 1 --2- 172.21.15.12:0/2833505328 >> v2:172.21.15.154:3300/0 conn(0x7f76fc15d6f0 0x7f76fc15a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.482+0000 7f7701bc5640 1 -- 172.21.15.12:0/2833505328 >> 172.21.15.12:0/2833505328 conn(0x7f76fc06bfe0 msgr2=0x7f76fc06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:42.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.482+0000 7f7701bc5640 1 -- 172.21.15.12:0/2833505328 shutdown_connections 2023-10-30T12:40:42.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.482+0000 7f7701bc5640 1 -- 172.21.15.12:0/2833505328 wait complete. 2023-10-30T12:40:42.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f7701bc5640 1 Processor -- start 2023-10-30T12:40:42.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f7701bc5640 1 -- start start 2023-10-30T12:40:42.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f7701bc5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76fc0cf160 0x7f76fc14e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:42.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f7701bc5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76fc1538a0 0x7f76fc0b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:42.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f7701bc5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76fc15d6f0 0x7f76fc0cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:42.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f7701bc5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc1efd70 con 0x7f76fc1538a0 2023-10-30T12:40:42.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f7701bc5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc1f0260 con 0x7f76fc0cf160 2023-10-30T12:40:42.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f7701bc5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc0ce7b0 con 0x7f76fc15d6f0 2023-10-30T12:40:42.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f76faffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76fc1538a0 0x7f76fc0b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:42.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f76faffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76fc1538a0 0x7f76fc0b49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60156/0 (socket says 172.21.15.12:60156) 2023-10-30T12:40:42.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f76faffd640 1 -- 172.21.15.12:0/1558402998 learned_addr learned my addr 172.21.15.12:0/1558402998 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:42.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f76fbfff640 1 --2- 172.21.15.12:0/1558402998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76fc15d6f0 0x7f76fc0cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:42.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f76fb7fe640 1 --2- 172.21.15.12:0/1558402998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76fc0cf160 0x7f76fc14e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:42.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f76faffd640 1 -- 172.21.15.12:0/1558402998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76fc15d6f0 msgr2=0x7f76fc0cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:42.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f76faffd640 1 --2- 172.21.15.12:0/1558402998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76fc15d6f0 0x7f76fc0cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f76faffd640 1 -- 172.21.15.12:0/1558402998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76fc0cf160 msgr2=0x7f76fc14e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:42.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f76faffd640 1 --2- 172.21.15.12:0/1558402998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76fc0cf160 0x7f76fc14e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f76faffd640 1 -- 172.21.15.12:0/1558402998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f0009420 con 0x7f76fc1538a0 2023-10-30T12:40:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f76fb7fe640 1 --2- 172.21.15.12:0/1558402998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76fc0cf160 0x7f76fc14e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:40:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f76faffd640 1 --2- 172.21.15.12:0/1558402998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76fc1538a0 0x7f76fc0b49f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f76f000e980 tx=0x7f76f000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f76e3fff640 1 -- 172.21.15.12:0/1558402998 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f000ca00 con 0x7f76fc1538a0 2023-10-30T12:40:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f76e3fff640 1 -- 172.21.15.12:0/1558402998 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f000cb60 con 0x7f76fc1538a0 2023-10-30T12:40:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f76e3fff640 1 -- 172.21.15.12:0/1558402998 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0013470 con 0x7f76fc1538a0 2023-10-30T12:40:42.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.486+0000 7f7701bc5640 1 -- 172.21.15.12:0/1558402998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f4009b30 con 0x7f76fc1538a0 2023-10-30T12:40:42.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.490+0000 7f7701bc5640 1 -- 172.21.15.12:0/1558402998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76fc1eff80 con 0x7f76fc1538a0 2023-10-30T12:40:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.490+0000 7f7701bc5640 1 -- 172.21.15.12:0/1558402998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76fc0b2a50 con 0x7f76fc1538a0 2023-10-30T12:40:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.490+0000 7f76e3fff640 1 -- 172.21.15.12:0/1558402998 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f76f00135d0 con 0x7f76fc1538a0 2023-10-30T12:40:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.490+0000 7f76e3fff640 1 --2- 172.21.15.12:0/1558402998 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f76e8066520 0x7f76e80689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.490+0000 7f76e3fff640 1 -- 172.21.15.12:0/1558402998 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f76f0084080 con 0x7f76fc1538a0 2023-10-30T12:40:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.490+0000 7f76fb7fe640 1 --2- 172.21.15.12:0/1558402998 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f76e8066520 0x7f76e80689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.494+0000 7f76fb7fe640 1 --2- 172.21.15.12:0/1558402998 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f76e8066520 0x7f76e80689e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f76ec000f70 tx=0x7f76ec007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.494+0000 7f76e3fff640 1 -- 172.21.15.12:0/1558402998 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f76f00b5090 con 0x7f76fc1538a0 2023-10-30T12:40:42.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.682+0000 7f7701bc5640 1 -- 172.21.15.12:0/1558402998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76fc04e170 con 0x7f76fc1538a0 2023-10-30T12:40:42.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.682+0000 7f76e3fff640 1 -- 172.21.15.12:0/1558402998 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5740 (secure 0 0 0) 0x7f76f0051500 con 0x7f76fc1538a0 2023-10-30T12:40:42.689 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:42.689 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:40.195798+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:42.690 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 26 2023-10-30T12:40:42.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.686+0000 7f7701bc5640 1 -- 172.21.15.12:0/1558402998 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f76e8066520 msgr2=0x7f76e80689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:42.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.686+0000 7f7701bc5640 1 --2- 172.21.15.12:0/1558402998 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f76e8066520 0x7f76e80689e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f76ec000f70 tx=0x7f76ec007d50 comp rx=0 tx=0).stop 2023-10-30T12:40:42.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.686+0000 7f7701bc5640 1 -- 172.21.15.12:0/1558402998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76fc1538a0 msgr2=0x7f76fc0b49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:42.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.686+0000 7f7701bc5640 1 --2- 172.21.15.12:0/1558402998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76fc1538a0 0x7f76fc0b49f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f76f000e980 tx=0x7f76f000ee40 comp rx=0 tx=0).stop 2023-10-30T12:40:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.686+0000 7f7701bc5640 1 -- 172.21.15.12:0/1558402998 shutdown_connections 2023-10-30T12:40:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.686+0000 7f7701bc5640 1 --2- 172.21.15.12:0/1558402998 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f76e8066520 0x7f76e80689e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.686+0000 7f7701bc5640 1 --2- 172.21.15.12:0/1558402998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76fc15d6f0 0x7f76fc0cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.686+0000 7f7701bc5640 1 --2- 172.21.15.12:0/1558402998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76fc1538a0 0x7f76fc0b49f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.686+0000 7f7701bc5640 1 --2- 172.21.15.12:0/1558402998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76fc0cf160 0x7f76fc14e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.686+0000 7f7701bc5640 1 -- 172.21.15.12:0/1558402998 >> 172.21.15.12:0/1558402998 conn(0x7f76fc06bfe0 msgr2=0x7f76fc0c48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.686+0000 7f7701bc5640 1 -- 172.21.15.12:0/1558402998 shutdown_connections 2023-10-30T12:40:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:42.686+0000 7f7701bc5640 1 -- 172.21.15.12:0/1558402998 wait complete. 2023-10-30T12:40:42.711 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 26, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 26, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4282": { "addr": "172.21.15.154:6839/1517672481", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1517672481, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 1517672481, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 4282, "incarnation": 10, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 7 }, "gid_4285": { "addr": "172.21.15.12:6836/291392212", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 291392212, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 291392212, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 4285, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 6 }, "gid_4971": { "addr": "172.21.15.12:6837/3124135109", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 3124135109, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 3124135109, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4971, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 38, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:40:40.195798+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4285, "mds_1": 4282, "mds_2": 4971 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:40:44.712 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:44.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.862+0000 7fe8f266b640 1 Processor -- start 2023-10-30T12:40:44.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.862+0000 7fe8f266b640 1 -- start start 2023-10-30T12:40:44.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.862+0000 7fe8f266b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe8ec15d6f0 0x7fe8ec15a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:44.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.862+0000 7fe8f266b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ec0cb800 con 0x7fe8ec0cf160 2023-10-30T12:40:44.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.862+0000 7fe8f266b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ec0cb330 con 0x7fe8ec1538a0 2023-10-30T12:40:44.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.862+0000 7fe8f266b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ec0cad70 con 0x7fe8ec15d6f0 2023-10-30T12:40:44.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.862+0000 7fe8eb7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe8ec15d6f0 0x7fe8ec15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:44.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.862+0000 7fe8eb7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe8ec15d6f0 0x7fe8ec15a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60166/0 (socket says 172.21.15.12:60166) 2023-10-30T12:40:44.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.862+0000 7fe8eb7fe640 1 -- 172.21.15.12:0/177136523 learned_addr learned my addr 172.21.15.12:0/177136523 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:44.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.862+0000 7fe8f0c08640 1 --1- 172.21.15.12:0/177136523 >> v1:172.21.15.12:6789/0 conn(0x7fe8ec0cf160 0x7fe8ec164070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39238/0 (socket says 172.21.15.12:39238) 2023-10-30T12:40:44.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.862+0000 7fe8eb7fe640 1 -- 172.21.15.12:0/177136523 >> v1:172.21.15.154:6789/0 conn(0x7fe8ec1538a0 legacy=0x7fe8ec0ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:44.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.862+0000 7fe8eb7fe640 1 -- 172.21.15.12:0/177136523 >> v1:172.21.15.12:6789/0 conn(0x7fe8ec0cf160 legacy=0x7fe8ec164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:44.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.862+0000 7fe8eb7fe640 1 -- 172.21.15.12:0/177136523 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec0cb050 con 0x7fe8ec15d6f0 2023-10-30T12:40:44.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.866+0000 7fe8eb7fe640 1 --2- 172.21.15.12:0/177136523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec15d6f0 0x7fe8ec15a090 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009870 tx=0x7fe8e001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216e9f334571c355 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:44.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.866+0000 7fe8eaffd640 1 -- 172.21.15.12:0/177136523 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00a8a30 con 0x7fe8ec15d6f0 2023-10-30T12:40:44.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.866+0000 7fe8eaffd640 1 -- 172.21.15.12:0/177136523 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e00a8b90 con 0x7fe8ec15d6f0 2023-10-30T12:40:44.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.866+0000 7fe8eaffd640 1 -- 172.21.15.12:0/177136523 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00b0440 con 0x7fe8ec15d6f0 2023-10-30T12:40:44.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.866+0000 7fe8f266b640 1 -- 172.21.15.12:0/177136523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec15d6f0 msgr2=0x7fe8ec15a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:44.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.866+0000 7fe8f266b640 1 --2- 172.21.15.12:0/177136523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec15d6f0 0x7fe8ec15a090 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009870 tx=0x7fe8e001ed30 comp rx=0 tx=0).stop 2023-10-30T12:40:44.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.866+0000 7fe8f266b640 1 -- 172.21.15.12:0/177136523 shutdown_connections 2023-10-30T12:40:44.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.866+0000 7fe8f266b640 1 --2- 172.21.15.12:0/177136523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec15d6f0 0x7fe8ec15a090 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:44.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.866+0000 7fe8f266b640 1 -- 172.21.15.12:0/177136523 >> 172.21.15.12:0/177136523 conn(0x7fe8ec06bfe0 msgr2=0x7fe8ec06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:44.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.866+0000 7fe8f266b640 1 -- 172.21.15.12:0/177136523 shutdown_connections 2023-10-30T12:40:44.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.866+0000 7fe8f266b640 1 -- 172.21.15.12:0/177136523 wait complete. 2023-10-30T12:40:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8f266b640 1 Processor -- start 2023-10-30T12:40:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8f266b640 1 -- start start 2023-10-30T12:40:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8f266b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8ec0cf160 0x7fe8ec1e3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8f266b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec1538a0 0x7fe8ec1592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8f266b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ec15d6f0 0x7fe8ec0c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:44.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8f266b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0c8ce0 con 0x7fe8ec1538a0 2023-10-30T12:40:44.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8f266b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0c8310 con 0x7fe8ec15d6f0 2023-10-30T12:40:44.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8f266b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0c7060 con 0x7fe8ec0cf160 2023-10-30T12:40:44.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8eb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec1538a0 0x7fe8ec1592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:44.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8eb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec1538a0 0x7fe8ec1592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60170/0 (socket says 172.21.15.12:60170) 2023-10-30T12:40:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8eb7fe640 1 -- 172.21.15.12:0/2099959174 learned_addr learned my addr 172.21.15.12:0/2099959174 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8f0c08640 1 --2- 172.21.15.12:0/2099959174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ec15d6f0 0x7fe8ec0c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8ebfff640 1 --2- 172.21.15.12:0/2099959174 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8ec0cf160 0x7fe8ec1e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8eb7fe640 1 -- 172.21.15.12:0/2099959174 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8ec0cf160 msgr2=0x7fe8ec1e3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8eb7fe640 1 --2- 172.21.15.12:0/2099959174 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8ec0cf160 0x7fe8ec1e3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8eb7fe640 1 -- 172.21.15.12:0/2099959174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ec15d6f0 msgr2=0x7fe8ec0c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8eb7fe640 1 --2- 172.21.15.12:0/2099959174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ec15d6f0 0x7fe8ec0c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8eb7fe640 1 -- 172.21.15.12:0/2099959174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e00094d0 con 0x7fe8ec1538a0 2023-10-30T12:40:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8ebfff640 1 --2- 172.21.15.12:0/2099959174 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8ec0cf160 0x7fe8ec1e3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:40:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8eb7fe640 1 --2- 172.21.15.12:0/2099959174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec1538a0 0x7fe8ec1592a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00b3040 tx=0x7fe8e001bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:44.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8e8ff9640 1 -- 172.21.15.12:0/2099959174 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00be070 con 0x7fe8ec1538a0 2023-10-30T12:40:44.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8e8ff9640 1 -- 172.21.15.12:0/2099959174 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e00ba400 con 0x7fe8ec1538a0 2023-10-30T12:40:44.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8e8ff9640 1 -- 172.21.15.12:0/2099959174 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00ba7b0 con 0x7fe8ec1538a0 2023-10-30T12:40:44.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8f266b640 1 -- 172.21.15.12:0/2099959174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ec0c7d00 con 0x7fe8ec1538a0 2023-10-30T12:40:44.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.870+0000 7fe8f266b640 1 -- 172.21.15.12:0/2099959174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ec0c6aa0 con 0x7fe8ec1538a0 2023-10-30T12:40:44.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.874+0000 7fe8f266b640 1 -- 172.21.15.12:0/2099959174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ec04e170 con 0x7fe8ec1538a0 2023-10-30T12:40:44.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.874+0000 7fe8e8ff9640 1 -- 172.21.15.12:0/2099959174 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe8e00ba910 con 0x7fe8ec1538a0 2023-10-30T12:40:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.874+0000 7fe8e8ff9640 1 --2- 172.21.15.12:0/2099959174 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe8d80664a0 0x7fe8d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.874+0000 7fe8e8ff9640 1 -- 172.21.15.12:0/2099959174 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fe8e0121e60 con 0x7fe8ec1538a0 2023-10-30T12:40:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.874+0000 7fe8ebfff640 1 --2- 172.21.15.12:0/2099959174 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe8d80664a0 0x7fe8d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.874+0000 7fe8ebfff640 1 --2- 172.21.15.12:0/2099959174 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe8d80664a0 0x7fe8d8068960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc000f70 tx=0x7fe8dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:44.878+0000 7fe8e8ff9640 1 -- 172.21.15.12:0/2099959174 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe8e00ef1b0 con 0x7fe8ec1538a0 2023-10-30T12:40:45.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:45.066+0000 7fe8f266b640 1 -- 172.21.15.12:0/2099959174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8ec0c8450 con 0x7fe8ec1538a0 2023-10-30T12:40:45.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:45.066+0000 7fe8e8ff9640 1 -- 172.21.15.12:0/2099959174 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5740 (secure 0 0 0) 0x7fe8e00b5020 con 0x7fe8ec1538a0 2023-10-30T12:40:45.073 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:45.073 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:40.195798+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:45.073 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 26 2023-10-30T12:40:45.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:45.070+0000 7fe8f266b640 1 -- 172.21.15.12:0/2099959174 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe8d80664a0 msgr2=0x7fe8d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:45.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:45.070+0000 7fe8f266b640 1 --2- 172.21.15.12:0/2099959174 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe8d80664a0 0x7fe8d8068960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc000f70 tx=0x7fe8dc009040 comp rx=0 tx=0).stop 2023-10-30T12:40:45.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:45.070+0000 7fe8f266b640 1 -- 172.21.15.12:0/2099959174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec1538a0 msgr2=0x7fe8ec1592a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:45.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:45.070+0000 7fe8f266b640 1 --2- 172.21.15.12:0/2099959174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec1538a0 0x7fe8ec1592a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00b3040 tx=0x7fe8e001bad0 comp rx=0 tx=0).stop 2023-10-30T12:40:45.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:45.070+0000 7fe8f266b640 1 -- 172.21.15.12:0/2099959174 shutdown_connections 2023-10-30T12:40:45.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:45.070+0000 7fe8f266b640 1 --2- 172.21.15.12:0/2099959174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ec15d6f0 0x7fe8ec0c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:45.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:45.070+0000 7fe8f266b640 1 --2- 172.21.15.12:0/2099959174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec1538a0 0x7fe8ec1592a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:45.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:45.070+0000 7fe8f266b640 1 --2- 172.21.15.12:0/2099959174 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe8d80664a0 0x7fe8d8068960 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:45.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:45.070+0000 7fe8f266b640 1 --2- 172.21.15.12:0/2099959174 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8ec0cf160 0x7fe8ec1e3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:45.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:45.070+0000 7fe8f266b640 1 -- 172.21.15.12:0/2099959174 >> 172.21.15.12:0/2099959174 conn(0x7fe8ec06bfe0 msgr2=0x7fe8ec0ce1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:45.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:45.070+0000 7fe8f266b640 1 -- 172.21.15.12:0/2099959174 shutdown_connections 2023-10-30T12:40:45.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:45.070+0000 7fe8f266b640 1 -- 172.21.15.12:0/2099959174 wait complete. 2023-10-30T12:40:45.094 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:40:47.095 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:47.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.242+0000 7fd1f3f1d640 1 Processor -- start 2023-10-30T12:40:47.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.246+0000 7fd1f3f1d640 1 -- start start 2023-10-30T12:40:47.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.246+0000 7fd1f3f1d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd1ec0c6b80 0x7fd1ec0bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:47.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.246+0000 7fd1f3f1d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1ec13f4a0 0x7fd1ec0c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:47.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.246+0000 7fd1f3f1d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ec0c0a50 con 0x7fd1ec0c6f60 2023-10-30T12:40:47.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.246+0000 7fd1f3f1d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec06f6e0 con 0x7fd1ec0c6b80 2023-10-30T12:40:47.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.246+0000 7fd1f3f1d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec06f9c0 con 0x7fd1ec13f4a0 2023-10-30T12:40:47.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.246+0000 7fd1f1cb9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd1ec0c6b80 0x7fd1ec0bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:47.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.246+0000 7fd1f1cb9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd1ec0c6b80 0x7fd1ec0bfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60174/0 (socket says 172.21.15.12:60174) 2023-10-30T12:40:47.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.246+0000 7fd1f1cb9640 1 -- 172.21.15.12:0/3980826981 learned_addr learned my addr 172.21.15.12:0/3980826981 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:47.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.246+0000 7fd1f24ba640 1 --2- 172.21.15.12:0/3980826981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1ec13f4a0 0x7fd1ec0c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:47.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.246+0000 7fd1f1cb9640 1 -- 172.21.15.12:0/3980826981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1ec13f4a0 msgr2=0x7fd1ec0c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:47.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.246+0000 7fd1f1cb9640 1 --2- 172.21.15.12:0/3980826981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1ec13f4a0 0x7fd1ec0c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:47.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.246+0000 7fd1f1cb9640 1 -- 172.21.15.12:0/3980826981 >> v1:172.21.15.154:6789/0 conn(0x7fd1ec0c6f60 legacy=0x7fd1ec13ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:47.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.246+0000 7fd1f1cb9640 1 -- 172.21.15.12:0/3980826981 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec06fca0 con 0x7fd1ec0c6b80 2023-10-30T12:40:47.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.250+0000 7fd1f1cb9640 1 --2- 172.21.15.12:0/3980826981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1ec0c6b80 0x7fd1ec0bfec0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009870 tx=0x7fd1e801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fc0393c4b0e0c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:47.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.250+0000 7fd1f0cb7640 1 -- 172.21.15.12:0/3980826981 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80ab650 con 0x7fd1ec0c6b80 2023-10-30T12:40:47.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.250+0000 7fd1f0cb7640 1 -- 172.21.15.12:0/3980826981 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e80ab7b0 con 0x7fd1ec0c6b80 2023-10-30T12:40:47.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.250+0000 7fd1f0cb7640 1 -- 172.21.15.12:0/3980826981 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80abb40 con 0x7fd1ec0c6b80 2023-10-30T12:40:47.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.250+0000 7fd1f3f1d640 1 -- 172.21.15.12:0/3980826981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1ec0c6b80 msgr2=0x7fd1ec0bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:47.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.250+0000 7fd1f3f1d640 1 --2- 172.21.15.12:0/3980826981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1ec0c6b80 0x7fd1ec0bfec0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009870 tx=0x7fd1e801c540 comp rx=0 tx=0).stop 2023-10-30T12:40:47.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.250+0000 7fd1f3f1d640 1 -- 172.21.15.12:0/3980826981 shutdown_connections 2023-10-30T12:40:47.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.250+0000 7fd1f3f1d640 1 --2- 172.21.15.12:0/3980826981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1ec13f4a0 0x7fd1ec0c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:47.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.250+0000 7fd1f3f1d640 1 --2- 172.21.15.12:0/3980826981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1ec0c6b80 0x7fd1ec0bfec0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:47.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.250+0000 7fd1f3f1d640 1 -- 172.21.15.12:0/3980826981 >> 172.21.15.12:0/3980826981 conn(0x7fd1ec0c7910 msgr2=0x7fd1ec0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:47.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.250+0000 7fd1f3f1d640 1 -- 172.21.15.12:0/3980826981 shutdown_connections 2023-10-30T12:40:47.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.250+0000 7fd1f3f1d640 1 -- 172.21.15.12:0/3980826981 wait complete. 2023-10-30T12:40:47.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.250+0000 7fd1f3f1d640 1 Processor -- start 2023-10-30T12:40:47.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.250+0000 7fd1f3f1d640 1 -- start start 2023-10-30T12:40:47.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f3f1d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1ec0c6b80 0x7fd1ec157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:47.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f3f1d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1ec0c6f60 0x7fd1ec0c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:47.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f3f1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1ec13f4a0 0x7fd1ec158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:47.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f3f1d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec1e9710 con 0x7fd1ec13f4a0 2023-10-30T12:40:47.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f3f1d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec157f70 con 0x7fd1ec0c6f60 2023-10-30T12:40:47.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f3f1d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec1e8de0 con 0x7fd1ec0c6b80 2023-10-30T12:40:47.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f24ba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1ec13f4a0 0x7fd1ec158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:47.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f24ba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1ec13f4a0 0x7fd1ec158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60184/0 (socket says 172.21.15.12:60184) 2023-10-30T12:40:47.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f24ba640 1 -- 172.21.15.12:0/2678141006 learned_addr learned my addr 172.21.15.12:0/2678141006 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:47.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f1cb9640 1 --2- 172.21.15.12:0/2678141006 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1ec0c6b80 0x7fd1ec157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:47.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f14b8640 1 --2- 172.21.15.12:0/2678141006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1ec0c6f60 0x7fd1ec0c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f24ba640 1 -- 172.21.15.12:0/2678141006 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1ec0c6b80 msgr2=0x7fd1ec157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f24ba640 1 --2- 172.21.15.12:0/2678141006 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1ec0c6b80 0x7fd1ec157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f24ba640 1 -- 172.21.15.12:0/2678141006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1ec0c6f60 msgr2=0x7fd1ec0c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f24ba640 1 --2- 172.21.15.12:0/2678141006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1ec0c6f60 0x7fd1ec0c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f24ba640 1 -- 172.21.15.12:0/2678141006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e80094d0 con 0x7fd1ec13f4a0 2023-10-30T12:40:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f24ba640 1 --2- 172.21.15.12:0/2678141006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1ec13f4a0 0x7fd1ec158400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4007e90 tx=0x7fd1e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1e27fc640 1 -- 172.21.15.12:0/2678141006 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4014420 con 0x7fd1ec13f4a0 2023-10-30T12:40:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1e27fc640 1 -- 172.21.15.12:0/2678141006 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e4014580 con 0x7fd1ec13f4a0 2023-10-30T12:40:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1e27fc640 1 -- 172.21.15.12:0/2678141006 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4014930 con 0x7fd1ec13f4a0 2023-10-30T12:40:47.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f3f1d640 1 -- 172.21.15.12:0/2678141006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec159770 con 0x7fd1ec13f4a0 2023-10-30T12:40:47.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f3f1d640 1 -- 172.21.15.12:0/2678141006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec1e8b00 con 0x7fd1ec13f4a0 2023-10-30T12:40:47.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1f3f1d640 1 -- 172.21.15.12:0/2678141006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec04e170 con 0x7fd1ec13f4a0 2023-10-30T12:40:47.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.254+0000 7fd1e27fc640 1 -- 172.21.15.12:0/2678141006 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd1e4014a90 con 0x7fd1ec13f4a0 2023-10-30T12:40:47.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.258+0000 7fd1e27fc640 1 --2- 172.21.15.12:0/2678141006 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1d00664a0 0x7fd1d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:47.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.258+0000 7fd1e27fc640 1 -- 172.21.15.12:0/2678141006 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fd1e4084800 con 0x7fd1ec13f4a0 2023-10-30T12:40:47.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.258+0000 7fd1f1cb9640 1 --2- 172.21.15.12:0/2678141006 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1d00664a0 0x7fd1d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:47.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.258+0000 7fd1f1cb9640 1 --2- 172.21.15.12:0/2678141006 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1d00664a0 0x7fd1d0068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80af000 tx=0x7fd1e80b6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:47.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.262+0000 7fd1e27fc640 1 -- 172.21.15.12:0/2678141006 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd1e4051b50 con 0x7fd1ec13f4a0 2023-10-30T12:40:47.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.450+0000 7fd1f3f1d640 1 -- 172.21.15.12:0/2678141006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1ec1580b0 con 0x7fd1ec13f4a0 2023-10-30T12:40:47.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.454+0000 7fd1e27fc640 1 -- 172.21.15.12:0/2678141006 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5742 (secure 0 0 0) 0x7fd1e4055170 con 0x7fd1ec13f4a0 2023-10-30T12:40:47.459 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:47.459 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:46.155051+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:resolve","state_seq":38,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:47.461 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2023-10-30T12:40:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.454+0000 7fd1f3f1d640 1 -- 172.21.15.12:0/2678141006 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1d00664a0 msgr2=0x7fd1d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.454+0000 7fd1f3f1d640 1 --2- 172.21.15.12:0/2678141006 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1d00664a0 0x7fd1d0068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80af000 tx=0x7fd1e80b6040 comp rx=0 tx=0).stop 2023-10-30T12:40:47.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.454+0000 7fd1f3f1d640 1 -- 172.21.15.12:0/2678141006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1ec13f4a0 msgr2=0x7fd1ec158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:47.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.454+0000 7fd1f3f1d640 1 --2- 172.21.15.12:0/2678141006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1ec13f4a0 0x7fd1ec158400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4007e90 tx=0x7fd1e400e4b0 comp rx=0 tx=0).stop 2023-10-30T12:40:47.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.454+0000 7fd1f3f1d640 1 -- 172.21.15.12:0/2678141006 shutdown_connections 2023-10-30T12:40:47.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.454+0000 7fd1f3f1d640 1 --2- 172.21.15.12:0/2678141006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1ec13f4a0 0x7fd1ec158400 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:47.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.454+0000 7fd1f3f1d640 1 --2- 172.21.15.12:0/2678141006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1ec0c6f60 0x7fd1ec0c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:47.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.454+0000 7fd1f3f1d640 1 --2- 172.21.15.12:0/2678141006 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1d00664a0 0x7fd1d0068960 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.454+0000 7fd1f3f1d640 1 --2- 172.21.15.12:0/2678141006 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1ec0c6b80 0x7fd1ec157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.454+0000 7fd1f3f1d640 1 -- 172.21.15.12:0/2678141006 >> 172.21.15.12:0/2678141006 conn(0x7fd1ec0c7910 msgr2=0x7fd1ec139510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:47.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.454+0000 7fd1f3f1d640 1 -- 172.21.15.12:0/2678141006 shutdown_connections 2023-10-30T12:40:47.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:47.454+0000 7fd1f3f1d640 1 -- 172.21.15.12:0/2678141006 wait complete. 2023-10-30T12:40:47.479 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:40:49.481 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:49.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.630+0000 7fa4e489e640 1 Processor -- start 2023-10-30T12:40:49.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.630+0000 7fa4e489e640 1 -- start start 2023-10-30T12:40:49.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.630+0000 7fa4e489e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa4e006f7a0 0x7fa4e006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:49.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.630+0000 7fa4e489e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e0066cf0 con 0x7fa4e00c6bd0 2023-10-30T12:40:49.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.630+0000 7fa4e489e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e00c70d0 con 0x7fa4e0071110 2023-10-30T12:40:49.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.634+0000 7fa4e489e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e00700c0 con 0x7fa4e006f7a0 2023-10-30T12:40:49.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.634+0000 7fa4de59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa4e006f7a0 0x7fa4e006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:49.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.634+0000 7fa4ddd9b640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa4e00c6bd0 0x7fa4e0069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39250/0 (socket says 172.21.15.12:39250) 2023-10-30T12:40:49.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.634+0000 7fa4de59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa4e006f7a0 0x7fa4e006fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60188/0 (socket says 172.21.15.12:60188) 2023-10-30T12:40:49.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.634+0000 7fa4ddd9b640 1 -- 172.21.15.12:0/1772226931 learned_addr learned my addr 172.21.15.12:0/1772226931 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:40:49.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.634+0000 7fa4de59c640 1 -- 172.21.15.12:0/1772226931 >> v1:172.21.15.154:6789/0 conn(0x7fa4e0071110 legacy=0x7fa4e0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:49.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.634+0000 7fa4de59c640 1 -- 172.21.15.12:0/1772226931 >> v1:172.21.15.12:6789/0 conn(0x7fa4e00c6bd0 legacy=0x7fa4e0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:49.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.634+0000 7fa4de59c640 1 -- 172.21.15.12:0/1772226931 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e00c7390 con 0x7fa4e006f7a0 2023-10-30T12:40:49.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.634+0000 7fa4dd59a640 1 -- 172.21.15.12:0/1772226931 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442227185 0 0) 0x7fa4e0066cf0 con 0x7fa4e00c6bd0 2023-10-30T12:40:49.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.634+0000 7fa4de59c640 1 --2- 172.21.15.12:0/1772226931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4e006f7a0 0x7fa4e006fb80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00097c0 tx=0x7fa4d001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ccbccf23ccacbcb server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:49.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.634+0000 7fa4dd59a640 1 -- 172.21.15.12:0/1772226931 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d00a8970 con 0x7fa4e006f7a0 2023-10-30T12:40:49.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.634+0000 7fa4dd59a640 1 -- 172.21.15.12:0/1772226931 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d00a8ad0 con 0x7fa4e006f7a0 2023-10-30T12:40:49.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4dd59a640 1 -- 172.21.15.12:0/1772226931 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d00a8e60 con 0x7fa4e006f7a0 2023-10-30T12:40:49.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4e489e640 1 -- 172.21.15.12:0/1772226931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4e006f7a0 msgr2=0x7fa4e006fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:49.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4e489e640 1 --2- 172.21.15.12:0/1772226931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4e006f7a0 0x7fa4e006fb80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00097c0 tx=0x7fa4d001ec70 comp rx=0 tx=0).stop 2023-10-30T12:40:49.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4e489e640 1 -- 172.21.15.12:0/1772226931 shutdown_connections 2023-10-30T12:40:49.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4e489e640 1 --2- 172.21.15.12:0/1772226931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4e006f7a0 0x7fa4e006fb80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:49.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4e489e640 1 -- 172.21.15.12:0/1772226931 >> 172.21.15.12:0/1772226931 conn(0x7fa4e00c7910 msgr2=0x7fa4e00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:49.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4e489e640 1 -- 172.21.15.12:0/1772226931 shutdown_connections 2023-10-30T12:40:49.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4e489e640 1 -- 172.21.15.12:0/1772226931 wait complete. 2023-10-30T12:40:49.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4e489e640 1 Processor -- start 2023-10-30T12:40:49.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4e489e640 1 -- start start 2023-10-30T12:40:49.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4e489e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4e006f7a0 0x7fa4e00c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:49.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4e489e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4e0071110 0x7fa4e01e1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:49.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4e489e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4e00c6bd0 0x7fa4e01ef010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:49.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4e489e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e01f31f0 con 0x7fa4e00c6bd0 2023-10-30T12:40:49.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4e489e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e01f2360 con 0x7fa4e006f7a0 2023-10-30T12:40:49.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.638+0000 7fa4e489e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e00c92b0 con 0x7fa4e0071110 2023-10-30T12:40:49.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4ded9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4e00c6bd0 0x7fa4e01ef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:49.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4ded9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4e00c6bd0 0x7fa4e01ef010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60194/0 (socket says 172.21.15.12:60194) 2023-10-30T12:40:49.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4ded9d640 1 -- 172.21.15.12:0/4057265286 learned_addr learned my addr 172.21.15.12:0/4057265286 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:49.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4de59c640 1 --2- 172.21.15.12:0/4057265286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4e006f7a0 0x7fa4e00c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:49.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4ddd9b640 1 --2- 172.21.15.12:0/4057265286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4e0071110 0x7fa4e01e1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:49.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4ded9d640 1 -- 172.21.15.12:0/4057265286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4e0071110 msgr2=0x7fa4e01e1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:49.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4ded9d640 1 --2- 172.21.15.12:0/4057265286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4e0071110 0x7fa4e01e1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:49.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4ded9d640 1 -- 172.21.15.12:0/4057265286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4e006f7a0 msgr2=0x7fa4e00c3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:49.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4ded9d640 1 --2- 172.21.15.12:0/4057265286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4e006f7a0 0x7fa4e00c3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:49.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4ded9d640 1 -- 172.21.15.12:0/4057265286 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d0009420 con 0x7fa4e00c6bd0 2023-10-30T12:40:49.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4ded9d640 1 --2- 172.21.15.12:0/4057265286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4e00c6bd0 0x7fa4e01ef010 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800c730 tx=0x7fa4d800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:49.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4ceffd640 1 -- 172.21.15.12:0/4057265286 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8003b80 con 0x7fa4e00c6bd0 2023-10-30T12:40:49.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4ceffd640 1 -- 172.21.15.12:0/4057265286 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d8003ce0 con 0x7fa4e00c6bd0 2023-10-30T12:40:49.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4ceffd640 1 -- 172.21.15.12:0/4057265286 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800aa30 con 0x7fa4e00c6bd0 2023-10-30T12:40:49.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4e489e640 1 -- 172.21.15.12:0/4057265286 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e01f2570 con 0x7fa4e00c6bd0 2023-10-30T12:40:49.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4e489e640 1 -- 172.21.15.12:0/4057265286 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e01f2b30 con 0x7fa4e00c6bd0 2023-10-30T12:40:49.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4e489e640 1 -- 172.21.15.12:0/4057265286 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e004e170 con 0x7fa4e00c6bd0 2023-10-30T12:40:49.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4ceffd640 1 -- 172.21.15.12:0/4057265286 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa4d800ab90 con 0x7fa4e00c6bd0 2023-10-30T12:40:49.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.642+0000 7fa4ceffd640 1 --2- 172.21.15.12:0/4057265286 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4c00664a0 0x7fa4c0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:49.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.646+0000 7fa4ceffd640 1 -- 172.21.15.12:0/4057265286 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fa4d8083d50 con 0x7fa4e00c6bd0 2023-10-30T12:40:49.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.646+0000 7fa4de59c640 1 --2- 172.21.15.12:0/4057265286 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4c00664a0 0x7fa4c0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:49.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.646+0000 7fa4de59c640 1 --2- 172.21.15.12:0/4057265286 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4c00664a0 0x7fa4c0068960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00b2000 tx=0x7fa4d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:49.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.646+0000 7fa4ceffd640 1 -- 172.21.15.12:0/4057265286 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa4d8051010 con 0x7fa4e00c6bd0 2023-10-30T12:40:49.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.838+0000 7fa4e489e640 1 -- 172.21.15.12:0/4057265286 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4e00c8c10 con 0x7fa4e00c6bd0 2023-10-30T12:40:49.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.838+0000 7fa4ceffd640 1 -- 172.21.15.12:0/4057265286 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5742 (secure 0 0 0) 0x7fa4d8054630 con 0x7fa4e00c6bd0 2023-10-30T12:40:49.845 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:49.845 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:46.155051+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:resolve","state_seq":38,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:49.846 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2023-10-30T12:40:49.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.842+0000 7fa4e489e640 1 -- 172.21.15.12:0/4057265286 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4c00664a0 msgr2=0x7fa4c0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:49.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.842+0000 7fa4e489e640 1 --2- 172.21.15.12:0/4057265286 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4c00664a0 0x7fa4c0068960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00b2000 tx=0x7fa4d0008040 comp rx=0 tx=0).stop 2023-10-30T12:40:49.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.842+0000 7fa4e489e640 1 -- 172.21.15.12:0/4057265286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4e00c6bd0 msgr2=0x7fa4e01ef010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:49.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.842+0000 7fa4e489e640 1 --2- 172.21.15.12:0/4057265286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4e00c6bd0 0x7fa4e01ef010 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800c730 tx=0x7fa4d800cbf0 comp rx=0 tx=0).stop 2023-10-30T12:40:49.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.842+0000 7fa4e489e640 1 -- 172.21.15.12:0/4057265286 shutdown_connections 2023-10-30T12:40:49.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.842+0000 7fa4e489e640 1 --2- 172.21.15.12:0/4057265286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4e00c6bd0 0x7fa4e01ef010 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:49.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.842+0000 7fa4e489e640 1 --2- 172.21.15.12:0/4057265286 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4c00664a0 0x7fa4c0068960 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:49.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.842+0000 7fa4e489e640 1 --2- 172.21.15.12:0/4057265286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4e0071110 0x7fa4e01e1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:49.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.842+0000 7fa4e489e640 1 --2- 172.21.15.12:0/4057265286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4e006f7a0 0x7fa4e00c3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:49.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.842+0000 7fa4e489e640 1 -- 172.21.15.12:0/4057265286 >> 172.21.15.12:0/4057265286 conn(0x7fa4e00c7910 msgr2=0x7fa4e00c54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:49.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.842+0000 7fa4e489e640 1 -- 172.21.15.12:0/4057265286 shutdown_connections 2023-10-30T12:40:49.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:49.842+0000 7fa4e489e640 1 -- 172.21.15.12:0/4057265286 wait complete. 2023-10-30T12:40:49.866 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:40:51.867 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:52.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.014+0000 7ff4673c4640 1 Processor -- start 2023-10-30T12:40:52.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.018+0000 7ff4673c4640 1 -- start start 2023-10-30T12:40:52.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.018+0000 7ff4673c4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff460156f60 0x7ff460071b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:52.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.018+0000 7ff4673c4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff46015d520 0x7ff460072050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:52.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.018+0000 7ff4673c4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff460157340 con 0x7ff46006fd50 2023-10-30T12:40:52.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.018+0000 7ff4673c4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff46006bea0 con 0x7ff460156f60 2023-10-30T12:40:52.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.018+0000 7ff4673c4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600caa80 con 0x7ff46015d520 2023-10-30T12:40:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.018+0000 7ff465160640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff460156f60 0x7ff460071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:52.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.018+0000 7ff465961640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff46015d520 0x7ff460072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:52.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.018+0000 7ff46495f640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff46006fd50 0x7ff46015ce10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:46376/0 (socket says 172.21.15.12:46376) 2023-10-30T12:40:52.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.018+0000 7ff46495f640 1 -- 172.21.15.12:0/1948137984 learned_addr learned my addr 172.21.15.12:0/1948137984 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:40:52.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.018+0000 7ff465160640 1 -- 172.21.15.12:0/1948137984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff46015d520 msgr2=0x7ff460072050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:52.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.018+0000 7ff465160640 1 --2- 172.21.15.12:0/1948137984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff46015d520 0x7ff460072050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:52.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.018+0000 7ff465160640 1 -- 172.21.15.12:0/1948137984 >> v1:172.21.15.154:6789/0 conn(0x7ff46006fd50 legacy=0x7ff46015ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:52.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.018+0000 7ff465160640 1 -- 172.21.15.12:0/1948137984 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46006ca30 con 0x7ff460156f60 2023-10-30T12:40:52.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.018+0000 7ff465961640 1 --2- 172.21.15.12:0/1948137984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff46015d520 0x7ff460072050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:40:52.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.022+0000 7ff465160640 1 --2- 172.21.15.12:0/1948137984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff460156f60 0x7ff460071b10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0097c0 tx=0x7ff45c01c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=546f1616406b4336 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:52.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.022+0000 7ff457fff640 1 -- 172.21.15.12:0/1948137984 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0ab660 con 0x7ff460156f60 2023-10-30T12:40:52.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.022+0000 7ff457fff640 1 -- 172.21.15.12:0/1948137984 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c0ab7c0 con 0x7ff460156f60 2023-10-30T12:40:52.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.022+0000 7ff457fff640 1 -- 172.21.15.12:0/1948137984 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0abb50 con 0x7ff460156f60 2023-10-30T12:40:52.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.022+0000 7ff4673c4640 1 -- 172.21.15.12:0/1948137984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff460156f60 msgr2=0x7ff460071b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:52.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.022+0000 7ff4673c4640 1 --2- 172.21.15.12:0/1948137984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff460156f60 0x7ff460071b10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0097c0 tx=0x7ff45c01c460 comp rx=0 tx=0).stop 2023-10-30T12:40:52.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.022+0000 7ff4673c4640 1 -- 172.21.15.12:0/1948137984 shutdown_connections 2023-10-30T12:40:52.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.022+0000 7ff4673c4640 1 --2- 172.21.15.12:0/1948137984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff46015d520 0x7ff460072050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:52.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.022+0000 7ff4673c4640 1 --2- 172.21.15.12:0/1948137984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff460156f60 0x7ff460071b10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:52.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.022+0000 7ff4673c4640 1 -- 172.21.15.12:0/1948137984 >> 172.21.15.12:0/1948137984 conn(0x7ff460069e90 msgr2=0x7ff46006a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:52.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.022+0000 7ff4673c4640 1 -- 172.21.15.12:0/1948137984 shutdown_connections 2023-10-30T12:40:52.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.022+0000 7ff4673c4640 1 -- 172.21.15.12:0/1948137984 wait complete. 2023-10-30T12:40:52.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff4673c4640 1 Processor -- start 2023-10-30T12:40:52.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff4673c4640 1 -- start start 2023-10-30T12:40:52.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff4673c4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff46006fd50 0x7ff4601f2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:52.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff4673c4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff460156f60 0x7ff46014df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:52.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff4673c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4600c7000 0x7ff4601593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:52.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff4673c4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46015c680 con 0x7ff4600c7000 2023-10-30T12:40:52.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff4673c4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4601f1a70 con 0x7ff46006fd50 2023-10-30T12:40:52.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff4673c4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600c6be0 con 0x7ff460156f60 2023-10-30T12:40:52.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff465961640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4600c7000 0x7ff4601593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:52.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff465961640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4600c7000 0x7ff4601593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40092/0 (socket says 172.21.15.12:40092) 2023-10-30T12:40:52.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff465961640 1 -- 172.21.15.12:0/1126675717 learned_addr learned my addr 172.21.15.12:0/1126675717 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:52.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff465160640 1 --2- 172.21.15.12:0/1126675717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff46006fd50 0x7ff4601f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:52.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff46495f640 1 --2- 172.21.15.12:0/1126675717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff460156f60 0x7ff46014df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:52.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff465961640 1 -- 172.21.15.12:0/1126675717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff460156f60 msgr2=0x7ff46014df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:52.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff465961640 1 --2- 172.21.15.12:0/1126675717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff460156f60 0x7ff46014df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:52.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff465961640 1 -- 172.21.15.12:0/1126675717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff46006fd50 msgr2=0x7ff4601f2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:52.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff465961640 1 --2- 172.21.15.12:0/1126675717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff46006fd50 0x7ff4601f2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:52.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff465961640 1 -- 172.21.15.12:0/1126675717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff458009a70 con 0x7ff4600c7000 2023-10-30T12:40:52.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff46495f640 1 --2- 172.21.15.12:0/1126675717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff460156f60 0x7ff46014df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:40:52.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff465961640 1 --2- 172.21.15.12:0/1126675717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4600c7000 0x7ff4601593b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff45800efc0 tx=0x7ff45800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:52.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff455ffb640 1 -- 172.21.15.12:0/1126675717 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45800cd30 con 0x7ff4600c7000 2023-10-30T12:40:52.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff455ffb640 1 -- 172.21.15.12:0/1126675717 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45800f040 con 0x7ff4600c7000 2023-10-30T12:40:52.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff455ffb640 1 -- 172.21.15.12:0/1126675717 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458003be0 con 0x7ff4600c7000 2023-10-30T12:40:52.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff4673c4640 1 -- 172.21.15.12:0/1126675717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45c009420 con 0x7ff4600c7000 2023-10-30T12:40:52.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff4673c4640 1 -- 172.21.15.12:0/1126675717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46015bde0 con 0x7ff4600c7000 2023-10-30T12:40:52.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.026+0000 7ff4673c4640 1 -- 172.21.15.12:0/1126675717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46004e170 con 0x7ff4600c7000 2023-10-30T12:40:52.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.030+0000 7ff455ffb640 1 -- 172.21.15.12:0/1126675717 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff458014480 con 0x7ff4600c7000 2023-10-30T12:40:52.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.030+0000 7ff455ffb640 1 --2- 172.21.15.12:0/1126675717 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff4440664a0 0x7ff444068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:52.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.030+0000 7ff455ffb640 1 -- 172.21.15.12:0/1126675717 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5997+0+0 (secure 0 0 0) 0x7ff458084a40 con 0x7ff4600c7000 2023-10-30T12:40:52.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.030+0000 7ff465160640 1 --2- 172.21.15.12:0/1126675717 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff4440664a0 0x7ff444068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:52.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.030+0000 7ff465160640 1 --2- 172.21.15.12:0/1126675717 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff4440664a0 0x7ff444068960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0af000 tx=0x7ff45c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:52.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.034+0000 7ff455ffb640 1 -- 172.21.15.12:0/1126675717 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff458086e60 con 0x7ff4600c7000 2023-10-30T12:40:52.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.226+0000 7ff4673c4640 1 -- 172.21.15.12:0/1126675717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff46006fb30 con 0x7ff4600c7000 2023-10-30T12:40:52.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.226+0000 7ff455ffb640 1 -- 172.21.15.12:0/1126675717 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5744 (secure 0 0 0) 0x7ff458051e30 con 0x7ff4600c7000 2023-10-30T12:40:52.232 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:52.232 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:51.480471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:reconnect","state_seq":40,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:52.232 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2023-10-30T12:40:52.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.226+0000 7ff4673c4640 1 -- 172.21.15.12:0/1126675717 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff4440664a0 msgr2=0x7ff444068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:52.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.226+0000 7ff4673c4640 1 --2- 172.21.15.12:0/1126675717 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff4440664a0 0x7ff444068960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0af000 tx=0x7ff45c0a7000 comp rx=0 tx=0).stop 2023-10-30T12:40:52.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.226+0000 7ff4673c4640 1 -- 172.21.15.12:0/1126675717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4600c7000 msgr2=0x7ff4601593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:52.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.226+0000 7ff4673c4640 1 --2- 172.21.15.12:0/1126675717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4600c7000 0x7ff4601593b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff45800efc0 tx=0x7ff45800c490 comp rx=0 tx=0).stop 2023-10-30T12:40:52.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.226+0000 7ff4673c4640 1 -- 172.21.15.12:0/1126675717 shutdown_connections 2023-10-30T12:40:52.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.226+0000 7ff4673c4640 1 --2- 172.21.15.12:0/1126675717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4600c7000 0x7ff4601593b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:52.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.226+0000 7ff4673c4640 1 --2- 172.21.15.12:0/1126675717 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff4440664a0 0x7ff444068960 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:52.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.226+0000 7ff4673c4640 1 --2- 172.21.15.12:0/1126675717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff460156f60 0x7ff46014df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:52.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.226+0000 7ff4673c4640 1 --2- 172.21.15.12:0/1126675717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff46006fd50 0x7ff4601f2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:52.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.226+0000 7ff4673c4640 1 -- 172.21.15.12:0/1126675717 >> 172.21.15.12:0/1126675717 conn(0x7ff460069e90 msgr2=0x7ff4600c81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:52.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.226+0000 7ff4673c4640 1 -- 172.21.15.12:0/1126675717 shutdown_connections 2023-10-30T12:40:52.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:52.230+0000 7ff4673c4640 1 -- 172.21.15.12:0/1126675717 wait complete. 2023-10-30T12:40:52.252 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:40:54.253 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:54.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.402+0000 7f197a794640 1 Processor -- start 2023-10-30T12:40:54.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.402+0000 7f197a794640 1 -- start start 2023-10-30T12:40:54.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.402+0000 7f197a794640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f197406e470 0x7f197406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:54.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.402+0000 7f197a794640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19740c8490 0x7f19740c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.402+0000 7f197a794640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197406afa0 con 0x7f1974153d70 2023-10-30T12:40:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.402+0000 7f197a794640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f197406d780 con 0x7f19740c8490 2023-10-30T12:40:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.402+0000 7f197a794640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1974154150 con 0x7f197406e470 2023-10-30T12:40:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.402+0000 7f1978d31640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19740c8490 0x7f19740c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.402+0000 7f1978d31640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19740c8490 0x7f19740c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40098/0 (socket says 172.21.15.12:40098) 2023-10-30T12:40:54.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.406+0000 7f1978d31640 1 -- 172.21.15.12:0/3739009619 learned_addr learned my addr 172.21.15.12:0/3739009619 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:54.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.406+0000 7f19737fe640 1 --2- 172.21.15.12:0/3739009619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f197406e470 0x7f197406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:54.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.406+0000 7f1978d31640 1 -- 172.21.15.12:0/3739009619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f197406e470 msgr2=0x7f197406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:54.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.406+0000 7f1978d31640 1 --2- 172.21.15.12:0/3739009619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f197406e470 0x7f197406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:54.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.406+0000 7f1978d31640 1 -- 172.21.15.12:0/3739009619 >> v1:172.21.15.154:6789/0 conn(0x7f1974153d70 legacy=0x7f19740cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:54.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.406+0000 7f1978d31640 1 -- 172.21.15.12:0/3739009619 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197406e030 con 0x7f19740c8490 2023-10-30T12:40:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.406+0000 7f1978d31640 1 --2- 172.21.15.12:0/3739009619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19740c8490 0x7f19740c8cc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f196c009e50 tx=0x7f196c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8dff2cb6a1e207 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.406+0000 7f1972ffd640 1 -- 172.21.15.12:0/3739009619 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c0a58a0 con 0x7f19740c8490 2023-10-30T12:40:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.406+0000 7f1972ffd640 1 -- 172.21.15.12:0/3739009619 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c0a5a00 con 0x7f19740c8490 2023-10-30T12:40:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.406+0000 7f1972ffd640 1 -- 172.21.15.12:0/3739009619 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c0a5d90 con 0x7f19740c8490 2023-10-30T12:40:54.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 -- 172.21.15.12:0/3739009619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19740c8490 msgr2=0x7f19740c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 --2- 172.21.15.12:0/3739009619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19740c8490 0x7f19740c8cc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f196c009e50 tx=0x7f196c01c9e0 comp rx=0 tx=0).stop 2023-10-30T12:40:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 -- 172.21.15.12:0/3739009619 shutdown_connections 2023-10-30T12:40:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 --2- 172.21.15.12:0/3739009619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19740c8490 0x7f19740c8cc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 --2- 172.21.15.12:0/3739009619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f197406e470 0x7f197406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 -- 172.21.15.12:0/3739009619 >> 172.21.15.12:0/3739009619 conn(0x7f1974071ea0 msgr2=0x7f1974065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 -- 172.21.15.12:0/3739009619 shutdown_connections 2023-10-30T12:40:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 -- 172.21.15.12:0/3739009619 wait complete. 2023-10-30T12:40:54.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 Processor -- start 2023-10-30T12:40:54.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 -- start start 2023-10-30T12:40:54.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f197406e470 0x7f1974147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:54.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19740c8490 0x7f1974145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:54.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1974153d70 0x7f19741493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:54.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1974143090 con 0x7f19740c8490 2023-10-30T12:40:54.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1974144210 con 0x7f197406e470 2023-10-30T12:40:54.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f197a794640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19741444f0 con 0x7f1974153d70 2023-10-30T12:40:54.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f19737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19740c8490 0x7f1974145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:54.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f19737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19740c8490 0x7f1974145be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40106/0 (socket says 172.21.15.12:40106) 2023-10-30T12:40:54.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f19737fe640 1 -- 172.21.15.12:0/1166437721 learned_addr learned my addr 172.21.15.12:0/1166437721 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:54.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f1978d31640 1 --2- 172.21.15.12:0/1166437721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1974153d70 0x7f19741493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:54.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f1973fff640 1 --2- 172.21.15.12:0/1166437721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f197406e470 0x7f1974147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:54.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f19737fe640 1 -- 172.21.15.12:0/1166437721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1974153d70 msgr2=0x7f19741493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f19737fe640 1 --2- 172.21.15.12:0/1166437721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1974153d70 0x7f19741493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f19737fe640 1 -- 172.21.15.12:0/1166437721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f197406e470 msgr2=0x7f1974147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f19737fe640 1 --2- 172.21.15.12:0/1166437721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f197406e470 0x7f1974147040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f19737fe640 1 -- 172.21.15.12:0/1166437721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c009b30 con 0x7f19740c8490 2023-10-30T12:40:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.410+0000 7f1973fff640 1 --2- 172.21.15.12:0/1166437721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f197406e470 0x7f1974147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:40:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.414+0000 7f19737fe640 1 --2- 172.21.15.12:0/1166437721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19740c8490 0x7f1974145be0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f196800d5f0 tx=0x7f196800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:54.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.414+0000 7f1970ff9640 1 -- 172.21.15.12:0/1166437721 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19680125f0 con 0x7f19740c8490 2023-10-30T12:40:54.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.414+0000 7f1970ff9640 1 -- 172.21.15.12:0/1166437721 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1968012750 con 0x7f19740c8490 2023-10-30T12:40:54.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.414+0000 7f197a794640 1 -- 172.21.15.12:0/1166437721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1974142760 con 0x7f19740c8490 2023-10-30T12:40:54.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.414+0000 7f1970ff9640 1 -- 172.21.15.12:0/1166437721 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968012ae0 con 0x7f19740c8490 2023-10-30T12:40:54.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.414+0000 7f197a794640 1 -- 172.21.15.12:0/1166437721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1974142d20 con 0x7f19740c8490 2023-10-30T12:40:54.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.414+0000 7f197a794640 1 -- 172.21.15.12:0/1166437721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1974143200 con 0x7f19740c8490 2023-10-30T12:40:54.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.414+0000 7f1970ff9640 1 -- 172.21.15.12:0/1166437721 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1968012c40 con 0x7f19740c8490 2023-10-30T12:40:54.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.414+0000 7f1970ff9640 1 --2- 172.21.15.12:0/1166437721 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19600664d0 0x7f1960068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:54.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.414+0000 7f1970ff9640 1 -- 172.21.15.12:0/1166437721 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f1968084410 con 0x7f19740c8490 2023-10-30T12:40:54.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.414+0000 7f1973fff640 1 --2- 172.21.15.12:0/1166437721 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19600664d0 0x7f1960068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:54.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.418+0000 7f1973fff640 1 --2- 172.21.15.12:0/1166437721 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19600664d0 0x7f1960068990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1964000f70 tx=0x7f1964009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:54.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.418+0000 7f1970ff9640 1 -- 172.21.15.12:0/1166437721 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f19680516d0 con 0x7f19740c8490 2023-10-30T12:40:54.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.610+0000 7f197a794640 1 -- 172.21.15.12:0/1166437721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f197404e170 con 0x7f19740c8490 2023-10-30T12:40:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.610+0000 7f1970ff9640 1 -- 172.21.15.12:0/1166437721 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5744 (secure 0 0 0) 0x7f1968007380 con 0x7f19740c8490 2023-10-30T12:40:54.614 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:54.615 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:51.480471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:reconnect","state_seq":40,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2023-10-30T12:40:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.610+0000 7f197a794640 1 -- 172.21.15.12:0/1166437721 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19600664d0 msgr2=0x7f1960068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.610+0000 7f197a794640 1 --2- 172.21.15.12:0/1166437721 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19600664d0 0x7f1960068990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1964000f70 tx=0x7f1964009040 comp rx=0 tx=0).stop 2023-10-30T12:40:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.610+0000 7f197a794640 1 -- 172.21.15.12:0/1166437721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19740c8490 msgr2=0x7f1974145be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.610+0000 7f197a794640 1 --2- 172.21.15.12:0/1166437721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19740c8490 0x7f1974145be0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f196800d5f0 tx=0x7f196800dab0 comp rx=0 tx=0).stop 2023-10-30T12:40:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.610+0000 7f197a794640 1 -- 172.21.15.12:0/1166437721 shutdown_connections 2023-10-30T12:40:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.610+0000 7f197a794640 1 --2- 172.21.15.12:0/1166437721 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19600664d0 0x7f1960068990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.610+0000 7f197a794640 1 --2- 172.21.15.12:0/1166437721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1974153d70 0x7f19741493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.610+0000 7f197a794640 1 --2- 172.21.15.12:0/1166437721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19740c8490 0x7f1974145be0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.610+0000 7f197a794640 1 --2- 172.21.15.12:0/1166437721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f197406e470 0x7f1974147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.610+0000 7f197a794640 1 -- 172.21.15.12:0/1166437721 >> 172.21.15.12:0/1166437721 conn(0x7f1974071ea0 msgr2=0x7f19740c7520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.610+0000 7f197a794640 1 -- 172.21.15.12:0/1166437721 shutdown_connections 2023-10-30T12:40:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:54.610+0000 7f197a794640 1 -- 172.21.15.12:0/1166437721 wait complete. 2023-10-30T12:40:54.636 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 28, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 28, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4282": { "addr": "172.21.15.154:6839/1517672481", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1517672481, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 1517672481, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 4282, "incarnation": 10, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 7 }, "gid_4285": { "addr": "172.21.15.12:6836/291392212", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 291392212, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 291392212, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 4285, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 6 }, "gid_4971": { "addr": "172.21.15.12:6837/3124135109", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 3124135109, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 3124135109, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4971, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:reconnect", "state_seq": 40 } }, "last_failure": 0, "last_failure_osd_epoch": 38, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:40:51.480471+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4285, "mds_1": 4282, "mds_2": 4971 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:40:56.637 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:56.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.786+0000 7ff0956ea640 1 Processor -- start 2023-10-30T12:40:56.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.786+0000 7ff0956ea640 1 -- start start 2023-10-30T12:40:56.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.786+0000 7ff0956ea640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff0900bfec0 0x7ff0900c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:56.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.786+0000 7ff0956ea640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff090071290 con 0x7ff09006a720 2023-10-30T12:40:56.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.786+0000 7ff0956ea640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff090071570 con 0x7ff090159e60 2023-10-30T12:40:56.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.786+0000 7ff0956ea640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff090071850 con 0x7ff0900bfec0 2023-10-30T12:40:56.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.786+0000 7ff087fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff0900bfec0 0x7ff0900c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:56.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.786+0000 7ff087fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff0900bfec0 0x7ff0900c1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40112/0 (socket says 172.21.15.12:40112) 2023-10-30T12:40:56.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.786+0000 7ff08f7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff09006a720 0x7ff090160610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:46226/0 (socket says 172.21.15.12:46226) 2023-10-30T12:40:56.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.786+0000 7ff087fff640 1 -- 172.21.15.12:0/3315455209 learned_addr learned my addr 172.21.15.12:0/3315455209 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:56.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.790+0000 7ff087fff640 1 -- 172.21.15.12:0/3315455209 >> v1:172.21.15.154:6789/0 conn(0x7ff090159e60 legacy=0x7ff09006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:56.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.790+0000 7ff087fff640 1 -- 172.21.15.12:0/3315455209 >> v1:172.21.15.12:6789/0 conn(0x7ff09006a720 legacy=0x7ff090160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:56.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.790+0000 7ff087fff640 1 -- 172.21.15.12:0/3315455209 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff090071b30 con 0x7ff0900bfec0 2023-10-30T12:40:56.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.790+0000 7ff08e7fc640 1 -- 172.21.15.12:0/3315455209 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795977843 0 0) 0x7ff090071290 con 0x7ff09006a720 2023-10-30T12:40:56.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.790+0000 7ff087fff640 1 --2- 172.21.15.12:0/3315455209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0900bfec0 0x7ff0900c1a40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff078009870 tx=0x7ff07801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf01b94718761ec server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:56.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.790+0000 7ff08e7fc640 1 -- 172.21.15.12:0/3315455209 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0780a8a30 con 0x7ff0900bfec0 2023-10-30T12:40:56.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.790+0000 7ff08e7fc640 1 -- 172.21.15.12:0/3315455209 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0780a8b90 con 0x7ff0900bfec0 2023-10-30T12:40:56.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.790+0000 7ff08e7fc640 1 -- 172.21.15.12:0/3315455209 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0780b0440 con 0x7ff0900bfec0 2023-10-30T12:40:56.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.790+0000 7ff0956ea640 1 -- 172.21.15.12:0/3315455209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0900bfec0 msgr2=0x7ff0900c1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:56.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.790+0000 7ff0956ea640 1 --2- 172.21.15.12:0/3315455209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0900bfec0 0x7ff0900c1a40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff078009870 tx=0x7ff07801ed30 comp rx=0 tx=0).stop 2023-10-30T12:40:56.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.790+0000 7ff0956ea640 1 -- 172.21.15.12:0/3315455209 shutdown_connections 2023-10-30T12:40:56.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.790+0000 7ff0956ea640 1 --2- 172.21.15.12:0/3315455209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0900bfec0 0x7ff0900c1a40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:56.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.790+0000 7ff0956ea640 1 -- 172.21.15.12:0/3315455209 >> 172.21.15.12:0/3315455209 conn(0x7ff0900c7910 msgr2=0x7ff0900c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:56.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.790+0000 7ff0956ea640 1 -- 172.21.15.12:0/3315455209 shutdown_connections 2023-10-30T12:40:56.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff0956ea640 1 -- 172.21.15.12:0/3315455209 wait complete. 2023-10-30T12:40:56.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff0956ea640 1 Processor -- start 2023-10-30T12:40:56.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff0956ea640 1 -- start start 2023-10-30T12:40:56.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff0956ea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff09006a720 0x7ff0901efad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:56.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff0956ea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0900bfec0 0x7ff0901df300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:56.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff0956ea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff090159e60 0x7ff0900ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:56.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff0956ea640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900af250 con 0x7ff090159e60 2023-10-30T12:40:56.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff0956ea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff090156e50 con 0x7ff0900bfec0 2023-10-30T12:40:56.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff0956ea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff090158bd0 con 0x7ff09006a720 2023-10-30T12:40:56.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff08f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff090159e60 0x7ff0900ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:56.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff08f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff090159e60 0x7ff0900ad8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40120/0 (socket says 172.21.15.12:40120) 2023-10-30T12:40:56.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff08f7fe640 1 -- 172.21.15.12:0/4162768898 learned_addr learned my addr 172.21.15.12:0/4162768898 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:56.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff087fff640 1 --2- 172.21.15.12:0/4162768898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0900bfec0 0x7ff0901df300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:56.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff08effd640 1 --2- 172.21.15.12:0/4162768898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff09006a720 0x7ff0901efad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:56.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff08f7fe640 1 -- 172.21.15.12:0/4162768898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff09006a720 msgr2=0x7ff0901efad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:56.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff08f7fe640 1 --2- 172.21.15.12:0/4162768898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff09006a720 0x7ff0901efad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:56.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff08f7fe640 1 -- 172.21.15.12:0/4162768898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0900bfec0 msgr2=0x7ff0901df300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:56.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff08f7fe640 1 --2- 172.21.15.12:0/4162768898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0900bfec0 0x7ff0901df300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:56.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff08f7fe640 1 -- 172.21.15.12:0/4162768898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0780094d0 con 0x7ff090159e60 2023-10-30T12:40:56.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff08f7fe640 1 --2- 172.21.15.12:0/4162768898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff090159e60 0x7ff0900ad8d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff088001570 tx=0x7ff08800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:56.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff0877fe640 1 -- 172.21.15.12:0/4162768898 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08800a640 con 0x7ff090159e60 2023-10-30T12:40:56.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff0877fe640 1 -- 172.21.15.12:0/4162768898 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08800a7c0 con 0x7ff090159e60 2023-10-30T12:40:56.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff0877fe640 1 -- 172.21.15.12:0/4162768898 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08800ab70 con 0x7ff090159e60 2023-10-30T12:40:56.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.794+0000 7ff0956ea640 1 -- 172.21.15.12:0/4162768898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff090158eb0 con 0x7ff090159e60 2023-10-30T12:40:56.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.798+0000 7ff0956ea640 1 -- 172.21.15.12:0/4162768898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff090157380 con 0x7ff090159e60 2023-10-30T12:40:56.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.798+0000 7ff0956ea640 1 -- 172.21.15.12:0/4162768898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09004e170 con 0x7ff090159e60 2023-10-30T12:40:56.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.798+0000 7ff0877fe640 1 -- 172.21.15.12:0/4162768898 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff08800acd0 con 0x7ff090159e60 2023-10-30T12:40:56.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.798+0000 7ff0877fe640 1 --2- 172.21.15.12:0/4162768898 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff0700664a0 0x7ff070068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:56.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.798+0000 7ff0877fe640 1 -- 172.21.15.12:0/4162768898 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5997+0+0 (secure 0 0 0) 0x7ff088083610 con 0x7ff090159e60 2023-10-30T12:40:56.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.798+0000 7ff08effd640 1 --2- 172.21.15.12:0/4162768898 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff0700664a0 0x7ff070068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:56.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.802+0000 7ff08effd640 1 --2- 172.21.15.12:0/4162768898 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff0700664a0 0x7ff070068960 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff080000f70 tx=0x7ff080009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:56.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.802+0000 7ff0877fe640 1 -- 172.21.15.12:0/4162768898 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff0880508d0 con 0x7ff090159e60 2023-10-30T12:40:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.994+0000 7ff0956ea640 1 -- 172.21.15.12:0/4162768898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff090066510 con 0x7ff090159e60 2023-10-30T12:40:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.994+0000 7ff0877fe640 1 -- 172.21.15.12:0/4162768898 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5744 (secure 0 0 0) 0x7ff088018090 con 0x7ff090159e60 2023-10-30T12:40:57.001 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:57.001 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:51.480471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:reconnect","state_seq":40,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:57.002 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2023-10-30T12:40:57.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.998+0000 7ff0956ea640 1 -- 172.21.15.12:0/4162768898 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff0700664a0 msgr2=0x7ff070068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:57.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.998+0000 7ff0956ea640 1 --2- 172.21.15.12:0/4162768898 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff0700664a0 0x7ff070068960 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff080000f70 tx=0x7ff080009040 comp rx=0 tx=0).stop 2023-10-30T12:40:57.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.998+0000 7ff0956ea640 1 -- 172.21.15.12:0/4162768898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff090159e60 msgr2=0x7ff0900ad8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:57.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.998+0000 7ff0956ea640 1 --2- 172.21.15.12:0/4162768898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff090159e60 0x7ff0900ad8d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff088001570 tx=0x7ff08800cca0 comp rx=0 tx=0).stop 2023-10-30T12:40:57.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.998+0000 7ff0956ea640 1 -- 172.21.15.12:0/4162768898 shutdown_connections 2023-10-30T12:40:57.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.998+0000 7ff0956ea640 1 --2- 172.21.15.12:0/4162768898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff090159e60 0x7ff0900ad8d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:57.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.998+0000 7ff0956ea640 1 --2- 172.21.15.12:0/4162768898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0900bfec0 0x7ff0901df300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:57.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.998+0000 7ff0956ea640 1 --2- 172.21.15.12:0/4162768898 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff0700664a0 0x7ff070068960 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:57.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.998+0000 7ff0956ea640 1 --2- 172.21.15.12:0/4162768898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff09006a720 0x7ff0901efad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:57.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.998+0000 7ff0956ea640 1 -- 172.21.15.12:0/4162768898 >> 172.21.15.12:0/4162768898 conn(0x7ff0900c7910 msgr2=0x7ff09006f540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:57.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.998+0000 7ff0956ea640 1 -- 172.21.15.12:0/4162768898 shutdown_connections 2023-10-30T12:40:57.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:56.998+0000 7ff0956ea640 1 -- 172.21.15.12:0/4162768898 wait complete. 2023-10-30T12:40:57.021 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:40:59.022 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:40:59.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.170+0000 7f2ce81c4640 1 Processor -- start 2023-10-30T12:40:59.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.170+0000 7f2ce81c4640 1 -- start start 2023-10-30T12:40:59.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.174+0000 7f2ce81c4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ce015d6f0 0x7f2ce015a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:59.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.174+0000 7f2ce81c4640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2ce00cf160 0x7f2ce00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:59.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.174+0000 7f2ce81c4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce00cb800 con 0x7f2ce01538a0 2023-10-30T12:40:59.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.174+0000 7f2ce81c4640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce00cb330 con 0x7f2ce00cf160 2023-10-30T12:40:59.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.174+0000 7f2ce81c4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce00cad70 con 0x7f2ce015d6f0 2023-10-30T12:40:59.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.174+0000 7f2ce6761640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2ce00cf160 0x7f2ce00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:59.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.174+0000 7f2ce6761640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2ce00cf160 0x7f2ce00cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40126/0 (socket says 172.21.15.12:40126) 2023-10-30T12:40:59.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.174+0000 7f2ce6761640 1 -- 172.21.15.12:0/2284237564 learned_addr learned my addr 172.21.15.12:0/2284237564 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:59.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.174+0000 7f2ce575f640 1 --2- 172.21.15.12:0/2284237564 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ce015d6f0 0x7f2ce015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.174+0000 7f2ce6761640 1 -- 172.21.15.12:0/2284237564 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ce015d6f0 msgr2=0x7f2ce015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:40:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.174+0000 7f2ce6761640 1 --2- 172.21.15.12:0/2284237564 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ce015d6f0 0x7f2ce015a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.174+0000 7f2ce6761640 1 -- 172.21.15.12:0/2284237564 >> v1:172.21.15.154:6789/0 conn(0x7f2ce01538a0 legacy=0x7f2ce00ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.174+0000 7f2ce6761640 1 -- 172.21.15.12:0/2284237564 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce00cb050 con 0x7f2ce00cf160 2023-10-30T12:40:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.178+0000 7f2ce6761640 1 --2- 172.21.15.12:0/2284237564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce00cf160 0x7f2ce00cf590 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009e50 tx=0x7f2cd801f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9843dbe2beeb48 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.178+0000 7f2ce4f5e640 1 -- 172.21.15.12:0/2284237564 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd80af400 con 0x7f2ce00cf160 2023-10-30T12:40:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.178+0000 7f2ce4f5e640 1 -- 172.21.15.12:0/2284237564 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd80ab050 con 0x7f2ce00cf160 2023-10-30T12:40:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.178+0000 7f2ce4f5e640 1 -- 172.21.15.12:0/2284237564 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd80af780 con 0x7f2ce00cf160 2023-10-30T12:40:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.178+0000 7f2ce81c4640 1 -- 172.21.15.12:0/2284237564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce00cf160 msgr2=0x7f2ce00cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.178+0000 7f2ce81c4640 1 --2- 172.21.15.12:0/2284237564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce00cf160 0x7f2ce00cf590 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009e50 tx=0x7f2cd801f1f0 comp rx=0 tx=0).stop 2023-10-30T12:40:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.178+0000 7f2ce81c4640 1 -- 172.21.15.12:0/2284237564 shutdown_connections 2023-10-30T12:40:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.178+0000 7f2ce81c4640 1 --2- 172.21.15.12:0/2284237564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce00cf160 0x7f2ce00cf590 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.178+0000 7f2ce81c4640 1 --2- 172.21.15.12:0/2284237564 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ce015d6f0 0x7f2ce015a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.178+0000 7f2ce81c4640 1 -- 172.21.15.12:0/2284237564 >> 172.21.15.12:0/2284237564 conn(0x7f2ce006bfe0 msgr2=0x7f2ce006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:59.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.178+0000 7f2ce81c4640 1 -- 172.21.15.12:0/2284237564 shutdown_connections 2023-10-30T12:40:59.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.178+0000 7f2ce81c4640 1 -- 172.21.15.12:0/2284237564 wait complete. 2023-10-30T12:40:59.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.178+0000 7f2ce81c4640 1 Processor -- start 2023-10-30T12:40:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.178+0000 7f2ce81c4640 1 -- start start 2023-10-30T12:40:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce81c4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ce00cf160 0x7f2ce01e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce81c4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ce01538a0 0x7f2ce00b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:59.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce81c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce015d6f0 0x7f2ce01f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:59.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce81c4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce01e3180 con 0x7f2ce015d6f0 2023-10-30T12:40:59.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce81c4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce01e2a10 con 0x7f2ce01538a0 2023-10-30T12:40:59.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce81c4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce01e2cf0 con 0x7f2ce00cf160 2023-10-30T12:40:59.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce6761640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce015d6f0 0x7f2ce01f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:59.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce6761640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce015d6f0 0x7f2ce01f2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40130/0 (socket says 172.21.15.12:40130) 2023-10-30T12:40:59.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce6761640 1 -- 172.21.15.12:0/2013788458 learned_addr learned my addr 172.21.15.12:0/2013788458 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:40:59.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce5f60640 1 --2- 172.21.15.12:0/2013788458 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ce00cf160 0x7f2ce01e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:59.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce575f640 1 --2- 172.21.15.12:0/2013788458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ce01538a0 0x7f2ce00b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:59.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce6761640 1 -- 172.21.15.12:0/2013788458 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ce00cf160 msgr2=0x7f2ce01e49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce6761640 1 --2- 172.21.15.12:0/2013788458 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ce00cf160 0x7f2ce01e49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce6761640 1 -- 172.21.15.12:0/2013788458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ce01538a0 msgr2=0x7f2ce00b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce6761640 1 --2- 172.21.15.12:0/2013788458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ce01538a0 0x7f2ce00b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce6761640 1 -- 172.21.15.12:0/2013788458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd8009b30 con 0x7f2ce015d6f0 2023-10-30T12:40:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce575f640 1 --2- 172.21.15.12:0/2013788458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ce01538a0 0x7f2ce00b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:40:59.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce6761640 1 --2- 172.21.15.12:0/2013788458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce015d6f0 0x7f2ce01f2250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2cd801ce20 tx=0x7f2cd801bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:59.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2cd67fc640 1 -- 172.21.15.12:0/2013788458 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd80c0070 con 0x7f2ce015d6f0 2023-10-30T12:40:59.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2cd67fc640 1 -- 172.21.15.12:0/2013788458 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd8009510 con 0x7f2ce015d6f0 2023-10-30T12:40:59.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce81c4640 1 -- 172.21.15.12:0/2013788458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce00b4a90 con 0x7f2ce015d6f0 2023-10-30T12:40:59.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2cd67fc640 1 -- 172.21.15.12:0/2013788458 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd80afd10 con 0x7f2ce015d6f0 2023-10-30T12:40:59.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce81c4640 1 -- 172.21.15.12:0/2013788458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce00b4d30 con 0x7f2ce015d6f0 2023-10-30T12:40:59.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2ce81c4640 1 -- 172.21.15.12:0/2013788458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce00b2a50 con 0x7f2ce015d6f0 2023-10-30T12:40:59.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.182+0000 7f2cd67fc640 1 -- 172.21.15.12:0/2013788458 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2cd80b6040 con 0x7f2ce015d6f0 2023-10-30T12:40:59.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.186+0000 7f2cd67fc640 1 --2- 172.21.15.12:0/2013788458 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2cc40664a0 0x7f2cc4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:40:59.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.186+0000 7f2cd67fc640 1 -- 172.21.15.12:0/2013788458 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f2cd8121960 con 0x7f2ce015d6f0 2023-10-30T12:40:59.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.186+0000 7f2ce5f60640 1 --2- 172.21.15.12:0/2013788458 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2cc40664a0 0x7f2cc4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:40:59.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.186+0000 7f2ce5f60640 1 --2- 172.21.15.12:0/2013788458 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2cc40664a0 0x7f2cc4068960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc000f70 tx=0x7f2cdc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:40:59.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.190+0000 7f2cd67fc640 1 -- 172.21.15.12:0/2013788458 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2cd80eed70 con 0x7f2ce015d6f0 2023-10-30T12:40:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.378+0000 7f2ce81c4640 1 -- 172.21.15.12:0/2013788458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ce01e2e60 con 0x7f2ce015d6f0 2023-10-30T12:40:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.382+0000 7f2cd67fc640 1 -- 172.21.15.12:0/2013788458 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5741 (secure 0 0 0) 0x7f2cd80f2390 con 0x7f2ce015d6f0 2023-10-30T12:40:59.386 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:40:59.386 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:56.499621+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:rejoin","state_seq":42,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:40:59.387 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2023-10-30T12:40:59.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.382+0000 7f2ce81c4640 1 -- 172.21.15.12:0/2013788458 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2cc40664a0 msgr2=0x7f2cc4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:59.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.382+0000 7f2ce81c4640 1 --2- 172.21.15.12:0/2013788458 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2cc40664a0 0x7f2cc4068960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc000f70 tx=0x7f2cdc009040 comp rx=0 tx=0).stop 2023-10-30T12:40:59.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.382+0000 7f2ce81c4640 1 -- 172.21.15.12:0/2013788458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce015d6f0 msgr2=0x7f2ce01f2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:40:59.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.382+0000 7f2ce81c4640 1 --2- 172.21.15.12:0/2013788458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce015d6f0 0x7f2ce01f2250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2cd801ce20 tx=0x7f2cd801bf90 comp rx=0 tx=0).stop 2023-10-30T12:40:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.382+0000 7f2ce81c4640 1 -- 172.21.15.12:0/2013788458 shutdown_connections 2023-10-30T12:40:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.382+0000 7f2ce81c4640 1 --2- 172.21.15.12:0/2013788458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce015d6f0 0x7f2ce01f2250 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.382+0000 7f2ce81c4640 1 --2- 172.21.15.12:0/2013788458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ce01538a0 0x7f2ce00b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.382+0000 7f2ce81c4640 1 --2- 172.21.15.12:0/2013788458 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2cc40664a0 0x7f2cc4068960 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.382+0000 7f2ce81c4640 1 --2- 172.21.15.12:0/2013788458 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ce00cf160 0x7f2ce01e49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:40:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.382+0000 7f2ce81c4640 1 -- 172.21.15.12:0/2013788458 >> 172.21.15.12:0/2013788458 conn(0x7f2ce006bfe0 msgr2=0x7f2ce0151cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:40:59.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.382+0000 7f2ce81c4640 1 -- 172.21.15.12:0/2013788458 shutdown_connections 2023-10-30T12:40:59.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:40:59.382+0000 7f2ce81c4640 1 -- 172.21.15.12:0/2013788458 wait complete. 2023-10-30T12:40:59.407 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:41:01.408 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:41:01.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.558+0000 7fb16fd3e640 1 Processor -- start 2023-10-30T12:41:01.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.558+0000 7fb16fd3e640 1 -- start start 2023-10-30T12:41:01.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.558+0000 7fb16fd3e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb1681538a0 0x7fb16815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:01.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.558+0000 7fb16fd3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1680cf160 0x7fb1680cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:01.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.558+0000 7fb16fd3e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1680cb800 con 0x7fb1680ca0e0 2023-10-30T12:41:01.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.558+0000 7fb16fd3e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb16815dc30 con 0x7fb1681538a0 2023-10-30T12:41:01.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.558+0000 7fb16fd3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680cb330 con 0x7fb1680cf160 2023-10-30T12:41:01.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.558+0000 7fb16dada640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb1681538a0 0x7fb16815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:01.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.558+0000 7fb16e2db640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1680cf160 0x7fb1680cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:01.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.558+0000 7fb16d2d9640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb1680ca0e0 0x7fb16815a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:54916/0 (socket says 172.21.15.12:54916) 2023-10-30T12:41:01.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.558+0000 7fb16d2d9640 1 -- 172.21.15.12:0/2148503039 learned_addr learned my addr 172.21.15.12:0/2148503039 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:41:01.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.558+0000 7fb16dada640 1 -- 172.21.15.12:0/2148503039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1680cf160 msgr2=0x7fb1680cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:41:01.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.558+0000 7fb16dada640 1 --2- 172.21.15.12:0/2148503039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1680cf160 0x7fb1680cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:01.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.558+0000 7fb16dada640 1 -- 172.21.15.12:0/2148503039 >> v1:172.21.15.154:6789/0 conn(0x7fb1680ca0e0 legacy=0x7fb16815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:01.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.558+0000 7fb16dada640 1 -- 172.21.15.12:0/2148503039 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680cad70 con 0x7fb1681538a0 2023-10-30T12:41:01.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.562+0000 7fb16dada640 1 --2- 172.21.15.12:0/2148503039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1681538a0 0x7fb16815d6f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb1640097c0 tx=0x7fb16401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9df45d5bb46b7e29 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:41:01.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.562+0000 7fb16cad8640 1 -- 172.21.15.12:0/2148503039 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1640a8970 con 0x7fb1681538a0 2023-10-30T12:41:01.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.562+0000 7fb16cad8640 1 -- 172.21.15.12:0/2148503039 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1640a8ad0 con 0x7fb1681538a0 2023-10-30T12:41:01.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.562+0000 7fb16cad8640 1 -- 172.21.15.12:0/2148503039 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1640a8e60 con 0x7fb1681538a0 2023-10-30T12:41:01.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.562+0000 7fb16fd3e640 1 -- 172.21.15.12:0/2148503039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1681538a0 msgr2=0x7fb16815d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:01.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.562+0000 7fb16fd3e640 1 --2- 172.21.15.12:0/2148503039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1681538a0 0x7fb16815d6f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb1640097c0 tx=0x7fb16401ec70 comp rx=0 tx=0).stop 2023-10-30T12:41:01.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.562+0000 7fb16fd3e640 1 -- 172.21.15.12:0/2148503039 shutdown_connections 2023-10-30T12:41:01.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.562+0000 7fb16fd3e640 1 --2- 172.21.15.12:0/2148503039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1680cf160 0x7fb1680cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:01.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.562+0000 7fb16fd3e640 1 --2- 172.21.15.12:0/2148503039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1681538a0 0x7fb16815d6f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:01.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.562+0000 7fb16fd3e640 1 -- 172.21.15.12:0/2148503039 >> 172.21.15.12:0/2148503039 conn(0x7fb16806bfe0 msgr2=0x7fb16806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:41:01.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.562+0000 7fb16fd3e640 1 -- 172.21.15.12:0/2148503039 shutdown_connections 2023-10-30T12:41:01.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.562+0000 7fb16fd3e640 1 -- 172.21.15.12:0/2148503039 wait complete. 2023-10-30T12:41:01.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16fd3e640 1 Processor -- start 2023-10-30T12:41:01.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16fd3e640 1 -- start start 2023-10-30T12:41:01.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16fd3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1680ca0e0 0x7fb1681592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:01.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16fd3e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1680cf160 0x7fb16815b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:01.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16fd3e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1681538a0 0x7fb16815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:01.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16fd3e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb168158ec0 con 0x7fb1681538a0 2023-10-30T12:41:01.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16fd3e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1681e5240 con 0x7fb1680cf160 2023-10-30T12:41:01.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16fd3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb168157560 con 0x7fb1680ca0e0 2023-10-30T12:41:01.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16e2db640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1681538a0 0x7fb16815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:01.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16e2db640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1681538a0 0x7fb16815ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43460/0 (socket says 172.21.15.12:43460) 2023-10-30T12:41:01.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16e2db640 1 -- 172.21.15.12:0/1474297362 learned_addr learned my addr 172.21.15.12:0/1474297362 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:41:01.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16d2d9640 1 --2- 172.21.15.12:0/1474297362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1680cf160 0x7fb16815b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:01.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16dada640 1 --2- 172.21.15.12:0/1474297362 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1680ca0e0 0x7fb1681592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:01.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16e2db640 1 -- 172.21.15.12:0/1474297362 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1680ca0e0 msgr2=0x7fb1681592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:01.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16e2db640 1 --2- 172.21.15.12:0/1474297362 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1680ca0e0 0x7fb1681592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:01.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16e2db640 1 -- 172.21.15.12:0/1474297362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1680cf160 msgr2=0x7fb16815b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:01.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16e2db640 1 --2- 172.21.15.12:0/1474297362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1680cf160 0x7fb16815b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:01.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16e2db640 1 -- 172.21.15.12:0/1474297362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb164009420 con 0x7fb1681538a0 2023-10-30T12:41:01.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16e2db640 1 --2- 172.21.15.12:0/1474297362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1681538a0 0x7fb16815ab50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb16000efc0 tx=0x7fb16000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:41:01.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb15e7fc640 1 -- 172.21.15.12:0/1474297362 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160009b10 con 0x7fb1681538a0 2023-10-30T12:41:01.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb15e7fc640 1 -- 172.21.15.12:0/1474297362 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16000f040 con 0x7fb1681538a0 2023-10-30T12:41:01.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb15e7fc640 1 -- 172.21.15.12:0/1474297362 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1600146c0 con 0x7fb1681538a0 2023-10-30T12:41:01.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16fd3e640 1 -- 172.21.15.12:0/1474297362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb168157820 con 0x7fb1681538a0 2023-10-30T12:41:01.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.566+0000 7fb16fd3e640 1 -- 172.21.15.12:0/1474297362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb168158800 con 0x7fb1681538a0 2023-10-30T12:41:01.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.570+0000 7fb16fd3e640 1 -- 172.21.15.12:0/1474297362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1680b2a50 con 0x7fb1681538a0 2023-10-30T12:41:01.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.570+0000 7fb15e7fc640 1 -- 172.21.15.12:0/1474297362 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb160014820 con 0x7fb1681538a0 2023-10-30T12:41:01.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.570+0000 7fb15e7fc640 1 --2- 172.21.15.12:0/1474297362 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb14c0664a0 0x7fb14c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:01.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.570+0000 7fb15e7fc640 1 -- 172.21.15.12:0/1474297362 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fb160084920 con 0x7fb1681538a0 2023-10-30T12:41:01.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.570+0000 7fb16dada640 1 --2- 172.21.15.12:0/1474297362 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb14c0664a0 0x7fb14c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:01.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.574+0000 7fb16dada640 1 --2- 172.21.15.12:0/1474297362 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb14c0664a0 0x7fb14c068960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb1640b2000 tx=0x7fb164008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:41:01.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.574+0000 7fb15e7fc640 1 -- 172.21.15.12:0/1474297362 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb160051d20 con 0x7fb1681538a0 2023-10-30T12:41:01.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.762+0000 7fb16fd3e640 1 -- 172.21.15.12:0/1474297362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb168159000 con 0x7fb1681538a0 2023-10-30T12:41:01.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.762+0000 7fb15e7fc640 1 -- 172.21.15.12:0/1474297362 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5741 (secure 0 0 0) 0x7fb168159000 con 0x7fb1681538a0 2023-10-30T12:41:01.769 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:41:01.770 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:40:56.499621+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:rejoin","state_seq":42,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:41:01.770 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2023-10-30T12:41:01.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.766+0000 7fb16fd3e640 1 -- 172.21.15.12:0/1474297362 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb14c0664a0 msgr2=0x7fb14c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:01.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.766+0000 7fb16fd3e640 1 --2- 172.21.15.12:0/1474297362 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb14c0664a0 0x7fb14c068960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb1640b2000 tx=0x7fb164008040 comp rx=0 tx=0).stop 2023-10-30T12:41:01.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.766+0000 7fb16fd3e640 1 -- 172.21.15.12:0/1474297362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1681538a0 msgr2=0x7fb16815ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:01.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.766+0000 7fb16fd3e640 1 --2- 172.21.15.12:0/1474297362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1681538a0 0x7fb16815ab50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb16000efc0 tx=0x7fb16000c490 comp rx=0 tx=0).stop 2023-10-30T12:41:01.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.766+0000 7fb16fd3e640 1 -- 172.21.15.12:0/1474297362 shutdown_connections 2023-10-30T12:41:01.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.766+0000 7fb16fd3e640 1 --2- 172.21.15.12:0/1474297362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1681538a0 0x7fb16815ab50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:01.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.766+0000 7fb16fd3e640 1 --2- 172.21.15.12:0/1474297362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1680cf160 0x7fb16815b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:01.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.766+0000 7fb16fd3e640 1 --2- 172.21.15.12:0/1474297362 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb14c0664a0 0x7fb14c068960 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:01.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.766+0000 7fb16fd3e640 1 --2- 172.21.15.12:0/1474297362 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1680ca0e0 0x7fb1681592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:01.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.766+0000 7fb16fd3e640 1 -- 172.21.15.12:0/1474297362 >> 172.21.15.12:0/1474297362 conn(0x7fb16806bfe0 msgr2=0x7fb16814fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:41:01.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.766+0000 7fb16fd3e640 1 -- 172.21.15.12:0/1474297362 shutdown_connections 2023-10-30T12:41:01.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:01.766+0000 7fb16fd3e640 1 -- 172.21.15.12:0/1474297362 wait complete. 2023-10-30T12:41:01.790 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:41:03.792 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:41:03.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.942+0000 7f59c1a9c640 1 Processor -- start 2023-10-30T12:41:03.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.942+0000 7f59c1a9c640 1 -- start start 2023-10-30T12:41:03.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.942+0000 7f59c1a9c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f59bc15d6f0 0x7f59bc15a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.942+0000 7f59c1a9c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59bc0cb800 con 0x7f59bc1538a0 2023-10-30T12:41:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.942+0000 7f59c1a9c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59bc0cb330 con 0x7f59bc0cf160 2023-10-30T12:41:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.942+0000 7f59c1a9c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59bc0cad70 con 0x7f59bc15d6f0 2023-10-30T12:41:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.942+0000 7f59baffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f59bc15d6f0 0x7f59bc15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:03.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.942+0000 7f59bb7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f59bc1538a0 0x7f59bc0ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:53838/0 (socket says 172.21.15.12:53838) 2023-10-30T12:41:03.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.942+0000 7f59bb7fe640 1 -- 172.21.15.12:0/1767872074 learned_addr learned my addr 172.21.15.12:0/1767872074 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:41:03.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.942+0000 7f59baffd640 1 -- 172.21.15.12:0/1767872074 >> v1:172.21.15.154:6789/0 conn(0x7f59bc0cf160 legacy=0x7f59bc164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:03.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.942+0000 7f59baffd640 1 -- 172.21.15.12:0/1767872074 >> v1:172.21.15.12:6789/0 conn(0x7f59bc1538a0 legacy=0x7f59bc0ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:03.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.942+0000 7f59baffd640 1 -- 172.21.15.12:0/1767872074 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc0cb050 con 0x7f59bc15d6f0 2023-10-30T12:41:03.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.942+0000 7f59ba7fc640 1 -- 172.21.15.12:0/1767872074 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201278419 0 0) 0x7f59bc0cb800 con 0x7f59bc1538a0 2023-10-30T12:41:03.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.946+0000 7f59baffd640 1 --2- 172.21.15.12:0/1767872074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc15d6f0 0x7f59bc15a090 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f59b00097c0 tx=0x7f59b001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b1e00c37128590 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:41:03.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.946+0000 7f59ba7fc640 1 -- 172.21.15.12:0/1767872074 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b00a8970 con 0x7f59bc15d6f0 2023-10-30T12:41:03.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.946+0000 7f59ba7fc640 1 -- 172.21.15.12:0/1767872074 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b00a8ad0 con 0x7f59bc15d6f0 2023-10-30T12:41:03.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.946+0000 7f59ba7fc640 1 -- 172.21.15.12:0/1767872074 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b00a8e60 con 0x7f59bc15d6f0 2023-10-30T12:41:03.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.946+0000 7f59c1a9c640 1 -- 172.21.15.12:0/1767872074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc15d6f0 msgr2=0x7f59bc15a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:03.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.946+0000 7f59c1a9c640 1 --2- 172.21.15.12:0/1767872074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc15d6f0 0x7f59bc15a090 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f59b00097c0 tx=0x7f59b001ec70 comp rx=0 tx=0).stop 2023-10-30T12:41:03.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.946+0000 7f59c1a9c640 1 -- 172.21.15.12:0/1767872074 shutdown_connections 2023-10-30T12:41:03.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.946+0000 7f59c1a9c640 1 --2- 172.21.15.12:0/1767872074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc15d6f0 0x7f59bc15a090 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:03.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.946+0000 7f59c1a9c640 1 -- 172.21.15.12:0/1767872074 >> 172.21.15.12:0/1767872074 conn(0x7f59bc06bfe0 msgr2=0x7f59bc06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:41:03.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.946+0000 7f59c1a9c640 1 -- 172.21.15.12:0/1767872074 shutdown_connections 2023-10-30T12:41:03.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.946+0000 7f59c1a9c640 1 -- 172.21.15.12:0/1767872074 wait complete. 2023-10-30T12:41:03.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59c1a9c640 1 Processor -- start 2023-10-30T12:41:03.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59c1a9c640 1 -- start start 2023-10-30T12:41:03.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59c1a9c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59bc0cf160 0x7f59bc1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:03.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59c1a9c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59bc1538a0 0x7f59bc15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:03.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59c1a9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc15d6f0 0x7f59bc1e3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:03.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59c1a9c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc15c500 con 0x7f59bc15d6f0 2023-10-30T12:41:03.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59c1a9c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc14d770 con 0x7f59bc1538a0 2023-10-30T12:41:03.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59c1a9c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc1e2f50 con 0x7f59bc0cf160 2023-10-30T12:41:03.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59bbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc15d6f0 0x7f59bc1e3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:03.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59bbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc15d6f0 0x7f59bc1e3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43474/0 (socket says 172.21.15.12:43474) 2023-10-30T12:41:03.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59bbfff640 1 -- 172.21.15.12:0/3260563946 learned_addr learned my addr 172.21.15.12:0/3260563946 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:41:03.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59bb7fe640 1 --2- 172.21.15.12:0/3260563946 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59bc0cf160 0x7f59bc1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59baffd640 1 --2- 172.21.15.12:0/3260563946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59bc1538a0 0x7f59bc15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59bbfff640 1 -- 172.21.15.12:0/3260563946 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59bc0cf160 msgr2=0x7f59bc1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59bbfff640 1 --2- 172.21.15.12:0/3260563946 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59bc0cf160 0x7f59bc1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59bbfff640 1 -- 172.21.15.12:0/3260563946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59bc1538a0 msgr2=0x7f59bc15b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59bbfff640 1 --2- 172.21.15.12:0/3260563946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59bc1538a0 0x7f59bc15b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59bbfff640 1 -- 172.21.15.12:0/3260563946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b0009420 con 0x7f59bc15d6f0 2023-10-30T12:41:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59bbfff640 1 --2- 172.21.15.12:0/3260563946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc15d6f0 0x7f59bc1e3990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f59b400c730 tx=0x7f59b400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:41:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59a3fff640 1 -- 172.21.15.12:0/3260563946 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4003b80 con 0x7f59bc15d6f0 2023-10-30T12:41:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59c1a9c640 1 -- 172.21.15.12:0/3260563946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc15c860 con 0x7f59bc15d6f0 2023-10-30T12:41:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59a3fff640 1 -- 172.21.15.12:0/3260563946 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b4003ce0 con 0x7f59bc15d6f0 2023-10-30T12:41:03.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59a3fff640 1 -- 172.21.15.12:0/3260563946 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4005310 con 0x7f59bc15d6f0 2023-10-30T12:41:03.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59c1a9c640 1 -- 172.21.15.12:0/3260563946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc0c6a70 con 0x7f59bc15d6f0 2023-10-30T12:41:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.950+0000 7f59c1a9c640 1 -- 172.21.15.12:0/3260563946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59bc153c80 con 0x7f59bc15d6f0 2023-10-30T12:41:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.954+0000 7f59a3fff640 1 -- 172.21.15.12:0/3260563946 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f59b4005680 con 0x7f59bc15d6f0 2023-10-30T12:41:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.954+0000 7f59a3fff640 1 --2- 172.21.15.12:0/3260563946 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f59a80664a0 0x7f59a8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.954+0000 7f59a3fff640 1 -- 172.21.15.12:0/3260563946 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f59b4083f30 con 0x7f59bc15d6f0 2023-10-30T12:41:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.954+0000 7f59bb7fe640 1 --2- 172.21.15.12:0/3260563946 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f59a80664a0 0x7f59a8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.954+0000 7f59bb7fe640 1 --2- 172.21.15.12:0/3260563946 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f59a80664a0 0x7f59a8068960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f59ac000f70 tx=0x7f59ac007240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:41:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:03.958+0000 7f59a3fff640 1 -- 172.21.15.12:0/3260563946 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f59b4051210 con 0x7f59bc15d6f0 2023-10-30T12:41:04.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:04.150+0000 7f59c1a9c640 1 -- 172.21.15.12:0/3260563946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59bc15a580 con 0x7f59bc15d6f0 2023-10-30T12:41:04.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:04.150+0000 7f59a3fff640 1 -- 172.21.15.12:0/3260563946 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5741 (secure 0 0 0) 0x7f59b4054830 con 0x7f59bc15d6f0 2023-10-30T12:41:04.155 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:41:04.155 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:41:02.236451+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:41:04.156 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 30 2023-10-30T12:41:04.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:04.150+0000 7f59c1a9c640 1 -- 172.21.15.12:0/3260563946 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f59a80664a0 msgr2=0x7f59a8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:04.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:04.150+0000 7f59c1a9c640 1 --2- 172.21.15.12:0/3260563946 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f59a80664a0 0x7f59a8068960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f59ac000f70 tx=0x7f59ac007240 comp rx=0 tx=0).stop 2023-10-30T12:41:04.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:04.150+0000 7f59c1a9c640 1 -- 172.21.15.12:0/3260563946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc15d6f0 msgr2=0x7f59bc1e3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:04.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:04.150+0000 7f59c1a9c640 1 --2- 172.21.15.12:0/3260563946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc15d6f0 0x7f59bc1e3990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f59b400c730 tx=0x7f59b400cbf0 comp rx=0 tx=0).stop 2023-10-30T12:41:04.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:04.150+0000 7f59c1a9c640 1 -- 172.21.15.12:0/3260563946 shutdown_connections 2023-10-30T12:41:04.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:04.150+0000 7f59c1a9c640 1 --2- 172.21.15.12:0/3260563946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc15d6f0 0x7f59bc1e3990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:04.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:04.150+0000 7f59c1a9c640 1 --2- 172.21.15.12:0/3260563946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59bc1538a0 0x7f59bc15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:04.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:04.150+0000 7f59c1a9c640 1 --2- 172.21.15.12:0/3260563946 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f59a80664a0 0x7f59a8068960 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:04.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:04.150+0000 7f59c1a9c640 1 --2- 172.21.15.12:0/3260563946 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59bc0cf160 0x7f59bc1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:04.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:04.150+0000 7f59c1a9c640 1 -- 172.21.15.12:0/3260563946 >> 172.21.15.12:0/3260563946 conn(0x7f59bc06bfe0 msgr2=0x7f59bc06c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:41:04.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:04.150+0000 7f59c1a9c640 1 -- 172.21.15.12:0/3260563946 shutdown_connections 2023-10-30T12:41:04.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:04.150+0000 7f59c1a9c640 1 -- 172.21.15.12:0/3260563946 wait complete. 2023-10-30T12:41:04.176 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=2, replacing gid=4262 2023-10-30T12:41:04.176 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 8 secs before reviving mds.a 2023-10-30T12:41:04.347 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:41:04.351 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:41:04.492 INFO:tasks.ceph.mds.e.smithi012.stderr:2023-10-30T12:41:04.486+0000 7fbd9da0c640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-30T12:41:09.977 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/21' 2023-10-30T12:41:09.978 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/27' 2023-10-30T12:41:09.978 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/31' 2023-10-30T12:41:09.978 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/36' 2023-10-30T12:41:09.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/20' 2023-10-30T12:41:09.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/39' 2023-10-30T12:41:09.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/34' 2023-10-30T12:41:09.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/38' 2023-10-30T12:41:09.979 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/28' 2023-10-30T12:41:09.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/24' 2023-10-30T12:41:09.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/29' 2023-10-30T12:41:09.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/32' 2023-10-30T12:41:09.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/22' 2023-10-30T12:41:09.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/25' 2023-10-30T12:41:09.980 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/30' 2023-10-30T12:41:09.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/33' 2023-10-30T12:41:09.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/23' 2023-10-30T12:41:09.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/35' 2023-10-30T12:41:09.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/26' 2023-10-30T12:41:09.981 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/37' 2023-10-30T12:41:09.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './287' 2023-10-30T12:41:09.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/21' 2023-10-30T12:41:09.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/27' 2023-10-30T12:41:09.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/31' 2023-10-30T12:41:09.982 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/36' 2023-10-30T12:41:09.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/20' 2023-10-30T12:41:09.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/39' 2023-10-30T12:41:09.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/34' 2023-10-30T12:41:09.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/38' 2023-10-30T12:41:09.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/28' 2023-10-30T12:41:09.983 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/24' 2023-10-30T12:41:09.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/29' 2023-10-30T12:41:09.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/32' 2023-10-30T12:41:09.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/22' 2023-10-30T12:41:09.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/25' 2023-10-30T12:41:09.984 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/30' 2023-10-30T12:41:09.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/33' 2023-10-30T12:41:09.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/23' 2023-10-30T12:41:09.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/35' 2023-10-30T12:41:09.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/26' 2023-10-30T12:41:09.985 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/37' 2023-10-30T12:41:09.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './288' 2023-10-30T12:41:09.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/21' 2023-10-30T12:41:09.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/27' 2023-10-30T12:41:09.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/31' 2023-10-30T12:41:09.986 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/36' 2023-10-30T12:41:09.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/20' 2023-10-30T12:41:09.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/39' 2023-10-30T12:41:09.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/34' 2023-10-30T12:41:09.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/38' 2023-10-30T12:41:09.987 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/28' 2023-10-30T12:41:09.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/24' 2023-10-30T12:41:09.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/29' 2023-10-30T12:41:09.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/32' 2023-10-30T12:41:09.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/22' 2023-10-30T12:41:09.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/25' 2023-10-30T12:41:09.988 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/30' 2023-10-30T12:41:09.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/33' 2023-10-30T12:41:09.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/23' 2023-10-30T12:41:09.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/35' 2023-10-30T12:41:09.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/26' 2023-10-30T12:41:09.989 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/37' 2023-10-30T12:41:09.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './289' 2023-10-30T12:41:09.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/21' 2023-10-30T12:41:09.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/27' 2023-10-30T12:41:09.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/31' 2023-10-30T12:41:09.990 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/36' 2023-10-30T12:41:09.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/20' 2023-10-30T12:41:09.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/39' 2023-10-30T12:41:09.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/34' 2023-10-30T12:41:09.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/38' 2023-10-30T12:41:09.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/28' 2023-10-30T12:41:09.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/24' 2023-10-30T12:41:09.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/29' 2023-10-30T12:41:09.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/32' 2023-10-30T12:41:09.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/22' 2023-10-30T12:41:09.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/25' 2023-10-30T12:41:09.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/30' 2023-10-30T12:41:09.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/33' 2023-10-30T12:41:09.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/23' 2023-10-30T12:41:09.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/35' 2023-10-30T12:41:09.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/26' 2023-10-30T12:41:09.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/37' 2023-10-30T12:41:09.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './290' 2023-10-30T12:41:09.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/21' 2023-10-30T12:41:09.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/27' 2023-10-30T12:41:09.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/31' 2023-10-30T12:41:09.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/36' 2023-10-30T12:41:09.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/20' 2023-10-30T12:41:09.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/39' 2023-10-30T12:41:09.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/34' 2023-10-30T12:41:09.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/38' 2023-10-30T12:41:09.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/28' 2023-10-30T12:41:09.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/24' 2023-10-30T12:41:09.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/29' 2023-10-30T12:41:09.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/32' 2023-10-30T12:41:09.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/22' 2023-10-30T12:41:09.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/25' 2023-10-30T12:41:09.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/30' 2023-10-30T12:41:09.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/33' 2023-10-30T12:41:09.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/23' 2023-10-30T12:41:09.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/35' 2023-10-30T12:41:09.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/26' 2023-10-30T12:41:09.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/37' 2023-10-30T12:41:09.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './291' 2023-10-30T12:41:09.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/21' 2023-10-30T12:41:09.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/27' 2023-10-30T12:41:09.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/31' 2023-10-30T12:41:09.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/36' 2023-10-30T12:41:09.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/20' 2023-10-30T12:41:09.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/39' 2023-10-30T12:41:09.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/34' 2023-10-30T12:41:09.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/38' 2023-10-30T12:41:09.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/28' 2023-10-30T12:41:09.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/24' 2023-10-30T12:41:09.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/29' 2023-10-30T12:41:10.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/32' 2023-10-30T12:41:10.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/22' 2023-10-30T12:41:10.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/25' 2023-10-30T12:41:10.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/30' 2023-10-30T12:41:10.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/33' 2023-10-30T12:41:10.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/23' 2023-10-30T12:41:10.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/35' 2023-10-30T12:41:10.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/26' 2023-10-30T12:41:10.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/37' 2023-10-30T12:41:10.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './292' 2023-10-30T12:41:10.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/21' 2023-10-30T12:41:10.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/27' 2023-10-30T12:41:10.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/31' 2023-10-30T12:41:10.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/36' 2023-10-30T12:41:10.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/20' 2023-10-30T12:41:10.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/39' 2023-10-30T12:41:10.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/34' 2023-10-30T12:41:10.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/38' 2023-10-30T12:41:10.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/28' 2023-10-30T12:41:10.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/24' 2023-10-30T12:41:10.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/29' 2023-10-30T12:41:10.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/32' 2023-10-30T12:41:10.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/22' 2023-10-30T12:41:10.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/25' 2023-10-30T12:41:10.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/30' 2023-10-30T12:41:10.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/33' 2023-10-30T12:41:10.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/23' 2023-10-30T12:41:10.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/35' 2023-10-30T12:41:10.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/26' 2023-10-30T12:41:10.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/37' 2023-10-30T12:41:10.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './293' 2023-10-30T12:41:10.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/21' 2023-10-30T12:41:10.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/27' 2023-10-30T12:41:10.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/31' 2023-10-30T12:41:10.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/36' 2023-10-30T12:41:10.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/20' 2023-10-30T12:41:10.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/39' 2023-10-30T12:41:10.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/34' 2023-10-30T12:41:10.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/38' 2023-10-30T12:41:10.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/28' 2023-10-30T12:41:10.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/24' 2023-10-30T12:41:10.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/29' 2023-10-30T12:41:10.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/32' 2023-10-30T12:41:10.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/22' 2023-10-30T12:41:10.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/25' 2023-10-30T12:41:10.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/30' 2023-10-30T12:41:10.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/33' 2023-10-30T12:41:10.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/23' 2023-10-30T12:41:10.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/35' 2023-10-30T12:41:10.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/26' 2023-10-30T12:41:10.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/37' 2023-10-30T12:41:10.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './294' 2023-10-30T12:41:10.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/21' 2023-10-30T12:41:10.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/27' 2023-10-30T12:41:10.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/31' 2023-10-30T12:41:10.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/36' 2023-10-30T12:41:10.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/20' 2023-10-30T12:41:10.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/39' 2023-10-30T12:41:10.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/34' 2023-10-30T12:41:10.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/38' 2023-10-30T12:41:10.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/28' 2023-10-30T12:41:10.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/24' 2023-10-30T12:41:10.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/29' 2023-10-30T12:41:10.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/32' 2023-10-30T12:41:10.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/22' 2023-10-30T12:41:10.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/25' 2023-10-30T12:41:10.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/30' 2023-10-30T12:41:10.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/33' 2023-10-30T12:41:10.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/23' 2023-10-30T12:41:10.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/35' 2023-10-30T12:41:10.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/26' 2023-10-30T12:41:10.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/37' 2023-10-30T12:41:10.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './295' 2023-10-30T12:41:10.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/21' 2023-10-30T12:41:10.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/27' 2023-10-30T12:41:10.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/31' 2023-10-30T12:41:10.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/36' 2023-10-30T12:41:10.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/20' 2023-10-30T12:41:10.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/39' 2023-10-30T12:41:10.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/34' 2023-10-30T12:41:10.015 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/38' 2023-10-30T12:41:10.015 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/28' 2023-10-30T12:41:10.015 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/24' 2023-10-30T12:41:10.015 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/29' 2023-10-30T12:41:10.015 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/32' 2023-10-30T12:41:10.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/22' 2023-10-30T12:41:10.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/25' 2023-10-30T12:41:10.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/30' 2023-10-30T12:41:10.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/33' 2023-10-30T12:41:10.016 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/23' 2023-10-30T12:41:10.017 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/35' 2023-10-30T12:41:10.017 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/26' 2023-10-30T12:41:10.017 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/37' 2023-10-30T12:41:10.017 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './296' 2023-10-30T12:41:10.017 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/21' 2023-10-30T12:41:10.018 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/27' 2023-10-30T12:41:11.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/31' 2023-10-30T12:41:11.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/36' 2023-10-30T12:41:11.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/20' 2023-10-30T12:41:11.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/39' 2023-10-30T12:41:11.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/34' 2023-10-30T12:41:11.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/38' 2023-10-30T12:41:11.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/28' 2023-10-30T12:41:11.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/24' 2023-10-30T12:41:11.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/29' 2023-10-30T12:41:11.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/32' 2023-10-30T12:41:11.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/22' 2023-10-30T12:41:11.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/25' 2023-10-30T12:41:11.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/30' 2023-10-30T12:41:11.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/33' 2023-10-30T12:41:11.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/23' 2023-10-30T12:41:11.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/35' 2023-10-30T12:41:11.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/26' 2023-10-30T12:41:11.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/37' 2023-10-30T12:41:11.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './297' 2023-10-30T12:41:11.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/21' 2023-10-30T12:41:11.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/27' 2023-10-30T12:41:11.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/31' 2023-10-30T12:41:11.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/36' 2023-10-30T12:41:11.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/20' 2023-10-30T12:41:11.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/39' 2023-10-30T12:41:11.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/34' 2023-10-30T12:41:11.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/38' 2023-10-30T12:41:11.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/28' 2023-10-30T12:41:11.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/24' 2023-10-30T12:41:11.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/29' 2023-10-30T12:41:11.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/32' 2023-10-30T12:41:11.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/22' 2023-10-30T12:41:11.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/25' 2023-10-30T12:41:11.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/30' 2023-10-30T12:41:11.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/33' 2023-10-30T12:41:11.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/23' 2023-10-30T12:41:11.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/35' 2023-10-30T12:41:11.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/26' 2023-10-30T12:41:11.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/37' 2023-10-30T12:41:11.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './298' 2023-10-30T12:41:11.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/21' 2023-10-30T12:41:11.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/27' 2023-10-30T12:41:11.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/31' 2023-10-30T12:41:11.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/36' 2023-10-30T12:41:11.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/20' 2023-10-30T12:41:11.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/39' 2023-10-30T12:41:11.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/34' 2023-10-30T12:41:11.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/38' 2023-10-30T12:41:11.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/28' 2023-10-30T12:41:11.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/24' 2023-10-30T12:41:11.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/29' 2023-10-30T12:41:11.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/32' 2023-10-30T12:41:11.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/22' 2023-10-30T12:41:11.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/25' 2023-10-30T12:41:11.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/30' 2023-10-30T12:41:11.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/33' 2023-10-30T12:41:11.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/23' 2023-10-30T12:41:11.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/35' 2023-10-30T12:41:11.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/26' 2023-10-30T12:41:11.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/37' 2023-10-30T12:41:11.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './299' 2023-10-30T12:41:11.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/19' 2023-10-30T12:41:11.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/15' 2023-10-30T12:41:11.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/20' 2023-10-30T12:41:11.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/12' 2023-10-30T12:41:11.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/10' 2023-10-30T12:41:11.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/11' 2023-10-30T12:41:11.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/16' 2023-10-30T12:41:11.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/14' 2023-10-30T12:41:11.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/18' 2023-10-30T12:41:11.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/13' 2023-10-30T12:41:11.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/17' 2023-10-30T12:41:11.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/166' 2023-10-30T12:41:11.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/19' 2023-10-30T12:41:11.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/15' 2023-10-30T12:41:11.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/20' 2023-10-30T12:41:11.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/12' 2023-10-30T12:41:11.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/10' 2023-10-30T12:41:11.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/11' 2023-10-30T12:41:11.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/16' 2023-10-30T12:41:11.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/14' 2023-10-30T12:41:11.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/18' 2023-10-30T12:41:11.083 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/13' 2023-10-30T12:41:11.083 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/17' 2023-10-30T12:41:11.083 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/147' 2023-10-30T12:41:11.083 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/19' 2023-10-30T12:41:11.083 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/15' 2023-10-30T12:41:11.083 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/20' 2023-10-30T12:41:11.084 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/12' 2023-10-30T12:41:11.084 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/10' 2023-10-30T12:41:11.084 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/11' 2023-10-30T12:41:11.084 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/16' 2023-10-30T12:41:11.084 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/14' 2023-10-30T12:41:11.085 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/18' 2023-10-30T12:41:11.085 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/13' 2023-10-30T12:41:11.085 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/17' 2023-10-30T12:41:11.085 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/151' 2023-10-30T12:41:11.085 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/19' 2023-10-30T12:41:11.085 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/15' 2023-10-30T12:41:11.086 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/20' 2023-10-30T12:41:11.086 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/12' 2023-10-30T12:41:11.086 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/10' 2023-10-30T12:41:11.086 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/11' 2023-10-30T12:41:11.087 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/16' 2023-10-30T12:41:11.087 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/14' 2023-10-30T12:41:11.087 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/18' 2023-10-30T12:41:11.087 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/13' 2023-10-30T12:41:11.087 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/17' 2023-10-30T12:41:11.088 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/170' 2023-10-30T12:41:11.088 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/19' 2023-10-30T12:41:11.088 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/15' 2023-10-30T12:41:11.088 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/20' 2023-10-30T12:41:11.088 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/12' 2023-10-30T12:41:11.088 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/10' 2023-10-30T12:41:11.089 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/11' 2023-10-30T12:41:11.089 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/16' 2023-10-30T12:41:11.089 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/14' 2023-10-30T12:41:11.089 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/18' 2023-10-30T12:41:11.089 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/13' 2023-10-30T12:41:11.090 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/17' 2023-10-30T12:41:11.090 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/167' 2023-10-30T12:41:11.090 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/19' 2023-10-30T12:41:11.090 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/15' 2023-10-30T12:41:11.090 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/20' 2023-10-30T12:41:11.091 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/12' 2023-10-30T12:41:11.091 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/10' 2023-10-30T12:41:11.091 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/11' 2023-10-30T12:41:11.091 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/16' 2023-10-30T12:41:11.091 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/14' 2023-10-30T12:41:11.091 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/18' 2023-10-30T12:41:11.092 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/13' 2023-10-30T12:41:11.092 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/17' 2023-10-30T12:41:11.092 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/140' 2023-10-30T12:41:11.092 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/19' 2023-10-30T12:41:11.092 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/15' 2023-10-30T12:41:11.092 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/20' 2023-10-30T12:41:11.093 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/12' 2023-10-30T12:41:11.093 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/10' 2023-10-30T12:41:11.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/11' 2023-10-30T12:41:11.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/16' 2023-10-30T12:41:11.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/14' 2023-10-30T12:41:11.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/18' 2023-10-30T12:41:11.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/13' 2023-10-30T12:41:11.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/17' 2023-10-30T12:41:11.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/142' 2023-10-30T12:41:11.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/19' 2023-10-30T12:41:11.885 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/15' 2023-10-30T12:41:11.885 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/20' 2023-10-30T12:41:11.885 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/12' 2023-10-30T12:41:11.885 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/10' 2023-10-30T12:41:11.885 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/11' 2023-10-30T12:41:11.886 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/16' 2023-10-30T12:41:11.886 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/14' 2023-10-30T12:41:11.886 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/18' 2023-10-30T12:41:11.886 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/13' 2023-10-30T12:41:11.886 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/17' 2023-10-30T12:41:11.887 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/150' 2023-10-30T12:41:11.887 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/19' 2023-10-30T12:41:11.887 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/15' 2023-10-30T12:41:11.887 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/20' 2023-10-30T12:41:11.887 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/12' 2023-10-30T12:41:11.888 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/10' 2023-10-30T12:41:11.888 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/11' 2023-10-30T12:41:11.888 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/16' 2023-10-30T12:41:11.888 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/14' 2023-10-30T12:41:11.888 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/18' 2023-10-30T12:41:11.888 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/13' 2023-10-30T12:41:11.889 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/17' 2023-10-30T12:41:11.889 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/105' 2023-10-30T12:41:11.889 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/19' 2023-10-30T12:41:11.889 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/15' 2023-10-30T12:41:11.889 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/20' 2023-10-30T12:41:11.890 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/12' 2023-10-30T12:41:11.890 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/10' 2023-10-30T12:41:11.890 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/11' 2023-10-30T12:41:11.890 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/16' 2023-10-30T12:41:11.890 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/14' 2023-10-30T12:41:11.891 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/18' 2023-10-30T12:41:11.891 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/13' 2023-10-30T12:41:11.891 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/17' 2023-10-30T12:41:11.891 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/149' 2023-10-30T12:41:11.891 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/19' 2023-10-30T12:41:11.891 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/15' 2023-10-30T12:41:11.892 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/20' 2023-10-30T12:41:11.892 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/12' 2023-10-30T12:41:11.892 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/10' 2023-10-30T12:41:11.892 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/11' 2023-10-30T12:41:11.892 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/16' 2023-10-30T12:41:11.893 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/14' 2023-10-30T12:41:11.893 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/18' 2023-10-30T12:41:11.893 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/13' 2023-10-30T12:41:11.893 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/17' 2023-10-30T12:41:11.893 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/109' 2023-10-30T12:41:11.894 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/19' 2023-10-30T12:41:11.894 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/15' 2023-10-30T12:41:11.894 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/20' 2023-10-30T12:41:11.894 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/12' 2023-10-30T12:41:11.894 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/10' 2023-10-30T12:41:11.895 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/11' 2023-10-30T12:41:11.895 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/16' 2023-10-30T12:41:11.895 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/14' 2023-10-30T12:41:11.895 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/18' 2023-10-30T12:41:11.895 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/13' 2023-10-30T12:41:11.895 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/17' 2023-10-30T12:41:11.896 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/158' 2023-10-30T12:41:11.896 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/19' 2023-10-30T12:41:11.896 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/15' 2023-10-30T12:41:11.896 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/20' 2023-10-30T12:41:11.896 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/12' 2023-10-30T12:41:11.897 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/10' 2023-10-30T12:41:11.897 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/11' 2023-10-30T12:41:11.897 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/16' 2023-10-30T12:41:11.897 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/14' 2023-10-30T12:41:11.897 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/18' 2023-10-30T12:41:11.897 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/13' 2023-10-30T12:41:11.898 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/17' 2023-10-30T12:41:11.898 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/106' 2023-10-30T12:41:11.898 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/19' 2023-10-30T12:41:11.898 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/15' 2023-10-30T12:41:11.899 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/20' 2023-10-30T12:41:11.899 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/12' 2023-10-30T12:41:11.899 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/10' 2023-10-30T12:41:11.899 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/11' 2023-10-30T12:41:11.899 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/16' 2023-10-30T12:41:11.899 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/14' 2023-10-30T12:41:11.900 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/18' 2023-10-30T12:41:11.900 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/13' 2023-10-30T12:41:11.900 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/17' 2023-10-30T12:41:11.900 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/189' 2023-10-30T12:41:11.900 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/19' 2023-10-30T12:41:11.901 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/15' 2023-10-30T12:41:11.901 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/20' 2023-10-30T12:41:11.901 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/12' 2023-10-30T12:41:11.901 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/10' 2023-10-30T12:41:11.901 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/11' 2023-10-30T12:41:11.902 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/16' 2023-10-30T12:41:11.902 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/14' 2023-10-30T12:41:11.902 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/18' 2023-10-30T12:41:11.902 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/13' 2023-10-30T12:41:11.902 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/17' 2023-10-30T12:41:11.902 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/133' 2023-10-30T12:41:11.903 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/19' 2023-10-30T12:41:11.903 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/15' 2023-10-30T12:41:11.903 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/20' 2023-10-30T12:41:11.903 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/12' 2023-10-30T12:41:11.903 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/10' 2023-10-30T12:41:11.904 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/11' 2023-10-30T12:41:12.177 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2023-10-30T12:41:12.177 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-30T12:41:12.178 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-30T12:41:12.182 INFO:tasks.ceph.mds.a:Started 2023-10-30T12:41:12.182 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:41:12.286 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2023-10-30T12:41:12.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f03a913e640 1 Processor -- start 2023-10-30T12:41:12.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f03a913e640 1 -- start start 2023-10-30T12:41:12.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f03a913e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f03a40c6b80 0x7f03a40bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:12.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f03a913e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f03a413f4a0 0x7f03a40c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:12.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f03a913e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a40c0a50 con 0x7f03a40c6f60 2023-10-30T12:41:12.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f03a913e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a406f6e0 con 0x7f03a413f4a0 2023-10-30T12:41:12.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f03a913e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a406f9c0 con 0x7f03a40c6b80 2023-10-30T12:41:12.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f039bfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f03a40c6f60 0x7f03a413ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38326/0 (socket says 172.21.15.12:38326) 2023-10-30T12:41:12.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f039bfff640 1 -- 172.21.15.12:0/2049906946 learned_addr learned my addr 172.21.15.12:0/2049906946 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:41:12.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f03a359e640 1 --2- 172.21.15.12:0/2049906946 >> v2:172.21.15.12:3300/0 conn(0x7f03a413f4a0 0x7f03a40c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:12.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f03a2d9d640 1 --2- 172.21.15.12:0/2049906946 >> v2:172.21.15.154:3300/0 conn(0x7f03a40c6b80 0x7f03a40bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:12.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f03a259c640 1 -- 172.21.15.12:0/2049906946 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260427011 0 0) 0x7f03a40c0a50 con 0x7f03a40c6f60 2023-10-30T12:41:12.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f03a259c640 1 -- 172.21.15.12:0/2049906946 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0390003530 con 0x7f03a40c6f60 2023-10-30T12:41:12.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f03a359e640 1 -- 172.21.15.12:0/2049906946 >> v2:172.21.15.154:3300/0 conn(0x7f03a40c6b80 msgr2=0x7f03a40bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:41:12.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f03a359e640 1 --2- 172.21.15.12:0/2049906946 >> v2:172.21.15.154:3300/0 conn(0x7f03a40c6b80 0x7f03a40bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:12.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f03a359e640 1 -- 172.21.15.12:0/2049906946 >> v1:172.21.15.12:6789/0 conn(0x7f03a40c6f60 legacy=0x7f03a413ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:12.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.334+0000 7f03a359e640 1 -- 172.21.15.12:0/2049906946 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a406fca0 con 0x7f03a413f4a0 2023-10-30T12:41:12.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.338+0000 7f03a359e640 1 --2- 172.21.15.12:0/2049906946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a413f4a0 0x7f03a40c0400 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f03a406f1b0 tx=0x7f039c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2edb3b07bad1e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:41:12.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.338+0000 7f03a259c640 1 -- 172.21.15.12:0/2049906946 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c0a58e0 con 0x7f03a413f4a0 2023-10-30T12:41:12.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.338+0000 7f03a259c640 1 -- 172.21.15.12:0/2049906946 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c0a5a40 con 0x7f03a413f4a0 2023-10-30T12:41:12.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.338+0000 7f03a259c640 1 -- 172.21.15.12:0/2049906946 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c0a5dd0 con 0x7f03a413f4a0 2023-10-30T12:41:12.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.338+0000 7f03a913e640 1 -- 172.21.15.12:0/2049906946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a413f4a0 msgr2=0x7f03a40c0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:12.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.338+0000 7f03a913e640 1 --2- 172.21.15.12:0/2049906946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a413f4a0 0x7f03a40c0400 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f03a406f1b0 tx=0x7f039c01c910 comp rx=0 tx=0).stop 2023-10-30T12:41:12.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.338+0000 7f03a913e640 1 -- 172.21.15.12:0/2049906946 shutdown_connections 2023-10-30T12:41:12.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.338+0000 7f03a913e640 1 --2- 172.21.15.12:0/2049906946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a413f4a0 0x7f03a40c0400 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:12.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.338+0000 7f03a913e640 1 --2- 172.21.15.12:0/2049906946 >> v2:172.21.15.154:3300/0 conn(0x7f03a40c6b80 0x7f03a40bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:12.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.338+0000 7f03a913e640 1 -- 172.21.15.12:0/2049906946 >> 172.21.15.12:0/2049906946 conn(0x7f03a40c7910 msgr2=0x7f03a40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:41:12.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.338+0000 7f03a913e640 1 -- 172.21.15.12:0/2049906946 shutdown_connections 2023-10-30T12:41:12.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.338+0000 7f03a913e640 1 -- 172.21.15.12:0/2049906946 wait complete. 2023-10-30T12:41:12.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a913e640 1 Processor -- start 2023-10-30T12:41:12.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a913e640 1 -- start start 2023-10-30T12:41:12.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a913e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03a40c6b80 0x7f03a4162f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:12.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a913e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03a40c6f60 0x7f03a415fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:12.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a913e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a413f4a0 0x7f03a4158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:12.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a913e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a40c9e10 con 0x7f03a413f4a0 2023-10-30T12:41:12.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a913e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a415ff90 con 0x7f03a40c6b80 2023-10-30T12:41:12.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a913e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a41635a0 con 0x7f03a40c6f60 2023-10-30T12:41:12.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a359e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a413f4a0 0x7f03a4158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:12.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a359e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a413f4a0 0x7f03a4158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50730/0 (socket says 172.21.15.12:50730) 2023-10-30T12:41:12.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a359e640 1 -- 172.21.15.12:0/4012647603 learned_addr learned my addr 172.21.15.12:0/4012647603 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:41:12.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f039bfff640 1 --2- 172.21.15.12:0/4012647603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03a40c6f60 0x7f03a415fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:12.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a2d9d640 1 --2- 172.21.15.12:0/4012647603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03a40c6b80 0x7f03a4162f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:12.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a359e640 1 -- 172.21.15.12:0/4012647603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03a40c6f60 msgr2=0x7f03a415fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:12.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a359e640 1 --2- 172.21.15.12:0/4012647603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03a40c6f60 0x7f03a415fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:12.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a359e640 1 -- 172.21.15.12:0/4012647603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03a40c6b80 msgr2=0x7f03a4162f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:12.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a359e640 1 --2- 172.21.15.12:0/4012647603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03a40c6b80 0x7f03a4162f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:12.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a359e640 1 -- 172.21.15.12:0/4012647603 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c009a70 con 0x7f03a413f4a0 2023-10-30T12:41:12.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a2d9d640 1 --2- 172.21.15.12:0/4012647603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03a40c6b80 0x7f03a4162f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:41:12.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a359e640 1 --2- 172.21.15.12:0/4012647603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a413f4a0 0x7f03a4158d80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f039c019520 tx=0x7f039c0196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:41:12.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f039b7fe640 1 -- 172.21.15.12:0/4012647603 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c0ad8d0 con 0x7f03a413f4a0 2023-10-30T12:41:12.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f039b7fe640 1 -- 172.21.15.12:0/4012647603 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c0ada30 con 0x7f03a413f4a0 2023-10-30T12:41:12.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f039b7fe640 1 -- 172.21.15.12:0/4012647603 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c0adde0 con 0x7f03a413f4a0 2023-10-30T12:41:12.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a913e640 1 -- 172.21.15.12:0/4012647603 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a40ca510 con 0x7f03a413f4a0 2023-10-30T12:41:12.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a913e640 1 -- 172.21.15.12:0/4012647603 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a40caad0 con 0x7f03a413f4a0 2023-10-30T12:41:12.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.342+0000 7f03a913e640 1 -- 172.21.15.12:0/4012647603 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a404e170 con 0x7f03a413f4a0 2023-10-30T12:41:12.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.346+0000 7f039b7fe640 1 -- 172.21.15.12:0/4012647603 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f039c0b4050 con 0x7f03a413f4a0 2023-10-30T12:41:12.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.346+0000 7f039b7fe640 1 --2- 172.21.15.12:0/4012647603 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f03840664a0 0x7f0384068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:12.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.346+0000 7f039b7fe640 1 -- 172.21.15.12:0/4012647603 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f039c11efe0 con 0x7f03a413f4a0 2023-10-30T12:41:12.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.346+0000 7f03a2d9d640 1 --2- 172.21.15.12:0/4012647603 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f03840664a0 0x7f0384068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:12.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.346+0000 7f03a2d9d640 1 --2- 172.21.15.12:0/4012647603 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f03840664a0 0x7f0384068960 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0394009bb0 tx=0x7f0394008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:41:12.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.350+0000 7f039b7fe640 1 -- 172.21.15.12:0/4012647603 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f039c0ec470 con 0x7f03a413f4a0 2023-10-30T12:41:12.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.538+0000 7f03a913e640 1 -- 172.21.15.12:0/4012647603 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03a40c7df0 con 0x7f03a413f4a0 2023-10-30T12:41:12.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.538+0000 7f039b7fe640 1 -- 172.21.15.12:0/4012647603 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5741 (secure 0 0 0) 0x7f039c0efa90 con 0x7f03a413f4a0 2023-10-30T12:41:12.546 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:41:12.546 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:41:02.236451+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:41:12.547 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 30 2023-10-30T12:41:12.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.542+0000 7f03a913e640 1 -- 172.21.15.12:0/4012647603 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f03840664a0 msgr2=0x7f0384068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:12.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.542+0000 7f03a913e640 1 --2- 172.21.15.12:0/4012647603 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f03840664a0 0x7f0384068960 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0394009bb0 tx=0x7f0394008040 comp rx=0 tx=0).stop 2023-10-30T12:41:12.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.542+0000 7f03a913e640 1 -- 172.21.15.12:0/4012647603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a413f4a0 msgr2=0x7f03a4158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:12.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.542+0000 7f03a913e640 1 --2- 172.21.15.12:0/4012647603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a413f4a0 0x7f03a4158d80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f039c019520 tx=0x7f039c0196b0 comp rx=0 tx=0).stop 2023-10-30T12:41:12.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.542+0000 7f03a913e640 1 -- 172.21.15.12:0/4012647603 shutdown_connections 2023-10-30T12:41:12.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.542+0000 7f03a913e640 1 --2- 172.21.15.12:0/4012647603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a413f4a0 0x7f03a4158d80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:12.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.542+0000 7f03a913e640 1 --2- 172.21.15.12:0/4012647603 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f03840664a0 0x7f0384068960 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.542+0000 7f03a913e640 1 --2- 172.21.15.12:0/4012647603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03a40c6f60 0x7f03a415fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.542+0000 7f03a913e640 1 --2- 172.21.15.12:0/4012647603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03a40c6b80 0x7f03a4162f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.542+0000 7f03a913e640 1 -- 172.21.15.12:0/4012647603 >> 172.21.15.12:0/4012647603 conn(0x7f03a40c7910 msgr2=0x7f03a40c0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:41:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.542+0000 7f03a913e640 1 -- 172.21.15.12:0/4012647603 shutdown_connections 2023-10-30T12:41:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:12.542+0000 7f03a913e640 1 -- 172.21.15.12:0/4012647603 wait complete. 2023-10-30T12:41:12.566 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2023-10-30T12:41:12.827 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/16' 2023-10-30T12:41:12.827 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/14' 2023-10-30T12:41:12.827 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/18' 2023-10-30T12:41:12.828 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/13' 2023-10-30T12:41:12.828 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/17' 2023-10-30T12:41:12.828 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/173' 2023-10-30T12:41:12.828 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/19' 2023-10-30T12:41:12.828 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/15' 2023-10-30T12:41:12.829 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/20' 2023-10-30T12:41:12.829 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/12' 2023-10-30T12:41:12.829 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/10' 2023-10-30T12:41:12.829 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/11' 2023-10-30T12:41:12.829 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/16' 2023-10-30T12:41:12.830 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/14' 2023-10-30T12:41:12.830 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/18' 2023-10-30T12:41:12.830 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/13' 2023-10-30T12:41:12.830 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/17' 2023-10-30T12:41:12.831 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/174' 2023-10-30T12:41:12.831 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/19' 2023-10-30T12:41:12.831 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/15' 2023-10-30T12:41:12.831 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/20' 2023-10-30T12:41:12.831 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/12' 2023-10-30T12:41:12.831 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/10' 2023-10-30T12:41:12.832 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/11' 2023-10-30T12:41:12.832 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/16' 2023-10-30T12:41:12.832 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/14' 2023-10-30T12:41:12.832 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/18' 2023-10-30T12:41:12.832 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/13' 2023-10-30T12:41:12.832 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/17' 2023-10-30T12:41:12.833 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/155' 2023-10-30T12:41:12.833 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/19' 2023-10-30T12:41:12.833 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/15' 2023-10-30T12:41:12.833 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/20' 2023-10-30T12:41:12.833 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/12' 2023-10-30T12:41:12.834 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/10' 2023-10-30T12:41:12.834 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/11' 2023-10-30T12:41:12.834 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/16' 2023-10-30T12:41:12.834 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/14' 2023-10-30T12:41:12.834 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/18' 2023-10-30T12:41:12.835 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/13' 2023-10-30T12:41:12.835 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/17' 2023-10-30T12:41:12.835 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/114' 2023-10-30T12:41:12.835 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/19' 2023-10-30T12:41:12.835 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/15' 2023-10-30T12:41:12.835 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/20' 2023-10-30T12:41:12.836 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/12' 2023-10-30T12:41:12.836 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/10' 2023-10-30T12:41:12.836 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/11' 2023-10-30T12:41:12.836 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/16' 2023-10-30T12:41:12.836 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/14' 2023-10-30T12:41:12.837 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/18' 2023-10-30T12:41:12.837 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/13' 2023-10-30T12:41:12.837 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/17' 2023-10-30T12:41:12.837 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/159' 2023-10-30T12:41:12.837 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/19' 2023-10-30T12:41:12.837 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/15' 2023-10-30T12:41:12.838 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/20' 2023-10-30T12:41:12.838 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/12' 2023-10-30T12:41:12.838 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/10' 2023-10-30T12:41:12.838 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/11' 2023-10-30T12:41:12.838 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/16' 2023-10-30T12:41:12.839 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/14' 2023-10-30T12:41:12.839 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/18' 2023-10-30T12:41:12.839 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/13' 2023-10-30T12:41:12.839 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/17' 2023-10-30T12:41:12.839 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/187' 2023-10-30T12:41:12.839 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/19' 2023-10-30T12:41:12.840 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/15' 2023-10-30T12:41:12.840 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/20' 2023-10-30T12:41:12.840 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/12' 2023-10-30T12:41:12.840 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/10' 2023-10-30T12:41:12.840 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/11' 2023-10-30T12:41:12.840 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/16' 2023-10-30T12:41:12.841 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/14' 2023-10-30T12:41:12.841 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/18' 2023-10-30T12:41:12.841 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/13' 2023-10-30T12:41:12.841 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/17' 2023-10-30T12:41:12.841 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/185' 2023-10-30T12:41:12.841 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/19' 2023-10-30T12:41:12.842 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/15' 2023-10-30T12:41:12.842 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/20' 2023-10-30T12:41:12.842 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/12' 2023-10-30T12:41:12.842 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/10' 2023-10-30T12:41:12.842 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/11' 2023-10-30T12:41:12.842 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/16' 2023-10-30T12:41:12.843 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/14' 2023-10-30T12:41:12.843 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/18' 2023-10-30T12:41:12.843 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/13' 2023-10-30T12:41:12.843 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/17' 2023-10-30T12:41:12.843 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/120' 2023-10-30T12:41:12.843 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/19' 2023-10-30T12:41:12.844 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/15' 2023-10-30T12:41:12.844 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/20' 2023-10-30T12:41:12.844 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/12' 2023-10-30T12:41:12.844 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/10' 2023-10-30T12:41:12.844 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/11' 2023-10-30T12:41:12.844 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/16' 2023-10-30T12:41:12.845 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/14' 2023-10-30T12:41:12.845 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/18' 2023-10-30T12:41:12.845 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/13' 2023-10-30T12:41:12.845 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/17' 2023-10-30T12:41:12.845 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/110' 2023-10-30T12:41:12.845 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/19' 2023-10-30T12:41:12.846 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/15' 2023-10-30T12:41:12.846 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/20' 2023-10-30T12:41:12.846 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/12' 2023-10-30T12:41:12.846 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/10' 2023-10-30T12:41:12.846 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/11' 2023-10-30T12:41:12.847 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/16' 2023-10-30T12:41:13.495 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/14' 2023-10-30T12:41:13.495 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/18' 2023-10-30T12:41:13.495 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/13' 2023-10-30T12:41:13.495 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/17' 2023-10-30T12:41:13.495 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/137' 2023-10-30T12:41:13.496 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/19' 2023-10-30T12:41:13.496 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/15' 2023-10-30T12:41:13.496 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/20' 2023-10-30T12:41:13.496 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/12' 2023-10-30T12:41:13.496 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/10' 2023-10-30T12:41:13.497 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/11' 2023-10-30T12:41:13.497 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/16' 2023-10-30T12:41:13.497 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/14' 2023-10-30T12:41:13.497 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/18' 2023-10-30T12:41:13.497 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/13' 2023-10-30T12:41:13.498 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/17' 2023-10-30T12:41:13.498 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/198' 2023-10-30T12:41:13.498 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/19' 2023-10-30T12:41:13.498 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/15' 2023-10-30T12:41:13.498 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/20' 2023-10-30T12:41:13.499 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/12' 2023-10-30T12:41:13.499 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/10' 2023-10-30T12:41:13.499 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/11' 2023-10-30T12:41:13.499 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/16' 2023-10-30T12:41:13.499 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/14' 2023-10-30T12:41:13.500 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/18' 2023-10-30T12:41:13.500 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/13' 2023-10-30T12:41:13.500 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/17' 2023-10-30T12:41:13.500 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/136' 2023-10-30T12:41:13.500 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/19' 2023-10-30T12:41:13.501 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/15' 2023-10-30T12:41:13.501 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/20' 2023-10-30T12:41:13.501 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/12' 2023-10-30T12:41:13.501 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/10' 2023-10-30T12:41:13.501 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/11' 2023-10-30T12:41:13.501 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/16' 2023-10-30T12:41:13.502 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/14' 2023-10-30T12:41:13.502 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/18' 2023-10-30T12:41:13.502 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/13' 2023-10-30T12:41:13.502 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/17' 2023-10-30T12:41:13.502 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/125' 2023-10-30T12:41:13.503 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/19' 2023-10-30T12:41:13.503 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/15' 2023-10-30T12:41:13.503 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/20' 2023-10-30T12:41:13.503 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/12' 2023-10-30T12:41:13.503 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/10' 2023-10-30T12:41:13.504 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/11' 2023-10-30T12:41:13.504 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/16' 2023-10-30T12:41:13.504 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/14' 2023-10-30T12:41:13.504 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/18' 2023-10-30T12:41:13.504 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/13' 2023-10-30T12:41:13.505 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/17' 2023-10-30T12:41:13.505 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/194' 2023-10-30T12:41:13.505 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/19' 2023-10-30T12:41:13.505 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/15' 2023-10-30T12:41:13.505 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/20' 2023-10-30T12:41:13.506 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/12' 2023-10-30T12:41:13.506 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/10' 2023-10-30T12:41:13.506 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/11' 2023-10-30T12:41:13.506 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/16' 2023-10-30T12:41:13.506 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/14' 2023-10-30T12:41:13.507 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/18' 2023-10-30T12:41:13.507 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/13' 2023-10-30T12:41:13.507 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/17' 2023-10-30T12:41:13.507 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/119' 2023-10-30T12:41:13.507 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/19' 2023-10-30T12:41:13.507 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/15' 2023-10-30T12:41:13.508 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/20' 2023-10-30T12:41:13.508 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/12' 2023-10-30T12:41:13.508 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/10' 2023-10-30T12:41:13.508 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/11' 2023-10-30T12:41:13.508 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/16' 2023-10-30T12:41:13.509 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/14' 2023-10-30T12:41:13.509 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/18' 2023-10-30T12:41:13.509 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/13' 2023-10-30T12:41:13.509 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/17' 2023-10-30T12:41:13.509 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/157' 2023-10-30T12:41:13.510 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/19' 2023-10-30T12:41:13.510 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/15' 2023-10-30T12:41:13.510 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/20' 2023-10-30T12:41:13.510 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/12' 2023-10-30T12:41:13.510 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/10' 2023-10-30T12:41:13.511 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/11' 2023-10-30T12:41:13.511 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/16' 2023-10-30T12:41:13.511 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/14' 2023-10-30T12:41:13.511 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/18' 2023-10-30T12:41:13.511 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/13' 2023-10-30T12:41:13.512 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/17' 2023-10-30T12:41:13.512 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/172' 2023-10-30T12:41:13.512 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/19' 2023-10-30T12:41:13.512 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/15' 2023-10-30T12:41:13.512 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/20' 2023-10-30T12:41:13.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/12' 2023-10-30T12:41:13.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/10' 2023-10-30T12:41:13.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/11' 2023-10-30T12:41:13.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/16' 2023-10-30T12:41:13.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/14' 2023-10-30T12:41:13.513 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/18' 2023-10-30T12:41:13.514 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/13' 2023-10-30T12:41:13.514 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/17' 2023-10-30T12:41:13.514 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/186' 2023-10-30T12:41:13.514 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/19' 2023-10-30T12:41:13.514 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/15' 2023-10-30T12:41:13.515 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/20' 2023-10-30T12:41:13.515 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/12' 2023-10-30T12:41:13.515 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/10' 2023-10-30T12:41:13.515 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/11' 2023-10-30T12:41:13.515 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/16' 2023-10-30T12:41:13.517 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/14' 2023-10-30T12:41:14.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/18' 2023-10-30T12:41:14.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/13' 2023-10-30T12:41:14.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/17' 2023-10-30T12:41:14.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/190' 2023-10-30T12:41:14.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/19' 2023-10-30T12:41:14.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/15' 2023-10-30T12:41:14.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/20' 2023-10-30T12:41:14.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/12' 2023-10-30T12:41:14.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/10' 2023-10-30T12:41:14.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/11' 2023-10-30T12:41:14.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/16' 2023-10-30T12:41:14.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/14' 2023-10-30T12:41:14.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/18' 2023-10-30T12:41:14.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/13' 2023-10-30T12:41:14.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/17' 2023-10-30T12:41:14.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/104' 2023-10-30T12:41:14.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/19' 2023-10-30T12:41:14.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/15' 2023-10-30T12:41:14.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/20' 2023-10-30T12:41:14.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/12' 2023-10-30T12:41:14.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/10' 2023-10-30T12:41:14.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/11' 2023-10-30T12:41:14.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/16' 2023-10-30T12:41:14.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/14' 2023-10-30T12:41:14.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/18' 2023-10-30T12:41:14.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/13' 2023-10-30T12:41:14.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/17' 2023-10-30T12:41:14.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/164' 2023-10-30T12:41:14.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/19' 2023-10-30T12:41:14.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/15' 2023-10-30T12:41:14.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/20' 2023-10-30T12:41:14.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/12' 2023-10-30T12:41:14.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/10' 2023-10-30T12:41:14.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/11' 2023-10-30T12:41:14.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/16' 2023-10-30T12:41:14.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/14' 2023-10-30T12:41:14.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/18' 2023-10-30T12:41:14.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/13' 2023-10-30T12:41:14.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/17' 2023-10-30T12:41:14.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/112' 2023-10-30T12:41:14.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/19' 2023-10-30T12:41:14.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/15' 2023-10-30T12:41:14.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/20' 2023-10-30T12:41:14.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/12' 2023-10-30T12:41:14.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/10' 2023-10-30T12:41:14.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/11' 2023-10-30T12:41:14.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/16' 2023-10-30T12:41:14.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/14' 2023-10-30T12:41:14.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/18' 2023-10-30T12:41:14.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/13' 2023-10-30T12:41:14.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/17' 2023-10-30T12:41:14.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/100' 2023-10-30T12:41:14.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/19' 2023-10-30T12:41:14.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/15' 2023-10-30T12:41:14.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/20' 2023-10-30T12:41:14.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/12' 2023-10-30T12:41:14.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/10' 2023-10-30T12:41:14.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/11' 2023-10-30T12:41:14.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/16' 2023-10-30T12:41:14.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/14' 2023-10-30T12:41:14.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/18' 2023-10-30T12:41:14.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/13' 2023-10-30T12:41:14.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/17' 2023-10-30T12:41:14.056 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/134' 2023-10-30T12:41:14.056 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/19' 2023-10-30T12:41:14.056 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/15' 2023-10-30T12:41:14.056 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/20' 2023-10-30T12:41:14.056 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/12' 2023-10-30T12:41:14.057 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/10' 2023-10-30T12:41:14.057 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/11' 2023-10-30T12:41:14.057 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/16' 2023-10-30T12:41:14.057 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/14' 2023-10-30T12:41:14.057 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/18' 2023-10-30T12:41:14.057 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/13' 2023-10-30T12:41:14.058 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/17' 2023-10-30T12:41:14.058 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/115' 2023-10-30T12:41:14.058 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/19' 2023-10-30T12:41:14.058 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/15' 2023-10-30T12:41:14.058 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/20' 2023-10-30T12:41:14.059 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/12' 2023-10-30T12:41:14.059 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/10' 2023-10-30T12:41:14.059 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/11' 2023-10-30T12:41:14.059 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/16' 2023-10-30T12:41:14.059 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/14' 2023-10-30T12:41:14.060 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/18' 2023-10-30T12:41:14.060 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/13' 2023-10-30T12:41:14.060 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/17' 2023-10-30T12:41:14.060 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/116' 2023-10-30T12:41:14.060 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/19' 2023-10-30T12:41:14.061 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/15' 2023-10-30T12:41:14.061 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/20' 2023-10-30T12:41:14.061 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/12' 2023-10-30T12:41:14.061 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/10' 2023-10-30T12:41:14.061 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/11' 2023-10-30T12:41:14.061 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/16' 2023-10-30T12:41:14.062 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/14' 2023-10-30T12:41:14.062 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/18' 2023-10-30T12:41:14.062 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/13' 2023-10-30T12:41:14.062 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/17' 2023-10-30T12:41:14.062 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/195' 2023-10-30T12:41:14.063 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/19' 2023-10-30T12:41:14.063 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/15' 2023-10-30T12:41:14.063 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/20' 2023-10-30T12:41:14.063 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/12' 2023-10-30T12:41:14.063 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/10' 2023-10-30T12:41:14.064 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/11' 2023-10-30T12:41:14.064 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/16' 2023-10-30T12:41:14.064 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/14' 2023-10-30T12:41:14.064 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/18' 2023-10-30T12:41:14.359 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/13' 2023-10-30T12:41:14.359 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/17' 2023-10-30T12:41:14.359 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/138' 2023-10-30T12:41:14.359 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/19' 2023-10-30T12:41:14.359 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/15' 2023-10-30T12:41:14.360 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/20' 2023-10-30T12:41:14.360 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/12' 2023-10-30T12:41:14.360 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/10' 2023-10-30T12:41:14.360 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/11' 2023-10-30T12:41:14.360 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/16' 2023-10-30T12:41:14.361 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/14' 2023-10-30T12:41:14.361 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/18' 2023-10-30T12:41:14.361 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/13' 2023-10-30T12:41:14.361 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/17' 2023-10-30T12:41:14.361 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/135' 2023-10-30T12:41:14.362 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/19' 2023-10-30T12:41:14.362 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/15' 2023-10-30T12:41:14.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/20' 2023-10-30T12:41:14.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/12' 2023-10-30T12:41:14.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/10' 2023-10-30T12:41:14.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/11' 2023-10-30T12:41:14.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/16' 2023-10-30T12:41:14.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/14' 2023-10-30T12:41:14.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/18' 2023-10-30T12:41:14.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/13' 2023-10-30T12:41:14.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/17' 2023-10-30T12:41:14.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/117' 2023-10-30T12:41:14.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/19' 2023-10-30T12:41:14.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/15' 2023-10-30T12:41:14.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/20' 2023-10-30T12:41:14.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/12' 2023-10-30T12:41:14.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/10' 2023-10-30T12:41:14.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/11' 2023-10-30T12:41:14.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/16' 2023-10-30T12:41:14.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/14' 2023-10-30T12:41:14.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/18' 2023-10-30T12:41:14.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/13' 2023-10-30T12:41:14.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/17' 2023-10-30T12:41:14.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/143' 2023-10-30T12:41:14.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/19' 2023-10-30T12:41:14.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/15' 2023-10-30T12:41:14.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/20' 2023-10-30T12:41:14.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/12' 2023-10-30T12:41:14.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/10' 2023-10-30T12:41:14.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/11' 2023-10-30T12:41:14.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/16' 2023-10-30T12:41:14.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/14' 2023-10-30T12:41:14.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/18' 2023-10-30T12:41:14.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/13' 2023-10-30T12:41:14.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/17' 2023-10-30T12:41:14.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/180' 2023-10-30T12:41:14.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/19' 2023-10-30T12:41:14.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/15' 2023-10-30T12:41:14.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/20' 2023-10-30T12:41:14.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/12' 2023-10-30T12:41:14.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/10' 2023-10-30T12:41:14.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/11' 2023-10-30T12:41:14.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/16' 2023-10-30T12:41:14.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/14' 2023-10-30T12:41:14.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/18' 2023-10-30T12:41:14.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/13' 2023-10-30T12:41:14.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/17' 2023-10-30T12:41:14.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/126' 2023-10-30T12:41:14.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/19' 2023-10-30T12:41:14.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/15' 2023-10-30T12:41:14.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/20' 2023-10-30T12:41:14.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/12' 2023-10-30T12:41:14.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/10' 2023-10-30T12:41:14.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/11' 2023-10-30T12:41:14.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/16' 2023-10-30T12:41:14.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/14' 2023-10-30T12:41:14.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/18' 2023-10-30T12:41:14.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/13' 2023-10-30T12:41:14.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/17' 2023-10-30T12:41:14.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/161' 2023-10-30T12:41:14.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/19' 2023-10-30T12:41:14.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/15' 2023-10-30T12:41:14.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/20' 2023-10-30T12:41:14.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/12' 2023-10-30T12:41:14.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/10' 2023-10-30T12:41:14.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/11' 2023-10-30T12:41:14.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/16' 2023-10-30T12:41:14.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/14' 2023-10-30T12:41:14.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/18' 2023-10-30T12:41:14.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/13' 2023-10-30T12:41:14.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/17' 2023-10-30T12:41:14.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/176' 2023-10-30T12:41:14.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/19' 2023-10-30T12:41:14.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/15' 2023-10-30T12:41:14.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/20' 2023-10-30T12:41:14.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/12' 2023-10-30T12:41:14.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/10' 2023-10-30T12:41:14.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/11' 2023-10-30T12:41:14.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/16' 2023-10-30T12:41:14.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/14' 2023-10-30T12:41:14.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/18' 2023-10-30T12:41:14.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/13' 2023-10-30T12:41:14.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/17' 2023-10-30T12:41:14.406 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/111' 2023-10-30T12:41:14.406 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/19' 2023-10-30T12:41:14.406 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/15' 2023-10-30T12:41:14.406 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/20' 2023-10-30T12:41:14.406 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/12' 2023-10-30T12:41:14.407 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/10' 2023-10-30T12:41:14.407 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/11' 2023-10-30T12:41:14.407 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/16' 2023-10-30T12:41:14.407 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/14' 2023-10-30T12:41:14.407 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/18' 2023-10-30T12:41:14.407 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/13' 2023-10-30T12:41:14.567 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:41:14.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff722bc4640 1 Processor -- start 2023-10-30T12:41:14.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff722bc4640 1 -- start start 2023-10-30T12:41:14.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff722bc4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff71c06f7a0 0x7ff71c06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:14.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff722bc4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff71c0c6bd0 0x7ff71c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:14.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff722bc4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff71c06a3d0 con 0x7ff71c071110 2023-10-30T12:41:14.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff722bc4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff71c0700c0 con 0x7ff71c06f7a0 2023-10-30T12:41:14.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff722bc4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c066cf0 con 0x7ff71c0c6bd0 2023-10-30T12:41:14.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff71bfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff71c0c6bd0 0x7ff71c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:14.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff720960640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff71c06f7a0 0x7ff71c06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:14.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff71bfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff71c0c6bd0 0x7ff71c069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:49242/0 (socket says 172.21.15.12:49242) 2023-10-30T12:41:14.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff71bfff640 1 -- 172.21.15.12:0/2302469807 learned_addr learned my addr 172.21.15.12:0/2302469807 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:41:14.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff721161640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff71c071110 0x7ff71c158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:44020/0 (socket says 172.21.15.12:44020) 2023-10-30T12:41:14.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff71bfff640 1 -- 172.21.15.12:0/2302469807 >> v2:172.21.15.154:3300/0 conn(0x7ff71c06f7a0 msgr2=0x7ff71c06fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:41:14.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff71bfff640 1 --2- 172.21.15.12:0/2302469807 >> v2:172.21.15.154:3300/0 conn(0x7ff71c06f7a0 0x7ff71c06fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:14.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff71bfff640 1 -- 172.21.15.12:0/2302469807 >> v1:172.21.15.154:6789/0 conn(0x7ff71c071110 legacy=0x7ff71c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:14.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff71bfff640 1 -- 172.21.15.12:0/2302469807 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c0c70d0 con 0x7ff71c0c6bd0 2023-10-30T12:41:14.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.734+0000 7ff71b7fe640 1 -- 172.21.15.12:0/2302469807 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100282151 0 0) 0x7ff71c06a3d0 con 0x7ff71c071110 2023-10-30T12:41:14.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.738+0000 7ff71bfff640 1 --2- 172.21.15.12:0/2302469807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff71c0c6bd0 0x7ff71c069e90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff708009870 tx=0x7ff70801ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d83dceee22e86ca server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:41:14.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.738+0000 7ff71b7fe640 1 -- 172.21.15.12:0/2302469807 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7080a8a30 con 0x7ff71c0c6bd0 2023-10-30T12:41:14.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.738+0000 7ff71b7fe640 1 -- 172.21.15.12:0/2302469807 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7080a8b90 con 0x7ff71c0c6bd0 2023-10-30T12:41:14.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.738+0000 7ff71b7fe640 1 -- 172.21.15.12:0/2302469807 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7080b0440 con 0x7ff71c0c6bd0 2023-10-30T12:41:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.738+0000 7ff722bc4640 1 -- 172.21.15.12:0/2302469807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff71c0c6bd0 msgr2=0x7ff71c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.738+0000 7ff722bc4640 1 --2- 172.21.15.12:0/2302469807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff71c0c6bd0 0x7ff71c069e90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff708009870 tx=0x7ff70801ed30 comp rx=0 tx=0).stop 2023-10-30T12:41:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.738+0000 7ff722bc4640 1 -- 172.21.15.12:0/2302469807 shutdown_connections 2023-10-30T12:41:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.738+0000 7ff722bc4640 1 --2- 172.21.15.12:0/2302469807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff71c0c6bd0 0x7ff71c069e90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.738+0000 7ff722bc4640 1 --2- 172.21.15.12:0/2302469807 >> v2:172.21.15.154:3300/0 conn(0x7ff71c06f7a0 0x7ff71c06fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:14.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.738+0000 7ff722bc4640 1 -- 172.21.15.12:0/2302469807 >> 172.21.15.12:0/2302469807 conn(0x7ff71c0c7910 msgr2=0x7ff71c0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:41:14.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.738+0000 7ff722bc4640 1 -- 172.21.15.12:0/2302469807 shutdown_connections 2023-10-30T12:41:14.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.738+0000 7ff722bc4640 1 -- 172.21.15.12:0/2302469807 wait complete. 2023-10-30T12:41:14.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff722bc4640 1 Processor -- start 2023-10-30T12:41:14.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff722bc4640 1 -- start start 2023-10-30T12:41:14.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff722bc4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff71c06f7a0 0x7ff71c1efd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:14.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff722bc4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff71c071110 0x7ff71c150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:14.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff722bc4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff71c0c6bd0 0x7ff71c150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:14.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff722bc4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c144080 con 0x7ff71c071110 2023-10-30T12:41:14.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff722bc4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c15a630 con 0x7ff71c0c6bd0 2023-10-30T12:41:14.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff722bc4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c159b20 con 0x7ff71c06f7a0 2023-10-30T12:41:14.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff71bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff71c071110 0x7ff71c150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:14.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff71bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff71c071110 0x7ff71c150130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50746/0 (socket says 172.21.15.12:50746) 2023-10-30T12:41:14.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff71bfff640 1 -- 172.21.15.12:0/61184266 learned_addr learned my addr 172.21.15.12:0/61184266 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:41:14.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff720960640 1 --2- 172.21.15.12:0/61184266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff71c06f7a0 0x7ff71c1efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:14.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff721161640 1 --2- 172.21.15.12:0/61184266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff71c0c6bd0 0x7ff71c150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:14.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff71bfff640 1 -- 172.21.15.12:0/61184266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff71c06f7a0 msgr2=0x7ff71c1efd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:14.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff71bfff640 1 --2- 172.21.15.12:0/61184266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff71c06f7a0 0x7ff71c1efd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:14.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff71bfff640 1 -- 172.21.15.12:0/61184266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff71c0c6bd0 msgr2=0x7ff71c150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:14.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff71bfff640 1 --2- 172.21.15.12:0/61184266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff71c0c6bd0 0x7ff71c150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:14.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff71bfff640 1 -- 172.21.15.12:0/61184266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7080094d0 con 0x7ff71c071110 2023-10-30T12:41:14.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff71bfff640 1 --2- 172.21.15.12:0/61184266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff71c071110 0x7ff71c150130 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff7080b3040 tx=0x7ff70801bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:41:14.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff7197fa640 1 -- 172.21.15.12:0/61184266 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7080b0dd0 con 0x7ff71c071110 2023-10-30T12:41:14.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff7197fa640 1 -- 172.21.15.12:0/61184266 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7080bb460 con 0x7ff71c071110 2023-10-30T12:41:14.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff7197fa640 1 -- 172.21.15.12:0/61184266 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7080bb810 con 0x7ff71c071110 2023-10-30T12:41:14.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff722bc4640 1 -- 172.21.15.12:0/61184266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff71c1498e0 con 0x7ff71c071110 2023-10-30T12:41:14.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff722bc4640 1 -- 172.21.15.12:0/61184266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71c159020 con 0x7ff71c071110 2023-10-30T12:41:14.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.742+0000 7ff722bc4640 1 -- 172.21.15.12:0/61184266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff71c1591b0 con 0x7ff71c071110 2023-10-30T12:41:14.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.746+0000 7ff7197fa640 1 -- 172.21.15.12:0/61184266 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff7080b83f0 con 0x7ff71c071110 2023-10-30T12:41:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.746+0000 7ff7197fa640 1 --2- 172.21.15.12:0/61184266 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff7000664a0 0x7ff700068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:41:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.746+0000 7ff7197fa640 1 -- 172.21.15.12:0/61184266 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5997+0+0 (secure 0 0 0) 0x7ff7081217b0 con 0x7ff71c071110 2023-10-30T12:41:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.746+0000 7ff720960640 1 --2- 172.21.15.12:0/61184266 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff7000664a0 0x7ff700068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:41:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.746+0000 7ff720960640 1 --2- 172.21.15.12:0/61184266 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff7000664a0 0x7ff700068960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff710009ac0 tx=0x7ff710009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:41:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.750+0000 7ff7197fa640 1 -- 172.21.15.12:0/61184266 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff7080eeb20 con 0x7ff71c071110 2023-10-30T12:41:14.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.942+0000 7ff722bc4640 1 -- 172.21.15.12:0/61184266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff71c04e170 con 0x7ff71c071110 2023-10-30T12:41:14.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.946+0000 7ff7197fa640 1 -- 172.21.15.12:0/61184266 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+6530 (secure 0 0 0) 0x7ff7080f2140 con 0x7ff71c071110 2023-10-30T12:41:14.953 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:41:14.954 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:41:13.242308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:41:14.955 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 31 2023-10-30T12:41:14.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.946+0000 7ff722bc4640 1 -- 172.21.15.12:0/61184266 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff7000664a0 msgr2=0x7ff700068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:14.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.946+0000 7ff722bc4640 1 --2- 172.21.15.12:0/61184266 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff7000664a0 0x7ff700068960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff710009ac0 tx=0x7ff710009190 comp rx=0 tx=0).stop 2023-10-30T12:41:14.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.946+0000 7ff722bc4640 1 -- 172.21.15.12:0/61184266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff71c071110 msgr2=0x7ff71c150130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:41:14.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.946+0000 7ff722bc4640 1 --2- 172.21.15.12:0/61184266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff71c071110 0x7ff71c150130 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff7080b3040 tx=0x7ff70801bad0 comp rx=0 tx=0).stop 2023-10-30T12:41:14.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.946+0000 7ff722bc4640 1 -- 172.21.15.12:0/61184266 shutdown_connections 2023-10-30T12:41:14.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.946+0000 7ff722bc4640 1 --2- 172.21.15.12:0/61184266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff71c0c6bd0 0x7ff71c150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:14.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.946+0000 7ff722bc4640 1 --2- 172.21.15.12:0/61184266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff71c071110 0x7ff71c150130 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:14.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.946+0000 7ff722bc4640 1 --2- 172.21.15.12:0/61184266 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff7000664a0 0x7ff700068960 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:14.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.946+0000 7ff722bc4640 1 --2- 172.21.15.12:0/61184266 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff71c06f7a0 0x7ff71c1efd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:41:14.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.946+0000 7ff722bc4640 1 -- 172.21.15.12:0/61184266 >> 172.21.15.12:0/61184266 conn(0x7ff71c0c7910 msgr2=0x7ff71c0c2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:41:14.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.946+0000 7ff722bc4640 1 -- 172.21.15.12:0/61184266 shutdown_connections 2023-10-30T12:41:14.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:41:14.946+0000 7ff722bc4640 1 -- 172.21.15.12:0/61184266 wait complete. 2023-10-30T12:41:14.973 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2023-10-30T12:41:14.973 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 102 secs before thrashing 2023-10-30T12:41:16.181 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/17' 2023-10-30T12:41:16.182 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/191' 2023-10-30T12:41:16.182 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/19' 2023-10-30T12:41:16.182 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/15' 2023-10-30T12:41:16.182 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/20' 2023-10-30T12:41:16.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/12' 2023-10-30T12:41:16.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/10' 2023-10-30T12:41:16.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/11' 2023-10-30T12:41:16.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/16' 2023-10-30T12:41:16.183 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/14' 2023-10-30T12:41:16.184 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/18' 2023-10-30T12:41:16.184 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/13' 2023-10-30T12:41:16.184 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/17' 2023-10-30T12:41:16.184 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/184' 2023-10-30T12:41:16.184 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/19' 2023-10-30T12:41:16.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/15' 2023-10-30T12:41:16.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/20' 2023-10-30T12:41:16.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/12' 2023-10-30T12:41:16.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/10' 2023-10-30T12:41:16.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/11' 2023-10-30T12:41:16.185 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/16' 2023-10-30T12:41:16.186 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/14' 2023-10-30T12:41:16.186 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/18' 2023-10-30T12:41:16.186 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/13' 2023-10-30T12:41:16.186 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/17' 2023-10-30T12:41:16.186 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/192' 2023-10-30T12:41:16.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/19' 2023-10-30T12:41:16.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/15' 2023-10-30T12:41:16.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/20' 2023-10-30T12:41:16.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/12' 2023-10-30T12:41:16.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/10' 2023-10-30T12:41:16.187 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/11' 2023-10-30T12:41:16.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/16' 2023-10-30T12:41:16.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/14' 2023-10-30T12:41:16.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/18' 2023-10-30T12:41:16.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/13' 2023-10-30T12:41:16.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/17' 2023-10-30T12:41:16.188 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/103' 2023-10-30T12:41:16.189 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/19' 2023-10-30T12:41:16.189 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/15' 2023-10-30T12:41:16.189 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/20' 2023-10-30T12:41:16.189 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/12' 2023-10-30T12:41:16.189 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/10' 2023-10-30T12:41:16.190 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/11' 2023-10-30T12:41:16.190 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/16' 2023-10-30T12:41:16.190 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/14' 2023-10-30T12:41:16.190 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/18' 2023-10-30T12:41:16.190 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/13' 2023-10-30T12:41:16.191 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/17' 2023-10-30T12:41:16.191 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/128' 2023-10-30T12:41:16.191 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/19' 2023-10-30T12:41:16.191 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/15' 2023-10-30T12:41:16.191 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/20' 2023-10-30T12:41:16.191 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/12' 2023-10-30T12:41:16.192 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/10' 2023-10-30T12:41:16.192 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/11' 2023-10-30T12:41:16.192 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/16' 2023-10-30T12:41:16.192 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/14' 2023-10-30T12:41:16.192 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/18' 2023-10-30T12:41:16.192 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/13' 2023-10-30T12:41:16.193 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/17' 2023-10-30T12:41:16.193 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/144' 2023-10-30T12:41:16.193 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/19' 2023-10-30T12:41:16.193 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/15' 2023-10-30T12:41:16.193 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/20' 2023-10-30T12:41:16.194 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/12' 2023-10-30T12:41:16.194 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/10' 2023-10-30T12:41:16.194 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/11' 2023-10-30T12:41:16.194 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/16' 2023-10-30T12:41:16.194 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/14' 2023-10-30T12:41:16.194 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/18' 2023-10-30T12:41:16.195 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/13' 2023-10-30T12:41:16.195 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/17' 2023-10-30T12:41:16.195 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/146' 2023-10-30T12:41:16.195 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/19' 2023-10-30T12:41:16.195 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/15' 2023-10-30T12:41:16.196 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/20' 2023-10-30T12:41:16.196 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/12' 2023-10-30T12:41:16.196 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/10' 2023-10-30T12:41:16.196 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/11' 2023-10-30T12:41:16.196 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/16' 2023-10-30T12:41:16.196 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/14' 2023-10-30T12:41:16.197 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/18' 2023-10-30T12:41:16.197 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/13' 2023-10-30T12:41:16.197 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/17' 2023-10-30T12:41:16.197 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/141' 2023-10-30T12:41:16.197 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/19' 2023-10-30T12:41:16.197 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/15' 2023-10-30T12:41:16.198 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/20' 2023-10-30T12:41:16.198 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/12' 2023-10-30T12:41:16.198 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/10' 2023-10-30T12:41:16.198 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/11' 2023-10-30T12:41:16.198 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/16' 2023-10-30T12:41:16.199 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/14' 2023-10-30T12:41:16.199 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/18' 2023-10-30T12:41:16.199 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/13' 2023-10-30T12:41:16.199 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/17' 2023-10-30T12:41:16.199 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/182' 2023-10-30T12:41:16.200 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/19' 2023-10-30T12:41:16.200 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/15' 2023-10-30T12:41:16.200 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/20' 2023-10-30T12:41:16.200 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/12' 2023-10-30T12:41:16.200 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/10' 2023-10-30T12:41:16.200 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/11' 2023-10-30T12:41:16.201 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/16' 2023-10-30T12:41:16.201 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/14' 2023-10-30T12:41:16.201 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/18' 2023-10-30T12:41:16.201 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/13' 2023-10-30T12:41:16.201 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/17' 2023-10-30T12:41:17.386 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/107' 2023-10-30T12:41:17.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/19' 2023-10-30T12:41:17.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/15' 2023-10-30T12:41:17.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/20' 2023-10-30T12:41:17.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/12' 2023-10-30T12:41:17.387 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/10' 2023-10-30T12:41:17.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/11' 2023-10-30T12:41:17.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/16' 2023-10-30T12:41:17.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/14' 2023-10-30T12:41:17.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/18' 2023-10-30T12:41:17.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/13' 2023-10-30T12:41:17.388 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/17' 2023-10-30T12:41:17.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/178' 2023-10-30T12:41:17.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/19' 2023-10-30T12:41:17.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/15' 2023-10-30T12:41:17.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/20' 2023-10-30T12:41:17.389 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/12' 2023-10-30T12:41:17.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/10' 2023-10-30T12:41:17.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/11' 2023-10-30T12:41:17.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/16' 2023-10-30T12:41:17.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/14' 2023-10-30T12:41:17.390 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/18' 2023-10-30T12:41:17.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/13' 2023-10-30T12:41:17.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/17' 2023-10-30T12:41:17.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/156' 2023-10-30T12:41:17.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/19' 2023-10-30T12:41:17.391 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/15' 2023-10-30T12:41:17.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/20' 2023-10-30T12:41:17.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/12' 2023-10-30T12:41:17.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/10' 2023-10-30T12:41:17.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/11' 2023-10-30T12:41:17.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/16' 2023-10-30T12:41:17.392 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/14' 2023-10-30T12:41:17.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/18' 2023-10-30T12:41:17.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/13' 2023-10-30T12:41:17.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/17' 2023-10-30T12:41:17.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/131' 2023-10-30T12:41:17.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/19' 2023-10-30T12:41:17.393 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/15' 2023-10-30T12:41:17.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/20' 2023-10-30T12:41:17.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/12' 2023-10-30T12:41:17.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/10' 2023-10-30T12:41:17.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/11' 2023-10-30T12:41:17.394 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/16' 2023-10-30T12:41:17.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/14' 2023-10-30T12:41:17.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/18' 2023-10-30T12:41:17.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/13' 2023-10-30T12:41:17.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/17' 2023-10-30T12:41:17.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/123' 2023-10-30T12:41:17.395 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/19' 2023-10-30T12:41:17.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/15' 2023-10-30T12:41:17.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/20' 2023-10-30T12:41:17.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/12' 2023-10-30T12:41:17.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/10' 2023-10-30T12:41:17.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/11' 2023-10-30T12:41:17.396 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/16' 2023-10-30T12:41:17.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/14' 2023-10-30T12:41:17.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/18' 2023-10-30T12:41:17.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/13' 2023-10-30T12:41:17.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/17' 2023-10-30T12:41:17.397 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/160' 2023-10-30T12:41:17.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/19' 2023-10-30T12:41:17.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/15' 2023-10-30T12:41:17.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/20' 2023-10-30T12:41:17.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/12' 2023-10-30T12:41:17.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/10' 2023-10-30T12:41:17.398 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/11' 2023-10-30T12:41:17.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/16' 2023-10-30T12:41:17.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/14' 2023-10-30T12:41:17.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/18' 2023-10-30T12:41:17.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/13' 2023-10-30T12:41:17.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/17' 2023-10-30T12:41:17.399 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/165' 2023-10-30T12:41:17.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/19' 2023-10-30T12:41:17.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/15' 2023-10-30T12:41:17.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/20' 2023-10-30T12:41:17.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/12' 2023-10-30T12:41:17.400 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/10' 2023-10-30T12:41:17.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/11' 2023-10-30T12:41:17.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/16' 2023-10-30T12:41:17.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/14' 2023-10-30T12:41:17.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/18' 2023-10-30T12:41:17.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/13' 2023-10-30T12:41:17.401 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/17' 2023-10-30T12:41:17.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/101' 2023-10-30T12:41:17.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/19' 2023-10-30T12:41:17.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/15' 2023-10-30T12:41:17.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/20' 2023-10-30T12:41:17.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/12' 2023-10-30T12:41:17.402 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/10' 2023-10-30T12:41:17.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/11' 2023-10-30T12:41:17.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/16' 2023-10-30T12:41:17.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/14' 2023-10-30T12:41:17.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/18' 2023-10-30T12:41:17.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/13' 2023-10-30T12:41:17.403 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/17' 2023-10-30T12:41:17.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/148' 2023-10-30T12:41:17.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/19' 2023-10-30T12:41:17.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/15' 2023-10-30T12:41:17.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/20' 2023-10-30T12:41:17.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/12' 2023-10-30T12:41:17.404 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/10' 2023-10-30T12:41:17.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/11' 2023-10-30T12:41:17.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/16' 2023-10-30T12:41:17.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/14' 2023-10-30T12:41:17.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/18' 2023-10-30T12:41:17.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/13' 2023-10-30T12:41:17.405 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/17' 2023-10-30T12:41:17.406 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/168' 2023-10-30T12:41:19.732 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/19' 2023-10-30T12:41:19.732 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/15' 2023-10-30T12:41:19.732 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/20' 2023-10-30T12:41:19.733 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/12' 2023-10-30T12:41:19.733 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/10' 2023-10-30T12:41:19.733 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/11' 2023-10-30T12:41:19.733 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/16' 2023-10-30T12:41:19.733 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/14' 2023-10-30T12:41:19.733 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/18' 2023-10-30T12:41:19.734 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/13' 2023-10-30T12:41:19.734 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/17' 2023-10-30T12:41:19.734 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/179' 2023-10-30T12:41:19.734 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/19' 2023-10-30T12:41:19.734 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/15' 2023-10-30T12:41:19.735 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/20' 2023-10-30T12:41:19.735 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/12' 2023-10-30T12:41:19.735 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/10' 2023-10-30T12:41:19.735 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/11' 2023-10-30T12:41:19.735 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/16' 2023-10-30T12:41:19.735 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/14' 2023-10-30T12:41:19.736 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/18' 2023-10-30T12:41:19.736 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/13' 2023-10-30T12:41:19.736 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/17' 2023-10-30T12:41:19.736 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/132' 2023-10-30T12:41:19.736 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/19' 2023-10-30T12:41:19.736 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/15' 2023-10-30T12:41:19.737 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/20' 2023-10-30T12:41:19.737 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/12' 2023-10-30T12:41:19.737 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/10' 2023-10-30T12:41:19.737 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/11' 2023-10-30T12:41:19.737 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/16' 2023-10-30T12:41:19.738 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/14' 2023-10-30T12:41:19.738 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/18' 2023-10-30T12:41:19.738 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/13' 2023-10-30T12:41:19.739 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/17' 2023-10-30T12:41:19.739 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/199' 2023-10-30T12:41:19.739 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/19' 2023-10-30T12:41:19.739 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/15' 2023-10-30T12:41:19.739 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/20' 2023-10-30T12:41:19.740 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/12' 2023-10-30T12:41:19.740 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/10' 2023-10-30T12:41:19.740 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/11' 2023-10-30T12:41:19.740 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/16' 2023-10-30T12:41:19.740 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/14' 2023-10-30T12:41:19.741 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/18' 2023-10-30T12:41:19.741 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/13' 2023-10-30T12:41:19.741 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/17' 2023-10-30T12:41:19.741 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/163' 2023-10-30T12:41:19.741 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/19' 2023-10-30T12:41:19.741 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/15' 2023-10-30T12:41:19.742 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/20' 2023-10-30T12:41:19.742 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/12' 2023-10-30T12:41:19.742 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/10' 2023-10-30T12:41:19.742 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/11' 2023-10-30T12:41:19.742 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/16' 2023-10-30T12:41:19.743 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/14' 2023-10-30T12:41:19.743 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/18' 2023-10-30T12:41:19.743 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/13' 2023-10-30T12:41:19.743 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/17' 2023-10-30T12:41:19.743 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/153' 2023-10-30T12:41:19.744 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/19' 2023-10-30T12:41:19.744 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/15' 2023-10-30T12:41:19.744 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/20' 2023-10-30T12:41:19.744 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/12' 2023-10-30T12:41:19.744 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/10' 2023-10-30T12:41:19.744 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/11' 2023-10-30T12:41:19.745 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/16' 2023-10-30T12:41:19.745 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/14' 2023-10-30T12:41:19.745 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/18' 2023-10-30T12:41:19.745 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/13' 2023-10-30T12:41:19.745 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/17' 2023-10-30T12:41:19.746 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/122' 2023-10-30T12:41:19.746 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/19' 2023-10-30T12:41:19.746 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/15' 2023-10-30T12:41:19.746 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/20' 2023-10-30T12:41:19.746 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/12' 2023-10-30T12:41:19.746 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/10' 2023-10-30T12:41:19.747 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/11' 2023-10-30T12:41:19.747 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/16' 2023-10-30T12:41:19.747 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/14' 2023-10-30T12:41:19.747 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/18' 2023-10-30T12:41:19.747 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/13' 2023-10-30T12:41:19.748 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/17' 2023-10-30T12:41:19.748 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/162' 2023-10-30T12:41:19.748 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/19' 2023-10-30T12:41:19.748 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/15' 2023-10-30T12:41:19.748 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/20' 2023-10-30T12:41:19.748 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/12' 2023-10-30T12:41:19.749 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/10' 2023-10-30T12:41:19.749 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/11' 2023-10-30T12:41:19.749 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/16' 2023-10-30T12:41:19.749 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/14' 2023-10-30T12:41:19.749 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/18' 2023-10-30T12:41:19.749 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/13' 2023-10-30T12:41:19.750 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/17' 2023-10-30T12:41:19.750 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/197' 2023-10-30T12:41:19.750 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/19' 2023-10-30T12:41:19.750 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/15' 2023-10-30T12:41:19.750 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/20' 2023-10-30T12:41:19.750 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/12' 2023-10-30T12:41:19.751 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/10' 2023-10-30T12:41:19.751 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/11' 2023-10-30T12:41:19.751 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/16' 2023-10-30T12:41:19.751 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/14' 2023-10-30T12:41:19.751 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/18' 2023-10-30T12:41:19.752 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/13' 2023-10-30T12:41:19.752 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/17' 2023-10-30T12:41:19.752 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/113' 2023-10-30T12:41:19.752 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/19' 2023-10-30T12:41:20.738 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/15' 2023-10-30T12:41:20.739 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/20' 2023-10-30T12:41:20.739 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/12' 2023-10-30T12:41:20.739 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/10' 2023-10-30T12:41:20.739 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/11' 2023-10-30T12:41:20.739 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/16' 2023-10-30T12:41:20.740 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/14' 2023-10-30T12:41:20.740 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/18' 2023-10-30T12:41:20.740 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/13' 2023-10-30T12:41:20.740 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/17' 2023-10-30T12:41:20.740 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/152' 2023-10-30T12:41:20.741 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/19' 2023-10-30T12:41:20.741 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/15' 2023-10-30T12:41:20.741 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/20' 2023-10-30T12:41:20.741 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/12' 2023-10-30T12:41:20.741 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/10' 2023-10-30T12:41:20.742 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/11' 2023-10-30T12:41:20.742 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/16' 2023-10-30T12:41:20.742 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/14' 2023-10-30T12:41:20.742 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/18' 2023-10-30T12:41:20.742 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/13' 2023-10-30T12:41:20.742 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/17' 2023-10-30T12:41:20.743 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/121' 2023-10-30T12:41:20.743 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/19' 2023-10-30T12:41:20.743 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/15' 2023-10-30T12:41:20.743 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/20' 2023-10-30T12:41:20.743 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/12' 2023-10-30T12:41:20.744 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/10' 2023-10-30T12:41:20.744 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/11' 2023-10-30T12:41:20.744 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/16' 2023-10-30T12:41:20.744 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/14' 2023-10-30T12:41:20.744 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/18' 2023-10-30T12:41:20.744 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/13' 2023-10-30T12:41:20.745 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/17' 2023-10-30T12:41:20.745 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/118' 2023-10-30T12:41:20.745 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/19' 2023-10-30T12:41:20.745 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/15' 2023-10-30T12:41:20.745 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/20' 2023-10-30T12:41:20.746 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/12' 2023-10-30T12:41:20.746 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/10' 2023-10-30T12:41:20.746 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/11' 2023-10-30T12:41:20.746 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/16' 2023-10-30T12:41:20.746 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/14' 2023-10-30T12:41:20.746 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/18' 2023-10-30T12:41:20.747 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/13' 2023-10-30T12:41:20.747 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/17' 2023-10-30T12:41:20.747 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/196' 2023-10-30T12:41:20.747 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/19' 2023-10-30T12:41:20.747 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/15' 2023-10-30T12:41:20.748 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/20' 2023-10-30T12:41:20.748 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/12' 2023-10-30T12:41:20.748 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/10' 2023-10-30T12:41:20.748 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/11' 2023-10-30T12:41:20.748 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/16' 2023-10-30T12:41:20.748 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/14' 2023-10-30T12:41:20.749 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/18' 2023-10-30T12:41:20.749 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/13' 2023-10-30T12:41:20.749 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/17' 2023-10-30T12:41:20.749 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/124' 2023-10-30T12:41:20.749 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/19' 2023-10-30T12:41:20.749 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/15' 2023-10-30T12:41:20.750 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/20' 2023-10-30T12:41:20.750 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/12' 2023-10-30T12:41:20.750 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/10' 2023-10-30T12:41:20.750 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/11' 2023-10-30T12:41:20.750 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/16' 2023-10-30T12:41:20.751 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/14' 2023-10-30T12:41:20.751 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/18' 2023-10-30T12:41:20.751 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/13' 2023-10-30T12:41:20.751 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/17' 2023-10-30T12:41:20.751 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/188' 2023-10-30T12:41:20.751 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/19' 2023-10-30T12:41:20.752 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/15' 2023-10-30T12:41:20.752 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/20' 2023-10-30T12:41:20.752 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/12' 2023-10-30T12:41:20.752 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/10' 2023-10-30T12:41:20.752 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/11' 2023-10-30T12:41:20.752 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/16' 2023-10-30T12:41:20.753 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/14' 2023-10-30T12:41:20.753 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/18' 2023-10-30T12:41:20.753 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/13' 2023-10-30T12:41:20.753 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/17' 2023-10-30T12:41:20.753 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/145' 2023-10-30T12:41:20.754 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/19' 2023-10-30T12:41:20.754 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/15' 2023-10-30T12:41:20.754 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/20' 2023-10-30T12:41:20.754 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/12' 2023-10-30T12:41:20.754 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/10' 2023-10-30T12:41:20.755 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/11' 2023-10-30T12:41:20.755 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/16' 2023-10-30T12:41:20.755 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/14' 2023-10-30T12:41:20.755 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/18' 2023-10-30T12:41:20.755 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/13' 2023-10-30T12:41:20.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/17' 2023-10-30T12:41:20.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/130' 2023-10-30T12:41:20.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/19' 2023-10-30T12:41:20.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/15' 2023-10-30T12:41:20.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/20' 2023-10-30T12:41:20.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/12' 2023-10-30T12:41:20.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/10' 2023-10-30T12:41:20.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/11' 2023-10-30T12:41:20.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/16' 2023-10-30T12:41:20.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/14' 2023-10-30T12:41:20.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/18' 2023-10-30T12:41:20.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/13' 2023-10-30T12:41:20.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/17' 2023-10-30T12:41:20.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/193' 2023-10-30T12:41:20.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/19' 2023-10-30T12:41:20.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/15' 2023-10-30T12:41:23.470 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/20' 2023-10-30T12:41:23.471 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/12' 2023-10-30T12:41:23.471 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/10' 2023-10-30T12:41:23.471 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/11' 2023-10-30T12:41:23.471 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/16' 2023-10-30T12:41:23.471 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/14' 2023-10-30T12:41:23.472 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/18' 2023-10-30T12:41:23.472 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/13' 2023-10-30T12:41:23.472 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/17' 2023-10-30T12:41:23.472 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/175' 2023-10-30T12:41:23.472 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/19' 2023-10-30T12:41:23.472 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/15' 2023-10-30T12:41:23.473 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/20' 2023-10-30T12:41:23.473 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/12' 2023-10-30T12:41:23.473 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/10' 2023-10-30T12:41:23.473 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/11' 2023-10-30T12:41:23.473 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/16' 2023-10-30T12:41:23.473 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/14' 2023-10-30T12:41:23.474 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/18' 2023-10-30T12:41:23.474 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/13' 2023-10-30T12:41:23.474 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/17' 2023-10-30T12:41:23.474 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/181' 2023-10-30T12:41:23.474 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/19' 2023-10-30T12:41:23.474 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/15' 2023-10-30T12:41:23.475 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/20' 2023-10-30T12:41:23.475 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/12' 2023-10-30T12:41:23.475 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/10' 2023-10-30T12:41:23.475 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/11' 2023-10-30T12:41:23.475 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/16' 2023-10-30T12:41:23.475 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/14' 2023-10-30T12:41:23.476 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/18' 2023-10-30T12:41:23.476 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/13' 2023-10-30T12:41:23.476 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/17' 2023-10-30T12:41:23.476 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/127' 2023-10-30T12:41:23.476 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/19' 2023-10-30T12:41:23.477 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/15' 2023-10-30T12:41:23.477 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/20' 2023-10-30T12:41:23.477 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/12' 2023-10-30T12:41:23.477 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/10' 2023-10-30T12:41:23.477 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/11' 2023-10-30T12:41:23.478 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/16' 2023-10-30T12:41:23.478 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/14' 2023-10-30T12:41:23.478 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/18' 2023-10-30T12:41:23.478 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/13' 2023-10-30T12:41:23.478 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/17' 2023-10-30T12:41:23.479 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/108' 2023-10-30T12:41:23.479 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/19' 2023-10-30T12:41:23.479 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/15' 2023-10-30T12:41:23.479 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/20' 2023-10-30T12:41:23.479 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/12' 2023-10-30T12:41:23.479 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/10' 2023-10-30T12:41:23.480 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/11' 2023-10-30T12:41:23.480 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/16' 2023-10-30T12:41:23.480 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/14' 2023-10-30T12:41:23.480 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/18' 2023-10-30T12:41:23.480 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/13' 2023-10-30T12:41:23.480 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/17' 2023-10-30T12:41:23.481 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/183' 2023-10-30T12:41:23.481 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/19' 2023-10-30T12:41:23.481 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/15' 2023-10-30T12:41:23.481 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/20' 2023-10-30T12:41:23.481 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/12' 2023-10-30T12:41:23.482 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/10' 2023-10-30T12:41:23.482 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/11' 2023-10-30T12:41:23.482 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/16' 2023-10-30T12:41:23.482 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/14' 2023-10-30T12:41:23.482 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/18' 2023-10-30T12:41:23.482 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/13' 2023-10-30T12:41:23.483 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/17' 2023-10-30T12:41:23.483 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/177' 2023-10-30T12:41:23.483 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/19' 2023-10-30T12:41:23.483 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/15' 2023-10-30T12:41:23.483 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/20' 2023-10-30T12:41:23.483 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/12' 2023-10-30T12:41:23.484 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/10' 2023-10-30T12:41:23.484 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/11' 2023-10-30T12:41:23.484 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/16' 2023-10-30T12:41:23.484 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/14' 2023-10-30T12:41:23.484 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/18' 2023-10-30T12:41:23.484 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/13' 2023-10-30T12:41:23.484 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/17' 2023-10-30T12:41:23.485 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/169' 2023-10-30T12:41:23.485 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/19' 2023-10-30T12:41:23.485 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/15' 2023-10-30T12:41:23.485 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/20' 2023-10-30T12:41:23.485 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/12' 2023-10-30T12:41:23.486 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/10' 2023-10-30T12:41:23.486 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/11' 2023-10-30T12:41:23.486 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/16' 2023-10-30T12:41:23.486 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/14' 2023-10-30T12:41:23.486 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/18' 2023-10-30T12:41:23.486 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/13' 2023-10-30T12:41:23.487 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/17' 2023-10-30T12:41:23.487 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/154' 2023-10-30T12:41:23.487 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/19' 2023-10-30T12:41:23.487 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/15' 2023-10-30T12:41:23.487 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/20' 2023-10-30T12:41:23.487 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/12' 2023-10-30T12:41:23.488 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/10' 2023-10-30T12:41:23.488 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/11' 2023-10-30T12:41:23.488 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/16' 2023-10-30T12:41:23.488 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/14' 2023-10-30T12:41:23.488 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/18' 2023-10-30T12:41:23.489 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/13' 2023-10-30T12:41:23.489 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/17' 2023-10-30T12:41:23.489 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/171' 2023-10-30T12:41:23.489 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/19' 2023-10-30T12:41:23.489 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/15' 2023-10-30T12:41:23.489 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/20' 2023-10-30T12:41:23.593 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/12' 2023-10-30T12:41:23.593 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/10' 2023-10-30T12:41:23.594 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/11' 2023-10-30T12:41:23.594 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/16' 2023-10-30T12:41:23.594 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/14' 2023-10-30T12:41:23.594 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/18' 2023-10-30T12:41:23.594 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/13' 2023-10-30T12:41:23.595 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/17' 2023-10-30T12:41:23.595 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/139' 2023-10-30T12:41:23.595 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/19' 2023-10-30T12:41:23.595 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/15' 2023-10-30T12:41:23.595 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/20' 2023-10-30T12:41:23.595 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/12' 2023-10-30T12:41:23.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/10' 2023-10-30T12:41:23.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/11' 2023-10-30T12:41:23.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/16' 2023-10-30T12:41:23.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/14' 2023-10-30T12:41:23.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/18' 2023-10-30T12:41:23.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/13' 2023-10-30T12:41:23.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/17' 2023-10-30T12:41:23.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/102' 2023-10-30T12:41:23.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/19' 2023-10-30T12:41:23.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/15' 2023-10-30T12:41:23.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/20' 2023-10-30T12:41:23.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/12' 2023-10-30T12:41:23.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/10' 2023-10-30T12:41:23.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/11' 2023-10-30T12:41:23.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/16' 2023-10-30T12:41:23.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/14' 2023-10-30T12:41:23.599 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/18' 2023-10-30T12:41:23.599 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/13' 2023-10-30T12:41:23.599 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/17' 2023-10-30T12:41:23.599 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/129' 2023-10-30T12:41:23.600 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1' 2023-10-30T12:41:23.600 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1' 2023-10-30T12:41:23.601 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-10-30T12:41:23.602 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:41:23.602 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:41:26.812 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-authwb.sh... 2023-10-30T12:41:26.812 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-authwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-authwb.sh 2023-10-30T12:41:31.396 INFO:tasks.workunit.client.0.smithi012.stderr:+ set -e 2023-10-30T12:41:31.396 INFO:tasks.workunit.client.0.smithi012.stderr:+ touch foo 2023-10-30T12:41:31.398 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod +x foo 2023-10-30T12:41:31.399 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir .snap/s 2023-10-30T12:41:31.414 INFO:tasks.workunit.client.0.smithi012.stderr:+ find .snap/s/foo -executable 2023-10-30T12:41:31.415 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep foo 2023-10-30T12:41:31.418 INFO:tasks.workunit.client.0.smithi012.stdout:.snap/s/foo 2023-10-30T12:41:31.419 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir .snap/s 2023-10-30T12:41:31.429 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm foo 2023-10-30T12:41:31.435 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo OK 2023-10-30T12:41:31.436 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-10-30T12:41:31.436 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:41:31.436 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:41:31.468 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-capwb.sh... 2023-10-30T12:41:31.468 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-capwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-capwb.sh 2023-10-30T12:41:31.530 INFO:tasks.workunit.client.0.smithi012.stderr:+ set -e 2023-10-30T12:41:31.530 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir foo 2023-10-30T12:41:31.533 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo x 2023-10-30T12:41:31.533 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:33.206 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir foo/.snap/ss 2023-10-30T12:41:33.219 INFO:tasks.workunit.client.0.smithi012.stderr:+ ln foo/x foo/xx 2023-10-30T12:41:33.222 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat foo/.snap/ss/x 2023-10-30T12:41:33.481 INFO:tasks.workunit.client.0.smithi012.stdout:x 2023-10-30T12:41:33.481 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir foo/.snap/ss 2023-10-30T12:41:33.492 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo a 2023-10-30T12:41:33.492 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo b 2023-10-30T12:41:33.492 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir foo/.snap/s 2023-10-30T12:41:33.506 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat foo/.snap/s/a 2023-10-30T12:41:33.518 INFO:tasks.workunit.client.0.smithi012.stderr:+ r=a 2023-10-30T12:41:33.519 INFO:tasks.workunit.client.0.smithi012.stderr:+ [ -z a ] 2023-10-30T12:41:33.519 INFO:tasks.workunit.client.0.smithi012.stderr:+ ln foo/b foo/b2 2023-10-30T12:41:33.520 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat foo/.snap/s/b 2023-10-30T12:41:33.969 INFO:tasks.workunit.client.0.smithi012.stdout:b 2023-10-30T12:41:33.969 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo this used to hang: 2023-10-30T12:41:33.969 INFO:tasks.workunit.client.0.smithi012.stdout:this used to hang: 2023-10-30T12:41:33.970 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo more 2023-10-30T12:41:33.970 INFO:tasks.workunit.client.0.smithi012.stdout:oh, it didn't hang! good job. 2023-10-30T12:41:33.971 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo oh, it didn't hang! good job. 2023-10-30T12:41:33.971 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat foo/b 2023-10-30T12:41:33.971 INFO:tasks.workunit.client.0.smithi012.stdout:b 2023-10-30T12:41:33.971 INFO:tasks.workunit.client.0.smithi012.stdout:more 2023-10-30T12:41:33.972 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir foo/.snap/s 2023-10-30T12:41:34.503 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:41:34.507 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:41:34.521 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm -r foo 2023-10-30T12:41:34.766 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo OK 2023-10-30T12:41:34.768 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-10-30T12:41:34.768 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:41:34.768 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:41:34.800 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-dir-rename.sh... 2023-10-30T12:41:34.800 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-dir-rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-dir-rename.sh 2023-10-30T12:41:36.395 INFO:tasks.workunit.client.0.smithi012.stderr:+ set -e 2023-10-30T12:41:36.396 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:36.397 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir .snap/bar 2023-10-30T12:41:36.411 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/foo 2023-10-30T12:41:36.420 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/foo 2023-10-30T12:41:36.429 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:36.431 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/bar/a 2023-10-30T12:41:41.387 INFO:tasks.workunit.client.0.smithi012.stdout: File: .snap/bar/a 2023-10-30T12:41:41.388 INFO:tasks.workunit.client.0.smithi012.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-10-30T12:41:41.388 INFO:tasks.workunit.client.0.smithi012.stdout:Device: fdh/253d Inode: 1099511633057 Links: 2 2023-10-30T12:41:41.388 INFO:tasks.workunit.client.0.smithi012.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1271/ ubuntu) 2023-10-30T12:41:41.388 INFO:tasks.workunit.client.0.smithi012.stdout:Access: 2023-10-30 12:41:36.389954338 +0000 2023-10-30T12:41:41.388 INFO:tasks.workunit.client.0.smithi012.stdout:Modify: 2023-10-30 12:41:36.389954338 +0000 2023-10-30T12:41:41.389 INFO:tasks.workunit.client.0.smithi012.stdout:Change: 2023-10-30 12:41:36.389954338 +0000 2023-10-30T12:41:41.389 INFO:tasks.workunit.client.0.smithi012.stdout: Birth: 2023-10-30 12:41:36.389954338 +0000 2023-10-30T12:41:41.389 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir .snap/bar 2023-10-30T12:41:41.397 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo OK 2023-10-30T12:41:41.398 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-10-30T12:41:41.399 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:41:41.399 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:41:41.427 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-double-null.sh... 2023-10-30T12:41:41.428 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-double-null.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-double-null.sh 2023-10-30T12:41:41.486 INFO:tasks.workunit.client.0.smithi012.stderr:+ set -e 2023-10-30T12:41:41.487 INFO:tasks.workunit.client.0.smithi012.stderr:+ seq 1 20 2023-10-30T12:41:41.488 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:41.743 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:41.744 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:41.759 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:41.768 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:41.769 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:41.996 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:42.013 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:42.013 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:42.013 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:42.013 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:42.014 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:42.014 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:42.014 INFO:tasks.workunit.client.0.smithi012.stdout: 48655 ? 00:00:00 ps 2023-10-30T12:41:42.015 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:42.030 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:42.032 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:42.040 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:42.042 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:42.043 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:42.057 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:42.065 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:42.067 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:42.137 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:42.153 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:42.153 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:42.153 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:42.153 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:42.153 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:42.154 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:42.154 INFO:tasks.workunit.client.0.smithi012.stdout: 48665 ? 00:00:00 ps 2023-10-30T12:41:42.154 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:42.168 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:42.171 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:42.178 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:42.180 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:42.181 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:42.442 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:42.451 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:42.452 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:42.588 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:42.603 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:42.603 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:42.603 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:42.604 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:42.604 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:42.604 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:42.604 INFO:tasks.workunit.client.0.smithi012.stdout: 48675 ? 00:00:00 ps 2023-10-30T12:41:42.605 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:42.621 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:42.623 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:42.630 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:42.633 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:42.634 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:42.647 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:42.655 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:42.656 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:42.713 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:42.728 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:42.728 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:42.728 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:42.729 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:42.729 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:42.729 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:42.729 INFO:tasks.workunit.client.0.smithi012.stdout: 48685 ? 00:00:00 ps 2023-10-30T12:41:42.730 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:42.744 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:42.746 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:42.754 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:42.757 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:42.758 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:42.770 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:42.780 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:42.781 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:42.855 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:42.870 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:42.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:42.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:42.870 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:42.870 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:42.871 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:42.871 INFO:tasks.workunit.client.0.smithi012.stdout: 48695 ? 00:00:00 ps 2023-10-30T12:41:42.871 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:42.885 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:42.887 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:42.895 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:42.897 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:42.898 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:42.911 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:42.920 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:42.921 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:42.996 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:43.011 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:43.012 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:43.012 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:43.012 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:43.012 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:43.012 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:43.013 INFO:tasks.workunit.client.0.smithi012.stdout: 48705 ? 00:00:00 ps 2023-10-30T12:41:43.013 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:43.028 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:43.030 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:43.037 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:43.039 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:43.041 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:43.054 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:43.063 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:43.064 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:43.114 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:43.128 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:43.129 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:43.129 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:43.129 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:43.129 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:43.129 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:43.130 INFO:tasks.workunit.client.0.smithi012.stdout: 48715 ? 00:00:00 ps 2023-10-30T12:41:43.130 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:43.145 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:43.147 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:43.154 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:43.157 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:43.158 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:43.170 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:43.179 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:43.180 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:43.256 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:43.271 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:43.271 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:43.271 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:43.271 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:43.272 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:43.272 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:43.272 INFO:tasks.workunit.client.0.smithi012.stdout: 48725 ? 00:00:00 ps 2023-10-30T12:41:43.273 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:43.287 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:43.290 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:43.298 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:43.301 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:43.302 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:43.314 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:43.323 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:43.324 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:43.397 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:43.412 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:43.412 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:43.413 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:43.413 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:43.413 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:43.413 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:43.413 INFO:tasks.workunit.client.0.smithi012.stdout: 48735 ? 00:00:00 ps 2023-10-30T12:41:43.414 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:43.430 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:43.432 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:43.439 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:43.442 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:43.443 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:43.455 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:43.465 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:43.466 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:43.523 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:43.537 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:43.538 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:43.538 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:43.538 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:43.538 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:43.539 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:43.539 INFO:tasks.workunit.client.0.smithi012.stdout: 48745 ? 00:00:00 ps 2023-10-30T12:41:43.539 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:43.554 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:43.556 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:43.562 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:43.565 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:43.566 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:43.580 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:43.588 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:43.589 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:43.656 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:43.671 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:43.671 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:43.672 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:43.672 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:43.672 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:43.672 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:43.673 INFO:tasks.workunit.client.0.smithi012.stdout: 48755 ? 00:00:00 ps 2023-10-30T12:41:43.673 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:43.688 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:43.690 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:43.697 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:43.700 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:43.701 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:43.714 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:43.975 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:43.976 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:44.099 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:44.114 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:44.114 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:44.115 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:44.115 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:44.115 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:44.115 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:44.115 INFO:tasks.workunit.client.0.smithi012.stdout: 48765 ? 00:00:00 ps 2023-10-30T12:41:44.116 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:44.130 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:44.132 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:44.140 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:44.142 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:44.143 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:44.155 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:44.165 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:44.166 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:44.223 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:44.238 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:44.239 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:44.239 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:44.239 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:44.239 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:44.239 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:44.240 INFO:tasks.workunit.client.0.smithi012.stdout: 48775 ? 00:00:00 ps 2023-10-30T12:41:44.240 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:44.255 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:44.257 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:44.263 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:44.266 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:44.267 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:44.279 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:44.288 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:44.289 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:44.366 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:44.381 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:44.381 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:44.381 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:44.382 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:44.382 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:44.382 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:44.382 INFO:tasks.workunit.client.0.smithi012.stdout: 48785 ? 00:00:00 ps 2023-10-30T12:41:44.383 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:44.397 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:44.399 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:44.406 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:44.408 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:44.409 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:44.422 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:44.430 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:44.431 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:44.624 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:44.640 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:44.640 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:44.640 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:44.640 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:44.640 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:44.641 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:44.641 INFO:tasks.workunit.client.0.smithi012.stdout: 48795 ? 00:00:00 ps 2023-10-30T12:41:44.641 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:44.655 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:44.658 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:44.664 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:44.667 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:44.669 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:44.681 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:44.942 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:44.943 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:45.116 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:45.133 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:45.133 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:45.133 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:45.134 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:45.134 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:45.134 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:45.134 INFO:tasks.workunit.client.0.smithi012.stdout: 48805 ? 00:00:00 ps 2023-10-30T12:41:45.135 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:45.150 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:45.152 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:45.159 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:45.162 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:45.163 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:45.524 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:45.534 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:45.535 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:45.616 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:45.631 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:45.632 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:45.632 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:45.632 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:45.632 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:45.632 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:45.633 INFO:tasks.workunit.client.0.smithi012.stdout: 48815 ? 00:00:00 ps 2023-10-30T12:41:45.633 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:45.647 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:45.650 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:45.657 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:45.660 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:45.661 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:45.674 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:45.683 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:45.685 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:45.863 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:45.878 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:45.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:45.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:45.879 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:45.879 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:45.879 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:45.879 INFO:tasks.workunit.client.0.smithi012.stdout: 48825 ? 00:00:00 ps 2023-10-30T12:41:45.887 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:45.903 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:45.905 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:46.222 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:46.227 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:46.228 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:46.329 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:46.357 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:46.361 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:46.660 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:46.689 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:46.690 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:46.690 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:46.690 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:46.690 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:46.691 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:46.691 INFO:tasks.workunit.client.0.smithi012.stdout: 48835 ? 00:00:00 ps 2023-10-30T12:41:46.691 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:47.210 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:47.212 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:47.221 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-10-30T12:41:47.225 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-10-30T12:41:47.227 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-10-30T12:41:47.269 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-10-30T12:41:47.277 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-10-30T12:41:47.278 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-10-30T12:41:47.576 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-10-30T12:41:47.591 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-10-30T12:41:47.591 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 systemd 2023-10-30T12:41:47.592 INFO:tasks.workunit.client.0.smithi012.stdout: 1197 ? 00:00:00 (sd-pam) 2023-10-30T12:41:47.592 INFO:tasks.workunit.client.0.smithi012.stdout: 27862 ? 00:00:01 sshd 2023-10-30T12:41:47.592 INFO:tasks.workunit.client.0.smithi012.stdout: 48644 ? 00:00:00 timeout 2023-10-30T12:41:47.592 INFO:tasks.workunit.client.0.smithi012.stdout: 48647 ? 00:00:00 snaptest-double 2023-10-30T12:41:47.593 INFO:tasks.workunit.client.0.smithi012.stdout: 48845 ? 00:00:00 ps 2023-10-30T12:41:47.593 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:41:47.609 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-10-30T12:41:47.612 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-10-30T12:41:47.618 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo OK 2023-10-30T12:41:47.619 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-10-30T12:41:47.620 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:41:47.620 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:41:47.690 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-estale.sh... 2023-10-30T12:41:47.691 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-estale.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-estale.sh 2023-10-30T12:41:51.399 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir .snap/foo 2023-10-30T12:41:51.410 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo We want ENOENT, not ESTALE, here. 2023-10-30T12:41:51.410 INFO:tasks.workunit.client.0.smithi012.stdout:We want ENOENT, not ESTALE, here. 2023-10-30T12:41:51.411 INFO:tasks.workunit.client.0.smithi012.stderr:+ seq 1 100 2023-10-30T12:41:51.412 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/1 2023-10-30T12:41:51.412 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.415 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/1': No such file or directory 2023-10-30T12:41:51.416 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/2 2023-10-30T12:41:51.416 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.417 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/2': No such file or directory 2023-10-30T12:41:51.418 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/3 2023-10-30T12:41:51.418 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.420 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/3': No such file or directory 2023-10-30T12:41:51.421 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/4 2023-10-30T12:41:51.421 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.423 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/4': No such file or directory 2023-10-30T12:41:51.423 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/5 2023-10-30T12:41:51.424 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.425 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/5': No such file or directory 2023-10-30T12:41:51.426 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/6 2023-10-30T12:41:51.426 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.428 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/6': No such file or directory 2023-10-30T12:41:51.429 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/7 2023-10-30T12:41:51.429 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.430 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/7': No such file or directory 2023-10-30T12:41:51.431 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/8 2023-10-30T12:41:51.431 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.433 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/8': No such file or directory 2023-10-30T12:41:51.434 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/9 2023-10-30T12:41:51.434 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.435 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/9': No such file or directory 2023-10-30T12:41:51.436 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/10 2023-10-30T12:41:51.436 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.438 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/10': No such file or directory 2023-10-30T12:41:51.439 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/11 2023-10-30T12:41:51.439 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.440 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/11': No such file or directory 2023-10-30T12:41:51.441 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/12 2023-10-30T12:41:51.441 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.443 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/12': No such file or directory 2023-10-30T12:41:51.444 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/13 2023-10-30T12:41:51.444 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.445 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/13': No such file or directory 2023-10-30T12:41:51.446 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/14 2023-10-30T12:41:51.447 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.448 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/14': No such file or directory 2023-10-30T12:41:51.449 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/15 2023-10-30T12:41:51.449 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.451 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/15': No such file or directory 2023-10-30T12:41:51.451 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/16 2023-10-30T12:41:51.452 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.453 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/16': No such file or directory 2023-10-30T12:41:51.454 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/17 2023-10-30T12:41:51.454 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.456 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/17': No such file or directory 2023-10-30T12:41:51.456 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/18 2023-10-30T12:41:51.457 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.458 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/18': No such file or directory 2023-10-30T12:41:51.459 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/19 2023-10-30T12:41:51.460 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.461 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/19': No such file or directory 2023-10-30T12:41:51.462 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/20 2023-10-30T12:41:51.462 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.463 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/20': No such file or directory 2023-10-30T12:41:51.464 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/21 2023-10-30T12:41:51.465 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.466 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/21': No such file or directory 2023-10-30T12:41:51.467 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/22 2023-10-30T12:41:51.467 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.468 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/22': No such file or directory 2023-10-30T12:41:51.469 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/23 2023-10-30T12:41:51.470 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.471 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/23': No such file or directory 2023-10-30T12:41:51.472 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/24 2023-10-30T12:41:51.472 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.473 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/24': No such file or directory 2023-10-30T12:41:51.474 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/25 2023-10-30T12:41:51.474 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.476 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/25': No such file or directory 2023-10-30T12:41:51.477 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/26 2023-10-30T12:41:51.477 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.478 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/26': No such file or directory 2023-10-30T12:41:51.479 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/27 2023-10-30T12:41:51.479 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.481 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/27': No such file or directory 2023-10-30T12:41:51.482 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/28 2023-10-30T12:41:51.482 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.483 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/28': No such file or directory 2023-10-30T12:41:51.484 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/29 2023-10-30T12:41:51.484 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.486 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/29': No such file or directory 2023-10-30T12:41:51.486 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/30 2023-10-30T12:41:51.487 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.488 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/30': No such file or directory 2023-10-30T12:41:51.489 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/31 2023-10-30T12:41:51.489 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.491 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/31': No such file or directory 2023-10-30T12:41:51.491 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/32 2023-10-30T12:41:51.492 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.493 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/32': No such file or directory 2023-10-30T12:41:51.494 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/33 2023-10-30T12:41:51.494 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.495 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/33': No such file or directory 2023-10-30T12:41:51.496 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/34 2023-10-30T12:41:51.496 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.498 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/34': No such file or directory 2023-10-30T12:41:51.499 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/35 2023-10-30T12:41:51.499 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.500 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/35': No such file or directory 2023-10-30T12:41:51.501 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/36 2023-10-30T12:41:51.501 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.503 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/36': No such file or directory 2023-10-30T12:41:51.503 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/37 2023-10-30T12:41:51.504 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.505 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/37': No such file or directory 2023-10-30T12:41:51.506 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/38 2023-10-30T12:41:51.506 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.507 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/38': No such file or directory 2023-10-30T12:41:51.508 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/39 2023-10-30T12:41:51.508 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.510 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/39': No such file or directory 2023-10-30T12:41:51.511 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/40 2023-10-30T12:41:51.511 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.512 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/40': No such file or directory 2023-10-30T12:41:51.513 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/41 2023-10-30T12:41:51.514 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.515 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/41': No such file or directory 2023-10-30T12:41:51.517 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/42 2023-10-30T12:41:51.517 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.518 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/42': No such file or directory 2023-10-30T12:41:51.519 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/43 2023-10-30T12:41:51.519 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.520 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/43': No such file or directory 2023-10-30T12:41:51.520 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/44 2023-10-30T12:41:51.521 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.522 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/44': No such file or directory 2023-10-30T12:41:51.523 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/45 2023-10-30T12:41:51.523 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.524 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/45': No such file or directory 2023-10-30T12:41:51.525 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/46 2023-10-30T12:41:51.526 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.527 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/46': No such file or directory 2023-10-30T12:41:51.528 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/47 2023-10-30T12:41:51.528 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.530 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/47': No such file or directory 2023-10-30T12:41:51.530 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/48 2023-10-30T12:41:51.531 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.532 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/48': No such file or directory 2023-10-30T12:41:51.533 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/49 2023-10-30T12:41:51.533 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.534 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/49': No such file or directory 2023-10-30T12:41:51.535 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/50 2023-10-30T12:41:51.535 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.537 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/50': No such file or directory 2023-10-30T12:41:51.537 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/51 2023-10-30T12:41:51.538 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.539 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/51': No such file or directory 2023-10-30T12:41:51.540 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/52 2023-10-30T12:41:51.540 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.542 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/52': No such file or directory 2023-10-30T12:41:51.542 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/53 2023-10-30T12:41:51.543 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.544 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/53': No such file or directory 2023-10-30T12:41:51.545 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/54 2023-10-30T12:41:51.545 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.546 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/54': No such file or directory 2023-10-30T12:41:51.547 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/55 2023-10-30T12:41:51.547 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.549 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/55': No such file or directory 2023-10-30T12:41:51.550 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/56 2023-10-30T12:41:51.550 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.551 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/56': No such file or directory 2023-10-30T12:41:51.552 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/57 2023-10-30T12:41:51.552 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.553 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/57': No such file or directory 2023-10-30T12:41:51.554 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/58 2023-10-30T12:41:51.555 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.556 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/58': No such file or directory 2023-10-30T12:41:51.557 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/59 2023-10-30T12:41:51.557 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.558 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/59': No such file or directory 2023-10-30T12:41:51.559 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/60 2023-10-30T12:41:51.559 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.560 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/60': No such file or directory 2023-10-30T12:41:51.561 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/61 2023-10-30T12:41:51.562 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.563 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/61': No such file or directory 2023-10-30T12:41:51.564 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/62 2023-10-30T12:41:51.564 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.565 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/62': No such file or directory 2023-10-30T12:41:51.566 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/63 2023-10-30T12:41:51.566 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.567 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/63': No such file or directory 2023-10-30T12:41:51.568 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/64 2023-10-30T12:41:51.569 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.570 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/64': No such file or directory 2023-10-30T12:41:51.571 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/65 2023-10-30T12:41:51.571 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.572 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/65': No such file or directory 2023-10-30T12:41:51.573 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/66 2023-10-30T12:41:51.573 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.574 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/66': No such file or directory 2023-10-30T12:41:51.575 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/67 2023-10-30T12:41:51.576 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.577 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/67': No such file or directory 2023-10-30T12:41:51.578 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/68 2023-10-30T12:41:51.578 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.579 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/68': No such file or directory 2023-10-30T12:41:51.580 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/69 2023-10-30T12:41:51.580 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.582 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/69': No such file or directory 2023-10-30T12:41:51.582 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/70 2023-10-30T12:41:51.583 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.584 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/70': No such file or directory 2023-10-30T12:41:51.585 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/71 2023-10-30T12:41:51.585 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.586 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/71': No such file or directory 2023-10-30T12:41:51.587 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/72 2023-10-30T12:41:51.587 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.588 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/72': No such file or directory 2023-10-30T12:41:51.589 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/73 2023-10-30T12:41:51.590 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.591 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/73': No such file or directory 2023-10-30T12:41:51.591 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/74 2023-10-30T12:41:51.592 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.593 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/74': No such file or directory 2023-10-30T12:41:51.594 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/75 2023-10-30T12:41:51.594 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.595 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/75': No such file or directory 2023-10-30T12:41:51.596 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/76 2023-10-30T12:41:51.596 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.598 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/76': No such file or directory 2023-10-30T12:41:51.599 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/77 2023-10-30T12:41:51.599 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.600 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/77': No such file or directory 2023-10-30T12:41:51.601 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/78 2023-10-30T12:41:51.601 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.603 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/78': No such file or directory 2023-10-30T12:41:51.604 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/79 2023-10-30T12:41:51.604 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.605 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/79': No such file or directory 2023-10-30T12:41:51.606 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/80 2023-10-30T12:41:51.606 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.607 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/80': No such file or directory 2023-10-30T12:41:51.608 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/81 2023-10-30T12:41:51.608 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.610 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/81': No such file or directory 2023-10-30T12:41:51.610 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/82 2023-10-30T12:41:51.611 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.612 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/82': No such file or directory 2023-10-30T12:41:51.613 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/83 2023-10-30T12:41:51.613 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.615 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/83': No such file or directory 2023-10-30T12:41:51.615 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/84 2023-10-30T12:41:51.616 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.617 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/84': No such file or directory 2023-10-30T12:41:51.618 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/85 2023-10-30T12:41:51.618 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.619 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/85': No such file or directory 2023-10-30T12:41:51.620 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/86 2023-10-30T12:41:51.620 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.622 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/86': No such file or directory 2023-10-30T12:41:51.623 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/87 2023-10-30T12:41:51.623 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.624 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/87': No such file or directory 2023-10-30T12:41:51.625 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/88 2023-10-30T12:41:51.625 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.626 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/88': No such file or directory 2023-10-30T12:41:51.627 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/89 2023-10-30T12:41:51.628 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.629 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/89': No such file or directory 2023-10-30T12:41:51.630 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/90 2023-10-30T12:41:51.630 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.631 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/90': No such file or directory 2023-10-30T12:41:51.632 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/91 2023-10-30T12:41:51.633 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.634 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/91': No such file or directory 2023-10-30T12:41:51.635 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/92 2023-10-30T12:41:51.635 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.636 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/92': No such file or directory 2023-10-30T12:41:51.637 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/93 2023-10-30T12:41:51.637 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.891 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/93': No such file or directory 2023-10-30T12:41:51.891 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/94 2023-10-30T12:41:51.892 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.893 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/94': No such file or directory 2023-10-30T12:41:51.894 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/95 2023-10-30T12:41:51.894 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.895 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/95': No such file or directory 2023-10-30T12:41:51.896 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/96 2023-10-30T12:41:51.896 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.899 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/96': No such file or directory 2023-10-30T12:41:51.900 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/97 2023-10-30T12:41:51.900 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.902 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/97': No such file or directory 2023-10-30T12:41:51.902 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/98 2023-10-30T12:41:51.903 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.904 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/98': No such file or directory 2023-10-30T12:41:51.905 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/99 2023-10-30T12:41:51.905 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.906 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/99': No such file or directory 2023-10-30T12:41:51.907 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/100 2023-10-30T12:41:51.907 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-10-30T12:41:51.909 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/100': No such file or directory 2023-10-30T12:41:51.909 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir .snap/foo 2023-10-30T12:41:51.919 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-10-30T12:41:51.920 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo OK 2023-10-30T12:41:51.920 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:41:51.921 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:41:51.946 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-git-ceph.sh... 2023-10-30T12:41:51.946 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-git-ceph.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh 2023-10-30T12:41:56.399 INFO:tasks.workunit.client.0.smithi012.stderr:+ set -e 2023-10-30T12:41:56.399 INFO:tasks.workunit.client.0.smithi012.stderr:+ sudo git config --global http.sslVerify false 2023-10-30T12:41:56.418 INFO:tasks.workunit.client.0.smithi012.stderr:+ sudo git config --global http.postBuffer 1048576000 2023-10-30T12:41:56.433 INFO:tasks.workunit.client.0.smithi012.stderr:+ retried=false 2023-10-30T12:41:56.434 INFO:tasks.workunit.client.0.smithi012.stderr:+ trap -- retry EXIT 2023-10-30T12:41:56.434 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm -rf ceph 2023-10-30T12:41:56.434 INFO:tasks.workunit.client.0.smithi012.stderr:+ timeout 1800 git clone https://git.ceph.com/ceph.git 2023-10-30T12:41:56.439 INFO:tasks.workunit.client.0.smithi012.stderr:Cloning into 'ceph'... 2023-10-30T12:42:04.564 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:42:04.569 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:42:34.624 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:42:34.629 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:42:56.974 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:42:57.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa475ac9640 1 Processor -- start 2023-10-30T12:42:57.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa475ac9640 1 -- start start 2023-10-30T12:42:57.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa475ac9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4701538a0 0x7fa47015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:57.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa475ac9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa4700ca0e0 0x7fa47015a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:57.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa475ac9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa4700cf160 0x7fa4700cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:57.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa475ac9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4700cb800 con 0x7fa4700ca0e0 2023-10-30T12:42:57.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa475ac9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47015dc30 con 0x7fa4700cf160 2023-10-30T12:42:57.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa475ac9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4700cb330 con 0x7fa4701538a0 2023-10-30T12:42:57.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa467fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa4700ca0e0 0x7fa47015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:57.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa467fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa4700ca0e0 0x7fa47015a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49058/0 (socket says 172.21.15.12:49058) 2023-10-30T12:42:57.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa467fff640 1 -- 172.21.15.12:0/3481357766 learned_addr learned my addr 172.21.15.12:0/3481357766 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:42:57.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa46ffff640 1 --2- 172.21.15.12:0/3481357766 >> v2:172.21.15.154:3300/0 conn(0x7fa4700cf160 0x7fa4700cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:57.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa46f7fe640 1 --2- 172.21.15.12:0/3481357766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4701538a0 0x7fa47015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:57.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa467fff640 1 -- 172.21.15.12:0/3481357766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4701538a0 msgr2=0x7fa47015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:42:57.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa467fff640 1 --2- 172.21.15.12:0/3481357766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4701538a0 0x7fa47015d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa467fff640 1 -- 172.21.15.12:0/3481357766 >> v2:172.21.15.154:3300/0 conn(0x7fa4700cf160 msgr2=0x7fa4700cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:42:57.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa467fff640 1 --2- 172.21.15.12:0/3481357766 >> v2:172.21.15.154:3300/0 conn(0x7fa4700cf160 0x7fa4700cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.129+0000 7fa467fff640 1 -- 172.21.15.12:0/3481357766 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4700cad70 con 0x7fa4700ca0e0 2023-10-30T12:42:57.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.133+0000 7fa467fff640 1 --2- 172.21.15.12:0/3481357766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4700ca0e0 0x7fa47015a090 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa47006cb00 tx=0x7fa45801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529ca4ef89af9711 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:42:57.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.133+0000 7fa46effd640 1 -- 172.21.15.12:0/3481357766 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4580a89b0 con 0x7fa4700ca0e0 2023-10-30T12:42:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.133+0000 7fa46effd640 1 -- 172.21.15.12:0/3481357766 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4580a8b10 con 0x7fa4700ca0e0 2023-10-30T12:42:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.133+0000 7fa46effd640 1 -- 172.21.15.12:0/3481357766 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4580a8ea0 con 0x7fa4700ca0e0 2023-10-30T12:42:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.133+0000 7fa475ac9640 1 -- 172.21.15.12:0/3481357766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4700ca0e0 msgr2=0x7fa47015a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.133+0000 7fa475ac9640 1 --2- 172.21.15.12:0/3481357766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4700ca0e0 0x7fa47015a090 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa47006cb00 tx=0x7fa45801ecf0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.133+0000 7fa475ac9640 1 -- 172.21.15.12:0/3481357766 shutdown_connections 2023-10-30T12:42:57.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.133+0000 7fa475ac9640 1 --2- 172.21.15.12:0/3481357766 >> v2:172.21.15.154:3300/0 conn(0x7fa4700cf160 0x7fa4700cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.133+0000 7fa475ac9640 1 --2- 172.21.15.12:0/3481357766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4700ca0e0 0x7fa47015a090 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.133+0000 7fa475ac9640 1 --2- 172.21.15.12:0/3481357766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4701538a0 0x7fa47015d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.133+0000 7fa475ac9640 1 -- 172.21.15.12:0/3481357766 >> 172.21.15.12:0/3481357766 conn(0x7fa47006bfe0 msgr2=0x7fa47006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:42:57.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.133+0000 7fa475ac9640 1 -- 172.21.15.12:0/3481357766 shutdown_connections 2023-10-30T12:42:57.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.133+0000 7fa475ac9640 1 -- 172.21.15.12:0/3481357766 wait complete. 2023-10-30T12:42:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa475ac9640 1 Processor -- start 2023-10-30T12:42:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa475ac9640 1 -- start start 2023-10-30T12:42:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa475ac9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4700ca0e0 0x7fa47015d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa475ac9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4700cf160 0x7fa47014a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa475ac9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4701538a0 0x7fa4701f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa475ac9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4701f2fd0 con 0x7fa4700cf160 2023-10-30T12:42:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa475ac9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47015ad70 con 0x7fa4700ca0e0 2023-10-30T12:42:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa475ac9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47014bab0 con 0x7fa4701538a0 2023-10-30T12:42:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa467fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4700cf160 0x7fa47014a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa467fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4700cf160 0x7fa47014a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49068/0 (socket says 172.21.15.12:49068) 2023-10-30T12:42:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa467fff640 1 -- 172.21.15.12:0/258838922 learned_addr learned my addr 172.21.15.12:0/258838922 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:42:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa46f7fe640 1 --2- 172.21.15.12:0/258838922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4700ca0e0 0x7fa47015d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:57.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa46ffff640 1 --2- 172.21.15.12:0/258838922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4701538a0 0x7fa4701f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:57.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa467fff640 1 -- 172.21.15.12:0/258838922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4701538a0 msgr2=0x7fa4701f33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:57.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa467fff640 1 --2- 172.21.15.12:0/258838922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4701538a0 0x7fa4701f33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa467fff640 1 -- 172.21.15.12:0/258838922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4700ca0e0 msgr2=0x7fa47015d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:57.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa467fff640 1 --2- 172.21.15.12:0/258838922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4700ca0e0 0x7fa47015d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa467fff640 1 -- 172.21.15.12:0/258838922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4580094d0 con 0x7fa4700cf160 2023-10-30T12:42:57.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa46f7fe640 1 --2- 172.21.15.12:0/258838922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4700ca0e0 0x7fa47015d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:42:57.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa467fff640 1 --2- 172.21.15.12:0/258838922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4700cf160 0x7fa47014a6f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa4580b3040 tx=0x7fa45801ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:42:57.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa46cff9640 1 -- 172.21.15.12:0/258838922 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4580ba600 con 0x7fa4700cf160 2023-10-30T12:42:57.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa46cff9640 1 -- 172.21.15.12:0/258838922 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4580ba760 con 0x7fa4700cf160 2023-10-30T12:42:57.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa475ac9640 1 -- 172.21.15.12:0/258838922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4700ce130 con 0x7fa4700cf160 2023-10-30T12:42:57.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa46cff9640 1 -- 172.21.15.12:0/258838922 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4580bab10 con 0x7fa4700cf160 2023-10-30T12:42:57.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.137+0000 7fa475ac9640 1 -- 172.21.15.12:0/258838922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4700ce380 con 0x7fa4700cf160 2023-10-30T12:42:57.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.141+0000 7fa475ac9640 1 -- 172.21.15.12:0/258838922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4700b2a50 con 0x7fa4700cf160 2023-10-30T12:42:57.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.141+0000 7fa46cff9640 1 -- 172.21.15.12:0/258838922 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa4580bac70 con 0x7fa4700cf160 2023-10-30T12:42:57.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.141+0000 7fa46cff9640 1 --2- 172.21.15.12:0/258838922 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4500664d0 0x7fa450068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:57.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.141+0000 7fa46cff9640 1 -- 172.21.15.12:0/258838922 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6025+0+0 (secure 0 0 0) 0x7fa458122710 con 0x7fa4700cf160 2023-10-30T12:42:57.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.141+0000 7fa46f7fe640 1 --2- 172.21.15.12:0/258838922 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4500664d0 0x7fa450068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:57.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.141+0000 7fa46f7fe640 1 --2- 172.21.15.12:0/258838922 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4500664d0 0x7fa450068990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa46000b9f0 tx=0x7fa460008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:42:57.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.145+0000 7fa46cff9640 1 -- 172.21.15.12:0/258838922 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa4580ef920 con 0x7fa4700cf160 2023-10-30T12:42:57.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.341+0000 7fa475ac9640 1 -- 172.21.15.12:0/258838922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4701f3110 con 0x7fa4700cf160 2023-10-30T12:42:57.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.341+0000 7fa46cff9640 1 -- 172.21.15.12:0/258838922 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+6528 (secure 0 0 0) 0x7fa4701f3110 con 0x7fa4700cf160 2023-10-30T12:42:57.348 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:42:57.348 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:42:12.327346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:42:57.350 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 34 2023-10-30T12:42:57.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.341+0000 7fa475ac9640 1 -- 172.21.15.12:0/258838922 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4500664d0 msgr2=0x7fa450068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.345+0000 7fa475ac9640 1 --2- 172.21.15.12:0/258838922 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4500664d0 0x7fa450068990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa46000b9f0 tx=0x7fa460008040 comp rx=0 tx=0).stop 2023-10-30T12:42:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.345+0000 7fa475ac9640 1 -- 172.21.15.12:0/258838922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4700cf160 msgr2=0x7fa47014a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.345+0000 7fa475ac9640 1 --2- 172.21.15.12:0/258838922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4700cf160 0x7fa47014a6f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa4580b3040 tx=0x7fa45801ba90 comp rx=0 tx=0).stop 2023-10-30T12:42:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.345+0000 7fa475ac9640 1 -- 172.21.15.12:0/258838922 shutdown_connections 2023-10-30T12:42:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.345+0000 7fa475ac9640 1 --2- 172.21.15.12:0/258838922 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4500664d0 0x7fa450068990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.345+0000 7fa475ac9640 1 --2- 172.21.15.12:0/258838922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4701538a0 0x7fa4701f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.345+0000 7fa475ac9640 1 --2- 172.21.15.12:0/258838922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4700cf160 0x7fa47014a6f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.345+0000 7fa475ac9640 1 --2- 172.21.15.12:0/258838922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4700ca0e0 0x7fa47015d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.345+0000 7fa475ac9640 1 -- 172.21.15.12:0/258838922 >> 172.21.15.12:0/258838922 conn(0x7fa47006bfe0 msgr2=0x7fa4701483a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:42:57.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.345+0000 7fa475ac9640 1 -- 172.21.15.12:0/258838922 shutdown_connections 2023-10-30T12:42:57.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.345+0000 7fa475ac9640 1 -- 172.21.15.12:0/258838922 wait complete. 2023-10-30T12:42:57.372 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=1) 2023-10-30T12:42:57.372 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-30T12:42:57.373 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:42:57.395 INFO:tasks.ceph.mds.d:Stopped 2023-10-30T12:42:57.396 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:42:57.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.533+0000 7fb6bbc46640 1 Processor -- start 2023-10-30T12:42:57.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.533+0000 7fb6bbc46640 1 -- start start 2023-10-30T12:42:57.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.537+0000 7fb6bbc46640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb6b4153d70 0x7fb6b406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:57.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.537+0000 7fb6bbc46640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b40c8490 0x7fb6b40c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:57.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.537+0000 7fb6bbc46640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b406afa0 con 0x7fb6b406e470 2023-10-30T12:42:57.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.537+0000 7fb6bbc46640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b406d780 con 0x7fb6b4153d70 2023-10-30T12:42:57.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.537+0000 7fb6bbc46640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b4154150 con 0x7fb6b40c8490 2023-10-30T12:42:57.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.537+0000 7fb6b99e2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb6b4153d70 0x7fb6b406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.537+0000 7fb6b91e1640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb6b406e470 0x7fb6b40cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56458/0 (socket says 172.21.15.12:56458) 2023-10-30T12:42:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.537+0000 7fb6b99e2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb6b4153d70 0x7fb6b406ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49072/0 (socket says 172.21.15.12:49072) 2023-10-30T12:42:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.537+0000 7fb6b91e1640 1 -- 172.21.15.12:0/3624271130 learned_addr learned my addr 172.21.15.12:0/3624271130 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:42:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.537+0000 7fb6ba1e3640 1 --2- 172.21.15.12:0/3624271130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b40c8490 0x7fb6b40c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:57.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.537+0000 7fb6b99e2640 1 -- 172.21.15.12:0/3624271130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b40c8490 msgr2=0x7fb6b40c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:42:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.537+0000 7fb6b99e2640 1 --2- 172.21.15.12:0/3624271130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b40c8490 0x7fb6b40c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.537+0000 7fb6b99e2640 1 -- 172.21.15.12:0/3624271130 >> v1:172.21.15.12:6789/0 conn(0x7fb6b406e470 legacy=0x7fb6b40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:57.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.537+0000 7fb6b99e2640 1 -- 172.21.15.12:0/3624271130 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b406e030 con 0x7fb6b4153d70 2023-10-30T12:42:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6b99e2640 1 --2- 172.21.15.12:0/3624271130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b4153d70 0x7fb6b406ca80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00097c0 tx=0x7fb6b001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7028bb4dc3a0c9d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:42:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6b89e0640 1 -- 172.21.15.12:0/3624271130 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b00ab650 con 0x7fb6b4153d70 2023-10-30T12:42:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6b89e0640 1 -- 172.21.15.12:0/3624271130 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b00ab7b0 con 0x7fb6b4153d70 2023-10-30T12:42:57.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6b89e0640 1 -- 172.21.15.12:0/3624271130 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b00abb40 con 0x7fb6b4153d70 2023-10-30T12:42:57.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 -- 172.21.15.12:0/3624271130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b4153d70 msgr2=0x7fb6b406ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:57.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 --2- 172.21.15.12:0/3624271130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b4153d70 0x7fb6b406ca80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00097c0 tx=0x7fb6b001c450 comp rx=0 tx=0).stop 2023-10-30T12:42:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 -- 172.21.15.12:0/3624271130 shutdown_connections 2023-10-30T12:42:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 --2- 172.21.15.12:0/3624271130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b40c8490 0x7fb6b40c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 --2- 172.21.15.12:0/3624271130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b4153d70 0x7fb6b406ca80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 -- 172.21.15.12:0/3624271130 >> 172.21.15.12:0/3624271130 conn(0x7fb6b4071ea0 msgr2=0x7fb6b4065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:42:57.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 -- 172.21.15.12:0/3624271130 shutdown_connections 2023-10-30T12:42:57.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 -- 172.21.15.12:0/3624271130 wait complete. 2023-10-30T12:42:57.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 Processor -- start 2023-10-30T12:42:57.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 -- start start 2023-10-30T12:42:57.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b406e470 0x7fb6b4143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:57.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b40c8490 0x7fb6b4142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:57.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b4153d70 0x7fb6b4154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:57.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b41446f0 con 0x7fb6b40c8490 2023-10-30T12:42:57.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b4144be0 con 0x7fb6b406e470 2023-10-30T12:42:57.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.541+0000 7fb6bbc46640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b4144900 con 0x7fb6b4153d70 2023-10-30T12:42:57.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6b91e1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b40c8490 0x7fb6b4142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:57.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6b91e1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b40c8490 0x7fb6b4142db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49086/0 (socket says 172.21.15.12:49086) 2023-10-30T12:42:57.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6b91e1640 1 -- 172.21.15.12:0/702941506 learned_addr learned my addr 172.21.15.12:0/702941506 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:42:57.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6b99e2640 1 --2- 172.21.15.12:0/702941506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b406e470 0x7fb6b4143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:57.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6ba1e3640 1 --2- 172.21.15.12:0/702941506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b4153d70 0x7fb6b4154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:57.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6b91e1640 1 -- 172.21.15.12:0/702941506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b4153d70 msgr2=0x7fb6b4154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:57.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6b91e1640 1 --2- 172.21.15.12:0/702941506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b4153d70 0x7fb6b4154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6b91e1640 1 -- 172.21.15.12:0/702941506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b406e470 msgr2=0x7fb6b4143b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:57.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6b91e1640 1 --2- 172.21.15.12:0/702941506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b406e470 0x7fb6b4143b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6b91e1640 1 -- 172.21.15.12:0/702941506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b0009420 con 0x7fb6b40c8490 2023-10-30T12:42:57.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6b99e2640 1 --2- 172.21.15.12:0/702941506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b406e470 0x7fb6b4143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:42:57.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6b91e1640 1 --2- 172.21.15.12:0/702941506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b40c8490 0x7fb6b4142db0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400c530 tx=0x7fb6a400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:42:57.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6aa7fc640 1 -- 172.21.15.12:0/702941506 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4005cf0 con 0x7fb6b40c8490 2023-10-30T12:42:57.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6aa7fc640 1 -- 172.21.15.12:0/702941506 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a4005e50 con 0x7fb6b40c8490 2023-10-30T12:42:57.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6bbc46640 1 -- 172.21.15.12:0/702941506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b4147f00 con 0x7fb6b40c8490 2023-10-30T12:42:57.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6aa7fc640 1 -- 172.21.15.12:0/702941506 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4007270 con 0x7fb6b40c8490 2023-10-30T12:42:57.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6bbc46640 1 -- 172.21.15.12:0/702941506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b4144f00 con 0x7fb6b40c8490 2023-10-30T12:42:57.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6bbc46640 1 -- 172.21.15.12:0/702941506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b404e170 con 0x7fb6b40c8490 2023-10-30T12:42:57.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6aa7fc640 1 -- 172.21.15.12:0/702941506 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb6a40073d0 con 0x7fb6b40c8490 2023-10-30T12:42:57.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.545+0000 7fb6aa7fc640 1 --2- 172.21.15.12:0/702941506 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb6980664d0 0x7fb698068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:57.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.549+0000 7fb6aa7fc640 1 -- 172.21.15.12:0/702941506 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6025+0+0 (secure 0 0 0) 0x7fb6a40834a0 con 0x7fb6b40c8490 2023-10-30T12:42:57.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.549+0000 7fb6b99e2640 1 --2- 172.21.15.12:0/702941506 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb6980664d0 0x7fb698068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:57.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.549+0000 7fb6b99e2640 1 --2- 172.21.15.12:0/702941506 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb6980664d0 0x7fb698068990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00af000 tx=0x7fb6b00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:42:57.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.549+0000 7fb6aa7fc640 1 -- 172.21.15.12:0/702941506 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb6a40507f0 con 0x7fb6b40c8490 2023-10-30T12:42:57.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.741+0000 7fb6bbc46640 1 -- 172.21.15.12:0/702941506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6b4144d70 con 0x7fb6b40c8490 2023-10-30T12:42:57.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.741+0000 7fb6aa7fc640 1 -- 172.21.15.12:0/702941506 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+6528 (secure 0 0 0) 0x7fb6a4053e10 con 0x7fb6b40c8490 2023-10-30T12:42:57.750 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:42:57.750 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:42:12.327346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:42:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 34 2023-10-30T12:42:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.745+0000 7fb6bbc46640 1 -- 172.21.15.12:0/702941506 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb6980664d0 msgr2=0x7fb698068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:57.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.745+0000 7fb6bbc46640 1 --2- 172.21.15.12:0/702941506 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb6980664d0 0x7fb698068990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00af000 tx=0x7fb6b00a7000 comp rx=0 tx=0).stop 2023-10-30T12:42:57.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.745+0000 7fb6bbc46640 1 -- 172.21.15.12:0/702941506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b40c8490 msgr2=0x7fb6b4142db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:57.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.745+0000 7fb6bbc46640 1 --2- 172.21.15.12:0/702941506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b40c8490 0x7fb6b4142db0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400c530 tx=0x7fb6a400c9f0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.745+0000 7fb6bbc46640 1 -- 172.21.15.12:0/702941506 shutdown_connections 2023-10-30T12:42:57.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.745+0000 7fb6bbc46640 1 --2- 172.21.15.12:0/702941506 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb6980664d0 0x7fb698068990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.745+0000 7fb6bbc46640 1 --2- 172.21.15.12:0/702941506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b4153d70 0x7fb6b4154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.745+0000 7fb6bbc46640 1 --2- 172.21.15.12:0/702941506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b40c8490 0x7fb6b4142db0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.745+0000 7fb6bbc46640 1 --2- 172.21.15.12:0/702941506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b406e470 0x7fb6b4143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:57.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.745+0000 7fb6bbc46640 1 -- 172.21.15.12:0/702941506 >> 172.21.15.12:0/702941506 conn(0x7fb6b4071ea0 msgr2=0x7fb6b414f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:42:57.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.745+0000 7fb6bbc46640 1 -- 172.21.15.12:0/702941506 shutdown_connections 2023-10-30T12:42:57.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:57.745+0000 7fb6bbc46640 1 -- 172.21.15.12:0/702941506 wait complete. 2023-10-30T12:42:57.770 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:42:59.773 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:42:59.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05eede4640 1 Processor -- start 2023-10-30T12:42:59.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05eede4640 1 -- start start 2023-10-30T12:42:59.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05eede4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f05e81538f0 0x7f05e8159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:59.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05eede4640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e80ca6a0 con 0x7f05e80cefa0 2023-10-30T12:42:59.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05eede4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e815d300 con 0x7f05e815cd50 2023-10-30T12:42:59.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05eede4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e815d5e0 con 0x7f05e81538f0 2023-10-30T12:42:59.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05ee5e3640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f05e80cefa0 0x7f05e81640e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56460/0 (socket says 172.21.15.12:56460) 2023-10-30T12:42:59.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05ee5e3640 1 -- 172.21.15.12:0/1443418935 learned_addr learned my addr 172.21.15.12:0/1443418935 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:42:59.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05edde2640 1 --2- 172.21.15.12:0/1443418935 >> v2:172.21.15.154:3300/0 conn(0x7f05e81538f0 0x7f05e8159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:59.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05ecde0640 1 -- 172.21.15.12:0/1443418935 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186316236 0 0) 0x7f05e80ca6a0 con 0x7f05e80cefa0 2023-10-30T12:42:59.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05ecde0640 1 -- 172.21.15.12:0/1443418935 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05d4003530 con 0x7f05e80cefa0 2023-10-30T12:42:59.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05ecde0640 1 -- 172.21.15.12:0/1443418935 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728200723 0 0) 0x7f05e815d300 con 0x7f05e815cd50 2023-10-30T12:42:59.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05ecde0640 1 -- 172.21.15.12:0/1443418935 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e80ca6a0 con 0x7f05e815cd50 2023-10-30T12:42:59.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05ecde0640 1 -- 172.21.15.12:0/1443418935 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2409869369 0 0) 0x7f05d4003530 con 0x7f05e80cefa0 2023-10-30T12:42:59.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05edde2640 1 -- 172.21.15.12:0/1443418935 >> v1:172.21.15.154:6789/0 conn(0x7f05e815cd50 legacy=0x7f05e80c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:59.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05edde2640 1 -- 172.21.15.12:0/1443418935 >> v1:172.21.15.12:6789/0 conn(0x7f05e80cefa0 legacy=0x7f05e81640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:59.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.921+0000 7f05edde2640 1 -- 172.21.15.12:0/1443418935 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e815d8c0 con 0x7f05e81538f0 2023-10-30T12:42:59.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.925+0000 7f05edde2640 1 --2- 172.21.15.12:0/1443418935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05e81538f0 0x7f05e8159fb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f05e40097c0 tx=0x7f05e401eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=665af13e9fbe68cd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:42:59.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.925+0000 7f05ecde0640 1 -- 172.21.15.12:0/1443418935 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e0001a00 con 0x7f05e81538f0 2023-10-30T12:42:59.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.925+0000 7f05ecde0640 1 -- 172.21.15.12:0/1443418935 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e40a8610 con 0x7f05e81538f0 2023-10-30T12:42:59.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.925+0000 7f05ecde0640 1 -- 172.21.15.12:0/1443418935 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40a8980 con 0x7f05e81538f0 2023-10-30T12:42:59.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.929+0000 7f05eede4640 1 -- 172.21.15.12:0/1443418935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05e81538f0 msgr2=0x7f05e8159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:59.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.929+0000 7f05eede4640 1 --2- 172.21.15.12:0/1443418935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05e81538f0 0x7f05e8159fb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f05e40097c0 tx=0x7f05e401eb90 comp rx=0 tx=0).stop 2023-10-30T12:42:59.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.929+0000 7f05eede4640 1 -- 172.21.15.12:0/1443418935 shutdown_connections 2023-10-30T12:42:59.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.929+0000 7f05eede4640 1 --2- 172.21.15.12:0/1443418935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05e81538f0 0x7f05e8159fb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:59.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.929+0000 7f05eede4640 1 -- 172.21.15.12:0/1443418935 >> 172.21.15.12:0/1443418935 conn(0x7f05e806a640 msgr2=0x7f05e8065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:42:59.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.929+0000 7f05eede4640 1 -- 172.21.15.12:0/1443418935 shutdown_connections 2023-10-30T12:42:59.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.929+0000 7f05eede4640 1 -- 172.21.15.12:0/1443418935 wait complete. 2023-10-30T12:42:59.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.929+0000 7f05eede4640 1 Processor -- start 2023-10-30T12:42:59.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.929+0000 7f05eede4640 1 -- start start 2023-10-30T12:42:59.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05eede4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05e80cefa0 0x7f05e8152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:59.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05eede4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05e81538f0 0x7f05e80b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:59.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05eede4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05e815cd50 0x7f05e80b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:59.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05eede4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e80b3050 con 0x7f05e81538f0 2023-10-30T12:42:59.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05eede4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e8152670 con 0x7f05e815cd50 2023-10-30T12:42:59.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05eede4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e8152b40 con 0x7f05e80cefa0 2023-10-30T12:42:59.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05ed5e1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05e81538f0 0x7f05e80b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:59.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05ed5e1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05e81538f0 0x7f05e80b4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49100/0 (socket says 172.21.15.12:49100) 2023-10-30T12:42:59.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05ed5e1640 1 -- 172.21.15.12:0/2001188651 learned_addr learned my addr 172.21.15.12:0/2001188651 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:42:59.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05edde2640 1 --2- 172.21.15.12:0/2001188651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05e80cefa0 0x7f05e8152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:59.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05ee5e3640 1 --2- 172.21.15.12:0/2001188651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05e815cd50 0x7f05e80b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:59.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05ed5e1640 1 -- 172.21.15.12:0/2001188651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05e80cefa0 msgr2=0x7f05e8152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:59.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05ed5e1640 1 --2- 172.21.15.12:0/2001188651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05e80cefa0 0x7f05e8152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:59.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05ed5e1640 1 -- 172.21.15.12:0/2001188651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05e815cd50 msgr2=0x7f05e80b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:42:59.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05ed5e1640 1 --2- 172.21.15.12:0/2001188651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05e815cd50 0x7f05e80b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:42:59.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05ed5e1640 1 -- 172.21.15.12:0/2001188651 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e4009420 con 0x7f05e81538f0 2023-10-30T12:42:59.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05ed5e1640 1 --2- 172.21.15.12:0/2001188651 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05e81538f0 0x7f05e80b4300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f05d800c460 tx=0x7f05d800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:42:59.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05de7fc640 1 -- 172.21.15.12:0/2001188651 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d80052d0 con 0x7f05e81538f0 2023-10-30T12:42:59.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05de7fc640 1 -- 172.21.15.12:0/2001188651 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d8005450 con 0x7f05e81538f0 2023-10-30T12:42:59.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05eede4640 1 -- 172.21.15.12:0/2001188651 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e80b3260 con 0x7f05e81538f0 2023-10-30T12:42:59.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05de7fc640 1 -- 172.21.15.12:0/2001188651 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8003840 con 0x7f05e81538f0 2023-10-30T12:42:59.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05eede4640 1 -- 172.21.15.12:0/2001188651 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e80b3820 con 0x7f05e81538f0 2023-10-30T12:42:59.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.933+0000 7f05eede4640 1 -- 172.21.15.12:0/2001188651 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e804e170 con 0x7f05e81538f0 2023-10-30T12:42:59.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.937+0000 7f05de7fc640 1 -- 172.21.15.12:0/2001188651 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f05d80039a0 con 0x7f05e81538f0 2023-10-30T12:42:59.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.937+0000 7f05de7fc640 1 --2- 172.21.15.12:0/2001188651 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f05cc0664a0 0x7f05cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:42:59.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.937+0000 7f05de7fc640 1 -- 172.21.15.12:0/2001188651 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6025+0+0 (secure 0 0 0) 0x7f05d8083580 con 0x7f05e81538f0 2023-10-30T12:42:59.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.937+0000 7f05edde2640 1 --2- 172.21.15.12:0/2001188651 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f05cc0664a0 0x7f05cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:42:59.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.937+0000 7f05edde2640 1 --2- 172.21.15.12:0/2001188651 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f05cc0664a0 0x7f05cc068960 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f05e40b2000 tx=0x7f05e40aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:42:59.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:42:59.941+0000 7f05de7fc640 1 -- 172.21.15.12:0/2001188651 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f05d80508d0 con 0x7f05e81538f0 2023-10-30T12:43:00.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:00.149+0000 7f05eede4640 1 -- 172.21.15.12:0/2001188651 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05e8151f10 con 0x7f05e81538f0 2023-10-30T12:43:00.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:00.149+0000 7f05de7fc640 1 -- 172.21.15.12:0/2001188651 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+6528 (secure 0 0 0) 0x7f05d8053ef0 con 0x7f05e81538f0 2023-10-30T12:43:00.157 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:00.157 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:42:12.327346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:00.158 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 34 2023-10-30T12:43:00.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:00.153+0000 7f05eede4640 1 -- 172.21.15.12:0/2001188651 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f05cc0664a0 msgr2=0x7f05cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:00.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:00.153+0000 7f05eede4640 1 --2- 172.21.15.12:0/2001188651 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f05cc0664a0 0x7f05cc068960 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f05e40b2000 tx=0x7f05e40aa070 comp rx=0 tx=0).stop 2023-10-30T12:43:00.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:00.153+0000 7f05eede4640 1 -- 172.21.15.12:0/2001188651 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05e81538f0 msgr2=0x7f05e80b4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:00.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:00.153+0000 7f05eede4640 1 --2- 172.21.15.12:0/2001188651 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05e81538f0 0x7f05e80b4300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f05d800c460 tx=0x7f05d800c920 comp rx=0 tx=0).stop 2023-10-30T12:43:00.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:00.153+0000 7f05eede4640 1 -- 172.21.15.12:0/2001188651 shutdown_connections 2023-10-30T12:43:00.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:00.153+0000 7f05eede4640 1 --2- 172.21.15.12:0/2001188651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05e815cd50 0x7f05e80b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:00.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:00.153+0000 7f05eede4640 1 --2- 172.21.15.12:0/2001188651 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05e81538f0 0x7f05e80b4300 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:00.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:00.153+0000 7f05eede4640 1 --2- 172.21.15.12:0/2001188651 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f05cc0664a0 0x7f05cc068960 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:00.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:00.153+0000 7f05eede4640 1 --2- 172.21.15.12:0/2001188651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05e80cefa0 0x7f05e8152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:00.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:00.153+0000 7f05eede4640 1 -- 172.21.15.12:0/2001188651 >> 172.21.15.12:0/2001188651 conn(0x7f05e806a640 msgr2=0x7f05e80caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:00.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:00.153+0000 7f05eede4640 1 -- 172.21.15.12:0/2001188651 shutdown_connections 2023-10-30T12:43:00.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:00.153+0000 7f05eede4640 1 -- 172.21.15.12:0/2001188651 wait complete. 2023-10-30T12:43:00.178 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:43:02.179 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:02.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.317+0000 7fb7b306b640 1 Processor -- start 2023-10-30T12:43:02.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.317+0000 7fb7b306b640 1 -- start start 2023-10-30T12:43:02.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7b306b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7ac06f7a0 0x7fb7ac06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:02.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7b306b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb7ac0c6bd0 0x7fb7ac069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:02.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7b306b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb7ac071110 0x7fb7ac158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:02.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7b306b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7ac0700c0 con 0x7fb7ac0c6bd0 2023-10-30T12:43:02.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7b306b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7ac06a3d0 con 0x7fb7ac071110 2023-10-30T12:43:02.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7b306b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac066cf0 con 0x7fb7ac06f7a0 2023-10-30T12:43:02.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7abfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb7ac0c6bd0 0x7fb7ac069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:02.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7abfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb7ac0c6bd0 0x7fb7ac069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33416/0 (socket says 172.21.15.12:33416) 2023-10-30T12:43:02.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7abfff640 1 -- 172.21.15.12:0/537483283 learned_addr learned my addr 172.21.15.12:0/537483283 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:02.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7b1608640 1 --2- 172.21.15.12:0/537483283 >> v2:172.21.15.154:3300/0 conn(0x7fb7ac071110 0x7fb7ac158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:02.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7b0e07640 1 --2- 172.21.15.12:0/537483283 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7ac06f7a0 0x7fb7ac06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:02.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7abfff640 1 -- 172.21.15.12:0/537483283 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7ac06f7a0 msgr2=0x7fb7ac06fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:02.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7abfff640 1 --2- 172.21.15.12:0/537483283 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7ac06f7a0 0x7fb7ac06fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:02.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7abfff640 1 -- 172.21.15.12:0/537483283 >> v2:172.21.15.154:3300/0 conn(0x7fb7ac071110 msgr2=0x7fb7ac158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:02.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7abfff640 1 --2- 172.21.15.12:0/537483283 >> v2:172.21.15.154:3300/0 conn(0x7fb7ac071110 0x7fb7ac158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:02.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.321+0000 7fb7abfff640 1 -- 172.21.15.12:0/537483283 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ac0c70d0 con 0x7fb7ac0c6bd0 2023-10-30T12:43:02.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7abfff640 1 --2- 172.21.15.12:0/537483283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7ac0c6bd0 0x7fb7ac069e90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac06c7d0 tx=0x7fb79801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae2b247c6c5d1d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:02.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7ab7fe640 1 -- 172.21.15.12:0/537483283 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7980a89b0 con 0x7fb7ac0c6bd0 2023-10-30T12:43:02.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7ab7fe640 1 -- 172.21.15.12:0/537483283 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7980a8b10 con 0x7fb7ac0c6bd0 2023-10-30T12:43:02.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7ab7fe640 1 -- 172.21.15.12:0/537483283 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7980a8ea0 con 0x7fb7ac0c6bd0 2023-10-30T12:43:02.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7b306b640 1 -- 172.21.15.12:0/537483283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7ac0c6bd0 msgr2=0x7fb7ac069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:02.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7b306b640 1 --2- 172.21.15.12:0/537483283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7ac0c6bd0 0x7fb7ac069e90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac06c7d0 tx=0x7fb79801ecf0 comp rx=0 tx=0).stop 2023-10-30T12:43:02.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7b306b640 1 -- 172.21.15.12:0/537483283 shutdown_connections 2023-10-30T12:43:02.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7b306b640 1 --2- 172.21.15.12:0/537483283 >> v2:172.21.15.154:3300/0 conn(0x7fb7ac071110 0x7fb7ac158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:02.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7b306b640 1 --2- 172.21.15.12:0/537483283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7ac0c6bd0 0x7fb7ac069e90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:02.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7b306b640 1 --2- 172.21.15.12:0/537483283 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7ac06f7a0 0x7fb7ac06fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:02.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7b306b640 1 -- 172.21.15.12:0/537483283 >> 172.21.15.12:0/537483283 conn(0x7fb7ac0c7910 msgr2=0x7fb7ac0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:02.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7b306b640 1 -- 172.21.15.12:0/537483283 shutdown_connections 2023-10-30T12:43:02.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7b306b640 1 -- 172.21.15.12:0/537483283 wait complete. 2023-10-30T12:43:02.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7b306b640 1 Processor -- start 2023-10-30T12:43:02.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7b306b640 1 -- start start 2023-10-30T12:43:02.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.325+0000 7fb7b306b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7ac06f7a0 0x7fb7ac0c1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:02.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7b306b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7ac071110 0x7fb7ac1ef370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:02.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7b306b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7ac0c6bd0 0x7fb7ac1ef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:02.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7b306b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac1eef20 con 0x7fb7ac071110 2023-10-30T12:43:02.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7b306b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac1f25c0 con 0x7fb7ac06f7a0 2023-10-30T12:43:02.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7b306b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac1f28a0 con 0x7fb7ac0c6bd0 2023-10-30T12:43:02.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7abfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7ac071110 0x7fb7ac1ef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:02.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7abfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7ac071110 0x7fb7ac1ef370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33422/0 (socket says 172.21.15.12:33422) 2023-10-30T12:43:02.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7abfff640 1 -- 172.21.15.12:0/1534266655 learned_addr learned my addr 172.21.15.12:0/1534266655 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:02.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7b0e07640 1 --2- 172.21.15.12:0/1534266655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7ac06f7a0 0x7fb7ac0c1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:02.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7b1608640 1 --2- 172.21.15.12:0/1534266655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7ac0c6bd0 0x7fb7ac1ef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:02.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7abfff640 1 -- 172.21.15.12:0/1534266655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7ac0c6bd0 msgr2=0x7fb7ac1ef8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:02.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7abfff640 1 --2- 172.21.15.12:0/1534266655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7ac0c6bd0 0x7fb7ac1ef8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:02.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7abfff640 1 -- 172.21.15.12:0/1534266655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7ac06f7a0 msgr2=0x7fb7ac0c1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:02.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7abfff640 1 --2- 172.21.15.12:0/1534266655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7ac06f7a0 0x7fb7ac0c1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:02.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7abfff640 1 -- 172.21.15.12:0/1534266655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7980094d0 con 0x7fb7ac071110 2023-10-30T12:43:02.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7b1608640 1 --2- 172.21.15.12:0/1534266655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7ac0c6bd0 0x7fb7ac1ef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:43:02.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7abfff640 1 --2- 172.21.15.12:0/1534266655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7ac071110 0x7fb7ac1ef370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb79801d3a0 tx=0x7fb79801ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:02.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7a97fa640 1 -- 172.21.15.12:0/1534266655 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7980ba600 con 0x7fb7ac071110 2023-10-30T12:43:02.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7a97fa640 1 -- 172.21.15.12:0/1534266655 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7980ba760 con 0x7fb7ac071110 2023-10-30T12:43:02.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7b306b640 1 -- 172.21.15.12:0/1534266655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ac1f2b80 con 0x7fb7ac071110 2023-10-30T12:43:02.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7a97fa640 1 -- 172.21.15.12:0/1534266655 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7980bab10 con 0x7fb7ac071110 2023-10-30T12:43:02.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7b306b640 1 -- 172.21.15.12:0/1534266655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ac1f3140 con 0x7fb7ac071110 2023-10-30T12:43:02.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7b306b640 1 -- 172.21.15.12:0/1534266655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7ac04e170 con 0x7fb7ac071110 2023-10-30T12:43:02.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.329+0000 7fb7a97fa640 1 -- 172.21.15.12:0/1534266655 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb7980bac70 con 0x7fb7ac071110 2023-10-30T12:43:02.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.333+0000 7fb7a97fa640 1 --2- 172.21.15.12:0/1534266655 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7900664a0 0x7fb790068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:02.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.333+0000 7fb7a97fa640 1 -- 172.21.15.12:0/1534266655 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6025+0+0 (secure 0 0 0) 0x7fb798122550 con 0x7fb7ac071110 2023-10-30T12:43:02.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.333+0000 7fb7b0e07640 1 --2- 172.21.15.12:0/1534266655 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7900664a0 0x7fb790068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:02.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.333+0000 7fb7b0e07640 1 --2- 172.21.15.12:0/1534266655 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7900664a0 0x7fb790068960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00045f0 tx=0x7fb7a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:02.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.337+0000 7fb7a97fa640 1 -- 172.21.15.12:0/1534266655 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb7980ef7f0 con 0x7fb7ac071110 2023-10-30T12:43:02.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.529+0000 7fb7b306b640 1 -- 172.21.15.12:0/1534266655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7ac06b350 con 0x7fb7ac071110 2023-10-30T12:43:02.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.529+0000 7fb7a97fa640 1 -- 172.21.15.12:0/1534266655 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+6528 (secure 0 0 0) 0x7fb7980f2e10 con 0x7fb7ac071110 2023-10-30T12:43:02.536 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:02.536 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:42:12.327346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:02.537 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 34 2023-10-30T12:43:02.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.533+0000 7fb7b306b640 1 -- 172.21.15.12:0/1534266655 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7900664a0 msgr2=0x7fb790068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:02.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.533+0000 7fb7b306b640 1 --2- 172.21.15.12:0/1534266655 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7900664a0 0x7fb790068960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00045f0 tx=0x7fb7a0008040 comp rx=0 tx=0).stop 2023-10-30T12:43:02.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.533+0000 7fb7b306b640 1 -- 172.21.15.12:0/1534266655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7ac071110 msgr2=0x7fb7ac1ef370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:02.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.533+0000 7fb7b306b640 1 --2- 172.21.15.12:0/1534266655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7ac071110 0x7fb7ac1ef370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb79801d3a0 tx=0x7fb79801ba90 comp rx=0 tx=0).stop 2023-10-30T12:43:02.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.533+0000 7fb7b306b640 1 -- 172.21.15.12:0/1534266655 shutdown_connections 2023-10-30T12:43:02.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.533+0000 7fb7b306b640 1 --2- 172.21.15.12:0/1534266655 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7900664a0 0x7fb790068960 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:02.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.533+0000 7fb7b306b640 1 --2- 172.21.15.12:0/1534266655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7ac0c6bd0 0x7fb7ac1ef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:02.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.533+0000 7fb7b306b640 1 --2- 172.21.15.12:0/1534266655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7ac071110 0x7fb7ac1ef370 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:02.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.533+0000 7fb7b306b640 1 --2- 172.21.15.12:0/1534266655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7ac06f7a0 0x7fb7ac0c1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:02.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.533+0000 7fb7b306b640 1 -- 172.21.15.12:0/1534266655 >> 172.21.15.12:0/1534266655 conn(0x7fb7ac0c7910 msgr2=0x7fb7ac06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:02.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.533+0000 7fb7b306b640 1 -- 172.21.15.12:0/1534266655 shutdown_connections 2023-10-30T12:43:02.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:02.533+0000 7fb7b306b640 1 -- 172.21.15.12:0/1534266655 wait complete. 2023-10-30T12:43:02.557 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:43:04.558 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:04.703 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:43:04.707 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:43:04.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.705+0000 7f73ce53e640 1 Processor -- start 2023-10-30T12:43:04.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.705+0000 7f73ce53e640 1 -- start start 2023-10-30T12:43:04.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.705+0000 7f73ce53e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f73c81538a0 0x7f73c815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:04.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.705+0000 7f73ce53e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f73c80ca0e0 0x7f73c815a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:04.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.705+0000 7f73ce53e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c80cb800 con 0x7f73c80cf160 2023-10-30T12:43:04.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.705+0000 7f73ce53e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c815dc30 con 0x7f73c80ca0e0 2023-10-30T12:43:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.705+0000 7f73ce53e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c80cb330 con 0x7f73c81538a0 2023-10-30T12:43:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.709+0000 7f73c77fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f73c80ca0e0 0x7f73c815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.709+0000 7f73c77fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f73c80ca0e0 0x7f73c815a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33436/0 (socket says 172.21.15.12:33436) 2023-10-30T12:43:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.709+0000 7f73c77fe640 1 -- 172.21.15.12:0/2689055838 learned_addr learned my addr 172.21.15.12:0/2689055838 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.709+0000 7f73ccadb640 1 --1- 172.21.15.12:0/2689055838 >> v1:172.21.15.12:6789/0 conn(0x7f73c80cf160 0x7f73c80cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45644/0 (socket says 172.21.15.12:45644) 2023-10-30T12:43:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.709+0000 7f73c7fff640 1 --2- 172.21.15.12:0/2689055838 >> v2:172.21.15.154:3300/0 conn(0x7f73c81538a0 0x7f73c815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.709+0000 7f73c77fe640 1 -- 172.21.15.12:0/2689055838 >> v2:172.21.15.154:3300/0 conn(0x7f73c81538a0 msgr2=0x7f73c815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.709+0000 7f73c77fe640 1 --2- 172.21.15.12:0/2689055838 >> v2:172.21.15.154:3300/0 conn(0x7f73c81538a0 0x7f73c815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.709+0000 7f73c77fe640 1 -- 172.21.15.12:0/2689055838 >> v1:172.21.15.12:6789/0 conn(0x7f73c80cf160 legacy=0x7f73c80cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.709+0000 7f73c77fe640 1 -- 172.21.15.12:0/2689055838 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c80cad70 con 0x7f73c80ca0e0 2023-10-30T12:43:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.709+0000 7f73c77fe640 1 --2- 172.21.15.12:0/2689055838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73c80ca0e0 0x7f73c815a090 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f73c80cfd10 tx=0x7f73bc01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef68e99e63e36b07 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.709+0000 7f73c6ffd640 1 -- 172.21.15.12:0/2689055838 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc0a89f0 con 0x7f73c80ca0e0 2023-10-30T12:43:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.709+0000 7f73c6ffd640 1 -- 172.21.15.12:0/2689055838 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73bc0a8b50 con 0x7f73c80ca0e0 2023-10-30T12:43:04.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.709+0000 7f73c6ffd640 1 -- 172.21.15.12:0/2689055838 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc0b0440 con 0x7f73c80ca0e0 2023-10-30T12:43:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.713+0000 7f73ce53e640 1 -- 172.21.15.12:0/2689055838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73c80ca0e0 msgr2=0x7f73c815a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.713+0000 7f73ce53e640 1 --2- 172.21.15.12:0/2689055838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73c80ca0e0 0x7f73c815a090 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f73c80cfd10 tx=0x7f73bc01ecf0 comp rx=0 tx=0).stop 2023-10-30T12:43:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.713+0000 7f73ce53e640 1 -- 172.21.15.12:0/2689055838 shutdown_connections 2023-10-30T12:43:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.713+0000 7f73ce53e640 1 --2- 172.21.15.12:0/2689055838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73c80ca0e0 0x7f73c815a090 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.713+0000 7f73ce53e640 1 --2- 172.21.15.12:0/2689055838 >> v2:172.21.15.154:3300/0 conn(0x7f73c81538a0 0x7f73c815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:04.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.713+0000 7f73ce53e640 1 -- 172.21.15.12:0/2689055838 >> 172.21.15.12:0/2689055838 conn(0x7f73c806bfe0 msgr2=0x7f73c806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:04.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.713+0000 7f73ce53e640 1 -- 172.21.15.12:0/2689055838 shutdown_connections 2023-10-30T12:43:04.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.713+0000 7f73ce53e640 1 -- 172.21.15.12:0/2689055838 wait complete. 2023-10-30T12:43:04.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.713+0000 7f73ce53e640 1 Processor -- start 2023-10-30T12:43:04.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.713+0000 7f73ce53e640 1 -- start start 2023-10-30T12:43:04.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73ce53e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73c80ca0e0 0x7f73c8140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:04.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73ce53e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73c80cf160 0x7f73c80ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:04.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73ce53e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73c81538a0 0x7f73c80cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:04.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73ce53e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c81e4380 con 0x7f73c80ca0e0 2023-10-30T12:43:04.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73ce53e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c8158620 con 0x7f73c81538a0 2023-10-30T12:43:04.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73ce53e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c8159690 con 0x7f73c80cf160 2023-10-30T12:43:04.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73c80ca0e0 0x7f73c8140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:04.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73c80ca0e0 0x7f73c8140ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33446/0 (socket says 172.21.15.12:33446) 2023-10-30T12:43:04.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73c7fff640 1 -- 172.21.15.12:0/2826887032 learned_addr learned my addr 172.21.15.12:0/2826887032 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:04.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73c77fe640 1 --2- 172.21.15.12:0/2826887032 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73c80cf160 0x7f73c80ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:04.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73ccadb640 1 --2- 172.21.15.12:0/2826887032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73c81538a0 0x7f73c80cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:04.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73c7fff640 1 -- 172.21.15.12:0/2826887032 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73c80cf160 msgr2=0x7f73c80ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:04.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73c7fff640 1 --2- 172.21.15.12:0/2826887032 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73c80cf160 0x7f73c80ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:04.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73c7fff640 1 -- 172.21.15.12:0/2826887032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73c81538a0 msgr2=0x7f73c80cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:04.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73c7fff640 1 --2- 172.21.15.12:0/2826887032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73c81538a0 0x7f73c80cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:04.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73c7fff640 1 -- 172.21.15.12:0/2826887032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc0094d0 con 0x7f73c80ca0e0 2023-10-30T12:43:04.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73c7fff640 1 --2- 172.21.15.12:0/2826887032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73c80ca0e0 0x7f73c8140ea0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f73b800d620 tx=0x7f73b800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:04.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73c4ff9640 1 -- 172.21.15.12:0/2826887032 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b800b620 con 0x7f73c80ca0e0 2023-10-30T12:43:04.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73c4ff9640 1 -- 172.21.15.12:0/2826887032 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b800b780 con 0x7f73c80ca0e0 2023-10-30T12:43:04.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73c4ff9640 1 -- 172.21.15.12:0/2826887032 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b800bb10 con 0x7f73c80ca0e0 2023-10-30T12:43:04.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73ce53e640 1 -- 172.21.15.12:0/2826887032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c81e51e0 con 0x7f73c80ca0e0 2023-10-30T12:43:04.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73ce53e640 1 -- 172.21.15.12:0/2826887032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c81e5610 con 0x7f73c80ca0e0 2023-10-30T12:43:04.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.717+0000 7f73ce53e640 1 -- 172.21.15.12:0/2826887032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c80b2a50 con 0x7f73c80ca0e0 2023-10-30T12:43:04.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.721+0000 7f73c4ff9640 1 -- 172.21.15.12:0/2826887032 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f73b800bc70 con 0x7f73c80ca0e0 2023-10-30T12:43:04.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.721+0000 7f73c4ff9640 1 --2- 172.21.15.12:0/2826887032 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f73b40664d0 0x7f73b4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:04.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.721+0000 7f73c4ff9640 1 -- 172.21.15.12:0/2826887032 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6025+0+0 (secure 0 0 0) 0x7f73b80842e0 con 0x7f73c80ca0e0 2023-10-30T12:43:04.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.721+0000 7f73c77fe640 1 --2- 172.21.15.12:0/2826887032 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f73b40664d0 0x7f73b4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:04.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.721+0000 7f73c77fe640 1 --2- 172.21.15.12:0/2826887032 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f73b40664d0 0x7f73b4068990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0b2000 tx=0x7f73bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:04.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.725+0000 7f73c4ff9640 1 -- 172.21.15.12:0/2826887032 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f73b80516c0 con 0x7f73c80ca0e0 2023-10-30T12:43:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.917+0000 7f73ce53e640 1 -- 172.21.15.12:0/2826887032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73c804e170 con 0x7f73c80ca0e0 2023-10-30T12:43:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.917+0000 7f73c4ff9640 1 -- 172.21.15.12:0/2826887032 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+6528 (secure 0 0 0) 0x7f73b8007380 con 0x7f73c80ca0e0 2023-10-30T12:43:04.923 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:04.923 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:42:12.327346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 34 2023-10-30T12:43:04.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.917+0000 7f73ce53e640 1 -- 172.21.15.12:0/2826887032 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f73b40664d0 msgr2=0x7f73b4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:04.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.917+0000 7f73ce53e640 1 --2- 172.21.15.12:0/2826887032 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f73b40664d0 0x7f73b4068990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0b2000 tx=0x7f73bc009040 comp rx=0 tx=0).stop 2023-10-30T12:43:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.917+0000 7f73ce53e640 1 -- 172.21.15.12:0/2826887032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73c80ca0e0 msgr2=0x7f73c8140ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.917+0000 7f73ce53e640 1 --2- 172.21.15.12:0/2826887032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73c80ca0e0 0x7f73c8140ea0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f73b800d620 tx=0x7f73b800dae0 comp rx=0 tx=0).stop 2023-10-30T12:43:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.917+0000 7f73ce53e640 1 -- 172.21.15.12:0/2826887032 shutdown_connections 2023-10-30T12:43:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.917+0000 7f73ce53e640 1 --2- 172.21.15.12:0/2826887032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73c81538a0 0x7f73c80cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.917+0000 7f73ce53e640 1 --2- 172.21.15.12:0/2826887032 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f73b40664d0 0x7f73b4068990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:04.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.917+0000 7f73ce53e640 1 --2- 172.21.15.12:0/2826887032 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73c80cf160 0x7f73c80ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:04.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.917+0000 7f73ce53e640 1 --2- 172.21.15.12:0/2826887032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73c80ca0e0 0x7f73c8140ea0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:04.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.917+0000 7f73ce53e640 1 -- 172.21.15.12:0/2826887032 >> 172.21.15.12:0/2826887032 conn(0x7f73c806bfe0 msgr2=0x7f73c80c5210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:04.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.917+0000 7f73ce53e640 1 -- 172.21.15.12:0/2826887032 shutdown_connections 2023-10-30T12:43:04.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:04.917+0000 7f73ce53e640 1 -- 172.21.15.12:0/2826887032 wait complete. 2023-10-30T12:43:04.944 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:43:06.945 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:07.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.093+0000 7f9dc9254640 1 Processor -- start 2023-10-30T12:43:07.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.093+0000 7f9dc9254640 1 -- start start 2023-10-30T12:43:07.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc9254640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9dc406e470 0x7f9dc406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:07.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc9254640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc406afa0 con 0x7f9dc4153d70 2023-10-30T12:43:07.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc9254640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc4154150 con 0x7f9dc40c8490 2023-10-30T12:43:07.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc9254640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc406d780 con 0x7f9dc406e470 2023-10-30T12:43:07.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc2d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9dc4153d70 0x7f9dc40cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45660/0 (socket says 172.21.15.12:45660) 2023-10-30T12:43:07.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc2d9d640 1 -- 172.21.15.12:0/3587993746 learned_addr learned my addr 172.21.15.12:0/3587993746 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:43:07.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dbbfff640 1 --2- 172.21.15.12:0/3587993746 >> v2:172.21.15.154:3300/0 conn(0x7f9dc406e470 0x7f9dc406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:07.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc259c640 1 -- 172.21.15.12:0/3587993746 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334395837 0 0) 0x7f9dc406afa0 con 0x7f9dc4153d70 2023-10-30T12:43:07.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc259c640 1 -- 172.21.15.12:0/3587993746 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9db0003530 con 0x7f9dc4153d70 2023-10-30T12:43:07.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc259c640 1 -- 172.21.15.12:0/3587993746 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 642728010 0 0) 0x7f9db0003530 con 0x7f9dc4153d70 2023-10-30T12:43:07.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc259c640 1 -- 172.21.15.12:0/3587993746 >> v2:172.21.15.154:3300/0 conn(0x7f9dc406e470 msgr2=0x7f9dc406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:07.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc259c640 1 --2- 172.21.15.12:0/3587993746 >> v2:172.21.15.154:3300/0 conn(0x7f9dc406e470 0x7f9dc406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:07.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc259c640 1 -- 172.21.15.12:0/3587993746 >> v1:172.21.15.154:6789/0 conn(0x7f9dc40c8490 legacy=0x7f9dc40c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:07.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc259c640 1 -- 172.21.15.12:0/3587993746 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc406e030 con 0x7f9dc4153d70 2023-10-30T12:43:07.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc259c640 1 -- 172.21.15.12:0/3587993746 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f9db4001680 con 0x7f9dc4153d70 2023-10-30T12:43:07.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc259c640 1 -- 172.21.15.12:0/3587993746 >> v1:172.21.15.12:6789/0 conn(0x7f9dc4153d70 legacy=0x7f9dc40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:07.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc259c640 1 --2- 172.21.15.12:0/3587993746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9db0003980 0x7f9db0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:07.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc259c640 1 --2- 172.21.15.12:0/3587993746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9db00047d0 0x7f9db0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:07.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc259c640 1 --2- 172.21.15.12:0/3587993746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9db0005190 0x7f9db000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:07.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc259c640 1 -- 172.21.15.12:0/3587993746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc406d780 con 0x7f9db0005190 2023-10-30T12:43:07.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc259c640 1 -- 172.21.15.12:0/3587993746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db000c030 con 0x7f9db0003980 2023-10-30T12:43:07.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc259c640 1 -- 172.21.15.12:0/3587993746 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db000c1f0 con 0x7f9db00047d0 2023-10-30T12:43:07.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc359e640 1 --2- 172.21.15.12:0/3587993746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9db0005190 0x7f9db000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:07.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc2d9d640 1 --2- 172.21.15.12:0/3587993746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9db0003980 0x7f9db0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:07.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dbbfff640 1 --2- 172.21.15.12:0/3587993746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9db00047d0 0x7f9db0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:07.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc359e640 1 -- 172.21.15.12:0/3587993746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9db00047d0 msgr2=0x7f9db0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:07.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc359e640 1 --2- 172.21.15.12:0/3587993746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9db00047d0 0x7f9db0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:07.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc359e640 1 -- 172.21.15.12:0/3587993746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9db0003980 msgr2=0x7f9db0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:07.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc359e640 1 --2- 172.21.15.12:0/3587993746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9db0003980 0x7f9db0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:07.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.097+0000 7f9dc359e640 1 -- 172.21.15.12:0/3587993746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db0003530 con 0x7f9db0005190 2023-10-30T12:43:07.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.101+0000 7f9dc359e640 1 -- 172.21.15.12:0/3587993746 reap_dead start 2023-10-30T12:43:07.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.101+0000 7f9dc359e640 1 --2- 172.21.15.12:0/3587993746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9db0005190 0x7f9db000b990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc001e70 tx=0x7f9dbc01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d4298ae2983346 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:07.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.101+0000 7f9dc259c640 1 -- 172.21.15.12:0/3587993746 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc003b80 con 0x7f9db0005190 2023-10-30T12:43:07.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.101+0000 7f9dc259c640 1 -- 172.21.15.12:0/3587993746 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc003d20 con 0x7f9db0005190 2023-10-30T12:43:07.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.101+0000 7f9dc259c640 1 -- 172.21.15.12:0/3587993746 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc005260 con 0x7f9db0005190 2023-10-30T12:43:07.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.101+0000 7f9dc9254640 1 -- 172.21.15.12:0/3587993746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9db0005190 msgr2=0x7f9db000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:07.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.101+0000 7f9dc9254640 1 --2- 172.21.15.12:0/3587993746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9db0005190 0x7f9db000b990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc001e70 tx=0x7f9dbc01c760 comp rx=0 tx=0).stop 2023-10-30T12:43:07.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.101+0000 7f9dc9254640 1 -- 172.21.15.12:0/3587993746 shutdown_connections 2023-10-30T12:43:07.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.101+0000 7f9dc9254640 1 --2- 172.21.15.12:0/3587993746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9db0005190 0x7f9db000b990 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:07.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.101+0000 7f9dc9254640 1 -- 172.21.15.12:0/3587993746 >> 172.21.15.12:0/3587993746 conn(0x7f9dc4071ea0 msgr2=0x7f9dc4065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:07.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.101+0000 7f9dc9254640 1 -- 172.21.15.12:0/3587993746 shutdown_connections 2023-10-30T12:43:07.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.101+0000 7f9dc9254640 1 -- 172.21.15.12:0/3587993746 wait complete. 2023-10-30T12:43:07.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dc9254640 1 Processor -- start 2023-10-30T12:43:07.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dc9254640 1 -- start start 2023-10-30T12:43:07.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dc9254640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9db0005190 0x7f9dc4149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:07.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dc9254640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dc414dce0 0x7f9dc41575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:07.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dc9254640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9dc4156230 0x7f9dc4157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:07.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dc9254640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc414a130 con 0x7f9dc414dce0 2023-10-30T12:43:07.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dc9254640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4157370 con 0x7f9db0005190 2023-10-30T12:43:07.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dc9254640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc41e5f60 con 0x7f9dc4156230 2023-10-30T12:43:07.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dbbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dc414dce0 0x7f9dc41575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:07.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dbbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dc414dce0 0x7f9dc41575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33466/0 (socket says 172.21.15.12:33466) 2023-10-30T12:43:07.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dbbfff640 1 -- 172.21.15.12:0/2509196499 learned_addr learned my addr 172.21.15.12:0/2509196499 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:07.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dc359e640 1 --2- 172.21.15.12:0/2509196499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9dc4156230 0x7f9dc4157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:07.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dc2d9d640 1 --2- 172.21.15.12:0/2509196499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9db0005190 0x7f9dc4149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:07.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dbbfff640 1 -- 172.21.15.12:0/2509196499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9dc4156230 msgr2=0x7f9dc4157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:07.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dbbfff640 1 --2- 172.21.15.12:0/2509196499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9dc4156230 0x7f9dc4157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:07.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dbbfff640 1 -- 172.21.15.12:0/2509196499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9db0005190 msgr2=0x7f9dc4149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:07.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dbbfff640 1 --2- 172.21.15.12:0/2509196499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9db0005190 0x7f9dc4149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:07.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dbbfff640 1 -- 172.21.15.12:0/2509196499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc4156dd0 con 0x7f9dc414dce0 2023-10-30T12:43:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dc359e640 1 --2- 172.21.15.12:0/2509196499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9dc4156230 0x7f9dc4157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:43:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dbbfff640 1 --2- 172.21.15.12:0/2509196499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dc414dce0 0x7f9dc41575b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9dac011590 tx=0x7f9dac011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dbb7fe640 1 -- 172.21.15.12:0/2509196499 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac015870 con 0x7f9dc414dce0 2023-10-30T12:43:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dbb7fe640 1 -- 172.21.15.12:0/2509196499 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac0159d0 con 0x7f9dc414dce0 2023-10-30T12:43:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dbb7fe640 1 -- 172.21.15.12:0/2509196499 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac015d40 con 0x7f9dc414dce0 2023-10-30T12:43:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dc9254640 1 -- 172.21.15.12:0/2509196499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc4157090 con 0x7f9dc414dce0 2023-10-30T12:43:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dc9254640 1 -- 172.21.15.12:0/2509196499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc41e64e0 con 0x7f9dc414dce0 2023-10-30T12:43:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.105+0000 7f9dc9254640 1 -- 172.21.15.12:0/2509196499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc404e170 con 0x7f9dc414dce0 2023-10-30T12:43:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.109+0000 7f9dbb7fe640 1 -- 172.21.15.12:0/2509196499 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9dac013400 con 0x7f9dc414dce0 2023-10-30T12:43:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.109+0000 7f9dbb7fe640 1 --2- 172.21.15.12:0/2509196499 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9da40664d0 0x7f9da4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:07.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.109+0000 7f9dbb7fe640 1 -- 172.21.15.12:0/2509196499 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6025+0+0 (secure 0 0 0) 0x7f9dac084400 con 0x7f9dc414dce0 2023-10-30T12:43:07.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.109+0000 7f9dc2d9d640 1 --2- 172.21.15.12:0/2509196499 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9da40664d0 0x7f9da4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:07.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.109+0000 7f9dc2d9d640 1 --2- 172.21.15.12:0/2509196499 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9da40664d0 0x7f9da4068990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9db4001060 tx=0x7f9db4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:07.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.113+0000 7f9dbb7fe640 1 -- 172.21.15.12:0/2509196499 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9dac0516a0 con 0x7f9dc414dce0 2023-10-30T12:43:07.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.305+0000 7f9dc9254640 1 -- 172.21.15.12:0/2509196499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dc41e6670 con 0x7f9dc414dce0 2023-10-30T12:43:07.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.305+0000 7f9dbb7fe640 1 -- 172.21.15.12:0/2509196499 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+6528 (secure 0 0 0) 0x7f9dac054cc0 con 0x7f9dc414dce0 2023-10-30T12:43:07.312 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:07.312 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:42:12.327346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:07.313 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 34 2023-10-30T12:43:07.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.305+0000 7f9dc9254640 1 -- 172.21.15.12:0/2509196499 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9da40664d0 msgr2=0x7f9da4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:07.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.305+0000 7f9dc9254640 1 --2- 172.21.15.12:0/2509196499 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9da40664d0 0x7f9da4068990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9db4001060 tx=0x7f9db4009040 comp rx=0 tx=0).stop 2023-10-30T12:43:07.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.305+0000 7f9dc9254640 1 -- 172.21.15.12:0/2509196499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dc414dce0 msgr2=0x7f9dc41575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:07.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.305+0000 7f9dc9254640 1 --2- 172.21.15.12:0/2509196499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dc414dce0 0x7f9dc41575b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9dac011590 tx=0x7f9dac011a50 comp rx=0 tx=0).stop 2023-10-30T12:43:07.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.305+0000 7f9dc9254640 1 -- 172.21.15.12:0/2509196499 shutdown_connections 2023-10-30T12:43:07.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.305+0000 7f9dc9254640 1 --2- 172.21.15.12:0/2509196499 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9da40664d0 0x7f9da4068990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:07.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.305+0000 7f9dc9254640 1 --2- 172.21.15.12:0/2509196499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9dc4156230 0x7f9dc4157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:07.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.305+0000 7f9dc9254640 1 --2- 172.21.15.12:0/2509196499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dc414dce0 0x7f9dc41575b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:07.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.305+0000 7f9dc9254640 1 --2- 172.21.15.12:0/2509196499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9db0005190 0x7f9dc4149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:07.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.305+0000 7f9dc9254640 1 -- 172.21.15.12:0/2509196499 >> 172.21.15.12:0/2509196499 conn(0x7f9dc4071ea0 msgr2=0x7f9dc40cc600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:07.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.309+0000 7f9dc9254640 1 -- 172.21.15.12:0/2509196499 shutdown_connections 2023-10-30T12:43:07.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:07.309+0000 7f9dc9254640 1 -- 172.21.15.12:0/2509196499 wait complete. 2023-10-30T12:43:07.333 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:43:09.334 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f14447f5640 1 Processor -- start 2023-10-30T12:43:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f14447f5640 1 -- start start 2023-10-30T12:43:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f14447f5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f143c1538a0 0x7f143c15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f14447f5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f143c0cf160 0x7f143c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:09.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f14447f5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143c0cb800 con 0x7f143c0ca0e0 2023-10-30T12:43:09.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f14447f5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f143c15dc30 con 0x7f143c0cf160 2023-10-30T12:43:09.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f14447f5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143c0cb330 con 0x7f143c1538a0 2023-10-30T12:43:09.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f1442591640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f143c1538a0 0x7f143c15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:09.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f1442d92640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f143c0cf160 0x7f143c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:09.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f1442d92640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f143c0cf160 0x7f143c0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:52692/0 (socket says 172.21.15.12:52692) 2023-10-30T12:43:09.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f1441d90640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f143c0ca0e0 0x7f143c15a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:47810/0 (socket says 172.21.15.12:47810) 2023-10-30T12:43:09.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f1442d92640 1 -- 172.21.15.12:0/2297035389 learned_addr learned my addr 172.21.15.12:0/2297035389 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:09.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f1442591640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f143c1538a0 0x7f143c15d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:53976/0 (socket says 172.21.15.12:53976) 2023-10-30T12:43:09.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f1442591640 1 -- 172.21.15.12:0/2297035389 >> v2:172.21.15.154:3300/0 conn(0x7f143c0cf160 msgr2=0x7f143c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:09.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f1442591640 1 --2- 172.21.15.12:0/2297035389 >> v2:172.21.15.154:3300/0 conn(0x7f143c0cf160 0x7f143c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f1442591640 1 -- 172.21.15.12:0/2297035389 >> v1:172.21.15.154:6789/0 conn(0x7f143c0ca0e0 legacy=0x7f143c15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f1442591640 1 -- 172.21.15.12:0/2297035389 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c0cad70 con 0x7f143c1538a0 2023-10-30T12:43:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.489+0000 7f1442d92640 1 --2- 172.21.15.12:0/2297035389 >> v2:172.21.15.154:3300/0 conn(0x7f143c0cf160 0x7f143c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:43:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.493+0000 7f1442591640 1 --2- 172.21.15.12:0/2297035389 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f143c1538a0 0x7f143c15d6f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f14380097c0 tx=0x7f143801ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef87171e5de51226 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.493+0000 7f144158f640 1 -- 172.21.15.12:0/2297035389 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14380a8970 con 0x7f143c1538a0 2023-10-30T12:43:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.493+0000 7f144158f640 1 -- 172.21.15.12:0/2297035389 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14380a8ad0 con 0x7f143c1538a0 2023-10-30T12:43:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.493+0000 7f144158f640 1 -- 172.21.15.12:0/2297035389 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14380a8e60 con 0x7f143c1538a0 2023-10-30T12:43:09.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.493+0000 7f14447f5640 1 -- 172.21.15.12:0/2297035389 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f143c1538a0 msgr2=0x7f143c15d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:09.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.493+0000 7f14447f5640 1 --2- 172.21.15.12:0/2297035389 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f143c1538a0 0x7f143c15d6f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f14380097c0 tx=0x7f143801ec70 comp rx=0 tx=0).stop 2023-10-30T12:43:09.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.493+0000 7f14447f5640 1 -- 172.21.15.12:0/2297035389 shutdown_connections 2023-10-30T12:43:09.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.493+0000 7f14447f5640 1 --2- 172.21.15.12:0/2297035389 >> v2:172.21.15.154:3300/0 conn(0x7f143c0cf160 0x7f143c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:09.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.493+0000 7f14447f5640 1 --2- 172.21.15.12:0/2297035389 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f143c1538a0 0x7f143c15d6f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:09.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.493+0000 7f14447f5640 1 -- 172.21.15.12:0/2297035389 >> 172.21.15.12:0/2297035389 conn(0x7f143c06bfe0 msgr2=0x7f143c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:09.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.493+0000 7f14447f5640 1 -- 172.21.15.12:0/2297035389 shutdown_connections 2023-10-30T12:43:09.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.493+0000 7f14447f5640 1 -- 172.21.15.12:0/2297035389 wait complete. 2023-10-30T12:43:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f14447f5640 1 Processor -- start 2023-10-30T12:43:09.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f14447f5640 1 -- start start 2023-10-30T12:43:09.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f14447f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f143c0ca0e0 0x7f143c1e3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:09.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f14447f5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f143c0cf160 0x7f143c1e3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:09.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f14447f5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f143c1538a0 0x7f143c0c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:09.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f14447f5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c1e2dc0 con 0x7f143c0ca0e0 2023-10-30T12:43:09.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f14447f5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c1e2ae0 con 0x7f143c0cf160 2023-10-30T12:43:09.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f14447f5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143c1e4df0 con 0x7f143c1538a0 2023-10-30T12:43:09.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f1442591640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f143c0ca0e0 0x7f143c1e3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:09.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f1442591640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f143c0ca0e0 0x7f143c1e3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33468/0 (socket says 172.21.15.12:33468) 2023-10-30T12:43:09.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f1442591640 1 -- 172.21.15.12:0/2526121949 learned_addr learned my addr 172.21.15.12:0/2526121949 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:09.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f1441d90640 1 --2- 172.21.15.12:0/2526121949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f143c0cf160 0x7f143c1e3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:09.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f1442d92640 1 --2- 172.21.15.12:0/2526121949 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f143c1538a0 0x7f143c0c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:09.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f1442591640 1 -- 172.21.15.12:0/2526121949 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f143c1538a0 msgr2=0x7f143c0c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:09.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f1442591640 1 --2- 172.21.15.12:0/2526121949 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f143c1538a0 0x7f143c0c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:09.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f1442591640 1 -- 172.21.15.12:0/2526121949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f143c0cf160 msgr2=0x7f143c1e3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:09.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f1442591640 1 --2- 172.21.15.12:0/2526121949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f143c0cf160 0x7f143c1e3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:09.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f1442591640 1 -- 172.21.15.12:0/2526121949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1434009b30 con 0x7f143c0ca0e0 2023-10-30T12:43:09.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f1442591640 1 --2- 172.21.15.12:0/2526121949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f143c0ca0e0 0x7f143c1e3250 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f143801d320 tx=0x7f143801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:09.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f1432ffd640 1 -- 172.21.15.12:0/2526121949 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14380c0070 con 0x7f143c0ca0e0 2023-10-30T12:43:09.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f1432ffd640 1 -- 172.21.15.12:0/2526121949 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1438006e20 con 0x7f143c0ca0e0 2023-10-30T12:43:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f1432ffd640 1 -- 172.21.15.12:0/2526121949 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14380ba640 con 0x7f143c0ca0e0 2023-10-30T12:43:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f14447f5640 1 -- 172.21.15.12:0/2526121949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1438009420 con 0x7f143c0ca0e0 2023-10-30T12:43:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.497+0000 7f14447f5640 1 -- 172.21.15.12:0/2526121949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143c0b49d0 con 0x7f143c0ca0e0 2023-10-30T12:43:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.501+0000 7f14447f5640 1 -- 172.21.15.12:0/2526121949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143c0b2a50 con 0x7f143c0ca0e0 2023-10-30T12:43:09.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.501+0000 7f1432ffd640 1 -- 172.21.15.12:0/2526121949 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f14380ba7a0 con 0x7f143c0ca0e0 2023-10-30T12:43:09.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.501+0000 7f1432ffd640 1 --2- 172.21.15.12:0/2526121949 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f14200664a0 0x7f1420068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:09.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.501+0000 7f1432ffd640 1 -- 172.21.15.12:0/2526121949 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6025+0+0 (secure 0 0 0) 0x7f1438121ca0 con 0x7f143c0ca0e0 2023-10-30T12:43:09.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.501+0000 7f1441d90640 1 --2- 172.21.15.12:0/2526121949 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f14200664a0 0x7f1420068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:09.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.501+0000 7f1441d90640 1 --2- 172.21.15.12:0/2526121949 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f14200664a0 0x7f1420068960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f143c151710 tx=0x7f142c007cd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:09.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.505+0000 7f1432ffd640 1 -- 172.21.15.12:0/2526121949 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f14380eeeb0 con 0x7f143c0ca0e0 2023-10-30T12:43:09.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.701+0000 7f14447f5640 1 -- 172.21.15.12:0/2526121949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f143c04e170 con 0x7f143c0ca0e0 2023-10-30T12:43:09.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.701+0000 7f1432ffd640 1 -- 172.21.15.12:0/2526121949 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+6528 (secure 0 0 0) 0x7f14380b5020 con 0x7f143c0ca0e0 2023-10-30T12:43:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:09.708 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:42:12.327346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:09.709 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 34 2023-10-30T12:43:09.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.705+0000 7f14447f5640 1 -- 172.21.15.12:0/2526121949 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f14200664a0 msgr2=0x7f1420068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:09.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.705+0000 7f14447f5640 1 --2- 172.21.15.12:0/2526121949 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f14200664a0 0x7f1420068960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f143c151710 tx=0x7f142c007cd0 comp rx=0 tx=0).stop 2023-10-30T12:43:09.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.705+0000 7f14447f5640 1 -- 172.21.15.12:0/2526121949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f143c0ca0e0 msgr2=0x7f143c1e3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:09.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.705+0000 7f14447f5640 1 --2- 172.21.15.12:0/2526121949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f143c0ca0e0 0x7f143c1e3250 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f143801d320 tx=0x7f143801ba10 comp rx=0 tx=0).stop 2023-10-30T12:43:09.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.705+0000 7f14447f5640 1 -- 172.21.15.12:0/2526121949 shutdown_connections 2023-10-30T12:43:09.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.705+0000 7f14447f5640 1 --2- 172.21.15.12:0/2526121949 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f14200664a0 0x7f1420068960 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:09.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.705+0000 7f14447f5640 1 --2- 172.21.15.12:0/2526121949 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f143c1538a0 0x7f143c0c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:09.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.705+0000 7f14447f5640 1 --2- 172.21.15.12:0/2526121949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f143c0cf160 0x7f143c1e3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:09.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.705+0000 7f14447f5640 1 --2- 172.21.15.12:0/2526121949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f143c0ca0e0 0x7f143c1e3250 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:09.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.705+0000 7f14447f5640 1 -- 172.21.15.12:0/2526121949 >> 172.21.15.12:0/2526121949 conn(0x7f143c06bfe0 msgr2=0x7f143c151740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:09.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.705+0000 7f14447f5640 1 -- 172.21.15.12:0/2526121949 shutdown_connections 2023-10-30T12:43:09.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:09.705+0000 7f14447f5640 1 -- 172.21.15.12:0/2526121949 wait complete. 2023-10-30T12:43:09.729 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:43:11.730 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:11.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.877+0000 7f3266ad6640 1 Processor -- start 2023-10-30T12:43:11.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.877+0000 7f3266ad6640 1 -- start start 2023-10-30T12:43:11.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f3266ad6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32600cf160 0x7f32600cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:11.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f3266ad6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32600cb800 con 0x7f326015d6f0 2023-10-30T12:43:11.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f3266ad6640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32600cb330 con 0x7f32601538a0 2023-10-30T12:43:11.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f3266ad6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32600cad70 con 0x7f32600cf160 2023-10-30T12:43:11.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325ffff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f326015d6f0 0x7f326015a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37770/0 (socket says 172.21.15.12:37770) 2023-10-30T12:43:11.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325ffff640 1 -- 172.21.15.12:0/406312809 learned_addr learned my addr 172.21.15.12:0/406312809 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:43:11.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f3265073640 1 --2- 172.21.15.12:0/406312809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32600cf160 0x7f32600cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:11.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325f7fe640 1 -- 172.21.15.12:0/406312809 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421584311 0 0) 0x7f32600cb800 con 0x7f326015d6f0 2023-10-30T12:43:11.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325f7fe640 1 -- 172.21.15.12:0/406312809 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3250003530 con 0x7f326015d6f0 2023-10-30T12:43:11.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325f7fe640 1 -- 172.21.15.12:0/406312809 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 505784342 0 0) 0x7f3250003530 con 0x7f326015d6f0 2023-10-30T12:43:11.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325f7fe640 1 -- 172.21.15.12:0/406312809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32600cf160 msgr2=0x7f32600cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:11.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325f7fe640 1 --2- 172.21.15.12:0/406312809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32600cf160 0x7f32600cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:11.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325f7fe640 1 -- 172.21.15.12:0/406312809 >> v1:172.21.15.154:6789/0 conn(0x7f32601538a0 legacy=0x7f32600ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:11.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325f7fe640 1 -- 172.21.15.12:0/406312809 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600cb050 con 0x7f326015d6f0 2023-10-30T12:43:11.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325f7fe640 1 -- 172.21.15.12:0/406312809 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f324c001680 con 0x7f326015d6f0 2023-10-30T12:43:11.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325f7fe640 1 -- 172.21.15.12:0/406312809 >> v1:172.21.15.12:6789/0 conn(0x7f326015d6f0 legacy=0x7f326015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:11.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325f7fe640 1 --2- 172.21.15.12:0/406312809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32600cf160 0x7f32500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:11.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325f7fe640 1 --2- 172.21.15.12:0/406312809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3250003ef0 0x7f32500047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:11.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325f7fe640 1 --2- 172.21.15.12:0/406312809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3250004d10 0x7f3250005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:11.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325f7fe640 1 -- 172.21.15.12:0/406312809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32600cad70 con 0x7f3250003ef0 2023-10-30T12:43:11.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325f7fe640 1 -- 172.21.15.12:0/406312809 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325000b9f0 con 0x7f3250004d10 2023-10-30T12:43:11.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325f7fe640 1 -- 172.21.15.12:0/406312809 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325000bbb0 con 0x7f32600cf160 2023-10-30T12:43:11.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325ffff640 1 --2- 172.21.15.12:0/406312809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3250003ef0 0x7f32500047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:11.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f3264872640 1 --2- 172.21.15.12:0/406312809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32600cf160 0x7f32500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:11.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f3265073640 1 --2- 172.21.15.12:0/406312809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3250004d10 0x7f3250005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:11.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325ffff640 1 -- 172.21.15.12:0/406312809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32600cf160 msgr2=0x7f32500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:11.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325ffff640 1 --2- 172.21.15.12:0/406312809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32600cf160 0x7f32500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:11.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325ffff640 1 -- 172.21.15.12:0/406312809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3250004d10 msgr2=0x7f3250005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:11.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325ffff640 1 --2- 172.21.15.12:0/406312809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3250004d10 0x7f3250005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:11.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f325ffff640 1 -- 172.21.15.12:0/406312809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3250003530 con 0x7f3250003ef0 2023-10-30T12:43:11.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.881+0000 7f3264872640 1 --2- 172.21.15.12:0/406312809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32600cf160 0x7f32500039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:43:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.885+0000 7f325ffff640 1 --2- 172.21.15.12:0/406312809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3250003ef0 0x7f32500047d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f324c00c190 tx=0x7f324c01f250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d590b58448bab15b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.885+0000 7f325f7fe640 1 -- 172.21.15.12:0/406312809 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c0a8e70 con 0x7f3250003ef0 2023-10-30T12:43:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.885+0000 7f325f7fe640 1 -- 172.21.15.12:0/406312809 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324c0058e0 con 0x7f3250003ef0 2023-10-30T12:43:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.885+0000 7f325f7fe640 1 -- 172.21.15.12:0/406312809 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c005cf0 con 0x7f3250003ef0 2023-10-30T12:43:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.885+0000 7f3266ad6640 1 -- 172.21.15.12:0/406312809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3250003ef0 msgr2=0x7f32500047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.885+0000 7f3266ad6640 1 --2- 172.21.15.12:0/406312809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3250003ef0 0x7f32500047d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f324c00c190 tx=0x7f324c01f250 comp rx=0 tx=0).stop 2023-10-30T12:43:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.885+0000 7f3265073640 1 -- 172.21.15.12:0/406312809 reap_dead start 2023-10-30T12:43:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.885+0000 7f3266ad6640 1 -- 172.21.15.12:0/406312809 shutdown_connections 2023-10-30T12:43:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.885+0000 7f3266ad6640 1 -- 172.21.15.12:0/406312809 >> 172.21.15.12:0/406312809 conn(0x7f326006bfe0 msgr2=0x7f326006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.885+0000 7f3266ad6640 1 -- 172.21.15.12:0/406312809 shutdown_connections 2023-10-30T12:43:11.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.885+0000 7f3266ad6640 1 -- 172.21.15.12:0/406312809 wait complete. 2023-10-30T12:43:11.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3266ad6640 1 Processor -- start 2023-10-30T12:43:11.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3266ad6640 1 -- start start 2023-10-30T12:43:11.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3266ad6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3250003ef0 0x7f32600ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:11.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3266ad6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32600c64d0 0x7f326014f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:11.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3266ad6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f326014f570 0x7f32601f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:11.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3266ad6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326014eba0 con 0x7f326014f570 2023-10-30T12:43:11.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3266ad6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32600c6c50 con 0x7f3250003ef0 2023-10-30T12:43:11.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3266ad6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326014f9f0 con 0x7f32600c64d0 2023-10-30T12:43:11.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3265073640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f326014f570 0x7f32601f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:11.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3265073640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f326014f570 0x7f32601f22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45976/0 (socket says 172.21.15.12:45976) 2023-10-30T12:43:11.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3265073640 1 -- 172.21.15.12:0/3532085441 learned_addr learned my addr 172.21.15.12:0/3532085441 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:11.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3264872640 1 --2- 172.21.15.12:0/3532085441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3250003ef0 0x7f32600ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:11.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f325ffff640 1 --2- 172.21.15.12:0/3532085441 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32600c64d0 0x7f326014f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:11.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3265073640 1 -- 172.21.15.12:0/3532085441 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32600c64d0 msgr2=0x7f326014f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:11.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3265073640 1 --2- 172.21.15.12:0/3532085441 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32600c64d0 0x7f326014f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3265073640 1 -- 172.21.15.12:0/3532085441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3250003ef0 msgr2=0x7f32600ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3265073640 1 --2- 172.21.15.12:0/3532085441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3250003ef0 0x7f32600ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3265073640 1 -- 172.21.15.12:0/3532085441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600ceee0 con 0x7f326014f570 2023-10-30T12:43:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3264872640 1 --2- 172.21.15.12:0/3532085441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3250003ef0 0x7f32600ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:43:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3265073640 1 --2- 172.21.15.12:0/3532085441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f326014f570 0x7f32601f22b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3260072ce0 tx=0x7f32580115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:11.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f325d7fa640 1 -- 172.21.15.12:0/3532085441 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325800e070 con 0x7f326014f570 2023-10-30T12:43:11.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f325d7fa640 1 -- 172.21.15.12:0/3532085441 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3258011e40 con 0x7f326014f570 2023-10-30T12:43:11.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3266ad6640 1 -- 172.21.15.12:0/3532085441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3260159440 con 0x7f326014f570 2023-10-30T12:43:11.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f325d7fa640 1 -- 172.21.15.12:0/3532085441 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32580039a0 con 0x7f326014f570 2023-10-30T12:43:11.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3266ad6640 1 -- 172.21.15.12:0/3532085441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32600c8640 con 0x7f326014f570 2023-10-30T12:43:11.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.889+0000 7f3266ad6640 1 -- 172.21.15.12:0/3532085441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32600c81c0 con 0x7f326014f570 2023-10-30T12:43:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.893+0000 7f325d7fa640 1 -- 172.21.15.12:0/3532085441 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3258003b20 con 0x7f326014f570 2023-10-30T12:43:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.893+0000 7f325d7fa640 1 --2- 172.21.15.12:0/3532085441 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f32440664d0 0x7f3244068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.893+0000 7f325d7fa640 1 -- 172.21.15.12:0/3532085441 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6025+0+0 (secure 0 0 0) 0x7f3258084620 con 0x7f326014f570 2023-10-30T12:43:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.893+0000 7f3264872640 1 --2- 172.21.15.12:0/3532085441 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f32440664d0 0x7f3244068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.893+0000 7f3264872640 1 --2- 172.21.15.12:0/3532085441 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f32440664d0 0x7f3244068990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3254005540 tx=0x7f3254009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:11.897+0000 7f325d7fa640 1 -- 172.21.15.12:0/3532085441 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f32600c81c0 con 0x7f326014f570 2023-10-30T12:43:12.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:12.093+0000 7f3266ad6640 1 -- 172.21.15.12:0/3532085441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32600c81c0 con 0x7f326014f570 2023-10-30T12:43:12.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:12.093+0000 7f325d7fa640 1 -- 172.21.15.12:0/3532085441 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+6528 (secure 0 0 0) 0x7f32600c81c0 con 0x7f326014f570 2023-10-30T12:43:12.099 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:12.099 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:42:12.327346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":4282,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.154:6839/1517672481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1517672481},{"type":"v1","addr":"172.21.15.154:6839","nonce":1517672481}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:12.100 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 34 2023-10-30T12:43:12.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:12.093+0000 7f3266ad6640 1 -- 172.21.15.12:0/3532085441 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f32440664d0 msgr2=0x7f3244068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:12.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:12.093+0000 7f3266ad6640 1 --2- 172.21.15.12:0/3532085441 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f32440664d0 0x7f3244068990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3254005540 tx=0x7f3254009040 comp rx=0 tx=0).stop 2023-10-30T12:43:12.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:12.093+0000 7f3266ad6640 1 -- 172.21.15.12:0/3532085441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f326014f570 msgr2=0x7f32601f22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:12.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:12.093+0000 7f3266ad6640 1 --2- 172.21.15.12:0/3532085441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f326014f570 0x7f32601f22b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3260072ce0 tx=0x7f32580115a0 comp rx=0 tx=0).stop 2023-10-30T12:43:12.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:12.093+0000 7f3266ad6640 1 -- 172.21.15.12:0/3532085441 shutdown_connections 2023-10-30T12:43:12.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:12.093+0000 7f3266ad6640 1 --2- 172.21.15.12:0/3532085441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f326014f570 0x7f32601f22b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:12.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:12.093+0000 7f3266ad6640 1 --2- 172.21.15.12:0/3532085441 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f32440664d0 0x7f3244068990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:12.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:12.093+0000 7f3266ad6640 1 --2- 172.21.15.12:0/3532085441 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32600c64d0 0x7f326014f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:12.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:12.093+0000 7f3266ad6640 1 --2- 172.21.15.12:0/3532085441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3250003ef0 0x7f32600ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:12.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:12.093+0000 7f3266ad6640 1 -- 172.21.15.12:0/3532085441 >> 172.21.15.12:0/3532085441 conn(0x7f326006bfe0 msgr2=0x7f326015afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:12.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:12.093+0000 7f3266ad6640 1 -- 172.21.15.12:0/3532085441 shutdown_connections 2023-10-30T12:43:12.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:12.093+0000 7f3266ad6640 1 -- 172.21.15.12:0/3532085441 wait complete. 2023-10-30T12:43:12.120 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:43:14.121 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:14.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.273+0000 7f90a0ba7640 1 Processor -- start 2023-10-30T12:43:14.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.273+0000 7f90a0ba7640 1 -- start start 2023-10-30T12:43:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.273+0000 7f90a0ba7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909c0717b0 0x7f909c15ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.273+0000 7f90a0ba7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f909c071b90 0x7f909c1513c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.273+0000 7f90a0ba7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909c15b060 con 0x7f909c0c6290 2023-10-30T12:43:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.273+0000 7f90a0ba7640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f909c157530 con 0x7f909c071b90 2023-10-30T12:43:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.273+0000 7f90a0ba7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c0c4ab0 con 0x7f909c0717b0 2023-10-30T12:43:14.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.273+0000 7f909b7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909c0717b0 0x7f909c15ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.273+0000 7f909affd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f909c071b90 0x7f909c1513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.273+0000 7f909bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f909c0c6290 0x7f909c069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:40768/0 (socket says 172.21.15.12:40768) 2023-10-30T12:43:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.273+0000 7f909bfff640 1 -- 172.21.15.12:0/2164092120 learned_addr learned my addr 172.21.15.12:0/2164092120 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:43:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.277+0000 7f909b7fe640 1 -- 172.21.15.12:0/2164092120 >> v2:172.21.15.154:3300/0 conn(0x7f909c071b90 msgr2=0x7f909c1513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.277+0000 7f909b7fe640 1 --2- 172.21.15.12:0/2164092120 >> v2:172.21.15.154:3300/0 conn(0x7f909c071b90 0x7f909c1513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.277+0000 7f909b7fe640 1 -- 172.21.15.12:0/2164092120 >> v1:172.21.15.154:6789/0 conn(0x7f909c0c6290 legacy=0x7f909c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.277+0000 7f909b7fe640 1 -- 172.21.15.12:0/2164092120 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c0c4d90 con 0x7f909c0717b0 2023-10-30T12:43:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.277+0000 7f909affd640 1 --2- 172.21.15.12:0/2164092120 >> v2:172.21.15.154:3300/0 conn(0x7f909c071b90 0x7f909c1513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:43:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.277+0000 7f909b7fe640 1 --2- 172.21.15.12:0/2164092120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909c0717b0 0x7f909c15ab20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f908c0097a0 tx=0x7f908c01ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87805ff4dc3830b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.277+0000 7f909a7fc640 1 -- 172.21.15.12:0/2164092120 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c0a8950 con 0x7f909c0717b0 2023-10-30T12:43:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.277+0000 7f909a7fc640 1 -- 172.21.15.12:0/2164092120 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908c0a8ab0 con 0x7f909c0717b0 2023-10-30T12:43:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.277+0000 7f909a7fc640 1 -- 172.21.15.12:0/2164092120 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c0a8e40 con 0x7f909c0717b0 2023-10-30T12:43:14.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 -- 172.21.15.12:0/2164092120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909c0717b0 msgr2=0x7f909c15ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:14.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 --2- 172.21.15.12:0/2164092120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909c0717b0 0x7f909c15ab20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f908c0097a0 tx=0x7f908c01ec50 comp rx=0 tx=0).stop 2023-10-30T12:43:14.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 -- 172.21.15.12:0/2164092120 shutdown_connections 2023-10-30T12:43:14.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 --2- 172.21.15.12:0/2164092120 >> v2:172.21.15.154:3300/0 conn(0x7f909c071b90 0x7f909c1513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 --2- 172.21.15.12:0/2164092120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909c0717b0 0x7f909c15ab20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 -- 172.21.15.12:0/2164092120 >> 172.21.15.12:0/2164092120 conn(0x7f909c0c75d0 msgr2=0x7f909c0c79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 -- 172.21.15.12:0/2164092120 shutdown_connections 2023-10-30T12:43:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 -- 172.21.15.12:0/2164092120 wait complete. 2023-10-30T12:43:14.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 Processor -- start 2023-10-30T12:43:14.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 -- start start 2023-10-30T12:43:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f909c0717b0 0x7f909c14ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909c071b90 0x7f909c15a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f909c0c6290 0x7f909c1f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c1f0570 con 0x7f909c0c6290 2023-10-30T12:43:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c149c50 con 0x7f909c0717b0 2023-10-30T12:43:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f90a0ba7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c1f0240 con 0x7f909c071b90 2023-10-30T12:43:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f909bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f909c0c6290 0x7f909c1f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f909bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f909c0c6290 0x7f909c1f0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45982/0 (socket says 172.21.15.12:45982) 2023-10-30T12:43:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f909bfff640 1 -- 172.21.15.12:0/126058535 learned_addr learned my addr 172.21.15.12:0/126058535 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.281+0000 7f909b7fe640 1 --2- 172.21.15.12:0/126058535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f909c0717b0 0x7f909c14ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:14.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.285+0000 7f909affd640 1 --2- 172.21.15.12:0/126058535 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909c071b90 0x7f909c15a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.285+0000 7f909bfff640 1 -- 172.21.15.12:0/126058535 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909c071b90 msgr2=0x7f909c15a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.285+0000 7f909bfff640 1 --2- 172.21.15.12:0/126058535 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909c071b90 0x7f909c15a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.285+0000 7f909bfff640 1 -- 172.21.15.12:0/126058535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f909c0717b0 msgr2=0x7f909c14ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:14.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.285+0000 7f909bfff640 1 --2- 172.21.15.12:0/126058535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f909c0717b0 0x7f909c14ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.285+0000 7f909bfff640 1 -- 172.21.15.12:0/126058535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9090009420 con 0x7f909c0c6290 2023-10-30T12:43:14.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.285+0000 7f909bfff640 1 --2- 172.21.15.12:0/126058535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f909c0c6290 0x7f909c1f0a00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f909400ca80 tx=0x7f909400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:14.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.285+0000 7f9083fff640 1 -- 172.21.15.12:0/126058535 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400ac30 con 0x7f909c0c6290 2023-10-30T12:43:14.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.285+0000 7f9083fff640 1 -- 172.21.15.12:0/126058535 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909400ad90 con 0x7f909c0c6290 2023-10-30T12:43:14.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.285+0000 7f9083fff640 1 -- 172.21.15.12:0/126058535 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094012600 con 0x7f909c0c6290 2023-10-30T12:43:14.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.285+0000 7f90a0ba7640 1 -- 172.21.15.12:0/126058535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908c0093b0 con 0x7f909c0c6290 2023-10-30T12:43:14.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.285+0000 7f90a0ba7640 1 -- 172.21.15.12:0/126058535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c149ea0 con 0x7f909c0c6290 2023-10-30T12:43:14.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.285+0000 7f90a0ba7640 1 -- 172.21.15.12:0/126058535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909c1f0720 con 0x7f909c0c6290 2023-10-30T12:43:14.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.285+0000 7f9083fff640 1 -- 172.21.15.12:0/126058535 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9094007070 con 0x7f909c0c6290 2023-10-30T12:43:14.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.285+0000 7f9083fff640 1 --2- 172.21.15.12:0/126058535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f90880664a0 0x7f9088068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:14.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.289+0000 7f9083fff640 1 -- 172.21.15.12:0/126058535 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f9094083ea0 con 0x7f909c0c6290 2023-10-30T12:43:14.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.289+0000 7f909b7fe640 1 --2- 172.21.15.12:0/126058535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f90880664a0 0x7f9088068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:14.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.289+0000 7f909b7fe640 1 --2- 172.21.15.12:0/126058535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f90880664a0 0x7f9088068960 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f908c0b2000 tx=0x7f908c0aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:14.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.293+0000 7f9083fff640 1 -- 172.21.15.12:0/126058535 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f909c1f0720 con 0x7f909c0c6290 2023-10-30T12:43:14.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.481+0000 7f90a0ba7640 1 -- 172.21.15.12:0/126058535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f909c1f0720 con 0x7f909c0c6290 2023-10-30T12:43:14.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.481+0000 7f9083fff640 1 -- 172.21.15.12:0/126058535 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5731 (secure 0 0 0) 0x7f909c1f0720 con 0x7f909c0c6290 2023-10-30T12:43:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:14.490 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:13.483945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 36 2023-10-30T12:43:14.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.485+0000 7f90a0ba7640 1 -- 172.21.15.12:0/126058535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f90880664a0 msgr2=0x7f9088068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:14.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.485+0000 7f90a0ba7640 1 --2- 172.21.15.12:0/126058535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f90880664a0 0x7f9088068960 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f908c0b2000 tx=0x7f908c0aa070 comp rx=0 tx=0).stop 2023-10-30T12:43:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.485+0000 7f90a0ba7640 1 -- 172.21.15.12:0/126058535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f909c0c6290 msgr2=0x7f909c1f0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.485+0000 7f90a0ba7640 1 --2- 172.21.15.12:0/126058535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f909c0c6290 0x7f909c1f0a00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f909400ca80 tx=0x7f909400cf40 comp rx=0 tx=0).stop 2023-10-30T12:43:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.485+0000 7f90a0ba7640 1 -- 172.21.15.12:0/126058535 shutdown_connections 2023-10-30T12:43:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.485+0000 7f90a0ba7640 1 --2- 172.21.15.12:0/126058535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f909c0c6290 0x7f909c1f0a00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.485+0000 7f90a0ba7640 1 --2- 172.21.15.12:0/126058535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f90880664a0 0x7f9088068960 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.485+0000 7f90a0ba7640 1 --2- 172.21.15.12:0/126058535 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f909c071b90 0x7f909c15a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.485+0000 7f90a0ba7640 1 --2- 172.21.15.12:0/126058535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f909c0717b0 0x7f909c14ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.485+0000 7f90a0ba7640 1 -- 172.21.15.12:0/126058535 >> 172.21.15.12:0/126058535 conn(0x7f909c0c75d0 msgr2=0x7f909c1553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:14.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.485+0000 7f90a0ba7640 1 -- 172.21.15.12:0/126058535 shutdown_connections 2023-10-30T12:43:14.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.485+0000 7f90a0ba7640 1 -- 172.21.15.12:0/126058535 wait complete. 2023-10-30T12:43:14.512 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2023-10-30T12:43:14.512 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:43:14.512 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:14.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.649+0000 7f57cf794640 1 Processor -- start 2023-10-30T12:43:14.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.649+0000 7f57cf794640 1 -- start start 2023-10-30T12:43:14.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.649+0000 7f57cf794640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57c8153d70 0x7f57c806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:14.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.649+0000 7f57cf794640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f57c806e470 0x7f57c80cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:14.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.649+0000 7f57cf794640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c806afa0 con 0x7f57c80c8490 2023-10-30T12:43:14.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.649+0000 7f57cf794640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c806d780 con 0x7f57c806e470 2023-10-30T12:43:14.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.649+0000 7f57cf794640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c8154150 con 0x7f57c8153d70 2023-10-30T12:43:14.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.649+0000 7f57cd530640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57c8153d70 0x7f57c806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:14.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.649+0000 7f57ccd2f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f57c806e470 0x7f57c80cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:14.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.649+0000 7f57cd530640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57c8153d70 0x7f57c806ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:42192/0 (socket says 172.21.15.12:42192) 2023-10-30T12:43:14.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.649+0000 7f57cd530640 1 -- 172.21.15.12:0/3186890302 learned_addr learned my addr 172.21.15.12:0/3186890302 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:14.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.653+0000 7f57ccd2f640 1 -- 172.21.15.12:0/3186890302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57c8153d70 msgr2=0x7f57c806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:14.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.653+0000 7f57ccd2f640 1 --2- 172.21.15.12:0/3186890302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57c8153d70 0x7f57c806ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.653+0000 7f57ccd2f640 1 -- 172.21.15.12:0/3186890302 >> v1:172.21.15.154:6789/0 conn(0x7f57c80c8490 legacy=0x7f57c80c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:14.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.653+0000 7f57ccd2f640 1 -- 172.21.15.12:0/3186890302 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c806e030 con 0x7f57c806e470 2023-10-30T12:43:14.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.653+0000 7f57cd530640 1 --2- 172.21.15.12:0/3186890302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57c8153d70 0x7f57c806ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:43:14.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.653+0000 7f57ccd2f640 1 --2- 172.21.15.12:0/3186890302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57c806e470 0x7f57c80cce00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f57b80097c0 tx=0x7f57b801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391fec93d94c66a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:14.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.653+0000 7f57bffff640 1 -- 172.21.15.12:0/3186890302 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80ab650 con 0x7f57c806e470 2023-10-30T12:43:14.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.653+0000 7f57bffff640 1 -- 172.21.15.12:0/3186890302 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b80ab7b0 con 0x7f57c806e470 2023-10-30T12:43:14.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.653+0000 7f57bffff640 1 -- 172.21.15.12:0/3186890302 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80abb40 con 0x7f57c806e470 2023-10-30T12:43:14.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 -- 172.21.15.12:0/3186890302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57c806e470 msgr2=0x7f57c80cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:14.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 --2- 172.21.15.12:0/3186890302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57c806e470 0x7f57c80cce00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f57b80097c0 tx=0x7f57b801c450 comp rx=0 tx=0).stop 2023-10-30T12:43:14.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 -- 172.21.15.12:0/3186890302 shutdown_connections 2023-10-30T12:43:14.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 --2- 172.21.15.12:0/3186890302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57c806e470 0x7f57c80cce00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 --2- 172.21.15.12:0/3186890302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57c8153d70 0x7f57c806ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 -- 172.21.15.12:0/3186890302 >> 172.21.15.12:0/3186890302 conn(0x7f57c8071ea0 msgr2=0x7f57c8065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:14.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 -- 172.21.15.12:0/3186890302 shutdown_connections 2023-10-30T12:43:14.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 -- 172.21.15.12:0/3186890302 wait complete. 2023-10-30T12:43:14.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 Processor -- start 2023-10-30T12:43:14.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 -- start start 2023-10-30T12:43:14.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57c806e470 0x7f57c8154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:14.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57c80c8490 0x7f57c8155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:14.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8153d70 0x7f57c81f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:14.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c80b0210 con 0x7f57c8153d70 2023-10-30T12:43:14.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c81542f0 con 0x7f57c80c8490 2023-10-30T12:43:14.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cf794640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c8154870 con 0x7f57c806e470 2023-10-30T12:43:14.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cdd31640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8153d70 0x7f57c81f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:14.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cdd31640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8153d70 0x7f57c81f60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45984/0 (socket says 172.21.15.12:45984) 2023-10-30T12:43:14.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cdd31640 1 -- 172.21.15.12:0/2477550792 learned_addr learned my addr 172.21.15.12:0/2477550792 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:14.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57cd530640 1 --2- 172.21.15.12:0/2477550792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57c806e470 0x7f57c8154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:14.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.657+0000 7f57ccd2f640 1 --2- 172.21.15.12:0/2477550792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57c80c8490 0x7f57c8155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:14.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57cdd31640 1 -- 172.21.15.12:0/2477550792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57c806e470 msgr2=0x7f57c8154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:14.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57cdd31640 1 --2- 172.21.15.12:0/2477550792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57c806e470 0x7f57c8154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57cdd31640 1 -- 172.21.15.12:0/2477550792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57c80c8490 msgr2=0x7f57c8155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:14.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57cdd31640 1 --2- 172.21.15.12:0/2477550792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57c80c8490 0x7f57c8155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57cdd31640 1 -- 172.21.15.12:0/2477550792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c40094d0 con 0x7f57c8153d70 2023-10-30T12:43:14.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57cdd31640 1 --2- 172.21.15.12:0/2477550792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8153d70 0x7f57c81f60e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f57c000c770 tx=0x7f57c000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:14.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57bdffb640 1 -- 172.21.15.12:0/2477550792 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0003b80 con 0x7f57c8153d70 2023-10-30T12:43:14.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57bdffb640 1 -- 172.21.15.12:0/2477550792 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c0003ce0 con 0x7f57c8153d70 2023-10-30T12:43:14.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57bdffb640 1 -- 172.21.15.12:0/2477550792 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c000aa90 con 0x7f57c8153d70 2023-10-30T12:43:14.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57cf794640 1 -- 172.21.15.12:0/2477550792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b8009420 con 0x7f57c8153d70 2023-10-30T12:43:14.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57cf794640 1 -- 172.21.15.12:0/2477550792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c8156cb0 con 0x7f57c8153d70 2023-10-30T12:43:14.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57cf794640 1 -- 172.21.15.12:0/2477550792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c804e170 con 0x7f57c8153d70 2023-10-30T12:43:14.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57bdffb640 1 -- 172.21.15.12:0/2477550792 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f57c000abf0 con 0x7f57c8153d70 2023-10-30T12:43:14.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57bdffb640 1 --2- 172.21.15.12:0/2477550792 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f57ac0664a0 0x7f57ac068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:14.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57bdffb640 1 -- 172.21.15.12:0/2477550792 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f57c0083db0 con 0x7f57c8153d70 2023-10-30T12:43:14.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.661+0000 7f57cd530640 1 --2- 172.21.15.12:0/2477550792 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f57ac0664a0 0x7f57ac068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:14.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.665+0000 7f57cd530640 1 --2- 172.21.15.12:0/2477550792 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f57ac0664a0 0x7f57ac068960 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f57c4006cb0 tx=0x7f57c4009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:14.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.665+0000 7f57bdffb640 1 -- 172.21.15.12:0/2477550792 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f57c0088070 con 0x7f57c8153d70 2023-10-30T12:43:14.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.857+0000 7f57cf794640 1 -- 172.21.15.12:0/2477550792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57c81549b0 con 0x7f57c8153d70 2023-10-30T12:43:14.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.857+0000 7f57bdffb640 1 -- 172.21.15.12:0/2477550792 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5731 (secure 0 0 0) 0x7f57c00510c0 con 0x7f57c8153d70 2023-10-30T12:43:14.865 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:14.865 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:13.483945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:14.865 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 36 2023-10-30T12:43:14.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.861+0000 7f57cf794640 1 -- 172.21.15.12:0/2477550792 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f57ac0664a0 msgr2=0x7f57ac068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:14.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.861+0000 7f57cf794640 1 --2- 172.21.15.12:0/2477550792 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f57ac0664a0 0x7f57ac068960 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f57c4006cb0 tx=0x7f57c4009240 comp rx=0 tx=0).stop 2023-10-30T12:43:14.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.861+0000 7f57cf794640 1 -- 172.21.15.12:0/2477550792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8153d70 msgr2=0x7f57c81f60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:14.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.861+0000 7f57cf794640 1 --2- 172.21.15.12:0/2477550792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8153d70 0x7f57c81f60e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f57c000c770 tx=0x7f57c000cc30 comp rx=0 tx=0).stop 2023-10-30T12:43:14.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.861+0000 7f57cf794640 1 -- 172.21.15.12:0/2477550792 shutdown_connections 2023-10-30T12:43:14.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.861+0000 7f57cf794640 1 --2- 172.21.15.12:0/2477550792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8153d70 0x7f57c81f60e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.861+0000 7f57cf794640 1 --2- 172.21.15.12:0/2477550792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57c80c8490 0x7f57c8155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.861+0000 7f57cf794640 1 --2- 172.21.15.12:0/2477550792 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f57ac0664a0 0x7f57ac068960 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.861+0000 7f57cf794640 1 --2- 172.21.15.12:0/2477550792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57c806e470 0x7f57c8154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:14.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.861+0000 7f57cf794640 1 -- 172.21.15.12:0/2477550792 >> 172.21.15.12:0/2477550792 conn(0x7f57c8071ea0 msgr2=0x7f57c80c7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:14.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.861+0000 7f57cf794640 1 -- 172.21.15.12:0/2477550792 shutdown_connections 2023-10-30T12:43:14.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:14.861+0000 7f57cf794640 1 -- 172.21.15.12:0/2477550792 wait complete. 2023-10-30T12:43:14.886 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 36, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 36, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4285": { "addr": "172.21.15.12:6836/291392212", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 291392212, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 291392212, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138322906710015, "flags": 0, "gid": 4285, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 6 }, "gid_4971": { "addr": "172.21.15.12:6837/3124135109", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 3124135109, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 3124135109, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 4971, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 44 }, "gid_5124": { "addr": "172.21.15.12:6835/559649789", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 559649789, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 559649789, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5124, "incarnation": 36, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 48, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:43:13.483945+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4285, "mds_1": 5124, "mds_2": 4971 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:43:16.888 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:17.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdc55f2640 1 Processor -- start 2023-10-30T12:43:17.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdc55f2640 1 -- start start 2023-10-30T12:43:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdc55f2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcdc01538f0 0x7fcdc0159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdc55f2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcdc015cd50 0x7fcdc00c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdc55f2640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc015d300 con 0x7fcdc00cefa0 2023-10-30T12:43:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdc55f2640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc015d5e0 con 0x7fcdc015cd50 2023-10-30T12:43:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdc55f2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc015d8c0 con 0x7fcdc01538f0 2023-10-30T12:43:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdbf7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcdc015cd50 0x7fcdc00c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdc4df1640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fcdc00cefa0 0x7fcdc00cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37780/0 (socket says 172.21.15.12:37780) 2023-10-30T12:43:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdc4df1640 1 -- 172.21.15.12:0/2464265156 learned_addr learned my addr 172.21.15.12:0/2464265156 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:43:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdbf7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcdc015cd50 0x7fcdc00c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45998/0 (socket says 172.21.15.12:45998) 2023-10-30T12:43:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdbffff640 1 --2- 172.21.15.12:0/2464265156 >> v2:172.21.15.154:3300/0 conn(0x7fcdc01538f0 0x7fcdc0159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:17.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdbf7fe640 1 -- 172.21.15.12:0/2464265156 >> v2:172.21.15.154:3300/0 conn(0x7fcdc01538f0 msgr2=0x7fcdc0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:17.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdbf7fe640 1 --2- 172.21.15.12:0/2464265156 >> v2:172.21.15.154:3300/0 conn(0x7fcdc01538f0 0x7fcdc0159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:17.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdbf7fe640 1 -- 172.21.15.12:0/2464265156 >> v1:172.21.15.12:6789/0 conn(0x7fcdc00cefa0 legacy=0x7fcdc00cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:17.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdbeffd640 1 -- 172.21.15.12:0/2464265156 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71668223 0 0) 0x7fcdc015d300 con 0x7fcdc00cefa0 2023-10-30T12:43:17.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdbf7fe640 1 -- 172.21.15.12:0/2464265156 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc015dba0 con 0x7fcdc015cd50 2023-10-30T12:43:17.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.037+0000 7fcdbffff640 1 --2- 172.21.15.12:0/2464265156 >> v2:172.21.15.154:3300/0 conn(0x7fcdc01538f0 0x7fcdc0159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:43:17.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.041+0000 7fcdbf7fe640 1 --2- 172.21.15.12:0/2464265156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdc015cd50 0x7fcdc00c9f90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00c9cb0 tx=0x7fcdb401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1222540ccf21cbd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:17.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.041+0000 7fcdbeffd640 1 -- 172.21.15.12:0/2464265156 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb40a8950 con 0x7fcdc015cd50 2023-10-30T12:43:17.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.041+0000 7fcdbeffd640 1 -- 172.21.15.12:0/2464265156 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb40a8ab0 con 0x7fcdc015cd50 2023-10-30T12:43:17.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.041+0000 7fcdbeffd640 1 -- 172.21.15.12:0/2464265156 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb40a8e40 con 0x7fcdc015cd50 2023-10-30T12:43:17.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.041+0000 7fcdc55f2640 1 -- 172.21.15.12:0/2464265156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdc015cd50 msgr2=0x7fcdc00c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:17.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.041+0000 7fcdc55f2640 1 --2- 172.21.15.12:0/2464265156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdc015cd50 0x7fcdc00c9f90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00c9cb0 tx=0x7fcdb401ec50 comp rx=0 tx=0).stop 2023-10-30T12:43:17.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.041+0000 7fcdc55f2640 1 -- 172.21.15.12:0/2464265156 shutdown_connections 2023-10-30T12:43:17.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.041+0000 7fcdc55f2640 1 --2- 172.21.15.12:0/2464265156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdc015cd50 0x7fcdc00c9f90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:17.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.041+0000 7fcdc55f2640 1 --2- 172.21.15.12:0/2464265156 >> v2:172.21.15.154:3300/0 conn(0x7fcdc01538f0 0x7fcdc0159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:17.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.041+0000 7fcdc55f2640 1 -- 172.21.15.12:0/2464265156 >> 172.21.15.12:0/2464265156 conn(0x7fcdc006a640 msgr2=0x7fcdc0065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:17.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.041+0000 7fcdc55f2640 1 -- 172.21.15.12:0/2464265156 shutdown_connections 2023-10-30T12:43:17.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.041+0000 7fcdc55f2640 1 -- 172.21.15.12:0/2464265156 wait complete. 2023-10-30T12:43:17.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.041+0000 7fcdc55f2640 1 Processor -- start 2023-10-30T12:43:17.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdc55f2640 1 -- start start 2023-10-30T12:43:17.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdc55f2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdc00cefa0 0x7fcdc0145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:17.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdc55f2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdc01538f0 0x7fcdc01519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:17.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdc55f2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdc015cd50 0x7fcdc00b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:17.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdc55f2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc015c5c0 con 0x7fcdc00cefa0 2023-10-30T12:43:17.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdc55f2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc0151530 con 0x7fcdc01538f0 2023-10-30T12:43:17.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdc55f2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc00b3dd0 con 0x7fcdc015cd50 2023-10-30T12:43:17.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdbffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdc00cefa0 0x7fcdc0145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:17.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdbffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdc00cefa0 0x7fcdc0145690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46008/0 (socket says 172.21.15.12:46008) 2023-10-30T12:43:17.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdbffff640 1 -- 172.21.15.12:0/1850519462 learned_addr learned my addr 172.21.15.12:0/1850519462 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:17.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdbf7fe640 1 --2- 172.21.15.12:0/1850519462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdc01538f0 0x7fcdc01519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:17.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdc4df1640 1 --2- 172.21.15.12:0/1850519462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdc015cd50 0x7fcdc00b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:17.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdbffff640 1 -- 172.21.15.12:0/1850519462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdc015cd50 msgr2=0x7fcdc00b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:17.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdbffff640 1 --2- 172.21.15.12:0/1850519462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdc015cd50 0x7fcdc00b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:17.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdbffff640 1 -- 172.21.15.12:0/1850519462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdc01538f0 msgr2=0x7fcdc01519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:17.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdbffff640 1 --2- 172.21.15.12:0/1850519462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdc01538f0 0x7fcdc01519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:17.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdbffff640 1 -- 172.21.15.12:0/1850519462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb4009420 con 0x7fcdc00cefa0 2023-10-30T12:43:17.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdbf7fe640 1 --2- 172.21.15.12:0/1850519462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdc01538f0 0x7fcdc01519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:43:17.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdbffff640 1 --2- 172.21.15.12:0/1850519462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdc00cefa0 0x7fcdc0145690 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000e840 tx=0x7fcdb000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:17.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdbcff9640 1 -- 172.21.15.12:0/1850519462 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb000c8d0 con 0x7fcdc00cefa0 2023-10-30T12:43:17.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdbcff9640 1 -- 172.21.15.12:0/1850519462 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb000ca30 con 0x7fcdc00cefa0 2023-10-30T12:43:17.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdbcff9640 1 -- 172.21.15.12:0/1850519462 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb000cdc0 con 0x7fcdc00cefa0 2023-10-30T12:43:17.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdc55f2640 1 -- 172.21.15.12:0/1850519462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc015c810 con 0x7fcdc00cefa0 2023-10-30T12:43:17.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdc55f2640 1 -- 172.21.15.12:0/1850519462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc01511c0 con 0x7fcdc00cefa0 2023-10-30T12:43:17.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.045+0000 7fcdc55f2640 1 -- 172.21.15.12:0/1850519462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc004e170 con 0x7fcdc00cefa0 2023-10-30T12:43:17.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.049+0000 7fcdbcff9640 1 -- 172.21.15.12:0/1850519462 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcdb0013410 con 0x7fcdc00cefa0 2023-10-30T12:43:17.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.049+0000 7fcdbcff9640 1 --2- 172.21.15.12:0/1850519462 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcdac0664a0 0x7fcdac068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:17.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.049+0000 7fcdbcff9640 1 -- 172.21.15.12:0/1850519462 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fcdb0083e70 con 0x7fcdc00cefa0 2023-10-30T12:43:17.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.049+0000 7fcdbf7fe640 1 --2- 172.21.15.12:0/1850519462 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcdac0664a0 0x7fcdac068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:17.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.049+0000 7fcdbf7fe640 1 --2- 172.21.15.12:0/1850519462 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcdac0664a0 0x7fcdac068960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40b2000 tx=0x7fcdb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:17.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.053+0000 7fcdbcff9640 1 -- 172.21.15.12:0/1850519462 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcdb0051160 con 0x7fcdc00cefa0 2023-10-30T12:43:17.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.241+0000 7fcdc55f2640 1 -- 172.21.15.12:0/1850519462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdc00cfca0 con 0x7fcdc00cefa0 2023-10-30T12:43:17.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.241+0000 7fcdbcff9640 1 -- 172.21.15.12:0/1850519462 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5731 (secure 0 0 0) 0x7fcdb0054780 con 0x7fcdc00cefa0 2023-10-30T12:43:17.248 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:17.249 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:13.483945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:17.249 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 36 2023-10-30T12:43:17.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.245+0000 7fcdc55f2640 1 -- 172.21.15.12:0/1850519462 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcdac0664a0 msgr2=0x7fcdac068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:17.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.245+0000 7fcdc55f2640 1 --2- 172.21.15.12:0/1850519462 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcdac0664a0 0x7fcdac068960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40b2000 tx=0x7fcdb4008040 comp rx=0 tx=0).stop 2023-10-30T12:43:17.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.245+0000 7fcdc55f2640 1 -- 172.21.15.12:0/1850519462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdc00cefa0 msgr2=0x7fcdc0145690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:17.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.245+0000 7fcdc55f2640 1 --2- 172.21.15.12:0/1850519462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdc00cefa0 0x7fcdc0145690 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000e840 tx=0x7fcdb000ed00 comp rx=0 tx=0).stop 2023-10-30T12:43:17.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.245+0000 7fcdc55f2640 1 -- 172.21.15.12:0/1850519462 shutdown_connections 2023-10-30T12:43:17.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.245+0000 7fcdc55f2640 1 --2- 172.21.15.12:0/1850519462 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcdac0664a0 0x7fcdac068960 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:17.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.245+0000 7fcdc55f2640 1 --2- 172.21.15.12:0/1850519462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdc015cd50 0x7fcdc00b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:17.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.245+0000 7fcdc55f2640 1 --2- 172.21.15.12:0/1850519462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdc01538f0 0x7fcdc01519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:17.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.245+0000 7fcdc55f2640 1 --2- 172.21.15.12:0/1850519462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdc00cefa0 0x7fcdc0145690 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:17.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.245+0000 7fcdc55f2640 1 -- 172.21.15.12:0/1850519462 >> 172.21.15.12:0/1850519462 conn(0x7fcdc006a640 msgr2=0x7fcdc00caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:17.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.245+0000 7fcdc55f2640 1 -- 172.21.15.12:0/1850519462 shutdown_connections 2023-10-30T12:43:17.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:17.245+0000 7fcdc55f2640 1 -- 172.21.15.12:0/1850519462 wait complete. 2023-10-30T12:43:17.269 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:43:19.270 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa4665c5640 1 Processor -- start 2023-10-30T12:43:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa4665c5640 1 -- start start 2023-10-30T12:43:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa4665c5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa4601570f0 0x7fa46006ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa4665c5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa46006b3b0 con 0x7fa4600b2300 2023-10-30T12:43:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa4665c5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4600c38a0 con 0x7fa46006f370 2023-10-30T12:43:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa4665c5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4600c3b20 con 0x7fa4601570f0 2023-10-30T12:43:19.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45f7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa4600b2300 0x7fa460071b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37796/0 (socket says 172.21.15.12:37796) 2023-10-30T12:43:19.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45f7fe640 1 -- 172.21.15.12:0/1863894646 learned_addr learned my addr 172.21.15.12:0/1863894646 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:43:19.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45ffff640 1 --2- 172.21.15.12:0/1863894646 >> v2:172.21.15.154:3300/0 conn(0x7fa4601570f0 0x7fa46006ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45effd640 1 -- 172.21.15.12:0/1863894646 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3095962832 0 0) 0x7fa46006b3b0 con 0x7fa4600b2300 2023-10-30T12:43:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45effd640 1 -- 172.21.15.12:0/1863894646 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa448003530 con 0x7fa4600b2300 2023-10-30T12:43:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45effd640 1 -- 172.21.15.12:0/1863894646 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1667817902 0 0) 0x7fa448003530 con 0x7fa4600b2300 2023-10-30T12:43:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45effd640 1 -- 172.21.15.12:0/1863894646 >> v2:172.21.15.154:3300/0 conn(0x7fa4601570f0 msgr2=0x7fa46006ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45effd640 1 --2- 172.21.15.12:0/1863894646 >> v2:172.21.15.154:3300/0 conn(0x7fa4601570f0 0x7fa46006ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45effd640 1 -- 172.21.15.12:0/1863894646 >> v1:172.21.15.154:6789/0 conn(0x7fa46006f370 legacy=0x7fa4601640d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:19.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45effd640 1 -- 172.21.15.12:0/1863894646 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4600ca9f0 con 0x7fa4600b2300 2023-10-30T12:43:19.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45effd640 1 -- 172.21.15.12:0/1863894646 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fa454001680 con 0x7fa4600b2300 2023-10-30T12:43:19.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45effd640 1 -- 172.21.15.12:0/1863894646 >> v1:172.21.15.12:6789/0 conn(0x7fa4600b2300 legacy=0x7fa460071b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:19.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45effd640 1 --2- 172.21.15.12:0/1863894646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa448003980 0x7fa448003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:19.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45effd640 1 --2- 172.21.15.12:0/1863894646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4480047d0 0x7fa448004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45effd640 1 --2- 172.21.15.12:0/1863894646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa448005190 0x7fa44800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45effd640 1 -- 172.21.15.12:0/1863894646 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4600c3b20 con 0x7fa448003980 2023-10-30T12:43:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45effd640 1 -- 172.21.15.12:0/1863894646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44800c030 con 0x7fa448005190 2023-10-30T12:43:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45effd640 1 -- 172.21.15.12:0/1863894646 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44800c1f0 con 0x7fa4480047d0 2023-10-30T12:43:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45ffff640 1 --2- 172.21.15.12:0/1863894646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa448003980 0x7fa448003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa464b62640 1 --2- 172.21.15.12:0/1863894646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa448005190 0x7fa44800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45f7fe640 1 --2- 172.21.15.12:0/1863894646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4480047d0 0x7fa448004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45ffff640 1 -- 172.21.15.12:0/1863894646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4480047d0 msgr2=0x7fa448004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45ffff640 1 --2- 172.21.15.12:0/1863894646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4480047d0 0x7fa448004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45ffff640 1 -- 172.21.15.12:0/1863894646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa448005190 msgr2=0x7fa44800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45ffff640 1 --2- 172.21.15.12:0/1863894646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa448005190 0x7fa44800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa45ffff640 1 -- 172.21.15.12:0/1863894646 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa448003530 con 0x7fa448003980 2023-10-30T12:43:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa464b62640 1 --2- 172.21.15.12:0/1863894646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa448005190 0x7fa44800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:43:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.409+0000 7fa464b62640 1 -- 172.21.15.12:0/1863894646 reap_dead start 2023-10-30T12:43:19.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.413+0000 7fa45ffff640 1 --2- 172.21.15.12:0/1863894646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa448003980 0x7fa448003da0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa450009ae0 tx=0x7fa45001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425d11c3ceff77ae server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:19.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.413+0000 7fa45effd640 1 -- 172.21.15.12:0/1863894646 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4500a5510 con 0x7fa448003980 2023-10-30T12:43:19.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.413+0000 7fa45effd640 1 -- 172.21.15.12:0/1863894646 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4500a5670 con 0x7fa448003980 2023-10-30T12:43:19.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.413+0000 7fa45effd640 1 -- 172.21.15.12:0/1863894646 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4500a5a20 con 0x7fa448003980 2023-10-30T12:43:19.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.413+0000 7fa4665c5640 1 -- 172.21.15.12:0/1863894646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa448003980 msgr2=0x7fa448003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:19.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.413+0000 7fa4665c5640 1 --2- 172.21.15.12:0/1863894646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa448003980 0x7fa448003da0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa450009ae0 tx=0x7fa45001ca50 comp rx=0 tx=0).stop 2023-10-30T12:43:19.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.413+0000 7fa4665c5640 1 -- 172.21.15.12:0/1863894646 shutdown_connections 2023-10-30T12:43:19.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.413+0000 7fa4665c5640 1 --2- 172.21.15.12:0/1863894646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa448003980 0x7fa448003da0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:19.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.413+0000 7fa4665c5640 1 -- 172.21.15.12:0/1863894646 >> 172.21.15.12:0/1863894646 conn(0x7fa460069e90 msgr2=0x7fa46006a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:19.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.413+0000 7fa4665c5640 1 -- 172.21.15.12:0/1863894646 shutdown_connections 2023-10-30T12:43:19.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.413+0000 7fa4665c5640 1 -- 172.21.15.12:0/1863894646 wait complete. 2023-10-30T12:43:19.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa4665c5640 1 Processor -- start 2023-10-30T12:43:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa4665c5640 1 -- start start 2023-10-30T12:43:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa4665c5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa448003980 0x7fa46015d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa4665c5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa46014d460 0x7fa460147300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa4665c5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa460147f50 0x7fa460147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa4665c5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4601fabb0 con 0x7fa46014d460 2023-10-30T12:43:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa4665c5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46015c7c0 con 0x7fa460147f50 2023-10-30T12:43:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa4665c5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4601fa610 con 0x7fa448003980 2023-10-30T12:43:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa45f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa46014d460 0x7fa460147300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa45f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa46014d460 0x7fa460147300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46028/0 (socket says 172.21.15.12:46028) 2023-10-30T12:43:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa45f7fe640 1 -- 172.21.15.12:0/4235641308 learned_addr learned my addr 172.21.15.12:0/4235641308 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa45ffff640 1 --2- 172.21.15.12:0/4235641308 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa448003980 0x7fa46015d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa464b62640 1 --2- 172.21.15.12:0/4235641308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa460147f50 0x7fa460147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa45f7fe640 1 -- 172.21.15.12:0/4235641308 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa448003980 msgr2=0x7fa46015d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa45f7fe640 1 --2- 172.21.15.12:0/4235641308 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa448003980 0x7fa46015d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa45f7fe640 1 -- 172.21.15.12:0/4235641308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa460147f50 msgr2=0x7fa460147840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:19.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa45f7fe640 1 --2- 172.21.15.12:0/4235641308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa460147f50 0x7fa460147840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa45f7fe640 1 -- 172.21.15.12:0/4235641308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4601fa8d0 con 0x7fa46014d460 2023-10-30T12:43:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa464b62640 1 --2- 172.21.15.12:0/4235641308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa460147f50 0x7fa460147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:43:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa45f7fe640 1 --2- 172.21.15.12:0/4235641308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa46014d460 0x7fa460147300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa45400e910 tx=0x7fa45400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa45cff9640 1 -- 172.21.15.12:0/4235641308 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454004a00 con 0x7fa46014d460 2023-10-30T12:43:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa45cff9640 1 -- 172.21.15.12:0/4235641308 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa454004b60 con 0x7fa46014d460 2023-10-30T12:43:19.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa4665c5640 1 -- 172.21.15.12:0/4235641308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46014e400 con 0x7fa46014d460 2023-10-30T12:43:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.417+0000 7fa45cff9640 1 -- 172.21.15.12:0/4235641308 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454005030 con 0x7fa46014d460 2023-10-30T12:43:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.421+0000 7fa4665c5640 1 -- 172.21.15.12:0/4235641308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46014e9c0 con 0x7fa46014d460 2023-10-30T12:43:19.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.421+0000 7fa4665c5640 1 -- 172.21.15.12:0/4235641308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46014ebe0 con 0x7fa46014d460 2023-10-30T12:43:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.421+0000 7fa45cff9640 1 -- 172.21.15.12:0/4235641308 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa454005190 con 0x7fa46014d460 2023-10-30T12:43:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.421+0000 7fa45cff9640 1 --2- 172.21.15.12:0/4235641308 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa44c0664d0 0x7fa44c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.421+0000 7fa45cff9640 1 -- 172.21.15.12:0/4235641308 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fa454083d10 con 0x7fa46014d460 2023-10-30T12:43:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.421+0000 7fa45ffff640 1 --2- 172.21.15.12:0/4235641308 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa44c0664d0 0x7fa44c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:19.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.421+0000 7fa45ffff640 1 --2- 172.21.15.12:0/4235641308 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa44c0664d0 0x7fa44c068990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa4500af000 tx=0x7fa4500046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:19.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.425+0000 7fa45cff9640 1 -- 172.21.15.12:0/4235641308 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa454089020 con 0x7fa46014d460 2023-10-30T12:43:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.621+0000 7fa4665c5640 1 -- 172.21.15.12:0/4235641308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa46004e170 con 0x7fa46014d460 2023-10-30T12:43:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.621+0000 7fa45cff9640 1 -- 172.21.15.12:0/4235641308 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5733 (secure 0 0 0) 0x7fa454019030 con 0x7fa46014d460 2023-10-30T12:43:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:19.627 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:19.143461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:resolve","state_seq":33,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 37 2023-10-30T12:43:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.621+0000 7fa4665c5640 1 -- 172.21.15.12:0/4235641308 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa44c0664d0 msgr2=0x7fa44c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:19.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.621+0000 7fa4665c5640 1 --2- 172.21.15.12:0/4235641308 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa44c0664d0 0x7fa44c068990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa4500af000 tx=0x7fa4500046d0 comp rx=0 tx=0).stop 2023-10-30T12:43:19.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.621+0000 7fa4665c5640 1 -- 172.21.15.12:0/4235641308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa46014d460 msgr2=0x7fa460147300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:19.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.621+0000 7fa4665c5640 1 --2- 172.21.15.12:0/4235641308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa46014d460 0x7fa460147300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa45400e910 tx=0x7fa45400edd0 comp rx=0 tx=0).stop 2023-10-30T12:43:19.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.621+0000 7fa4665c5640 1 -- 172.21.15.12:0/4235641308 shutdown_connections 2023-10-30T12:43:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.621+0000 7fa4665c5640 1 --2- 172.21.15.12:0/4235641308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa460147f50 0x7fa460147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.621+0000 7fa4665c5640 1 --2- 172.21.15.12:0/4235641308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa46014d460 0x7fa460147300 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.621+0000 7fa4665c5640 1 --2- 172.21.15.12:0/4235641308 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa44c0664d0 0x7fa44c068990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.621+0000 7fa4665c5640 1 --2- 172.21.15.12:0/4235641308 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa448003980 0x7fa46015d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.621+0000 7fa4665c5640 1 -- 172.21.15.12:0/4235641308 >> 172.21.15.12:0/4235641308 conn(0x7fa460069e90 msgr2=0x7fa4600c1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.621+0000 7fa4665c5640 1 -- 172.21.15.12:0/4235641308 shutdown_connections 2023-10-30T12:43:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:19.621+0000 7fa4665c5640 1 -- 172.21.15.12:0/4235641308 wait complete. 2023-10-30T12:43:19.647 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:43:21.648 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:21.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.792+0000 7fd45c1d3640 1 Processor -- start 2023-10-30T12:43:21.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd45c1d3640 1 -- start start 2023-10-30T12:43:21.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd45c1d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4540c93d0 0x7fd45406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:21.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd45c1d3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45406afa0 con 0x7fd45406e470 2023-10-30T12:43:21.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd45c1d3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd454154150 con 0x7fd454153d70 2023-10-30T12:43:21.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd45c1d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45406d780 con 0x7fd4540c93d0 2023-10-30T12:43:21.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd45976e640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd45406e470 0x7fd4540c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50832/0 (socket says 172.21.15.12:50832) 2023-10-30T12:43:21.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd45976e640 1 -- 172.21.15.12:0/3361444023 learned_addr learned my addr 172.21.15.12:0/3361444023 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:43:21.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd45a770640 1 --2- 172.21.15.12:0/3361444023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4540c93d0 0x7fd45406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd458f6d640 1 -- 172.21.15.12:0/3361444023 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986259651 0 0) 0x7fd45406afa0 con 0x7fd45406e470 2023-10-30T12:43:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd458f6d640 1 -- 172.21.15.12:0/3361444023 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd440003530 con 0x7fd45406e470 2023-10-30T12:43:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd458f6d640 1 -- 172.21.15.12:0/3361444023 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2872807693 0 0) 0x7fd440003530 con 0x7fd45406e470 2023-10-30T12:43:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd458f6d640 1 -- 172.21.15.12:0/3361444023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4540c93d0 msgr2=0x7fd45406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd458f6d640 1 --2- 172.21.15.12:0/3361444023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4540c93d0 0x7fd45406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd458f6d640 1 -- 172.21.15.12:0/3361444023 >> v1:172.21.15.154:6789/0 conn(0x7fd454153d70 legacy=0x7fd4540cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd458f6d640 1 -- 172.21.15.12:0/3361444023 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45406e030 con 0x7fd45406e470 2023-10-30T12:43:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd458f6d640 1 -- 172.21.15.12:0/3361444023 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fd444001680 con 0x7fd45406e470 2023-10-30T12:43:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd458f6d640 1 -- 172.21.15.12:0/3361444023 >> v1:172.21.15.12:6789/0 conn(0x7fd45406e470 legacy=0x7fd4540c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd458f6d640 1 --2- 172.21.15.12:0/3361444023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4400039b0 0x7fd440003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd458f6d640 1 --2- 172.21.15.12:0/3361444023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4540c93d0 0x7fd4400047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd458f6d640 1 --2- 172.21.15.12:0/3361444023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd440004d10 0x7fd440005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd458f6d640 1 -- 172.21.15.12:0/3361444023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45406d780 con 0x7fd4400039b0 2023-10-30T12:43:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd458f6d640 1 -- 172.21.15.12:0/3361444023 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44000b990 con 0x7fd440004d10 2023-10-30T12:43:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd458f6d640 1 -- 172.21.15.12:0/3361444023 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44000bb50 con 0x7fd4540c93d0 2023-10-30T12:43:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd459f6f640 1 --2- 172.21.15.12:0/3361444023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4400039b0 0x7fd440003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:21.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd45976e640 1 --2- 172.21.15.12:0/3361444023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4540c93d0 0x7fd4400047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:21.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd45a770640 1 --2- 172.21.15.12:0/3361444023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd440004d10 0x7fd440005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:21.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd459f6f640 1 -- 172.21.15.12:0/3361444023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4540c93d0 msgr2=0x7fd4400047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:21.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd459f6f640 1 --2- 172.21.15.12:0/3361444023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4540c93d0 0x7fd4400047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:21.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd459f6f640 1 -- 172.21.15.12:0/3361444023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd440004d10 msgr2=0x7fd440005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:21.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd459f6f640 1 --2- 172.21.15.12:0/3361444023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd440004d10 0x7fd440005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:21.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.796+0000 7fd459f6f640 1 -- 172.21.15.12:0/3361444023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd440003530 con 0x7fd4400039b0 2023-10-30T12:43:21.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.800+0000 7fd459f6f640 1 --2- 172.21.15.12:0/3361444023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4400039b0 0x7fd440003d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd450001910 tx=0x7fd45001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad15ea5fbf475cd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:21.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.800+0000 7fd458f6d640 1 -- 172.21.15.12:0/3361444023 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4500ad640 con 0x7fd4400039b0 2023-10-30T12:43:21.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.800+0000 7fd458f6d640 1 -- 172.21.15.12:0/3361444023 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4500ad7c0 con 0x7fd4400039b0 2023-10-30T12:43:21.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.800+0000 7fd458f6d640 1 -- 172.21.15.12:0/3361444023 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4500adb70 con 0x7fd4400039b0 2023-10-30T12:43:21.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.800+0000 7fd45c1d3640 1 -- 172.21.15.12:0/3361444023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4400039b0 msgr2=0x7fd440003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:21.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.800+0000 7fd45c1d3640 1 --2- 172.21.15.12:0/3361444023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4400039b0 0x7fd440003d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd450001910 tx=0x7fd45001c470 comp rx=0 tx=0).stop 2023-10-30T12:43:21.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.800+0000 7fd45a770640 1 -- 172.21.15.12:0/3361444023 reap_dead start 2023-10-30T12:43:21.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.800+0000 7fd45c1d3640 1 -- 172.21.15.12:0/3361444023 shutdown_connections 2023-10-30T12:43:21.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.800+0000 7fd45c1d3640 1 -- 172.21.15.12:0/3361444023 >> 172.21.15.12:0/3361444023 conn(0x7fd454071ea0 msgr2=0x7fd454065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:21.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.800+0000 7fd45c1d3640 1 -- 172.21.15.12:0/3361444023 shutdown_connections 2023-10-30T12:43:21.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.800+0000 7fd45c1d3640 1 -- 172.21.15.12:0/3361444023 wait complete. 2023-10-30T12:43:21.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45c1d3640 1 Processor -- start 2023-10-30T12:43:21.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45c1d3640 1 -- start start 2023-10-30T12:43:21.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45c1d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4400039b0 0x7fd454154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:21.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45c1d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4541f5c30 0x7fd454145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:21.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45c1d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd454145a10 0x7fd45414ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:21.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45c1d3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4541560e0 con 0x7fd4541f5c30 2023-10-30T12:43:21.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45c1d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4540cc520 con 0x7fd454145a10 2023-10-30T12:43:21.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45c1d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4540cb670 con 0x7fd4400039b0 2023-10-30T12:43:21.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45976e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4541f5c30 0x7fd454145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:21.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45976e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4541f5c30 0x7fd454145110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51188/0 (socket says 172.21.15.12:51188) 2023-10-30T12:43:21.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45976e640 1 -- 172.21.15.12:0/539139739 learned_addr learned my addr 172.21.15.12:0/539139739 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:21.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd459f6f640 1 --2- 172.21.15.12:0/539139739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4400039b0 0x7fd454154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:21.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45a770640 1 --2- 172.21.15.12:0/539139739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd454145a10 0x7fd45414ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:21.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45976e640 1 -- 172.21.15.12:0/539139739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4400039b0 msgr2=0x7fd454154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:21.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45976e640 1 --2- 172.21.15.12:0/539139739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4400039b0 0x7fd454154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:21.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45976e640 1 -- 172.21.15.12:0/539139739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd454145a10 msgr2=0x7fd45414ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:21.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45976e640 1 --2- 172.21.15.12:0/539139739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd454145a10 0x7fd45414ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:21.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45976e640 1 -- 172.21.15.12:0/539139739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4540cb910 con 0x7fd4541f5c30 2023-10-30T12:43:21.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45976e640 1 --2- 172.21.15.12:0/539139739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4541f5c30 0x7fd454145110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd4440072f0 tx=0x7fd4440077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:21.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd44a7fc640 1 -- 172.21.15.12:0/539139739 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44400f9c0 con 0x7fd4541f5c30 2023-10-30T12:43:21.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd44a7fc640 1 -- 172.21.15.12:0/539139739 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44400fb40 con 0x7fd4541f5c30 2023-10-30T12:43:21.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd44a7fc640 1 -- 172.21.15.12:0/539139739 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4440133f0 con 0x7fd4541f5c30 2023-10-30T12:43:21.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45c1d3640 1 -- 172.21.15.12:0/539139739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4540cbbf0 con 0x7fd4541f5c30 2023-10-30T12:43:21.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.804+0000 7fd45c1d3640 1 -- 172.21.15.12:0/539139739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4540cc1b0 con 0x7fd4541f5c30 2023-10-30T12:43:21.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.808+0000 7fd45c1d3640 1 -- 172.21.15.12:0/539139739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4540cc6d0 con 0x7fd4541f5c30 2023-10-30T12:43:21.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.808+0000 7fd44a7fc640 1 -- 172.21.15.12:0/539139739 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd444013550 con 0x7fd4541f5c30 2023-10-30T12:43:21.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.808+0000 7fd44a7fc640 1 --2- 172.21.15.12:0/539139739 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd4380664a0 0x7fd438068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:21.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.808+0000 7fd44a7fc640 1 -- 172.21.15.12:0/539139739 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fd444084390 con 0x7fd4541f5c30 2023-10-30T12:43:21.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.808+0000 7fd459f6f640 1 --2- 172.21.15.12:0/539139739 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd4380664a0 0x7fd438068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:21.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.808+0000 7fd459f6f640 1 --2- 172.21.15.12:0/539139739 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd4380664a0 0x7fd438068960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd4500af000 tx=0x7fd450001330 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:21.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:21.812+0000 7fd44a7fc640 1 -- 172.21.15.12:0/539139739 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd4540cc6d0 con 0x7fd4541f5c30 2023-10-30T12:43:22.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:22.004+0000 7fd45c1d3640 1 -- 172.21.15.12:0/539139739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4540cc6d0 con 0x7fd4541f5c30 2023-10-30T12:43:22.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:22.004+0000 7fd44a7fc640 1 -- 172.21.15.12:0/539139739 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5733 (secure 0 0 0) 0x7fd4540cc6d0 con 0x7fd4541f5c30 2023-10-30T12:43:22.011 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:22.011 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:19.143461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:resolve","state_seq":33,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:22.012 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 37 2023-10-30T12:43:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:22.004+0000 7fd45c1d3640 1 -- 172.21.15.12:0/539139739 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd4380664a0 msgr2=0x7fd438068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:22.004+0000 7fd45c1d3640 1 --2- 172.21.15.12:0/539139739 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd4380664a0 0x7fd438068960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd4500af000 tx=0x7fd450001330 comp rx=0 tx=0).stop 2023-10-30T12:43:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:22.004+0000 7fd45c1d3640 1 -- 172.21.15.12:0/539139739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4541f5c30 msgr2=0x7fd454145110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:22.004+0000 7fd45c1d3640 1 --2- 172.21.15.12:0/539139739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4541f5c30 0x7fd454145110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd4440072f0 tx=0x7fd4440077b0 comp rx=0 tx=0).stop 2023-10-30T12:43:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:22.004+0000 7fd45c1d3640 1 -- 172.21.15.12:0/539139739 shutdown_connections 2023-10-30T12:43:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:22.004+0000 7fd45c1d3640 1 --2- 172.21.15.12:0/539139739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd454145a10 0x7fd45414ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:22.004+0000 7fd45c1d3640 1 --2- 172.21.15.12:0/539139739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4541f5c30 0x7fd454145110 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:22.004+0000 7fd45c1d3640 1 --2- 172.21.15.12:0/539139739 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd4380664a0 0x7fd438068960 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:22.004+0000 7fd45c1d3640 1 --2- 172.21.15.12:0/539139739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd4400039b0 0x7fd454154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:22.004+0000 7fd45c1d3640 1 -- 172.21.15.12:0/539139739 >> 172.21.15.12:0/539139739 conn(0x7fd454071ea0 msgr2=0x7fd454150600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:22.004+0000 7fd45c1d3640 1 -- 172.21.15.12:0/539139739 shutdown_connections 2023-10-30T12:43:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:22.004+0000 7fd45c1d3640 1 -- 172.21.15.12:0/539139739 wait complete. 2023-10-30T12:43:22.031 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:43:24.033 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:24.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.180+0000 7fb71413e640 1 Processor -- start 2023-10-30T12:43:24.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.180+0000 7fb71413e640 1 -- start start 2023-10-30T12:43:24.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.180+0000 7fb71413e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb70c0cb550 0x7fb70c0cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:24.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.180+0000 7fb71413e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb70c071250 0x7fb70c0716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:24.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.180+0000 7fb71413e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb70c071be0 con 0x7fb70c14fd50 2023-10-30T12:43:24.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.180+0000 7fb71413e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70c1560a0 con 0x7fb70c071250 2023-10-30T12:43:24.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.180+0000 7fb71413e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c156380 con 0x7fb70c0cb550 2023-10-30T12:43:24.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.180+0000 7fb7126db640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb70c071250 0x7fb70c0716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:24.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.180+0000 7fb711eda640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb70c14fd50 0x7fb70c0c65d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50842/0 (socket says 172.21.15.12:50842) 2023-10-30T12:43:24.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.180+0000 7fb711eda640 1 -- 172.21.15.12:0/1870991112 learned_addr learned my addr 172.21.15.12:0/1870991112 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:43:24.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.180+0000 7fb7116d9640 1 --2- 172.21.15.12:0/1870991112 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb70c0cb550 0x7fb70c0cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:24.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.184+0000 7fb7126db640 1 -- 172.21.15.12:0/1870991112 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb70c0cb550 msgr2=0x7fb70c0cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:24.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.184+0000 7fb7126db640 1 --2- 172.21.15.12:0/1870991112 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb70c0cb550 0x7fb70c0cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:24.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.184+0000 7fb7126db640 1 -- 172.21.15.12:0/1870991112 >> v1:172.21.15.12:6789/0 conn(0x7fb70c14fd50 legacy=0x7fb70c0c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:24.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.184+0000 7fb7126db640 1 -- 172.21.15.12:0/1870991112 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c156660 con 0x7fb70c071250 2023-10-30T12:43:24.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.184+0000 7fb7126db640 1 --2- 172.21.15.12:0/1870991112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb70c071250 0x7fb70c0716a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb704009d90 tx=0x7fb70401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4525cdc6556c575 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:24.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.184+0000 7fb710ed8640 1 -- 172.21.15.12:0/1870991112 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7040a5920 con 0x7fb70c071250 2023-10-30T12:43:24.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.184+0000 7fb710ed8640 1 -- 172.21.15.12:0/1870991112 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7040a5a80 con 0x7fb70c071250 2023-10-30T12:43:24.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.184+0000 7fb710ed8640 1 -- 172.21.15.12:0/1870991112 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7040a5e10 con 0x7fb70c071250 2023-10-30T12:43:24.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.184+0000 7fb71413e640 1 -- 172.21.15.12:0/1870991112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb70c071250 msgr2=0x7fb70c0716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:24.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.184+0000 7fb71413e640 1 --2- 172.21.15.12:0/1870991112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb70c071250 0x7fb70c0716a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb704009d90 tx=0x7fb70401c920 comp rx=0 tx=0).stop 2023-10-30T12:43:24.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.184+0000 7fb71413e640 1 -- 172.21.15.12:0/1870991112 shutdown_connections 2023-10-30T12:43:24.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.184+0000 7fb71413e640 1 --2- 172.21.15.12:0/1870991112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb70c071250 0x7fb70c0716a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:24.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.184+0000 7fb71413e640 1 --2- 172.21.15.12:0/1870991112 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb70c0cb550 0x7fb70c0cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:24.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.184+0000 7fb71413e640 1 -- 172.21.15.12:0/1870991112 >> 172.21.15.12:0/1870991112 conn(0x7fb70c06c6e0 msgr2=0x7fb70c069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:24.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb71413e640 1 -- 172.21.15.12:0/1870991112 shutdown_connections 2023-10-30T12:43:24.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb71413e640 1 -- 172.21.15.12:0/1870991112 wait complete. 2023-10-30T12:43:24.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb71413e640 1 Processor -- start 2023-10-30T12:43:24.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb71413e640 1 -- start start 2023-10-30T12:43:24.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb71413e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb70c071250 0x7fb70c13f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:24.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb71413e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb70c0cb550 0x7fb70c0c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:24.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb71413e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb70c14fd50 0x7fb70c0c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:24.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb71413e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c13f960 con 0x7fb70c071250 2023-10-30T12:43:24.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb71413e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c14d900 con 0x7fb70c0cb550 2023-10-30T12:43:24.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb71413e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c14f5a0 con 0x7fb70c14fd50 2023-10-30T12:43:24.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb711eda640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb70c071250 0x7fb70c13f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:24.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb711eda640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb70c071250 0x7fb70c13f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51196/0 (socket says 172.21.15.12:51196) 2023-10-30T12:43:24.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb711eda640 1 -- 172.21.15.12:0/501634637 learned_addr learned my addr 172.21.15.12:0/501634637 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:24.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb7116d9640 1 --2- 172.21.15.12:0/501634637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb70c0cb550 0x7fb70c0c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:24.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb7126db640 1 --2- 172.21.15.12:0/501634637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb70c14fd50 0x7fb70c0c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:24.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb711eda640 1 -- 172.21.15.12:0/501634637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb70c14fd50 msgr2=0x7fb70c0c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:24.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb711eda640 1 --2- 172.21.15.12:0/501634637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb70c14fd50 0x7fb70c0c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:24.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb711eda640 1 -- 172.21.15.12:0/501634637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb70c0cb550 msgr2=0x7fb70c0c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:24.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb711eda640 1 --2- 172.21.15.12:0/501634637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb70c0cb550 0x7fb70c0c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:24.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb711eda640 1 -- 172.21.15.12:0/501634637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb704009a70 con 0x7fb70c071250 2023-10-30T12:43:24.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb7126db640 1 --2- 172.21.15.12:0/501634637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb70c14fd50 0x7fb70c0c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:43:24.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb711eda640 1 --2- 172.21.15.12:0/501634637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb70c071250 0x7fb70c13f420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c370 tx=0x7fb70800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:24.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb7027fc640 1 -- 172.21.15.12:0/501634637 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708005cf0 con 0x7fb70c071250 2023-10-30T12:43:24.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb7027fc640 1 -- 172.21.15.12:0/501634637 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb708005e50 con 0x7fb70c071250 2023-10-30T12:43:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb7027fc640 1 -- 172.21.15.12:0/501634637 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7080072f0 con 0x7fb70c071250 2023-10-30T12:43:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.188+0000 7fb71413e640 1 -- 172.21.15.12:0/501634637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70c13fd20 con 0x7fb70c071250 2023-10-30T12:43:24.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.192+0000 7fb71413e640 1 -- 172.21.15.12:0/501634637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70c14d590 con 0x7fb70c071250 2023-10-30T12:43:24.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.192+0000 7fb71413e640 1 -- 172.21.15.12:0/501634637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70c14d750 con 0x7fb70c071250 2023-10-30T12:43:24.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.192+0000 7fb7027fc640 1 -- 172.21.15.12:0/501634637 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb708007450 con 0x7fb70c071250 2023-10-30T12:43:24.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.192+0000 7fb7027fc640 1 --2- 172.21.15.12:0/501634637 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb6f00664a0 0x7fb6f0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:24.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.192+0000 7fb7027fc640 1 -- 172.21.15.12:0/501634637 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fb7080835c0 con 0x7fb70c071250 2023-10-30T12:43:24.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.192+0000 7fb7116d9640 1 --2- 172.21.15.12:0/501634637 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb6f00664a0 0x7fb6f0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:24.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.196+0000 7fb7116d9640 1 --2- 172.21.15.12:0/501634637 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb6f00664a0 0x7fb6f0068960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb70c15a340 tx=0x7fb6fc00d520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:24.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.196+0000 7fb7027fc640 1 -- 172.21.15.12:0/501634637 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb708050840 con 0x7fb70c071250 2023-10-30T12:43:24.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.388+0000 7fb71413e640 1 -- 172.21.15.12:0/501634637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb70c140240 con 0x7fb70c071250 2023-10-30T12:43:24.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.388+0000 7fb7027fc640 1 -- 172.21.15.12:0/501634637 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5733 (secure 0 0 0) 0x7fb70c140240 con 0x7fb70c071250 2023-10-30T12:43:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:24.395 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:19.143461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:resolve","state_seq":33,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:24.396 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 37 2023-10-30T12:43:24.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.388+0000 7fb71413e640 1 -- 172.21.15.12:0/501634637 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb6f00664a0 msgr2=0x7fb6f0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:24.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.388+0000 7fb71413e640 1 --2- 172.21.15.12:0/501634637 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb6f00664a0 0x7fb6f0068960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb70c15a340 tx=0x7fb6fc00d520 comp rx=0 tx=0).stop 2023-10-30T12:43:24.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.388+0000 7fb71413e640 1 -- 172.21.15.12:0/501634637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb70c071250 msgr2=0x7fb70c13f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:24.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.388+0000 7fb71413e640 1 --2- 172.21.15.12:0/501634637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb70c071250 0x7fb70c13f420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c370 tx=0x7fb70800c830 comp rx=0 tx=0).stop 2023-10-30T12:43:24.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.388+0000 7fb71413e640 1 -- 172.21.15.12:0/501634637 shutdown_connections 2023-10-30T12:43:24.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.388+0000 7fb71413e640 1 --2- 172.21.15.12:0/501634637 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb6f00664a0 0x7fb6f0068960 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:24.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.388+0000 7fb71413e640 1 --2- 172.21.15.12:0/501634637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb70c14fd50 0x7fb70c0c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:24.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.388+0000 7fb71413e640 1 --2- 172.21.15.12:0/501634637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb70c0cb550 0x7fb70c0c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:24.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.388+0000 7fb71413e640 1 --2- 172.21.15.12:0/501634637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb70c071250 0x7fb70c13f420 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:24.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.388+0000 7fb71413e640 1 -- 172.21.15.12:0/501634637 >> 172.21.15.12:0/501634637 conn(0x7fb70c06c6e0 msgr2=0x7fb70c143a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:24.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.388+0000 7fb71413e640 1 -- 172.21.15.12:0/501634637 shutdown_connections 2023-10-30T12:43:24.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:24.388+0000 7fb71413e640 1 -- 172.21.15.12:0/501634637 wait complete. 2023-10-30T12:43:24.415 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:43:26.416 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:26.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.552+0000 7fd36c73e640 1 Processor -- start 2023-10-30T12:43:26.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.552+0000 7fd36c73e640 1 -- start start 2023-10-30T12:43:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.552+0000 7fd36c73e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3640c6b80 0x7fd3640bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.552+0000 7fd36c73e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3640c0a50 con 0x7fd36413f4a0 2023-10-30T12:43:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.552+0000 7fd36c73e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36406f6e0 con 0x7fd3640c6f60 2023-10-30T12:43:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.552+0000 7fd36c73e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36406f9c0 con 0x7fd3640c6b80 2023-10-30T12:43:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.552+0000 7fd36acdb640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd36413f4a0 0x7fd3641566a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50856/0 (socket says 172.21.15.12:50856) 2023-10-30T12:43:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.552+0000 7fd36acdb640 1 -- 172.21.15.12:0/593009154 learned_addr learned my addr 172.21.15.12:0/593009154 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:43:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.556+0000 7fd36a4da640 1 --2- 172.21.15.12:0/593009154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3640c6b80 0x7fd3640bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.556+0000 7fd3694d8640 1 -- 172.21.15.12:0/593009154 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438485675 0 0) 0x7fd3640c0a50 con 0x7fd36413f4a0 2023-10-30T12:43:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.556+0000 7fd3694d8640 1 -- 172.21.15.12:0/593009154 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd350003530 con 0x7fd36413f4a0 2023-10-30T12:43:26.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.556+0000 7fd36a4da640 1 -- 172.21.15.12:0/593009154 >> v1:172.21.15.154:6789/0 conn(0x7fd3640c6f60 legacy=0x7fd36413ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:26.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.556+0000 7fd3694d8640 1 -- 172.21.15.12:0/593009154 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2926519849 0 0) 0x7fd350003530 con 0x7fd36413f4a0 2023-10-30T12:43:26.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.556+0000 7fd36a4da640 1 -- 172.21.15.12:0/593009154 >> v1:172.21.15.12:6789/0 conn(0x7fd36413f4a0 legacy=0x7fd3641566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:26.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.556+0000 7fd36a4da640 1 -- 172.21.15.12:0/593009154 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36406fca0 con 0x7fd3640c6b80 2023-10-30T12:43:26.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.556+0000 7fd36a4da640 1 --2- 172.21.15.12:0/593009154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3640c6b80 0x7fd3640bfec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3600097f0 tx=0x7fd36001c290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc60e7d8aaab788 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:26.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd3694d8640 1 -- 172.21.15.12:0/593009154 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c001a00 con 0x7fd3640c6b80 2023-10-30T12:43:26.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd3694d8640 1 -- 172.21.15.12:0/593009154 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3600a7050 con 0x7fd3640c6b80 2023-10-30T12:43:26.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd3694d8640 1 -- 172.21.15.12:0/593009154 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3600ab600 con 0x7fd3640c6b80 2023-10-30T12:43:26.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36c73e640 1 -- 172.21.15.12:0/593009154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3640c6b80 msgr2=0x7fd3640bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:26.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36c73e640 1 --2- 172.21.15.12:0/593009154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3640c6b80 0x7fd3640bfec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3600097f0 tx=0x7fd36001c290 comp rx=0 tx=0).stop 2023-10-30T12:43:26.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36c73e640 1 -- 172.21.15.12:0/593009154 shutdown_connections 2023-10-30T12:43:26.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36c73e640 1 --2- 172.21.15.12:0/593009154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3640c6b80 0x7fd3640bfec0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:26.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36c73e640 1 -- 172.21.15.12:0/593009154 >> 172.21.15.12:0/593009154 conn(0x7fd3640c7910 msgr2=0x7fd3640c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:26.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36c73e640 1 -- 172.21.15.12:0/593009154 shutdown_connections 2023-10-30T12:43:26.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36c73e640 1 -- 172.21.15.12:0/593009154 wait complete. 2023-10-30T12:43:26.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36c73e640 1 Processor -- start 2023-10-30T12:43:26.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36c73e640 1 -- start start 2023-10-30T12:43:26.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36c73e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3640c6b80 0x7fd3640ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:26.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36c73e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3640c6f60 0x7fd3640c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:26.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36c73e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd36413f4a0 0x7fd3640c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:26.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36c73e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3641f0600 con 0x7fd36413f4a0 2023-10-30T12:43:26.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36c73e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640c3070 con 0x7fd3640c6b80 2023-10-30T12:43:26.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36c73e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640c32c0 con 0x7fd3640c6f60 2023-10-30T12:43:26.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36acdb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd36413f4a0 0x7fd3640c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:26.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36acdb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd36413f4a0 0x7fd3640c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51206/0 (socket says 172.21.15.12:51206) 2023-10-30T12:43:26.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36acdb640 1 -- 172.21.15.12:0/4169989083 learned_addr learned my addr 172.21.15.12:0/4169989083 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:26.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd36a4da640 1 --2- 172.21.15.12:0/4169989083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3640c6b80 0x7fd3640ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:26.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.560+0000 7fd369cd9640 1 --2- 172.21.15.12:0/4169989083 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3640c6f60 0x7fd3640c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:26.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd36acdb640 1 -- 172.21.15.12:0/4169989083 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3640c6f60 msgr2=0x7fd3640c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:26.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd36acdb640 1 --2- 172.21.15.12:0/4169989083 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3640c6f60 0x7fd3640c3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:26.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd36acdb640 1 -- 172.21.15.12:0/4169989083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3640c6b80 msgr2=0x7fd3640ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:26.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd36acdb640 1 --2- 172.21.15.12:0/4169989083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3640c6b80 0x7fd3640ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:26.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd36acdb640 1 -- 172.21.15.12:0/4169989083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd360009420 con 0x7fd36413f4a0 2023-10-30T12:43:26.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd36a4da640 1 --2- 172.21.15.12:0/4169989083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3640c6b80 0x7fd3640ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:43:26.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd36acdb640 1 --2- 172.21.15.12:0/4169989083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd36413f4a0 0x7fd3640c42a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00cac0 tx=0x7fd35c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:26.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd35affd640 1 -- 172.21.15.12:0/4169989083 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c00a640 con 0x7fd36413f4a0 2023-10-30T12:43:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd35affd640 1 -- 172.21.15.12:0/4169989083 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c00a7c0 con 0x7fd36413f4a0 2023-10-30T12:43:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd35affd640 1 -- 172.21.15.12:0/4169989083 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c00abf0 con 0x7fd36413f4a0 2023-10-30T12:43:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd36c73e640 1 -- 172.21.15.12:0/4169989083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3640c3560 con 0x7fd36413f4a0 2023-10-30T12:43:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd36c73e640 1 -- 172.21.15.12:0/4169989083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3641effb0 con 0x7fd36413f4a0 2023-10-30T12:43:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd36c73e640 1 -- 172.21.15.12:0/4169989083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd36404e170 con 0x7fd36413f4a0 2023-10-30T12:43:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd35affd640 1 -- 172.21.15.12:0/4169989083 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd35c00ad50 con 0x7fd36413f4a0 2023-10-30T12:43:26.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd35affd640 1 --2- 172.21.15.12:0/4169989083 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd3480664a0 0x7fd348068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:26.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd35affd640 1 -- 172.21.15.12:0/4169989083 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fd35c083630 con 0x7fd36413f4a0 2023-10-30T12:43:26.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.564+0000 7fd36a4da640 1 --2- 172.21.15.12:0/4169989083 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd3480664a0 0x7fd348068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:26.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.568+0000 7fd36a4da640 1 --2- 172.21.15.12:0/4169989083 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd3480664a0 0x7fd348068960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd3600ad000 tx=0x7fd3600a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:26.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.568+0000 7fd35affd640 1 -- 172.21.15.12:0/4169989083 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd35c0508b0 con 0x7fd36413f4a0 2023-10-30T12:43:26.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.756+0000 7fd36c73e640 1 -- 172.21.15.12:0/4169989083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3641f0740 con 0x7fd36413f4a0 2023-10-30T12:43:26.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.760+0000 7fd35affd640 1 -- 172.21.15.12:0/4169989083 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5735 (secure 0 0 0) 0x7fd3641f0740 con 0x7fd36413f4a0 2023-10-30T12:43:26.767 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:26.767 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:24.527164+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:reconnect","state_seq":35,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 38 2023-10-30T12:43:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.760+0000 7fd36c73e640 1 -- 172.21.15.12:0/4169989083 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd3480664a0 msgr2=0x7fd348068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.760+0000 7fd36c73e640 1 --2- 172.21.15.12:0/4169989083 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd3480664a0 0x7fd348068960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd3600ad000 tx=0x7fd3600a6000 comp rx=0 tx=0).stop 2023-10-30T12:43:26.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.760+0000 7fd36c73e640 1 -- 172.21.15.12:0/4169989083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd36413f4a0 msgr2=0x7fd3640c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:26.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.760+0000 7fd36c73e640 1 --2- 172.21.15.12:0/4169989083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd36413f4a0 0x7fd3640c42a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00cac0 tx=0x7fd35c00cf80 comp rx=0 tx=0).stop 2023-10-30T12:43:26.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.760+0000 7fd36c73e640 1 -- 172.21.15.12:0/4169989083 shutdown_connections 2023-10-30T12:43:26.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.760+0000 7fd36c73e640 1 --2- 172.21.15.12:0/4169989083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd36413f4a0 0x7fd3640c42a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.760+0000 7fd36c73e640 1 --2- 172.21.15.12:0/4169989083 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd3480664a0 0x7fd348068960 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.760+0000 7fd36c73e640 1 --2- 172.21.15.12:0/4169989083 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3640c6f60 0x7fd3640c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.760+0000 7fd36c73e640 1 --2- 172.21.15.12:0/4169989083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3640c6b80 0x7fd3640ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.760+0000 7fd36c73e640 1 -- 172.21.15.12:0/4169989083 >> 172.21.15.12:0/4169989083 conn(0x7fd3640c7910 msgr2=0x7fd36413aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.760+0000 7fd36c73e640 1 -- 172.21.15.12:0/4169989083 shutdown_connections 2023-10-30T12:43:26.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:26.760+0000 7fd36c73e640 1 -- 172.21.15.12:0/4169989083 wait complete. 2023-10-30T12:43:26.787 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 38, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 38, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4285": { "addr": "172.21.15.12:6836/291392212", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 291392212, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 291392212, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138322906710015, "flags": 0, "gid": 4285, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 6 }, "gid_4971": { "addr": "172.21.15.12:6837/3124135109", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 3124135109, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 3124135109, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 4971, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 44 }, "gid_5124": { "addr": "172.21.15.12:6835/559649789", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 559649789, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 559649789, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5124, "incarnation": 36, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:reconnect", "state_seq": 35 } }, "last_failure": 0, "last_failure_osd_epoch": 48, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:43:24.527164+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4285, "mds_1": 5124, "mds_2": 4971 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:43:28.789 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:28.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.936+0000 7f29c5a46640 1 Processor -- start 2023-10-30T12:43:28.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.936+0000 7f29c5a46640 1 -- start start 2023-10-30T12:43:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29c5a46640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f29c015d6f0 0x7f29c015a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29c5a46640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c00cb800 con 0x7f29c01538a0 2023-10-30T12:43:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29c5a46640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c00cb330 con 0x7f29c00cf160 2023-10-30T12:43:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29c5a46640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c00cad70 con 0x7f29c015d6f0 2023-10-30T12:43:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29beffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f29c01538a0 0x7f29c00ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50870/0 (socket says 172.21.15.12:50870) 2023-10-30T12:43:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29beffd640 1 -- 172.21.15.12:0/2088037685 learned_addr learned my addr 172.21.15.12:0/2088037685 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:43:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29b7fff640 1 --2- 172.21.15.12:0/2088037685 >> v2:172.21.15.154:3300/0 conn(0x7f29c015d6f0 0x7f29c015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29be7fc640 1 -- 172.21.15.12:0/2088037685 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545389659 0 0) 0x7f29c00cb800 con 0x7f29c01538a0 2023-10-30T12:43:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29be7fc640 1 -- 172.21.15.12:0/2088037685 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29ac003530 con 0x7f29c01538a0 2023-10-30T12:43:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29be7fc640 1 -- 172.21.15.12:0/2088037685 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1746698204 0 0) 0x7f29ac003530 con 0x7f29c01538a0 2023-10-30T12:43:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29be7fc640 1 -- 172.21.15.12:0/2088037685 >> v2:172.21.15.154:3300/0 conn(0x7f29c015d6f0 msgr2=0x7f29c015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29be7fc640 1 --2- 172.21.15.12:0/2088037685 >> v2:172.21.15.154:3300/0 conn(0x7f29c015d6f0 0x7f29c015a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:28.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29be7fc640 1 -- 172.21.15.12:0/2088037685 >> v1:172.21.15.154:6789/0 conn(0x7f29c00cf160 legacy=0x7f29c0164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:28.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29be7fc640 1 -- 172.21.15.12:0/2088037685 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c00cb050 con 0x7f29c01538a0 2023-10-30T12:43:28.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29be7fc640 1 -- 172.21.15.12:0/2088037685 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f29b0001680 con 0x7f29c01538a0 2023-10-30T12:43:28.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29be7fc640 1 -- 172.21.15.12:0/2088037685 >> v1:172.21.15.12:6789/0 conn(0x7f29c01538a0 legacy=0x7f29c00ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:28.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29be7fc640 1 --2- 172.21.15.12:0/2088037685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29ac003980 0x7f29ac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29be7fc640 1 --2- 172.21.15.12:0/2088037685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29ac0047d0 0x7f29ac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29be7fc640 1 --2- 172.21.15.12:0/2088037685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29ac005190 0x7f29ac00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29be7fc640 1 -- 172.21.15.12:0/2088037685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00cad70 con 0x7f29ac0047d0 2023-10-30T12:43:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29be7fc640 1 -- 172.21.15.12:0/2088037685 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac00c040 con 0x7f29ac005190 2023-10-30T12:43:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29be7fc640 1 -- 172.21.15.12:0/2088037685 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac00c200 con 0x7f29ac003980 2023-10-30T12:43:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29b7fff640 1 --2- 172.21.15.12:0/2088037685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29ac0047d0 0x7f29ac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29beffd640 1 --2- 172.21.15.12:0/2088037685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29ac003980 0x7f29ac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29bf7fe640 1 --2- 172.21.15.12:0/2088037685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29ac005190 0x7f29ac00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29b7fff640 1 -- 172.21.15.12:0/2088037685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29ac003980 msgr2=0x7f29ac003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29b7fff640 1 --2- 172.21.15.12:0/2088037685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29ac003980 0x7f29ac003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29b7fff640 1 -- 172.21.15.12:0/2088037685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29ac005190 msgr2=0x7f29ac00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29b7fff640 1 --2- 172.21.15.12:0/2088037685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29ac005190 0x7f29ac00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29b7fff640 1 -- 172.21.15.12:0/2088037685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ac003530 con 0x7f29ac0047d0 2023-10-30T12:43:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.940+0000 7f29bf7fe640 1 -- 172.21.15.12:0/2088037685 reap_dead start 2023-10-30T12:43:28.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.944+0000 7f29b7fff640 1 --2- 172.21.15.12:0/2088037685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29ac0047d0 0x7f29ac004c50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009ae0 tx=0x7f29a801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c80162ec860d579 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:28.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.944+0000 7f29be7fc640 1 -- 172.21.15.12:0/2088037685 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a80a5640 con 0x7f29ac0047d0 2023-10-30T12:43:28.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.944+0000 7f29be7fc640 1 -- 172.21.15.12:0/2088037685 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a80a57a0 con 0x7f29ac0047d0 2023-10-30T12:43:28.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.944+0000 7f29be7fc640 1 -- 172.21.15.12:0/2088037685 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a80a5b50 con 0x7f29ac0047d0 2023-10-30T12:43:28.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.944+0000 7f29c5a46640 1 -- 172.21.15.12:0/2088037685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29ac0047d0 msgr2=0x7f29ac004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:28.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.944+0000 7f29c5a46640 1 --2- 172.21.15.12:0/2088037685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29ac0047d0 0x7f29ac004c50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009ae0 tx=0x7f29a801ca50 comp rx=0 tx=0).stop 2023-10-30T12:43:28.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.944+0000 7f29c5a46640 1 -- 172.21.15.12:0/2088037685 shutdown_connections 2023-10-30T12:43:28.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.944+0000 7f29c5a46640 1 --2- 172.21.15.12:0/2088037685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29ac0047d0 0x7f29ac004c50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:28.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.944+0000 7f29c5a46640 1 -- 172.21.15.12:0/2088037685 >> 172.21.15.12:0/2088037685 conn(0x7f29c006bfe0 msgr2=0x7f29c006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:28.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.944+0000 7f29c5a46640 1 -- 172.21.15.12:0/2088037685 shutdown_connections 2023-10-30T12:43:28.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.944+0000 7f29c5a46640 1 -- 172.21.15.12:0/2088037685 wait complete. 2023-10-30T12:43:28.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29c5a46640 1 Processor -- start 2023-10-30T12:43:28.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29c5a46640 1 -- start start 2023-10-30T12:43:28.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29c5a46640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29ac0047d0 0x7f29c01f0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:28.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29c5a46640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29c01f01b0 0x7f29c00b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:28.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29c5a46640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29c0158790 0x7f29c01f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:28.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29c5a46640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c01e55b0 con 0x7f29c01f01b0 2023-10-30T12:43:28.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29c5a46640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c01ef4c0 con 0x7f29ac0047d0 2023-10-30T12:43:28.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29c5a46640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c01f3400 con 0x7f29c0158790 2023-10-30T12:43:28.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29b7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29c01f01b0 0x7f29c00b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:28.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29b7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29c01f01b0 0x7f29c00b4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51218/0 (socket says 172.21.15.12:51218) 2023-10-30T12:43:28.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29b7fff640 1 -- 172.21.15.12:0/940855197 learned_addr learned my addr 172.21.15.12:0/940855197 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:28.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29beffd640 1 --2- 172.21.15.12:0/940855197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29ac0047d0 0x7f29c01f0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:28.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29bf7fe640 1 --2- 172.21.15.12:0/940855197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29c0158790 0x7f29c01f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29b7fff640 1 -- 172.21.15.12:0/940855197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29c0158790 msgr2=0x7f29c01f2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29b7fff640 1 --2- 172.21.15.12:0/940855197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29c0158790 0x7f29c01f2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29b7fff640 1 -- 172.21.15.12:0/940855197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29ac0047d0 msgr2=0x7f29c01f0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29b7fff640 1 --2- 172.21.15.12:0/940855197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29ac0047d0 0x7f29c01f0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29b7fff640 1 -- 172.21.15.12:0/940855197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c01f36e0 con 0x7f29c01f01b0 2023-10-30T12:43:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29b7fff640 1 --2- 172.21.15.12:0/940855197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29c01f01b0 0x7f29c00b4380 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f29a801c880 tx=0x7f29a80197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:28.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29bcff9640 1 -- 172.21.15.12:0/940855197 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a80bd070 con 0x7f29c01f01b0 2023-10-30T12:43:28.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29bcff9640 1 -- 172.21.15.12:0/940855197 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a8004020 con 0x7f29c01f01b0 2023-10-30T12:43:28.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29c5a46640 1 -- 172.21.15.12:0/940855197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c01ef6d0 con 0x7f29c01f01b0 2023-10-30T12:43:28.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29bcff9640 1 -- 172.21.15.12:0/940855197 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a80043f0 con 0x7f29c01f01b0 2023-10-30T12:43:28.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.948+0000 7f29c5a46640 1 -- 172.21.15.12:0/940855197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c01efc90 con 0x7f29c01f01b0 2023-10-30T12:43:28.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.952+0000 7f29c5a46640 1 -- 172.21.15.12:0/940855197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c004e170 con 0x7f29c01f01b0 2023-10-30T12:43:28.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.952+0000 7f29bcff9640 1 -- 172.21.15.12:0/940855197 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f29a80b4070 con 0x7f29c01f01b0 2023-10-30T12:43:28.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.952+0000 7f29bcff9640 1 --2- 172.21.15.12:0/940855197 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f29a00664d0 0x7f29a0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:28.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.952+0000 7f29bcff9640 1 -- 172.21.15.12:0/940855197 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f29a811e360 con 0x7f29c01f01b0 2023-10-30T12:43:28.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.952+0000 7f29beffd640 1 --2- 172.21.15.12:0/940855197 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f29a00664d0 0x7f29a0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:28.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.952+0000 7f29beffd640 1 --2- 172.21.15.12:0/940855197 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f29a00664d0 0x7f29a0068990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f29b0000c10 tx=0x7f29b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:28.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:28.956+0000 7f29bcff9640 1 -- 172.21.15.12:0/940855197 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f29a80eb6e0 con 0x7f29c01f01b0 2023-10-30T12:43:29.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:29.144+0000 7f29c5a46640 1 -- 172.21.15.12:0/940855197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29c00b1dd0 con 0x7f29c01f01b0 2023-10-30T12:43:29.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:29.148+0000 7f29bcff9640 1 -- 172.21.15.12:0/940855197 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5735 (secure 0 0 0) 0x7f29a80eed00 con 0x7f29c01f01b0 2023-10-30T12:43:29.154 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:29.155 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:24.527164+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:reconnect","state_seq":35,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:29.155 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 38 2023-10-30T12:43:29.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:29.148+0000 7f29c5a46640 1 -- 172.21.15.12:0/940855197 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f29a00664d0 msgr2=0x7f29a0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:29.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:29.148+0000 7f29c5a46640 1 --2- 172.21.15.12:0/940855197 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f29a00664d0 0x7f29a0068990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f29b0000c10 tx=0x7f29b0009040 comp rx=0 tx=0).stop 2023-10-30T12:43:29.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:29.148+0000 7f29c5a46640 1 -- 172.21.15.12:0/940855197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29c01f01b0 msgr2=0x7f29c00b4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:29.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:29.148+0000 7f29c5a46640 1 --2- 172.21.15.12:0/940855197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29c01f01b0 0x7f29c00b4380 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f29a801c880 tx=0x7f29a80197f0 comp rx=0 tx=0).stop 2023-10-30T12:43:29.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:29.148+0000 7f29c5a46640 1 -- 172.21.15.12:0/940855197 shutdown_connections 2023-10-30T12:43:29.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:29.148+0000 7f29c5a46640 1 --2- 172.21.15.12:0/940855197 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f29a00664d0 0x7f29a0068990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:29.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:29.148+0000 7f29c5a46640 1 --2- 172.21.15.12:0/940855197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29c0158790 0x7f29c01f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:29.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:29.148+0000 7f29c5a46640 1 --2- 172.21.15.12:0/940855197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29c01f01b0 0x7f29c00b4380 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:29.148+0000 7f29c5a46640 1 --2- 172.21.15.12:0/940855197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29ac0047d0 0x7f29c01f0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:29.148+0000 7f29c5a46640 1 -- 172.21.15.12:0/940855197 >> 172.21.15.12:0/940855197 conn(0x7f29c006bfe0 msgr2=0x7f29c006c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:29.148+0000 7f29c5a46640 1 -- 172.21.15.12:0/940855197 shutdown_connections 2023-10-30T12:43:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:29.148+0000 7f29c5a46640 1 -- 172.21.15.12:0/940855197 wait complete. 2023-10-30T12:43:29.175 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:43:31.176 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:31.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.324+0000 7f12cbd72640 1 Processor -- start 2023-10-30T12:43:31.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.324+0000 7f12cbd72640 1 -- start start 2023-10-30T12:43:31.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.324+0000 7f12cbd72640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c4153d70 0x7f12c406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:31.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.324+0000 7f12cbd72640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c406afa0 con 0x7f12c40c8490 2023-10-30T12:43:31.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.324+0000 7f12cbd72640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c4154150 con 0x7f12c406e470 2023-10-30T12:43:31.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.324+0000 7f12cbd72640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c406d780 con 0x7f12c4153d70 2023-10-30T12:43:31.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.324+0000 7f12ca30f640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f12c40c8490 0x7f12c40c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56548/0 (socket says 172.21.15.12:56548) 2023-10-30T12:43:31.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.324+0000 7f12ca30f640 1 -- 172.21.15.12:0/4107720809 learned_addr learned my addr 172.21.15.12:0/4107720809 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:43:31.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.324+0000 7f12c9b0e640 1 --2- 172.21.15.12:0/4107720809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c4153d70 0x7f12c406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:31.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.324+0000 7f12c8b0c640 1 -- 172.21.15.12:0/4107720809 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2789346193 0 0) 0x7f12c406afa0 con 0x7f12c40c8490 2023-10-30T12:43:31.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.324+0000 7f12c8b0c640 1 -- 172.21.15.12:0/4107720809 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12b0003530 con 0x7f12c40c8490 2023-10-30T12:43:31.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c8b0c640 1 -- 172.21.15.12:0/4107720809 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4106324533 0 0) 0x7f12b0003530 con 0x7f12c40c8490 2023-10-30T12:43:31.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c8b0c640 1 -- 172.21.15.12:0/4107720809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c4153d70 msgr2=0x7f12c406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:31.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c8b0c640 1 --2- 172.21.15.12:0/4107720809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c4153d70 0x7f12c406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:31.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c8b0c640 1 -- 172.21.15.12:0/4107720809 >> v1:172.21.15.154:6789/0 conn(0x7f12c406e470 legacy=0x7f12c40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:31.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c8b0c640 1 -- 172.21.15.12:0/4107720809 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c406e030 con 0x7f12c40c8490 2023-10-30T12:43:31.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c8b0c640 1 -- 172.21.15.12:0/4107720809 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f12bc001a00 con 0x7f12c40c8490 2023-10-30T12:43:31.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c8b0c640 1 -- 172.21.15.12:0/4107720809 >> v1:172.21.15.12:6789/0 conn(0x7f12c40c8490 legacy=0x7f12c40c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:31.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c8b0c640 1 --2- 172.21.15.12:0/4107720809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12b00039b0 0x7f12b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:31.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c8b0c640 1 --2- 172.21.15.12:0/4107720809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12b00047d0 0x7f12b0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:31.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c8b0c640 1 --2- 172.21.15.12:0/4107720809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c4153d70 0x7f12b0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:31.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c8b0c640 1 -- 172.21.15.12:0/4107720809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c406d780 con 0x7f12b00047d0 2023-10-30T12:43:31.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c8b0c640 1 -- 172.21.15.12:0/4107720809 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b000ba40 con 0x7f12b00039b0 2023-10-30T12:43:31.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c8b0c640 1 -- 172.21.15.12:0/4107720809 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b000bc00 con 0x7f12c4153d70 2023-10-30T12:43:31.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c930d640 1 --2- 172.21.15.12:0/4107720809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12b00047d0 0x7f12b0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:31.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c9b0e640 1 --2- 172.21.15.12:0/4107720809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12b00039b0 0x7f12b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:31.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12ca30f640 1 --2- 172.21.15.12:0/4107720809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c4153d70 0x7f12b0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:31.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c930d640 1 -- 172.21.15.12:0/4107720809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c4153d70 msgr2=0x7f12b0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:31.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c930d640 1 --2- 172.21.15.12:0/4107720809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c4153d70 0x7f12b0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:31.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c930d640 1 -- 172.21.15.12:0/4107720809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12b00039b0 msgr2=0x7f12b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:31.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c930d640 1 --2- 172.21.15.12:0/4107720809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12b00039b0 0x7f12b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:31.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c930d640 1 -- 172.21.15.12:0/4107720809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b0003530 con 0x7f12b00047d0 2023-10-30T12:43:31.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c930d640 1 --2- 172.21.15.12:0/4107720809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12b00047d0 0x7f12b0004c50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f12b4001750 tx=0x7f12b401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ad447d70230e1f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:31.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.328+0000 7f12c8b0c640 1 -- 172.21.15.12:0/4107720809 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b40052d0 con 0x7f12b00047d0 2023-10-30T12:43:31.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12c8b0c640 1 -- 172.21.15.12:0/4107720809 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b40a7050 con 0x7f12b00047d0 2023-10-30T12:43:31.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12c8b0c640 1 -- 172.21.15.12:0/4107720809 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4003840 con 0x7f12b00047d0 2023-10-30T12:43:31.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12cbd72640 1 -- 172.21.15.12:0/4107720809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12b00047d0 msgr2=0x7f12b0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:31.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12cbd72640 1 --2- 172.21.15.12:0/4107720809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12b00047d0 0x7f12b0004c50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f12b4001750 tx=0x7f12b401c3a0 comp rx=0 tx=0).stop 2023-10-30T12:43:31.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12ca30f640 1 -- 172.21.15.12:0/4107720809 reap_dead start 2023-10-30T12:43:31.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12cbd72640 1 -- 172.21.15.12:0/4107720809 shutdown_connections 2023-10-30T12:43:31.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12cbd72640 1 -- 172.21.15.12:0/4107720809 >> 172.21.15.12:0/4107720809 conn(0x7f12c4071ea0 msgr2=0x7f12c4065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:31.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12cbd72640 1 -- 172.21.15.12:0/4107720809 shutdown_connections 2023-10-30T12:43:31.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12cbd72640 1 -- 172.21.15.12:0/4107720809 wait complete. 2023-10-30T12:43:31.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12cbd72640 1 Processor -- start 2023-10-30T12:43:31.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12cbd72640 1 -- start start 2023-10-30T12:43:31.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12cbd72640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12b00047d0 0x7f12c41e05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:31.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12cbd72640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12c41e1ed0 0x7f12c40cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:31.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12cbd72640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c40cb2b0 0x7f12c41e11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:31.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12cbd72640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c41f0010 con 0x7f12b00047d0 2023-10-30T12:43:31.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12cbd72640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4154790 con 0x7f12c41e1ed0 2023-10-30T12:43:31.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12cbd72640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c40cb730 con 0x7f12c40cb2b0 2023-10-30T12:43:31.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12c9b0e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12b00047d0 0x7f12c41e05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:31.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12c9b0e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12b00047d0 0x7f12c41e05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46020/0 (socket says 172.21.15.12:46020) 2023-10-30T12:43:31.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.332+0000 7f12c9b0e640 1 -- 172.21.15.12:0/2324544307 learned_addr learned my addr 172.21.15.12:0/2324544307 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:31.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.336+0000 7f12c930d640 1 --2- 172.21.15.12:0/2324544307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12c41e1ed0 0x7f12c40cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:31.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.336+0000 7f12ca30f640 1 --2- 172.21.15.12:0/2324544307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c40cb2b0 0x7f12c41e11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:31.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.336+0000 7f12c9b0e640 1 -- 172.21.15.12:0/2324544307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c40cb2b0 msgr2=0x7f12c41e11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:31.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.336+0000 7f12c9b0e640 1 --2- 172.21.15.12:0/2324544307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c40cb2b0 0x7f12c41e11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:31.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.336+0000 7f12c9b0e640 1 -- 172.21.15.12:0/2324544307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12c41e1ed0 msgr2=0x7f12c40cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:31.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.336+0000 7f12c9b0e640 1 --2- 172.21.15.12:0/2324544307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12c41e1ed0 0x7f12c40cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:31.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.336+0000 7f12c9b0e640 1 -- 172.21.15.12:0/2324544307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4152b90 con 0x7f12b00047d0 2023-10-30T12:43:31.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.336+0000 7f12c9b0e640 1 --2- 172.21.15.12:0/2324544307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12b00047d0 0x7f12c41e05a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f12c0011610 tx=0x7f12c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:31.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.336+0000 7f12ba7fc640 1 -- 172.21.15.12:0/2324544307 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0015870 con 0x7f12b00047d0 2023-10-30T12:43:31.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.336+0000 7f12ba7fc640 1 -- 172.21.15.12:0/2324544307 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c00159d0 con 0x7f12b00047d0 2023-10-30T12:43:31.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.336+0000 7f12cbd72640 1 -- 172.21.15.12:0/2324544307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c41f0200 con 0x7f12b00047d0 2023-10-30T12:43:31.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.336+0000 7f12ba7fc640 1 -- 172.21.15.12:0/2324544307 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0015d40 con 0x7f12b00047d0 2023-10-30T12:43:31.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.336+0000 7f12cbd72640 1 -- 172.21.15.12:0/2324544307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c41f04a0 con 0x7f12b00047d0 2023-10-30T12:43:31.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.336+0000 7f12cbd72640 1 -- 172.21.15.12:0/2324544307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c404e170 con 0x7f12b00047d0 2023-10-30T12:43:31.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.336+0000 7f12ba7fc640 1 -- 172.21.15.12:0/2324544307 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f12c000f480 con 0x7f12b00047d0 2023-10-30T12:43:31.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.340+0000 7f12ba7fc640 1 --2- 172.21.15.12:0/2324544307 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f12a80664a0 0x7f12a8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:31.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.340+0000 7f12ba7fc640 1 -- 172.21.15.12:0/2324544307 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f12c0084470 con 0x7f12b00047d0 2023-10-30T12:43:31.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.340+0000 7f12c930d640 1 --2- 172.21.15.12:0/2324544307 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f12a80664a0 0x7f12a8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:31.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.340+0000 7f12c930d640 1 --2- 172.21.15.12:0/2324544307 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f12a80664a0 0x7f12a8068960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f12b00057b0 tx=0x7f12b40011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:31.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.344+0000 7f12ba7fc640 1 -- 172.21.15.12:0/2324544307 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f12c0051760 con 0x7f12b00047d0 2023-10-30T12:43:31.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.540+0000 7f12cbd72640 1 -- 172.21.15.12:0/2324544307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12c4157360 con 0x7f12b00047d0 2023-10-30T12:43:31.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.540+0000 7f12ba7fc640 1 -- 172.21.15.12:0/2324544307 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5732 (secure 0 0 0) 0x7f12c0054d80 con 0x7f12b00047d0 2023-10-30T12:43:31.548 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:31.549 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:29.547565+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:rejoin","state_seq":37,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:31.549 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 39 2023-10-30T12:43:31.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.544+0000 7f12cbd72640 1 -- 172.21.15.12:0/2324544307 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f12a80664a0 msgr2=0x7f12a8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:31.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.544+0000 7f12cbd72640 1 --2- 172.21.15.12:0/2324544307 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f12a80664a0 0x7f12a8068960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f12b00057b0 tx=0x7f12b40011a0 comp rx=0 tx=0).stop 2023-10-30T12:43:31.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.544+0000 7f12cbd72640 1 -- 172.21.15.12:0/2324544307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12b00047d0 msgr2=0x7f12c41e05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:31.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.544+0000 7f12cbd72640 1 --2- 172.21.15.12:0/2324544307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12b00047d0 0x7f12c41e05a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f12c0011610 tx=0x7f12c0011ad0 comp rx=0 tx=0).stop 2023-10-30T12:43:31.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.544+0000 7f12cbd72640 1 -- 172.21.15.12:0/2324544307 shutdown_connections 2023-10-30T12:43:31.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.544+0000 7f12cbd72640 1 --2- 172.21.15.12:0/2324544307 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f12a80664a0 0x7f12a8068960 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:31.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.544+0000 7f12cbd72640 1 --2- 172.21.15.12:0/2324544307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c40cb2b0 0x7f12c41e11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:31.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.544+0000 7f12cbd72640 1 --2- 172.21.15.12:0/2324544307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12c41e1ed0 0x7f12c40cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:31.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.544+0000 7f12cbd72640 1 --2- 172.21.15.12:0/2324544307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12b00047d0 0x7f12c41e05a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:31.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.544+0000 7f12cbd72640 1 -- 172.21.15.12:0/2324544307 >> 172.21.15.12:0/2324544307 conn(0x7f12c4071ea0 msgr2=0x7f12c406ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:31.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.544+0000 7f12cbd72640 1 -- 172.21.15.12:0/2324544307 shutdown_connections 2023-10-30T12:43:31.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:31.544+0000 7f12cbd72640 1 -- 172.21.15.12:0/2324544307 wait complete. 2023-10-30T12:43:31.569 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:43:33.570 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.716+0000 7f152f846640 1 Processor -- start 2023-10-30T12:43:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.716+0000 7f152f846640 1 -- start start 2023-10-30T12:43:33.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.720+0000 7f152f846640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f15281580e0 0x7f1528070740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:33.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.720+0000 7f152f846640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152815e700 0x7f152806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:33.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.720+0000 7f152f846640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15280c5810 con 0x7f15280c6540 2023-10-30T12:43:33.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.720+0000 7f152f846640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15280c6990 con 0x7f15281580e0 2023-10-30T12:43:33.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.720+0000 7f152f846640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152815eb90 con 0x7f152815e700 2023-10-30T12:43:33.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.720+0000 7f152d5e2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f15281580e0 0x7f1528070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.720+0000 7f152dde3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152815e700 0x7f152806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.720+0000 7f152cde1640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f15280c6540 0x7f1528069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:50586/0 (socket says 172.21.15.12:50586) 2023-10-30T12:43:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.720+0000 7f152d5e2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f15281580e0 0x7f1528070740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:44684/0 (socket says 172.21.15.12:44684) 2023-10-30T12:43:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.720+0000 7f152cde1640 1 -- 172.21.15.12:0/2163314053 learned_addr learned my addr 172.21.15.12:0/2163314053 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:43:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.720+0000 7f152d5e2640 1 -- 172.21.15.12:0/2163314053 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152815e700 msgr2=0x7f152806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:33.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.720+0000 7f152d5e2640 1 --2- 172.21.15.12:0/2163314053 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152815e700 0x7f152806a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:33.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.720+0000 7f152d5e2640 1 -- 172.21.15.12:0/2163314053 >> v1:172.21.15.154:6789/0 conn(0x7f15280c6540 legacy=0x7f1528069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:33.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.720+0000 7f152d5e2640 1 -- 172.21.15.12:0/2163314053 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15280c6bb0 con 0x7f15281580e0 2023-10-30T12:43:33.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.724+0000 7f152d5e2640 1 --2- 172.21.15.12:0/2163314053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15281580e0 0x7f1528070740 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f15240097c0 tx=0x7f152401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b0b556f9dff5fc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:33.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.724+0000 7f151ffff640 1 -- 172.21.15.12:0/2163314053 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15240ab650 con 0x7f15281580e0 2023-10-30T12:43:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.724+0000 7f151ffff640 1 -- 172.21.15.12:0/2163314053 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15240ab7b0 con 0x7f15281580e0 2023-10-30T12:43:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.724+0000 7f151ffff640 1 -- 172.21.15.12:0/2163314053 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15240abb40 con 0x7f15281580e0 2023-10-30T12:43:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.724+0000 7f152f846640 1 -- 172.21.15.12:0/2163314053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15281580e0 msgr2=0x7f1528070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:33.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.724+0000 7f152f846640 1 --2- 172.21.15.12:0/2163314053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15281580e0 0x7f1528070740 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f15240097c0 tx=0x7f152401c450 comp rx=0 tx=0).stop 2023-10-30T12:43:33.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.724+0000 7f152f846640 1 -- 172.21.15.12:0/2163314053 shutdown_connections 2023-10-30T12:43:33.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.724+0000 7f152f846640 1 --2- 172.21.15.12:0/2163314053 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152815e700 0x7f152806a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:33.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.724+0000 7f152f846640 1 --2- 172.21.15.12:0/2163314053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15281580e0 0x7f1528070740 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:33.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.724+0000 7f152f846640 1 -- 172.21.15.12:0/2163314053 >> 172.21.15.12:0/2163314053 conn(0x7f15280c7910 msgr2=0x7f15280c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:33.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.724+0000 7f152f846640 1 -- 172.21.15.12:0/2163314053 shutdown_connections 2023-10-30T12:43:33.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.724+0000 7f152f846640 1 -- 172.21.15.12:0/2163314053 wait complete. 2023-10-30T12:43:33.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.724+0000 7f152f846640 1 Processor -- start 2023-10-30T12:43:33.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.724+0000 7f152f846640 1 -- start start 2023-10-30T12:43:33.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152f846640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15280c6540 0x7f1528071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:33.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152f846640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15281580e0 0x7f15280c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:33.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152f846640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152815e700 0x7f15280c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:33.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152f846640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15280c28e0 con 0x7f152815e700 2023-10-30T12:43:33.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152f846640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15280c20b0 con 0x7f15281580e0 2023-10-30T12:43:33.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152f846640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15280c3b00 con 0x7f15280c6540 2023-10-30T12:43:33.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152dde3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152815e700 0x7f15280c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:33.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152dde3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152815e700 0x7f15280c4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46030/0 (socket says 172.21.15.12:46030) 2023-10-30T12:43:33.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152dde3640 1 -- 172.21.15.12:0/3681752587 learned_addr learned my addr 172.21.15.12:0/3681752587 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:33.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152d5e2640 1 --2- 172.21.15.12:0/3681752587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15280c6540 0x7f1528071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:33.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152cde1640 1 --2- 172.21.15.12:0/3681752587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15281580e0 0x7f15280c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:33.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152dde3640 1 -- 172.21.15.12:0/3681752587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15280c6540 msgr2=0x7f1528071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:33.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152dde3640 1 --2- 172.21.15.12:0/3681752587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15280c6540 0x7f1528071ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:33.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152dde3640 1 -- 172.21.15.12:0/3681752587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15281580e0 msgr2=0x7f15280c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:33.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152dde3640 1 --2- 172.21.15.12:0/3681752587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15281580e0 0x7f15280c48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:33.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152dde3640 1 -- 172.21.15.12:0/3681752587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1524009420 con 0x7f152815e700 2023-10-30T12:43:33.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152cde1640 1 --2- 172.21.15.12:0/3681752587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15281580e0 0x7f15280c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:43:33.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152dde3640 1 --2- 172.21.15.12:0/3681752587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152815e700 0x7f15280c4e00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f152000efc0 tx=0x7f152000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:33.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f151dffb640 1 -- 172.21.15.12:0/3681752587 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520009ba0 con 0x7f152815e700 2023-10-30T12:43:33.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f151dffb640 1 -- 172.21.15.12:0/3681752587 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152000f040 con 0x7f152815e700 2023-10-30T12:43:33.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f151dffb640 1 -- 172.21.15.12:0/3681752587 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15200146c0 con 0x7f152815e700 2023-10-30T12:43:33.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152f846640 1 -- 172.21.15.12:0/3681752587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15280c3de0 con 0x7f152815e700 2023-10-30T12:43:33.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152f846640 1 -- 172.21.15.12:0/3681752587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15280c43a0 con 0x7f152815e700 2023-10-30T12:43:33.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f152f846640 1 -- 172.21.15.12:0/3681752587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15280c45c0 con 0x7f152815e700 2023-10-30T12:43:33.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.728+0000 7f151dffb640 1 -- 172.21.15.12:0/3681752587 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1520014820 con 0x7f152815e700 2023-10-30T12:43:33.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.732+0000 7f151dffb640 1 --2- 172.21.15.12:0/3681752587 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f150c0664d0 0x7f150c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:33.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.732+0000 7f151dffb640 1 -- 172.21.15.12:0/3681752587 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f1520084d80 con 0x7f152815e700 2023-10-30T12:43:33.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.732+0000 7f152d5e2640 1 --2- 172.21.15.12:0/3681752587 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f150c0664d0 0x7f150c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:33.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.732+0000 7f152d5e2640 1 --2- 172.21.15.12:0/3681752587 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f150c0664d0 0x7f150c068990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f15240af000 tx=0x7f15240a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:33.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.736+0000 7f151dffb640 1 -- 172.21.15.12:0/3681752587 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1520052080 con 0x7f152815e700 2023-10-30T12:43:33.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.924+0000 7f152f846640 1 -- 172.21.15.12:0/3681752587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f152804e170 con 0x7f152815e700 2023-10-30T12:43:33.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.924+0000 7f151dffb640 1 -- 172.21.15.12:0/3681752587 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5731 (secure 0 0 0) 0x7f1520018090 con 0x7f152815e700 2023-10-30T12:43:33.932 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:33.932 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:33.244974+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:rejoin","state_seq":37,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:33.933 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 40 2023-10-30T12:43:33.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.928+0000 7f152f846640 1 -- 172.21.15.12:0/3681752587 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f150c0664d0 msgr2=0x7f150c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:33.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.928+0000 7f152f846640 1 --2- 172.21.15.12:0/3681752587 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f150c0664d0 0x7f150c068990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f15240af000 tx=0x7f15240a7000 comp rx=0 tx=0).stop 2023-10-30T12:43:33.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.928+0000 7f152f846640 1 -- 172.21.15.12:0/3681752587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152815e700 msgr2=0x7f15280c4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:33.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.928+0000 7f152f846640 1 --2- 172.21.15.12:0/3681752587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152815e700 0x7f15280c4e00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f152000efc0 tx=0x7f152000c490 comp rx=0 tx=0).stop 2023-10-30T12:43:33.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.928+0000 7f152f846640 1 -- 172.21.15.12:0/3681752587 shutdown_connections 2023-10-30T12:43:33.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.928+0000 7f152f846640 1 --2- 172.21.15.12:0/3681752587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152815e700 0x7f15280c4e00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:33.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.928+0000 7f152f846640 1 --2- 172.21.15.12:0/3681752587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15281580e0 0x7f15280c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:33.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.928+0000 7f152f846640 1 --2- 172.21.15.12:0/3681752587 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f150c0664d0 0x7f150c068990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:33.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.928+0000 7f152f846640 1 --2- 172.21.15.12:0/3681752587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15280c6540 0x7f1528071ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:33.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.928+0000 7f152f846640 1 -- 172.21.15.12:0/3681752587 >> 172.21.15.12:0/3681752587 conn(0x7f15280c7910 msgr2=0x7f1528156e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:33.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.928+0000 7f152f846640 1 -- 172.21.15.12:0/3681752587 shutdown_connections 2023-10-30T12:43:33.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:33.928+0000 7f152f846640 1 -- 172.21.15.12:0/3681752587 wait complete. 2023-10-30T12:43:33.953 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:43:34.989 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:43:34.994 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:43:35.954 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:36.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.100+0000 7fc1d4046640 1 Processor -- start 2023-10-30T12:43:36.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d4046640 1 -- start start 2023-10-30T12:43:36.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d4046640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1cc153bf0 0x7fc1cc071260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:36.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d4046640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc1cc0c6fe0 0x7fc1cc0c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:36.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d4046640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc1cc0cb550 0x7fc1cc0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:36.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d4046640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1cc15d870 con 0x7fc1cc0c6fe0 2023-10-30T12:43:36.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d4046640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1cc0c60e0 con 0x7fc1cc0cb550 2023-10-30T12:43:36.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d4046640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc0c63a0 con 0x7fc1cc153bf0 2023-10-30T12:43:36.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d15e1640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc1cc0c6fe0 0x7fc1cc0c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d15e1640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc1cc0c6fe0 0x7fc1cc0c7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46036/0 (socket says 172.21.15.12:46036) 2023-10-30T12:43:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d15e1640 1 -- 172.21.15.12:0/3209733247 learned_addr learned my addr 172.21.15.12:0/3209733247 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d1de2640 1 --2- 172.21.15.12:0/3209733247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1cc153bf0 0x7fc1cc071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d25e3640 1 --2- 172.21.15.12:0/3209733247 >> v2:172.21.15.154:3300/0 conn(0x7fc1cc0cb550 0x7fc1cc0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d15e1640 1 -- 172.21.15.12:0/3209733247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1cc153bf0 msgr2=0x7fc1cc071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:36.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d15e1640 1 --2- 172.21.15.12:0/3209733247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1cc153bf0 0x7fc1cc071260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:36.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d15e1640 1 -- 172.21.15.12:0/3209733247 >> v2:172.21.15.154:3300/0 conn(0x7fc1cc0cb550 msgr2=0x7fc1cc0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:43:36.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d15e1640 1 --2- 172.21.15.12:0/3209733247 >> v2:172.21.15.154:3300/0 conn(0x7fc1cc0cb550 0x7fc1cc0cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:36.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.104+0000 7fc1d15e1640 1 -- 172.21.15.12:0/3209733247 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc0c6680 con 0x7fc1cc0c6fe0 2023-10-30T12:43:36.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.108+0000 7fc1d15e1640 1 --2- 172.21.15.12:0/3209733247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1cc0c6fe0 0x7fc1cc0c7480 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc06b990 tx=0x7fc1bc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1bc94c0c0cd3c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:36.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.108+0000 7fc1d0de0640 1 -- 172.21.15.12:0/3209733247 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0aa5f0 con 0x7fc1cc0c6fe0 2023-10-30T12:43:36.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.108+0000 7fc1d0de0640 1 -- 172.21.15.12:0/3209733247 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc0aa750 con 0x7fc1cc0c6fe0 2023-10-30T12:43:36.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.108+0000 7fc1d0de0640 1 -- 172.21.15.12:0/3209733247 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0aaae0 con 0x7fc1cc0c6fe0 2023-10-30T12:43:36.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.108+0000 7fc1d4046640 1 -- 172.21.15.12:0/3209733247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1cc0c6fe0 msgr2=0x7fc1cc0c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:36.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.108+0000 7fc1d4046640 1 --2- 172.21.15.12:0/3209733247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1cc0c6fe0 0x7fc1cc0c7480 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc06b990 tx=0x7fc1bc01c4d0 comp rx=0 tx=0).stop 2023-10-30T12:43:36.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.108+0000 7fc1d4046640 1 -- 172.21.15.12:0/3209733247 shutdown_connections 2023-10-30T12:43:36.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.108+0000 7fc1d4046640 1 --2- 172.21.15.12:0/3209733247 >> v2:172.21.15.154:3300/0 conn(0x7fc1cc0cb550 0x7fc1cc0cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:36.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.108+0000 7fc1d4046640 1 --2- 172.21.15.12:0/3209733247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1cc0c6fe0 0x7fc1cc0c7480 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:36.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.108+0000 7fc1d4046640 1 --2- 172.21.15.12:0/3209733247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1cc153bf0 0x7fc1cc071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:36.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.108+0000 7fc1d4046640 1 -- 172.21.15.12:0/3209733247 >> 172.21.15.12:0/3209733247 conn(0x7fc1cc06c6e0 msgr2=0x7fc1cc069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:36.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.108+0000 7fc1d4046640 1 -- 172.21.15.12:0/3209733247 shutdown_connections 2023-10-30T12:43:36.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.108+0000 7fc1d4046640 1 -- 172.21.15.12:0/3209733247 wait complete. 2023-10-30T12:43:36.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.108+0000 7fc1d4046640 1 Processor -- start 2023-10-30T12:43:36.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d4046640 1 -- start start 2023-10-30T12:43:36.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d4046640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1cc0c6fe0 0x7fc1cc0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:36.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d4046640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1cc0cb550 0x7fc1cc15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:36.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d4046640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1cc153bf0 0x7fc1cc158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d4046640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc1eb0b0 con 0x7fc1cc0cb550 2023-10-30T12:43:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d4046640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc1f1a20 con 0x7fc1cc0c6fe0 2023-10-30T12:43:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d4046640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc1f1cc0 con 0x7fc1cc153bf0 2023-10-30T12:43:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d15e1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1cc0cb550 0x7fc1cc15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d15e1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1cc0cb550 0x7fc1cc15c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46042/0 (socket says 172.21.15.12:46042) 2023-10-30T12:43:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d15e1640 1 -- 172.21.15.12:0/1228454913 learned_addr learned my addr 172.21.15.12:0/1228454913 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d1de2640 1 --2- 172.21.15.12:0/1228454913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1cc0c6fe0 0x7fc1cc0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d25e3640 1 --2- 172.21.15.12:0/1228454913 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1cc153bf0 0x7fc1cc158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d15e1640 1 -- 172.21.15.12:0/1228454913 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1cc153bf0 msgr2=0x7fc1cc158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d15e1640 1 --2- 172.21.15.12:0/1228454913 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1cc153bf0 0x7fc1cc158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:36.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d15e1640 1 -- 172.21.15.12:0/1228454913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1cc0c6fe0 msgr2=0x7fc1cc0c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:36.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d15e1640 1 --2- 172.21.15.12:0/1228454913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1cc0c6fe0 0x7fc1cc0c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:36.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d15e1640 1 -- 172.21.15.12:0/1228454913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc0094d0 con 0x7fc1cc0cb550 2023-10-30T12:43:36.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d15e1640 1 --2- 172.21.15.12:0/1228454913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1cc0cb550 0x7fc1cc15c550 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc01ab80 tx=0x7fc1bc019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:36.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1c27fc640 1 -- 172.21.15.12:0/1228454913 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0bd5f0 con 0x7fc1cc0cb550 2023-10-30T12:43:36.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1c27fc640 1 -- 172.21.15.12:0/1228454913 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc0bd750 con 0x7fc1cc0cb550 2023-10-30T12:43:36.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1c27fc640 1 -- 172.21.15.12:0/1228454913 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0bdac0 con 0x7fc1cc0cb550 2023-10-30T12:43:36.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d4046640 1 -- 172.21.15.12:0/1228454913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1cc1f2820 con 0x7fc1cc0cb550 2023-10-30T12:43:36.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d4046640 1 -- 172.21.15.12:0/1228454913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc1f2260 con 0x7fc1cc0cb550 2023-10-30T12:43:36.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.112+0000 7fc1d4046640 1 -- 172.21.15.12:0/1228454913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1cc1eb220 con 0x7fc1cc0cb550 2023-10-30T12:43:36.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.116+0000 7fc1c27fc640 1 -- 172.21.15.12:0/1228454913 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc1bc0b33f0 con 0x7fc1cc0cb550 2023-10-30T12:43:36.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.116+0000 7fc1c27fc640 1 --2- 172.21.15.12:0/1228454913 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc1b00664a0 0x7fc1b0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:36.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.116+0000 7fc1c27fc640 1 -- 172.21.15.12:0/1228454913 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fc1bc0a7070 con 0x7fc1cc0cb550 2023-10-30T12:43:36.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.116+0000 7fc1d1de2640 1 --2- 172.21.15.12:0/1228454913 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc1b00664a0 0x7fc1b0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:36.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.116+0000 7fc1d1de2640 1 --2- 172.21.15.12:0/1228454913 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc1b00664a0 0x7fc1b0068960 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009ac0 tx=0x7fc1c8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:36.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.120+0000 7fc1c27fc640 1 -- 172.21.15.12:0/1228454913 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc1bc0ebec0 con 0x7fc1cc0cb550 2023-10-30T12:43:36.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.312+0000 7fc1d4046640 1 -- 172.21.15.12:0/1228454913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1cc04e170 con 0x7fc1cc0cb550 2023-10-30T12:43:36.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.316+0000 7fc1c27fc640 1 -- 172.21.15.12:0/1228454913 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5731 (secure 0 0 0) 0x7fc1bc0ef4e0 con 0x7fc1cc0cb550 2023-10-30T12:43:36.323 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:36.323 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:35.236829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:36.323 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 41 2023-10-30T12:43:36.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.316+0000 7fc1d4046640 1 -- 172.21.15.12:0/1228454913 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc1b00664a0 msgr2=0x7fc1b0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:36.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.316+0000 7fc1d4046640 1 --2- 172.21.15.12:0/1228454913 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc1b00664a0 0x7fc1b0068960 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009ac0 tx=0x7fc1c8009190 comp rx=0 tx=0).stop 2023-10-30T12:43:36.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.316+0000 7fc1d4046640 1 -- 172.21.15.12:0/1228454913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1cc0cb550 msgr2=0x7fc1cc15c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:36.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.316+0000 7fc1d4046640 1 --2- 172.21.15.12:0/1228454913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1cc0cb550 0x7fc1cc15c550 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc01ab80 tx=0x7fc1bc019270 comp rx=0 tx=0).stop 2023-10-30T12:43:36.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.316+0000 7fc1d4046640 1 -- 172.21.15.12:0/1228454913 shutdown_connections 2023-10-30T12:43:36.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.316+0000 7fc1d4046640 1 --2- 172.21.15.12:0/1228454913 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc1b00664a0 0x7fc1b0068960 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:36.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.316+0000 7fc1d4046640 1 --2- 172.21.15.12:0/1228454913 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1cc153bf0 0x7fc1cc158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:36.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.316+0000 7fc1d4046640 1 --2- 172.21.15.12:0/1228454913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1cc0cb550 0x7fc1cc15c550 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:36.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.316+0000 7fc1d4046640 1 --2- 172.21.15.12:0/1228454913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1cc0c6fe0 0x7fc1cc0c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:36.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.316+0000 7fc1d4046640 1 -- 172.21.15.12:0/1228454913 >> 172.21.15.12:0/1228454913 conn(0x7fc1cc06c6e0 msgr2=0x7fc1cc069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:36.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.316+0000 7fc1d4046640 1 -- 172.21.15.12:0/1228454913 shutdown_connections 2023-10-30T12:43:36.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:36.316+0000 7fc1d4046640 1 -- 172.21.15.12:0/1228454913 wait complete. 2023-10-30T12:43:36.343 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=1, replacing gid=4282 2023-10-30T12:43:36.344 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 6 secs before reviving mds.d 2023-10-30T12:43:42.344 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2023-10-30T12:43:42.344 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-30T12:43:42.345 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-30T12:43:42.348 INFO:tasks.ceph.mds.d:Started 2023-10-30T12:43:42.348 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:42.424 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2023-10-30T12:43:42.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.496+0000 7f795c0ea640 1 Processor -- start 2023-10-30T12:43:42.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.496+0000 7f795c0ea640 1 -- start start 2023-10-30T12:43:42.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.496+0000 7f795c0ea640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f79540c6b80 0x7f79540bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:42.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.496+0000 7f795c0ea640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79540c0a50 con 0x7f795413f4a0 2023-10-30T12:43:42.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.496+0000 7f795c0ea640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f795406f6e0 con 0x7f79540c6f60 2023-10-30T12:43:42.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.496+0000 7f795c0ea640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f795406f9c0 con 0x7f79540c6b80 2023-10-30T12:43:42.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.500+0000 7f7959e86640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f79540c6b80 0x7f79540bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:42.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.500+0000 7f7959e86640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f79540c6b80 0x7f79540bfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36688/0 (socket says 172.21.15.12:36688) 2023-10-30T12:43:42.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.500+0000 7f7959e86640 1 -- 172.21.15.12:0/63225127 learned_addr learned my addr 172.21.15.12:0/63225127 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:42.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.500+0000 7f7959e86640 1 -- 172.21.15.12:0/63225127 >> v1:172.21.15.154:6789/0 conn(0x7f79540c6f60 legacy=0x7f795413ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:42.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.500+0000 7f7959e86640 1 -- 172.21.15.12:0/63225127 >> v1:172.21.15.12:6789/0 conn(0x7f795413f4a0 legacy=0x7f79541566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:42.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.500+0000 7f7959e86640 1 -- 172.21.15.12:0/63225127 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795406fca0 con 0x7f79540c6b80 2023-10-30T12:43:42.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.500+0000 7f7958e84640 1 -- 172.21.15.12:0/63225127 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563750056 0 0) 0x7f79540c0a50 con 0x7f795413f4a0 2023-10-30T12:43:42.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.500+0000 7f7959e86640 1 --2- 172.21.15.12:0/63225127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79540c6b80 0x7f79540bfec0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7950009870 tx=0x7f795001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1edfe4e17f1266e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:42.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.500+0000 7f7958e84640 1 -- 172.21.15.12:0/63225127 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79500ab650 con 0x7f79540c6b80 2023-10-30T12:43:42.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.500+0000 7f7958e84640 1 -- 172.21.15.12:0/63225127 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79500ab7b0 con 0x7f79540c6b80 2023-10-30T12:43:42.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.500+0000 7f7958e84640 1 -- 172.21.15.12:0/63225127 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79500abb40 con 0x7f79540c6b80 2023-10-30T12:43:42.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795c0ea640 1 -- 172.21.15.12:0/63225127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79540c6b80 msgr2=0x7f79540bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:42.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795c0ea640 1 --2- 172.21.15.12:0/63225127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79540c6b80 0x7f79540bfec0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7950009870 tx=0x7f795001c540 comp rx=0 tx=0).stop 2023-10-30T12:43:42.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795c0ea640 1 -- 172.21.15.12:0/63225127 shutdown_connections 2023-10-30T12:43:42.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795c0ea640 1 --2- 172.21.15.12:0/63225127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79540c6b80 0x7f79540bfec0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:42.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795c0ea640 1 -- 172.21.15.12:0/63225127 >> 172.21.15.12:0/63225127 conn(0x7f79540c7910 msgr2=0x7f79540c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795c0ea640 1 -- 172.21.15.12:0/63225127 shutdown_connections 2023-10-30T12:43:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795c0ea640 1 -- 172.21.15.12:0/63225127 wait complete. 2023-10-30T12:43:42.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795c0ea640 1 Processor -- start 2023-10-30T12:43:42.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795c0ea640 1 -- start start 2023-10-30T12:43:42.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795c0ea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79540c6b80 0x7f795413afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:42.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795c0ea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79540c6f60 0x7f795413ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:42.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795c0ea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f795413f4a0 0x7f79541396f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:42.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795c0ea640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7954156490 con 0x7f795413f4a0 2023-10-30T12:43:42.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795c0ea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7954155240 con 0x7f79540c6b80 2023-10-30T12:43:42.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795c0ea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7954154db0 con 0x7f79540c6f60 2023-10-30T12:43:42.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795a687640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f795413f4a0 0x7f79541396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:42.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795a687640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f795413f4a0 0x7f79541396f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36690/0 (socket says 172.21.15.12:36690) 2023-10-30T12:43:42.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f795a687640 1 -- 172.21.15.12:0/4179961507 learned_addr learned my addr 172.21.15.12:0/4179961507 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:42.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f7959685640 1 --2- 172.21.15.12:0/4179961507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79540c6f60 0x7f795413ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:42.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.504+0000 7f7959e86640 1 --2- 172.21.15.12:0/4179961507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79540c6b80 0x7f795413afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:42.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f795a687640 1 -- 172.21.15.12:0/4179961507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79540c6f60 msgr2=0x7f795413ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:42.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f795a687640 1 --2- 172.21.15.12:0/4179961507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79540c6f60 0x7f795413ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f795a687640 1 -- 172.21.15.12:0/4179961507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79540c6b80 msgr2=0x7f795413afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f795a687640 1 --2- 172.21.15.12:0/4179961507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79540c6b80 0x7f795413afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f795a687640 1 -- 172.21.15.12:0/4179961507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79500094d0 con 0x7f795413f4a0 2023-10-30T12:43:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f795a687640 1 --2- 172.21.15.12:0/4179961507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f795413f4a0 0x7f79541396f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f794c002240 tx=0x7f794c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f794a7fc640 1 -- 172.21.15.12:0/4179961507 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c003db0 con 0x7f795413f4a0 2023-10-30T12:43:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f794a7fc640 1 -- 172.21.15.12:0/4179961507 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c00ce30 con 0x7f795413f4a0 2023-10-30T12:43:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f794a7fc640 1 -- 172.21.15.12:0/4179961507 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c005270 con 0x7f795413f4a0 2023-10-30T12:43:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f795c0ea640 1 -- 172.21.15.12:0/4179961507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79540c9050 con 0x7f795413f4a0 2023-10-30T12:43:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f795c0ea640 1 -- 172.21.15.12:0/4179961507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7954154480 con 0x7f795413f4a0 2023-10-30T12:43:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f795c0ea640 1 -- 172.21.15.12:0/4179961507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7954154f20 con 0x7f795413f4a0 2023-10-30T12:43:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f794a7fc640 1 -- 172.21.15.12:0/4179961507 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f794c0053d0 con 0x7f795413f4a0 2023-10-30T12:43:42.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f794a7fc640 1 --2- 172.21.15.12:0/4179961507 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f79380664a0 0x7f7938068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:42.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f794a7fc640 1 -- 172.21.15.12:0/4179961507 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f794c0837d0 con 0x7f795413f4a0 2023-10-30T12:43:42.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.508+0000 7f7959e86640 1 --2- 172.21.15.12:0/4179961507 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f79380664a0 0x7f7938068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:42.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.512+0000 7f7959e86640 1 --2- 172.21.15.12:0/4179961507 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f79380664a0 0x7f7938068960 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f79500af000 tx=0x7f79500b6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:42.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.512+0000 7f794a7fc640 1 -- 172.21.15.12:0/4179961507 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f794c050af0 con 0x7f795413f4a0 2023-10-30T12:43:42.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.700+0000 7f795c0ea640 1 -- 172.21.15.12:0/4179961507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f795404e170 con 0x7f795413f4a0 2023-10-30T12:43:42.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.700+0000 7f794a7fc640 1 -- 172.21.15.12:0/4179961507 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5731 (secure 0 0 0) 0x7f794c017070 con 0x7f795413f4a0 2023-10-30T12:43:42.710 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:42.710 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:35.236829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:42.711 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 41 2023-10-30T12:43:42.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.704+0000 7f795c0ea640 1 -- 172.21.15.12:0/4179961507 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f79380664a0 msgr2=0x7f7938068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:42.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.704+0000 7f795c0ea640 1 --2- 172.21.15.12:0/4179961507 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f79380664a0 0x7f7938068960 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f79500af000 tx=0x7f79500b6040 comp rx=0 tx=0).stop 2023-10-30T12:43:42.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.704+0000 7f795c0ea640 1 -- 172.21.15.12:0/4179961507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f795413f4a0 msgr2=0x7f79541396f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:42.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.704+0000 7f795c0ea640 1 --2- 172.21.15.12:0/4179961507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f795413f4a0 0x7f79541396f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f794c002240 tx=0x7f794c00cbe0 comp rx=0 tx=0).stop 2023-10-30T12:43:42.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.704+0000 7f795c0ea640 1 -- 172.21.15.12:0/4179961507 shutdown_connections 2023-10-30T12:43:42.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.704+0000 7f795c0ea640 1 --2- 172.21.15.12:0/4179961507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f795413f4a0 0x7f79541396f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:42.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.704+0000 7f795c0ea640 1 --2- 172.21.15.12:0/4179961507 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f79380664a0 0x7f7938068960 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:42.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.704+0000 7f795c0ea640 1 --2- 172.21.15.12:0/4179961507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79540c6f60 0x7f795413ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:42.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.704+0000 7f795c0ea640 1 --2- 172.21.15.12:0/4179961507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79540c6b80 0x7f795413afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:42.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.704+0000 7f795c0ea640 1 -- 172.21.15.12:0/4179961507 >> 172.21.15.12:0/4179961507 conn(0x7f79540c7910 msgr2=0x7f79540c1500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:42.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.704+0000 7f795c0ea640 1 -- 172.21.15.12:0/4179961507 shutdown_connections 2023-10-30T12:43:42.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:42.704+0000 7f795c0ea640 1 -- 172.21.15.12:0/4179961507 wait complete. 2023-10-30T12:43:42.730 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2023-10-30T12:43:44.731 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:43:44.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.880+0000 7fce6afc5640 1 Processor -- start 2023-10-30T12:43:44.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.880+0000 7fce6afc5640 1 -- start start 2023-10-30T12:43:44.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.880+0000 7fce6afc5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce640c05d0 0x7fce6413ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:44.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.880+0000 7fce6afc5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce640c0a50 con 0x7fce640c6f60 2023-10-30T12:43:44.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.880+0000 7fce6afc5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6406f6e0 con 0x7fce640c6b80 2023-10-30T12:43:44.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.880+0000 7fce6afc5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6406f9c0 con 0x7fce640c05d0 2023-10-30T12:43:44.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.880+0000 7fce69562640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fce640c6f60 0x7fce641566a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36284/0 (socket says 172.21.15.12:36284) 2023-10-30T12:43:44.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.880+0000 7fce69562640 1 -- 172.21.15.12:0/2836846021 learned_addr learned my addr 172.21.15.12:0/2836846021 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:43:44.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.880+0000 7fce63fff640 1 --2- 172.21.15.12:0/2836846021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce640c05d0 0x7fce6413ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:44.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.880+0000 7fce637fe640 1 -- 172.21.15.12:0/2836846021 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435948670 0 0) 0x7fce640c0a50 con 0x7fce640c6f60 2023-10-30T12:43:44.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.880+0000 7fce637fe640 1 -- 172.21.15.12:0/2836846021 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce54003530 con 0x7fce640c6f60 2023-10-30T12:43:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.880+0000 7fce63fff640 1 -- 172.21.15.12:0/2836846021 >> v1:172.21.15.154:6789/0 conn(0x7fce640c6b80 legacy=0x7fce640bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.880+0000 7fce63fff640 1 -- 172.21.15.12:0/2836846021 >> v1:172.21.15.12:6789/0 conn(0x7fce640c6f60 legacy=0x7fce641566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.880+0000 7fce637fe640 1 -- 172.21.15.12:0/2836846021 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2478176218 0 0) 0x7fce54003530 con 0x7fce640c6f60 2023-10-30T12:43:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.880+0000 7fce63fff640 1 -- 172.21.15.12:0/2836846021 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6406fca0 con 0x7fce640c05d0 2023-10-30T12:43:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.884+0000 7fce63fff640 1 --2- 172.21.15.12:0/2836846021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce640c05d0 0x7fce6413ed90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fce500097f0 tx=0x7fce5001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbadd0eef22c23e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.884+0000 7fce637fe640 1 -- 172.21.15.12:0/2836846021 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c001a00 con 0x7fce640c05d0 2023-10-30T12:43:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.884+0000 7fce637fe640 1 -- 172.21.15.12:0/2836846021 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce500a7050 con 0x7fce640c05d0 2023-10-30T12:43:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.884+0000 7fce637fe640 1 -- 172.21.15.12:0/2836846021 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce500ab600 con 0x7fce640c05d0 2023-10-30T12:43:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.884+0000 7fce6afc5640 1 -- 172.21.15.12:0/2836846021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce640c05d0 msgr2=0x7fce6413ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.884+0000 7fce6afc5640 1 --2- 172.21.15.12:0/2836846021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce640c05d0 0x7fce6413ed90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fce500097f0 tx=0x7fce5001c3e0 comp rx=0 tx=0).stop 2023-10-30T12:43:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.884+0000 7fce6afc5640 1 -- 172.21.15.12:0/2836846021 shutdown_connections 2023-10-30T12:43:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.884+0000 7fce6afc5640 1 --2- 172.21.15.12:0/2836846021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce640c05d0 0x7fce6413ed90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.884+0000 7fce6afc5640 1 -- 172.21.15.12:0/2836846021 >> 172.21.15.12:0/2836846021 conn(0x7fce640c7910 msgr2=0x7fce640c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:44.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.884+0000 7fce6afc5640 1 -- 172.21.15.12:0/2836846021 shutdown_connections 2023-10-30T12:43:44.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.884+0000 7fce6afc5640 1 -- 172.21.15.12:0/2836846021 wait complete. 2023-10-30T12:43:44.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce6afc5640 1 Processor -- start 2023-10-30T12:43:44.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce6afc5640 1 -- start start 2023-10-30T12:43:44.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce6afc5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce640c05d0 0x7fce641591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:44.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce6afc5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce640c6b80 0x7fce64134130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:44.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce6afc5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce640c6f60 0x7fce640c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:44.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce6afc5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce64138e50 con 0x7fce640c05d0 2023-10-30T12:43:44.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce6afc5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce640b0390 con 0x7fce640c6b80 2023-10-30T12:43:44.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce6afc5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6413abc0 con 0x7fce640c6f60 2023-10-30T12:43:44.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce68d61640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce640c05d0 0x7fce641591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:44.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce68d61640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce640c05d0 0x7fce641591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36706/0 (socket says 172.21.15.12:36706) 2023-10-30T12:43:44.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce68d61640 1 -- 172.21.15.12:0/1527794984 learned_addr learned my addr 172.21.15.12:0/1527794984 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:43:44.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce69562640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce640c6f60 0x7fce640c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:44.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce63fff640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce640c6b80 0x7fce64134130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:44.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce68d61640 1 -- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce640c6f60 msgr2=0x7fce640c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:44.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce68d61640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce640c6f60 0x7fce640c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:44.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce68d61640 1 -- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce640c6b80 msgr2=0x7fce64134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:44.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce68d61640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce640c6b80 0x7fce64134130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:44.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce68d61640 1 -- 172.21.15.12:0/1527794984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce50009420 con 0x7fce640c05d0 2023-10-30T12:43:44.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce63fff640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce640c6b80 0x7fce64134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:43:44.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce69562640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce640c6f60 0x7fce640c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:43:44.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce68d61640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce640c05d0 0x7fce641591f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fce5800c530 tx=0x7fce5800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:44.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce617fa640 1 -- 172.21.15.12:0/1527794984 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58005220 con 0x7fce640c05d0 2023-10-30T12:43:44.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce617fa640 1 -- 172.21.15.12:0/1527794984 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce58005380 con 0x7fce640c05d0 2023-10-30T12:43:44.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce617fa640 1 -- 172.21.15.12:0/1527794984 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58007270 con 0x7fce640c05d0 2023-10-30T12:43:44.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce6afc5640 1 -- 172.21.15.12:0/1527794984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce64138bf0 con 0x7fce640c05d0 2023-10-30T12:43:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.888+0000 7fce6afc5640 1 -- 172.21.15.12:0/1527794984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce640caab0 con 0x7fce640c05d0 2023-10-30T12:43:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.892+0000 7fce6afc5640 1 -- 172.21.15.12:0/1527794984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6404e170 con 0x7fce640c05d0 2023-10-30T12:43:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.892+0000 7fce617fa640 1 -- 172.21.15.12:0/1527794984 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fce580073d0 con 0x7fce640c05d0 2023-10-30T12:43:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.892+0000 7fce617fa640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fce480664d0 0x7fce48068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:43:44.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.892+0000 7fce617fa640 1 -- 172.21.15.12:0/1527794984 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fce58083940 con 0x7fce640c05d0 2023-10-30T12:43:44.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.892+0000 7fce63fff640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fce480664d0 0x7fce48068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:43:44.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.892+0000 7fce63fff640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fce480664d0 0x7fce48068990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fce500ad000 tx=0x7fce500a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:43:44.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:44.896+0000 7fce617fa640 1 -- 172.21.15.12:0/1527794984 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fce58050c30 con 0x7fce640c05d0 2023-10-30T12:43:45.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:45.088+0000 7fce6afc5640 1 -- 172.21.15.12:0/1527794984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce640c64b0 con 0x7fce640c05d0 2023-10-30T12:43:45.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:45.092+0000 7fce617fa640 1 -- 172.21.15.12:0/1527794984 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+6525 (secure 0 0 0) 0x7fce58054250 con 0x7fce640c05d0 2023-10-30T12:43:45.098 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:43:45.099 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:35.236829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:43:45.099 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 42 2023-10-30T12:43:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:45.092+0000 7fce6afc5640 1 -- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fce480664d0 msgr2=0x7fce48068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:45.092+0000 7fce6afc5640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fce480664d0 0x7fce48068990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fce500ad000 tx=0x7fce500a6000 comp rx=0 tx=0).stop 2023-10-30T12:43:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:45.092+0000 7fce6afc5640 1 -- 172.21.15.12:0/1527794984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce640c05d0 msgr2=0x7fce641591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:43:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:45.092+0000 7fce6afc5640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce640c05d0 0x7fce641591f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fce5800c530 tx=0x7fce5800c9f0 comp rx=0 tx=0).stop 2023-10-30T12:43:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:45.092+0000 7fce6afc5640 1 -- 172.21.15.12:0/1527794984 shutdown_connections 2023-10-30T12:43:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:45.092+0000 7fce6afc5640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fce480664d0 0x7fce48068990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:45.092+0000 7fce6afc5640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fce640c6f60 0x7fce640c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:45.092+0000 7fce6afc5640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce640c6b80 0x7fce64134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:45.092+0000 7fce6afc5640 1 --2- 172.21.15.12:0/1527794984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce640c05d0 0x7fce641591f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:43:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:45.092+0000 7fce6afc5640 1 -- 172.21.15.12:0/1527794984 >> 172.21.15.12:0/1527794984 conn(0x7fce640c7910 msgr2=0x7fce6406a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:43:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:45.092+0000 7fce6afc5640 1 -- 172.21.15.12:0/1527794984 shutdown_connections 2023-10-30T12:43:45.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:43:45.092+0000 7fce6afc5640 1 -- 172.21.15.12:0/1527794984 wait complete. 2023-10-30T12:43:45.119 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2023-10-30T12:43:45.119 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 32 secs before thrashing 2023-10-30T12:44:05.129 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:44:05.133 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:44:17.121 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:17.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.264+0000 7fac6ab3e640 1 Processor -- start 2023-10-30T12:44:17.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.264+0000 7fac6ab3e640 1 -- start start 2023-10-30T12:44:17.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.264+0000 7fac6ab3e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac64153d70 0x7fac6406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:17.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.264+0000 7fac6ab3e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fac640c8490 0x7fac640c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:17.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.264+0000 7fac6ab3e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6406afa0 con 0x7fac6406e470 2023-10-30T12:44:17.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.264+0000 7fac6ab3e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6406d780 con 0x7fac64153d70 2023-10-30T12:44:17.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.264+0000 7fac6ab3e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac64154150 con 0x7fac640c8490 2023-10-30T12:44:17.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.264+0000 7fac688da640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac64153d70 0x7fac6406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:17.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.264+0000 7fac688da640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac64153d70 0x7fac6406ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43430/0 (socket says 172.21.15.12:43430) 2023-10-30T12:44:17.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.264+0000 7fac688da640 1 -- 172.21.15.12:0/3818499290 learned_addr learned my addr 172.21.15.12:0/3818499290 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:17.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.264+0000 7fac690db640 1 --2- 172.21.15.12:0/3818499290 >> v2:172.21.15.154:3300/0 conn(0x7fac640c8490 0x7fac640c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:17.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.264+0000 7fac688da640 1 -- 172.21.15.12:0/3818499290 >> v2:172.21.15.154:3300/0 conn(0x7fac640c8490 msgr2=0x7fac640c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:17.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.264+0000 7fac688da640 1 --2- 172.21.15.12:0/3818499290 >> v2:172.21.15.154:3300/0 conn(0x7fac640c8490 0x7fac640c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.264+0000 7fac688da640 1 -- 172.21.15.12:0/3818499290 >> v1:172.21.15.154:6789/0 conn(0x7fac6406e470 legacy=0x7fac640cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:17.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.264+0000 7fac688da640 1 -- 172.21.15.12:0/3818499290 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6406e030 con 0x7fac64153d70 2023-10-30T12:44:17.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.268+0000 7fac688da640 1 --2- 172.21.15.12:0/3818499290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac64153d70 0x7fac6406ca80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fac640cddf0 tx=0x7fac5801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2699cce3dec7f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:17.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.268+0000 7fac637fe640 1 -- 172.21.15.12:0/3818499290 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac580aa430 con 0x7fac64153d70 2023-10-30T12:44:17.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.268+0000 7fac637fe640 1 -- 172.21.15.12:0/3818499290 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac580aa590 con 0x7fac64153d70 2023-10-30T12:44:17.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.268+0000 7fac637fe640 1 -- 172.21.15.12:0/3818499290 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac580aa920 con 0x7fac64153d70 2023-10-30T12:44:17.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.268+0000 7fac6ab3e640 1 -- 172.21.15.12:0/3818499290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac64153d70 msgr2=0x7fac6406ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:17.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.268+0000 7fac6ab3e640 1 --2- 172.21.15.12:0/3818499290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac64153d70 0x7fac6406ca80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fac640cddf0 tx=0x7fac5801c4d0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.268+0000 7fac6ab3e640 1 -- 172.21.15.12:0/3818499290 shutdown_connections 2023-10-30T12:44:17.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.268+0000 7fac6ab3e640 1 --2- 172.21.15.12:0/3818499290 >> v2:172.21.15.154:3300/0 conn(0x7fac640c8490 0x7fac640c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.268+0000 7fac6ab3e640 1 --2- 172.21.15.12:0/3818499290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac64153d70 0x7fac6406ca80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.268+0000 7fac6ab3e640 1 -- 172.21.15.12:0/3818499290 >> 172.21.15.12:0/3818499290 conn(0x7fac64071ea0 msgr2=0x7fac64065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:17.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.268+0000 7fac6ab3e640 1 -- 172.21.15.12:0/3818499290 shutdown_connections 2023-10-30T12:44:17.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.268+0000 7fac6ab3e640 1 -- 172.21.15.12:0/3818499290 wait complete. 2023-10-30T12:44:17.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac6ab3e640 1 Processor -- start 2023-10-30T12:44:17.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac6ab3e640 1 -- start start 2023-10-30T12:44:17.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac6ab3e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac6406e470 0x7fac64148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:17.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac6ab3e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac640c8490 0x7fac64140a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:17.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac6ab3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac64153d70 0x7fac64145fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:17.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac6ab3e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac641f2900 con 0x7fac6406e470 2023-10-30T12:44:17.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac6ab3e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac640ccbd0 con 0x7fac640c8490 2023-10-30T12:44:17.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac6ab3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac641f3160 con 0x7fac64153d70 2023-10-30T12:44:17.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac688da640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac6406e470 0x7fac64148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:17.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac688da640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac6406e470 0x7fac64148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43438/0 (socket says 172.21.15.12:43438) 2023-10-30T12:44:17.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac688da640 1 -- 172.21.15.12:0/1741061591 learned_addr learned my addr 172.21.15.12:0/1741061591 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:17.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac63fff640 1 --2- 172.21.15.12:0/1741061591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac640c8490 0x7fac64140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:17.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac690db640 1 --2- 172.21.15.12:0/1741061591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac64153d70 0x7fac64145fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:17.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac688da640 1 -- 172.21.15.12:0/1741061591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac64153d70 msgr2=0x7fac64145fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:17.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac688da640 1 --2- 172.21.15.12:0/1741061591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac64153d70 0x7fac64145fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac688da640 1 -- 172.21.15.12:0/1741061591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac640c8490 msgr2=0x7fac64140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:17.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac688da640 1 --2- 172.21.15.12:0/1741061591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac640c8490 0x7fac64140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac688da640 1 -- 172.21.15.12:0/1741061591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac580094d0 con 0x7fac6406e470 2023-10-30T12:44:17.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac688da640 1 --2- 172.21.15.12:0/1741061591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac6406e470 0x7fac64148ae0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fac580190e0 tx=0x7fac5801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:17.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac617fa640 1 -- 172.21.15.12:0/1741061591 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac580b3800 con 0x7fac6406e470 2023-10-30T12:44:17.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac617fa640 1 -- 172.21.15.12:0/1741061591 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac580b3960 con 0x7fac6406e470 2023-10-30T12:44:17.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac617fa640 1 -- 172.21.15.12:0/1741061591 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac580b3cf0 con 0x7fac6406e470 2023-10-30T12:44:17.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac6ab3e640 1 -- 172.21.15.12:0/1741061591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac641503b0 con 0x7fac6406e470 2023-10-30T12:44:17.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.272+0000 7fac6ab3e640 1 -- 172.21.15.12:0/1741061591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac641480d0 con 0x7fac6406e470 2023-10-30T12:44:17.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.276+0000 7fac6ab3e640 1 -- 172.21.15.12:0/1741061591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6404e170 con 0x7fac6406e470 2023-10-30T12:44:17.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.276+0000 7fac617fa640 1 -- 172.21.15.12:0/1741061591 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fac580b2380 con 0x7fac6406e470 2023-10-30T12:44:17.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.276+0000 7fac617fa640 1 --2- 172.21.15.12:0/1741061591 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac480664a0 0x7fac48068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:17.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.276+0000 7fac617fa640 1 -- 172.21.15.12:0/1741061591 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fac5811e500 con 0x7fac6406e470 2023-10-30T12:44:17.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.276+0000 7fac63fff640 1 --2- 172.21.15.12:0/1741061591 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac480664a0 0x7fac48068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:17.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.276+0000 7fac63fff640 1 --2- 172.21.15.12:0/1741061591 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac480664a0 0x7fac48068960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fac641519a0 tx=0x7fac50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:17.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.280+0000 7fac617fa640 1 -- 172.21.15.12:0/1741061591 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fac580eb8b0 con 0x7fac6406e470 2023-10-30T12:44:17.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.472+0000 7fac6ab3e640 1 -- 172.21.15.12:0/1741061591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac64157360 con 0x7fac6406e470 2023-10-30T12:44:17.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.472+0000 7fac617fa640 1 -- 172.21.15.12:0/1741061591 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6527 (secure 0 0 0) 0x7fac580eeed0 con 0x7fac6406e470 2023-10-30T12:44:17.480 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:17.480 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:46.382577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:17.481 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 43 2023-10-30T12:44:17.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.472+0000 7fac6ab3e640 1 -- 172.21.15.12:0/1741061591 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac480664a0 msgr2=0x7fac48068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:17.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.472+0000 7fac6ab3e640 1 --2- 172.21.15.12:0/1741061591 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac480664a0 0x7fac48068960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fac641519a0 tx=0x7fac50009040 comp rx=0 tx=0).stop 2023-10-30T12:44:17.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.472+0000 7fac6ab3e640 1 -- 172.21.15.12:0/1741061591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac6406e470 msgr2=0x7fac64148ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:17.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.472+0000 7fac6ab3e640 1 --2- 172.21.15.12:0/1741061591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac6406e470 0x7fac64148ae0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fac580190e0 tx=0x7fac5801ad10 comp rx=0 tx=0).stop 2023-10-30T12:44:17.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.476+0000 7fac6ab3e640 1 -- 172.21.15.12:0/1741061591 shutdown_connections 2023-10-30T12:44:17.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.476+0000 7fac6ab3e640 1 --2- 172.21.15.12:0/1741061591 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac480664a0 0x7fac48068960 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.476+0000 7fac6ab3e640 1 --2- 172.21.15.12:0/1741061591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac64153d70 0x7fac64145fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.476+0000 7fac6ab3e640 1 --2- 172.21.15.12:0/1741061591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac640c8490 0x7fac64140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.476+0000 7fac6ab3e640 1 --2- 172.21.15.12:0/1741061591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac6406e470 0x7fac64148ae0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.476+0000 7fac6ab3e640 1 -- 172.21.15.12:0/1741061591 >> 172.21.15.12:0/1741061591 conn(0x7fac64071ea0 msgr2=0x7fac641473d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:17.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.476+0000 7fac6ab3e640 1 -- 172.21.15.12:0/1741061591 shutdown_connections 2023-10-30T12:44:17.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.476+0000 7fac6ab3e640 1 -- 172.21.15.12:0/1741061591 wait complete. 2023-10-30T12:44:17.501 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=0) 2023-10-30T12:44:17.501 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-10-30T12:44:17.501 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:44:17.521 INFO:tasks.ceph.mds.e:Stopped 2023-10-30T12:44:17.522 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.704+0000 7f59f9980640 1 Processor -- start 2023-10-30T12:44:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.704+0000 7f59f9980640 1 -- start start 2023-10-30T12:44:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.704+0000 7f59f9980640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f59f41538a0 0x7f59f415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f9980640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f59f40ca0e0 0x7f59f415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f9980640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f40cb800 con 0x7f59f40cf160 2023-10-30T12:44:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f9980640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f415dc30 con 0x7f59f41538a0 2023-10-30T12:44:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f9980640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f40cb330 con 0x7f59f40ca0e0 2023-10-30T12:44:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f37fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f59f40cf160 0x7f59f40cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39556/0 (socket says 172.21.15.12:39556) 2023-10-30T12:44:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f37fe640 1 -- 172.21.15.12:0/2105163758 learned_addr learned my addr 172.21.15.12:0/2105163758 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:44:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f2ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f59f41538a0 0x7f59f415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f27fc640 1 --2- 172.21.15.12:0/2105163758 >> v2:172.21.15.154:3300/0 conn(0x7f59f40ca0e0 0x7f59f415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f2ffd640 1 -- 172.21.15.12:0/2105163758 >> v2:172.21.15.154:3300/0 conn(0x7f59f40ca0e0 msgr2=0x7f59f415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f2ffd640 1 --2- 172.21.15.12:0/2105163758 >> v2:172.21.15.154:3300/0 conn(0x7f59f40ca0e0 0x7f59f415a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f2ffd640 1 -- 172.21.15.12:0/2105163758 >> v1:172.21.15.12:6789/0 conn(0x7f59f40cf160 legacy=0x7f59f40cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f2ffd640 1 -- 172.21.15.12:0/2105163758 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f40cad70 con 0x7f59f41538a0 2023-10-30T12:44:17.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f2ffd640 1 --2- 172.21.15.12:0/2105163758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f41538a0 0x7f59f415d6f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f59f40b30a0 tx=0x7f59e401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e53e5f7b2eec273 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:17.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f1ffb640 1 -- 172.21.15.12:0/2105163758 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e40a8950 con 0x7f59f41538a0 2023-10-30T12:44:17.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f1ffb640 1 -- 172.21.15.12:0/2105163758 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e40a8ab0 con 0x7f59f41538a0 2023-10-30T12:44:17.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.708+0000 7f59f1ffb640 1 -- 172.21.15.12:0/2105163758 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e40a8e40 con 0x7f59f41538a0 2023-10-30T12:44:17.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 -- 172.21.15.12:0/2105163758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f41538a0 msgr2=0x7f59f415d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:17.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 --2- 172.21.15.12:0/2105163758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f41538a0 0x7f59f415d6f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f59f40b30a0 tx=0x7f59e401ec50 comp rx=0 tx=0).stop 2023-10-30T12:44:17.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 -- 172.21.15.12:0/2105163758 shutdown_connections 2023-10-30T12:44:17.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 --2- 172.21.15.12:0/2105163758 >> v2:172.21.15.154:3300/0 conn(0x7f59f40ca0e0 0x7f59f415a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 --2- 172.21.15.12:0/2105163758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f41538a0 0x7f59f415d6f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 -- 172.21.15.12:0/2105163758 >> 172.21.15.12:0/2105163758 conn(0x7f59f406bfe0 msgr2=0x7f59f406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 -- 172.21.15.12:0/2105163758 shutdown_connections 2023-10-30T12:44:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 -- 172.21.15.12:0/2105163758 wait complete. 2023-10-30T12:44:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 Processor -- start 2023-10-30T12:44:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 -- start start 2023-10-30T12:44:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59f40ca0e0 0x7f59f4158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f40cf160 0x7f59f40c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59f41538a0 0x7f59f41ef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f41e4db0 con 0x7f59f40cf160 2023-10-30T12:44:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f41ef520 con 0x7f59f40ca0e0 2023-10-30T12:44:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f9980640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f41f2960 con 0x7f59f41538a0 2023-10-30T12:44:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f40cf160 0x7f59f40c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f40cf160 0x7f59f40c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43464/0 (socket says 172.21.15.12:43464) 2023-10-30T12:44:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f27fc640 1 -- 172.21.15.12:0/2321094261 learned_addr learned my addr 172.21.15.12:0/2321094261 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f2ffd640 1 --2- 172.21.15.12:0/2321094261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59f40ca0e0 0x7f59f4158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.712+0000 7f59f37fe640 1 --2- 172.21.15.12:0/2321094261 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59f41538a0 0x7f59f41ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59f27fc640 1 -- 172.21.15.12:0/2321094261 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59f41538a0 msgr2=0x7f59f41ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59f27fc640 1 --2- 172.21.15.12:0/2321094261 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59f41538a0 0x7f59f41ef9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59f27fc640 1 -- 172.21.15.12:0/2321094261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59f40ca0e0 msgr2=0x7f59f4158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59f27fc640 1 --2- 172.21.15.12:0/2321094261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59f40ca0e0 0x7f59f4158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59f27fc640 1 -- 172.21.15.12:0/2321094261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e4009420 con 0x7f59f40cf160 2023-10-30T12:44:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59f2ffd640 1 --2- 172.21.15.12:0/2321094261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59f40ca0e0 0x7f59f4158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:44:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59f27fc640 1 --2- 172.21.15.12:0/2321094261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f40cf160 0x7f59f40c5230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f59e800d800 tx=0x7f59e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59dbfff640 1 -- 172.21.15.12:0/2321094261 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e800b6c0 con 0x7f59f40cf160 2023-10-30T12:44:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59dbfff640 1 -- 172.21.15.12:0/2321094261 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e800b820 con 0x7f59f40cf160 2023-10-30T12:44:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59f9980640 1 -- 172.21.15.12:0/2321094261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f41f2c20 con 0x7f59f40cf160 2023-10-30T12:44:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59dbfff640 1 -- 172.21.15.12:0/2321094261 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e800bb90 con 0x7f59f40cf160 2023-10-30T12:44:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59f9980640 1 -- 172.21.15.12:0/2321094261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f41f31e0 con 0x7f59f40cf160 2023-10-30T12:44:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59f9980640 1 -- 172.21.15.12:0/2321094261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f40b2a50 con 0x7f59f40cf160 2023-10-30T12:44:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59dbfff640 1 -- 172.21.15.12:0/2321094261 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f59e800bcf0 con 0x7f59f40cf160 2023-10-30T12:44:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59dbfff640 1 --2- 172.21.15.12:0/2321094261 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f59e00664d0 0x7f59e0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59dbfff640 1 -- 172.21.15.12:0/2321094261 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f59e80845c0 con 0x7f59f40cf160 2023-10-30T12:44:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.716+0000 7f59f2ffd640 1 --2- 172.21.15.12:0/2321094261 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f59e00664d0 0x7f59e0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.720+0000 7f59f2ffd640 1 --2- 172.21.15.12:0/2321094261 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f59e00664d0 0x7f59e0068990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f59e40b2000 tx=0x7f59e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.720+0000 7f59dbfff640 1 -- 172.21.15.12:0/2321094261 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f59e8051820 con 0x7f59f40cf160 2023-10-30T12:44:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.908+0000 7f59f9980640 1 -- 172.21.15.12:0/2321094261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59f41ef660 con 0x7f59f40cf160 2023-10-30T12:44:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.908+0000 7f59dbfff640 1 -- 172.21.15.12:0/2321094261 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6527 (secure 0 0 0) 0x7f59e8054e40 con 0x7f59f40cf160 2023-10-30T12:44:17.917 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:17.917 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:46.382577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:17.917 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 43 2023-10-30T12:44:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.912+0000 7f59f9980640 1 -- 172.21.15.12:0/2321094261 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f59e00664d0 msgr2=0x7f59e0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.912+0000 7f59f9980640 1 --2- 172.21.15.12:0/2321094261 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f59e00664d0 0x7f59e0068990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f59e40b2000 tx=0x7f59e4008040 comp rx=0 tx=0).stop 2023-10-30T12:44:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.912+0000 7f59f9980640 1 -- 172.21.15.12:0/2321094261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f40cf160 msgr2=0x7f59f40c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.912+0000 7f59f9980640 1 --2- 172.21.15.12:0/2321094261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f40cf160 0x7f59f40c5230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f59e800d800 tx=0x7f59e800dcc0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.912+0000 7f59f9980640 1 -- 172.21.15.12:0/2321094261 shutdown_connections 2023-10-30T12:44:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.912+0000 7f59f9980640 1 --2- 172.21.15.12:0/2321094261 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f59e00664d0 0x7f59e0068990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.912+0000 7f59f9980640 1 --2- 172.21.15.12:0/2321094261 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59f41538a0 0x7f59f41ef9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.912+0000 7f59f9980640 1 --2- 172.21.15.12:0/2321094261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f40cf160 0x7f59f40c5230 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.912+0000 7f59f9980640 1 --2- 172.21.15.12:0/2321094261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59f40ca0e0 0x7f59f4158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:17.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.912+0000 7f59f9980640 1 -- 172.21.15.12:0/2321094261 >> 172.21.15.12:0/2321094261 conn(0x7f59f406bfe0 msgr2=0x7f59f4151f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:17.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.912+0000 7f59f9980640 1 -- 172.21.15.12:0/2321094261 shutdown_connections 2023-10-30T12:44:17.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:17.912+0000 7f59f9980640 1 -- 172.21.15.12:0/2321094261 wait complete. 2023-10-30T12:44:17.938 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:44:19.940 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d99b5640 1 Processor -- start 2023-10-30T12:44:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d99b5640 1 -- start start 2023-10-30T12:44:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d99b5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2d41538f0 0x7ff2d4159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d99b5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff2d415cd50 0x7ff2d40c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:20.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d99b5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d415d300 con 0x7ff2d40cefa0 2023-10-30T12:44:20.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d99b5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d415d5e0 con 0x7ff2d415cd50 2023-10-30T12:44:20.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d99b5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d415d8c0 con 0x7ff2d41538f0 2023-10-30T12:44:20.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d3fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff2d415cd50 0x7ff2d40c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:20.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d89b3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2d41538f0 0x7ff2d4159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:20.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d91b4640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff2d40cefa0 0x7ff2d40cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:37578/0 (socket says 172.21.15.12:37578) 2023-10-30T12:44:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d3fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff2d415cd50 0x7ff2d40c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:45518/0 (socket says 172.21.15.12:45518) 2023-10-30T12:44:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d91b4640 1 -- 172.21.15.12:0/371406230 learned_addr learned my addr 172.21.15.12:0/371406230 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:44:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d3fff640 1 -- 172.21.15.12:0/371406230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2d41538f0 msgr2=0x7ff2d4159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:20.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d3fff640 1 --2- 172.21.15.12:0/371406230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2d41538f0 0x7ff2d4159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:20.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d3fff640 1 -- 172.21.15.12:0/371406230 >> v1:172.21.15.154:6789/0 conn(0x7ff2d40cefa0 legacy=0x7ff2d40cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:20.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.092+0000 7ff2d3fff640 1 -- 172.21.15.12:0/371406230 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d415dba0 con 0x7ff2d415cd50 2023-10-30T12:44:20.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.096+0000 7ff2d3fff640 1 --2- 172.21.15.12:0/371406230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2d415cd50 0x7ff2d40c9f90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00097c0 tx=0x7ff2c001ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54dc84e65fc1bbc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:20.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.096+0000 7ff2d37fe640 1 -- 172.21.15.12:0/371406230 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00a8970 con 0x7ff2d415cd50 2023-10-30T12:44:20.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.096+0000 7ff2d37fe640 1 -- 172.21.15.12:0/371406230 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c00a8ad0 con 0x7ff2d415cd50 2023-10-30T12:44:20.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.096+0000 7ff2d37fe640 1 -- 172.21.15.12:0/371406230 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00a8e60 con 0x7ff2d415cd50 2023-10-30T12:44:20.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.096+0000 7ff2d99b5640 1 -- 172.21.15.12:0/371406230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2d415cd50 msgr2=0x7ff2d40c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:20.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.096+0000 7ff2d99b5640 1 --2- 172.21.15.12:0/371406230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2d415cd50 0x7ff2d40c9f90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00097c0 tx=0x7ff2c001ec70 comp rx=0 tx=0).stop 2023-10-30T12:44:20.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.096+0000 7ff2d99b5640 1 -- 172.21.15.12:0/371406230 shutdown_connections 2023-10-30T12:44:20.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.096+0000 7ff2d99b5640 1 --2- 172.21.15.12:0/371406230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2d415cd50 0x7ff2d40c9f90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:20.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.096+0000 7ff2d99b5640 1 --2- 172.21.15.12:0/371406230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2d41538f0 0x7ff2d4159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:20.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.096+0000 7ff2d99b5640 1 -- 172.21.15.12:0/371406230 >> 172.21.15.12:0/371406230 conn(0x7ff2d406a640 msgr2=0x7ff2d4065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:20.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.096+0000 7ff2d99b5640 1 -- 172.21.15.12:0/371406230 shutdown_connections 2023-10-30T12:44:20.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.096+0000 7ff2d99b5640 1 -- 172.21.15.12:0/371406230 wait complete. 2023-10-30T12:44:20.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d99b5640 1 Processor -- start 2023-10-30T12:44:20.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d99b5640 1 -- start start 2023-10-30T12:44:20.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d99b5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2d40cefa0 0x7ff2d4144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:20.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d99b5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2d41538f0 0x7ff2d41eef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:20.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d99b5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2d415cd50 0x7ff2d413f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:20.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d99b5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d41f2e20 con 0x7ff2d40cefa0 2023-10-30T12:44:20.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d99b5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40b3290 con 0x7ff2d415cd50 2023-10-30T12:44:20.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d99b5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40b3570 con 0x7ff2d41538f0 2023-10-30T12:44:20.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d89b3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2d40cefa0 0x7ff2d4144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:20.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d89b3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2d40cefa0 0x7ff2d4144ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43466/0 (socket says 172.21.15.12:43466) 2023-10-30T12:44:20.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d89b3640 1 -- 172.21.15.12:0/2805399647 learned_addr learned my addr 172.21.15.12:0/2805399647 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:20.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d3fff640 1 --2- 172.21.15.12:0/2805399647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2d41538f0 0x7ff2d41eef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:20.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d91b4640 1 --2- 172.21.15.12:0/2805399647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2d415cd50 0x7ff2d413f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:20.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d89b3640 1 -- 172.21.15.12:0/2805399647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2d41538f0 msgr2=0x7ff2d41eef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:20.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d89b3640 1 --2- 172.21.15.12:0/2805399647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2d41538f0 0x7ff2d41eef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:20.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d89b3640 1 -- 172.21.15.12:0/2805399647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2d415cd50 msgr2=0x7ff2d413f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:20.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d89b3640 1 --2- 172.21.15.12:0/2805399647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2d415cd50 0x7ff2d413f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:20.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d89b3640 1 -- 172.21.15.12:0/2805399647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c0009420 con 0x7ff2d40cefa0 2023-10-30T12:44:20.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d89b3640 1 --2- 172.21.15.12:0/2805399647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2d40cefa0 0x7ff2d4144ee0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800e930 tx=0x7ff2c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:20.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d17fa640 1 -- 172.21.15.12:0/2805399647 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8009690 con 0x7ff2d40cefa0 2023-10-30T12:44:20.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d17fa640 1 -- 172.21.15.12:0/2805399647 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c800ca00 con 0x7ff2d40cefa0 2023-10-30T12:44:20.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d99b5640 1 -- 172.21.15.12:0/2805399647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d41f24f0 con 0x7ff2d40cefa0 2023-10-30T12:44:20.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d17fa640 1 -- 172.21.15.12:0/2805399647 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c800cd70 con 0x7ff2d40cefa0 2023-10-30T12:44:20.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.100+0000 7ff2d99b5640 1 -- 172.21.15.12:0/2805399647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d41f2ab0 con 0x7ff2d40cefa0 2023-10-30T12:44:20.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.104+0000 7ff2d99b5640 1 -- 172.21.15.12:0/2805399647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d404e170 con 0x7ff2d40cefa0 2023-10-30T12:44:20.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.104+0000 7ff2d17fa640 1 -- 172.21.15.12:0/2805399647 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff2c8013470 con 0x7ff2d40cefa0 2023-10-30T12:44:20.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.104+0000 7ff2d17fa640 1 --2- 172.21.15.12:0/2805399647 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2b80664d0 0x7ff2b8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:20.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.104+0000 7ff2d17fa640 1 -- 172.21.15.12:0/2805399647 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7ff2c8084190 con 0x7ff2d40cefa0 2023-10-30T12:44:20.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.104+0000 7ff2d3fff640 1 --2- 172.21.15.12:0/2805399647 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2b80664d0 0x7ff2b8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:20.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.104+0000 7ff2d3fff640 1 --2- 172.21.15.12:0/2805399647 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2b80664d0 0x7ff2b8068990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00b2000 tx=0x7ff2c00aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:20.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.108+0000 7ff2d17fa640 1 -- 172.21.15.12:0/2805399647 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff2c80515f0 con 0x7ff2d40cefa0 2023-10-30T12:44:20.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.300+0000 7ff2d99b5640 1 -- 172.21.15.12:0/2805399647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2d40c5690 con 0x7ff2d40cefa0 2023-10-30T12:44:20.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.300+0000 7ff2d17fa640 1 -- 172.21.15.12:0/2805399647 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6527 (secure 0 0 0) 0x7ff2c8054c10 con 0x7ff2d40cefa0 2023-10-30T12:44:20.309 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:20.309 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:46.382577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:20.311 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 43 2023-10-30T12:44:20.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.304+0000 7ff2d99b5640 1 -- 172.21.15.12:0/2805399647 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2b80664d0 msgr2=0x7ff2b8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:20.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.304+0000 7ff2d99b5640 1 --2- 172.21.15.12:0/2805399647 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2b80664d0 0x7ff2b8068990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00b2000 tx=0x7ff2c00aa070 comp rx=0 tx=0).stop 2023-10-30T12:44:20.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.304+0000 7ff2d99b5640 1 -- 172.21.15.12:0/2805399647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2d40cefa0 msgr2=0x7ff2d4144ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:20.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.304+0000 7ff2d99b5640 1 --2- 172.21.15.12:0/2805399647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2d40cefa0 0x7ff2d4144ee0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800e930 tx=0x7ff2c800edf0 comp rx=0 tx=0).stop 2023-10-30T12:44:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.304+0000 7ff2d99b5640 1 -- 172.21.15.12:0/2805399647 shutdown_connections 2023-10-30T12:44:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.304+0000 7ff2d99b5640 1 --2- 172.21.15.12:0/2805399647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2d415cd50 0x7ff2d413f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.304+0000 7ff2d99b5640 1 --2- 172.21.15.12:0/2805399647 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2b80664d0 0x7ff2b8068990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:20.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.304+0000 7ff2d99b5640 1 --2- 172.21.15.12:0/2805399647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2d41538f0 0x7ff2d41eef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:20.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.304+0000 7ff2d99b5640 1 --2- 172.21.15.12:0/2805399647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2d40cefa0 0x7ff2d4144ee0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:20.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.304+0000 7ff2d99b5640 1 -- 172.21.15.12:0/2805399647 >> 172.21.15.12:0/2805399647 conn(0x7ff2d406a640 msgr2=0x7ff2d40cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:20.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.304+0000 7ff2d99b5640 1 -- 172.21.15.12:0/2805399647 shutdown_connections 2023-10-30T12:44:20.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:20.304+0000 7ff2d99b5640 1 -- 172.21.15.12:0/2805399647 wait complete. 2023-10-30T12:44:20.330 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:44:22.331 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:22.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.480+0000 7f6e1e609640 1 Processor -- start 2023-10-30T12:44:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.480+0000 7f6e1e609640 1 -- start start 2023-10-30T12:44:22.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.484+0000 7f6e1e609640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e180cb550 0x7f6e180cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:22.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.484+0000 7f6e1e609640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6e18071250 0x7f6e180716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:22.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.484+0000 7f6e1e609640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e18071be0 con 0x7f6e1814fd50 2023-10-30T12:44:22.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.484+0000 7f6e1e609640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e181560a0 con 0x7f6e18071250 2023-10-30T12:44:22.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.484+0000 7f6e1e609640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e18156380 con 0x7f6e180cb550 2023-10-30T12:44:22.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.484+0000 7f6e177fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e180cb550 0x7f6e180cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:22.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.484+0000 7f6e177fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e180cb550 0x7f6e180cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:33392/0 (socket says 172.21.15.12:33392) 2023-10-30T12:44:22.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.484+0000 7f6e177fe640 1 -- 172.21.15.12:0/3186457476 learned_addr learned my addr 172.21.15.12:0/3186457476 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:22.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.484+0000 7f6e1cba6640 1 --2- 172.21.15.12:0/3186457476 >> v2:172.21.15.154:3300/0 conn(0x7f6e18071250 0x7f6e180716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:22.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.484+0000 7f6e177fe640 1 -- 172.21.15.12:0/3186457476 >> v2:172.21.15.154:3300/0 conn(0x7f6e18071250 msgr2=0x7f6e180716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:22.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.484+0000 7f6e177fe640 1 --2- 172.21.15.12:0/3186457476 >> v2:172.21.15.154:3300/0 conn(0x7f6e18071250 0x7f6e180716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:22.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.484+0000 7f6e177fe640 1 -- 172.21.15.12:0/3186457476 >> v1:172.21.15.154:6789/0 conn(0x7f6e1814fd50 legacy=0x7f6e180c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:22.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.484+0000 7f6e177fe640 1 -- 172.21.15.12:0/3186457476 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e18156660 con 0x7f6e180cb550 2023-10-30T12:44:22.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.484+0000 7f6e1cba6640 1 --2- 172.21.15.12:0/3186457476 >> v2:172.21.15.154:3300/0 conn(0x7f6e18071250 0x7f6e180716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:44:22.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.488+0000 7f6e177fe640 1 --2- 172.21.15.12:0/3186457476 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e180cb550 0x7f6e180cb9b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c006fd0 tx=0x7f6e0c01e9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c320a3911b25d38e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:22.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.488+0000 7f6e16ffd640 1 -- 172.21.15.12:0/3186457476 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c004230 con 0x7f6e180cb550 2023-10-30T12:44:22.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.488+0000 7f6e16ffd640 1 -- 172.21.15.12:0/3186457476 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0c0ab070 con 0x7f6e180cb550 2023-10-30T12:44:22.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.488+0000 7f6e16ffd640 1 -- 172.21.15.12:0/3186457476 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c0a77b0 con 0x7f6e180cb550 2023-10-30T12:44:22.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.488+0000 7f6e1e609640 1 -- 172.21.15.12:0/3186457476 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e180cb550 msgr2=0x7f6e180cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:22.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.488+0000 7f6e1e609640 1 --2- 172.21.15.12:0/3186457476 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e180cb550 0x7f6e180cb9b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c006fd0 tx=0x7f6e0c01e9a0 comp rx=0 tx=0).stop 2023-10-30T12:44:22.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.488+0000 7f6e1e609640 1 -- 172.21.15.12:0/3186457476 shutdown_connections 2023-10-30T12:44:22.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.488+0000 7f6e1e609640 1 --2- 172.21.15.12:0/3186457476 >> v2:172.21.15.154:3300/0 conn(0x7f6e18071250 0x7f6e180716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:22.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.488+0000 7f6e1e609640 1 --2- 172.21.15.12:0/3186457476 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e180cb550 0x7f6e180cb9b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:22.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.488+0000 7f6e1e609640 1 -- 172.21.15.12:0/3186457476 >> 172.21.15.12:0/3186457476 conn(0x7f6e1806c6e0 msgr2=0x7f6e18069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:22.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.488+0000 7f6e1e609640 1 -- 172.21.15.12:0/3186457476 shutdown_connections 2023-10-30T12:44:22.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.488+0000 7f6e1e609640 1 -- 172.21.15.12:0/3186457476 wait complete. 2023-10-30T12:44:22.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.488+0000 7f6e1e609640 1 Processor -- start 2023-10-30T12:44:22.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.488+0000 7f6e1e609640 1 -- start start 2023-10-30T12:44:22.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e1e609640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e18071250 0x7f6e181487f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:22.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e1e609640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e180cb550 0x7f6e180c1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:22.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e1e609640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e1814fd50 0x7f6e180c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:22.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e1e609640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e18155c50 con 0x7f6e18071250 2023-10-30T12:44:22.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e1e609640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e18150a00 con 0x7f6e180cb550 2023-10-30T12:44:22.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e1e609640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e181f7060 con 0x7f6e1814fd50 2023-10-30T12:44:22.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e17fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e18071250 0x7f6e181487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:22.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e17fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e18071250 0x7f6e181487f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57570/0 (socket says 172.21.15.12:57570) 2023-10-30T12:44:22.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e17fff640 1 -- 172.21.15.12:0/3225721914 learned_addr learned my addr 172.21.15.12:0/3225721914 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:22.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e1cba6640 1 --2- 172.21.15.12:0/3225721914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e1814fd50 0x7f6e180c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e177fe640 1 --2- 172.21.15.12:0/3225721914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e180cb550 0x7f6e180c1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e17fff640 1 -- 172.21.15.12:0/3225721914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e1814fd50 msgr2=0x7f6e180c0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e17fff640 1 --2- 172.21.15.12:0/3225721914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e1814fd50 0x7f6e180c0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e17fff640 1 -- 172.21.15.12:0/3225721914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e180cb550 msgr2=0x7f6e180c1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e17fff640 1 --2- 172.21.15.12:0/3225721914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e180cb550 0x7f6e180c1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e17fff640 1 -- 172.21.15.12:0/3225721914 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e10009a70 con 0x7f6e18071250 2023-10-30T12:44:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e17fff640 1 --2- 172.21.15.12:0/3225721914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e18071250 0x7f6e181487f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c3c0 tx=0x7f6e0800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e14ff9640 1 -- 172.21.15.12:0/3225721914 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e080052d0 con 0x7f6e18071250 2023-10-30T12:44:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e14ff9640 1 -- 172.21.15.12:0/3225721914 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e08005430 con 0x7f6e18071250 2023-10-30T12:44:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e1e609640 1 -- 172.21.15.12:0/3225721914 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0c00b780 con 0x7f6e18071250 2023-10-30T12:44:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e14ff9640 1 -- 172.21.15.12:0/3225721914 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08003840 con 0x7f6e18071250 2023-10-30T12:44:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e1e609640 1 -- 172.21.15.12:0/3225721914 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e181f67c0 con 0x7f6e18071250 2023-10-30T12:44:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e1e609640 1 -- 172.21.15.12:0/3225721914 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1804e170 con 0x7f6e18071250 2023-10-30T12:44:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.492+0000 7f6e14ff9640 1 -- 172.21.15.12:0/3225721914 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6e080039a0 con 0x7f6e18071250 2023-10-30T12:44:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.496+0000 7f6e14ff9640 1 --2- 172.21.15.12:0/3225721914 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6e040664a0 0x7f6e04068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.496+0000 7f6e14ff9640 1 -- 172.21.15.12:0/3225721914 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f6e080835d0 con 0x7f6e18071250 2023-10-30T12:44:22.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.496+0000 7f6e177fe640 1 --2- 172.21.15.12:0/3225721914 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6e040664a0 0x7f6e04068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:22.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.496+0000 7f6e177fe640 1 --2- 172.21.15.12:0/3225721914 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6e040664a0 0x7f6e04068960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0b1000 tx=0x7f6e0c0a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:22.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.500+0000 7f6e14ff9640 1 -- 172.21.15.12:0/3225721914 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6e080508e0 con 0x7f6e18071250 2023-10-30T12:44:22.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.696+0000 7f6e1e609640 1 -- 172.21.15.12:0/3225721914 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e180c8300 con 0x7f6e18071250 2023-10-30T12:44:22.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.696+0000 7f6e14ff9640 1 -- 172.21.15.12:0/3225721914 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6527 (secure 0 0 0) 0x7f6e08053f00 con 0x7f6e18071250 2023-10-30T12:44:22.706 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:22.706 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:46.382577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:22.706 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 43 2023-10-30T12:44:22.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.700+0000 7f6e1e609640 1 -- 172.21.15.12:0/3225721914 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6e040664a0 msgr2=0x7f6e04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:22.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.700+0000 7f6e1e609640 1 --2- 172.21.15.12:0/3225721914 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6e040664a0 0x7f6e04068960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0b1000 tx=0x7f6e0c0a9070 comp rx=0 tx=0).stop 2023-10-30T12:44:22.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.700+0000 7f6e1e609640 1 -- 172.21.15.12:0/3225721914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e18071250 msgr2=0x7f6e181487f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:22.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.700+0000 7f6e1e609640 1 --2- 172.21.15.12:0/3225721914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e18071250 0x7f6e181487f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c3c0 tx=0x7f6e0800c7c0 comp rx=0 tx=0).stop 2023-10-30T12:44:22.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.700+0000 7f6e1e609640 1 -- 172.21.15.12:0/3225721914 shutdown_connections 2023-10-30T12:44:22.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.700+0000 7f6e1e609640 1 --2- 172.21.15.12:0/3225721914 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6e040664a0 0x7f6e04068960 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:22.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.700+0000 7f6e1e609640 1 --2- 172.21.15.12:0/3225721914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e1814fd50 0x7f6e180c0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:22.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.700+0000 7f6e1e609640 1 --2- 172.21.15.12:0/3225721914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e180cb550 0x7f6e180c1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:22.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.700+0000 7f6e1e609640 1 --2- 172.21.15.12:0/3225721914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e18071250 0x7f6e181487f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:22.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.700+0000 7f6e1e609640 1 -- 172.21.15.12:0/3225721914 >> 172.21.15.12:0/3225721914 conn(0x7f6e1806c6e0 msgr2=0x7f6e18147140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:22.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.700+0000 7f6e1e609640 1 -- 172.21.15.12:0/3225721914 shutdown_connections 2023-10-30T12:44:22.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:22.700+0000 7f6e1e609640 1 -- 172.21.15.12:0/3225721914 wait complete. 2023-10-30T12:44:22.727 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:44:24.728 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:24.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.876+0000 7f556219d640 1 Processor -- start 2023-10-30T12:44:24.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.876+0000 7f556219d640 1 -- start start 2023-10-30T12:44:24.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.876+0000 7f556219d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f555c1538a0 0x7f555c15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:24.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.876+0000 7f556219d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f555c0cf160 0x7f555c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:24.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.876+0000 7f556219d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c0cb800 con 0x7f555c0ca0e0 2023-10-30T12:44:24.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.876+0000 7f556219d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555c15dc30 con 0x7f555c1538a0 2023-10-30T12:44:24.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.876+0000 7f556219d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555c0cb330 con 0x7f555c0cf160 2023-10-30T12:44:24.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.876+0000 7f555b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f555c1538a0 0x7f555c15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:24.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.876+0000 7f555b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f555c1538a0 0x7f555c15d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57574/0 (socket says 172.21.15.12:57574) 2023-10-30T12:44:24.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.876+0000 7f555b7fe640 1 -- 172.21.15.12:0/3592149912 learned_addr learned my addr 172.21.15.12:0/3592149912 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:24.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.876+0000 7f555bfff640 1 --2- 172.21.15.12:0/3592149912 >> v2:172.21.15.154:3300/0 conn(0x7f555c0cf160 0x7f555c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:24.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.876+0000 7f555b7fe640 1 -- 172.21.15.12:0/3592149912 >> v2:172.21.15.154:3300/0 conn(0x7f555c0cf160 msgr2=0x7f555c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:24.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.876+0000 7f555b7fe640 1 --2- 172.21.15.12:0/3592149912 >> v2:172.21.15.154:3300/0 conn(0x7f555c0cf160 0x7f555c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:24.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.876+0000 7f555b7fe640 1 -- 172.21.15.12:0/3592149912 >> v1:172.21.15.154:6789/0 conn(0x7f555c0ca0e0 legacy=0x7f555c15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:24.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.876+0000 7f555b7fe640 1 -- 172.21.15.12:0/3592149912 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c0cad70 con 0x7f555c1538a0 2023-10-30T12:44:24.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.880+0000 7f555b7fe640 1 --2- 172.21.15.12:0/3592149912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f555c1538a0 0x7f555c15d6f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f555c0b5330 tx=0x7f554c01ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ef9700098a704d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:24.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.880+0000 7f555a7fc640 1 -- 172.21.15.12:0/3592149912 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c0a89b0 con 0x7f555c1538a0 2023-10-30T12:44:24.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.880+0000 7f555a7fc640 1 -- 172.21.15.12:0/3592149912 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554c0a8b10 con 0x7f555c1538a0 2023-10-30T12:44:24.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.880+0000 7f555a7fc640 1 -- 172.21.15.12:0/3592149912 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c0a8ea0 con 0x7f555c1538a0 2023-10-30T12:44:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.880+0000 7f556219d640 1 -- 172.21.15.12:0/3592149912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f555c1538a0 msgr2=0x7f555c15d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.880+0000 7f556219d640 1 --2- 172.21.15.12:0/3592149912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f555c1538a0 0x7f555c15d6f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f555c0b5330 tx=0x7f554c01ecb0 comp rx=0 tx=0).stop 2023-10-30T12:44:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.880+0000 7f556219d640 1 -- 172.21.15.12:0/3592149912 shutdown_connections 2023-10-30T12:44:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.880+0000 7f556219d640 1 --2- 172.21.15.12:0/3592149912 >> v2:172.21.15.154:3300/0 conn(0x7f555c0cf160 0x7f555c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.880+0000 7f556219d640 1 --2- 172.21.15.12:0/3592149912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f555c1538a0 0x7f555c15d6f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.880+0000 7f556219d640 1 -- 172.21.15.12:0/3592149912 >> 172.21.15.12:0/3592149912 conn(0x7f555c06bfe0 msgr2=0x7f555c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.880+0000 7f556219d640 1 -- 172.21.15.12:0/3592149912 shutdown_connections 2023-10-30T12:44:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.880+0000 7f556219d640 1 -- 172.21.15.12:0/3592149912 wait complete. 2023-10-30T12:44:24.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f556219d640 1 Processor -- start 2023-10-30T12:44:24.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f556219d640 1 -- start start 2023-10-30T12:44:24.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f556219d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f555c0ca0e0 0x7f555c1e2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:24.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f556219d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f555c0cf160 0x7f555c1e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:24.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f556219d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f555c1538a0 0x7f555c1e5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:24.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f556219d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c15ba00 con 0x7f555c0ca0e0 2023-10-30T12:44:24.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f556219d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c1e4750 con 0x7f555c0cf160 2023-10-30T12:44:24.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f556219d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555c15b700 con 0x7f555c1538a0 2023-10-30T12:44:24.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f555b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f555c0ca0e0 0x7f555c1e2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:24.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f555b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f555c0ca0e0 0x7f555c1e2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57580/0 (socket says 172.21.15.12:57580) 2023-10-30T12:44:24.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f555b7fe640 1 -- 172.21.15.12:0/833878779 learned_addr learned my addr 172.21.15.12:0/833878779 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:24.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f555affd640 1 --2- 172.21.15.12:0/833878779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f555c0cf160 0x7f555c1e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:24.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f555bfff640 1 --2- 172.21.15.12:0/833878779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f555c1538a0 0x7f555c1e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:24.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f555b7fe640 1 -- 172.21.15.12:0/833878779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f555c1538a0 msgr2=0x7f555c1e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:24.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f555b7fe640 1 --2- 172.21.15.12:0/833878779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f555c1538a0 0x7f555c1e5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:24.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f555b7fe640 1 -- 172.21.15.12:0/833878779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f555c0cf160 msgr2=0x7f555c1e4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:24.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f555b7fe640 1 --2- 172.21.15.12:0/833878779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f555c0cf160 0x7f555c1e4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f555b7fe640 1 -- 172.21.15.12:0/833878779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c0094d0 con 0x7f555c0ca0e0 2023-10-30T12:44:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f555b7fe640 1 --2- 172.21.15.12:0/833878779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f555c0ca0e0 0x7f555c1e2d40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f554c01d360 tx=0x7f554c01ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f5543fff640 1 -- 172.21.15.12:0/833878779 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c0ba600 con 0x7f555c0ca0e0 2023-10-30T12:44:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f5543fff640 1 -- 172.21.15.12:0/833878779 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554c0ba760 con 0x7f555c0ca0e0 2023-10-30T12:44:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f5543fff640 1 -- 172.21.15.12:0/833878779 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c0bab10 con 0x7f555c0ca0e0 2023-10-30T12:44:24.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f556219d640 1 -- 172.21.15.12:0/833878779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555c0b4300 con 0x7f555c0ca0e0 2023-10-30T12:44:24.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f556219d640 1 -- 172.21.15.12:0/833878779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555c0b4860 con 0x7f555c0ca0e0 2023-10-30T12:44:24.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.884+0000 7f556219d640 1 -- 172.21.15.12:0/833878779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555c0b2a50 con 0x7f555c0ca0e0 2023-10-30T12:44:24.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.888+0000 7f5543fff640 1 -- 172.21.15.12:0/833878779 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f554c0bac70 con 0x7f555c0ca0e0 2023-10-30T12:44:24.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.888+0000 7f5543fff640 1 --2- 172.21.15.12:0/833878779 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f55480664f0 0x7f55480689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:24.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.888+0000 7f5543fff640 1 -- 172.21.15.12:0/833878779 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f554c122620 con 0x7f555c0ca0e0 2023-10-30T12:44:24.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.888+0000 7f555affd640 1 --2- 172.21.15.12:0/833878779 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f55480664f0 0x7f55480689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:24.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.888+0000 7f555affd640 1 --2- 172.21.15.12:0/833878779 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f55480664f0 0x7f55480689b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f555c0b1da0 tx=0x7f5550008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:24.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:24.892+0000 7f5543fff640 1 -- 172.21.15.12:0/833878779 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f554c0ef990 con 0x7f555c0ca0e0 2023-10-30T12:44:25.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:25.084+0000 7f556219d640 1 -- 172.21.15.12:0/833878779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f555c0b2a50 con 0x7f555c0ca0e0 2023-10-30T12:44:25.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:25.084+0000 7f5543fff640 1 -- 172.21.15.12:0/833878779 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6527 (secure 0 0 0) 0x7f554c0b5090 con 0x7f555c0ca0e0 2023-10-30T12:44:25.091 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:25.091 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:46.382577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:25.091 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 43 2023-10-30T12:44:25.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:25.084+0000 7f556219d640 1 -- 172.21.15.12:0/833878779 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f55480664f0 msgr2=0x7f55480689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:25.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:25.084+0000 7f556219d640 1 --2- 172.21.15.12:0/833878779 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f55480664f0 0x7f55480689b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f555c0b1da0 tx=0x7f5550008040 comp rx=0 tx=0).stop 2023-10-30T12:44:25.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:25.084+0000 7f556219d640 1 -- 172.21.15.12:0/833878779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f555c0ca0e0 msgr2=0x7f555c1e2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:25.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:25.084+0000 7f556219d640 1 --2- 172.21.15.12:0/833878779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f555c0ca0e0 0x7f555c1e2d40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f554c01d360 tx=0x7f554c01ba50 comp rx=0 tx=0).stop 2023-10-30T12:44:25.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:25.084+0000 7f556219d640 1 -- 172.21.15.12:0/833878779 shutdown_connections 2023-10-30T12:44:25.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:25.084+0000 7f556219d640 1 --2- 172.21.15.12:0/833878779 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f55480664f0 0x7f55480689b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:25.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:25.084+0000 7f556219d640 1 --2- 172.21.15.12:0/833878779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f555c1538a0 0x7f555c1e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:25.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:25.084+0000 7f556219d640 1 --2- 172.21.15.12:0/833878779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f555c0cf160 0x7f555c1e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:25.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:25.084+0000 7f556219d640 1 --2- 172.21.15.12:0/833878779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f555c0ca0e0 0x7f555c1e2d40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:25.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:25.084+0000 7f556219d640 1 -- 172.21.15.12:0/833878779 >> 172.21.15.12:0/833878779 conn(0x7f555c06bfe0 msgr2=0x7f555c151cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:25.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:25.084+0000 7f556219d640 1 -- 172.21.15.12:0/833878779 shutdown_connections 2023-10-30T12:44:25.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:25.084+0000 7f556219d640 1 -- 172.21.15.12:0/833878779 wait complete. 2023-10-30T12:44:25.112 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:44:27.114 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:27.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.260+0000 7f8f4a9ab640 1 Processor -- start 2023-10-30T12:44:27.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.260+0000 7f8f4a9ab640 1 -- start start 2023-10-30T12:44:27.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f4a9ab640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44070a00 0x7f8f44070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:27.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f4a9ab640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8f44069e90 0x7f8f4406a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:27.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f4a9ab640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f44071380 con 0x7f8f441539a0 2023-10-30T12:44:27.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f4a9ab640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f4406fbb0 con 0x7f8f44069e90 2023-10-30T12:44:27.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f4a9ab640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f44153d80 con 0x7f8f44070a00 2023-10-30T12:44:27.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f43fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8f441539a0 0x7f8f4415a380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49218/0 (socket says 172.21.15.12:49218) 2023-10-30T12:44:27.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f43fff640 1 -- 172.21.15.12:0/865768665 learned_addr learned my addr 172.21.15.12:0/865768665 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:44:27.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f48f48640 1 --2- 172.21.15.12:0/865768665 >> v2:172.21.15.12:3300/0 conn(0x7f8f44069e90 0x7f8f4406a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:27.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f3bfff640 1 --2- 172.21.15.12:0/865768665 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44070a00 0x7f8f44070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:27.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f437fe640 1 -- 172.21.15.12:0/865768665 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1370270127 0 0) 0x7f8f44071380 con 0x7f8f441539a0 2023-10-30T12:44:27.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f437fe640 1 -- 172.21.15.12:0/865768665 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f30003530 con 0x7f8f441539a0 2023-10-30T12:44:27.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f48f48640 1 -- 172.21.15.12:0/865768665 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44070a00 msgr2=0x7f8f44070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:27.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f48f48640 1 --2- 172.21.15.12:0/865768665 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44070a00 0x7f8f44070e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:27.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f48f48640 1 -- 172.21.15.12:0/865768665 >> v1:172.21.15.12:6789/0 conn(0x7f8f441539a0 legacy=0x7f8f4415a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:27.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f48f48640 1 -- 172.21.15.12:0/865768665 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f440c5c50 con 0x7f8f44069e90 2023-10-30T12:44:27.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.264+0000 7f8f3bfff640 1 --2- 172.21.15.12:0/865768665 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44070a00 0x7f8f44070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:44:27.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.268+0000 7f8f48f48640 1 --2- 172.21.15.12:0/865768665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 0x7f8f4406a300 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009d90 tx=0x7f8f3c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5006965900d85969 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:27.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.268+0000 7f8f437fe640 1 -- 172.21.15.12:0/865768665 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0a5860 con 0x7f8f44069e90 2023-10-30T12:44:27.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.268+0000 7f8f437fe640 1 -- 172.21.15.12:0/865768665 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c0a59c0 con 0x7f8f44069e90 2023-10-30T12:44:27.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.268+0000 7f8f437fe640 1 -- 172.21.15.12:0/865768665 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0a5d50 con 0x7f8f44069e90 2023-10-30T12:44:27.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.268+0000 7f8f4a9ab640 1 -- 172.21.15.12:0/865768665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 msgr2=0x7f8f4406a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:27.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.268+0000 7f8f4a9ab640 1 --2- 172.21.15.12:0/865768665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 0x7f8f4406a300 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009d90 tx=0x7f8f3c01c890 comp rx=0 tx=0).stop 2023-10-30T12:44:27.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.268+0000 7f8f4a9ab640 1 -- 172.21.15.12:0/865768665 shutdown_connections 2023-10-30T12:44:27.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.268+0000 7f8f4a9ab640 1 --2- 172.21.15.12:0/865768665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 0x7f8f4406a300 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:27.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.268+0000 7f8f4a9ab640 1 --2- 172.21.15.12:0/865768665 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f44070a00 0x7f8f44070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:27.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.268+0000 7f8f4a9ab640 1 -- 172.21.15.12:0/865768665 >> 172.21.15.12:0/865768665 conn(0x7f8f440c7910 msgr2=0x7f8f440c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:27.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.268+0000 7f8f4a9ab640 1 -- 172.21.15.12:0/865768665 shutdown_connections 2023-10-30T12:44:27.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.268+0000 7f8f4a9ab640 1 -- 172.21.15.12:0/865768665 wait complete. 2023-10-30T12:44:27.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.268+0000 7f8f4a9ab640 1 Processor -- start 2023-10-30T12:44:27.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.268+0000 7f8f4a9ab640 1 -- start start 2023-10-30T12:44:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f4a9ab640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 0x7f8f440c1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f4a9ab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f44070a00 0x7f8f441f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f4a9ab640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f441539a0 0x7f8f441f6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f4a9ab640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440c0ca0 con 0x7f8f44069e90 2023-10-30T12:44:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f4a9ab640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f441f68c0 con 0x7f8f44070a00 2023-10-30T12:44:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f4a9ab640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440c0970 con 0x7f8f441539a0 2023-10-30T12:44:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 0x7f8f440c1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 0x7f8f440c1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57596/0 (socket says 172.21.15.12:57596) 2023-10-30T12:44:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f43fff640 1 -- 172.21.15.12:0/4217385160 learned_addr learned my addr 172.21.15.12:0/4217385160 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f48f48640 1 --2- 172.21.15.12:0/4217385160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f441539a0 0x7f8f441f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f3bfff640 1 --2- 172.21.15.12:0/4217385160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f44070a00 0x7f8f441f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f43fff640 1 -- 172.21.15.12:0/4217385160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f441539a0 msgr2=0x7f8f441f6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f43fff640 1 --2- 172.21.15.12:0/4217385160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f441539a0 0x7f8f441f6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f43fff640 1 -- 172.21.15.12:0/4217385160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f44070a00 msgr2=0x7f8f441f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f43fff640 1 --2- 172.21.15.12:0/4217385160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f44070a00 0x7f8f441f5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f43fff640 1 -- 172.21.15.12:0/4217385160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c009a70 con 0x7f8f44069e90 2023-10-30T12:44:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f43fff640 1 --2- 172.21.15.12:0/4217385160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 0x7f8f440c1130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400c4d0 tx=0x7f8f3400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f417fa640 1 -- 172.21.15.12:0/4217385160 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34005cf0 con 0x7f8f44069e90 2023-10-30T12:44:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f417fa640 1 -- 172.21.15.12:0/4217385160 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f34005e50 con 0x7f8f44069e90 2023-10-30T12:44:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f417fa640 1 -- 172.21.15.12:0/4217385160 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34007270 con 0x7f8f44069e90 2023-10-30T12:44:27.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f4a9ab640 1 -- 172.21.15.12:0/4217385160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f440c9fa0 con 0x7f8f44069e90 2023-10-30T12:44:27.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f4a9ab640 1 -- 172.21.15.12:0/4217385160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f440c8c10 con 0x7f8f44069e90 2023-10-30T12:44:27.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f4a9ab640 1 -- 172.21.15.12:0/4217385160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4404e170 con 0x7f8f44069e90 2023-10-30T12:44:27.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.272+0000 7f8f417fa640 1 -- 172.21.15.12:0/4217385160 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8f340073d0 con 0x7f8f44069e90 2023-10-30T12:44:27.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.276+0000 7f8f417fa640 1 --2- 172.21.15.12:0/4217385160 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f240664d0 0x7f8f24068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:27.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.276+0000 7f8f417fa640 1 -- 172.21.15.12:0/4217385160 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f8f34083570 con 0x7f8f44069e90 2023-10-30T12:44:27.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.276+0000 7f8f3bfff640 1 --2- 172.21.15.12:0/4217385160 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f240664d0 0x7f8f24068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:27.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.276+0000 7f8f3bfff640 1 --2- 172.21.15.12:0/4217385160 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f240664d0 0x7f8f24068990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8f4406c5e0 tx=0x7f8f2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:27.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.280+0000 7f8f417fa640 1 -- 172.21.15.12:0/4217385160 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8f340507f0 con 0x7f8f44069e90 2023-10-30T12:44:27.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.472+0000 7f8f4a9ab640 1 -- 172.21.15.12:0/4217385160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f441517b0 con 0x7f8f44069e90 2023-10-30T12:44:27.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.472+0000 7f8f417fa640 1 -- 172.21.15.12:0/4217385160 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6527 (secure 0 0 0) 0x7f8f34017090 con 0x7f8f44069e90 2023-10-30T12:44:27.479 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:27.479 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:46.382577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 43 2023-10-30T12:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.472+0000 7f8f4a9ab640 1 -- 172.21.15.12:0/4217385160 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f240664d0 msgr2=0x7f8f24068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.472+0000 7f8f4a9ab640 1 --2- 172.21.15.12:0/4217385160 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f240664d0 0x7f8f24068990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8f4406c5e0 tx=0x7f8f2c008040 comp rx=0 tx=0).stop 2023-10-30T12:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.472+0000 7f8f4a9ab640 1 -- 172.21.15.12:0/4217385160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 msgr2=0x7f8f440c1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.472+0000 7f8f4a9ab640 1 --2- 172.21.15.12:0/4217385160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 0x7f8f440c1130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400c4d0 tx=0x7f8f3400c990 comp rx=0 tx=0).stop 2023-10-30T12:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.472+0000 7f8f4a9ab640 1 -- 172.21.15.12:0/4217385160 shutdown_connections 2023-10-30T12:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.472+0000 7f8f4a9ab640 1 --2- 172.21.15.12:0/4217385160 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8f240664d0 0x7f8f24068990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.472+0000 7f8f4a9ab640 1 --2- 172.21.15.12:0/4217385160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f441539a0 0x7f8f441f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.472+0000 7f8f4a9ab640 1 --2- 172.21.15.12:0/4217385160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f44070a00 0x7f8f441f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.472+0000 7f8f4a9ab640 1 --2- 172.21.15.12:0/4217385160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f44069e90 0x7f8f440c1130 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.472+0000 7f8f4a9ab640 1 -- 172.21.15.12:0/4217385160 >> 172.21.15.12:0/4217385160 conn(0x7f8f440c7910 msgr2=0x7f8f44150230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.472+0000 7f8f4a9ab640 1 -- 172.21.15.12:0/4217385160 shutdown_connections 2023-10-30T12:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:27.472+0000 7f8f4a9ab640 1 -- 172.21.15.12:0/4217385160 wait complete. 2023-10-30T12:44:27.500 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:44:29.501 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:29.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.648+0000 7fda9e091640 1 Processor -- start 2023-10-30T12:44:29.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.648+0000 7fda9e091640 1 -- start start 2023-10-30T12:44:29.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.648+0000 7fda9e091640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fda9806f7a0 0x7fda9806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:29.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.648+0000 7fda9e091640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda98066cf0 con 0x7fda980c6bd0 2023-10-30T12:44:29.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.648+0000 7fda9e091640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda980c70d0 con 0x7fda98071110 2023-10-30T12:44:29.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.648+0000 7fda9e091640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda980700c0 con 0x7fda9806f7a0 2023-10-30T12:44:29.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.648+0000 7fda96ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fda980c6bd0 0x7fda98069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49230/0 (socket says 172.21.15.12:49230) 2023-10-30T12:44:29.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.648+0000 7fda96ffd640 1 -- 172.21.15.12:0/355900162 learned_addr learned my addr 172.21.15.12:0/355900162 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:44:29.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.648+0000 7fda977fe640 1 --2- 172.21.15.12:0/355900162 >> v2:172.21.15.154:3300/0 conn(0x7fda9806f7a0 0x7fda9806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:29.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.648+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6612807 0 0) 0x7fda98066cf0 con 0x7fda980c6bd0 2023-10-30T12:44:29.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.648+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda80003530 con 0x7fda980c6bd0 2023-10-30T12:44:29.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232377299 0 0) 0x7fda980c70d0 con 0x7fda98071110 2023-10-30T12:44:29.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda98066cf0 con 0x7fda98071110 2023-10-30T12:44:29.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 208192046 0 0) 0x7fda80003530 con 0x7fda980c6bd0 2023-10-30T12:44:29.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 >> v2:172.21.15.154:3300/0 conn(0x7fda9806f7a0 msgr2=0x7fda9806fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:29.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda967fc640 1 --2- 172.21.15.12:0/355900162 >> v2:172.21.15.154:3300/0 conn(0x7fda9806f7a0 0x7fda9806fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:29.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 >> v1:172.21.15.154:6789/0 conn(0x7fda98071110 legacy=0x7fda98158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:29.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda980c7390 con 0x7fda980c6bd0 2023-10-30T12:44:29.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fda8c001680 con 0x7fda980c6bd0 2023-10-30T12:44:29.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 >> v1:172.21.15.12:6789/0 conn(0x7fda980c6bd0 legacy=0x7fda98069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:29.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda967fc640 1 --2- 172.21.15.12:0/355900162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda80003b10 0x7fda80003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:29.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda967fc640 1 --2- 172.21.15.12:0/355900162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda800047d0 0x7fda80004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:29.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda967fc640 1 --2- 172.21.15.12:0/355900162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda80005190 0x7fda8000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:29.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda980700c0 con 0x7fda80005190 2023-10-30T12:44:29.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8000bf80 con 0x7fda80003b10 2023-10-30T12:44:29.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8000c140 con 0x7fda800047d0 2023-10-30T12:44:29.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda97fff640 1 --2- 172.21.15.12:0/355900162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda80005190 0x7fda8000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:29.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda977fe640 1 --2- 172.21.15.12:0/355900162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda80003b10 0x7fda80003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:29.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda96ffd640 1 --2- 172.21.15.12:0/355900162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda800047d0 0x7fda80004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:29.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda97fff640 1 -- 172.21.15.12:0/355900162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda800047d0 msgr2=0x7fda80004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:29.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda97fff640 1 --2- 172.21.15.12:0/355900162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda800047d0 0x7fda80004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:29.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda97fff640 1 -- 172.21.15.12:0/355900162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda80003b10 msgr2=0x7fda80003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:29.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda97fff640 1 --2- 172.21.15.12:0/355900162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda80003b10 0x7fda80003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:29.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda97fff640 1 -- 172.21.15.12:0/355900162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda80003530 con 0x7fda80005190 2023-10-30T12:44:29.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.652+0000 7fda97fff640 1 -- 172.21.15.12:0/355900162 reap_dead start 2023-10-30T12:44:29.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda97fff640 1 --2- 172.21.15.12:0/355900162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda80005190 0x7fda8000b990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fda90007fd0 tx=0x7fda9001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e441fb4d3eaec787 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:29.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90003b80 con 0x7fda80005190 2023-10-30T12:44:29.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda90003ce0 con 0x7fda80005190 2023-10-30T12:44:29.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda967fc640 1 -- 172.21.15.12:0/355900162 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda900052c0 con 0x7fda80005190 2023-10-30T12:44:29.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda9e091640 1 -- 172.21.15.12:0/355900162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda80005190 msgr2=0x7fda8000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:29.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda9e091640 1 --2- 172.21.15.12:0/355900162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda80005190 0x7fda8000b990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fda90007fd0 tx=0x7fda9001c8c0 comp rx=0 tx=0).stop 2023-10-30T12:44:29.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda9e091640 1 -- 172.21.15.12:0/355900162 shutdown_connections 2023-10-30T12:44:29.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda9e091640 1 --2- 172.21.15.12:0/355900162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda80005190 0x7fda8000b990 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:29.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda9e091640 1 -- 172.21.15.12:0/355900162 >> 172.21.15.12:0/355900162 conn(0x7fda980c7910 msgr2=0x7fda980c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:29.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda9e091640 1 -- 172.21.15.12:0/355900162 shutdown_connections 2023-10-30T12:44:29.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda9e091640 1 -- 172.21.15.12:0/355900162 wait complete. 2023-10-30T12:44:29.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda9e091640 1 Processor -- start 2023-10-30T12:44:29.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda9e091640 1 -- start start 2023-10-30T12:44:29.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda9e091640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda80005190 0x7fda980afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:29.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda9e091640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda9814dd80 0x7fda9814b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:29.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda9e091640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda980c5d70 0x7fda9814bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:29.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda9e091640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9814f800 con 0x7fda9814dd80 2023-10-30T12:44:29.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda9e091640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9814ea70 con 0x7fda80005190 2023-10-30T12:44:29.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.656+0000 7fda9e091640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9814e510 con 0x7fda980c5d70 2023-10-30T12:44:29.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda96ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda9814dd80 0x7fda9814b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:29.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda96ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda9814dd80 0x7fda9814b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57618/0 (socket says 172.21.15.12:57618) 2023-10-30T12:44:29.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda96ffd640 1 -- 172.21.15.12:0/3815706128 learned_addr learned my addr 172.21.15.12:0/3815706128 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:29.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda97fff640 1 --2- 172.21.15.12:0/3815706128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda980c5d70 0x7fda9814bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:29.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda977fe640 1 --2- 172.21.15.12:0/3815706128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda80005190 0x7fda980afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:29.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda96ffd640 1 -- 172.21.15.12:0/3815706128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda980c5d70 msgr2=0x7fda9814bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:29.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda96ffd640 1 --2- 172.21.15.12:0/3815706128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda980c5d70 0x7fda9814bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:29.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda96ffd640 1 -- 172.21.15.12:0/3815706128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda80005190 msgr2=0x7fda980afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:29.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda96ffd640 1 --2- 172.21.15.12:0/3815706128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda80005190 0x7fda980afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:29.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda96ffd640 1 -- 172.21.15.12:0/3815706128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9814eea0 con 0x7fda9814dd80 2023-10-30T12:44:29.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda96ffd640 1 --2- 172.21.15.12:0/3815706128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda9814dd80 0x7fda9814b750 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fda8c007340 tx=0x7fda8c007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:29.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda7ffff640 1 -- 172.21.15.12:0/3815706128 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c00f9c0 con 0x7fda9814dd80 2023-10-30T12:44:29.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda7ffff640 1 -- 172.21.15.12:0/3815706128 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8c00fb60 con 0x7fda9814dd80 2023-10-30T12:44:29.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda9e091640 1 -- 172.21.15.12:0/3815706128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9814e790 con 0x7fda9814dd80 2023-10-30T12:44:29.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda7ffff640 1 -- 172.21.15.12:0/3815706128 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c0133f0 con 0x7fda9814dd80 2023-10-30T12:44:29.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda9e091640 1 -- 172.21.15.12:0/3815706128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9814f420 con 0x7fda9814dd80 2023-10-30T12:44:29.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda9e091640 1 -- 172.21.15.12:0/3815706128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9814f940 con 0x7fda9814dd80 2023-10-30T12:44:29.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.660+0000 7fda7ffff640 1 -- 172.21.15.12:0/3815706128 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fda8c013550 con 0x7fda9814dd80 2023-10-30T12:44:29.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.664+0000 7fda7ffff640 1 --2- 172.21.15.12:0/3815706128 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda840664d0 0x7fda84068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:29.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.664+0000 7fda7ffff640 1 -- 172.21.15.12:0/3815706128 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fda8c083d20 con 0x7fda9814dd80 2023-10-30T12:44:29.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.664+0000 7fda977fe640 1 --2- 172.21.15.12:0/3815706128 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda840664d0 0x7fda84068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:29.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.664+0000 7fda977fe640 1 --2- 172.21.15.12:0/3815706128 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda840664d0 0x7fda84068990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fda88009ac0 tx=0x7fda88009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:29.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.664+0000 7fda7ffff640 1 -- 172.21.15.12:0/3815706128 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fda9814f940 con 0x7fda9814dd80 2023-10-30T12:44:29.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.856+0000 7fda9e091640 1 -- 172.21.15.12:0/3815706128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda9814f940 con 0x7fda9814dd80 2023-10-30T12:44:29.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.856+0000 7fda7ffff640 1 -- 172.21.15.12:0/3815706128 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6527 (secure 0 0 0) 0x7fda9814f940 con 0x7fda9814dd80 2023-10-30T12:44:29.866 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:29.866 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:46.382577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 43 2023-10-30T12:44:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.860+0000 7fda9e091640 1 -- 172.21.15.12:0/3815706128 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda840664d0 msgr2=0x7fda84068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.860+0000 7fda9e091640 1 --2- 172.21.15.12:0/3815706128 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda840664d0 0x7fda84068990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fda88009ac0 tx=0x7fda88009190 comp rx=0 tx=0).stop 2023-10-30T12:44:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.860+0000 7fda9e091640 1 -- 172.21.15.12:0/3815706128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda9814dd80 msgr2=0x7fda9814b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.860+0000 7fda9e091640 1 --2- 172.21.15.12:0/3815706128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda9814dd80 0x7fda9814b750 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fda8c007340 tx=0x7fda8c007740 comp rx=0 tx=0).stop 2023-10-30T12:44:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.860+0000 7fda9e091640 1 -- 172.21.15.12:0/3815706128 shutdown_connections 2023-10-30T12:44:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.860+0000 7fda9e091640 1 --2- 172.21.15.12:0/3815706128 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda840664d0 0x7fda84068990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.860+0000 7fda9e091640 1 --2- 172.21.15.12:0/3815706128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda980c5d70 0x7fda9814bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.860+0000 7fda9e091640 1 --2- 172.21.15.12:0/3815706128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda9814dd80 0x7fda9814b750 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.860+0000 7fda9e091640 1 --2- 172.21.15.12:0/3815706128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda80005190 0x7fda980afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.860+0000 7fda9e091640 1 -- 172.21.15.12:0/3815706128 >> 172.21.15.12:0/3815706128 conn(0x7fda980c7910 msgr2=0x7fda980c2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.860+0000 7fda9e091640 1 -- 172.21.15.12:0/3815706128 shutdown_connections 2023-10-30T12:44:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:29.860+0000 7fda9e091640 1 -- 172.21.15.12:0/3815706128 wait complete. 2023-10-30T12:44:29.888 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:44:31.889 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:32.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.036+0000 7fac54446640 1 Processor -- start 2023-10-30T12:44:32.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.036+0000 7fac54446640 1 -- start start 2023-10-30T12:44:32.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.036+0000 7fac54446640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac4c0c6b80 0x7fac4c0bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:32.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.036+0000 7fac54446640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac4c13f4a0 0x7fac4c0c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:32.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.036+0000 7fac54446640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4c0c0a50 con 0x7fac4c0c6f60 2023-10-30T12:44:32.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.036+0000 7fac54446640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c06f6e0 con 0x7fac4c0c6b80 2023-10-30T12:44:32.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.036+0000 7fac54446640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c06f9c0 con 0x7fac4c13f4a0 2023-10-30T12:44:32.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.036+0000 7fac521e2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac4c0c6b80 0x7fac4c0bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:32.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.036+0000 7fac519e1640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fac4c0c6f60 0x7fac4c13ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50006/0 (socket says 172.21.15.12:50006) 2023-10-30T12:44:32.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.036+0000 7fac519e1640 1 -- 172.21.15.12:0/2162305068 learned_addr learned my addr 172.21.15.12:0/2162305068 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:44:32.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.040+0000 7fac529e3640 1 --2- 172.21.15.12:0/2162305068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac4c13f4a0 0x7fac4c0c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:32.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.040+0000 7fac521e2640 1 -- 172.21.15.12:0/2162305068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac4c13f4a0 msgr2=0x7fac4c0c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:32.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.040+0000 7fac521e2640 1 --2- 172.21.15.12:0/2162305068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac4c13f4a0 0x7fac4c0c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:32.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.040+0000 7fac521e2640 1 -- 172.21.15.12:0/2162305068 >> v1:172.21.15.12:6789/0 conn(0x7fac4c0c6f60 legacy=0x7fac4c13ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:32.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.040+0000 7fac521e2640 1 -- 172.21.15.12:0/2162305068 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c06fca0 con 0x7fac4c0c6b80 2023-10-30T12:44:32.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.040+0000 7fac511e0640 1 -- 172.21.15.12:0/2162305068 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598834308 0 0) 0x7fac4c0c0a50 con 0x7fac4c0c6f60 2023-10-30T12:44:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.040+0000 7fac521e2640 1 --2- 172.21.15.12:0/2162305068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac4c0c6b80 0x7fac4c0bfec0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fac480097c0 tx=0x7fac4801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c108097ee9341f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.040+0000 7fac511e0640 1 -- 172.21.15.12:0/2162305068 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac480ad640 con 0x7fac4c0c6b80 2023-10-30T12:44:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.040+0000 7fac511e0640 1 -- 172.21.15.12:0/2162305068 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac480ad7a0 con 0x7fac4c0c6b80 2023-10-30T12:44:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.040+0000 7fac511e0640 1 -- 172.21.15.12:0/2162305068 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac480adb30 con 0x7fac4c0c6b80 2023-10-30T12:44:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.040+0000 7fac54446640 1 -- 172.21.15.12:0/2162305068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac4c0c6b80 msgr2=0x7fac4c0bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.040+0000 7fac54446640 1 --2- 172.21.15.12:0/2162305068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac4c0c6b80 0x7fac4c0bfec0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fac480097c0 tx=0x7fac4801c480 comp rx=0 tx=0).stop 2023-10-30T12:44:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac54446640 1 -- 172.21.15.12:0/2162305068 shutdown_connections 2023-10-30T12:44:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac54446640 1 --2- 172.21.15.12:0/2162305068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac4c13f4a0 0x7fac4c0c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac54446640 1 --2- 172.21.15.12:0/2162305068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac4c0c6b80 0x7fac4c0bfec0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:32.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac54446640 1 -- 172.21.15.12:0/2162305068 >> 172.21.15.12:0/2162305068 conn(0x7fac4c0c7910 msgr2=0x7fac4c0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:32.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac54446640 1 -- 172.21.15.12:0/2162305068 shutdown_connections 2023-10-30T12:44:32.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac54446640 1 -- 172.21.15.12:0/2162305068 wait complete. 2023-10-30T12:44:32.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac54446640 1 Processor -- start 2023-10-30T12:44:32.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac54446640 1 -- start start 2023-10-30T12:44:32.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac54446640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac4c0c6b80 0x7fac4c135d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:32.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac54446640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac4c0c6f60 0x7fac4c136c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:32.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac54446640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac4c13f4a0 0x7fac4c1f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:32.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac54446640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c136270 con 0x7fac4c13f4a0 2023-10-30T12:44:32.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac54446640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c135450 con 0x7fac4c0c6b80 2023-10-30T12:44:32.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac54446640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c135730 con 0x7fac4c0c6f60 2023-10-30T12:44:32.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac529e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac4c13f4a0 0x7fac4c1f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac529e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac4c13f4a0 0x7fac4c1f57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41546/0 (socket says 172.21.15.12:41546) 2023-10-30T12:44:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac529e3640 1 -- 172.21.15.12:0/2793566951 learned_addr learned my addr 172.21.15.12:0/2793566951 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac519e1640 1 --2- 172.21.15.12:0/2793566951 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac4c0c6f60 0x7fac4c136c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac521e2640 1 --2- 172.21.15.12:0/2793566951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac4c0c6b80 0x7fac4c135d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac529e3640 1 -- 172.21.15.12:0/2793566951 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac4c0c6f60 msgr2=0x7fac4c136c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac529e3640 1 --2- 172.21.15.12:0/2793566951 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac4c0c6f60 0x7fac4c136c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac529e3640 1 -- 172.21.15.12:0/2793566951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac4c0c6b80 msgr2=0x7fac4c135d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac529e3640 1 --2- 172.21.15.12:0/2793566951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac4c0c6b80 0x7fac4c135d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac529e3640 1 -- 172.21.15.12:0/2793566951 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac48009420 con 0x7fac4c13f4a0 2023-10-30T12:44:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.044+0000 7fac529e3640 1 --2- 172.21.15.12:0/2793566951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac4c13f4a0 0x7fac4c1f57c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fac44007e60 tx=0x7fac4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.048+0000 7fac42ffd640 1 -- 172.21.15.12:0/2793566951 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac44014420 con 0x7fac4c13f4a0 2023-10-30T12:44:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.048+0000 7fac42ffd640 1 -- 172.21.15.12:0/2793566951 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac440145a0 con 0x7fac4c13f4a0 2023-10-30T12:44:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.048+0000 7fac42ffd640 1 -- 172.21.15.12:0/2793566951 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac44014950 con 0x7fac4c13f4a0 2023-10-30T12:44:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.048+0000 7fac54446640 1 -- 172.21.15.12:0/2793566951 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c134d20 con 0x7fac4c13f4a0 2023-10-30T12:44:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.048+0000 7fac54446640 1 -- 172.21.15.12:0/2793566951 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c1f52a0 con 0x7fac4c13f4a0 2023-10-30T12:44:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.048+0000 7fac54446640 1 -- 172.21.15.12:0/2793566951 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4c06b0c0 con 0x7fac4c13f4a0 2023-10-30T12:44:32.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.048+0000 7fac42ffd640 1 -- 172.21.15.12:0/2793566951 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fac44014ab0 con 0x7fac4c13f4a0 2023-10-30T12:44:32.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.048+0000 7fac42ffd640 1 --2- 172.21.15.12:0/2793566951 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac300664a0 0x7fac30068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:32.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.048+0000 7fac42ffd640 1 -- 172.21.15.12:0/2793566951 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fac44084a50 con 0x7fac4c13f4a0 2023-10-30T12:44:32.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.048+0000 7fac521e2640 1 --2- 172.21.15.12:0/2793566951 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac300664a0 0x7fac30068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:32.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.052+0000 7fac521e2640 1 --2- 172.21.15.12:0/2793566951 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac300664a0 0x7fac30068960 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fac480af000 tx=0x7fac48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:32.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.052+0000 7fac42ffd640 1 -- 172.21.15.12:0/2793566951 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fac44051c20 con 0x7fac4c13f4a0 2023-10-30T12:44:32.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.244+0000 7fac54446640 1 -- 172.21.15.12:0/2793566951 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac4c04e170 con 0x7fac4c13f4a0 2023-10-30T12:44:32.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.244+0000 7fac42ffd640 1 -- 172.21.15.12:0/2793566951 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6527 (secure 0 0 0) 0x7fac44055240 con 0x7fac4c13f4a0 2023-10-30T12:44:32.252 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:32.253 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:46.382577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:32.253 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 43 2023-10-30T12:44:32.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.248+0000 7fac54446640 1 -- 172.21.15.12:0/2793566951 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac300664a0 msgr2=0x7fac30068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:32.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.248+0000 7fac54446640 1 --2- 172.21.15.12:0/2793566951 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac300664a0 0x7fac30068960 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fac480af000 tx=0x7fac48008040 comp rx=0 tx=0).stop 2023-10-30T12:44:32.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.248+0000 7fac54446640 1 -- 172.21.15.12:0/2793566951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac4c13f4a0 msgr2=0x7fac4c1f57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:32.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.248+0000 7fac54446640 1 --2- 172.21.15.12:0/2793566951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac4c13f4a0 0x7fac4c1f57c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fac44007e60 tx=0x7fac4400e4b0 comp rx=0 tx=0).stop 2023-10-30T12:44:32.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.248+0000 7fac54446640 1 -- 172.21.15.12:0/2793566951 shutdown_connections 2023-10-30T12:44:32.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.248+0000 7fac54446640 1 --2- 172.21.15.12:0/2793566951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac4c13f4a0 0x7fac4c1f57c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:32.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.248+0000 7fac54446640 1 --2- 172.21.15.12:0/2793566951 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac300664a0 0x7fac30068960 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:32.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.248+0000 7fac54446640 1 --2- 172.21.15.12:0/2793566951 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac4c0c6f60 0x7fac4c136c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:32.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.248+0000 7fac54446640 1 --2- 172.21.15.12:0/2793566951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac4c0c6b80 0x7fac4c135d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:32.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.248+0000 7fac54446640 1 -- 172.21.15.12:0/2793566951 >> 172.21.15.12:0/2793566951 conn(0x7fac4c0c7910 msgr2=0x7fac4c0c0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:32.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.248+0000 7fac54446640 1 -- 172.21.15.12:0/2793566951 shutdown_connections 2023-10-30T12:44:32.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:32.248+0000 7fac54446640 1 -- 172.21.15.12:0/2793566951 wait complete. 2023-10-30T12:44:32.273 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:44:34.274 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:34.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.420+0000 7f46eca46640 1 Processor -- start 2023-10-30T12:44:34.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.420+0000 7f46eca46640 1 -- start start 2023-10-30T12:44:34.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46eca46640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46e815d6f0 0x7f46e815a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:34.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46eca46640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e80cb800 con 0x7f46e80cf160 2023-10-30T12:44:34.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46eca46640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e80cb330 con 0x7f46e81538a0 2023-10-30T12:44:34.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46eca46640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e80cad70 con 0x7f46e815d6f0 2023-10-30T12:44:34.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e6d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f46e80cf160 0x7f46e8164070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50022/0 (socket says 172.21.15.12:50022) 2023-10-30T12:44:34.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e6d9d640 1 -- 172.21.15.12:0/2647968656 learned_addr learned my addr 172.21.15.12:0/2647968656 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:44:34.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46dffff640 1 --2- 172.21.15.12:0/2647968656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46e815d6f0 0x7f46e815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:34.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e5d9b640 1 -- 172.21.15.12:0/2647968656 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305596669 0 0) 0x7f46e80cb800 con 0x7f46e80cf160 2023-10-30T12:44:34.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e5d9b640 1 -- 172.21.15.12:0/2647968656 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46d4003530 con 0x7f46e80cf160 2023-10-30T12:44:34.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e5d9b640 1 -- 172.21.15.12:0/2647968656 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1358046190 0 0) 0x7f46d4003530 con 0x7f46e80cf160 2023-10-30T12:44:34.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e5d9b640 1 -- 172.21.15.12:0/2647968656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46e815d6f0 msgr2=0x7f46e815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:34.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e5d9b640 1 --2- 172.21.15.12:0/2647968656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46e815d6f0 0x7f46e815a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:34.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e5d9b640 1 -- 172.21.15.12:0/2647968656 >> v1:172.21.15.154:6789/0 conn(0x7f46e81538a0 legacy=0x7f46e80ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:34.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e5d9b640 1 -- 172.21.15.12:0/2647968656 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e80cb050 con 0x7f46e80cf160 2023-10-30T12:44:34.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e5d9b640 1 -- 172.21.15.12:0/2647968656 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f46e0001a00 con 0x7f46e80cf160 2023-10-30T12:44:34.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e5d9b640 1 -- 172.21.15.12:0/2647968656 >> v1:172.21.15.12:6789/0 conn(0x7f46e80cf160 legacy=0x7f46e8164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:34.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e5d9b640 1 --2- 172.21.15.12:0/2647968656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d40039b0 0x7f46d4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:34.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e5d9b640 1 --2- 172.21.15.12:0/2647968656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46d4004360 0x7f46d4004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:34.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e5d9b640 1 --2- 172.21.15.12:0/2647968656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46e815d6f0 0x7f46d4005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:34.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e5d9b640 1 -- 172.21.15.12:0/2647968656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e80cad70 con 0x7f46d40039b0 2023-10-30T12:44:34.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e5d9b640 1 -- 172.21.15.12:0/2647968656 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d400b9c0 con 0x7f46d4004360 2023-10-30T12:44:34.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e5d9b640 1 -- 172.21.15.12:0/2647968656 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d400bb80 con 0x7f46e815d6f0 2023-10-30T12:44:34.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e659c640 1 --2- 172.21.15.12:0/2647968656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d40039b0 0x7f46d4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:34.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46dffff640 1 --2- 172.21.15.12:0/2647968656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46d4004360 0x7f46d4004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:34.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e6d9d640 1 --2- 172.21.15.12:0/2647968656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46e815d6f0 0x7f46d4005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:34.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e659c640 1 -- 172.21.15.12:0/2647968656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46e815d6f0 msgr2=0x7f46d4005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:34.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e659c640 1 --2- 172.21.15.12:0/2647968656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46e815d6f0 0x7f46d4005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:34.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e659c640 1 -- 172.21.15.12:0/2647968656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46d4004360 msgr2=0x7f46d4004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:34.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e659c640 1 --2- 172.21.15.12:0/2647968656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46d4004360 0x7f46d4004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:34.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.424+0000 7f46e659c640 1 -- 172.21.15.12:0/2647968656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d4003530 con 0x7f46d40039b0 2023-10-30T12:44:34.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.428+0000 7f46e659c640 1 --2- 172.21.15.12:0/2647968656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d40039b0 0x7f46d4003d90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f46d8001860 tx=0x7f46d801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0822694df6b0e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:34.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.428+0000 7f46e5d9b640 1 -- 172.21.15.12:0/2647968656 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d80a8890 con 0x7f46d40039b0 2023-10-30T12:44:34.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.428+0000 7f46e5d9b640 1 -- 172.21.15.12:0/2647968656 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d80a89f0 con 0x7f46d40039b0 2023-10-30T12:44:34.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.428+0000 7f46e5d9b640 1 -- 172.21.15.12:0/2647968656 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d80a8da0 con 0x7f46d40039b0 2023-10-30T12:44:34.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.428+0000 7f46eca46640 1 -- 172.21.15.12:0/2647968656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d40039b0 msgr2=0x7f46d4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:34.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.428+0000 7f46eca46640 1 --2- 172.21.15.12:0/2647968656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d40039b0 0x7f46d4003d90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f46d8001860 tx=0x7f46d801ec70 comp rx=0 tx=0).stop 2023-10-30T12:44:34.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.428+0000 7f46e6d9d640 1 -- 172.21.15.12:0/2647968656 reap_dead start 2023-10-30T12:44:34.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.428+0000 7f46eca46640 1 -- 172.21.15.12:0/2647968656 shutdown_connections 2023-10-30T12:44:34.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.428+0000 7f46eca46640 1 -- 172.21.15.12:0/2647968656 >> 172.21.15.12:0/2647968656 conn(0x7f46e806bfe0 msgr2=0x7f46e806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:34.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.428+0000 7f46eca46640 1 -- 172.21.15.12:0/2647968656 shutdown_connections 2023-10-30T12:44:34.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.428+0000 7f46eca46640 1 -- 172.21.15.12:0/2647968656 wait complete. 2023-10-30T12:44:34.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.428+0000 7f46eca46640 1 Processor -- start 2023-10-30T12:44:34.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46eca46640 1 -- start start 2023-10-30T12:44:34.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46eca46640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d40039b0 0x7f46e81ef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:34.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46eca46640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46e8154000 0x7f46e814c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:34.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46eca46640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46e814cd60 0x7f46e814db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:34.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46eca46640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e81f3150 con 0x7f46d40039b0 2023-10-30T12:44:34.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46eca46640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e81eeef0 con 0x7f46e814cd60 2023-10-30T12:44:34.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46eca46640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e80ce130 con 0x7f46e8154000 2023-10-30T12:44:34.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46e659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d40039b0 0x7f46e81ef6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:34.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46e659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d40039b0 0x7f46e81ef6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41564/0 (socket says 172.21.15.12:41564) 2023-10-30T12:44:34.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46e659c640 1 -- 172.21.15.12:0/311705701 learned_addr learned my addr 172.21.15.12:0/311705701 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:34.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46e6d9d640 1 --2- 172.21.15.12:0/311705701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46e814cd60 0x7f46e814db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:34.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46dffff640 1 --2- 172.21.15.12:0/311705701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46e8154000 0x7f46e814c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:34.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46e659c640 1 -- 172.21.15.12:0/311705701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46e8154000 msgr2=0x7f46e814c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:34.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46e659c640 1 --2- 172.21.15.12:0/311705701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46e8154000 0x7f46e814c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:34.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46e659c640 1 -- 172.21.15.12:0/311705701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46e814cd60 msgr2=0x7f46e814db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:34.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46e659c640 1 --2- 172.21.15.12:0/311705701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46e814cd60 0x7f46e814db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:34.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46e659c640 1 -- 172.21.15.12:0/311705701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e80ce3f0 con 0x7f46d40039b0 2023-10-30T12:44:34.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46e659c640 1 --2- 172.21.15.12:0/311705701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d40039b0 0x7f46e81ef6b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f46d80b3040 tx=0x7f46d4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:34.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46deffd640 1 -- 172.21.15.12:0/311705701 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d8005270 con 0x7f46d40039b0 2023-10-30T12:44:34.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46eca46640 1 -- 172.21.15.12:0/311705701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e80ce6d0 con 0x7f46d40039b0 2023-10-30T12:44:34.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46deffd640 1 -- 172.21.15.12:0/311705701 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d80053d0 con 0x7f46d40039b0 2023-10-30T12:44:34.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46deffd640 1 -- 172.21.15.12:0/311705701 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d80069e0 con 0x7f46d40039b0 2023-10-30T12:44:34.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46eca46640 1 -- 172.21.15.12:0/311705701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e8158770 con 0x7f46d40039b0 2023-10-30T12:44:34.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.432+0000 7f46eca46640 1 -- 172.21.15.12:0/311705701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e804e170 con 0x7f46d40039b0 2023-10-30T12:44:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.436+0000 7f46deffd640 1 -- 172.21.15.12:0/311705701 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f46d80b7040 con 0x7f46d40039b0 2023-10-30T12:44:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.436+0000 7f46deffd640 1 --2- 172.21.15.12:0/311705701 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f46c80664d0 0x7f46c8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.436+0000 7f46deffd640 1 -- 172.21.15.12:0/311705701 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f46d8120680 con 0x7f46d40039b0 2023-10-30T12:44:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.436+0000 7f46dffff640 1 --2- 172.21.15.12:0/311705701 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f46c80664d0 0x7f46c8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.436+0000 7f46dffff640 1 --2- 172.21.15.12:0/311705701 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f46c80664d0 0x7f46c8068990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f46e81573a0 tx=0x7f46d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.440+0000 7f46deffd640 1 -- 172.21.15.12:0/311705701 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f46d80ed9a0 con 0x7f46d40039b0 2023-10-30T12:44:34.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.628+0000 7f46eca46640 1 -- 172.21.15.12:0/311705701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46e80b1dd0 con 0x7f46d40039b0 2023-10-30T12:44:34.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.628+0000 7f46deffd640 1 -- 172.21.15.12:0/311705701 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6527 (secure 0 0 0) 0x7f46d80a8850 con 0x7f46d40039b0 2023-10-30T12:44:34.636 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:34.637 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:43:46.382577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4285,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.12:6836/291392212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":291392212},{"type":"v1","addr":"172.21.15.12:6836","nonce":291392212}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:34.637 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 43 2023-10-30T12:44:34.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.628+0000 7f46eca46640 1 -- 172.21.15.12:0/311705701 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f46c80664d0 msgr2=0x7f46c8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:34.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.628+0000 7f46eca46640 1 --2- 172.21.15.12:0/311705701 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f46c80664d0 0x7f46c8068990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f46e81573a0 tx=0x7f46d0008040 comp rx=0 tx=0).stop 2023-10-30T12:44:34.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.628+0000 7f46eca46640 1 -- 172.21.15.12:0/311705701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d40039b0 msgr2=0x7f46e81ef6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:34.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.632+0000 7f46eca46640 1 --2- 172.21.15.12:0/311705701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d40039b0 0x7f46e81ef6b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f46d80b3040 tx=0x7f46d4005800 comp rx=0 tx=0).stop 2023-10-30T12:44:34.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.632+0000 7f46eca46640 1 -- 172.21.15.12:0/311705701 shutdown_connections 2023-10-30T12:44:34.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.632+0000 7f46eca46640 1 --2- 172.21.15.12:0/311705701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46e814cd60 0x7f46e814db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:34.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.632+0000 7f46eca46640 1 --2- 172.21.15.12:0/311705701 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f46c80664d0 0x7f46c8068990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:34.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.632+0000 7f46eca46640 1 --2- 172.21.15.12:0/311705701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46e8154000 0x7f46e814c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:34.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.632+0000 7f46eca46640 1 --2- 172.21.15.12:0/311705701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d40039b0 0x7f46e81ef6b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:34.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.632+0000 7f46eca46640 1 -- 172.21.15.12:0/311705701 >> 172.21.15.12:0/311705701 conn(0x7f46e806bfe0 msgr2=0x7f46e815afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:34.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.632+0000 7f46eca46640 1 -- 172.21.15.12:0/311705701 shutdown_connections 2023-10-30T12:44:34.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:34.632+0000 7f46eca46640 1 -- 172.21.15.12:0/311705701 wait complete. 2023-10-30T12:44:34.657 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:44:35.181 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:44:35.185 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:44:36.658 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:36.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.804+0000 7fa72c91d640 1 Processor -- start 2023-10-30T12:44:36.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.804+0000 7fa72c91d640 1 -- start start 2023-10-30T12:44:36.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.804+0000 7fa72c91d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7280c6b80 0x7fa7280bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:36.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.804+0000 7fa72c91d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa7280c6f60 0x7fa7280c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:36.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.804+0000 7fa72c91d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa7280c8430 0x7fa72813ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:36.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.804+0000 7fa72c91d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7280c0a50 con 0x7fa7280c6f60 2023-10-30T12:44:36.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.804+0000 7fa72c91d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72806f6e0 con 0x7fa7280c8430 2023-10-30T12:44:36.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.804+0000 7fa72c91d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72806f9c0 con 0x7fa7280c6b80 2023-10-30T12:44:36.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.808+0000 7fa71ffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa7280c6f60 0x7fa7280c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:36.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.808+0000 7fa71ffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa7280c6f60 0x7fa7280c0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41580/0 (socket says 172.21.15.12:41580) 2023-10-30T12:44:36.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.808+0000 7fa71ffff640 1 -- 172.21.15.12:0/3290321092 learned_addr learned my addr 172.21.15.12:0/3290321092 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:36.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.808+0000 7fa72659c640 1 --2- 172.21.15.12:0/3290321092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7280c6b80 0x7fa7280bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.808+0000 7fa726d9d640 1 --2- 172.21.15.12:0/3290321092 >> v2:172.21.15.154:3300/0 conn(0x7fa7280c8430 0x7fa72813ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.808+0000 7fa71ffff640 1 -- 172.21.15.12:0/3290321092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7280c6b80 msgr2=0x7fa7280bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.808+0000 7fa71ffff640 1 --2- 172.21.15.12:0/3290321092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7280c6b80 0x7fa7280bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.808+0000 7fa71ffff640 1 -- 172.21.15.12:0/3290321092 >> v2:172.21.15.154:3300/0 conn(0x7fa7280c8430 msgr2=0x7fa72813ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.808+0000 7fa71ffff640 1 --2- 172.21.15.12:0/3290321092 >> v2:172.21.15.154:3300/0 conn(0x7fa7280c8430 0x7fa72813ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.808+0000 7fa71ffff640 1 -- 172.21.15.12:0/3290321092 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72806fca0 con 0x7fa7280c6f60 2023-10-30T12:44:36.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.808+0000 7fa71ffff640 1 --2- 172.21.15.12:0/3290321092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7280c6f60 0x7fa7280c0400 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa7280b0860 tx=0x7fa71001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d97eb708b8a919 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:36.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.808+0000 7fa725d9b640 1 -- 172.21.15.12:0/3290321092 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7100ac630 con 0x7fa7280c6f60 2023-10-30T12:44:36.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.808+0000 7fa725d9b640 1 -- 172.21.15.12:0/3290321092 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7100ac790 con 0x7fa7280c6f60 2023-10-30T12:44:36.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.808+0000 7fa725d9b640 1 -- 172.21.15.12:0/3290321092 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7100acb20 con 0x7fa7280c6f60 2023-10-30T12:44:36.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 -- 172.21.15.12:0/3290321092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7280c6f60 msgr2=0x7fa7280c0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:36.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 --2- 172.21.15.12:0/3290321092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7280c6f60 0x7fa7280c0400 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa7280b0860 tx=0x7fa71001c500 comp rx=0 tx=0).stop 2023-10-30T12:44:36.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 -- 172.21.15.12:0/3290321092 shutdown_connections 2023-10-30T12:44:36.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 --2- 172.21.15.12:0/3290321092 >> v2:172.21.15.154:3300/0 conn(0x7fa7280c8430 0x7fa72813ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 --2- 172.21.15.12:0/3290321092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7280c6f60 0x7fa7280c0400 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 --2- 172.21.15.12:0/3290321092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7280c6b80 0x7fa7280bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 -- 172.21.15.12:0/3290321092 >> 172.21.15.12:0/3290321092 conn(0x7fa7280c7910 msgr2=0x7fa7280c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 -- 172.21.15.12:0/3290321092 shutdown_connections 2023-10-30T12:44:36.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 -- 172.21.15.12:0/3290321092 wait complete. 2023-10-30T12:44:36.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 Processor -- start 2023-10-30T12:44:36.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 -- start start 2023-10-30T12:44:36.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7280c6b80 0x7fa72813a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:36.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7280c6f60 0x7fa72813cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:36.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7280c8430 0x7fa7281579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:36.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72813d2e0 con 0x7fa7280c8430 2023-10-30T12:44:36.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa728138180 con 0x7fa7280c6b80 2023-10-30T12:44:36.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa72c91d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa728137be0 con 0x7fa7280c6f60 2023-10-30T12:44:36.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa726d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7280c8430 0x7fa7281579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:36.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa726d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7280c8430 0x7fa7281579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41584/0 (socket says 172.21.15.12:41584) 2023-10-30T12:44:36.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.812+0000 7fa726d9d640 1 -- 172.21.15.12:0/3390785625 learned_addr learned my addr 172.21.15.12:0/3390785625 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:36.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa71ffff640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7280c6f60 0x7fa72813cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:36.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa72659c640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7280c6b80 0x7fa72813a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:36.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa726d9d640 1 -- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7280c6f60 msgr2=0x7fa72813cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:36.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa726d9d640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7280c6f60 0x7fa72813cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:36.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa726d9d640 1 -- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7280c6b80 msgr2=0x7fa72813a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:36.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa726d9d640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7280c6b80 0x7fa72813a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:36.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa726d9d640 1 -- 172.21.15.12:0/3390785625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7100094d0 con 0x7fa7280c8430 2023-10-30T12:44:36.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa71ffff640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7280c6f60 0x7fa72813cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:44:36.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa72659c640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7280c6b80 0x7fa72813a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:44:36.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa726d9d640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7280c8430 0x7fa7281579f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa720007e60 tx=0x7fa72000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:36.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa71effd640 1 -- 172.21.15.12:0/3390785625 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720014420 con 0x7fa7280c8430 2023-10-30T12:44:36.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa71effd640 1 -- 172.21.15.12:0/3390785625 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7200145a0 con 0x7fa7280c8430 2023-10-30T12:44:36.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa71effd640 1 -- 172.21.15.12:0/3390785625 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720014950 con 0x7fa7280c8430 2023-10-30T12:44:36.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa72c91d640 1 -- 172.21.15.12:0/3390785625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa728137ea0 con 0x7fa7280c8430 2023-10-30T12:44:36.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa72c91d640 1 -- 172.21.15.12:0/3390785625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa728158610 con 0x7fa7280c8430 2023-10-30T12:44:36.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa72c91d640 1 -- 172.21.15.12:0/3390785625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7280c1e40 con 0x7fa7280c8430 2023-10-30T12:44:36.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.816+0000 7fa71effd640 1 -- 172.21.15.12:0/3390785625 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa720014ab0 con 0x7fa7280c8430 2023-10-30T12:44:36.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.820+0000 7fa71effd640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7080664a0 0x7fa708068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:36.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.820+0000 7fa71effd640 1 -- 172.21.15.12:0/3390785625 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fa720084ab0 con 0x7fa7280c8430 2023-10-30T12:44:36.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.820+0000 7fa72659c640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7080664a0 0x7fa708068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:36.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.820+0000 7fa72659c640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7080664a0 0x7fa708068960 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa718004660 tx=0x7fa718008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:36.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:36.824+0000 7fa71effd640 1 -- 172.21.15.12:0/3390785625 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa720051c30 con 0x7fa7280c8430 2023-10-30T12:44:37.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.012+0000 7fa72c91d640 1 -- 172.21.15.12:0/3390785625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa72804e170 con 0x7fa7280c8430 2023-10-30T12:44:37.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.012+0000 7fa71effd640 1 -- 172.21.15.12:0/3390785625 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+5737 (secure 0 0 0) 0x7fa720055250 con 0x7fa7280c8430 2023-10-30T12:44:37.019 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:37.020 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:44:35.272605+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2023-10-30T12:44:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.012+0000 7fa72c91d640 1 -- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7080664a0 msgr2=0x7fa708068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.012+0000 7fa72c91d640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7080664a0 0x7fa708068960 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa718004660 tx=0x7fa718008040 comp rx=0 tx=0).stop 2023-10-30T12:44:37.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.012+0000 7fa72c91d640 1 -- 172.21.15.12:0/3390785625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7280c8430 msgr2=0x7fa7281579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:37.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.012+0000 7fa72c91d640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7280c8430 0x7fa7281579f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa720007e60 tx=0x7fa72000e4b0 comp rx=0 tx=0).stop 2023-10-30T12:44:37.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.012+0000 7fa72c91d640 1 -- 172.21.15.12:0/3390785625 shutdown_connections 2023-10-30T12:44:37.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.012+0000 7fa72c91d640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7280c8430 0x7fa7281579f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:37.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.012+0000 7fa72c91d640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7080664a0 0x7fa708068960 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:37.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.012+0000 7fa72c91d640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7280c6f60 0x7fa72813cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:37.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.012+0000 7fa72c91d640 1 --2- 172.21.15.12:0/3390785625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7280c6b80 0x7fa72813a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:37.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.012+0000 7fa72c91d640 1 -- 172.21.15.12:0/3390785625 >> 172.21.15.12:0/3390785625 conn(0x7fa7280c7910 msgr2=0x7fa7280c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:37.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.012+0000 7fa72c91d640 1 -- 172.21.15.12:0/3390785625 shutdown_connections 2023-10-30T12:44:37.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.012+0000 7fa72c91d640 1 -- 172.21.15.12:0/3390785625 wait complete. 2023-10-30T12:44:37.041 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2023-10-30T12:44:37.041 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:44:37.042 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.176+0000 7fa787e09640 1 Processor -- start 2023-10-30T12:44:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.176+0000 7fa787e09640 1 -- start start 2023-10-30T12:44:37.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.180+0000 7fa787e09640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa7800c6f60 0x7fa7801566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:37.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.180+0000 7fa787e09640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7800c0a50 con 0x7fa7800c6b80 2023-10-30T12:44:37.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.180+0000 7fa787e09640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78006f6e0 con 0x7fa7800c05d0 2023-10-30T12:44:37.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.180+0000 7fa787e09640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa78006f9c0 con 0x7fa7800c6f60 2023-10-30T12:44:37.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.180+0000 7fa785ba5640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa7800c6b80 0x7fa7800bfec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50032/0 (socket says 172.21.15.12:50032) 2023-10-30T12:44:37.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.180+0000 7fa785ba5640 1 -- 172.21.15.12:0/789463960 learned_addr learned my addr 172.21.15.12:0/789463960 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:44:37.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.180+0000 7fa7863a6640 1 --2- 172.21.15.12:0/789463960 >> v2:172.21.15.12:3300/0 conn(0x7fa7800c6f60 0x7fa7801566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:37.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.180+0000 7fa784ba3640 1 -- 172.21.15.12:0/789463960 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839077294 0 0) 0x7fa7800c0a50 con 0x7fa7800c6b80 2023-10-30T12:44:37.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.180+0000 7fa784ba3640 1 -- 172.21.15.12:0/789463960 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa76c003530 con 0x7fa7800c6b80 2023-10-30T12:44:37.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.180+0000 7fa7863a6640 1 -- 172.21.15.12:0/789463960 >> v1:172.21.15.154:6789/0 conn(0x7fa7800c05d0 legacy=0x7fa78013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:37.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.180+0000 7fa7863a6640 1 -- 172.21.15.12:0/789463960 >> v1:172.21.15.12:6789/0 conn(0x7fa7800c6b80 legacy=0x7fa7800bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:37.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.180+0000 7fa7863a6640 1 -- 172.21.15.12:0/789463960 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78006fca0 con 0x7fa7800c6f60 2023-10-30T12:44:37.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.180+0000 7fa7863a6640 1 --2- 172.21.15.12:0/789463960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7800c6f60 0x7fa7801566a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa778009d90 tx=0x7fa77801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc166268d2d80211 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:37.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa784ba3640 1 -- 172.21.15.12:0/789463960 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7780a5820 con 0x7fa7800c6f60 2023-10-30T12:44:37.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa784ba3640 1 -- 172.21.15.12:0/789463960 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7780a5980 con 0x7fa7800c6f60 2023-10-30T12:44:37.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa784ba3640 1 -- 172.21.15.12:0/789463960 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7780a5d10 con 0x7fa7800c6f60 2023-10-30T12:44:37.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa787e09640 1 -- 172.21.15.12:0/789463960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7800c6f60 msgr2=0x7fa7801566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:37.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa787e09640 1 --2- 172.21.15.12:0/789463960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7800c6f60 0x7fa7801566a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa778009d90 tx=0x7fa77801c950 comp rx=0 tx=0).stop 2023-10-30T12:44:37.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa787e09640 1 -- 172.21.15.12:0/789463960 shutdown_connections 2023-10-30T12:44:37.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa787e09640 1 --2- 172.21.15.12:0/789463960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7800c6f60 0x7fa7801566a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:37.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa787e09640 1 -- 172.21.15.12:0/789463960 >> 172.21.15.12:0/789463960 conn(0x7fa7800c7910 msgr2=0x7fa7800c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:37.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa787e09640 1 -- 172.21.15.12:0/789463960 shutdown_connections 2023-10-30T12:44:37.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa787e09640 1 -- 172.21.15.12:0/789463960 wait complete. 2023-10-30T12:44:37.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa787e09640 1 Processor -- start 2023-10-30T12:44:37.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa787e09640 1 -- start start 2023-10-30T12:44:37.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa787e09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7800c05d0 0x7fa780154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:37.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa787e09640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7800c6b80 0x7fa78006a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:37.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa787e09640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7800c6f60 0x7fa78013cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:37.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa787e09640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7800c9450 con 0x7fa7800c05d0 2023-10-30T12:44:37.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa787e09640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa780154c40 con 0x7fa7800c6f60 2023-10-30T12:44:37.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa787e09640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa780155cf0 con 0x7fa7800c6b80 2023-10-30T12:44:37.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa785ba5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7800c05d0 0x7fa780154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:37.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa785ba5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7800c05d0 0x7fa780154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41598/0 (socket says 172.21.15.12:41598) 2023-10-30T12:44:37.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.184+0000 7fa785ba5640 1 -- 172.21.15.12:0/2842269545 learned_addr learned my addr 172.21.15.12:0/2842269545 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:37.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa7853a4640 1 --2- 172.21.15.12:0/2842269545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7800c6b80 0x7fa78006a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:37.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa7863a6640 1 --2- 172.21.15.12:0/2842269545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7800c6f60 0x7fa78013cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:37.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa785ba5640 1 -- 172.21.15.12:0/2842269545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7800c6b80 msgr2=0x7fa78006a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:37.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa785ba5640 1 --2- 172.21.15.12:0/2842269545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7800c6b80 0x7fa78006a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:37.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa785ba5640 1 -- 172.21.15.12:0/2842269545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7800c6f60 msgr2=0x7fa78013cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:37.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa785ba5640 1 --2- 172.21.15.12:0/2842269545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7800c6f60 0x7fa78013cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:37.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa785ba5640 1 -- 172.21.15.12:0/2842269545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa778009a70 con 0x7fa7800c05d0 2023-10-30T12:44:37.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa785ba5640 1 --2- 172.21.15.12:0/2842269545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7800c05d0 0x7fa780154700 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00c500 tx=0x7fa77c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:37.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa7767fc640 1 -- 172.21.15.12:0/2842269545 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c005cf0 con 0x7fa7800c05d0 2023-10-30T12:44:37.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa7767fc640 1 -- 172.21.15.12:0/2842269545 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77c005e50 con 0x7fa7800c05d0 2023-10-30T12:44:37.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa787e09640 1 -- 172.21.15.12:0/2842269545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7800c9ad0 con 0x7fa7800c05d0 2023-10-30T12:44:37.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa7767fc640 1 -- 172.21.15.12:0/2842269545 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c0072f0 con 0x7fa7800c05d0 2023-10-30T12:44:37.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa787e09640 1 -- 172.21.15.12:0/2842269545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7800c91b0 con 0x7fa7800c05d0 2023-10-30T12:44:37.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa787e09640 1 -- 172.21.15.12:0/2842269545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7800c9ce0 con 0x7fa7800c05d0 2023-10-30T12:44:37.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa7767fc640 1 -- 172.21.15.12:0/2842269545 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa77c007450 con 0x7fa7800c05d0 2023-10-30T12:44:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa7767fc640 1 --2- 172.21.15.12:0/2842269545 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7640664a0 0x7fa764068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.188+0000 7fa7767fc640 1 -- 172.21.15.12:0/2842269545 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fa77c0835b0 con 0x7fa7800c05d0 2023-10-30T12:44:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.192+0000 7fa7853a4640 1 --2- 172.21.15.12:0/2842269545 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7640664a0 0x7fa764068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.192+0000 7fa7853a4640 1 --2- 172.21.15.12:0/2842269545 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7640664a0 0x7fa764068960 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa78006b0d0 tx=0x7fa770009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.192+0000 7fa7767fc640 1 -- 172.21.15.12:0/2842269545 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa7800c9ce0 con 0x7fa7800c05d0 2023-10-30T12:44:37.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.380+0000 7fa787e09640 1 -- 172.21.15.12:0/2842269545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7800c9ce0 con 0x7fa7800c05d0 2023-10-30T12:44:37.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.380+0000 7fa7767fc640 1 -- 172.21.15.12:0/2842269545 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+5737 (secure 0 0 0) 0x7fa7800c9ce0 con 0x7fa7800c05d0 2023-10-30T12:44:37.390 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:37.390 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:44:35.272605+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:37.390 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2023-10-30T12:44:37.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.384+0000 7fa787e09640 1 -- 172.21.15.12:0/2842269545 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7640664a0 msgr2=0x7fa764068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:37.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.384+0000 7fa787e09640 1 --2- 172.21.15.12:0/2842269545 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7640664a0 0x7fa764068960 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa78006b0d0 tx=0x7fa770009040 comp rx=0 tx=0).stop 2023-10-30T12:44:37.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.384+0000 7fa787e09640 1 -- 172.21.15.12:0/2842269545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7800c05d0 msgr2=0x7fa780154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:37.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.384+0000 7fa787e09640 1 --2- 172.21.15.12:0/2842269545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7800c05d0 0x7fa780154700 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00c500 tx=0x7fa77c00c9c0 comp rx=0 tx=0).stop 2023-10-30T12:44:37.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.384+0000 7fa787e09640 1 -- 172.21.15.12:0/2842269545 shutdown_connections 2023-10-30T12:44:37.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.384+0000 7fa787e09640 1 --2- 172.21.15.12:0/2842269545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7800c6f60 0x7fa78013cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:37.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.384+0000 7fa787e09640 1 --2- 172.21.15.12:0/2842269545 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa7640664a0 0x7fa764068960 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:37.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.384+0000 7fa787e09640 1 --2- 172.21.15.12:0/2842269545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa7800c6b80 0x7fa78006a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:37.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.384+0000 7fa787e09640 1 --2- 172.21.15.12:0/2842269545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7800c05d0 0x7fa780154700 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:37.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.384+0000 7fa787e09640 1 -- 172.21.15.12:0/2842269545 >> 172.21.15.12:0/2842269545 conn(0x7fa7800c7910 msgr2=0x7fa78014cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:37.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.384+0000 7fa787e09640 1 -- 172.21.15.12:0/2842269545 shutdown_connections 2023-10-30T12:44:37.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:37.384+0000 7fa787e09640 1 -- 172.21.15.12:0/2842269545 wait complete. 2023-10-30T12:44:37.411 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 44, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 44, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4971": { "addr": "172.21.15.12:6837/3124135109", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 3124135109, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 3124135109, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4971, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 44 }, "gid_5000": { "addr": "172.21.15.154:6839/1635719967", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1635719967, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 1635719967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5000, "incarnation": 44, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 }, "gid_5124": { "addr": "172.21.15.12:6835/559649789", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 559649789, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 559649789, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5124, "incarnation": 36, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 39 } }, "last_failure": 0, "last_failure_osd_epoch": 49, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:44:35.272605+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5000, "mds_1": 5124, "mds_2": 4971 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:44:39.412 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:39.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.560+0000 7fb5e8e79640 1 Processor -- start 2023-10-30T12:44:39.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.560+0000 7fb5e8e79640 1 -- start start 2023-10-30T12:44:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.560+0000 7fb5e8e79640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb5e4151800 0x7fb5e406e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:39.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.560+0000 7fb5e8e79640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5e40c1ee0 0x7fb5e40c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:39.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.560+0000 7fb5e8e79640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e406e090 con 0x7fb5e40c9230 2023-10-30T12:44:39.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.560+0000 7fb5e8e79640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e406dc70 con 0x7fb5e4151800 2023-10-30T12:44:39.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.560+0000 7fb5e8e79640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40c8a50 con 0x7fb5e40c1ee0 2023-10-30T12:44:39.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.560+0000 7fb5e1d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5e40c1ee0 0x7fb5e40c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:39.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.560+0000 7fb5e259c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb5e4151800 0x7fb5e406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.560+0000 7fb5e1d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5e40c1ee0 0x7fb5e40c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:51732/0 (socket says 172.21.15.12:51732) 2023-10-30T12:44:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.560+0000 7fb5e1d9b640 1 -- 172.21.15.12:0/391811978 learned_addr learned my addr 172.21.15.12:0/391811978 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.564+0000 7fb5e259c640 1 -- 172.21.15.12:0/391811978 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5e40c1ee0 msgr2=0x7fb5e40c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.564+0000 7fb5e259c640 1 --2- 172.21.15.12:0/391811978 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5e40c1ee0 0x7fb5e40c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.564+0000 7fb5e259c640 1 -- 172.21.15.12:0/391811978 >> v1:172.21.15.154:6789/0 conn(0x7fb5e40c9230 legacy=0x7fb5e40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:39.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.564+0000 7fb5e259c640 1 -- 172.21.15.12:0/391811978 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e40c3b10 con 0x7fb5e4151800 2023-10-30T12:44:39.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.564+0000 7fb5e1d9b640 1 --2- 172.21.15.12:0/391811978 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5e40c1ee0 0x7fb5e40c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:44:39.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.564+0000 7fb5e259c640 1 --2- 172.21.15.12:0/391811978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5e4151800 0x7fb5e406e470 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40097c0 tx=0x7fb5d401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6efc334f2452b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e159a640 1 -- 172.21.15.12:0/391811978 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d40ab650 con 0x7fb5e4151800 2023-10-30T12:44:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e159a640 1 -- 172.21.15.12:0/391811978 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d40ab7b0 con 0x7fb5e4151800 2023-10-30T12:44:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e159a640 1 -- 172.21.15.12:0/391811978 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d40abb40 con 0x7fb5e4151800 2023-10-30T12:44:39.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 -- 172.21.15.12:0/391811978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5e4151800 msgr2=0x7fb5e406e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:39.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 --2- 172.21.15.12:0/391811978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5e4151800 0x7fb5e406e470 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40097c0 tx=0x7fb5d401c450 comp rx=0 tx=0).stop 2023-10-30T12:44:39.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 -- 172.21.15.12:0/391811978 shutdown_connections 2023-10-30T12:44:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 --2- 172.21.15.12:0/391811978 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5e40c1ee0 0x7fb5e40c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 --2- 172.21.15.12:0/391811978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5e4151800 0x7fb5e406e470 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 -- 172.21.15.12:0/391811978 >> 172.21.15.12:0/391811978 conn(0x7fb5e4071ea0 msgr2=0x7fb5e4065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 -- 172.21.15.12:0/391811978 shutdown_connections 2023-10-30T12:44:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 -- 172.21.15.12:0/391811978 wait complete. 2023-10-30T12:44:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 Processor -- start 2023-10-30T12:44:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 -- start start 2023-10-30T12:44:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5e40c1ee0 0x7fb5e4151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:39.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5e40c9230 0x7fb5e40cc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:39.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5e4151800 0x7fb5e414a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:39.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e414a0a0 con 0x7fb5e40c9230 2023-10-30T12:44:39.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e41f5910 con 0x7fb5e4151800 2023-10-30T12:44:39.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.568+0000 7fb5e8e79640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e41f5bf0 con 0x7fb5e40c1ee0 2023-10-30T12:44:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5e1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5e40c9230 0x7fb5e40cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5e259c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5e40c1ee0 0x7fb5e4151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5e1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5e40c9230 0x7fb5e40cc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41602/0 (socket says 172.21.15.12:41602) 2023-10-30T12:44:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5e1d9b640 1 -- 172.21.15.12:0/2359089522 learned_addr learned my addr 172.21.15.12:0/2359089522 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5e259c640 1 --2- 172.21.15.12:0/2359089522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5e40c1ee0 0x7fb5e4151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:51736/0 (socket says 172.21.15.12:51736) 2023-10-30T12:44:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5e2d9d640 1 --2- 172.21.15.12:0/2359089522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5e4151800 0x7fb5e414a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5e1d9b640 1 -- 172.21.15.12:0/2359089522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5e40c1ee0 msgr2=0x7fb5e4151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5e1d9b640 1 --2- 172.21.15.12:0/2359089522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5e40c1ee0 0x7fb5e4151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5e1d9b640 1 -- 172.21.15.12:0/2359089522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5e4151800 msgr2=0x7fb5e414a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5e1d9b640 1 --2- 172.21.15.12:0/2359089522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5e4151800 0x7fb5e414a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5e1d9b640 1 -- 172.21.15.12:0/2359089522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d80094d0 con 0x7fb5e40c9230 2023-10-30T12:44:39.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5e1d9b640 1 --2- 172.21.15.12:0/2359089522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5e40c9230 0x7fb5e40cc050 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800e9b0 tx=0x7fb5d800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:39.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5d2ffd640 1 -- 172.21.15.12:0/2359089522 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d800c9a0 con 0x7fb5e40c9230 2023-10-30T12:44:39.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5d2ffd640 1 -- 172.21.15.12:0/2359089522 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d800cb00 con 0x7fb5e40c9230 2023-10-30T12:44:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5d2ffd640 1 -- 172.21.15.12:0/2359089522 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8013470 con 0x7fb5e40c9230 2023-10-30T12:44:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5e8e79640 1 -- 172.21.15.12:0/2359089522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d4009420 con 0x7fb5e40c9230 2023-10-30T12:44:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5e8e79640 1 -- 172.21.15.12:0/2359089522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e41f61b0 con 0x7fb5e40c9230 2023-10-30T12:44:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5e8e79640 1 -- 172.21.15.12:0/2359089522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e404e170 con 0x7fb5e40c9230 2023-10-30T12:44:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.572+0000 7fb5d2ffd640 1 -- 172.21.15.12:0/2359089522 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb5d80135d0 con 0x7fb5e40c9230 2023-10-30T12:44:39.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.576+0000 7fb5d2ffd640 1 --2- 172.21.15.12:0/2359089522 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb5c40664d0 0x7fb5c4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:39.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.576+0000 7fb5d2ffd640 1 -- 172.21.15.12:0/2359089522 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fb5d8084200 con 0x7fb5e40c9230 2023-10-30T12:44:39.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.576+0000 7fb5e259c640 1 --2- 172.21.15.12:0/2359089522 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb5c40664d0 0x7fb5c4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:39.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.576+0000 7fb5e259c640 1 --2- 172.21.15.12:0/2359089522 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb5c40664d0 0x7fb5c4068990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40af000 tx=0x7fb5d40a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:39.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.580+0000 7fb5d2ffd640 1 -- 172.21.15.12:0/2359089522 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb5d80514c0 con 0x7fb5e40c9230 2023-10-30T12:44:39.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.768+0000 7fb5e8e79640 1 -- 172.21.15.12:0/2359089522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5e40c6de0 con 0x7fb5e40c9230 2023-10-30T12:44:39.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.768+0000 7fb5d2ffd640 1 -- 172.21.15.12:0/2359089522 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+5737 (secure 0 0 0) 0x7fb5d8054ae0 con 0x7fb5e40c9230 2023-10-30T12:44:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:39.776 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:44:35.272605+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:39.777 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2023-10-30T12:44:39.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.768+0000 7fb5e8e79640 1 -- 172.21.15.12:0/2359089522 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb5c40664d0 msgr2=0x7fb5c4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:39.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.768+0000 7fb5e8e79640 1 --2- 172.21.15.12:0/2359089522 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb5c40664d0 0x7fb5c4068990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40af000 tx=0x7fb5d40a7000 comp rx=0 tx=0).stop 2023-10-30T12:44:39.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.768+0000 7fb5e8e79640 1 -- 172.21.15.12:0/2359089522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5e40c9230 msgr2=0x7fb5e40cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.768+0000 7fb5e8e79640 1 --2- 172.21.15.12:0/2359089522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5e40c9230 0x7fb5e40cc050 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800e9b0 tx=0x7fb5d800ee70 comp rx=0 tx=0).stop 2023-10-30T12:44:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.768+0000 7fb5e8e79640 1 -- 172.21.15.12:0/2359089522 shutdown_connections 2023-10-30T12:44:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.768+0000 7fb5e8e79640 1 --2- 172.21.15.12:0/2359089522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5e4151800 0x7fb5e414a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.768+0000 7fb5e8e79640 1 --2- 172.21.15.12:0/2359089522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5e40c9230 0x7fb5e40cc050 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.768+0000 7fb5e8e79640 1 --2- 172.21.15.12:0/2359089522 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb5c40664d0 0x7fb5c4068990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:39.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.768+0000 7fb5e8e79640 1 --2- 172.21.15.12:0/2359089522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5e40c1ee0 0x7fb5e4151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:39.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.768+0000 7fb5e8e79640 1 -- 172.21.15.12:0/2359089522 >> 172.21.15.12:0/2359089522 conn(0x7fb5e4071ea0 msgr2=0x7fb5e40c7520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:39.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.768+0000 7fb5e8e79640 1 -- 172.21.15.12:0/2359089522 shutdown_connections 2023-10-30T12:44:39.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:39.768+0000 7fb5e8e79640 1 -- 172.21.15.12:0/2359089522 wait complete. 2023-10-30T12:44:39.796 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:44:41.797 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:41.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.944+0000 7f2b77f3e640 1 Processor -- start 2023-10-30T12:44:41.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.944+0000 7f2b77f3e640 1 -- start start 2023-10-30T12:44:41.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.944+0000 7f2b77f3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b7014fd50 0x7f2b700c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:41.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.948+0000 7f2b77f3e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b701560a0 con 0x7f2b70065ee0 2023-10-30T12:44:41.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.948+0000 7f2b77f3e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b70156380 con 0x7f2b70065a80 2023-10-30T12:44:41.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.948+0000 7f2b77f3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b70156660 con 0x7f2b7014fd50 2023-10-30T12:44:41.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.948+0000 7f2b764db640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2b70065ee0 0x7f2b70071250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:46254/0 (socket says 172.21.15.12:46254) 2023-10-30T12:44:41.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.948+0000 7f2b764db640 1 -- 172.21.15.12:0/1812141428 learned_addr learned my addr 172.21.15.12:0/1812141428 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:44:41.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.948+0000 7f2b75cda640 1 --2- 172.21.15.12:0/1812141428 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b7014fd50 0x7f2b700c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:41.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.948+0000 7f2b74cd8640 1 -- 172.21.15.12:0/1812141428 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992243004 0 0) 0x7f2b701560a0 con 0x7f2b70065ee0 2023-10-30T12:44:41.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.948+0000 7f2b74cd8640 1 -- 172.21.15.12:0/1812141428 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b5c005740 con 0x7f2b70065ee0 2023-10-30T12:44:41.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.948+0000 7f2b74cd8640 1 -- 172.21.15.12:0/1812141428 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678629949 0 0) 0x7f2b70156380 con 0x7f2b70065a80 2023-10-30T12:44:41.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.948+0000 7f2b75cda640 1 -- 172.21.15.12:0/1812141428 >> v1:172.21.15.154:6789/0 conn(0x7f2b70065a80 legacy=0x7f2b700cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:41.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.948+0000 7f2b75cda640 1 -- 172.21.15.12:0/1812141428 >> v1:172.21.15.12:6789/0 conn(0x7f2b70065ee0 legacy=0x7f2b70071250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:41.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.948+0000 7f2b75cda640 1 -- 172.21.15.12:0/1812141428 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b70156940 con 0x7f2b7014fd50 2023-10-30T12:44:41.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.948+0000 7f2b75cda640 1 --2- 172.21.15.12:0/1812141428 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b7014fd50 0x7f2b700c65d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0097f0 tx=0x7f2b6c01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46e0f9fd1cf6096f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:41.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b74cd8640 1 -- 172.21.15.12:0/1812141428 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c004230 con 0x7f2b7014fd50 2023-10-30T12:44:41.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b74cd8640 1 -- 172.21.15.12:0/1812141428 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6c0a7050 con 0x7f2b7014fd50 2023-10-30T12:44:41.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b74cd8640 1 -- 172.21.15.12:0/1812141428 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c003620 con 0x7f2b7014fd50 2023-10-30T12:44:41.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b77f3e640 1 -- 172.21.15.12:0/1812141428 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b7014fd50 msgr2=0x7f2b700c65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:41.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b77f3e640 1 --2- 172.21.15.12:0/1812141428 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b7014fd50 0x7f2b700c65d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0097f0 tx=0x7f2b6c01c490 comp rx=0 tx=0).stop 2023-10-30T12:44:41.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b77f3e640 1 -- 172.21.15.12:0/1812141428 shutdown_connections 2023-10-30T12:44:41.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b77f3e640 1 --2- 172.21.15.12:0/1812141428 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b7014fd50 0x7f2b700c65d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:41.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b77f3e640 1 -- 172.21.15.12:0/1812141428 >> 172.21.15.12:0/1812141428 conn(0x7f2b7006c6e0 msgr2=0x7f2b70069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:41.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b77f3e640 1 -- 172.21.15.12:0/1812141428 shutdown_connections 2023-10-30T12:44:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b77f3e640 1 -- 172.21.15.12:0/1812141428 wait complete. 2023-10-30T12:44:41.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b77f3e640 1 Processor -- start 2023-10-30T12:44:41.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b77f3e640 1 -- start start 2023-10-30T12:44:41.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b77f3e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b70065a80 0x7f2b70160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:41.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b77f3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b70065ee0 0x7f2b701530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:41.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b77f3e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b7014fd50 0x7f2b70160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:41.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b77f3e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7015fc70 con 0x7f2b70065a80 2023-10-30T12:44:41.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b77f3e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b70150e40 con 0x7f2b7014fd50 2023-10-30T12:44:41.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b77f3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b701f1e20 con 0x7f2b70065ee0 2023-10-30T12:44:41.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b754d9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b70065ee0 0x7f2b701530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:41.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b75cda640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b70065a80 0x7f2b70160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:41.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b754d9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b70065ee0 0x7f2b701530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:38026/0 (socket says 172.21.15.12:38026) 2023-10-30T12:44:41.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b754d9640 1 -- 172.21.15.12:0/3578422223 learned_addr learned my addr 172.21.15.12:0/3578422223 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:41.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.952+0000 7f2b764db640 1 --2- 172.21.15.12:0/3578422223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b7014fd50 0x7f2b70160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:41.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.956+0000 7f2b754d9640 1 -- 172.21.15.12:0/3578422223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b7014fd50 msgr2=0x7f2b70160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:41.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.956+0000 7f2b754d9640 1 --2- 172.21.15.12:0/3578422223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b7014fd50 0x7f2b70160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:41.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.956+0000 7f2b754d9640 1 -- 172.21.15.12:0/3578422223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b70065a80 msgr2=0x7f2b70160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:41.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.956+0000 7f2b754d9640 1 --2- 172.21.15.12:0/3578422223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b70065a80 0x7f2b70160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:41.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.956+0000 7f2b754d9640 1 -- 172.21.15.12:0/3578422223 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c009420 con 0x7f2b70065ee0 2023-10-30T12:44:41.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.956+0000 7f2b754d9640 1 --2- 172.21.15.12:0/3578422223 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b70065ee0 0x7f2b701530c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2b60001ec0 tx=0x7f2b6000c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:41.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.956+0000 7f2b667fc640 1 -- 172.21.15.12:0/3578422223 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b600054e0 con 0x7f2b70065ee0 2023-10-30T12:44:41.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.956+0000 7f2b667fc640 1 -- 172.21.15.12:0/3578422223 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b60003600 con 0x7f2b70065ee0 2023-10-30T12:44:41.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.956+0000 7f2b77f3e640 1 -- 172.21.15.12:0/3578422223 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b701f2080 con 0x7f2b70065ee0 2023-10-30T12:44:41.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.956+0000 7f2b667fc640 1 -- 172.21.15.12:0/3578422223 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b600039b0 con 0x7f2b70065ee0 2023-10-30T12:44:41.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.956+0000 7f2b77f3e640 1 -- 172.21.15.12:0/3578422223 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b701f2640 con 0x7f2b70065ee0 2023-10-30T12:44:41.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.956+0000 7f2b77f3e640 1 -- 172.21.15.12:0/3578422223 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b700c2850 con 0x7f2b70065ee0 2023-10-30T12:44:41.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.956+0000 7f2b667fc640 1 -- 172.21.15.12:0/3578422223 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2b60003b10 con 0x7f2b70065ee0 2023-10-30T12:44:41.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.956+0000 7f2b667fc640 1 --2- 172.21.15.12:0/3578422223 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2b540664a0 0x7f2b54068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:41.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.960+0000 7f2b667fc640 1 -- 172.21.15.12:0/3578422223 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f2b60083800 con 0x7f2b70065ee0 2023-10-30T12:44:41.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.960+0000 7f2b75cda640 1 --2- 172.21.15.12:0/3578422223 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2b540664a0 0x7f2b54068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:41.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.960+0000 7f2b75cda640 1 --2- 172.21.15.12:0/3578422223 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2b540664a0 0x7f2b54068960 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0ad000 tx=0x7f2b6c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:41.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:41.964+0000 7f2b667fc640 1 -- 172.21.15.12:0/3578422223 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2b60050ab0 con 0x7f2b70065ee0 2023-10-30T12:44:42.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:42.156+0000 7f2b77f3e640 1 -- 172.21.15.12:0/3578422223 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b7015fdb0 con 0x7f2b70065ee0 2023-10-30T12:44:42.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:42.156+0000 7f2b667fc640 1 -- 172.21.15.12:0/3578422223 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+5737 (secure 0 0 0) 0x7f2b7015fdb0 con 0x7f2b70065ee0 2023-10-30T12:44:42.164 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:42.164 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:44:35.272605+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:42.164 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2023-10-30T12:44:42.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:42.156+0000 7f2b77f3e640 1 -- 172.21.15.12:0/3578422223 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2b540664a0 msgr2=0x7f2b54068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:42.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:42.156+0000 7f2b77f3e640 1 --2- 172.21.15.12:0/3578422223 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2b540664a0 0x7f2b54068960 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0ad000 tx=0x7f2b6c0a6000 comp rx=0 tx=0).stop 2023-10-30T12:44:42.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:42.156+0000 7f2b77f3e640 1 -- 172.21.15.12:0/3578422223 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b70065ee0 msgr2=0x7f2b701530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:42.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:42.156+0000 7f2b77f3e640 1 --2- 172.21.15.12:0/3578422223 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b70065ee0 0x7f2b701530c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2b60001ec0 tx=0x7f2b6000c8e0 comp rx=0 tx=0).stop 2023-10-30T12:44:42.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:42.156+0000 7f2b77f3e640 1 -- 172.21.15.12:0/3578422223 shutdown_connections 2023-10-30T12:44:42.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:42.156+0000 7f2b77f3e640 1 --2- 172.21.15.12:0/3578422223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b7014fd50 0x7f2b70160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:42.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:42.156+0000 7f2b77f3e640 1 --2- 172.21.15.12:0/3578422223 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2b540664a0 0x7f2b54068960 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:42.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:42.156+0000 7f2b77f3e640 1 --2- 172.21.15.12:0/3578422223 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b70065ee0 0x7f2b701530c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:42.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:42.156+0000 7f2b77f3e640 1 --2- 172.21.15.12:0/3578422223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b70065a80 0x7f2b70160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:42.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:42.156+0000 7f2b77f3e640 1 -- 172.21.15.12:0/3578422223 >> 172.21.15.12:0/3578422223 conn(0x7f2b7006c6e0 msgr2=0x7f2b701491f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:42.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:42.160+0000 7f2b77f3e640 1 -- 172.21.15.12:0/3578422223 shutdown_connections 2023-10-30T12:44:42.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:42.160+0000 7f2b77f3e640 1 -- 172.21.15.12:0/3578422223 wait complete. 2023-10-30T12:44:42.185 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:44:44.186 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:44.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.336+0000 7f60ec33e640 1 Processor -- start 2023-10-30T12:44:44.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.336+0000 7f60ec33e640 1 -- start start 2023-10-30T12:44:44.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.340+0000 7f60ec33e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f60e4153d70 0x7f60e406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:44.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.340+0000 7f60ec33e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f60e40c8490 0x7f60e40c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:44.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.340+0000 7f60ec33e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e406afa0 con 0x7f60e406e470 2023-10-30T12:44:44.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.340+0000 7f60ec33e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60e406d780 con 0x7f60e40c8490 2023-10-30T12:44:44.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.340+0000 7f60ec33e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60e4154150 con 0x7f60e4153d70 2023-10-30T12:44:44.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.340+0000 7f60ea8db640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f60e40c8490 0x7f60e40c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:44.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.340+0000 7f60e98d9640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f60e406e470 0x7f60e40cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:46260/0 (socket says 172.21.15.12:46260) 2023-10-30T12:44:44.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.340+0000 7f60e98d9640 1 -- 172.21.15.12:0/2896482554 learned_addr learned my addr 172.21.15.12:0/2896482554 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:44:44.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.340+0000 7f60ea0da640 1 --2- 172.21.15.12:0/2896482554 >> v2:172.21.15.154:3300/0 conn(0x7f60e4153d70 0x7f60e406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:44.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.340+0000 7f60ea8db640 1 -- 172.21.15.12:0/2896482554 >> v2:172.21.15.154:3300/0 conn(0x7f60e4153d70 msgr2=0x7f60e406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:44.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.340+0000 7f60ea8db640 1 --2- 172.21.15.12:0/2896482554 >> v2:172.21.15.154:3300/0 conn(0x7f60e4153d70 0x7f60e406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:44.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.340+0000 7f60ea8db640 1 -- 172.21.15.12:0/2896482554 >> v1:172.21.15.12:6789/0 conn(0x7f60e406e470 legacy=0x7f60e40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:44.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.340+0000 7f60ea8db640 1 -- 172.21.15.12:0/2896482554 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e406e030 con 0x7f60e40c8490 2023-10-30T12:44:44.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.340+0000 7f60e90d8640 1 -- 172.21.15.12:0/2896482554 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090315817 0 0) 0x7f60e406afa0 con 0x7f60e406e470 2023-10-30T12:44:44.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.344+0000 7f60ea8db640 1 --2- 172.21.15.12:0/2896482554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e40c8490 0x7f60e40c8cc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f60e406d180 tx=0x7f60dc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94029cb69ef5139e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:44.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.344+0000 7f60e90d8640 1 -- 172.21.15.12:0/2896482554 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0a59a0 con 0x7f60e40c8490 2023-10-30T12:44:44.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.344+0000 7f60e90d8640 1 -- 172.21.15.12:0/2896482554 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc0a5b00 con 0x7f60e40c8490 2023-10-30T12:44:44.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.344+0000 7f60e90d8640 1 -- 172.21.15.12:0/2896482554 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0a5e90 con 0x7f60e40c8490 2023-10-30T12:44:44.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.344+0000 7f60ec33e640 1 -- 172.21.15.12:0/2896482554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e40c8490 msgr2=0x7f60e40c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:44.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.344+0000 7f60ec33e640 1 --2- 172.21.15.12:0/2896482554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e40c8490 0x7f60e40c8cc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f60e406d180 tx=0x7f60dc01c8e0 comp rx=0 tx=0).stop 2023-10-30T12:44:44.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.344+0000 7f60ec33e640 1 -- 172.21.15.12:0/2896482554 shutdown_connections 2023-10-30T12:44:44.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.344+0000 7f60ec33e640 1 --2- 172.21.15.12:0/2896482554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e40c8490 0x7f60e40c8cc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.344+0000 7f60ec33e640 1 --2- 172.21.15.12:0/2896482554 >> v2:172.21.15.154:3300/0 conn(0x7f60e4153d70 0x7f60e406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.344+0000 7f60ec33e640 1 -- 172.21.15.12:0/2896482554 >> 172.21.15.12:0/2896482554 conn(0x7f60e4071ea0 msgr2=0x7f60e4065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.344+0000 7f60ec33e640 1 -- 172.21.15.12:0/2896482554 shutdown_connections 2023-10-30T12:44:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.344+0000 7f60ec33e640 1 -- 172.21.15.12:0/2896482554 wait complete. 2023-10-30T12:44:44.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.344+0000 7f60ec33e640 1 Processor -- start 2023-10-30T12:44:44.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60ec33e640 1 -- start start 2023-10-30T12:44:44.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60ec33e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60e406e470 0x7f60e4144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:44.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60ec33e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e40c8490 0x7f60e40cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:44.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60ec33e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60e4153d70 0x7f60e41388e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:44.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60ec33e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e415a880 con 0x7f60e40c8490 2023-10-30T12:44:44.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60ec33e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e40cb0f0 con 0x7f60e406e470 2023-10-30T12:44:44.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60ec33e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e4144f90 con 0x7f60e4153d70 2023-10-30T12:44:44.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60e98d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e40c8490 0x7f60e40cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:44.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60e98d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e40c8490 0x7f60e40cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38806/0 (socket says 172.21.15.12:38806) 2023-10-30T12:44:44.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60e98d9640 1 -- 172.21.15.12:0/2915805149 learned_addr learned my addr 172.21.15.12:0/2915805149 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:44.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60ea0da640 1 --2- 172.21.15.12:0/2915805149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60e406e470 0x7f60e4144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:44.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60ea8db640 1 --2- 172.21.15.12:0/2915805149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60e4153d70 0x7f60e41388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:44.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60e98d9640 1 -- 172.21.15.12:0/2915805149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60e4153d70 msgr2=0x7f60e41388e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:44.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60e98d9640 1 --2- 172.21.15.12:0/2915805149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60e4153d70 0x7f60e41388e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:44.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60e98d9640 1 -- 172.21.15.12:0/2915805149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60e406e470 msgr2=0x7f60e4144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60e98d9640 1 --2- 172.21.15.12:0/2915805149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60e406e470 0x7f60e4144900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60e98d9640 1 -- 172.21.15.12:0/2915805149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60dc009a70 con 0x7f60e40c8490 2023-10-30T12:44:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60ea8db640 1 --2- 172.21.15.12:0/2915805149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60e4153d70 0x7f60e41388e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:44:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60e98d9640 1 --2- 172.21.15.12:0/2915805149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e40c8490 0x7f60e40cb580 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f60d40011f0 tx=0x7f60d400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60daffd640 1 -- 172.21.15.12:0/2915805149 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d40074e0 con 0x7f60e40c8490 2023-10-30T12:44:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60daffd640 1 -- 172.21.15.12:0/2915805149 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d4007640 con 0x7f60e40c8490 2023-10-30T12:44:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60ec33e640 1 -- 172.21.15.12:0/2915805149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e4143380 con 0x7f60e40c8490 2023-10-30T12:44:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60daffd640 1 -- 172.21.15.12:0/2915805149 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4004a10 con 0x7f60e40c8490 2023-10-30T12:44:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60ec33e640 1 -- 172.21.15.12:0/2915805149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e4142dc0 con 0x7f60e40c8490 2023-10-30T12:44:44.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.348+0000 7f60ec33e640 1 -- 172.21.15.12:0/2915805149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e40cb3b0 con 0x7f60e40c8490 2023-10-30T12:44:44.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.352+0000 7f60daffd640 1 -- 172.21.15.12:0/2915805149 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f60d4004b70 con 0x7f60e40c8490 2023-10-30T12:44:44.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.352+0000 7f60daffd640 1 --2- 172.21.15.12:0/2915805149 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f60c80664a0 0x7f60c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:44.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.352+0000 7f60daffd640 1 -- 172.21.15.12:0/2915805149 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f60d4083680 con 0x7f60e40c8490 2023-10-30T12:44:44.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.352+0000 7f60ea0da640 1 --2- 172.21.15.12:0/2915805149 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f60c80664a0 0x7f60c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:44.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.352+0000 7f60ea0da640 1 --2- 172.21.15.12:0/2915805149 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f60c80664a0 0x7f60c8068960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f60e000b930 tx=0x7f60e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:44.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.356+0000 7f60daffd640 1 -- 172.21.15.12:0/2915805149 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f60d4050890 con 0x7f60e40c8490 2023-10-30T12:44:44.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.548+0000 7f60ec33e640 1 -- 172.21.15.12:0/2915805149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60e404e170 con 0x7f60e40c8490 2023-10-30T12:44:44.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.548+0000 7f60daffd640 1 -- 172.21.15.12:0/2915805149 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5739 (secure 0 0 0) 0x7f60d4053eb0 con 0x7f60e40c8490 2023-10-30T12:44:44.552 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:44.552 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:44:42.540611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:resolve","state_seq":17,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:44.552 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 45 2023-10-30T12:44:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.548+0000 7f60ec33e640 1 -- 172.21.15.12:0/2915805149 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f60c80664a0 msgr2=0x7f60c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.548+0000 7f60ec33e640 1 --2- 172.21.15.12:0/2915805149 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f60c80664a0 0x7f60c8068960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f60e000b930 tx=0x7f60e0008040 comp rx=0 tx=0).stop 2023-10-30T12:44:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.548+0000 7f60ec33e640 1 -- 172.21.15.12:0/2915805149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e40c8490 msgr2=0x7f60e40cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.548+0000 7f60ec33e640 1 --2- 172.21.15.12:0/2915805149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e40c8490 0x7f60e40cb580 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f60d40011f0 tx=0x7f60d400c9b0 comp rx=0 tx=0).stop 2023-10-30T12:44:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.548+0000 7f60ec33e640 1 -- 172.21.15.12:0/2915805149 shutdown_connections 2023-10-30T12:44:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.548+0000 7f60ec33e640 1 --2- 172.21.15.12:0/2915805149 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f60c80664a0 0x7f60c8068960 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.548+0000 7f60ec33e640 1 --2- 172.21.15.12:0/2915805149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60e4153d70 0x7f60e41388e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.548+0000 7f60ec33e640 1 --2- 172.21.15.12:0/2915805149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e40c8490 0x7f60e40cb580 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.548+0000 7f60ec33e640 1 --2- 172.21.15.12:0/2915805149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60e406e470 0x7f60e4144900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.548+0000 7f60ec33e640 1 -- 172.21.15.12:0/2915805149 >> 172.21.15.12:0/2915805149 conn(0x7f60e4071ea0 msgr2=0x7f60e4151d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.548+0000 7f60ec33e640 1 -- 172.21.15.12:0/2915805149 shutdown_connections 2023-10-30T12:44:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:44.548+0000 7f60ec33e640 1 -- 172.21.15.12:0/2915805149 wait complete. 2023-10-30T12:44:44.572 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:44:46.574 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:46.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.724+0000 7ff1b2e46640 1 Processor -- start 2023-10-30T12:44:46.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.724+0000 7ff1b2e46640 1 -- start start 2023-10-30T12:44:46.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.728+0000 7ff1b2e46640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ac06f7a0 0x7ff1ac06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:46.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.728+0000 7ff1b2e46640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff1ac071110 0x7ff1ac158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:46.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.728+0000 7ff1b2e46640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ac066cf0 con 0x7ff1ac0c6bd0 2023-10-30T12:44:46.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.728+0000 7ff1b2e46640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ac0700c0 con 0x7ff1ac071110 2023-10-30T12:44:46.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.728+0000 7ff1b2e46640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac0c70d0 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.728+0000 7ff1b13e3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff1ac071110 0x7ff1ac158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:46.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.728+0000 7ff1b0be2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ac06f7a0 0x7ff1ac06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:46.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.728+0000 7ff1abfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff1ac0c6bd0 0x7ff1ac069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:60650/0 (socket says 172.21.15.12:60650) 2023-10-30T12:44:46.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.728+0000 7ff1abfff640 1 -- 172.21.15.12:0/3471715499 learned_addr learned my addr 172.21.15.12:0/3471715499 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:44:46.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.728+0000 7ff1ab7fe640 1 -- 172.21.15.12:0/3471715499 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4173765955 0 0) 0x7ff1ac066cf0 con 0x7ff1ac0c6bd0 2023-10-30T12:44:46.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.728+0000 7ff1b0be2640 1 -- 172.21.15.12:0/3471715499 >> v2:172.21.15.154:3300/0 conn(0x7ff1ac071110 msgr2=0x7ff1ac158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:46.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.728+0000 7ff1b0be2640 1 --2- 172.21.15.12:0/3471715499 >> v2:172.21.15.154:3300/0 conn(0x7ff1ac071110 0x7ff1ac158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:46.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.728+0000 7ff1b0be2640 1 -- 172.21.15.12:0/3471715499 >> v1:172.21.15.154:6789/0 conn(0x7ff1ac0c6bd0 legacy=0x7ff1ac069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:46.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.728+0000 7ff1b0be2640 1 -- 172.21.15.12:0/3471715499 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ac0c7390 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.732+0000 7ff1b0be2640 1 --2- 172.21.15.12:0/3471715499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ac06f7a0 0x7ff1ac06fb80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00097c0 tx=0x7ff1a001ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f9b4da21b9efa8c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:46.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.732+0000 7ff1ab7fe640 1 -- 172.21.15.12:0/3471715499 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00a8970 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.732+0000 7ff1ab7fe640 1 -- 172.21.15.12:0/3471715499 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a00a8ad0 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.732+0000 7ff1ab7fe640 1 -- 172.21.15.12:0/3471715499 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00a8e60 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.732+0000 7ff1b2e46640 1 -- 172.21.15.12:0/3471715499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ac06f7a0 msgr2=0x7ff1ac06fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:46.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.732+0000 7ff1b2e46640 1 --2- 172.21.15.12:0/3471715499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ac06f7a0 0x7ff1ac06fb80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00097c0 tx=0x7ff1a001ec70 comp rx=0 tx=0).stop 2023-10-30T12:44:46.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.732+0000 7ff1b2e46640 1 -- 172.21.15.12:0/3471715499 shutdown_connections 2023-10-30T12:44:46.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.732+0000 7ff1b2e46640 1 --2- 172.21.15.12:0/3471715499 >> v2:172.21.15.154:3300/0 conn(0x7ff1ac071110 0x7ff1ac158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:46.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.732+0000 7ff1b2e46640 1 --2- 172.21.15.12:0/3471715499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ac06f7a0 0x7ff1ac06fb80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:46.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.732+0000 7ff1b2e46640 1 -- 172.21.15.12:0/3471715499 >> 172.21.15.12:0/3471715499 conn(0x7ff1ac0c7910 msgr2=0x7ff1ac0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:46.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.732+0000 7ff1b2e46640 1 -- 172.21.15.12:0/3471715499 shutdown_connections 2023-10-30T12:44:46.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.732+0000 7ff1b2e46640 1 -- 172.21.15.12:0/3471715499 wait complete. 2023-10-30T12:44:46.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.732+0000 7ff1b2e46640 1 Processor -- start 2023-10-30T12:44:46.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b2e46640 1 -- start start 2023-10-30T12:44:46.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b2e46640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1ac06f7a0 0x7ff1ac0af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:46.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b2e46640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ac071110 0x7ff1ac1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:46.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b2e46640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac0c6bd0 0x7ff1ac159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:46.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b2e46640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac15b130 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b2e46640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac1ef740 con 0x7ff1ac0c6bd0 2023-10-30T12:44:46.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b2e46640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac1eeac0 con 0x7ff1ac071110 2023-10-30T12:44:46.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b0be2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1ac06f7a0 0x7ff1ac0af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:46.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b0be2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1ac06f7a0 0x7ff1ac0af250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38814/0 (socket says 172.21.15.12:38814) 2023-10-30T12:44:46.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b0be2640 1 -- 172.21.15.12:0/362848005 learned_addr learned my addr 172.21.15.12:0/362848005 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:46.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1abfff640 1 --2- 172.21.15.12:0/362848005 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ac071110 0x7ff1ac1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:46.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b13e3640 1 --2- 172.21.15.12:0/362848005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac0c6bd0 0x7ff1ac159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:46.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b0be2640 1 -- 172.21.15.12:0/362848005 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ac071110 msgr2=0x7ff1ac1599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:46.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b0be2640 1 --2- 172.21.15.12:0/362848005 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ac071110 0x7ff1ac1599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:46.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b0be2640 1 -- 172.21.15.12:0/362848005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac0c6bd0 msgr2=0x7ff1ac159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b0be2640 1 --2- 172.21.15.12:0/362848005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac0c6bd0 0x7ff1ac159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b0be2640 1 -- 172.21.15.12:0/362848005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a0009420 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b0be2640 1 --2- 172.21.15.12:0/362848005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1ac06f7a0 0x7ff1ac0af250 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff1a001c8a0 tx=0x7ff1a001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1a97fa640 1 -- 172.21.15.12:0/362848005 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00b9630 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1a97fa640 1 -- 172.21.15.12:0/362848005 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a00b97b0 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1a97fa640 1 -- 172.21.15.12:0/362848005 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00b9b60 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b2e46640 1 -- 172.21.15.12:0/362848005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ac1eeda0 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b2e46640 1 -- 172.21.15.12:0/362848005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ac1ef360 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.736+0000 7ff1b2e46640 1 -- 172.21.15.12:0/362848005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ac1ef880 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.740+0000 7ff1a97fa640 1 -- 172.21.15.12:0/362848005 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff1a00b7040 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.740+0000 7ff1a97fa640 1 --2- 172.21.15.12:0/362848005 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff1900664d0 0x7ff190068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:46.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.740+0000 7ff1a97fa640 1 -- 172.21.15.12:0/362848005 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7ff1a0121fe0 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.740+0000 7ff1abfff640 1 --2- 172.21.15.12:0/362848005 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff1900664d0 0x7ff190068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:46.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.740+0000 7ff1abfff640 1 --2- 172.21.15.12:0/362848005 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff1900664d0 0x7ff190068990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff198007fd0 tx=0x7ff198007d60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.744+0000 7ff1a97fa640 1 -- 172.21.15.12:0/362848005 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff1ac1ef880 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.932+0000 7ff1b2e46640 1 -- 172.21.15.12:0/362848005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1ac1ef880 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.932+0000 7ff1a97fa640 1 -- 172.21.15.12:0/362848005 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5739 (secure 0 0 0) 0x7ff1ac1ef880 con 0x7ff1ac06f7a0 2023-10-30T12:44:46.937 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:46.938 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:44:42.540611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:resolve","state_seq":17,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:46.938 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 45 2023-10-30T12:44:46.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.936+0000 7ff1b2e46640 1 -- 172.21.15.12:0/362848005 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff1900664d0 msgr2=0x7ff190068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:46.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.936+0000 7ff1b2e46640 1 --2- 172.21.15.12:0/362848005 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff1900664d0 0x7ff190068990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff198007fd0 tx=0x7ff198007d60 comp rx=0 tx=0).stop 2023-10-30T12:44:46.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.936+0000 7ff1b2e46640 1 -- 172.21.15.12:0/362848005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1ac06f7a0 msgr2=0x7ff1ac0af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:46.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.936+0000 7ff1b2e46640 1 --2- 172.21.15.12:0/362848005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1ac06f7a0 0x7ff1ac0af250 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff1a001c8a0 tx=0x7ff1a001ba10 comp rx=0 tx=0).stop 2023-10-30T12:44:46.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.936+0000 7ff1b2e46640 1 -- 172.21.15.12:0/362848005 shutdown_connections 2023-10-30T12:44:46.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.936+0000 7ff1b2e46640 1 --2- 172.21.15.12:0/362848005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac0c6bd0 0x7ff1ac159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:46.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.936+0000 7ff1b2e46640 1 --2- 172.21.15.12:0/362848005 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff1900664d0 0x7ff190068990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:46.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.936+0000 7ff1b2e46640 1 --2- 172.21.15.12:0/362848005 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ac071110 0x7ff1ac1599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:46.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.936+0000 7ff1b2e46640 1 --2- 172.21.15.12:0/362848005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1ac06f7a0 0x7ff1ac0af250 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:46.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.936+0000 7ff1b2e46640 1 -- 172.21.15.12:0/362848005 >> 172.21.15.12:0/362848005 conn(0x7ff1ac0c7910 msgr2=0x7ff1ac0c2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:46.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.936+0000 7ff1b2e46640 1 -- 172.21.15.12:0/362848005 shutdown_connections 2023-10-30T12:44:46.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:46.936+0000 7ff1b2e46640 1 -- 172.21.15.12:0/362848005 wait complete. 2023-10-30T12:44:46.958 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:44:48.961 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:49.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.112+0000 7fcdd4046640 1 Processor -- start 2023-10-30T12:44:49.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.112+0000 7fcdd4046640 1 -- start start 2023-10-30T12:44:49.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd4046640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcdcc06fd50 0x7fcdcc15ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:49.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd4046640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdcc06afa0 0x7fcdcc15d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:49.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd4046640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdcc157340 con 0x7fcdcc156f60 2023-10-30T12:44:49.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd4046640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdcc06bea0 con 0x7fcdcc06fd50 2023-10-30T12:44:49.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd4046640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0caa80 con 0x7fcdcc06afa0 2023-10-30T12:44:49.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd1de2640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fcdcc156f60 0x7fcdcc071b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:46266/0 (socket says 172.21.15.12:46266) 2023-10-30T12:44:49.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd1de2640 1 -- 172.21.15.12:0/2359461379 learned_addr learned my addr 172.21.15.12:0/2359461379 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:44:49.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd15e1640 1 --2- 172.21.15.12:0/2359461379 >> v2:172.21.15.154:3300/0 conn(0x7fcdcc06fd50 0x7fcdcc15ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:49.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd25e3640 1 --2- 172.21.15.12:0/2359461379 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdcc06afa0 0x7fcdcc15d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:49.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 -- 172.21.15.12:0/2359461379 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559141129 0 0) 0x7fcdcc157340 con 0x7fcdcc156f60 2023-10-30T12:44:49.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 -- 172.21.15.12:0/2359461379 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdb8003530 con 0x7fcdcc156f60 2023-10-30T12:44:49.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 -- 172.21.15.12:0/2359461379 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4210806274 0 0) 0x7fcdb8003530 con 0x7fcdcc156f60 2023-10-30T12:44:49.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 -- 172.21.15.12:0/2359461379 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdcc06afa0 msgr2=0x7fcdcc15d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:49.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 --2- 172.21.15.12:0/2359461379 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdcc06afa0 0x7fcdcc15d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:49.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 -- 172.21.15.12:0/2359461379 >> v2:172.21.15.154:3300/0 conn(0x7fcdcc06fd50 msgr2=0x7fcdcc15ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:49.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 --2- 172.21.15.12:0/2359461379 >> v2:172.21.15.154:3300/0 conn(0x7fcdcc06fd50 0x7fcdcc15ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:49.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 -- 172.21.15.12:0/2359461379 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc06ca30 con 0x7fcdcc156f60 2023-10-30T12:44:49.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 -- 172.21.15.12:0/2359461379 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fcdc8001680 con 0x7fcdcc156f60 2023-10-30T12:44:49.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 -- 172.21.15.12:0/2359461379 >> v1:172.21.15.12:6789/0 conn(0x7fcdcc156f60 legacy=0x7fcdcc071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:49.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 --2- 172.21.15.12:0/2359461379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdb8003980 0x7fcdb8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:49.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 --2- 172.21.15.12:0/2359461379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdb80047d0 0x7fcdb8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:49.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 --2- 172.21.15.12:0/2359461379 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdcc06afa0 0x7fcdb8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:49.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 -- 172.21.15.12:0/2359461379 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc06bea0 con 0x7fcdb80047d0 2023-10-30T12:44:49.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 -- 172.21.15.12:0/2359461379 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0caa80 con 0x7fcdb8003980 2023-10-30T12:44:49.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd0de0640 1 -- 172.21.15.12:0/2359461379 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb800b990 con 0x7fcdcc06afa0 2023-10-30T12:44:49.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd15e1640 1 --2- 172.21.15.12:0/2359461379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdb80047d0 0x7fcdb8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:49.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd1de2640 1 --2- 172.21.15.12:0/2359461379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdb8003980 0x7fcdb8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:49.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd25e3640 1 --2- 172.21.15.12:0/2359461379 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdcc06afa0 0x7fcdb8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd15e1640 1 -- 172.21.15.12:0/2359461379 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdcc06afa0 msgr2=0x7fcdb8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd15e1640 1 --2- 172.21.15.12:0/2359461379 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdcc06afa0 0x7fcdb8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd15e1640 1 -- 172.21.15.12:0/2359461379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdb8003980 msgr2=0x7fcdb8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd15e1640 1 --2- 172.21.15.12:0/2359461379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdb8003980 0x7fcdb8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.116+0000 7fcdd15e1640 1 -- 172.21.15.12:0/2359461379 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb8003530 con 0x7fcdb80047d0 2023-10-30T12:44:49.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.120+0000 7fcdd15e1640 1 --2- 172.21.15.12:0/2359461379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdb80047d0 0x7fcdb8004c50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009ae0 tx=0x7fcdbc01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665d2b93443c82bc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:49.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.120+0000 7fcdd0de0640 1 -- 172.21.15.12:0/2359461379 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0a55a0 con 0x7fcdb80047d0 2023-10-30T12:44:49.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.120+0000 7fcdd0de0640 1 -- 172.21.15.12:0/2359461379 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdbc0a5700 con 0x7fcdb80047d0 2023-10-30T12:44:49.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.120+0000 7fcdd0de0640 1 -- 172.21.15.12:0/2359461379 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0a5ab0 con 0x7fcdb80047d0 2023-10-30T12:44:49.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.120+0000 7fcdd4046640 1 -- 172.21.15.12:0/2359461379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdb80047d0 msgr2=0x7fcdb8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:49.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.120+0000 7fcdd4046640 1 --2- 172.21.15.12:0/2359461379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdb80047d0 0x7fcdb8004c50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009ae0 tx=0x7fcdbc01ca90 comp rx=0 tx=0).stop 2023-10-30T12:44:49.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.120+0000 7fcdd25e3640 1 -- 172.21.15.12:0/2359461379 reap_dead start 2023-10-30T12:44:49.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.120+0000 7fcdd4046640 1 -- 172.21.15.12:0/2359461379 shutdown_connections 2023-10-30T12:44:49.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.120+0000 7fcdd4046640 1 -- 172.21.15.12:0/2359461379 >> 172.21.15.12:0/2359461379 conn(0x7fcdcc069e90 msgr2=0x7fcdcc06a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:49.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.120+0000 7fcdd4046640 1 -- 172.21.15.12:0/2359461379 shutdown_connections 2023-10-30T12:44:49.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.120+0000 7fcdd4046640 1 -- 172.21.15.12:0/2359461379 wait complete. 2023-10-30T12:44:49.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd4046640 1 Processor -- start 2023-10-30T12:44:49.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd4046640 1 -- start start 2023-10-30T12:44:49.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd4046640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdb80047d0 0x7fcdcc0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:49.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd4046640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdcc0c6790 0x7fcdcc0c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:49.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd4046640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdcc0cc070 0x7fcdcc1590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:49.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd4046640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0cbea0 con 0x7fcdcc0cc070 2023-10-30T12:44:49.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd4046640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0c6dd0 con 0x7fcdcc0c6790 2023-10-30T12:44:49.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd4046640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0c5160 con 0x7fcdb80047d0 2023-10-30T12:44:49.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd25e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdcc0cc070 0x7fcdcc1590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:49.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd25e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdcc0cc070 0x7fcdcc1590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38832/0 (socket says 172.21.15.12:38832) 2023-10-30T12:44:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd25e3640 1 -- 172.21.15.12:0/2473799318 learned_addr learned my addr 172.21.15.12:0/2473799318 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd1de2640 1 --2- 172.21.15.12:0/2473799318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdb80047d0 0x7fcdcc0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd15e1640 1 --2- 172.21.15.12:0/2473799318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdcc0c6790 0x7fcdcc0c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd25e3640 1 -- 172.21.15.12:0/2473799318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdb80047d0 msgr2=0x7fcdcc0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd25e3640 1 --2- 172.21.15.12:0/2473799318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdb80047d0 0x7fcdcc0cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd25e3640 1 -- 172.21.15.12:0/2473799318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdcc0c6790 msgr2=0x7fcdcc0c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd25e3640 1 --2- 172.21.15.12:0/2473799318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdcc0c6790 0x7fcdcc0c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd25e3640 1 -- 172.21.15.12:0/2473799318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc0cb8e0 con 0x7fcdcc0cc070 2023-10-30T12:44:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd1de2640 1 --2- 172.21.15.12:0/2473799318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdb80047d0 0x7fcdcc0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:44:49.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd25e3640 1 --2- 172.21.15.12:0/2473799318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdcc0cc070 0x7fcdcc1590f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc06e440 tx=0x7fcdc400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:49.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdc27fc640 1 -- 172.21.15.12:0/2473799318 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc400e040 con 0x7fcdcc0cc070 2023-10-30T12:44:49.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdc27fc640 1 -- 172.21.15.12:0/2473799318 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc400f040 con 0x7fcdcc0cc070 2023-10-30T12:44:49.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdc27fc640 1 -- 172.21.15.12:0/2473799318 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4003c00 con 0x7fcdcc0cc070 2023-10-30T12:44:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd4046640 1 -- 172.21.15.12:0/2473799318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdcc0cbbc0 con 0x7fcdcc0cc070 2023-10-30T12:44:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd4046640 1 -- 172.21.15.12:0/2473799318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdcc0c4df0 con 0x7fcdcc0cc070 2023-10-30T12:44:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.124+0000 7fcdd4046640 1 -- 172.21.15.12:0/2473799318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdcc0c4970 con 0x7fcdcc0cc070 2023-10-30T12:44:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.128+0000 7fcdc27fc640 1 -- 172.21.15.12:0/2473799318 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcdc4004030 con 0x7fcdcc0cc070 2023-10-30T12:44:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.128+0000 7fcdc27fc640 1 --2- 172.21.15.12:0/2473799318 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcdb00664d0 0x7fcdb0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:49.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.128+0000 7fcdc27fc640 1 -- 172.21.15.12:0/2473799318 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fcdc4084880 con 0x7fcdcc0cc070 2023-10-30T12:44:49.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.128+0000 7fcdd1de2640 1 --2- 172.21.15.12:0/2473799318 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcdb00664d0 0x7fcdb0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.128+0000 7fcdd1de2640 1 --2- 172.21.15.12:0/2473799318 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcdb00664d0 0x7fcdb0068990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0ca2c0 tx=0x7fcdc8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.132+0000 7fcdc27fc640 1 -- 172.21.15.12:0/2473799318 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcdcc0c4970 con 0x7fcdcc0cc070 2023-10-30T12:44:49.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.324+0000 7fcdd4046640 1 -- 172.21.15.12:0/2473799318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdcc0c4970 con 0x7fcdcc0cc070 2023-10-30T12:44:49.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.324+0000 7fcdc27fc640 1 -- 172.21.15.12:0/2473799318 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5739 (secure 0 0 0) 0x7fcdcc0c4970 con 0x7fcdcc0cc070 2023-10-30T12:44:49.328 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:49.328 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:44:42.540611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:resolve","state_seq":17,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:49.329 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 45 2023-10-30T12:44:49.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.324+0000 7fcdd4046640 1 -- 172.21.15.12:0/2473799318 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcdb00664d0 msgr2=0x7fcdb0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:49.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.324+0000 7fcdd4046640 1 --2- 172.21.15.12:0/2473799318 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcdb00664d0 0x7fcdb0068990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0ca2c0 tx=0x7fcdc8009040 comp rx=0 tx=0).stop 2023-10-30T12:44:49.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.324+0000 7fcdd4046640 1 -- 172.21.15.12:0/2473799318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdcc0cc070 msgr2=0x7fcdcc1590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:49.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.324+0000 7fcdd4046640 1 --2- 172.21.15.12:0/2473799318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdcc0cc070 0x7fcdcc1590f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc06e440 tx=0x7fcdc400f580 comp rx=0 tx=0).stop 2023-10-30T12:44:49.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.324+0000 7fcdd4046640 1 -- 172.21.15.12:0/2473799318 shutdown_connections 2023-10-30T12:44:49.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.324+0000 7fcdd4046640 1 --2- 172.21.15.12:0/2473799318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdcc0cc070 0x7fcdcc1590f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:49.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.324+0000 7fcdd4046640 1 --2- 172.21.15.12:0/2473799318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdcc0c6790 0x7fcdcc0c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:49.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.324+0000 7fcdd4046640 1 --2- 172.21.15.12:0/2473799318 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcdb00664d0 0x7fcdb0068990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:49.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.324+0000 7fcdd4046640 1 --2- 172.21.15.12:0/2473799318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcdb80047d0 0x7fcdcc0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:49.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.324+0000 7fcdd4046640 1 -- 172.21.15.12:0/2473799318 >> 172.21.15.12:0/2473799318 conn(0x7fcdcc069e90 msgr2=0x7fcdcc0c2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:49.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.328+0000 7fcdd4046640 1 -- 172.21.15.12:0/2473799318 shutdown_connections 2023-10-30T12:44:49.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:49.328+0000 7fcdd4046640 1 -- 172.21.15.12:0/2473799318 wait complete. 2023-10-30T12:44:49.349 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 45, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 45, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4971": { "addr": "172.21.15.12:6837/3124135109", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 3124135109, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 3124135109, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4971, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 44 }, "gid_5000": { "addr": "172.21.15.154:6839/1635719967", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1635719967, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 1635719967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5000, "incarnation": 44, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 17 }, "gid_5124": { "addr": "172.21.15.12:6835/559649789", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 559649789, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 559649789, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5124, "incarnation": 36, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 39 } }, "last_failure": 0, "last_failure_osd_epoch": 49, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:44:42.540611+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5000, "mds_1": 5124, "mds_2": 4971 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:44:51.351 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.500+0000 7f648dc17640 1 Processor -- start 2023-10-30T12:44:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.500+0000 7f648dc17640 1 -- start start 2023-10-30T12:44:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.500+0000 7f648dc17640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f648806f7a0 0x7f648806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.500+0000 7f648dc17640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6488071110 0x7f6488158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.500+0000 7f648dc17640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6488066cf0 con 0x7f64880c6bd0 2023-10-30T12:44:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.500+0000 7f648dc17640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64880700c0 con 0x7f6488071110 2023-10-30T12:44:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.500+0000 7f648dc17640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64880c70d0 con 0x7f648806f7a0 2023-10-30T12:44:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f6486ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f64880c6bd0 0x7f6488069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48058/0 (socket says 172.21.15.12:48058) 2023-10-30T12:44:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f6486ffd640 1 -- 172.21.15.12:0/1051423048 learned_addr learned my addr 172.21.15.12:0/1051423048 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:44:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f6487fff640 1 --2- 172.21.15.12:0/1051423048 >> v2:172.21.15.154:3300/0 conn(0x7f6488071110 0x7f6488158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64877fe640 1 --2- 172.21.15.12:0/1051423048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f648806f7a0 0x7f648806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:51.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 -- 172.21.15.12:0/1051423048 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644363656 0 0) 0x7f6488066cf0 con 0x7f64880c6bd0 2023-10-30T12:44:51.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 -- 172.21.15.12:0/1051423048 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6470003530 con 0x7f64880c6bd0 2023-10-30T12:44:51.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 -- 172.21.15.12:0/1051423048 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1332910263 0 0) 0x7f6470003530 con 0x7f64880c6bd0 2023-10-30T12:44:51.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 -- 172.21.15.12:0/1051423048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f648806f7a0 msgr2=0x7f648806fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:51.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 --2- 172.21.15.12:0/1051423048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f648806f7a0 0x7f648806fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:51.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 -- 172.21.15.12:0/1051423048 >> v2:172.21.15.154:3300/0 conn(0x7f6488071110 msgr2=0x7f6488158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:51.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 --2- 172.21.15.12:0/1051423048 >> v2:172.21.15.154:3300/0 conn(0x7f6488071110 0x7f6488158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:51.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 -- 172.21.15.12:0/1051423048 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64880c7390 con 0x7f64880c6bd0 2023-10-30T12:44:51.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 -- 172.21.15.12:0/1051423048 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f647c001680 con 0x7f64880c6bd0 2023-10-30T12:44:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 -- 172.21.15.12:0/1051423048 >> v1:172.21.15.12:6789/0 conn(0x7f64880c6bd0 legacy=0x7f6488069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 --2- 172.21.15.12:0/1051423048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6470003980 0x7f6470003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 --2- 172.21.15.12:0/1051423048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f648806f7a0 0x7f64700047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 --2- 172.21.15.12:0/1051423048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6470004d10 0x7f6470005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 -- 172.21.15.12:0/1051423048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64880700c0 con 0x7f6470003980 2023-10-30T12:44:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 -- 172.21.15.12:0/1051423048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64880c70d0 con 0x7f6470004d10 2023-10-30T12:44:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64867fc640 1 -- 172.21.15.12:0/1051423048 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647000b990 con 0x7f648806f7a0 2023-10-30T12:44:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64877fe640 1 --2- 172.21.15.12:0/1051423048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6470003980 0x7f6470003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f6487fff640 1 --2- 172.21.15.12:0/1051423048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6470004d10 0x7f6470005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f6486ffd640 1 --2- 172.21.15.12:0/1051423048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f648806f7a0 0x7f64700047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64877fe640 1 -- 172.21.15.12:0/1051423048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f648806f7a0 msgr2=0x7f64700047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:51.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64877fe640 1 --2- 172.21.15.12:0/1051423048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f648806f7a0 0x7f64700047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:51.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64877fe640 1 -- 172.21.15.12:0/1051423048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6470004d10 msgr2=0x7f6470005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:51.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64877fe640 1 --2- 172.21.15.12:0/1051423048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6470004d10 0x7f6470005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:51.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f64877fe640 1 -- 172.21.15.12:0/1051423048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6470003530 con 0x7f6470003980 2023-10-30T12:44:51.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.504+0000 7f6486ffd640 1 --2- 172.21.15.12:0/1051423048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f648806f7a0 0x7f64700047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:44:51.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.508+0000 7f64877fe640 1 --2- 172.21.15.12:0/1051423048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6470003980 0x7f6470003da0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6478009ae0 tx=0x7f647801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951929ccf9ac60bd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:51.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.508+0000 7f64867fc640 1 -- 172.21.15.12:0/1051423048 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64780a5640 con 0x7f6470003980 2023-10-30T12:44:51.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.508+0000 7f64867fc640 1 -- 172.21.15.12:0/1051423048 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64780a57a0 con 0x7f6470003980 2023-10-30T12:44:51.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.508+0000 7f64867fc640 1 -- 172.21.15.12:0/1051423048 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64780a5b50 con 0x7f6470003980 2023-10-30T12:44:51.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.508+0000 7f648dc17640 1 -- 172.21.15.12:0/1051423048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6470003980 msgr2=0x7f6470003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:51.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.508+0000 7f648dc17640 1 --2- 172.21.15.12:0/1051423048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6470003980 0x7f6470003da0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6478009ae0 tx=0x7f647801ca50 comp rx=0 tx=0).stop 2023-10-30T12:44:51.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.508+0000 7f6487fff640 1 -- 172.21.15.12:0/1051423048 reap_dead start 2023-10-30T12:44:51.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.508+0000 7f648dc17640 1 -- 172.21.15.12:0/1051423048 shutdown_connections 2023-10-30T12:44:51.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.508+0000 7f648dc17640 1 -- 172.21.15.12:0/1051423048 >> 172.21.15.12:0/1051423048 conn(0x7f64880c7910 msgr2=0x7f64880c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:51.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.508+0000 7f648dc17640 1 -- 172.21.15.12:0/1051423048 shutdown_connections 2023-10-30T12:44:51.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.508+0000 7f648dc17640 1 -- 172.21.15.12:0/1051423048 wait complete. 2023-10-30T12:44:51.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.508+0000 7f648dc17640 1 Processor -- start 2023-10-30T12:44:51.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.508+0000 7f648dc17640 1 -- start start 2023-10-30T12:44:51.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f648dc17640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6470003980 0x7f64881577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:51.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f648dc17640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6488154c00 0x7f648815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:51.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f648dc17640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64880af120 0x7f64880af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:51.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f648dc17640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6488155fa0 con 0x7f64880af120 2023-10-30T12:44:51.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f648dc17640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64881564f0 con 0x7f6470003980 2023-10-30T12:44:51.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f648dc17640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6488156210 con 0x7f6488154c00 2023-10-30T12:44:51.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f6487fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64880af120 0x7f64880af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:51.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f6487fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64880af120 0x7f64880af5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32958/0 (socket says 172.21.15.12:32958) 2023-10-30T12:44:51.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f6487fff640 1 -- 172.21.15.12:0/3084029704 learned_addr learned my addr 172.21.15.12:0/3084029704 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:51.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f6486ffd640 1 --2- 172.21.15.12:0/3084029704 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6488154c00 0x7f648815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:51.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f64877fe640 1 --2- 172.21.15.12:0/3084029704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6470003980 0x7f64881577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:51.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f6487fff640 1 -- 172.21.15.12:0/3084029704 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6488154c00 msgr2=0x7f648815b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:51.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f6487fff640 1 --2- 172.21.15.12:0/3084029704 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6488154c00 0x7f648815b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:51.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f6487fff640 1 -- 172.21.15.12:0/3084029704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6470003980 msgr2=0x7f64881577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:51.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f6487fff640 1 --2- 172.21.15.12:0/3084029704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6470003980 0x7f64881577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f6487fff640 1 -- 172.21.15.12:0/3084029704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6488156cf0 con 0x7f64880af120 2023-10-30T12:44:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f6487fff640 1 --2- 172.21.15.12:0/3084029704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64880af120 0x7f64880af5a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f64881552b0 tx=0x7f648000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f646ffff640 1 -- 172.21.15.12:0/3084029704 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648000e040 con 0x7f64880af120 2023-10-30T12:44:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f648dc17640 1 -- 172.21.15.12:0/3084029704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6488156fd0 con 0x7f64880af120 2023-10-30T12:44:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f646ffff640 1 -- 172.21.15.12:0/3084029704 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648000f040 con 0x7f64880af120 2023-10-30T12:44:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f646ffff640 1 -- 172.21.15.12:0/3084029704 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480015430 con 0x7f64880af120 2023-10-30T12:44:51.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f648dc17640 1 -- 172.21.15.12:0/3084029704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64881583b0 con 0x7f64880af120 2023-10-30T12:44:51.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f648dc17640 1 -- 172.21.15.12:0/3084029704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64880c8870 con 0x7f64880af120 2023-10-30T12:44:51.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.512+0000 7f646ffff640 1 -- 172.21.15.12:0/3084029704 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6480015590 con 0x7f64880af120 2023-10-30T12:44:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.516+0000 7f646ffff640 1 --2- 172.21.15.12:0/3084029704 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f64740664a0 0x7f6474068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.516+0000 7f646ffff640 1 -- 172.21.15.12:0/3084029704 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f64800848b0 con 0x7f64880af120 2023-10-30T12:44:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.516+0000 7f64877fe640 1 --2- 172.21.15.12:0/3084029704 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f64740664a0 0x7f6474068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.516+0000 7f64877fe640 1 --2- 172.21.15.12:0/3084029704 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f64740664a0 0x7f6474068960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f64780af000 tx=0x7f64780046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.520+0000 7f646ffff640 1 -- 172.21.15.12:0/3084029704 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6480051a30 con 0x7f64880af120 2023-10-30T12:44:51.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.716+0000 7f648dc17640 1 -- 172.21.15.12:0/3084029704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64880c4590 con 0x7f64880af120 2023-10-30T12:44:51.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.716+0000 7f646ffff640 1 -- 172.21.15.12:0/3084029704 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+5741 (secure 0 0 0) 0x7f6480055050 con 0x7f64880af120 2023-10-30T12:44:51.719 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:51.719 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:44:48.592083+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:reconnect","state_seq":19,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:51.719 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 46 2023-10-30T12:44:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.716+0000 7f648dc17640 1 -- 172.21.15.12:0/3084029704 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f64740664a0 msgr2=0x7f6474068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:51.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.716+0000 7f648dc17640 1 --2- 172.21.15.12:0/3084029704 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f64740664a0 0x7f6474068960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f64780af000 tx=0x7f64780046d0 comp rx=0 tx=0).stop 2023-10-30T12:44:51.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.716+0000 7f648dc17640 1 -- 172.21.15.12:0/3084029704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64880af120 msgr2=0x7f64880af5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:51.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.716+0000 7f648dc17640 1 --2- 172.21.15.12:0/3084029704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64880af120 0x7f64880af5a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f64881552b0 tx=0x7f648000f580 comp rx=0 tx=0).stop 2023-10-30T12:44:51.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.716+0000 7f648dc17640 1 -- 172.21.15.12:0/3084029704 shutdown_connections 2023-10-30T12:44:51.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.716+0000 7f648dc17640 1 --2- 172.21.15.12:0/3084029704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64880af120 0x7f64880af5a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:51.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.716+0000 7f648dc17640 1 --2- 172.21.15.12:0/3084029704 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f64740664a0 0x7f6474068960 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:51.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.716+0000 7f648dc17640 1 --2- 172.21.15.12:0/3084029704 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6488154c00 0x7f648815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:51.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.716+0000 7f648dc17640 1 --2- 172.21.15.12:0/3084029704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6470003980 0x7f64881577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:51.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.716+0000 7f648dc17640 1 -- 172.21.15.12:0/3084029704 >> 172.21.15.12:0/3084029704 conn(0x7f64880c7910 msgr2=0x7f64880c8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:51.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.716+0000 7f648dc17640 1 -- 172.21.15.12:0/3084029704 shutdown_connections 2023-10-30T12:44:51.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:51.716+0000 7f648dc17640 1 -- 172.21.15.12:0/3084029704 wait complete. 2023-10-30T12:44:51.740 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:44:53.741 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:53.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.892+0000 7f5be7565640 1 Processor -- start 2023-10-30T12:44:53.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.892+0000 7f5be7565640 1 -- start start 2023-10-30T12:44:53.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.892+0000 7f5be7565640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5be00c6540 0x7f5be015e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.892+0000 7f5be7565640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be00c5810 con 0x7f5be0070740 2023-10-30T12:44:53.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.892+0000 7f5be7565640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be00c69b0 con 0x7f5be01580e0 2023-10-30T12:44:53.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.892+0000 7f5be7565640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be006acb0 con 0x7f5be00c6540 2023-10-30T12:44:53.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5be4b00640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5be0070740 0x7f5be006a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48068/0 (socket says 172.21.15.12:48068) 2023-10-30T12:44:53.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5be4b00640 1 -- 172.21.15.12:0/2228076423 learned_addr learned my addr 172.21.15.12:0/2228076423 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:44:53.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5be5b02640 1 --2- 172.21.15.12:0/2228076423 >> v2:172.21.15.154:3300/0 conn(0x7f5be00c6540 0x7f5be015e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5bd7fff640 1 -- 172.21.15.12:0/2228076423 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047002797 0 0) 0x7f5be00c5810 con 0x7f5be0070740 2023-10-30T12:44:53.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5bd7fff640 1 -- 172.21.15.12:0/2228076423 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bcc003530 con 0x7f5be0070740 2023-10-30T12:44:53.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5bd7fff640 1 -- 172.21.15.12:0/2228076423 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3434016349 0 0) 0x7f5bcc003530 con 0x7f5be0070740 2023-10-30T12:44:53.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5bd7fff640 1 -- 172.21.15.12:0/2228076423 >> v2:172.21.15.154:3300/0 conn(0x7f5be00c6540 msgr2=0x7f5be015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:53.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5bd7fff640 1 --2- 172.21.15.12:0/2228076423 >> v2:172.21.15.154:3300/0 conn(0x7f5be00c6540 0x7f5be015e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5bd7fff640 1 -- 172.21.15.12:0/2228076423 >> v1:172.21.15.154:6789/0 conn(0x7f5be01580e0 legacy=0x7f5be0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:53.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5bd7fff640 1 -- 172.21.15.12:0/2228076423 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be00c6bb0 con 0x7f5be0070740 2023-10-30T12:44:53.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5bd7fff640 1 -- 172.21.15.12:0/2228076423 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f5bd0001680 con 0x7f5be0070740 2023-10-30T12:44:53.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5bd7fff640 1 -- 172.21.15.12:0/2228076423 >> v1:172.21.15.12:6789/0 conn(0x7f5be0070740 legacy=0x7f5be006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:53.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5bd7fff640 1 --2- 172.21.15.12:0/2228076423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bcc003980 0x7f5bcc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5bd7fff640 1 --2- 172.21.15.12:0/2228076423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bcc0047d0 0x7f5bcc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5bd7fff640 1 --2- 172.21.15.12:0/2228076423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc005190 0x7f5bcc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5bd7fff640 1 -- 172.21.15.12:0/2228076423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc00c000 con 0x7f5bcc0047d0 2023-10-30T12:44:53.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5bd7fff640 1 -- 172.21.15.12:0/2228076423 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc00c170 con 0x7f5bcc005190 2023-10-30T12:44:53.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5bd7fff640 1 -- 172.21.15.12:0/2228076423 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc00c330 con 0x7f5bcc003980 2023-10-30T12:44:53.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5be4b00640 1 --2- 172.21.15.12:0/2228076423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bcc0047d0 0x7f5bcc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5be5301640 1 --2- 172.21.15.12:0/2228076423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bcc003980 0x7f5bcc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5be5b02640 1 --2- 172.21.15.12:0/2228076423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc005190 0x7f5bcc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5be4b00640 1 -- 172.21.15.12:0/2228076423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bcc003980 msgr2=0x7f5bcc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:53.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5be4b00640 1 --2- 172.21.15.12:0/2228076423 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bcc003980 0x7f5bcc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5be4b00640 1 -- 172.21.15.12:0/2228076423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc005190 msgr2=0x7f5bcc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:53.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5be4b00640 1 --2- 172.21.15.12:0/2228076423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc005190 0x7f5bcc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5be4b00640 1 -- 172.21.15.12:0/2228076423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc003530 con 0x7f5bcc0047d0 2023-10-30T12:44:53.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5be5b02640 1 --2- 172.21.15.12:0/2228076423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc005190 0x7f5bcc00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:44:53.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.896+0000 7f5be5b02640 1 -- 172.21.15.12:0/2228076423 reap_dead start 2023-10-30T12:44:53.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.900+0000 7f5be4b00640 1 --2- 172.21.15.12:0/2228076423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bcc0047d0 0x7f5bcc004c50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c240 tx=0x7f5bd001f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2035df3b60ce5c64 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:53.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.900+0000 7f5bd7fff640 1 -- 172.21.15.12:0/2228076423 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0005ae0 con 0x7f5bcc0047d0 2023-10-30T12:44:53.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.900+0000 7f5bd7fff640 1 -- 172.21.15.12:0/2228076423 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd00ab070 con 0x7f5bcc0047d0 2023-10-30T12:44:53.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.900+0000 7f5bd7fff640 1 -- 172.21.15.12:0/2228076423 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0007070 con 0x7f5bcc0047d0 2023-10-30T12:44:53.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.900+0000 7f5be7565640 1 -- 172.21.15.12:0/2228076423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bcc0047d0 msgr2=0x7f5bcc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:53.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.900+0000 7f5be7565640 1 --2- 172.21.15.12:0/2228076423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bcc0047d0 0x7f5bcc004c50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c240 tx=0x7f5bd001f370 comp rx=0 tx=0).stop 2023-10-30T12:44:53.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.900+0000 7f5be7565640 1 -- 172.21.15.12:0/2228076423 shutdown_connections 2023-10-30T12:44:53.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.900+0000 7f5be7565640 1 --2- 172.21.15.12:0/2228076423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bcc0047d0 0x7f5bcc004c50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.900+0000 7f5be7565640 1 -- 172.21.15.12:0/2228076423 >> 172.21.15.12:0/2228076423 conn(0x7f5be00c7910 msgr2=0x7f5be00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:53.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.900+0000 7f5be7565640 1 -- 172.21.15.12:0/2228076423 shutdown_connections 2023-10-30T12:44:53.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.900+0000 7f5be7565640 1 -- 172.21.15.12:0/2228076423 wait complete. 2023-10-30T12:44:53.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.900+0000 7f5be7565640 1 Processor -- start 2023-10-30T12:44:53.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.900+0000 7f5be7565640 1 -- start start 2023-10-30T12:44:53.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be7565640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc0047d0 0x7f5be00c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be7565640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be01f4900 0x7f5be01f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be7565640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be01f6500 0x7f5be01f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be7565640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be01600a0 con 0x7f5be01f4900 2023-10-30T12:44:53.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be7565640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be00c2730 con 0x7f5bcc0047d0 2023-10-30T12:44:53.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be7565640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be01f4470 con 0x7f5be01f6500 2023-10-30T12:44:53.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be4b00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be01f4900 0x7f5be01f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be4b00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be01f4900 0x7f5be01f4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32974/0 (socket says 172.21.15.12:32974) 2023-10-30T12:44:53.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be4b00640 1 -- 172.21.15.12:0/401224875 learned_addr learned my addr 172.21.15.12:0/401224875 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be5b02640 1 --2- 172.21.15.12:0/401224875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be01f6500 0x7f5be01f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be5301640 1 --2- 172.21.15.12:0/401224875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc0047d0 0x7f5be00c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be4b00640 1 -- 172.21.15.12:0/401224875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be01f6500 msgr2=0x7f5be01f6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be4b00640 1 --2- 172.21.15.12:0/401224875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be01f6500 0x7f5be01f6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be4b00640 1 -- 172.21.15.12:0/401224875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc0047d0 msgr2=0x7f5be00c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:53.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be4b00640 1 --2- 172.21.15.12:0/401224875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc0047d0 0x7f5be00c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be4b00640 1 -- 172.21.15.12:0/401224875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be00c5d80 con 0x7f5be01f4900 2023-10-30T12:44:53.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be4b00640 1 --2- 172.21.15.12:0/401224875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be01f4900 0x7f5be01f4d60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5bd001f1a0 tx=0x7f5bcc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:53.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5bd5ffb640 1 -- 172.21.15.12:0/401224875 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0001680 con 0x7f5be01f4900 2023-10-30T12:44:53.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5bd5ffb640 1 -- 172.21.15.12:0/401224875 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd0007070 con 0x7f5be01f4900 2023-10-30T12:44:53.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be7565640 1 -- 172.21.15.12:0/401224875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be01f52a0 con 0x7f5be01f4900 2023-10-30T12:44:53.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5bd5ffb640 1 -- 172.21.15.12:0/401224875 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0007400 con 0x7f5be01f4900 2023-10-30T12:44:53.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be7565640 1 -- 172.21.15.12:0/401224875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be0160fc0 con 0x7f5be01f4900 2023-10-30T12:44:53.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5be7565640 1 -- 172.21.15.12:0/401224875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be01f4620 con 0x7f5be01f4900 2023-10-30T12:44:53.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.904+0000 7f5bd5ffb640 1 -- 172.21.15.12:0/401224875 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5bd00b9030 con 0x7f5be01f4900 2023-10-30T12:44:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.908+0000 7f5bd5ffb640 1 --2- 172.21.15.12:0/401224875 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5bc4066520 0x7f5bc40689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.908+0000 7f5bd5ffb640 1 -- 172.21.15.12:0/401224875 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f5bd00b7070 con 0x7f5be01f4900 2023-10-30T12:44:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.908+0000 7f5be5301640 1 --2- 172.21.15.12:0/401224875 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5bc4066520 0x7f5bc40689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.908+0000 7f5be5301640 1 --2- 172.21.15.12:0/401224875 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5bc4066520 0x7f5bc40689e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc007fd0 tx=0x7f5bdc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:53.912+0000 7f5bd5ffb640 1 -- 172.21.15.12:0/401224875 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5be01f4620 con 0x7f5be01f4900 2023-10-30T12:44:54.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:54.104+0000 7f5be7565640 1 -- 172.21.15.12:0/401224875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5be01f4620 con 0x7f5be01f4900 2023-10-30T12:44:54.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:54.104+0000 7f5bd5ffb640 1 -- 172.21.15.12:0/401224875 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+5741 (secure 0 0 0) 0x7f5be01f4620 con 0x7f5be01f4900 2023-10-30T12:44:54.107 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:54.108 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:44:48.592083+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:reconnect","state_seq":19,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 46 2023-10-30T12:44:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:54.104+0000 7f5be7565640 1 -- 172.21.15.12:0/401224875 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5bc4066520 msgr2=0x7f5bc40689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:54.104+0000 7f5be7565640 1 --2- 172.21.15.12:0/401224875 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5bc4066520 0x7f5bc40689e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc007fd0 tx=0x7f5bdc009040 comp rx=0 tx=0).stop 2023-10-30T12:44:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:54.104+0000 7f5be7565640 1 -- 172.21.15.12:0/401224875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be01f4900 msgr2=0x7f5be01f4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:54.104+0000 7f5be7565640 1 --2- 172.21.15.12:0/401224875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be01f4900 0x7f5be01f4d60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5bd001f1a0 tx=0x7f5bcc005800 comp rx=0 tx=0).stop 2023-10-30T12:44:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:54.104+0000 7f5be7565640 1 -- 172.21.15.12:0/401224875 shutdown_connections 2023-10-30T12:44:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:54.104+0000 7f5be7565640 1 --2- 172.21.15.12:0/401224875 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5bc4066520 0x7f5bc40689e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:54.104+0000 7f5be7565640 1 --2- 172.21.15.12:0/401224875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5be01f6500 0x7f5be01f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:54.104+0000 7f5be7565640 1 --2- 172.21.15.12:0/401224875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be01f4900 0x7f5be01f4d60 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:54.104+0000 7f5be7565640 1 --2- 172.21.15.12:0/401224875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bcc0047d0 0x7f5be00c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:54.104+0000 7f5be7565640 1 -- 172.21.15.12:0/401224875 >> 172.21.15.12:0/401224875 conn(0x7f5be00c7910 msgr2=0x7f5be0070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:54.104+0000 7f5be7565640 1 -- 172.21.15.12:0/401224875 shutdown_connections 2023-10-30T12:44:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:54.104+0000 7f5be7565640 1 -- 172.21.15.12:0/401224875 wait complete. 2023-10-30T12:44:54.128 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:44:56.129 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:56.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.280+0000 7fba89fc5640 1 Processor -- start 2023-10-30T12:44:56.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.280+0000 7fba89fc5640 1 -- start start 2023-10-30T12:44:56.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.280+0000 7fba89fc5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba840c6bd0 0x7fba8406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:56.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.280+0000 7fba89fc5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fba84071110 0x7fba84158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:56.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.280+0000 7fba89fc5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba84066cf0 con 0x7fba8406f7a0 2023-10-30T12:44:56.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.280+0000 7fba89fc5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba840700c0 con 0x7fba84071110 2023-10-30T12:44:56.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.280+0000 7fba89fc5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba840c70d0 con 0x7fba840c6bd0 2023-10-30T12:44:56.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.280+0000 7fba83fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fba84071110 0x7fba84158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:56.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.280+0000 7fba83fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fba84071110 0x7fba84158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32976/0 (socket says 172.21.15.12:32976) 2023-10-30T12:44:56.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.280+0000 7fba83fff640 1 -- 172.21.15.12:0/2307691845 learned_addr learned my addr 172.21.15.12:0/2307691845 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:56.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.280+0000 7fba82ffd640 1 --2- 172.21.15.12:0/2307691845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba840c6bd0 0x7fba8406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:56.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.280+0000 7fba83fff640 1 -- 172.21.15.12:0/2307691845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba840c6bd0 msgr2=0x7fba8406fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:56.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.280+0000 7fba83fff640 1 --2- 172.21.15.12:0/2307691845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba840c6bd0 0x7fba8406fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.280+0000 7fba83fff640 1 -- 172.21.15.12:0/2307691845 >> v1:172.21.15.154:6789/0 conn(0x7fba8406f7a0 legacy=0x7fba84069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:56.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.280+0000 7fba83fff640 1 -- 172.21.15.12:0/2307691845 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba840c7390 con 0x7fba84071110 2023-10-30T12:44:56.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.284+0000 7fba83fff640 1 --2- 172.21.15.12:0/2307691845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba84071110 0x7fba84158110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009e50 tx=0x7fba7c01f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfaa283439c5f297 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:56.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.284+0000 7fba827fc640 1 -- 172.21.15.12:0/2307691845 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c0af400 con 0x7fba84071110 2023-10-30T12:44:56.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.284+0000 7fba827fc640 1 -- 172.21.15.12:0/2307691845 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c0ab050 con 0x7fba84071110 2023-10-30T12:44:56.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.284+0000 7fba827fc640 1 -- 172.21.15.12:0/2307691845 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c0af780 con 0x7fba84071110 2023-10-30T12:44:56.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.284+0000 7fba89fc5640 1 -- 172.21.15.12:0/2307691845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba84071110 msgr2=0x7fba84158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:56.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.284+0000 7fba89fc5640 1 --2- 172.21.15.12:0/2307691845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba84071110 0x7fba84158110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009e50 tx=0x7fba7c01f1f0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.284+0000 7fba89fc5640 1 -- 172.21.15.12:0/2307691845 shutdown_connections 2023-10-30T12:44:56.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.284+0000 7fba89fc5640 1 --2- 172.21.15.12:0/2307691845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba84071110 0x7fba84158110 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.284+0000 7fba89fc5640 1 --2- 172.21.15.12:0/2307691845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba840c6bd0 0x7fba8406fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.284+0000 7fba89fc5640 1 -- 172.21.15.12:0/2307691845 >> 172.21.15.12:0/2307691845 conn(0x7fba840c7910 msgr2=0x7fba840c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:56.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.284+0000 7fba89fc5640 1 -- 172.21.15.12:0/2307691845 shutdown_connections 2023-10-30T12:44:56.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.284+0000 7fba89fc5640 1 -- 172.21.15.12:0/2307691845 wait complete. 2023-10-30T12:44:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba89fc5640 1 Processor -- start 2023-10-30T12:44:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba89fc5640 1 -- start start 2023-10-30T12:44:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba89fc5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8406f7a0 0x7fba841588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba89fc5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba84071110 0x7fba841574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba89fc5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba840c6bd0 0x7fba840aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:56.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba89fc5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8415aaf0 con 0x7fba8406f7a0 2023-10-30T12:44:56.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba89fc5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba84156400 con 0x7fba840c6bd0 2023-10-30T12:44:56.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba89fc5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba84154a30 con 0x7fba84071110 2023-10-30T12:44:56.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba837fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8406f7a0 0x7fba841588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:56.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba837fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8406f7a0 0x7fba841588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32978/0 (socket says 172.21.15.12:32978) 2023-10-30T12:44:56.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba837fe640 1 -- 172.21.15.12:0/989861337 learned_addr learned my addr 172.21.15.12:0/989861337 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:56.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba82ffd640 1 --2- 172.21.15.12:0/989861337 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba84071110 0x7fba841574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:56.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba83fff640 1 --2- 172.21.15.12:0/989861337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba840c6bd0 0x7fba840aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:56.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba837fe640 1 -- 172.21.15.12:0/989861337 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba84071110 msgr2=0x7fba841574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:56.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba837fe640 1 --2- 172.21.15.12:0/989861337 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba84071110 0x7fba841574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba837fe640 1 -- 172.21.15.12:0/989861337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba840c6bd0 msgr2=0x7fba840aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:56.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba837fe640 1 --2- 172.21.15.12:0/989861337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba840c6bd0 0x7fba840aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba837fe640 1 -- 172.21.15.12:0/989861337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7c009b30 con 0x7fba8406f7a0 2023-10-30T12:44:56.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba837fe640 1 --2- 172.21.15.12:0/989861337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8406f7a0 0x7fba841588a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fba7400c3c0 tx=0x7fba7400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:56.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba6bfff640 1 -- 172.21.15.12:0/989861337 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba740052d0 con 0x7fba8406f7a0 2023-10-30T12:44:56.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba6bfff640 1 -- 172.21.15.12:0/989861337 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba74005430 con 0x7fba8406f7a0 2023-10-30T12:44:56.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba6bfff640 1 -- 172.21.15.12:0/989861337 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74003840 con 0x7fba8406f7a0 2023-10-30T12:44:56.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba89fc5640 1 -- 172.21.15.12:0/989861337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba84154d10 con 0x7fba8406f7a0 2023-10-30T12:44:56.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.288+0000 7fba89fc5640 1 -- 172.21.15.12:0/989861337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba84156a00 con 0x7fba8406f7a0 2023-10-30T12:44:56.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.292+0000 7fba89fc5640 1 -- 172.21.15.12:0/989861337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8404e170 con 0x7fba8406f7a0 2023-10-30T12:44:56.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.292+0000 7fba6bfff640 1 -- 172.21.15.12:0/989861337 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fba740039a0 con 0x7fba8406f7a0 2023-10-30T12:44:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.292+0000 7fba6bfff640 1 --2- 172.21.15.12:0/989861337 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba700664d0 0x7fba70068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.292+0000 7fba6bfff640 1 -- 172.21.15.12:0/989861337 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fba74083670 con 0x7fba8406f7a0 2023-10-30T12:44:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.292+0000 7fba82ffd640 1 --2- 172.21.15.12:0/989861337 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba700664d0 0x7fba70068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.292+0000 7fba82ffd640 1 --2- 172.21.15.12:0/989861337 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba700664d0 0x7fba70068990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fba840c7e70 tx=0x7fba78008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.296+0000 7fba6bfff640 1 -- 172.21.15.12:0/989861337 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fba740507f0 con 0x7fba8406f7a0 2023-10-30T12:44:56.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.484+0000 7fba89fc5640 1 -- 172.21.15.12:0/989861337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba840c7f40 con 0x7fba8406f7a0 2023-10-30T12:44:56.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.488+0000 7fba6bfff640 1 -- 172.21.15.12:0/989861337 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+5738 (secure 0 0 0) 0x7fba74053e10 con 0x7fba8406f7a0 2023-10-30T12:44:56.491 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:56.491 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:44:54.566454+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:56.492 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 47 2023-10-30T12:44:56.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.488+0000 7fba89fc5640 1 -- 172.21.15.12:0/989861337 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba700664d0 msgr2=0x7fba70068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:56.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.488+0000 7fba89fc5640 1 --2- 172.21.15.12:0/989861337 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba700664d0 0x7fba70068990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fba840c7e70 tx=0x7fba78008040 comp rx=0 tx=0).stop 2023-10-30T12:44:56.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.488+0000 7fba89fc5640 1 -- 172.21.15.12:0/989861337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8406f7a0 msgr2=0x7fba841588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:56.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.488+0000 7fba89fc5640 1 --2- 172.21.15.12:0/989861337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8406f7a0 0x7fba841588a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fba7400c3c0 tx=0x7fba7400c880 comp rx=0 tx=0).stop 2023-10-30T12:44:56.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.488+0000 7fba89fc5640 1 -- 172.21.15.12:0/989861337 shutdown_connections 2023-10-30T12:44:56.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.488+0000 7fba89fc5640 1 --2- 172.21.15.12:0/989861337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba840c6bd0 0x7fba840aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.488+0000 7fba89fc5640 1 --2- 172.21.15.12:0/989861337 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fba700664d0 0x7fba70068990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.488+0000 7fba89fc5640 1 --2- 172.21.15.12:0/989861337 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba84071110 0x7fba841574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.488+0000 7fba89fc5640 1 --2- 172.21.15.12:0/989861337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8406f7a0 0x7fba841588a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.488+0000 7fba89fc5640 1 -- 172.21.15.12:0/989861337 >> 172.21.15.12:0/989861337 conn(0x7fba840c7910 msgr2=0x7fba840c8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:56.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.488+0000 7fba89fc5640 1 -- 172.21.15.12:0/989861337 shutdown_connections 2023-10-30T12:44:56.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:56.488+0000 7fba89fc5640 1 -- 172.21.15.12:0/989861337 wait complete. 2023-10-30T12:44:56.512 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:44:58.513 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:44:58.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.664+0000 7fa4f2468640 1 Processor -- start 2023-10-30T12:44:58.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.664+0000 7fa4f2468640 1 -- start start 2023-10-30T12:44:58.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.664+0000 7fa4f2468640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4ec0c05d0 0x7fa4ec13ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:58.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.664+0000 7fa4f2468640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa4ec0c6f60 0x7fa4ec13f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:58.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.664+0000 7fa4f2468640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4ec0c0a50 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.664+0000 7fa4f2468640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4ec06f6e0 con 0x7fa4ec0c6f60 2023-10-30T12:44:58.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.664+0000 7fa4f2468640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec06f9c0 con 0x7fa4ec0c05d0 2023-10-30T12:44:58.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.664+0000 7fa4ebfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa4ec0c6b80 0x7fa4ec0bfec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48076/0 (socket says 172.21.15.12:48076) 2023-10-30T12:44:58.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.664+0000 7fa4ebfff640 1 -- 172.21.15.12:0/2737821776 learned_addr learned my addr 172.21.15.12:0/2737821776 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:44:58.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.668+0000 7fa4eb7fe640 1 --2- 172.21.15.12:0/2737821776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4ec0c05d0 0x7fa4ec13ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:58.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.668+0000 7fa4f0a05640 1 --2- 172.21.15.12:0/2737821776 >> v2:172.21.15.154:3300/0 conn(0x7fa4ec0c6f60 0x7fa4ec13f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:58.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.668+0000 7fa4eaffd640 1 -- 172.21.15.12:0/2737821776 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24033882 0 0) 0x7fa4ec0c0a50 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.668+0000 7fa4eaffd640 1 -- 172.21.15.12:0/2737821776 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4d4003530 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.668+0000 7fa4eb7fe640 1 -- 172.21.15.12:0/2737821776 >> v2:172.21.15.154:3300/0 conn(0x7fa4ec0c6f60 msgr2=0x7fa4ec13f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:58.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.668+0000 7fa4eb7fe640 1 --2- 172.21.15.12:0/2737821776 >> v2:172.21.15.154:3300/0 conn(0x7fa4ec0c6f60 0x7fa4ec13f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:58.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.668+0000 7fa4eb7fe640 1 -- 172.21.15.12:0/2737821776 >> v1:172.21.15.12:6789/0 conn(0x7fa4ec0c6b80 legacy=0x7fa4ec0bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:58.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.668+0000 7fa4eaffd640 1 -- 172.21.15.12:0/2737821776 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2472884555 0 0) 0x7fa4d4003530 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.668+0000 7fa4eb7fe640 1 -- 172.21.15.12:0/2737821776 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ec06fca0 con 0x7fa4ec0c05d0 2023-10-30T12:44:58.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.668+0000 7fa4f0a05640 1 --2- 172.21.15.12:0/2737821776 >> v2:172.21.15.154:3300/0 conn(0x7fa4ec0c6f60 0x7fa4ec13f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:44:58.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.668+0000 7fa4eb7fe640 1 --2- 172.21.15.12:0/2737821776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4ec0c05d0 0x7fa4ec13ed90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00097c0 tx=0x7fa4e001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb33b732f3b5d123 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:58.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4eaffd640 1 -- 172.21.15.12:0/2737821776 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00ad640 con 0x7fa4ec0c05d0 2023-10-30T12:44:58.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4eaffd640 1 -- 172.21.15.12:0/2737821776 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e00ad7a0 con 0x7fa4ec0c05d0 2023-10-30T12:44:58.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4eaffd640 1 -- 172.21.15.12:0/2737821776 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00adb10 con 0x7fa4ec0c05d0 2023-10-30T12:44:58.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 -- 172.21.15.12:0/2737821776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4ec0c05d0 msgr2=0x7fa4ec13ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:58.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 --2- 172.21.15.12:0/2737821776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4ec0c05d0 0x7fa4ec13ed90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00097c0 tx=0x7fa4e001c480 comp rx=0 tx=0).stop 2023-10-30T12:44:58.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 -- 172.21.15.12:0/2737821776 shutdown_connections 2023-10-30T12:44:58.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 --2- 172.21.15.12:0/2737821776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4ec0c05d0 0x7fa4ec13ed90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:58.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 --2- 172.21.15.12:0/2737821776 >> v2:172.21.15.154:3300/0 conn(0x7fa4ec0c6f60 0x7fa4ec13f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:58.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 -- 172.21.15.12:0/2737821776 >> 172.21.15.12:0/2737821776 conn(0x7fa4ec0c7910 msgr2=0x7fa4ec0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 -- 172.21.15.12:0/2737821776 shutdown_connections 2023-10-30T12:44:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 -- 172.21.15.12:0/2737821776 wait complete. 2023-10-30T12:44:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 Processor -- start 2023-10-30T12:44:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 -- start start 2023-10-30T12:44:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4ec0c05d0 0x7fa4ec1f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4ec0c6b80 0x7fa4ec1f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4ec0c6f60 0x7fa4ec13a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec1f5e80 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec154890 con 0x7fa4ec0c05d0 2023-10-30T12:44:58.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.672+0000 7fa4f2468640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec1f50c0 con 0x7fa4ec0c6f60 2023-10-30T12:44:58.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4eb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4ec0c6b80 0x7fa4ec1f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:58.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4eb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4ec0c6b80 0x7fa4ec1f1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32992/0 (socket says 172.21.15.12:32992) 2023-10-30T12:44:58.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4eb7fe640 1 -- 172.21.15.12:0/4032589535 learned_addr learned my addr 172.21.15.12:0/4032589535 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:44:58.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4f0a05640 1 --2- 172.21.15.12:0/4032589535 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4ec0c6f60 0x7fa4ec13a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:58.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4ebfff640 1 --2- 172.21.15.12:0/4032589535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4ec0c05d0 0x7fa4ec1f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:58.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4eb7fe640 1 -- 172.21.15.12:0/4032589535 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4ec0c6f60 msgr2=0x7fa4ec13a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:58.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4eb7fe640 1 --2- 172.21.15.12:0/4032589535 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4ec0c6f60 0x7fa4ec13a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:58.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4eb7fe640 1 -- 172.21.15.12:0/4032589535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4ec0c05d0 msgr2=0x7fa4ec1f19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:58.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4eb7fe640 1 --2- 172.21.15.12:0/4032589535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4ec0c05d0 0x7fa4ec1f19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:58.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4eb7fe640 1 -- 172.21.15.12:0/4032589535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e4009a70 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4f0a05640 1 --2- 172.21.15.12:0/4032589535 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4ec0c6f60 0x7fa4ec13a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:44:58.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4eb7fe640 1 --2- 172.21.15.12:0/4032589535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4ec0c6b80 0x7fa4ec1f1f10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0000c10 tx=0x7fa4e0019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:58.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4e8ff9640 1 -- 172.21.15.12:0/4032589535 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00add20 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4e8ff9640 1 -- 172.21.15.12:0/4032589535 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e00a6070 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4f2468640 1 -- 172.21.15.12:0/4032589535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e0009420 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4e8ff9640 1 -- 172.21.15.12:0/4032589535 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0003820 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4f2468640 1 -- 172.21.15.12:0/4032589535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ec1f5680 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4f2468640 1 -- 172.21.15.12:0/4032589535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4ec04e170 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4e8ff9640 1 -- 172.21.15.12:0/4032589535 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa4e00b4040 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.676+0000 7fa4e8ff9640 1 --2- 172.21.15.12:0/4032589535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4d80664a0 0x7fa4d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:58.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.680+0000 7fa4e8ff9640 1 -- 172.21.15.12:0/4032589535 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fa4e011e430 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.680+0000 7fa4ebfff640 1 --2- 172.21.15.12:0/4032589535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4d80664a0 0x7fa4d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:58.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.680+0000 7fa4ebfff640 1 --2- 172.21.15.12:0/4032589535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4d80664a0 0x7fa4d8068960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc001710 tx=0x7fa4dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:58.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.680+0000 7fa4e8ff9640 1 -- 172.21.15.12:0/4032589535 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa4e00eb5b0 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.872+0000 7fa4f2468640 1 -- 172.21.15.12:0/4032589535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4ec137100 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.872+0000 7fa4e8ff9640 1 -- 172.21.15.12:0/4032589535 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+5738 (secure 0 0 0) 0x7fa4e00eebd0 con 0x7fa4ec0c6b80 2023-10-30T12:44:58.876 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:44:58.877 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:44:54.566454+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:44:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 47 2023-10-30T12:44:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.872+0000 7fa4f2468640 1 -- 172.21.15.12:0/4032589535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4d80664a0 msgr2=0x7fa4d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.872+0000 7fa4f2468640 1 --2- 172.21.15.12:0/4032589535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4d80664a0 0x7fa4d8068960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc001710 tx=0x7fa4dc009040 comp rx=0 tx=0).stop 2023-10-30T12:44:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.872+0000 7fa4f2468640 1 -- 172.21.15.12:0/4032589535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4ec0c6b80 msgr2=0x7fa4ec1f1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.872+0000 7fa4f2468640 1 --2- 172.21.15.12:0/4032589535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4ec0c6b80 0x7fa4ec1f1f10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0000c10 tx=0x7fa4e0019220 comp rx=0 tx=0).stop 2023-10-30T12:44:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.872+0000 7fa4f2468640 1 -- 172.21.15.12:0/4032589535 shutdown_connections 2023-10-30T12:44:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.872+0000 7fa4f2468640 1 --2- 172.21.15.12:0/4032589535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4d80664a0 0x7fa4d8068960 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.872+0000 7fa4f2468640 1 --2- 172.21.15.12:0/4032589535 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4ec0c6f60 0x7fa4ec13a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.872+0000 7fa4f2468640 1 --2- 172.21.15.12:0/4032589535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4ec0c6b80 0x7fa4ec1f1f10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.872+0000 7fa4f2468640 1 --2- 172.21.15.12:0/4032589535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4ec0c05d0 0x7fa4ec1f19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.872+0000 7fa4f2468640 1 -- 172.21.15.12:0/4032589535 >> 172.21.15.12:0/4032589535 conn(0x7fa4ec0c7910 msgr2=0x7fa4ec0c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:58.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.876+0000 7fa4f2468640 1 -- 172.21.15.12:0/4032589535 shutdown_connections 2023-10-30T12:44:58.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:44:58.876+0000 7fa4f2468640 1 -- 172.21.15.12:0/4032589535 wait complete. 2023-10-30T12:44:58.897 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:45:00.898 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:45:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.048+0000 7fe282d65640 1 Processor -- start 2023-10-30T12:45:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.048+0000 7fe282d65640 1 -- start start 2023-10-30T12:45:01.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.052+0000 7fe282d65640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe27c156f80 0x7fe27c159880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:01.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.052+0000 7fe282d65640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe27c070c90 0x7fe27c164120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:01.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.052+0000 7fe282d65640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27c0c2f20 con 0x7fe27c070650 2023-10-30T12:45:01.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.052+0000 7fe282d65640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27c157360 con 0x7fe27c156f80 2023-10-30T12:45:01.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.052+0000 7fe282d65640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27c0c85c0 con 0x7fe27c070c90 2023-10-30T12:45:01.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.052+0000 7fe280b01640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe27c156f80 0x7fe27c159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:01.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.052+0000 7fe280b01640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe27c156f80 0x7fe27c159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36740/0 (socket says 172.21.15.12:36740) 2023-10-30T12:45:01.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.052+0000 7fe280b01640 1 -- 172.21.15.12:0/3994535101 learned_addr learned my addr 172.21.15.12:0/3994535101 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:45:01.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.052+0000 7fe281302640 1 --2- 172.21.15.12:0/3994535101 >> v2:172.21.15.154:3300/0 conn(0x7fe27c070c90 0x7fe27c164120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:01.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.052+0000 7fe280b01640 1 -- 172.21.15.12:0/3994535101 >> v2:172.21.15.154:3300/0 conn(0x7fe27c070c90 msgr2=0x7fe27c164120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:01.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.052+0000 7fe280b01640 1 --2- 172.21.15.12:0/3994535101 >> v2:172.21.15.154:3300/0 conn(0x7fe27c070c90 0x7fe27c164120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:01.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.052+0000 7fe280b01640 1 -- 172.21.15.12:0/3994535101 >> v1:172.21.15.154:6789/0 conn(0x7fe27c070650 legacy=0x7fe27c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:01.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.052+0000 7fe280b01640 1 -- 172.21.15.12:0/3994535101 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c071230 con 0x7fe27c156f80 2023-10-30T12:45:01.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.056+0000 7fe280b01640 1 --2- 172.21.15.12:0/3994535101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe27c156f80 0x7fe27c159880 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe27c15de90 tx=0x7fe27001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b5b15a60480296 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:01.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.056+0000 7fe27b7fe640 1 -- 172.21.15.12:0/3994535101 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2700aa620 con 0x7fe27c156f80 2023-10-30T12:45:01.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.056+0000 7fe27b7fe640 1 -- 172.21.15.12:0/3994535101 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2700aa780 con 0x7fe27c156f80 2023-10-30T12:45:01.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.056+0000 7fe27b7fe640 1 -- 172.21.15.12:0/3994535101 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2700aab10 con 0x7fe27c156f80 2023-10-30T12:45:01.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.056+0000 7fe282d65640 1 -- 172.21.15.12:0/3994535101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe27c156f80 msgr2=0x7fe27c159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:01.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.056+0000 7fe282d65640 1 --2- 172.21.15.12:0/3994535101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe27c156f80 0x7fe27c159880 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe27c15de90 tx=0x7fe27001c500 comp rx=0 tx=0).stop 2023-10-30T12:45:01.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.056+0000 7fe282d65640 1 -- 172.21.15.12:0/3994535101 shutdown_connections 2023-10-30T12:45:01.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.056+0000 7fe282d65640 1 --2- 172.21.15.12:0/3994535101 >> v2:172.21.15.154:3300/0 conn(0x7fe27c070c90 0x7fe27c164120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:01.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.056+0000 7fe282d65640 1 --2- 172.21.15.12:0/3994535101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe27c156f80 0x7fe27c159880 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:01.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.056+0000 7fe282d65640 1 -- 172.21.15.12:0/3994535101 >> 172.21.15.12:0/3994535101 conn(0x7fe27c0cbeb0 msgr2=0x7fe27c0cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:01.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.056+0000 7fe282d65640 1 -- 172.21.15.12:0/3994535101 shutdown_connections 2023-10-30T12:45:01.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.056+0000 7fe282d65640 1 -- 172.21.15.12:0/3994535101 wait complete. 2023-10-30T12:45:01.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.056+0000 7fe282d65640 1 Processor -- start 2023-10-30T12:45:01.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.056+0000 7fe282d65640 1 -- start start 2023-10-30T12:45:01.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe282d65640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe27c070650 0x7fe27c140c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:01.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe282d65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe27c070c90 0x7fe27c0c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:01.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe282d65640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe27c156f80 0x7fe27c0c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:01.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe282d65640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c141150 con 0x7fe27c070c90 2023-10-30T12:45:01.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe282d65640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c06a610 con 0x7fe27c070650 2023-10-30T12:45:01.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe282d65640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c14e210 con 0x7fe27c156f80 2023-10-30T12:45:01.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe27bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe27c070c90 0x7fe27c0c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:01.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe27bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe27c070c90 0x7fe27c0c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36746/0 (socket says 172.21.15.12:36746) 2023-10-30T12:45:01.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe27bfff640 1 -- 172.21.15.12:0/4173013263 learned_addr learned my addr 172.21.15.12:0/4173013263 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:45:01.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe281302640 1 --2- 172.21.15.12:0/4173013263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe27c156f80 0x7fe27c0c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:01.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe280b01640 1 --2- 172.21.15.12:0/4173013263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe27c070650 0x7fe27c140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:01.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe27bfff640 1 -- 172.21.15.12:0/4173013263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe27c156f80 msgr2=0x7fe27c0c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:01.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe27bfff640 1 --2- 172.21.15.12:0/4173013263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe27c156f80 0x7fe27c0c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:01.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe27bfff640 1 -- 172.21.15.12:0/4173013263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe27c070650 msgr2=0x7fe27c140c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:01.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe27bfff640 1 --2- 172.21.15.12:0/4173013263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe27c070650 0x7fe27c140c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:01.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe27bfff640 1 -- 172.21.15.12:0/4173013263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2700094d0 con 0x7fe27c070c90 2023-10-30T12:45:01.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe27bfff640 1 --2- 172.21.15.12:0/4173013263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe27c070c90 0x7fe27c0c7040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe26800c3c0 tx=0x7fe26800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:01.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe2797fa640 1 -- 172.21.15.12:0/4173013263 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2680052d0 con 0x7fe27c070c90 2023-10-30T12:45:01.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe2797fa640 1 -- 172.21.15.12:0/4173013263 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe268005430 con 0x7fe27c070c90 2023-10-30T12:45:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe282d65640 1 -- 172.21.15.12:0/4173013263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27c06a860 con 0x7fe27c070c90 2023-10-30T12:45:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe2797fa640 1 -- 172.21.15.12:0/4173013263 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe268003820 con 0x7fe27c070c90 2023-10-30T12:45:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe282d65640 1 -- 172.21.15.12:0/4173013263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27c0cb810 con 0x7fe27c070c90 2023-10-30T12:45:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe282d65640 1 -- 172.21.15.12:0/4173013263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27c04e170 con 0x7fe27c070c90 2023-10-30T12:45:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.060+0000 7fe2797fa640 1 -- 172.21.15.12:0/4173013263 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe268003980 con 0x7fe27c070c90 2023-10-30T12:45:01.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.064+0000 7fe2797fa640 1 --2- 172.21.15.12:0/4173013263 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe2600664a0 0x7fe260068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:01.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.064+0000 7fe2797fa640 1 -- 172.21.15.12:0/4173013263 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fe2680835e0 con 0x7fe27c070c90 2023-10-30T12:45:01.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.064+0000 7fe280b01640 1 --2- 172.21.15.12:0/4173013263 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe2600664a0 0x7fe260068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:01.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.064+0000 7fe280b01640 1 --2- 172.21.15.12:0/4173013263 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe2600664a0 0x7fe260068960 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe2700ae000 tx=0x7fe2700b5040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:01.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.068+0000 7fe2797fa640 1 -- 172.21.15.12:0/4173013263 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe268050810 con 0x7fe27c070c90 2023-10-30T12:45:01.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.256+0000 7fe282d65640 1 -- 172.21.15.12:0/4173013263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe27c142850 con 0x7fe27c070c90 2023-10-30T12:45:01.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.256+0000 7fe2797fa640 1 -- 172.21.15.12:0/4173013263 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+5738 (secure 0 0 0) 0x7fe268053e30 con 0x7fe27c070c90 2023-10-30T12:45:01.260 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:45:01.261 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:44:54.566454+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:45:01.261 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 47 2023-10-30T12:45:01.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.260+0000 7fe282d65640 1 -- 172.21.15.12:0/4173013263 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe2600664a0 msgr2=0x7fe260068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:01.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.260+0000 7fe282d65640 1 --2- 172.21.15.12:0/4173013263 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe2600664a0 0x7fe260068960 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe2700ae000 tx=0x7fe2700b5040 comp rx=0 tx=0).stop 2023-10-30T12:45:01.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.260+0000 7fe282d65640 1 -- 172.21.15.12:0/4173013263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe27c070c90 msgr2=0x7fe27c0c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:01.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.260+0000 7fe282d65640 1 --2- 172.21.15.12:0/4173013263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe27c070c90 0x7fe27c0c7040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe26800c3c0 tx=0x7fe26800c880 comp rx=0 tx=0).stop 2023-10-30T12:45:01.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.260+0000 7fe282d65640 1 -- 172.21.15.12:0/4173013263 shutdown_connections 2023-10-30T12:45:01.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.260+0000 7fe282d65640 1 --2- 172.21.15.12:0/4173013263 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe2600664a0 0x7fe260068960 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:01.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.260+0000 7fe282d65640 1 --2- 172.21.15.12:0/4173013263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe27c156f80 0x7fe27c0c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:01.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.260+0000 7fe282d65640 1 --2- 172.21.15.12:0/4173013263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe27c070c90 0x7fe27c0c7040 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:01.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.260+0000 7fe282d65640 1 --2- 172.21.15.12:0/4173013263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe27c070650 0x7fe27c140c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:01.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.260+0000 7fe282d65640 1 -- 172.21.15.12:0/4173013263 >> 172.21.15.12:0/4173013263 conn(0x7fe27c0cbeb0 msgr2=0x7fe27c0cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:01.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.260+0000 7fe282d65640 1 -- 172.21.15.12:0/4173013263 shutdown_connections 2023-10-30T12:45:01.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:01.260+0000 7fe282d65640 1 -- 172.21.15.12:0/4173013263 wait complete. 2023-10-30T12:45:01.282 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 47, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 47, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4971": { "addr": "172.21.15.12:6837/3124135109", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 3124135109, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 3124135109, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4971, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 44 }, "gid_5000": { "addr": "172.21.15.154:6839/1635719967", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1635719967, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 1635719967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5000, "incarnation": 44, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:rejoin", "state_seq": 21 }, "gid_5124": { "addr": "172.21.15.12:6835/559649789", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 559649789, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 559649789, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5124, "incarnation": 36, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 39 } }, "last_failure": 0, "last_failure_osd_epoch": 49, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:44:54.566454+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5000, "mds_1": 5124, "mds_2": 4971 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:45:03.283 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:45:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.432+0000 7fec82f65640 1 Processor -- start 2023-10-30T12:45:03.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.432+0000 7fec82f65640 1 -- start start 2023-10-30T12:45:03.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.436+0000 7fec82f65640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec7c0c6fe0 0x7fec7c071260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:03.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.436+0000 7fec82f65640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fec7c0c5f60 0x7fec7c0cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:03.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.436+0000 7fec82f65640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec7c0700b0 con 0x7fec7c153bf0 2023-10-30T12:45:03.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.436+0000 7fec82f65640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7c0c6810 con 0x7fec7c0c5f60 2023-10-30T12:45:03.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.436+0000 7fec82f65640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c159900 con 0x7fec7c0c6fe0 2023-10-30T12:45:03.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.436+0000 7fec81502640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fec7c0c5f60 0x7fec7c0cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:03.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.436+0000 7fec81502640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fec7c0c5f60 0x7fec7c0cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36758/0 (socket says 172.21.15.12:36758) 2023-10-30T12:45:03.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.436+0000 7fec81502640 1 -- 172.21.15.12:0/2801312179 learned_addr learned my addr 172.21.15.12:0/2801312179 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:45:03.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.436+0000 7fec7bfff640 1 --2- 172.21.15.12:0/2801312179 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec7c0c6fe0 0x7fec7c071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:03.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.436+0000 7fec81502640 1 -- 172.21.15.12:0/2801312179 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec7c0c6fe0 msgr2=0x7fec7c071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:03.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.436+0000 7fec81502640 1 --2- 172.21.15.12:0/2801312179 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec7c0c6fe0 0x7fec7c071260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.436+0000 7fec81502640 1 -- 172.21.15.12:0/2801312179 >> v1:172.21.15.154:6789/0 conn(0x7fec7c153bf0 legacy=0x7fec7c0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:03.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.436+0000 7fec81502640 1 -- 172.21.15.12:0/2801312179 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c0c6530 con 0x7fec7c0c5f60 2023-10-30T12:45:03.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.440+0000 7fec81502640 1 --2- 172.21.15.12:0/2801312179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec7c0c5f60 0x7fec7c0cb570 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fec74009e50 tx=0x7fec7401f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f0a3e3c36d7894 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:03.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.440+0000 7fec7b7fe640 1 -- 172.21.15.12:0/2801312179 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec740af400 con 0x7fec7c0c5f60 2023-10-30T12:45:03.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.440+0000 7fec7b7fe640 1 -- 172.21.15.12:0/2801312179 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec740ab050 con 0x7fec7c0c5f60 2023-10-30T12:45:03.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.440+0000 7fec7b7fe640 1 -- 172.21.15.12:0/2801312179 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec740af780 con 0x7fec7c0c5f60 2023-10-30T12:45:03.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.440+0000 7fec82f65640 1 -- 172.21.15.12:0/2801312179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec7c0c5f60 msgr2=0x7fec7c0cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:03.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.440+0000 7fec82f65640 1 --2- 172.21.15.12:0/2801312179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec7c0c5f60 0x7fec7c0cb570 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fec74009e50 tx=0x7fec7401f1f0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.440+0000 7fec82f65640 1 -- 172.21.15.12:0/2801312179 shutdown_connections 2023-10-30T12:45:03.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.440+0000 7fec82f65640 1 --2- 172.21.15.12:0/2801312179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec7c0c5f60 0x7fec7c0cb570 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.440+0000 7fec82f65640 1 --2- 172.21.15.12:0/2801312179 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec7c0c6fe0 0x7fec7c071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.440+0000 7fec82f65640 1 -- 172.21.15.12:0/2801312179 >> 172.21.15.12:0/2801312179 conn(0x7fec7c06c6e0 msgr2=0x7fec7c069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:03.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.440+0000 7fec82f65640 1 -- 172.21.15.12:0/2801312179 shutdown_connections 2023-10-30T12:45:03.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.440+0000 7fec82f65640 1 -- 172.21.15.12:0/2801312179 wait complete. 2023-10-30T12:45:03.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.440+0000 7fec82f65640 1 Processor -- start 2023-10-30T12:45:03.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.440+0000 7fec82f65640 1 -- start start 2023-10-30T12:45:03.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec82f65640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec7c0c5f60 0x7fec7c152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:03.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec82f65640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec7c0c6fe0 0x7fec7c143500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:03.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec82f65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec7c153bf0 0x7fec7c143a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:03.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec82f65640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c143f80 con 0x7fec7c153bf0 2023-10-30T12:45:03.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec82f65640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c1485a0 con 0x7fec7c0c5f60 2023-10-30T12:45:03.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec82f65640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c14dea0 con 0x7fec7c0c6fe0 2023-10-30T12:45:03.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec81502640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec7c153bf0 0x7fec7c143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:03.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec7bfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec7c0c6fe0 0x7fec7c143500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:03.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec80d01640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec7c0c5f60 0x7fec7c152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:03.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec81502640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec7c153bf0 0x7fec7c143a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36766/0 (socket says 172.21.15.12:36766) 2023-10-30T12:45:03.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec7bfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec7c0c6fe0 0x7fec7c143500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:57572/0 (socket says 172.21.15.12:57572) 2023-10-30T12:45:03.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec81502640 1 -- 172.21.15.12:0/1057593687 learned_addr learned my addr 172.21.15.12:0/1057593687 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:45:03.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec81502640 1 -- 172.21.15.12:0/1057593687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec7c0c6fe0 msgr2=0x7fec7c143500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:03.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec81502640 1 --2- 172.21.15.12:0/1057593687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec7c0c6fe0 0x7fec7c143500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec81502640 1 -- 172.21.15.12:0/1057593687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec7c0c5f60 msgr2=0x7fec7c152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:03.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec81502640 1 --2- 172.21.15.12:0/1057593687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec7c0c5f60 0x7fec7c152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec81502640 1 -- 172.21.15.12:0/1057593687 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec74009b30 con 0x7fec7c153bf0 2023-10-30T12:45:03.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec81502640 1 --2- 172.21.15.12:0/1057593687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec7c153bf0 0x7fec7c143a40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fec7401ce20 tx=0x7fec7401bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:03.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec797fa640 1 -- 172.21.15.12:0/1057593687 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec740afde0 con 0x7fec7c153bf0 2023-10-30T12:45:03.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec797fa640 1 -- 172.21.15.12:0/1057593687 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec74009510 con 0x7fec7c153bf0 2023-10-30T12:45:03.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec797fa640 1 -- 172.21.15.12:0/1057593687 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec740ba660 con 0x7fec7c153bf0 2023-10-30T12:45:03.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec82f65640 1 -- 172.21.15.12:0/1057593687 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7c14f990 con 0x7fec7c153bf0 2023-10-30T12:45:03.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec82f65640 1 -- 172.21.15.12:0/1057593687 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7c14e0f0 con 0x7fec7c153bf0 2023-10-30T12:45:03.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec82f65640 1 -- 172.21.15.12:0/1057593687 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7c04e170 con 0x7fec7c153bf0 2023-10-30T12:45:03.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.444+0000 7fec797fa640 1 -- 172.21.15.12:0/1057593687 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fec740b6040 con 0x7fec7c153bf0 2023-10-30T12:45:03.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.448+0000 7fec797fa640 1 --2- 172.21.15.12:0/1057593687 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fec600664a0 0x7fec60068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:03.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.448+0000 7fec797fa640 1 -- 172.21.15.12:0/1057593687 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fec74122140 con 0x7fec7c153bf0 2023-10-30T12:45:03.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.448+0000 7fec80d01640 1 --2- 172.21.15.12:0/1057593687 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fec600664a0 0x7fec60068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:03.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.448+0000 7fec80d01640 1 --2- 172.21.15.12:0/1057593687 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fec600664a0 0x7fec60068960 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec70007fd0 tx=0x7fec70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:03.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.452+0000 7fec797fa640 1 -- 172.21.15.12:0/1057593687 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fec740ef540 con 0x7fec7c153bf0 2023-10-30T12:45:03.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.640+0000 7fec82f65640 1 -- 172.21.15.12:0/1057593687 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec7c06fe90 con 0x7fec7c153bf0 2023-10-30T12:45:03.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.640+0000 7fec797fa640 1 -- 172.21.15.12:0/1057593687 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+5744 (secure 0 0 0) 0x7fec740f2b60 con 0x7fec7c153bf0 2023-10-30T12:45:03.644 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:45:03.644 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:45:00.691908+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:clientreplay","state_seq":23,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:45:03.645 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 48 2023-10-30T12:45:03.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.640+0000 7fec82f65640 1 -- 172.21.15.12:0/1057593687 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fec600664a0 msgr2=0x7fec60068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:03.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.640+0000 7fec82f65640 1 --2- 172.21.15.12:0/1057593687 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fec600664a0 0x7fec60068960 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec70007fd0 tx=0x7fec70009040 comp rx=0 tx=0).stop 2023-10-30T12:45:03.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.640+0000 7fec82f65640 1 -- 172.21.15.12:0/1057593687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec7c153bf0 msgr2=0x7fec7c143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:03.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.640+0000 7fec82f65640 1 --2- 172.21.15.12:0/1057593687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec7c153bf0 0x7fec7c143a40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fec7401ce20 tx=0x7fec7401bf90 comp rx=0 tx=0).stop 2023-10-30T12:45:03.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.644+0000 7fec82f65640 1 -- 172.21.15.12:0/1057593687 shutdown_connections 2023-10-30T12:45:03.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.644+0000 7fec82f65640 1 --2- 172.21.15.12:0/1057593687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec7c153bf0 0x7fec7c143a40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.644+0000 7fec82f65640 1 --2- 172.21.15.12:0/1057593687 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fec600664a0 0x7fec60068960 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.644+0000 7fec82f65640 1 --2- 172.21.15.12:0/1057593687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec7c0c6fe0 0x7fec7c143500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.644+0000 7fec82f65640 1 --2- 172.21.15.12:0/1057593687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec7c0c5f60 0x7fec7c152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.644+0000 7fec82f65640 1 -- 172.21.15.12:0/1057593687 >> 172.21.15.12:0/1057593687 conn(0x7fec7c06c6e0 msgr2=0x7fec7c070520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:03.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.644+0000 7fec82f65640 1 -- 172.21.15.12:0/1057593687 shutdown_connections 2023-10-30T12:45:03.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:03.644+0000 7fec82f65640 1 -- 172.21.15.12:0/1057593687 wait complete. 2023-10-30T12:45:03.666 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:45:05.305 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:45:05.309 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:45:05.668 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:45:05.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.820+0000 7ffa4d667640 1 Processor -- start 2023-10-30T12:45:05.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.820+0000 7ffa4d667640 1 -- start start 2023-10-30T12:45:05.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.820+0000 7ffa4d667640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa481538f0 0x7ffa48159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:05.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.820+0000 7ffa4d667640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa480ca6a0 con 0x7ffa4815cd50 2023-10-30T12:45:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.820+0000 7ffa4d667640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa4815d300 con 0x7ffa480cefa0 2023-10-30T12:45:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.820+0000 7ffa4d667640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4815d5e0 con 0x7ffa481538f0 2023-10-30T12:45:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.820+0000 7ffa477fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ffa4815cd50 0x7ffa480c9f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54358/0 (socket says 172.21.15.12:54358) 2023-10-30T12:45:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.820+0000 7ffa477fe640 1 -- 172.21.15.12:0/3566324063 learned_addr learned my addr 172.21.15.12:0/3566324063 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:45:05.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.820+0000 7ffa47fff640 1 --2- 172.21.15.12:0/3566324063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa481538f0 0x7ffa48159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:05.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.820+0000 7ffa46ffd640 1 -- 172.21.15.12:0/3566324063 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770792963 0 0) 0x7ffa480ca6a0 con 0x7ffa4815cd50 2023-10-30T12:45:05.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.820+0000 7ffa46ffd640 1 -- 172.21.15.12:0/3566324063 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa30003530 con 0x7ffa4815cd50 2023-10-30T12:45:05.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.820+0000 7ffa47fff640 1 -- 172.21.15.12:0/3566324063 >> v1:172.21.15.154:6789/0 conn(0x7ffa480cefa0 legacy=0x7ffa481640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:05.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.820+0000 7ffa46ffd640 1 -- 172.21.15.12:0/3566324063 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 996267005 0 0) 0x7ffa30003530 con 0x7ffa4815cd50 2023-10-30T12:45:05.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.820+0000 7ffa47fff640 1 -- 172.21.15.12:0/3566324063 >> v1:172.21.15.12:6789/0 conn(0x7ffa4815cd50 legacy=0x7ffa480c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:05.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.820+0000 7ffa47fff640 1 -- 172.21.15.12:0/3566324063 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4815d8c0 con 0x7ffa481538f0 2023-10-30T12:45:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.824+0000 7ffa47fff640 1 --2- 172.21.15.12:0/3566324063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa481538f0 0x7ffa48159fb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffa380097f0 tx=0x7ffa3801ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c5adfb8f2c117af server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:05.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.824+0000 7ffa46ffd640 1 -- 172.21.15.12:0/3566324063 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c001680 con 0x7ffa481538f0 2023-10-30T12:45:05.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.824+0000 7ffa46ffd640 1 -- 172.21.15.12:0/3566324063 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa380a8650 con 0x7ffa481538f0 2023-10-30T12:45:05.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.824+0000 7ffa46ffd640 1 -- 172.21.15.12:0/3566324063 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa380a89e0 con 0x7ffa481538f0 2023-10-30T12:45:05.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.824+0000 7ffa4d667640 1 -- 172.21.15.12:0/3566324063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa481538f0 msgr2=0x7ffa48159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:05.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.824+0000 7ffa4d667640 1 --2- 172.21.15.12:0/3566324063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa481538f0 0x7ffa48159fb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffa380097f0 tx=0x7ffa3801ebd0 comp rx=0 tx=0).stop 2023-10-30T12:45:05.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.824+0000 7ffa4d667640 1 -- 172.21.15.12:0/3566324063 shutdown_connections 2023-10-30T12:45:05.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.824+0000 7ffa4d667640 1 --2- 172.21.15.12:0/3566324063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa481538f0 0x7ffa48159fb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:05.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.824+0000 7ffa4d667640 1 -- 172.21.15.12:0/3566324063 >> 172.21.15.12:0/3566324063 conn(0x7ffa4806a640 msgr2=0x7ffa48065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:05.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.824+0000 7ffa4d667640 1 -- 172.21.15.12:0/3566324063 shutdown_connections 2023-10-30T12:45:05.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.824+0000 7ffa4d667640 1 -- 172.21.15.12:0/3566324063 wait complete. 2023-10-30T12:45:05.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa4d667640 1 Processor -- start 2023-10-30T12:45:05.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa4d667640 1 -- start start 2023-10-30T12:45:05.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa4d667640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa480cefa0 0x7ffa4815c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:05.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa4d667640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa481538f0 0x7ffa481f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:05.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa4d667640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa4815cd50 0x7ffa480ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:05.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa4d667640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4815c610 con 0x7ffa480cefa0 2023-10-30T12:45:05.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa4d667640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4814dee0 con 0x7ffa481538f0 2023-10-30T12:45:05.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa4d667640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480b4c50 con 0x7ffa4815cd50 2023-10-30T12:45:05.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa47fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa480cefa0 0x7ffa4815c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:05.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa47fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa480cefa0 0x7ffa4815c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36774/0 (socket says 172.21.15.12:36774) 2023-10-30T12:45:05.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa47fff640 1 -- 172.21.15.12:0/3452619230 learned_addr learned my addr 172.21.15.12:0/3452619230 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:45:05.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa477fe640 1 --2- 172.21.15.12:0/3452619230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa481538f0 0x7ffa481f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:05.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa4ce66640 1 --2- 172.21.15.12:0/3452619230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa4815cd50 0x7ffa480ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:05.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa47fff640 1 -- 172.21.15.12:0/3452619230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa4815cd50 msgr2=0x7ffa480ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:05.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa47fff640 1 --2- 172.21.15.12:0/3452619230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa4815cd50 0x7ffa480ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:05.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa47fff640 1 -- 172.21.15.12:0/3452619230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa481538f0 msgr2=0x7ffa481f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:05.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa47fff640 1 --2- 172.21.15.12:0/3452619230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa481538f0 0x7ffa481f2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:05.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa47fff640 1 -- 172.21.15.12:0/3452619230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa38009420 con 0x7ffa480cefa0 2023-10-30T12:45:05.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa47fff640 1 --2- 172.21.15.12:0/3452619230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa480cefa0 0x7ffa4815c0d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffa380098e0 tx=0x7ffa3801b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:05.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa44ff9640 1 -- 172.21.15.12:0/3452619230 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38004220 con 0x7ffa480cefa0 2023-10-30T12:45:05.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa44ff9640 1 -- 172.21.15.12:0/3452619230 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa38004380 con 0x7ffa480cefa0 2023-10-30T12:45:05.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa4d667640 1 -- 172.21.15.12:0/3452619230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa480b4ed0 con 0x7ffa480cefa0 2023-10-30T12:45:05.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa44ff9640 1 -- 172.21.15.12:0/3452619230 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa380b0890 con 0x7ffa480cefa0 2023-10-30T12:45:05.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.828+0000 7ffa4d667640 1 -- 172.21.15.12:0/3452619230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa481f23b0 con 0x7ffa480cefa0 2023-10-30T12:45:05.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.832+0000 7ffa4d667640 1 -- 172.21.15.12:0/3452619230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4804e170 con 0x7ffa480cefa0 2023-10-30T12:45:05.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.832+0000 7ffa44ff9640 1 -- 172.21.15.12:0/3452619230 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ffa380b7040 con 0x7ffa480cefa0 2023-10-30T12:45:05.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.832+0000 7ffa44ff9640 1 --2- 172.21.15.12:0/3452619230 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa340664d0 0x7ffa34068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:05.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.832+0000 7ffa44ff9640 1 -- 172.21.15.12:0/3452619230 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7ffa38120a10 con 0x7ffa480cefa0 2023-10-30T12:45:05.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.832+0000 7ffa477fe640 1 --2- 172.21.15.12:0/3452619230 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa340664d0 0x7ffa34068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:05.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.832+0000 7ffa477fe640 1 --2- 172.21.15.12:0/3452619230 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa340664d0 0x7ffa34068990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffa4814ff90 tx=0x7ffa3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:05.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:05.836+0000 7ffa44ff9640 1 -- 172.21.15.12:0/3452619230 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ffa380edc20 con 0x7ffa480cefa0 2023-10-30T12:45:06.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:06.028+0000 7ffa4d667640 1 -- 172.21.15.12:0/3452619230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa480b2290 con 0x7ffa480cefa0 2023-10-30T12:45:06.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:06.028+0000 7ffa44ff9640 1 -- 172.21.15.12:0/3452619230 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+5744 (secure 0 0 0) 0x7ffa380b5090 con 0x7ffa480cefa0 2023-10-30T12:45:06.031 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:45:06.032 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:45:00.691908+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:clientreplay","state_seq":23,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:45:06.032 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 48 2023-10-30T12:45:06.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:06.028+0000 7ffa4d667640 1 -- 172.21.15.12:0/3452619230 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa340664d0 msgr2=0x7ffa34068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:06.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:06.028+0000 7ffa4d667640 1 --2- 172.21.15.12:0/3452619230 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa340664d0 0x7ffa34068990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffa4814ff90 tx=0x7ffa3c009040 comp rx=0 tx=0).stop 2023-10-30T12:45:06.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:06.028+0000 7ffa4d667640 1 -- 172.21.15.12:0/3452619230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa480cefa0 msgr2=0x7ffa4815c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:06.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:06.028+0000 7ffa4d667640 1 --2- 172.21.15.12:0/3452619230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa480cefa0 0x7ffa4815c0d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffa380098e0 tx=0x7ffa3801b970 comp rx=0 tx=0).stop 2023-10-30T12:45:06.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:06.028+0000 7ffa4d667640 1 -- 172.21.15.12:0/3452619230 shutdown_connections 2023-10-30T12:45:06.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:06.028+0000 7ffa4d667640 1 --2- 172.21.15.12:0/3452619230 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ffa340664d0 0x7ffa34068990 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:06.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:06.028+0000 7ffa4d667640 1 --2- 172.21.15.12:0/3452619230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa4815cd50 0x7ffa480ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:06.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:06.028+0000 7ffa4d667640 1 --2- 172.21.15.12:0/3452619230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa481538f0 0x7ffa481f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:06.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:06.028+0000 7ffa4d667640 1 --2- 172.21.15.12:0/3452619230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa480cefa0 0x7ffa4815c0d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:06.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:06.028+0000 7ffa4d667640 1 -- 172.21.15.12:0/3452619230 >> 172.21.15.12:0/3452619230 conn(0x7ffa4806a640 msgr2=0x7ffa480c5080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:06.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:06.028+0000 7ffa4d667640 1 -- 172.21.15.12:0/3452619230 shutdown_connections 2023-10-30T12:45:06.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:06.028+0000 7ffa4d667640 1 -- 172.21.15.12:0/3452619230 wait complete. 2023-10-30T12:45:06.052 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:45:06.500 INFO:tasks.workunit.client.0.smithi012.stderr:Updating files: 0% (48/12071) Updating files: 1% (121/12071) Updating files: 1% (237/12071) Updating files: 2% (242/12071) Updating files: 3% (363/12071) Updating files: 3% (424/12071) Updating files: 4% (483/12071) Updating files: 5% (604/12071) Updating files: 5% (721/12071) Updating files: 6% (725/12071) Updating files: 7% (845/12071) Updating files: 7% (919/12071) Updating files: 7% (937/12071) Updating files: 8% (966/12071) Updating files: 8% (996/12071) Updating files: 8% (1000/12071) Updating files: 8% (1075/12071) Updating files: 9% (1087/12071) Updating files: 9% (1161/12071) Updating files: 10% (1208/12071) Updating files: 10% (1250/12071) Updating files: 11% (1328/12071) Updating files: 12% (1449/12071) Updating files: 12% (1508/12071) Updating files: 13% (1570/12071) Updating files: 13% (1588/12071) Updating files: 14% (1690/12071) Updating files: 14% (1714/12071) Updating files: 14% (1794/12071) Updating files: 15% (1811/12071) Updating files: 16% (1932/12071) Updating files: 16% (2045/12071) Updating files: 17% (2053/12071) Updating files: 18% (2173/12071) Updating files: 19% (2294/12071) Updating files: 19% (2413/12071) Updating files: 20% (2415/12071) Updating files: 20% (2530/12071) Updating files: 21% (2535/12071) Updating files: 22% (2656/12071) Updating files: 22% (2697/12071) Updating files: 22% (2752/12071) Updating files: 23% (2777/12071) Updating files: 23% (2791/12071) Updating files: 24% (2898/12071) Updating files: 24% (2919/12071) Updating files: 25% (3018/12071) Updating files: 25% (3044/12071) Updating files: 25% (3117/12071) Updating files: 25% (3119/12071) Updating files: 26% (3139/12071) Updating files: 27% (3260/12071) Updating files: 27% (3370/12071) Updating files: 27% (3374/12071) Updating files: 28% (3380/12071) Updating files: 28% (3429/12071) Updating files: 29% (3501/12071) Updating files: 29% (3537/12071) Updating files: 29% (3583/12071) Updating files: 30% (3622/12071) Updating files: 30% (3647/12071) Updating files: 30% (3712/12071) Updating files: 31% (3743/12071) Updating files: 32% (3863/12071) Updating files: 32% (3887/12071) Updating files: 33% (3984/12071) Updating files: 34% (4105/12071) Updating files: 34% (4177/12071) Updating files: 35% (4225/12071) Updating files: 35% (4269/12071) Updating files: 35% (4319/12071) Updating files: 36% (4346/12071) Updating files: 37% (4467/12071) Updating files: 37% (4534/12071) Updating files: 38% (4587/12071) Updating files: 39% (4708/12071) Updating files: 39% (4761/12071) Updating files: 39% (4811/12071) Updating files: 40% (4829/12071) Updating files: 40% (4910/12071) Updating files: 41% (4950/12071) Updating files: 41% (4998/12071) Updating files: 42% (5070/12071) Updating files: 42% (5174/12071) Updating files: 43% (5191/12071) Updating files: 44% (5312/12071) Updating files: 44% (5329/12071) Updating files: 44% (5346/12071) Updating files: 45% (5432/12071) error: unable to create file src/common/AsyncOpTracker.cc: File exists 2023-10-30T12:45:08.053 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:45:08.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18e4ad6640 1 Processor -- start 2023-10-30T12:45:08.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18e4ad6640 1 -- start start 2023-10-30T12:45:08.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18e4ad6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f18e00c6b80 0x7f18e00bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:08.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18e4ad6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e00c0a50 con 0x7f18e00c6f60 2023-10-30T12:45:08.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18e4ad6640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e006f6e0 con 0x7f18e013f4a0 2023-10-30T12:45:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18e4ad6640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e006f9c0 con 0x7f18e00c6b80 2023-10-30T12:45:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18ddd9b640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f18e00c6f60 0x7f18e013ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54370/0 (socket says 172.21.15.12:54370) 2023-10-30T12:45:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18ddd9b640 1 -- 172.21.15.12:0/568395303 learned_addr learned my addr 172.21.15.12:0/568395303 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:45:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18de59c640 1 --2- 172.21.15.12:0/568395303 >> v2:172.21.15.154:3300/0 conn(0x7f18e00c6b80 0x7f18e00bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18dd59a640 1 -- 172.21.15.12:0/568395303 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4274091681 0 0) 0x7f18e00c0a50 con 0x7f18e00c6f60 2023-10-30T12:45:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18dd59a640 1 -- 172.21.15.12:0/568395303 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18c8003530 con 0x7f18e00c6f60 2023-10-30T12:45:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18dd59a640 1 -- 172.21.15.12:0/568395303 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2054733463 0 0) 0x7f18c8003530 con 0x7f18e00c6f60 2023-10-30T12:45:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18dd59a640 1 -- 172.21.15.12:0/568395303 >> v2:172.21.15.154:3300/0 conn(0x7f18e00c6b80 msgr2=0x7f18e00bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18dd59a640 1 --2- 172.21.15.12:0/568395303 >> v2:172.21.15.154:3300/0 conn(0x7f18e00c6b80 0x7f18e00bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18dd59a640 1 -- 172.21.15.12:0/568395303 >> v1:172.21.15.154:6789/0 conn(0x7f18e013f4a0 legacy=0x7f18e01566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18dd59a640 1 -- 172.21.15.12:0/568395303 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e006fca0 con 0x7f18e00c6f60 2023-10-30T12:45:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18dd59a640 1 -- 172.21.15.12:0/568395303 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f18d4001680 con 0x7f18e00c6f60 2023-10-30T12:45:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18dd59a640 1 -- 172.21.15.12:0/568395303 >> v1:172.21.15.12:6789/0 conn(0x7f18e00c6f60 legacy=0x7f18e013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18dd59a640 1 --2- 172.21.15.12:0/568395303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c8003980 0x7f18c8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18dd59a640 1 --2- 172.21.15.12:0/568395303 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18c80047d0 0x7f18c8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18dd59a640 1 --2- 172.21.15.12:0/568395303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18c8005190 0x7f18c800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18dd59a640 1 -- 172.21.15.12:0/568395303 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e006f9c0 con 0x7f18c8003980 2023-10-30T12:45:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18dd59a640 1 -- 172.21.15.12:0/568395303 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c800c030 con 0x7f18c8005190 2023-10-30T12:45:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18dd59a640 1 -- 172.21.15.12:0/568395303 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c800c1f0 con 0x7f18c80047d0 2023-10-30T12:45:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18de59c640 1 --2- 172.21.15.12:0/568395303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c8003980 0x7f18c8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18ddd9b640 1 --2- 172.21.15.12:0/568395303 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18c80047d0 0x7f18c8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:08.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18ded9d640 1 --2- 172.21.15.12:0/568395303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18c8005190 0x7f18c800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:08.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18de59c640 1 -- 172.21.15.12:0/568395303 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18c80047d0 msgr2=0x7f18c8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:08.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18de59c640 1 --2- 172.21.15.12:0/568395303 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18c80047d0 0x7f18c8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:08.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18de59c640 1 -- 172.21.15.12:0/568395303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18c8005190 msgr2=0x7f18c800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:08.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18de59c640 1 --2- 172.21.15.12:0/568395303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18c8005190 0x7f18c800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:08.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18de59c640 1 -- 172.21.15.12:0/568395303 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c8003530 con 0x7f18c8003980 2023-10-30T12:45:08.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.204+0000 7f18ded9d640 1 -- 172.21.15.12:0/568395303 reap_dead start 2023-10-30T12:45:08.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.208+0000 7f18de59c640 1 --2- 172.21.15.12:0/568395303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c8003980 0x7f18c8003da0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009ae0 tx=0x7f18d001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1903feac467a3ead server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:08.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.208+0000 7f18dd59a640 1 -- 172.21.15.12:0/568395303 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d00a5530 con 0x7f18c8003980 2023-10-30T12:45:08.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.208+0000 7f18dd59a640 1 -- 172.21.15.12:0/568395303 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d00a5690 con 0x7f18c8003980 2023-10-30T12:45:08.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.208+0000 7f18dd59a640 1 -- 172.21.15.12:0/568395303 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d00a5a40 con 0x7f18c8003980 2023-10-30T12:45:08.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.208+0000 7f18e4ad6640 1 -- 172.21.15.12:0/568395303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c8003980 msgr2=0x7f18c8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:08.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.208+0000 7f18e4ad6640 1 --2- 172.21.15.12:0/568395303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c8003980 0x7f18c8003da0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009ae0 tx=0x7f18d001ca50 comp rx=0 tx=0).stop 2023-10-30T12:45:08.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.208+0000 7f18e4ad6640 1 -- 172.21.15.12:0/568395303 shutdown_connections 2023-10-30T12:45:08.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.208+0000 7f18e4ad6640 1 --2- 172.21.15.12:0/568395303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c8003980 0x7f18c8003da0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:08.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.208+0000 7f18e4ad6640 1 -- 172.21.15.12:0/568395303 >> 172.21.15.12:0/568395303 conn(0x7f18e00c7910 msgr2=0x7f18e00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:08.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.208+0000 7f18e4ad6640 1 -- 172.21.15.12:0/568395303 shutdown_connections 2023-10-30T12:45:08.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.208+0000 7f18e4ad6640 1 -- 172.21.15.12:0/568395303 wait complete. 2023-10-30T12:45:08.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18e4ad6640 1 Processor -- start 2023-10-30T12:45:08.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18e4ad6640 1 -- start start 2023-10-30T12:45:08.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18e4ad6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18c8003980 0x7f18e00aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:08.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18e4ad6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e01562c0 0x7f18e0154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:08.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18e4ad6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18e013a810 0x7f18e01386c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:08.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18e4ad6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e013a3f0 con 0x7f18e013a810 2023-10-30T12:45:08.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18e4ad6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e00af300 con 0x7f18e01562c0 2023-10-30T12:45:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18e4ad6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e0139520 con 0x7f18c8003980 2023-10-30T12:45:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18ded9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18e013a810 0x7f18e01386c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18ded9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18e013a810 0x7f18e01386c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36798/0 (socket says 172.21.15.12:36798) 2023-10-30T12:45:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18ded9d640 1 -- 172.21.15.12:0/20810166 learned_addr learned my addr 172.21.15.12:0/20810166 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:45:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18de59c640 1 --2- 172.21.15.12:0/20810166 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18c8003980 0x7f18e00aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18ddd9b640 1 --2- 172.21.15.12:0/20810166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e01562c0 0x7f18e0154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18ded9d640 1 -- 172.21.15.12:0/20810166 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18c8003980 msgr2=0x7f18e00aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18ded9d640 1 --2- 172.21.15.12:0/20810166 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18c8003980 0x7f18e00aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18ded9d640 1 -- 172.21.15.12:0/20810166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e01562c0 msgr2=0x7f18e0154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18ded9d640 1 --2- 172.21.15.12:0/20810166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e01562c0 0x7f18e0154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18ded9d640 1 -- 172.21.15.12:0/20810166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e0139770 con 0x7f18e013a810 2023-10-30T12:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18ddd9b640 1 --2- 172.21.15.12:0/20810166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e01562c0 0x7f18e0154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18ded9d640 1 --2- 172.21.15.12:0/20810166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18e013a810 0x7f18e01386c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f18e013f880 tx=0x7f18d800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18ceffd640 1 -- 172.21.15.12:0/20810166 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8006d70 con 0x7f18e013a810 2023-10-30T12:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18ceffd640 1 -- 172.21.15.12:0/20810166 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d8003980 con 0x7f18e013a810 2023-10-30T12:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18e4ad6640 1 -- 172.21.15.12:0/20810166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e0139a50 con 0x7f18e013a810 2023-10-30T12:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.212+0000 7f18ceffd640 1 -- 172.21.15.12:0/20810166 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8003d30 con 0x7f18e013a810 2023-10-30T12:45:08.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.216+0000 7f18e4ad6640 1 -- 172.21.15.12:0/20810166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e013a010 con 0x7f18e013a810 2023-10-30T12:45:08.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.216+0000 7f18e4ad6640 1 -- 172.21.15.12:0/20810166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e013a530 con 0x7f18e013a810 2023-10-30T12:45:08.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.216+0000 7f18ceffd640 1 -- 172.21.15.12:0/20810166 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f18d8005070 con 0x7f18e013a810 2023-10-30T12:45:08.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.216+0000 7f18ceffd640 1 --2- 172.21.15.12:0/20810166 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f18c00664d0 0x7f18c0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:08.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.216+0000 7f18ceffd640 1 -- 172.21.15.12:0/20810166 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f18d8083560 con 0x7f18e013a810 2023-10-30T12:45:08.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.216+0000 7f18de59c640 1 --2- 172.21.15.12:0/20810166 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f18c00664d0 0x7f18c0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:08.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.220+0000 7f18de59c640 1 --2- 172.21.15.12:0/20810166 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f18c00664d0 0x7f18c0068990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f18d00af000 tx=0x7f18d00046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:08.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.220+0000 7f18ceffd640 1 -- 172.21.15.12:0/20810166 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f18e013a530 con 0x7f18e013a810 2023-10-30T12:45:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.412+0000 7f18e4ad6640 1 -- 172.21.15.12:0/20810166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18e013a530 con 0x7f18e013a810 2023-10-30T12:45:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.412+0000 7f18ceffd640 1 -- 172.21.15.12:0/20810166 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+5738 (secure 0 0 0) 0x7f18e013a530 con 0x7f18e013a810 2023-10-30T12:45:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:45:08.418 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:45:06.708275+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:45:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 49 2023-10-30T12:45:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.412+0000 7f18e4ad6640 1 -- 172.21.15.12:0/20810166 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f18c00664d0 msgr2=0x7f18c0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.412+0000 7f18e4ad6640 1 --2- 172.21.15.12:0/20810166 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f18c00664d0 0x7f18c0068990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f18d00af000 tx=0x7f18d00046d0 comp rx=0 tx=0).stop 2023-10-30T12:45:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.412+0000 7f18e4ad6640 1 -- 172.21.15.12:0/20810166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18e013a810 msgr2=0x7f18e01386c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.412+0000 7f18e4ad6640 1 --2- 172.21.15.12:0/20810166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18e013a810 0x7f18e01386c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f18e013f880 tx=0x7f18d800e960 comp rx=0 tx=0).stop 2023-10-30T12:45:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.412+0000 7f18e4ad6640 1 -- 172.21.15.12:0/20810166 shutdown_connections 2023-10-30T12:45:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.412+0000 7f18e4ad6640 1 --2- 172.21.15.12:0/20810166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18e013a810 0x7f18e01386c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.412+0000 7f18e4ad6640 1 --2- 172.21.15.12:0/20810166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e01562c0 0x7f18e0154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.412+0000 7f18e4ad6640 1 --2- 172.21.15.12:0/20810166 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f18c00664d0 0x7f18c0068990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.412+0000 7f18e4ad6640 1 --2- 172.21.15.12:0/20810166 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18c8003980 0x7f18e00aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.412+0000 7f18e4ad6640 1 -- 172.21.15.12:0/20810166 >> 172.21.15.12:0/20810166 conn(0x7f18e00c7910 msgr2=0x7f18e013b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.412+0000 7f18e4ad6640 1 -- 172.21.15.12:0/20810166 shutdown_connections 2023-10-30T12:45:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:08.412+0000 7f18e4ad6640 1 -- 172.21.15.12:0/20810166 wait complete. 2023-10-30T12:45:08.437 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d has gained rank=0, replacing gid=4285 2023-10-30T12:45:08.438 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 4 secs before reviving mds.e 2023-10-30T12:45:12.438 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2023-10-30T12:45:12.439 INFO:tasks.ceph.mds.e:Restarting daemon 2023-10-30T12:45:12.439 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-10-30T12:45:12.443 INFO:tasks.ceph.mds.e:Started 2023-10-30T12:45:12.444 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:45:12.537 INFO:tasks.ceph.mds.e.smithi012.stdout:starting mds.e at 2023-10-30T12:45:12.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.591+0000 7f343826b640 1 Processor -- start 2023-10-30T12:45:12.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.591+0000 7f343826b640 1 -- start start 2023-10-30T12:45:12.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.595+0000 7f343826b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f343015d6f0 0x7f343015a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.595+0000 7f343826b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34300cf160 0x7f34300cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.595+0000 7f343826b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34300cb800 con 0x7f34301538a0 2023-10-30T12:45:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.595+0000 7f343826b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34300cb330 con 0x7f343015d6f0 2023-10-30T12:45:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.595+0000 7f343826b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34300cad70 con 0x7f34300cf160 2023-10-30T12:45:12.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.595+0000 7f3435806640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f343015d6f0 0x7f343015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:12.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.595+0000 7f3436007640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f34301538a0 0x7f34300ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50468/0 (socket says 172.21.15.12:50468) 2023-10-30T12:45:12.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.595+0000 7f3436007640 1 -- 172.21.15.12:0/2979623635 learned_addr learned my addr 172.21.15.12:0/2979623635 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:45:12.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.595+0000 7f3436808640 1 --2- 172.21.15.12:0/2979623635 >> v2:172.21.15.154:3300/0 conn(0x7f34300cf160 0x7f34300cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:12.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.595+0000 7f3435005640 1 -- 172.21.15.12:0/2979623635 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98539242 0 0) 0x7f34300cb800 con 0x7f34301538a0 2023-10-30T12:45:12.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.595+0000 7f3435005640 1 -- 172.21.15.12:0/2979623635 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f341c003530 con 0x7f34301538a0 2023-10-30T12:45:12.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.595+0000 7f3435806640 1 -- 172.21.15.12:0/2979623635 >> v2:172.21.15.154:3300/0 conn(0x7f34300cf160 msgr2=0x7f34300cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:12.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.595+0000 7f3435806640 1 --2- 172.21.15.12:0/2979623635 >> v2:172.21.15.154:3300/0 conn(0x7f34300cf160 0x7f34300cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.595+0000 7f3435806640 1 -- 172.21.15.12:0/2979623635 >> v1:172.21.15.12:6789/0 conn(0x7f34301538a0 legacy=0x7f34300ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:12.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.595+0000 7f3435806640 1 -- 172.21.15.12:0/2979623635 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34300cb050 con 0x7f343015d6f0 2023-10-30T12:45:12.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.599+0000 7f3435806640 1 --2- 172.21.15.12:0/2979623635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f343015d6f0 0x7f343015a090 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f34300b5330 tx=0x7f342001ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186b6dff8857e50e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:12.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.599+0000 7f3435005640 1 -- 172.21.15.12:0/2979623635 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34200a8950 con 0x7f343015d6f0 2023-10-30T12:45:12.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.599+0000 7f3435005640 1 -- 172.21.15.12:0/2979623635 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34200a8ab0 con 0x7f343015d6f0 2023-10-30T12:45:12.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.599+0000 7f3435005640 1 -- 172.21.15.12:0/2979623635 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34200a8e40 con 0x7f343015d6f0 2023-10-30T12:45:12.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.599+0000 7f343826b640 1 -- 172.21.15.12:0/2979623635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f343015d6f0 msgr2=0x7f343015a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:12.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.599+0000 7f343826b640 1 --2- 172.21.15.12:0/2979623635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f343015d6f0 0x7f343015a090 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f34300b5330 tx=0x7f342001ec50 comp rx=0 tx=0).stop 2023-10-30T12:45:12.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.599+0000 7f343826b640 1 -- 172.21.15.12:0/2979623635 shutdown_connections 2023-10-30T12:45:12.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.599+0000 7f343826b640 1 --2- 172.21.15.12:0/2979623635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f343015d6f0 0x7f343015a090 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.599+0000 7f343826b640 1 --2- 172.21.15.12:0/2979623635 >> v2:172.21.15.154:3300/0 conn(0x7f34300cf160 0x7f34300cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.599+0000 7f343826b640 1 -- 172.21.15.12:0/2979623635 >> 172.21.15.12:0/2979623635 conn(0x7f343006bfe0 msgr2=0x7f343006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:12.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.599+0000 7f343826b640 1 -- 172.21.15.12:0/2979623635 shutdown_connections 2023-10-30T12:45:12.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.599+0000 7f343826b640 1 -- 172.21.15.12:0/2979623635 wait complete. 2023-10-30T12:45:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.599+0000 7f343826b640 1 Processor -- start 2023-10-30T12:45:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.599+0000 7f343826b640 1 -- start start 2023-10-30T12:45:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f343826b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34300cf160 0x7f34301ef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f343826b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34301538a0 0x7f3430158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f343826b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f343015d6f0 0x7f3430158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f343826b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34301eff20 con 0x7f34301538a0 2023-10-30T12:45:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f343826b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343015a8b0 con 0x7f343015d6f0 2023-10-30T12:45:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f343826b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343015bf40 con 0x7f34300cf160 2023-10-30T12:45:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3436007640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34300cf160 0x7f34301ef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3435806640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34301538a0 0x7f3430158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3436007640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34300cf160 0x7f34301ef9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:33288/0 (socket says 172.21.15.12:33288) 2023-10-30T12:45:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3436007640 1 -- 172.21.15.12:0/1549817039 learned_addr learned my addr 172.21.15.12:0/1549817039 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:45:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3436808640 1 --2- 172.21.15.12:0/1549817039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f343015d6f0 0x7f3430158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3435806640 1 -- 172.21.15.12:0/1549817039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34300cf160 msgr2=0x7f34301ef9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3435806640 1 --2- 172.21.15.12:0/1549817039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34300cf160 0x7f34301ef9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3435806640 1 -- 172.21.15.12:0/1549817039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f343015d6f0 msgr2=0x7f3430158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3435806640 1 --2- 172.21.15.12:0/1549817039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f343015d6f0 0x7f3430158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3435806640 1 -- 172.21.15.12:0/1549817039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3420009420 con 0x7f34301538a0 2023-10-30T12:45:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3436007640 1 --2- 172.21.15.12:0/1549817039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34300cf160 0x7f34301ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:12.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3435806640 1 --2- 172.21.15.12:0/1549817039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34301538a0 0x7f3430158940 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f34200af000 tx=0x7f342001b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:12.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3426ffd640 1 -- 172.21.15.12:0/1549817039 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34200bd070 con 0x7f34301538a0 2023-10-30T12:45:12.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3426ffd640 1 -- 172.21.15.12:0/1549817039 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34200b0ba0 con 0x7f34301538a0 2023-10-30T12:45:12.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f343826b640 1 -- 172.21.15.12:0/1549817039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343015c210 con 0x7f34301538a0 2023-10-30T12:45:12.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3426ffd640 1 -- 172.21.15.12:0/1549817039 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34200b9430 con 0x7f34301538a0 2023-10-30T12:45:12.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f343826b640 1 -- 172.21.15.12:0/1549817039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343015c7d0 con 0x7f34301538a0 2023-10-30T12:45:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f343826b640 1 -- 172.21.15.12:0/1549817039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34300b2a50 con 0x7f34301538a0 2023-10-30T12:45:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.603+0000 7f3426ffd640 1 -- 172.21.15.12:0/1549817039 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f34200b7040 con 0x7f34301538a0 2023-10-30T12:45:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.607+0000 7f3426ffd640 1 --2- 172.21.15.12:0/1549817039 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34140664a0 0x7f3414068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.607+0000 7f3426ffd640 1 -- 172.21.15.12:0/1549817039 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f34201211c0 con 0x7f34301538a0 2023-10-30T12:45:12.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.607+0000 7f3436007640 1 --2- 172.21.15.12:0/1549817039 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34140664a0 0x7f3414068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:12.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.607+0000 7f3436007640 1 --2- 172.21.15.12:0/1549817039 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34140664a0 0x7f3414068960 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f342c0011f0 tx=0x7f342c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:12.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.611+0000 7f3426ffd640 1 -- 172.21.15.12:0/1549817039 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f34200ee3f0 con 0x7f34301538a0 2023-10-30T12:45:12.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.799+0000 7f343826b640 1 -- 172.21.15.12:0/1549817039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f343004e170 con 0x7f34301538a0 2023-10-30T12:45:12.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.799+0000 7f3426ffd640 1 -- 172.21.15.12:0/1549817039 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+5738 (secure 0 0 0) 0x7f34200f1a10 con 0x7f34301538a0 2023-10-30T12:45:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:45:12.806 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:45:06.708275+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:45:12.807 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 49 2023-10-30T12:45:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.803+0000 7f343826b640 1 -- 172.21.15.12:0/1549817039 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34140664a0 msgr2=0x7f3414068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.803+0000 7f343826b640 1 --2- 172.21.15.12:0/1549817039 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34140664a0 0x7f3414068960 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f342c0011f0 tx=0x7f342c009040 comp rx=0 tx=0).stop 2023-10-30T12:45:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.803+0000 7f343826b640 1 -- 172.21.15.12:0/1549817039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34301538a0 msgr2=0x7f3430158940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.803+0000 7f343826b640 1 --2- 172.21.15.12:0/1549817039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34301538a0 0x7f3430158940 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f34200af000 tx=0x7f342001b9f0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.803+0000 7f343826b640 1 -- 172.21.15.12:0/1549817039 shutdown_connections 2023-10-30T12:45:12.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.803+0000 7f343826b640 1 --2- 172.21.15.12:0/1549817039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f343015d6f0 0x7f3430158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.803+0000 7f343826b640 1 --2- 172.21.15.12:0/1549817039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34301538a0 0x7f3430158940 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.803+0000 7f343826b640 1 --2- 172.21.15.12:0/1549817039 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34140664a0 0x7f3414068960 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.803+0000 7f343826b640 1 --2- 172.21.15.12:0/1549817039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34300cf160 0x7f34301ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.803+0000 7f343826b640 1 -- 172.21.15.12:0/1549817039 >> 172.21.15.12:0/1549817039 conn(0x7f343006bfe0 msgr2=0x7f343006c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:12.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.803+0000 7f343826b640 1 -- 172.21.15.12:0/1549817039 shutdown_connections 2023-10-30T12:45:12.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:12.803+0000 7f343826b640 1 -- 172.21.15.12:0/1549817039 wait complete. 2023-10-30T12:45:12.826 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2023-10-30T12:45:13.343 INFO:tasks.ceph.mds.d.smithi154.stderr:2023-10-30T12:45:13.347+0000 7f45c531e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-30T12:45:13.343 INFO:tasks.ceph.mds.d.smithi154.stderr:2023-10-30T12:45:13.347+0000 7f45c531e640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-30T12:45:13.343 INFO:tasks.ceph.mds.d.smithi154.stderr:2023-10-30T12:45:13.347+0000 7f45c531e640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-30T12:45:14.827 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:45:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.979+0000 7f9b75409640 1 Processor -- start 2023-10-30T12:45:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.979+0000 7f9b75409640 1 -- start start 2023-10-30T12:45:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.979+0000 7f9b75409640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9b7015d6f0 0x7f9b7015a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.979+0000 7f9b75409640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b700cb800 con 0x7f9b700cf160 2023-10-30T12:45:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.979+0000 7f9b75409640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b700cb330 con 0x7f9b701538a0 2023-10-30T12:45:14.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.979+0000 7f9b75409640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b700cad70 con 0x7f9b7015d6f0 2023-10-30T12:45:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.979+0000 7f9b6f7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9b700cf160 0x7f9b70164070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50470/0 (socket says 172.21.15.12:50470) 2023-10-30T12:45:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.979+0000 7f9b6f7fe640 1 -- 172.21.15.12:0/210846697 learned_addr learned my addr 172.21.15.12:0/210846697 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:45:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.979+0000 7f9b667fc640 1 --2- 172.21.15.12:0/210846697 >> v2:172.21.15.154:3300/0 conn(0x7f9b7015d6f0 0x7f9b7015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.979+0000 7f9b6e7fc640 1 -- 172.21.15.12:0/210846697 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023064267 0 0) 0x7f9b700cb800 con 0x7f9b700cf160 2023-10-30T12:45:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.979+0000 7f9b6e7fc640 1 -- 172.21.15.12:0/210846697 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b5c003530 con 0x7f9b700cf160 2023-10-30T12:45:14.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.979+0000 7f9b6e7fc640 1 -- 172.21.15.12:0/210846697 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3603155289 0 0) 0x7f9b5c003530 con 0x7f9b700cf160 2023-10-30T12:45:14.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.979+0000 7f9b667fc640 1 -- 172.21.15.12:0/210846697 >> v1:172.21.15.154:6789/0 conn(0x7f9b701538a0 legacy=0x7f9b700ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.979+0000 7f9b667fc640 1 -- 172.21.15.12:0/210846697 >> v1:172.21.15.12:6789/0 conn(0x7f9b700cf160 legacy=0x7f9b70164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.979+0000 7f9b667fc640 1 -- 172.21.15.12:0/210846697 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b700cb050 con 0x7f9b7015d6f0 2023-10-30T12:45:14.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.983+0000 7f9b667fc640 1 --2- 172.21.15.12:0/210846697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b7015d6f0 0x7f9b7015a090 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9b600097c0 tx=0x7f9b6001eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3e1067b05608c70 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:14.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.983+0000 7f9b6e7fc640 1 -- 172.21.15.12:0/210846697 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68001a00 con 0x7f9b7015d6f0 2023-10-30T12:45:14.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.983+0000 7f9b6e7fc640 1 -- 172.21.15.12:0/210846697 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b600a8610 con 0x7f9b7015d6f0 2023-10-30T12:45:14.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.983+0000 7f9b6e7fc640 1 -- 172.21.15.12:0/210846697 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b600a8980 con 0x7f9b7015d6f0 2023-10-30T12:45:14.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.983+0000 7f9b75409640 1 -- 172.21.15.12:0/210846697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b7015d6f0 msgr2=0x7f9b7015a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.983+0000 7f9b75409640 1 --2- 172.21.15.12:0/210846697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b7015d6f0 0x7f9b7015a090 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9b600097c0 tx=0x7f9b6001eb90 comp rx=0 tx=0).stop 2023-10-30T12:45:14.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b75409640 1 -- 172.21.15.12:0/210846697 shutdown_connections 2023-10-30T12:45:14.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b75409640 1 --2- 172.21.15.12:0/210846697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b7015d6f0 0x7f9b7015a090 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b75409640 1 -- 172.21.15.12:0/210846697 >> 172.21.15.12:0/210846697 conn(0x7f9b7006bfe0 msgr2=0x7f9b7006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:14.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b75409640 1 -- 172.21.15.12:0/210846697 shutdown_connections 2023-10-30T12:45:14.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b75409640 1 -- 172.21.15.12:0/210846697 wait complete. 2023-10-30T12:45:14.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b75409640 1 Processor -- start 2023-10-30T12:45:14.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b75409640 1 -- start start 2023-10-30T12:45:14.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b75409640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b700cf160 0x7f9b700cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b75409640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b701538a0 0x7f9b700cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b75409640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9b7015d6f0 0x7f9b7014cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b75409640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b700cd2b0 con 0x7f9b700cf160 2023-10-30T12:45:14.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b75409640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7014e880 con 0x7f9b701538a0 2023-10-30T12:45:14.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b75409640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b70158ab0 con 0x7f9b7015d6f0 2023-10-30T12:45:14.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b6effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b700cf160 0x7f9b700cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b6effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b700cf160 0x7f9b700cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40080/0 (socket says 172.21.15.12:40080) 2023-10-30T12:45:14.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b6effd640 1 -- 172.21.15.12:0/1856264088 learned_addr learned my addr 172.21.15.12:0/1856264088 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:45:14.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b667fc640 1 --2- 172.21.15.12:0/1856264088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b701538a0 0x7f9b700cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b6f7fe640 1 --2- 172.21.15.12:0/1856264088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9b7015d6f0 0x7f9b7014cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b6effd640 1 -- 172.21.15.12:0/1856264088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9b7015d6f0 msgr2=0x7f9b7014cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b6effd640 1 --2- 172.21.15.12:0/1856264088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9b7015d6f0 0x7f9b7014cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b6effd640 1 -- 172.21.15.12:0/1856264088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b701538a0 msgr2=0x7f9b700cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b6effd640 1 --2- 172.21.15.12:0/1856264088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b701538a0 0x7f9b700cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b6effd640 1 -- 172.21.15.12:0/1856264088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b60009420 con 0x7f9b700cf160 2023-10-30T12:45:14.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.987+0000 7f9b6f7fe640 1 --2- 172.21.15.12:0/1856264088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9b7015d6f0 0x7f9b7014cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:14.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.991+0000 7f9b6effd640 1 --2- 172.21.15.12:0/1856264088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b700cf160 0x7f9b700cd450 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800c500 tx=0x7f9b5800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.991+0000 7f9b67fff640 1 -- 172.21.15.12:0/1856264088 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b58005220 con 0x7f9b700cf160 2023-10-30T12:45:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.991+0000 7f9b67fff640 1 -- 172.21.15.12:0/1856264088 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b58005380 con 0x7f9b700cf160 2023-10-30T12:45:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.991+0000 7f9b67fff640 1 -- 172.21.15.12:0/1856264088 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b58007270 con 0x7f9b700cf160 2023-10-30T12:45:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.991+0000 7f9b75409640 1 -- 172.21.15.12:0/1856264088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b70157f00 con 0x7f9b700cf160 2023-10-30T12:45:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.991+0000 7f9b75409640 1 -- 172.21.15.12:0/1856264088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b70158180 con 0x7f9b700cf160 2023-10-30T12:45:14.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.991+0000 7f9b75409640 1 -- 172.21.15.12:0/1856264088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b7004e170 con 0x7f9b700cf160 2023-10-30T12:45:15.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.991+0000 7f9b67fff640 1 -- 172.21.15.12:0/1856264088 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9b580073d0 con 0x7f9b700cf160 2023-10-30T12:45:15.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.991+0000 7f9b67fff640 1 --2- 172.21.15.12:0/1856264088 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9b500664a0 0x7f9b50068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.991+0000 7f9b67fff640 1 -- 172.21.15.12:0/1856264088 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f9b580835f0 con 0x7f9b700cf160 2023-10-30T12:45:15.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.991+0000 7f9b667fc640 1 --2- 172.21.15.12:0/1856264088 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9b500664a0 0x7f9b50068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.995+0000 7f9b667fc640 1 --2- 172.21.15.12:0/1856264088 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9b500664a0 0x7f9b50068960 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9b600b2000 tx=0x7f9b600aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:15.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:14.995+0000 7f9b67fff640 1 -- 172.21.15.12:0/1856264088 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9b58050810 con 0x7f9b700cf160 2023-10-30T12:45:15.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:15.191+0000 7f9b75409640 1 -- 172.21.15.12:0/1856264088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b7015c8c0 con 0x7f9b700cf160 2023-10-30T12:45:15.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:15.191+0000 7f9b67fff640 1 -- 172.21.15.12:0/1856264088 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6529 (secure 0 0 0) 0x7f9b58053e30 con 0x7f9b700cf160 2023-10-30T12:45:15.197 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:45:15.198 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:45:06.708275+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:45:15.198 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 50 2023-10-30T12:45:15.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:15.195+0000 7f9b75409640 1 -- 172.21.15.12:0/1856264088 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9b500664a0 msgr2=0x7f9b50068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:15.195+0000 7f9b75409640 1 --2- 172.21.15.12:0/1856264088 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9b500664a0 0x7f9b50068960 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9b600b2000 tx=0x7f9b600aa070 comp rx=0 tx=0).stop 2023-10-30T12:45:15.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:15.195+0000 7f9b75409640 1 -- 172.21.15.12:0/1856264088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b700cf160 msgr2=0x7f9b700cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:15.195+0000 7f9b75409640 1 --2- 172.21.15.12:0/1856264088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b700cf160 0x7f9b700cd450 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800c500 tx=0x7f9b5800c9c0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:15.195+0000 7f9b75409640 1 -- 172.21.15.12:0/1856264088 shutdown_connections 2023-10-30T12:45:15.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:15.195+0000 7f9b75409640 1 --2- 172.21.15.12:0/1856264088 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9b500664a0 0x7f9b50068960 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:15.195+0000 7f9b75409640 1 --2- 172.21.15.12:0/1856264088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9b7015d6f0 0x7f9b7014cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:15.195+0000 7f9b75409640 1 --2- 172.21.15.12:0/1856264088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b701538a0 0x7f9b700cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:15.195+0000 7f9b75409640 1 --2- 172.21.15.12:0/1856264088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b700cf160 0x7f9b700cd450 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:15.195+0000 7f9b75409640 1 -- 172.21.15.12:0/1856264088 >> 172.21.15.12:0/1856264088 conn(0x7f9b7006bfe0 msgr2=0x7f9b7015b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:15.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:15.195+0000 7f9b75409640 1 -- 172.21.15.12:0/1856264088 shutdown_connections 2023-10-30T12:45:15.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:45:15.195+0000 7f9b75409640 1 -- 172.21.15.12:0/1856264088 wait complete. 2023-10-30T12:45:15.219 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2023-10-30T12:45:15.219 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 67 secs before thrashing 2023-10-30T12:45:35.369 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:45:35.375 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:46:05.432 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:46:05.436 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:46:11.227 INFO:tasks.workunit.client.0.smithi012.stderr:Updating files: 45% (5498/12071) Updating files: 45% (5499/12071) Updating files: 46% (5553/12071) Updating files: 47% (5674/12071) Updating files: 47% (5678/12071) Updating files: 48% (5795/12071) Updating files: 48% (5847/12071) Updating files: 48% (5862/12071) Updating files: 49% (5915/12071) Updating files: 49% (5931/12071) Updating files: 50% (6036/12071) Updating files: 50% (6097/12071) Updating files: 51% (6157/12071) Updating files: 51% (6234/12071) Updating files: 52% (6277/12071) Updating files: 52% (6330/12071) Updating files: 53% (6398/12071) Updating files: 53% (6516/12071) Updating files: 54% (6519/12071) Updating files: 55% (6640/12071) Updating files: 56% (6760/12071) Updating files: 57% (6881/12071) Updating files: 57% (6894/12071) Updating files: 58% (7002/12071) Updating files: 58% (7035/12071) Updating files: 59% (7122/12071) Updating files: 59% (7133/12071) Updating files: 60% (7243/12071) Updating files: 61% (7364/12071) Updating files: 62% (7485/12071) Updating files: 62% (7513/12071) Updating files: 63% (7605/12071) Updating files: 63% (7624/12071) Updating files: 64% (7726/12071) Updating files: 64% (7787/12071) Updating files: 65% (7847/12071) Updating files: 66% (7967/12071) Updating files: 66% (8004/12071) Updating files: 67% (8088/12071) Updating files: 68% (8209/12071) Updating files: 68% (8252/12071) Updating files: 69% (8329/12071) Updating files: 69% (8357/12071) Updating files: 69% (8372/12071) Updating files: 70% (8450/12071) Updating files: 70% (8559/12071) Updating files: 71% (8571/12071) Updating files: 72% (8692/12071) Updating files: 73% (8812/12071) Updating files: 73% (8820/12071) Updating files: 73% (8931/12071) Updating files: 74% (8933/12071) Updating files: 75% (9054/12071) Updating files: 75% (9061/12071) Updating files: 75% (9105/12071) Updating files: 75% (9124/12071) Updating files: 76% (9174/12071) Updating files: 77% (9295/12071) Updating files: 77% (9314/12071) Updating files: 77% (9406/12071) Updating files: 78% (9416/12071) Updating files: 78% (9532/12071) Updating files: 79% (9537/12071) Updating files: 80% (9657/12071) Updating files: 80% (9702/12071) Updating files: 80% (9718/12071) Updating files: 81% (9778/12071) Updating files: 81% (9833/12071) Updating files: 82% (9899/12071) Updating files: 83% (10019/12071) Updating files: 83% (10047/12071) Updating files: 84% (10140/12071) Updating files: 85% (10261/12071) Updating files: 85% (10297/12071) Updating files: 85% (10347/12071) Updating files: 86% (10382/12071) Updating files: 86% (10469/12071) Updating files: 86% (10499/12071) Updating files: 87% (10502/12071) Updating files: 87% (10620/12071) Updating files: 88% (10623/12071) Updating files: 88% (10632/12071) Updating files: 89% (10744/12071) Updating files: 90% (10864/12071) Updating files: 91% (10985/12071) Updating files: 91% (10991/12071) Updating files: 92% (11106/12071) Updating files: 93% (11227/12071) Updating files: 94% (11347/12071) Updating files: 94% (11370/12071) Updating files: 95% (11468/12071) Updating files: 95% (11476/12071) Updating files: 96% (11589/12071) Updating files: 96% (11603/12071) Updating files: 96% (11700/12071) Updating files: 97% (11709/12071) Updating files: 98% (11830/12071) Updating files: 99% (11951/12071) Updating files: 99% (11989/12071) Updating files: 99% (12011/12071) Updating files: 99% (12038/12071) Updating files: 100% (12071/12071) Updating files: 100% (12071/12071), done. 2023-10-30T12:46:11.228 INFO:tasks.workunit.client.0.smithi012.stderr:fatal: unable to checkout working tree 2023-10-30T12:46:11.232 INFO:tasks.workunit.client.0.smithi012.stderr:warning: Clone succeeded, but checkout failed. 2023-10-30T12:46:11.232 INFO:tasks.workunit.client.0.smithi012.stderr:You can inspect what was checked out with 'git status' 2023-10-30T12:46:11.233 INFO:tasks.workunit.client.0.smithi012.stderr:and retry with 'git restore --source=HEAD :/' 2023-10-30T12:46:11.233 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-10-30T12:46:11.300 INFO:tasks.workunit.client.0.smithi012.stderr:+ retry 2023-10-30T12:46:11.301 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm -rf ceph 2023-10-30T12:46:22.220 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:22.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb45dd3e640 1 Processor -- start 2023-10-30T12:46:22.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb45dd3e640 1 -- start start 2023-10-30T12:46:22.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb45dd3e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb45815d6f0 0x7fb45815a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:22.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb45dd3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4580cf160 0x7fb4580cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:22.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb45dd3e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4580cb800 con 0x7fb4581538a0 2023-10-30T12:46:22.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb45dd3e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4580cb330 con 0x7fb45815d6f0 2023-10-30T12:46:22.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb45dd3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4580cad70 con 0x7fb4580cf160 2023-10-30T12:46:22.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb457fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4580cf160 0x7fb4580cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:22.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb456ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb45815d6f0 0x7fb45815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:22.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb4577fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb4581538a0 0x7fb4580ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:57254/0 (socket says 172.21.15.12:57254) 2023-10-30T12:46:22.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb4577fe640 1 -- 172.21.15.12:0/925088154 learned_addr learned my addr 172.21.15.12:0/925088154 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:46:22.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb457fff640 1 -- 172.21.15.12:0/925088154 >> v2:172.21.15.154:3300/0 conn(0x7fb45815d6f0 msgr2=0x7fb45815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:22.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb457fff640 1 --2- 172.21.15.12:0/925088154 >> v2:172.21.15.154:3300/0 conn(0x7fb45815d6f0 0x7fb45815a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb457fff640 1 -- 172.21.15.12:0/925088154 >> v1:172.21.15.154:6789/0 conn(0x7fb4581538a0 legacy=0x7fb4580ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:22.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb457fff640 1 -- 172.21.15.12:0/925088154 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4580cb050 con 0x7fb4580cf160 2023-10-30T12:46:22.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.371+0000 7fb456ffd640 1 --2- 172.21.15.12:0/925088154 >> v2:172.21.15.154:3300/0 conn(0x7fb45815d6f0 0x7fb45815a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:46:22.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.375+0000 7fb457fff640 1 --2- 172.21.15.12:0/925088154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4580cf160 0x7fb4580cf590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb450009d90 tx=0x7fb45001f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77712f8f720ff0be server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:22.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.375+0000 7fb4567fc640 1 -- 172.21.15.12:0/925088154 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4500af600 con 0x7fb4580cf160 2023-10-30T12:46:22.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.375+0000 7fb4567fc640 1 -- 172.21.15.12:0/925088154 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4500ab050 con 0x7fb4580cf160 2023-10-30T12:46:22.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.375+0000 7fb4567fc640 1 -- 172.21.15.12:0/925088154 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4500af980 con 0x7fb4580cf160 2023-10-30T12:46:22.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 -- 172.21.15.12:0/925088154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4580cf160 msgr2=0x7fb4580cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:22.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 --2- 172.21.15.12:0/925088154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4580cf160 0x7fb4580cf590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb450009d90 tx=0x7fb45001f130 comp rx=0 tx=0).stop 2023-10-30T12:46:22.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 -- 172.21.15.12:0/925088154 shutdown_connections 2023-10-30T12:46:22.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 --2- 172.21.15.12:0/925088154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4580cf160 0x7fb4580cf590 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 --2- 172.21.15.12:0/925088154 >> v2:172.21.15.154:3300/0 conn(0x7fb45815d6f0 0x7fb45815a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 -- 172.21.15.12:0/925088154 >> 172.21.15.12:0/925088154 conn(0x7fb45806bfe0 msgr2=0x7fb45806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:22.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 -- 172.21.15.12:0/925088154 shutdown_connections 2023-10-30T12:46:22.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 -- 172.21.15.12:0/925088154 wait complete. 2023-10-30T12:46:22.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 Processor -- start 2023-10-30T12:46:22.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 -- start start 2023-10-30T12:46:22.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4580cf160 0x7fb45814e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:22.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4581538a0 0x7fb4580b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:22.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb45815d6f0 0x7fb4580cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:22.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581efd70 con 0x7fb4580cf160 2023-10-30T12:46:22.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581f0260 con 0x7fb45815d6f0 2023-10-30T12:46:22.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb45dd3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4580ce7b0 con 0x7fb4581538a0 2023-10-30T12:46:22.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb4577fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4580cf160 0x7fb45814e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:22.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb4577fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4580cf160 0x7fb45814e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38340/0 (socket says 172.21.15.12:38340) 2023-10-30T12:46:22.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb4577fe640 1 -- 172.21.15.12:0/1850122917 learned_addr learned my addr 172.21.15.12:0/1850122917 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:22.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb456ffd640 1 --2- 172.21.15.12:0/1850122917 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4581538a0 0x7fb4580b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:22.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb457fff640 1 --2- 172.21.15.12:0/1850122917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb45815d6f0 0x7fb4580cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:22.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb4577fe640 1 -- 172.21.15.12:0/1850122917 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4581538a0 msgr2=0x7fb4580b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:22.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb4577fe640 1 --2- 172.21.15.12:0/1850122917 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4581538a0 0x7fb4580b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb4577fe640 1 -- 172.21.15.12:0/1850122917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb45815d6f0 msgr2=0x7fb4580cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb4577fe640 1 --2- 172.21.15.12:0/1850122917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb45815d6f0 0x7fb4580cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.379+0000 7fb4577fe640 1 -- 172.21.15.12:0/1850122917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c009420 con 0x7fb4580cf160 2023-10-30T12:46:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.383+0000 7fb4577fe640 1 --2- 172.21.15.12:0/1850122917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4580cf160 0x7fb45814e220 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb44800c500 tx=0x7fb44800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.383+0000 7fb43ffff640 1 -- 172.21.15.12:0/1850122917 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448005cf0 con 0x7fb4580cf160 2023-10-30T12:46:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.383+0000 7fb43ffff640 1 -- 172.21.15.12:0/1850122917 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb448005e50 con 0x7fb4580cf160 2023-10-30T12:46:22.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.383+0000 7fb43ffff640 1 -- 172.21.15.12:0/1850122917 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4480072f0 con 0x7fb4580cf160 2023-10-30T12:46:22.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.383+0000 7fb45dd3e640 1 -- 172.21.15.12:0/1850122917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb450009a70 con 0x7fb4580cf160 2023-10-30T12:46:22.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.383+0000 7fb45dd3e640 1 -- 172.21.15.12:0/1850122917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4581eff80 con 0x7fb4580cf160 2023-10-30T12:46:22.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.383+0000 7fb45dd3e640 1 -- 172.21.15.12:0/1850122917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4580b2a50 con 0x7fb4580cf160 2023-10-30T12:46:22.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.383+0000 7fb43ffff640 1 -- 172.21.15.12:0/1850122917 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb448007450 con 0x7fb4580cf160 2023-10-30T12:46:22.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.383+0000 7fb43ffff640 1 --2- 172.21.15.12:0/1850122917 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb4440664d0 0x7fb444068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:22.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.383+0000 7fb43ffff640 1 -- 172.21.15.12:0/1850122917 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fb448083460 con 0x7fb4580cf160 2023-10-30T12:46:22.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.383+0000 7fb456ffd640 1 --2- 172.21.15.12:0/1850122917 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb4440664d0 0x7fb444068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:22.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.387+0000 7fb456ffd640 1 --2- 172.21.15.12:0/1850122917 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb4440664d0 0x7fb444068990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb4580c98a0 tx=0x7fb44c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:22.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.387+0000 7fb43ffff640 1 -- 172.21.15.12:0/1850122917 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb448050850 con 0x7fb4580cf160 2023-10-30T12:46:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.607+0000 7fb45dd3e640 1 -- 172.21.15.12:0/1850122917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb45804e170 con 0x7fb4580cf160 2023-10-30T12:46:22.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.607+0000 7fb43ffff640 1 -- 172.21.15.12:0/1850122917 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6538 (secure 0 0 0) 0x7fb448053e70 con 0x7fb4580cf160 2023-10-30T12:46:22.611 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:22.611 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:18.260197+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:22.612 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 56 2023-10-30T12:46:22.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.607+0000 7fb45dd3e640 1 -- 172.21.15.12:0/1850122917 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb4440664d0 msgr2=0x7fb444068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:22.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.607+0000 7fb45dd3e640 1 --2- 172.21.15.12:0/1850122917 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb4440664d0 0x7fb444068990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb4580c98a0 tx=0x7fb44c009190 comp rx=0 tx=0).stop 2023-10-30T12:46:22.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.607+0000 7fb45dd3e640 1 -- 172.21.15.12:0/1850122917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4580cf160 msgr2=0x7fb45814e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:22.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.607+0000 7fb45dd3e640 1 --2- 172.21.15.12:0/1850122917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4580cf160 0x7fb45814e220 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb44800c500 tx=0x7fb44800c9c0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.607+0000 7fb45dd3e640 1 -- 172.21.15.12:0/1850122917 shutdown_connections 2023-10-30T12:46:22.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.607+0000 7fb45dd3e640 1 --2- 172.21.15.12:0/1850122917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb45815d6f0 0x7fb4580cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.607+0000 7fb45dd3e640 1 --2- 172.21.15.12:0/1850122917 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb4440664d0 0x7fb444068990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.607+0000 7fb45dd3e640 1 --2- 172.21.15.12:0/1850122917 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4581538a0 0x7fb4580b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.607+0000 7fb45dd3e640 1 --2- 172.21.15.12:0/1850122917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4580cf160 0x7fb45814e220 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.607+0000 7fb45dd3e640 1 -- 172.21.15.12:0/1850122917 >> 172.21.15.12:0/1850122917 conn(0x7fb45806bfe0 msgr2=0x7fb4580c48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:22.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.607+0000 7fb45dd3e640 1 -- 172.21.15.12:0/1850122917 shutdown_connections 2023-10-30T12:46:22.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.607+0000 7fb45dd3e640 1 -- 172.21.15.12:0/1850122917 wait complete. 2023-10-30T12:46:22.632 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=2) 2023-10-30T12:46:22.632 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-30T12:46:22.633 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:46:22.651 INFO:tasks.ceph.mds.c:Stopped 2023-10-30T12:46:22.651 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:22.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.835+0000 7f07cacc9640 1 Processor -- start 2023-10-30T12:46:22.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.835+0000 7f07cacc9640 1 -- start start 2023-10-30T12:46:22.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.835+0000 7f07cacc9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f07c40c6f60 0x7f07c41566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:22.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.835+0000 7f07cacc9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07c40c0a50 con 0x7f07c40c6b80 2023-10-30T12:46:22.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.835+0000 7f07cacc9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07c406f6e0 con 0x7f07c40c05d0 2023-10-30T12:46:22.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.835+0000 7f07cacc9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07c406f9c0 con 0x7f07c40c6f60 2023-10-30T12:46:22.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.835+0000 7f07c8a65640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f07c40c6b80 0x7f07c40bfec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50170/0 (socket says 172.21.15.12:50170) 2023-10-30T12:46:22.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.835+0000 7f07c8a65640 1 -- 172.21.15.12:0/2848126299 learned_addr learned my addr 172.21.15.12:0/2848126299 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:46:22.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.835+0000 7f07c9266640 1 --2- 172.21.15.12:0/2848126299 >> v2:172.21.15.154:3300/0 conn(0x7f07c40c6f60 0x7f07c41566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:22.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.835+0000 7f07c37fe640 1 -- 172.21.15.12:0/2848126299 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107469559 0 0) 0x7f07c40c0a50 con 0x7f07c40c6b80 2023-10-30T12:46:22.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c37fe640 1 -- 172.21.15.12:0/2848126299 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07b4005740 con 0x7f07c40c6b80 2023-10-30T12:46:22.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c37fe640 1 -- 172.21.15.12:0/2848126299 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 962970483 0 0) 0x7f07b4005740 con 0x7f07c40c6b80 2023-10-30T12:46:22.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c37fe640 1 -- 172.21.15.12:0/2848126299 >> v2:172.21.15.154:3300/0 conn(0x7f07c40c6f60 msgr2=0x7f07c41566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:22.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c37fe640 1 --2- 172.21.15.12:0/2848126299 >> v2:172.21.15.154:3300/0 conn(0x7f07c40c6f60 0x7f07c41566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c37fe640 1 -- 172.21.15.12:0/2848126299 >> v1:172.21.15.154:6789/0 conn(0x7f07c40c05d0 legacy=0x7f07c413ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:22.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c37fe640 1 -- 172.21.15.12:0/2848126299 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c406fca0 con 0x7f07c40c6b80 2023-10-30T12:46:22.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c37fe640 1 -- 172.21.15.12:0/2848126299 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f07b8001680 con 0x7f07c40c6b80 2023-10-30T12:46:22.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c37fe640 1 -- 172.21.15.12:0/2848126299 >> v1:172.21.15.12:6789/0 conn(0x7f07c40c6b80 legacy=0x7f07c40bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:22.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c37fe640 1 --2- 172.21.15.12:0/2848126299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b4005b90 0x7f07b4005f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:22.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c37fe640 1 --2- 172.21.15.12:0/2848126299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07b4006590 0x7f07b4006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:22.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c37fe640 1 --2- 172.21.15.12:0/2848126299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b4007260 0x7f07b400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:22.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c37fe640 1 -- 172.21.15.12:0/2848126299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c406f9c0 con 0x7f07b4006590 2023-10-30T12:46:22.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c37fe640 1 -- 172.21.15.12:0/2848126299 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b400e1c0 con 0x7f07b4007260 2023-10-30T12:46:22.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c37fe640 1 -- 172.21.15.12:0/2848126299 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b400e380 con 0x7f07b4005b90 2023-10-30T12:46:22.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c3fff640 1 --2- 172.21.15.12:0/2848126299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07b4006590 0x7f07b4006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:22.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c8a65640 1 --2- 172.21.15.12:0/2848126299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b4005b90 0x7f07b4005f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:22.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c9266640 1 --2- 172.21.15.12:0/2848126299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b4007260 0x7f07b400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:22.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c3fff640 1 -- 172.21.15.12:0/2848126299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b4005b90 msgr2=0x7f07b4005f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:22.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c3fff640 1 --2- 172.21.15.12:0/2848126299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b4005b90 0x7f07b4005f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c3fff640 1 -- 172.21.15.12:0/2848126299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b4007260 msgr2=0x7f07b400dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:22.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c3fff640 1 --2- 172.21.15.12:0/2848126299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b4007260 0x7f07b400dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c3fff640 1 -- 172.21.15.12:0/2848126299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b4005740 con 0x7f07b4006590 2023-10-30T12:46:22.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c9266640 1 -- 172.21.15.12:0/2848126299 reap_dead start 2023-10-30T12:46:22.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.839+0000 7f07c3fff640 1 --2- 172.21.15.12:0/2848126299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07b4006590 0x7f07b4006a60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f07c406c9a0 tx=0x7f07b001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ddbdcc9838b4bcd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:22.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07c37fe640 1 -- 172.21.15.12:0/2848126299 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b00a8890 con 0x7f07b4006590 2023-10-30T12:46:22.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07c37fe640 1 -- 172.21.15.12:0/2848126299 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b00a8a10 con 0x7f07b4006590 2023-10-30T12:46:22.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07c37fe640 1 -- 172.21.15.12:0/2848126299 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b00a8dc0 con 0x7f07b4006590 2023-10-30T12:46:22.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07cacc9640 1 -- 172.21.15.12:0/2848126299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07b4006590 msgr2=0x7f07b4006a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:22.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07cacc9640 1 --2- 172.21.15.12:0/2848126299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07b4006590 0x7f07b4006a60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f07c406c9a0 tx=0x7f07b001ec70 comp rx=0 tx=0).stop 2023-10-30T12:46:22.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07cacc9640 1 -- 172.21.15.12:0/2848126299 shutdown_connections 2023-10-30T12:46:22.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07cacc9640 1 --2- 172.21.15.12:0/2848126299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07b4006590 0x7f07b4006a60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07cacc9640 1 -- 172.21.15.12:0/2848126299 >> 172.21.15.12:0/2848126299 conn(0x7f07c40c7910 msgr2=0x7f07c40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:22.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07cacc9640 1 -- 172.21.15.12:0/2848126299 shutdown_connections 2023-10-30T12:46:22.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07cacc9640 1 -- 172.21.15.12:0/2848126299 wait complete. 2023-10-30T12:46:22.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07cacc9640 1 Processor -- start 2023-10-30T12:46:22.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07cacc9640 1 -- start start 2023-10-30T12:46:22.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07cacc9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07b4006590 0x7f07c41eafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:22.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07cacc9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07c4158ce0 0x7f07c4155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:22.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07cacc9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07c4159160 0x7f07c414f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:22.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07cacc9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c4157470 con 0x7f07b4006590 2023-10-30T12:46:22.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07cacc9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c41584b0 con 0x7f07c4159160 2023-10-30T12:46:22.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07cacc9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c41eb4f0 con 0x7f07c4158ce0 2023-10-30T12:46:22.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07c8a65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07b4006590 0x7f07c41eafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:22.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07c8a65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07b4006590 0x7f07c41eafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38362/0 (socket says 172.21.15.12:38362) 2023-10-30T12:46:22.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.843+0000 7f07c8a65640 1 -- 172.21.15.12:0/1699740807 learned_addr learned my addr 172.21.15.12:0/1699740807 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:22.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07c3fff640 1 --2- 172.21.15.12:0/1699740807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07c4158ce0 0x7f07c4155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:22.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07c9266640 1 --2- 172.21.15.12:0/1699740807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07c4159160 0x7f07c414f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:22.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07c8a65640 1 -- 172.21.15.12:0/1699740807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07c4158ce0 msgr2=0x7f07c4155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:22.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07c8a65640 1 --2- 172.21.15.12:0/1699740807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07c4158ce0 0x7f07c4155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07c8a65640 1 -- 172.21.15.12:0/1699740807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07c4159160 msgr2=0x7f07c414f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07c8a65640 1 --2- 172.21.15.12:0/1699740807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07c4159160 0x7f07c414f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07c8a65640 1 -- 172.21.15.12:0/1699740807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c41595e0 con 0x7f07b4006590 2023-10-30T12:46:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07c9266640 1 --2- 172.21.15.12:0/1699740807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07c4159160 0x7f07c414f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:46:22.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07c8a65640 1 --2- 172.21.15.12:0/1699740807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07b4006590 0x7f07c41eafb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e780 tx=0x7f07b800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:22.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07c17fa640 1 -- 172.21.15.12:0/1699740807 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b800c6c0 con 0x7f07b4006590 2023-10-30T12:46:22.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07c17fa640 1 -- 172.21.15.12:0/1699740807 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b800c860 con 0x7f07b4006590 2023-10-30T12:46:22.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07c17fa640 1 -- 172.21.15.12:0/1699740807 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b800cc10 con 0x7f07b4006590 2023-10-30T12:46:22.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07cacc9640 1 -- 172.21.15.12:0/1699740807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c4157830 con 0x7f07b4006590 2023-10-30T12:46:22.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07cacc9640 1 -- 172.21.15.12:0/1699740807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c4157df0 con 0x7f07b4006590 2023-10-30T12:46:22.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07cacc9640 1 -- 172.21.15.12:0/1699740807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c404e170 con 0x7f07b4006590 2023-10-30T12:46:22.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07c17fa640 1 -- 172.21.15.12:0/1699740807 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f07b801d460 con 0x7f07b4006590 2023-10-30T12:46:22.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07c17fa640 1 --2- 172.21.15.12:0/1699740807 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f07a80664a0 0x7f07a8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:22.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.847+0000 7f07c17fa640 1 -- 172.21.15.12:0/1699740807 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f07b8083f90 con 0x7f07b4006590 2023-10-30T12:46:22.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.851+0000 7f07c3fff640 1 --2- 172.21.15.12:0/1699740807 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f07a80664a0 0x7f07a8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:22.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.851+0000 7f07c3fff640 1 --2- 172.21.15.12:0/1699740807 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f07a80664a0 0x7f07a8068960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f07b00b2000 tx=0x7f07b0001330 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:22.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:22.851+0000 7f07c17fa640 1 -- 172.21.15.12:0/1699740807 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f07b8051110 con 0x7f07b4006590 2023-10-30T12:46:23.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:23.043+0000 7f07cacc9640 1 -- 172.21.15.12:0/1699740807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07c413adc0 con 0x7f07b4006590 2023-10-30T12:46:23.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:23.043+0000 7f07c17fa640 1 -- 172.21.15.12:0/1699740807 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6538 (secure 0 0 0) 0x7f07b8054730 con 0x7f07b4006590 2023-10-30T12:46:23.047 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:23.047 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:18.260197+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:23.047 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 56 2023-10-30T12:46:23.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:23.043+0000 7f07cacc9640 1 -- 172.21.15.12:0/1699740807 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f07a80664a0 msgr2=0x7f07a8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:23.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:23.043+0000 7f07cacc9640 1 --2- 172.21.15.12:0/1699740807 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f07a80664a0 0x7f07a8068960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f07b00b2000 tx=0x7f07b0001330 comp rx=0 tx=0).stop 2023-10-30T12:46:23.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:23.043+0000 7f07cacc9640 1 -- 172.21.15.12:0/1699740807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07b4006590 msgr2=0x7f07c41eafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:23.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:23.043+0000 7f07cacc9640 1 --2- 172.21.15.12:0/1699740807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07b4006590 0x7f07c41eafb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e780 tx=0x7f07b800ec40 comp rx=0 tx=0).stop 2023-10-30T12:46:23.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:23.043+0000 7f07cacc9640 1 -- 172.21.15.12:0/1699740807 shutdown_connections 2023-10-30T12:46:23.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:23.043+0000 7f07cacc9640 1 --2- 172.21.15.12:0/1699740807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07c4159160 0x7f07c414f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:23.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:23.043+0000 7f07cacc9640 1 --2- 172.21.15.12:0/1699740807 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f07a80664a0 0x7f07a8068960 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:23.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:23.043+0000 7f07cacc9640 1 --2- 172.21.15.12:0/1699740807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07c4158ce0 0x7f07c4155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:23.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:23.043+0000 7f07cacc9640 1 --2- 172.21.15.12:0/1699740807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07b4006590 0x7f07c41eafb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:23.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:23.043+0000 7f07cacc9640 1 -- 172.21.15.12:0/1699740807 >> 172.21.15.12:0/1699740807 conn(0x7f07c40c7910 msgr2=0x7f07c40c0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:23.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:23.043+0000 7f07cacc9640 1 -- 172.21.15.12:0/1699740807 shutdown_connections 2023-10-30T12:46:23.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:23.043+0000 7f07cacc9640 1 -- 172.21.15.12:0/1699740807 wait complete. 2023-10-30T12:46:23.068 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:46:25.069 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.219+0000 7fbe6929c640 1 Processor -- start 2023-10-30T12:46:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.219+0000 7fbe6929c640 1 -- start start 2023-10-30T12:46:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.223+0000 7fbe6929c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbe6415d6f0 0x7fbe6415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.223+0000 7fbe6929c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbe640cf160 0x7fbe640cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.223+0000 7fbe6929c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe640cb800 con 0x7fbe641538a0 2023-10-30T12:46:25.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.223+0000 7fbe6929c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe640cb330 con 0x7fbe6415d6f0 2023-10-30T12:46:25.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.223+0000 7fbe6929c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe640cad70 con 0x7fbe640cf160 2023-10-30T12:46:25.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.223+0000 7fbe627fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbe6415d6f0 0x7fbe6415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:25.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.223+0000 7fbe627fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbe6415d6f0 0x7fbe6415a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38366/0 (socket says 172.21.15.12:38366) 2023-10-30T12:46:25.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.223+0000 7fbe62ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbe641538a0 0x7fbe640ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50186/0 (socket says 172.21.15.12:50186) 2023-10-30T12:46:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.223+0000 7fbe627fc640 1 -- 172.21.15.12:0/2762848990 learned_addr learned my addr 172.21.15.12:0/2762848990 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.223+0000 7fbe637fe640 1 --2- 172.21.15.12:0/2762848990 >> v2:172.21.15.154:3300/0 conn(0x7fbe640cf160 0x7fbe640cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.223+0000 7fbe627fc640 1 -- 172.21.15.12:0/2762848990 >> v2:172.21.15.154:3300/0 conn(0x7fbe640cf160 msgr2=0x7fbe640cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.223+0000 7fbe627fc640 1 --2- 172.21.15.12:0/2762848990 >> v2:172.21.15.154:3300/0 conn(0x7fbe640cf160 0x7fbe640cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.223+0000 7fbe627fc640 1 -- 172.21.15.12:0/2762848990 >> v1:172.21.15.12:6789/0 conn(0x7fbe641538a0 legacy=0x7fbe640ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:25.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.223+0000 7fbe627fc640 1 -- 172.21.15.12:0/2762848990 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe640cb050 con 0x7fbe6415d6f0 2023-10-30T12:46:25.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.223+0000 7fbe61ffb640 1 -- 172.21.15.12:0/2762848990 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611459216 0 0) 0x7fbe640cb800 con 0x7fbe641538a0 2023-10-30T12:46:25.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.227+0000 7fbe627fc640 1 --2- 172.21.15.12:0/2762848990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe6415d6f0 0x7fbe6415a090 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbe640b5330 tx=0x7fbe5801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0e2e0af3ac50f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:25.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.227+0000 7fbe61ffb640 1 -- 172.21.15.12:0/2762848990 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe580a89f0 con 0x7fbe6415d6f0 2023-10-30T12:46:25.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.227+0000 7fbe61ffb640 1 -- 172.21.15.12:0/2762848990 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe580a8b50 con 0x7fbe6415d6f0 2023-10-30T12:46:25.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.227+0000 7fbe61ffb640 1 -- 172.21.15.12:0/2762848990 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe580b0440 con 0x7fbe6415d6f0 2023-10-30T12:46:25.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.227+0000 7fbe6929c640 1 -- 172.21.15.12:0/2762848990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe6415d6f0 msgr2=0x7fbe6415a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:25.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.227+0000 7fbe6929c640 1 --2- 172.21.15.12:0/2762848990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe6415d6f0 0x7fbe6415a090 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbe640b5330 tx=0x7fbe5801ecf0 comp rx=0 tx=0).stop 2023-10-30T12:46:25.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.227+0000 7fbe6929c640 1 -- 172.21.15.12:0/2762848990 shutdown_connections 2023-10-30T12:46:25.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.227+0000 7fbe6929c640 1 --2- 172.21.15.12:0/2762848990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe6415d6f0 0x7fbe6415a090 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:25.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.227+0000 7fbe6929c640 1 --2- 172.21.15.12:0/2762848990 >> v2:172.21.15.154:3300/0 conn(0x7fbe640cf160 0x7fbe640cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:25.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.227+0000 7fbe6929c640 1 -- 172.21.15.12:0/2762848990 >> 172.21.15.12:0/2762848990 conn(0x7fbe6406bfe0 msgr2=0x7fbe6406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:25.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.227+0000 7fbe6929c640 1 -- 172.21.15.12:0/2762848990 shutdown_connections 2023-10-30T12:46:25.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.227+0000 7fbe6929c640 1 -- 172.21.15.12:0/2762848990 wait complete. 2023-10-30T12:46:25.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.227+0000 7fbe6929c640 1 Processor -- start 2023-10-30T12:46:25.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.227+0000 7fbe6929c640 1 -- start start 2023-10-30T12:46:25.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe6929c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe640cf160 0x7fbe641464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:25.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe6929c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe641538a0 0x7fbe640b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:25.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe6929c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe6415d6f0 0x7fbe640b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:25.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe6929c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640ce8f0 con 0x7fbe6415d6f0 2023-10-30T12:46:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe6929c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640b4380 con 0x7fbe641538a0 2023-10-30T12:46:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe6929c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640cd3c0 con 0x7fbe640cf160 2023-10-30T12:46:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe6415d6f0 0x7fbe640b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe6415d6f0 0x7fbe640b4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38374/0 (socket says 172.21.15.12:38374) 2023-10-30T12:46:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe637fe640 1 -- 172.21.15.12:0/4195138494 learned_addr learned my addr 172.21.15.12:0/4195138494 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:25.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe62ffd640 1 --2- 172.21.15.12:0/4195138494 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe640cf160 0x7fbe641464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:25.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe627fc640 1 --2- 172.21.15.12:0/4195138494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe641538a0 0x7fbe640b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:25.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe637fe640 1 -- 172.21.15.12:0/4195138494 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe640cf160 msgr2=0x7fbe641464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:25.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe637fe640 1 --2- 172.21.15.12:0/4195138494 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe640cf160 0x7fbe641464a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:25.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe637fe640 1 -- 172.21.15.12:0/4195138494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe641538a0 msgr2=0x7fbe640b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:25.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe637fe640 1 --2- 172.21.15.12:0/4195138494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe641538a0 0x7fbe640b48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:25.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe637fe640 1 -- 172.21.15.12:0/4195138494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe580094d0 con 0x7fbe6415d6f0 2023-10-30T12:46:25.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe637fe640 1 --2- 172.21.15.12:0/4195138494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe6415d6f0 0x7fbe640b4e00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c007d90 tx=0x7fbe5c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:25.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe4b7fe640 1 -- 172.21.15.12:0/4195138494 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c014420 con 0x7fbe6415d6f0 2023-10-30T12:46:25.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe4b7fe640 1 -- 172.21.15.12:0/4195138494 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5c0145a0 con 0x7fbe6415d6f0 2023-10-30T12:46:25.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe4b7fe640 1 -- 172.21.15.12:0/4195138494 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c014950 con 0x7fbe6415d6f0 2023-10-30T12:46:25.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe6929c640 1 -- 172.21.15.12:0/4195138494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe64145950 con 0x7fbe6415d6f0 2023-10-30T12:46:25.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe6929c640 1 -- 172.21.15.12:0/4195138494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe640cd050 con 0x7fbe6415d6f0 2023-10-30T12:46:25.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe6929c640 1 -- 172.21.15.12:0/4195138494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe640b2a50 con 0x7fbe6415d6f0 2023-10-30T12:46:25.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.231+0000 7fbe4b7fe640 1 -- 172.21.15.12:0/4195138494 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbe5c014ab0 con 0x7fbe6415d6f0 2023-10-30T12:46:25.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.235+0000 7fbe4b7fe640 1 --2- 172.21.15.12:0/4195138494 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbe500664d0 0x7fbe50068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:25.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.235+0000 7fbe4b7fe640 1 -- 172.21.15.12:0/4195138494 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fbe5c084820 con 0x7fbe6415d6f0 2023-10-30T12:46:25.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.235+0000 7fbe62ffd640 1 --2- 172.21.15.12:0/4195138494 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbe500664d0 0x7fbe50068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:25.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.235+0000 7fbe62ffd640 1 --2- 172.21.15.12:0/4195138494 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbe500664d0 0x7fbe50068990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbe54007fd0 tx=0x7fbe54007d60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:25.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.239+0000 7fbe4b7fe640 1 -- 172.21.15.12:0/4195138494 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbe5c051ae0 con 0x7fbe6415d6f0 2023-10-30T12:46:25.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.427+0000 7fbe6929c640 1 -- 172.21.15.12:0/4195138494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe640cd530 con 0x7fbe6415d6f0 2023-10-30T12:46:25.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.427+0000 7fbe4b7fe640 1 -- 172.21.15.12:0/4195138494 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6538 (secure 0 0 0) 0x7fbe5c055100 con 0x7fbe6415d6f0 2023-10-30T12:46:25.433 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:25.433 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:18.260197+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:25.434 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 56 2023-10-30T12:46:25.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.431+0000 7fbe6929c640 1 -- 172.21.15.12:0/4195138494 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbe500664d0 msgr2=0x7fbe50068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:25.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.431+0000 7fbe6929c640 1 --2- 172.21.15.12:0/4195138494 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbe500664d0 0x7fbe50068990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbe54007fd0 tx=0x7fbe54007d60 comp rx=0 tx=0).stop 2023-10-30T12:46:25.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.431+0000 7fbe6929c640 1 -- 172.21.15.12:0/4195138494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe6415d6f0 msgr2=0x7fbe640b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:25.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.431+0000 7fbe6929c640 1 --2- 172.21.15.12:0/4195138494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe6415d6f0 0x7fbe640b4e00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c007d90 tx=0x7fbe5c00e4b0 comp rx=0 tx=0).stop 2023-10-30T12:46:25.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.431+0000 7fbe6929c640 1 -- 172.21.15.12:0/4195138494 shutdown_connections 2023-10-30T12:46:25.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.431+0000 7fbe6929c640 1 --2- 172.21.15.12:0/4195138494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe6415d6f0 0x7fbe640b4e00 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:25.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.431+0000 7fbe6929c640 1 --2- 172.21.15.12:0/4195138494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe641538a0 0x7fbe640b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:25.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.431+0000 7fbe6929c640 1 --2- 172.21.15.12:0/4195138494 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbe500664d0 0x7fbe50068990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:25.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.431+0000 7fbe6929c640 1 --2- 172.21.15.12:0/4195138494 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe640cf160 0x7fbe641464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:25.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.431+0000 7fbe6929c640 1 -- 172.21.15.12:0/4195138494 >> 172.21.15.12:0/4195138494 conn(0x7fbe6406bfe0 msgr2=0x7fbe6406c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:25.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.431+0000 7fbe6929c640 1 -- 172.21.15.12:0/4195138494 shutdown_connections 2023-10-30T12:46:25.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:25.431+0000 7fbe6929c640 1 -- 172.21.15.12:0/4195138494 wait complete. 2023-10-30T12:46:25.456 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:46:27.457 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:27.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.607+0000 7fb7bc13e640 1 Processor -- start 2023-10-30T12:46:27.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.607+0000 7fb7bc13e640 1 -- start start 2023-10-30T12:46:27.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.607+0000 7fb7bc13e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb7b40cb550 0x7fb7b40cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:27.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.607+0000 7fb7bc13e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7b4071250 0x7fb7b40716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.607+0000 7fb7bc13e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b4071be0 con 0x7fb7b414fd50 2023-10-30T12:46:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.607+0000 7fb7bc13e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b41560a0 con 0x7fb7b40cb550 2023-10-30T12:46:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.607+0000 7fb7bc13e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b4156380 con 0x7fb7b4071250 2023-10-30T12:46:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.607+0000 7fb7ba6db640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7b4071250 0x7fb7b40716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.607+0000 7fb7b96d9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb7b40cb550 0x7fb7b40cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.607+0000 7fb7b9eda640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb7b414fd50 0x7fb7b40c65d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:57274/0 (socket says 172.21.15.12:57274) 2023-10-30T12:46:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.607+0000 7fb7ba6db640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7b4071250 0x7fb7b40716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:40380/0 (socket says 172.21.15.12:40380) 2023-10-30T12:46:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.607+0000 7fb7b9eda640 1 -- 172.21.15.12:0/1797903103 learned_addr learned my addr 172.21.15.12:0/1797903103 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:46:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.611+0000 7fb7b8ed8640 1 -- 172.21.15.12:0/1797903103 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532917791 0 0) 0x7fb7b4071be0 con 0x7fb7b414fd50 2023-10-30T12:46:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.611+0000 7fb7b8ed8640 1 -- 172.21.15.12:0/1797903103 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7a0003450 con 0x7fb7b414fd50 2023-10-30T12:46:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.611+0000 7fb7ba6db640 1 -- 172.21.15.12:0/1797903103 >> v2:172.21.15.154:3300/0 conn(0x7fb7b40cb550 msgr2=0x7fb7b40cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.611+0000 7fb7ba6db640 1 --2- 172.21.15.12:0/1797903103 >> v2:172.21.15.154:3300/0 conn(0x7fb7b40cb550 0x7fb7b40cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:27.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.611+0000 7fb7ba6db640 1 -- 172.21.15.12:0/1797903103 >> v1:172.21.15.154:6789/0 conn(0x7fb7b414fd50 legacy=0x7fb7b40c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:27.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.611+0000 7fb7ba6db640 1 -- 172.21.15.12:0/1797903103 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b4156660 con 0x7fb7b4071250 2023-10-30T12:46:27.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.611+0000 7fb7b96d9640 1 --2- 172.21.15.12:0/1797903103 >> v2:172.21.15.154:3300/0 conn(0x7fb7b40cb550 0x7fb7b40cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:46:27.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.611+0000 7fb7ba6db640 1 --2- 172.21.15.12:0/1797903103 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7b4071250 0x7fb7b40716a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009d90 tx=0x7fb7ac01c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88404e35ca9e5344 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:27.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.611+0000 7fb7b8ed8640 1 -- 172.21.15.12:0/1797903103 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac0a5920 con 0x7fb7b4071250 2023-10-30T12:46:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.611+0000 7fb7b8ed8640 1 -- 172.21.15.12:0/1797903103 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ac0a5a80 con 0x7fb7b4071250 2023-10-30T12:46:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.611+0000 7fb7b8ed8640 1 -- 172.21.15.12:0/1797903103 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac0a5e10 con 0x7fb7b4071250 2023-10-30T12:46:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 -- 172.21.15.12:0/1797903103 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7b4071250 msgr2=0x7fb7b40716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 --2- 172.21.15.12:0/1797903103 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7b4071250 0x7fb7b40716a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009d90 tx=0x7fb7ac01c920 comp rx=0 tx=0).stop 2023-10-30T12:46:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 -- 172.21.15.12:0/1797903103 shutdown_connections 2023-10-30T12:46:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 --2- 172.21.15.12:0/1797903103 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7b4071250 0x7fb7b40716a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 --2- 172.21.15.12:0/1797903103 >> v2:172.21.15.154:3300/0 conn(0x7fb7b40cb550 0x7fb7b40cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 -- 172.21.15.12:0/1797903103 >> 172.21.15.12:0/1797903103 conn(0x7fb7b406c6e0 msgr2=0x7fb7b4069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 -- 172.21.15.12:0/1797903103 shutdown_connections 2023-10-30T12:46:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 -- 172.21.15.12:0/1797903103 wait complete. 2023-10-30T12:46:27.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 Processor -- start 2023-10-30T12:46:27.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 -- start start 2023-10-30T12:46:27.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7b4071250 0x7fb7b4142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7b40cb550 0x7fb7b41445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b414fd50 0x7fb7b4144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b41ee510 con 0x7fb7b414fd50 2023-10-30T12:46:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b41ed960 con 0x7fb7b40cb550 2023-10-30T12:46:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7bc13e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b41f1050 con 0x7fb7b4071250 2023-10-30T12:46:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7ba6db640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b414fd50 0x7fb7b4144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7ba6db640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b414fd50 0x7fb7b4144b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38378/0 (socket says 172.21.15.12:38378) 2023-10-30T12:46:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7ba6db640 1 -- 172.21.15.12:0/227861910 learned_addr learned my addr 172.21.15.12:0/227861910 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7b9eda640 1 --2- 172.21.15.12:0/227861910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7b4071250 0x7fb7b4142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.615+0000 7fb7b96d9640 1 --2- 172.21.15.12:0/227861910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7b40cb550 0x7fb7b41445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.619+0000 7fb7ba6db640 1 -- 172.21.15.12:0/227861910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7b4071250 msgr2=0x7fb7b4142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.619+0000 7fb7ba6db640 1 --2- 172.21.15.12:0/227861910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7b4071250 0x7fb7b4142470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.619+0000 7fb7ba6db640 1 -- 172.21.15.12:0/227861910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7b40cb550 msgr2=0x7fb7b41445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.619+0000 7fb7ba6db640 1 --2- 172.21.15.12:0/227861910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7b40cb550 0x7fb7b41445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.619+0000 7fb7ba6db640 1 -- 172.21.15.12:0/227861910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a4009420 con 0x7fb7b414fd50 2023-10-30T12:46:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.619+0000 7fb7ba6db640 1 --2- 172.21.15.12:0/227861910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b414fd50 0x7fb7b4144b00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009d90 tx=0x7fb7ac0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:27.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.619+0000 7fb7aa7fc640 1 -- 172.21.15.12:0/227861910 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac0aecb0 con 0x7fb7b414fd50 2023-10-30T12:46:27.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.619+0000 7fb7aa7fc640 1 -- 172.21.15.12:0/227861910 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ac0aee10 con 0x7fb7b414fd50 2023-10-30T12:46:27.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.619+0000 7fb7bc13e640 1 -- 172.21.15.12:0/227861910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ac009a70 con 0x7fb7b414fd50 2023-10-30T12:46:27.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.619+0000 7fb7aa7fc640 1 -- 172.21.15.12:0/227861910 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac0b7650 con 0x7fb7b414fd50 2023-10-30T12:46:27.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.619+0000 7fb7bc13e640 1 -- 172.21.15.12:0/227861910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b41edec0 con 0x7fb7b414fd50 2023-10-30T12:46:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.619+0000 7fb7bc13e640 1 -- 172.21.15.12:0/227861910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b41ee680 con 0x7fb7b414fd50 2023-10-30T12:46:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.619+0000 7fb7aa7fc640 1 -- 172.21.15.12:0/227861910 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb7ac0b4050 con 0x7fb7b414fd50 2023-10-30T12:46:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.619+0000 7fb7aa7fc640 1 --2- 172.21.15.12:0/227861910 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7980664a0 0x7fb798068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.623+0000 7fb7aa7fc640 1 -- 172.21.15.12:0/227861910 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fb7ac11f0c0 con 0x7fb7b414fd50 2023-10-30T12:46:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.623+0000 7fb7b9eda640 1 --2- 172.21.15.12:0/227861910 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7980664a0 0x7fb798068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:27.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.623+0000 7fb7b9eda640 1 --2- 172.21.15.12:0/227861910 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7980664a0 0x7fb798068960 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0007fd0 tx=0x7fb7b0000bd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:27.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.627+0000 7fb7aa7fc640 1 -- 172.21.15.12:0/227861910 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb7ac0ec2f0 con 0x7fb7b414fd50 2023-10-30T12:46:27.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.819+0000 7fb7bc13e640 1 -- 172.21.15.12:0/227861910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7b404e170 con 0x7fb7b414fd50 2023-10-30T12:46:27.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.819+0000 7fb7aa7fc640 1 -- 172.21.15.12:0/227861910 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6538 (secure 0 0 0) 0x7fb7ac0b2020 con 0x7fb7b414fd50 2023-10-30T12:46:27.824 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:27.824 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:18.260197+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:27.825 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 56 2023-10-30T12:46:27.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.819+0000 7fb7bc13e640 1 -- 172.21.15.12:0/227861910 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7980664a0 msgr2=0x7fb798068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:27.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.819+0000 7fb7bc13e640 1 --2- 172.21.15.12:0/227861910 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7980664a0 0x7fb798068960 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0007fd0 tx=0x7fb7b0000bd0 comp rx=0 tx=0).stop 2023-10-30T12:46:27.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.819+0000 7fb7bc13e640 1 -- 172.21.15.12:0/227861910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b414fd50 msgr2=0x7fb7b4144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:27.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.819+0000 7fb7bc13e640 1 --2- 172.21.15.12:0/227861910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b414fd50 0x7fb7b4144b00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009d90 tx=0x7fb7ac0196c0 comp rx=0 tx=0).stop 2023-10-30T12:46:27.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.823+0000 7fb7bc13e640 1 -- 172.21.15.12:0/227861910 shutdown_connections 2023-10-30T12:46:27.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.823+0000 7fb7bc13e640 1 --2- 172.21.15.12:0/227861910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7b414fd50 0x7fb7b4144b00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:27.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.823+0000 7fb7bc13e640 1 --2- 172.21.15.12:0/227861910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7b40cb550 0x7fb7b41445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:27.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.823+0000 7fb7bc13e640 1 --2- 172.21.15.12:0/227861910 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7980664a0 0x7fb798068960 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:27.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.823+0000 7fb7bc13e640 1 --2- 172.21.15.12:0/227861910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7b4071250 0x7fb7b4142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:27.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.823+0000 7fb7bc13e640 1 -- 172.21.15.12:0/227861910 >> 172.21.15.12:0/227861910 conn(0x7fb7b406c6e0 msgr2=0x7fb7b4071630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:27.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.823+0000 7fb7bc13e640 1 -- 172.21.15.12:0/227861910 shutdown_connections 2023-10-30T12:46:27.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:27.823+0000 7fb7bc13e640 1 -- 172.21.15.12:0/227861910 wait complete. 2023-10-30T12:46:27.845 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:46:29.846 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:30.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.995+0000 7f5c2146b640 1 Processor -- start 2023-10-30T12:46:30.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.995+0000 7f5c2146b640 1 -- start start 2023-10-30T12:46:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c2146b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c1c0c6b80 0x7f5c1c0bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c2146b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1c0c0a50 con 0x7f5c1c0c6f60 2023-10-30T12:46:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c2146b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1c06f6e0 con 0x7f5c1c13f4a0 2023-10-30T12:46:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c2146b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1c06f9c0 con 0x7f5c1c0c6b80 2023-10-30T12:46:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c13fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5c1c0c6f60 0x7f5c1c13ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50194/0 (socket says 172.21.15.12:50194) 2023-10-30T12:46:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c13fff640 1 -- 172.21.15.12:0/3634615965 learned_addr learned my addr 172.21.15.12:0/3634615965 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:46:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1affd640 1 --2- 172.21.15.12:0/3634615965 >> v2:172.21.15.154:3300/0 conn(0x7f5c1c0c6b80 0x7f5c1c0bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112729390 0 0) 0x7f5c1c0c0a50 con 0x7f5c1c0c6f60 2023-10-30T12:46:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c08003530 con 0x7f5c1c0c6f60 2023-10-30T12:46:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136710162 0 0) 0x7f5c1c06f6e0 con 0x7f5c1c13f4a0 2023-10-30T12:46:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c1c0c0a50 con 0x7f5c1c13f4a0 2023-10-30T12:46:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1387989317 0 0) 0x7f5c08003530 con 0x7f5c1c0c6f60 2023-10-30T12:46:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 >> v2:172.21.15.154:3300/0 conn(0x7f5c1c0c6b80 msgr2=0x7f5c1c0bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 --2- 172.21.15.12:0/3634615965 >> v2:172.21.15.154:3300/0 conn(0x7f5c1c0c6b80 0x7f5c1c0bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 >> v1:172.21.15.154:6789/0 conn(0x7f5c1c13f4a0 legacy=0x7f5c1c1566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c06fca0 con 0x7f5c1c0c6f60 2023-10-30T12:46:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f5c04001680 con 0x7f5c1c0c6f60 2023-10-30T12:46:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1affd640 1 --2- 172.21.15.12:0/3634615965 >> v2:172.21.15.154:3300/0 conn(0x7f5c1c0c6b80 0x7f5c1c0bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:46:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 >> v1:172.21.15.12:6789/0 conn(0x7f5c1c0c6f60 legacy=0x7f5c1c13ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 --2- 172.21.15.12:0/3634615965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c08003b10 0x7f5c08003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 --2- 172.21.15.12:0/3634615965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c080047d0 0x7f5c08004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 --2- 172.21.15.12:0/3634615965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c08005190 0x7f5c0800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c06f9c0 con 0x7f5c08003b10 2023-10-30T12:46:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0800c010 con 0x7f5c080047d0 2023-10-30T12:46:30.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0800c1d0 con 0x7f5c08005190 2023-10-30T12:46:30.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1affd640 1 --2- 172.21.15.12:0/3634615965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c08003b10 0x7f5c08003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:30.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c13fff640 1 --2- 172.21.15.12:0/3634615965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c080047d0 0x7f5c08004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:30.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1b7fe640 1 --2- 172.21.15.12:0/3634615965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c08005190 0x7f5c0800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:30.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1affd640 1 -- 172.21.15.12:0/3634615965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c08005190 msgr2=0x7f5c0800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:30.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1affd640 1 --2- 172.21.15.12:0/3634615965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c08005190 0x7f5c0800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:30.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1affd640 1 -- 172.21.15.12:0/3634615965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c080047d0 msgr2=0x7f5c08004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:30.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1affd640 1 --2- 172.21.15.12:0/3634615965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c080047d0 0x7f5c08004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:30.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1affd640 1 -- 172.21.15.12:0/3634615965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c08003530 con 0x7f5c08003b10 2023-10-30T12:46:30.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1b7fe640 1 --2- 172.21.15.12:0/3634615965 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c08005190 0x7f5c0800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:46:30.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:29.999+0000 7f5c1b7fe640 1 -- 172.21.15.12:0/3634615965 reap_dead start 2023-10-30T12:46:30.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.003+0000 7f5c1affd640 1 --2- 172.21.15.12:0/3634615965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c08003b10 0x7f5c08003f10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009cb0 tx=0x7f5c0c01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa10fb1ef9a04eb server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:30.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.003+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0a57a0 con 0x7f5c08003b10 2023-10-30T12:46:30.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.003+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c0a5900 con 0x7f5c08003b10 2023-10-30T12:46:30.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.003+0000 7f5c1a7fc640 1 -- 172.21.15.12:0/3634615965 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0a5cf0 con 0x7f5c08003b10 2023-10-30T12:46:30.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.003+0000 7f5c2146b640 1 -- 172.21.15.12:0/3634615965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c08003b10 msgr2=0x7f5c08003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:30.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.003+0000 7f5c2146b640 1 --2- 172.21.15.12:0/3634615965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c08003b10 0x7f5c08003f10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009cb0 tx=0x7f5c0c01cbf0 comp rx=0 tx=0).stop 2023-10-30T12:46:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.003+0000 7f5c2146b640 1 -- 172.21.15.12:0/3634615965 shutdown_connections 2023-10-30T12:46:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.003+0000 7f5c2146b640 1 --2- 172.21.15.12:0/3634615965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c08003b10 0x7f5c08003f10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.003+0000 7f5c2146b640 1 -- 172.21.15.12:0/3634615965 >> 172.21.15.12:0/3634615965 conn(0x7f5c1c0c7910 msgr2=0x7f5c1c0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.003+0000 7f5c2146b640 1 -- 172.21.15.12:0/3634615965 shutdown_connections 2023-10-30T12:46:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.003+0000 7f5c2146b640 1 -- 172.21.15.12:0/3634615965 wait complete. 2023-10-30T12:46:30.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c2146b640 1 Processor -- start 2023-10-30T12:46:30.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c2146b640 1 -- start start 2023-10-30T12:46:30.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c2146b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c08003b10 0x7f5c1c1530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:30.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c2146b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c1c15a020 0x7f5c1c139e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:30.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c2146b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c1c1ee160 0x7f5c1c1524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c2146b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c1f1fe0 con 0x7f5c1c1ee160 2023-10-30T12:46:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c2146b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c1f16d0 con 0x7f5c08003b10 2023-10-30T12:46:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c2146b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c15a560 con 0x7f5c1c15a020 2023-10-30T12:46:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c1b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c1c1ee160 0x7f5c1c1524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c1b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c1c1ee160 0x7f5c1c1524c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38400/0 (socket says 172.21.15.12:38400) 2023-10-30T12:46:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c1b7fe640 1 -- 172.21.15.12:0/4198237716 learned_addr learned my addr 172.21.15.12:0/4198237716 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c13fff640 1 --2- 172.21.15.12:0/4198237716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c1c15a020 0x7f5c1c139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c1affd640 1 --2- 172.21.15.12:0/4198237716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c08003b10 0x7f5c1c1530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c1b7fe640 1 -- 172.21.15.12:0/4198237716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c1c15a020 msgr2=0x7f5c1c139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c1b7fe640 1 --2- 172.21.15.12:0/4198237716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c1c15a020 0x7f5c1c139e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c1b7fe640 1 -- 172.21.15.12:0/4198237716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c08003b10 msgr2=0x7f5c1c1530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c1b7fe640 1 --2- 172.21.15.12:0/4198237716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c08003b10 0x7f5c1c1530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c1b7fe640 1 -- 172.21.15.12:0/4198237716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c15a840 con 0x7f5c1c1ee160 2023-10-30T12:46:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c1b7fe640 1 --2- 172.21.15.12:0/4198237716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c1c1ee160 0x7f5c1c1524c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0b1190 tx=0x7f5c1400dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c137fe640 1 -- 172.21.15.12:0/4198237716 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c14003bb0 con 0x7f5c1c1ee160 2023-10-30T12:46:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c137fe640 1 -- 172.21.15.12:0/4198237716 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c14003d30 con 0x7f5c1c1ee160 2023-10-30T12:46:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c137fe640 1 -- 172.21.15.12:0/4198237716 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c14003ea0 con 0x7f5c1c1ee160 2023-10-30T12:46:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c2146b640 1 -- 172.21.15.12:0/4198237716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c1f1920 con 0x7f5c1c1ee160 2023-10-30T12:46:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c2146b640 1 -- 172.21.15.12:0/4198237716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c13a370 con 0x7f5c1c1ee160 2023-10-30T12:46:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.007+0000 7f5c2146b640 1 -- 172.21.15.12:0/4198237716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1c04e170 con 0x7f5c1c1ee160 2023-10-30T12:46:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.011+0000 7f5c137fe640 1 -- 172.21.15.12:0/4198237716 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5c14005070 con 0x7f5c1c1ee160 2023-10-30T12:46:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.011+0000 7f5c137fe640 1 --2- 172.21.15.12:0/4198237716 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5bfc0664d0 0x7f5bfc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.011+0000 7f5c137fe640 1 -- 172.21.15.12:0/4198237716 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f5c14082ff0 con 0x7f5c1c1ee160 2023-10-30T12:46:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.011+0000 7f5c1affd640 1 --2- 172.21.15.12:0/4198237716 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5bfc0664d0 0x7f5bfc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.011+0000 7f5c1affd640 1 --2- 172.21.15.12:0/4198237716 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5bfc0664d0 0x7f5bfc068990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0af000 tx=0x7f5c0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.015+0000 7f5c137fe640 1 -- 172.21.15.12:0/4198237716 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5c140502a0 con 0x7f5c1c1ee160 2023-10-30T12:46:30.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.203+0000 7f5c2146b640 1 -- 172.21.15.12:0/4198237716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c1c15ad00 con 0x7f5c1c1ee160 2023-10-30T12:46:30.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.203+0000 7f5c137fe640 1 -- 172.21.15.12:0/4198237716 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6538 (secure 0 0 0) 0x7f5c14018090 con 0x7f5c1c1ee160 2023-10-30T12:46:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:30.209 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:18.260197+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:30.211 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 56 2023-10-30T12:46:30.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.207+0000 7f5c2146b640 1 -- 172.21.15.12:0/4198237716 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5bfc0664d0 msgr2=0x7f5bfc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:30.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.207+0000 7f5c2146b640 1 --2- 172.21.15.12:0/4198237716 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5bfc0664d0 0x7f5bfc068990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0af000 tx=0x7f5c0c008040 comp rx=0 tx=0).stop 2023-10-30T12:46:30.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.207+0000 7f5c2146b640 1 -- 172.21.15.12:0/4198237716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c1c1ee160 msgr2=0x7f5c1c1524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:30.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.207+0000 7f5c2146b640 1 --2- 172.21.15.12:0/4198237716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c1c1ee160 0x7f5c1c1524c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0b1190 tx=0x7f5c1400dbf0 comp rx=0 tx=0).stop 2023-10-30T12:46:30.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.207+0000 7f5c2146b640 1 -- 172.21.15.12:0/4198237716 shutdown_connections 2023-10-30T12:46:30.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.207+0000 7f5c2146b640 1 --2- 172.21.15.12:0/4198237716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c1c1ee160 0x7f5c1c1524c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:30.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.207+0000 7f5c2146b640 1 --2- 172.21.15.12:0/4198237716 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5bfc0664d0 0x7f5bfc068990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:30.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.207+0000 7f5c2146b640 1 --2- 172.21.15.12:0/4198237716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c1c15a020 0x7f5c1c139e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:30.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.207+0000 7f5c2146b640 1 --2- 172.21.15.12:0/4198237716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c08003b10 0x7f5c1c1530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:30.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.207+0000 7f5c2146b640 1 -- 172.21.15.12:0/4198237716 >> 172.21.15.12:0/4198237716 conn(0x7f5c1c0c7910 msgr2=0x7f5c1c06c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:30.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.207+0000 7f5c2146b640 1 -- 172.21.15.12:0/4198237716 shutdown_connections 2023-10-30T12:46:30.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:30.207+0000 7f5c2146b640 1 -- 172.21.15.12:0/4198237716 wait complete. 2023-10-30T12:46:30.230 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:46:32.231 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.379+0000 7f6df689c640 1 Processor -- start 2023-10-30T12:46:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.379+0000 7f6df689c640 1 -- start start 2023-10-30T12:46:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.383+0000 7f6df689c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6df0153bf0 0x7f6df0071260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.383+0000 7f6df689c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6df00c5f60 0x7f6df00cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.383+0000 7f6df689c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df00700b0 con 0x7f6df00c6fe0 2023-10-30T12:46:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.383+0000 7f6df689c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df00c6810 con 0x7f6df0153bf0 2023-10-30T12:46:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.383+0000 7f6df689c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df0159900 con 0x7f6df00c5f60 2023-10-30T12:46:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.383+0000 7f6deffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6df0153bf0 0x7f6df0071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.383+0000 7f6deffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6df0153bf0 0x7f6df0071260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58784/0 (socket says 172.21.15.12:58784) 2023-10-30T12:46:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.383+0000 7f6deffff640 1 -- 172.21.15.12:0/3528623587 learned_addr learned my addr 172.21.15.12:0/3528623587 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.383+0000 7f6df4e39640 1 --2- 172.21.15.12:0/3528623587 >> v2:172.21.15.154:3300/0 conn(0x7f6df00c5f60 0x7f6df00cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.383+0000 7f6deffff640 1 -- 172.21.15.12:0/3528623587 >> v2:172.21.15.154:3300/0 conn(0x7f6df00c5f60 msgr2=0x7f6df00cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.383+0000 7f6deffff640 1 --2- 172.21.15.12:0/3528623587 >> v2:172.21.15.154:3300/0 conn(0x7f6df00c5f60 0x7f6df00cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:32.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.383+0000 7f6deffff640 1 -- 172.21.15.12:0/3528623587 >> v1:172.21.15.154:6789/0 conn(0x7f6df00c6fe0 legacy=0x7f6df00c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:32.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.383+0000 7f6deffff640 1 -- 172.21.15.12:0/3528623587 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df00c6530 con 0x7f6df0153bf0 2023-10-30T12:46:32.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.387+0000 7f6deffff640 1 --2- 172.21.15.12:0/3528623587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df0153bf0 0x7f6df0071260 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6df006ce90 tx=0x7f6de001ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a45c34a6b1a41b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:32.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.387+0000 7f6deeffd640 1 -- 172.21.15.12:0/3528623587 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de00a89b0 con 0x7f6df0153bf0 2023-10-30T12:46:32.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.387+0000 7f6deeffd640 1 -- 172.21.15.12:0/3528623587 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de00a8b10 con 0x7f6df0153bf0 2023-10-30T12:46:32.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.387+0000 7f6deeffd640 1 -- 172.21.15.12:0/3528623587 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de00a8ea0 con 0x7f6df0153bf0 2023-10-30T12:46:32.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.387+0000 7f6df689c640 1 -- 172.21.15.12:0/3528623587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df0153bf0 msgr2=0x7f6df0071260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:32.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.387+0000 7f6df689c640 1 --2- 172.21.15.12:0/3528623587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df0153bf0 0x7f6df0071260 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6df006ce90 tx=0x7f6de001ecb0 comp rx=0 tx=0).stop 2023-10-30T12:46:32.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.387+0000 7f6df689c640 1 -- 172.21.15.12:0/3528623587 shutdown_connections 2023-10-30T12:46:32.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.387+0000 7f6df689c640 1 --2- 172.21.15.12:0/3528623587 >> v2:172.21.15.154:3300/0 conn(0x7f6df00c5f60 0x7f6df00cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:32.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.387+0000 7f6df689c640 1 --2- 172.21.15.12:0/3528623587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df0153bf0 0x7f6df0071260 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:32.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.387+0000 7f6df689c640 1 -- 172.21.15.12:0/3528623587 >> 172.21.15.12:0/3528623587 conn(0x7f6df006c6e0 msgr2=0x7f6df0069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:32.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.387+0000 7f6df689c640 1 -- 172.21.15.12:0/3528623587 shutdown_connections 2023-10-30T12:46:32.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.387+0000 7f6df689c640 1 -- 172.21.15.12:0/3528623587 wait complete. 2023-10-30T12:46:32.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.387+0000 7f6df689c640 1 Processor -- start 2023-10-30T12:46:32.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.387+0000 7f6df689c640 1 -- start start 2023-10-30T12:46:32.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df689c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6df00c5f60 0x7f6df00c2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:32.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df689c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6df00c6fe0 0x7f6df014e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:32.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df689c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df0153bf0 0x7f6df0155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:32.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df689c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df01efb40 con 0x7f6df0153bf0 2023-10-30T12:46:32.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df689c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df0157130 con 0x7f6df00c6fe0 2023-10-30T12:46:32.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df689c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df0157410 con 0x7f6df00c5f60 2023-10-30T12:46:32.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df4e39640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df0153bf0 0x7f6df0155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:32.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df4e39640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df0153bf0 0x7f6df0155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58798/0 (socket says 172.21.15.12:58798) 2023-10-30T12:46:32.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df4e39640 1 -- 172.21.15.12:0/628922791 learned_addr learned my addr 172.21.15.12:0/628922791 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:32.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6deffff640 1 --2- 172.21.15.12:0/628922791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6df00c5f60 0x7f6df00c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:32.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6def7fe640 1 --2- 172.21.15.12:0/628922791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6df00c6fe0 0x7f6df014e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:32.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df4e39640 1 -- 172.21.15.12:0/628922791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6df00c5f60 msgr2=0x7f6df00c2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:32.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df4e39640 1 --2- 172.21.15.12:0/628922791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6df00c5f60 0x7f6df00c2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:32.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df4e39640 1 -- 172.21.15.12:0/628922791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6df00c6fe0 msgr2=0x7f6df014e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:32.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df4e39640 1 --2- 172.21.15.12:0/628922791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6df00c6fe0 0x7f6df014e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:32.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df4e39640 1 -- 172.21.15.12:0/628922791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de00094d0 con 0x7f6df0153bf0 2023-10-30T12:46:32.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df4e39640 1 --2- 172.21.15.12:0/628922791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df0153bf0 0x7f6df0155b40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6de8007e40 tx=0x7f6de800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:32.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6decff9640 1 -- 172.21.15.12:0/628922791 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8014620 con 0x7f6df0153bf0 2023-10-30T12:46:32.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6decff9640 1 -- 172.21.15.12:0/628922791 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de800ee90 con 0x7f6df0153bf0 2023-10-30T12:46:32.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df689c640 1 -- 172.21.15.12:0/628922791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df0155340 con 0x7f6df0153bf0 2023-10-30T12:46:32.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6decff9640 1 -- 172.21.15.12:0/628922791 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de80149e0 con 0x7f6df0153bf0 2023-10-30T12:46:32.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df689c640 1 -- 172.21.15.12:0/628922791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df0159080 con 0x7f6df0153bf0 2023-10-30T12:46:32.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6df689c640 1 -- 172.21.15.12:0/628922791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df0155860 con 0x7f6df0153bf0 2023-10-30T12:46:32.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.391+0000 7f6decff9640 1 -- 172.21.15.12:0/628922791 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6de8014b40 con 0x7f6df0153bf0 2023-10-30T12:46:32.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.395+0000 7f6decff9640 1 --2- 172.21.15.12:0/628922791 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6ddc0664d0 0x7f6ddc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:32.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.395+0000 7f6decff9640 1 -- 172.21.15.12:0/628922791 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f6de80848a0 con 0x7f6df0153bf0 2023-10-30T12:46:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.395+0000 7f6deffff640 1 --2- 172.21.15.12:0/628922791 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6ddc0664d0 0x7f6ddc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.395+0000 7f6deffff640 1 --2- 172.21.15.12:0/628922791 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6ddc0664d0 0x7f6ddc068990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6de00b2000 tx=0x7f6de0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.399+0000 7f6decff9640 1 -- 172.21.15.12:0/628922791 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6df0155860 con 0x7f6df0153bf0 2023-10-30T12:46:32.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.591+0000 7f6df689c640 1 -- 172.21.15.12:0/628922791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6df0155860 con 0x7f6df0153bf0 2023-10-30T12:46:32.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.591+0000 7f6decff9640 1 -- 172.21.15.12:0/628922791 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6538 (secure 0 0 0) 0x7f6df0155860 con 0x7f6df0153bf0 2023-10-30T12:46:32.595 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:32.596 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:18.260197+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:32.597 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 56 2023-10-30T12:46:32.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.591+0000 7f6df689c640 1 -- 172.21.15.12:0/628922791 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6ddc0664d0 msgr2=0x7f6ddc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:32.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.591+0000 7f6df689c640 1 --2- 172.21.15.12:0/628922791 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6ddc0664d0 0x7f6ddc068990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6de00b2000 tx=0x7f6de0009040 comp rx=0 tx=0).stop 2023-10-30T12:46:32.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.591+0000 7f6df689c640 1 -- 172.21.15.12:0/628922791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df0153bf0 msgr2=0x7f6df0155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:32.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.591+0000 7f6df689c640 1 --2- 172.21.15.12:0/628922791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df0153bf0 0x7f6df0155b40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6de8007e40 tx=0x7f6de800e4b0 comp rx=0 tx=0).stop 2023-10-30T12:46:32.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.591+0000 7f6df689c640 1 -- 172.21.15.12:0/628922791 shutdown_connections 2023-10-30T12:46:32.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.591+0000 7f6df689c640 1 --2- 172.21.15.12:0/628922791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df0153bf0 0x7f6df0155b40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:32.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.591+0000 7f6df689c640 1 --2- 172.21.15.12:0/628922791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6df00c6fe0 0x7f6df014e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:32.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.591+0000 7f6df689c640 1 --2- 172.21.15.12:0/628922791 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6ddc0664d0 0x7f6ddc068990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:32.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.591+0000 7f6df689c640 1 --2- 172.21.15.12:0/628922791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6df00c5f60 0x7f6df00c2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:32.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.591+0000 7f6df689c640 1 -- 172.21.15.12:0/628922791 >> 172.21.15.12:0/628922791 conn(0x7f6df006c6e0 msgr2=0x7f6df00c5160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:32.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.591+0000 7f6df689c640 1 -- 172.21.15.12:0/628922791 shutdown_connections 2023-10-30T12:46:32.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:32.591+0000 7f6df689c640 1 -- 172.21.15.12:0/628922791 wait complete. 2023-10-30T12:46:32.616 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:46:34.617 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:34.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.767+0000 7fd621980640 1 Processor -- start 2023-10-30T12:46:34.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.771+0000 7fd621980640 1 -- start start 2023-10-30T12:46:34.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.771+0000 7fd621980640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd61c15d6f0 0x7fd61c15a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:34.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.771+0000 7fd621980640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd61c0cf160 0x7fd61c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:34.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.771+0000 7fd621980640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61c0cb800 con 0x7fd61c1538a0 2023-10-30T12:46:34.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.771+0000 7fd621980640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61c0cb330 con 0x7fd61c15d6f0 2023-10-30T12:46:34.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.771+0000 7fd621980640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c0cad70 con 0x7fd61c0cf160 2023-10-30T12:46:34.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.771+0000 7fd61b7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd61c0cf160 0x7fd61c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:34.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.771+0000 7fd613fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd61c15d6f0 0x7fd61c15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:34.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.771+0000 7fd61b7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd61c0cf160 0x7fd61c0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:51602/0 (socket says 172.21.15.12:51602) 2023-10-30T12:46:34.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.771+0000 7fd61b7fe640 1 -- 172.21.15.12:0/9460497 learned_addr learned my addr 172.21.15.12:0/9460497 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:34.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.771+0000 7fd61affd640 1 --1- 172.21.15.12:0/9460497 >> v1:172.21.15.154:6789/0 conn(0x7fd61c1538a0 0x7fd61c0ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:60396/0 (socket says 172.21.15.12:60396) 2023-10-30T12:46:34.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.771+0000 7fd61b7fe640 1 -- 172.21.15.12:0/9460497 >> v2:172.21.15.154:3300/0 conn(0x7fd61c15d6f0 msgr2=0x7fd61c15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:34.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.771+0000 7fd61b7fe640 1 --2- 172.21.15.12:0/9460497 >> v2:172.21.15.154:3300/0 conn(0x7fd61c15d6f0 0x7fd61c15a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:34.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.771+0000 7fd61b7fe640 1 -- 172.21.15.12:0/9460497 >> v1:172.21.15.154:6789/0 conn(0x7fd61c1538a0 legacy=0x7fd61c0ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:34.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.771+0000 7fd61b7fe640 1 -- 172.21.15.12:0/9460497 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c0cb050 con 0x7fd61c0cf160 2023-10-30T12:46:34.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.775+0000 7fd61b7fe640 1 --2- 172.21.15.12:0/9460497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd61c0cf160 0x7fd61c0cf590 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd614009e50 tx=0x7fd61401f1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=556cbc9ed5f5412f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:34.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.775+0000 7fd61a7fc640 1 -- 172.21.15.12:0/9460497 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140af400 con 0x7fd61c0cf160 2023-10-30T12:46:34.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.775+0000 7fd61a7fc640 1 -- 172.21.15.12:0/9460497 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6140ab050 con 0x7fd61c0cf160 2023-10-30T12:46:34.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.775+0000 7fd61a7fc640 1 -- 172.21.15.12:0/9460497 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140af780 con 0x7fd61c0cf160 2023-10-30T12:46:34.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.775+0000 7fd621980640 1 -- 172.21.15.12:0/9460497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd61c0cf160 msgr2=0x7fd61c0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:34.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.775+0000 7fd621980640 1 --2- 172.21.15.12:0/9460497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd61c0cf160 0x7fd61c0cf590 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd614009e50 tx=0x7fd61401f1f0 comp rx=0 tx=0).stop 2023-10-30T12:46:34.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.775+0000 7fd621980640 1 -- 172.21.15.12:0/9460497 shutdown_connections 2023-10-30T12:46:34.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.775+0000 7fd621980640 1 --2- 172.21.15.12:0/9460497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd61c0cf160 0x7fd61c0cf590 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:34.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.775+0000 7fd621980640 1 --2- 172.21.15.12:0/9460497 >> v2:172.21.15.154:3300/0 conn(0x7fd61c15d6f0 0x7fd61c15a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:34.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.775+0000 7fd621980640 1 -- 172.21.15.12:0/9460497 >> 172.21.15.12:0/9460497 conn(0x7fd61c06bfe0 msgr2=0x7fd61c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:34.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.775+0000 7fd621980640 1 -- 172.21.15.12:0/9460497 shutdown_connections 2023-10-30T12:46:34.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.775+0000 7fd621980640 1 -- 172.21.15.12:0/9460497 wait complete. 2023-10-30T12:46:34.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd621980640 1 Processor -- start 2023-10-30T12:46:34.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd621980640 1 -- start start 2023-10-30T12:46:34.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd621980640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd61c0cf160 0x7fd61c14e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:34.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd621980640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd61c1538a0 0x7fd61c0b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:34.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd621980640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61c15d6f0 0x7fd61c0cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:34.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd621980640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c1efd70 con 0x7fd61c1538a0 2023-10-30T12:46:34.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd621980640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c1f0260 con 0x7fd61c15d6f0 2023-10-30T12:46:34.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd621980640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c0ce7b0 con 0x7fd61c0cf160 2023-10-30T12:46:34.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd613fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd61c1538a0 0x7fd61c0b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:34.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd61affd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd61c0cf160 0x7fd61c14e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:34.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd613fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd61c1538a0 0x7fd61c0b49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58804/0 (socket says 172.21.15.12:58804) 2023-10-30T12:46:34.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd613fff640 1 -- 172.21.15.12:0/2374652072 learned_addr learned my addr 172.21.15.12:0/2374652072 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:34.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd61b7fe640 1 --2- 172.21.15.12:0/2374652072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61c15d6f0 0x7fd61c0cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:34.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd613fff640 1 -- 172.21.15.12:0/2374652072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd61c0cf160 msgr2=0x7fd61c14e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:34.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd613fff640 1 --2- 172.21.15.12:0/2374652072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd61c0cf160 0x7fd61c14e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:34.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd613fff640 1 -- 172.21.15.12:0/2374652072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61c15d6f0 msgr2=0x7fd61c0cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:34.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd613fff640 1 --2- 172.21.15.12:0/2374652072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61c15d6f0 0x7fd61c0cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:34.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd613fff640 1 -- 172.21.15.12:0/2374652072 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd614009b30 con 0x7fd61c1538a0 2023-10-30T12:46:34.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd613fff640 1 --2- 172.21.15.12:0/2374652072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd61c1538a0 0x7fd61c0b49f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd60400e870 tx=0x7fd60400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:34.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd618ff9640 1 -- 172.21.15.12:0/2374652072 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd604009510 con 0x7fd61c1538a0 2023-10-30T12:46:34.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd618ff9640 1 -- 172.21.15.12:0/2374652072 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60400eec0 con 0x7fd61c1538a0 2023-10-30T12:46:34.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd618ff9640 1 -- 172.21.15.12:0/2374652072 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60400cb40 con 0x7fd61c1538a0 2023-10-30T12:46:34.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd621980640 1 -- 172.21.15.12:0/2374652072 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61c14e760 con 0x7fd61c1538a0 2023-10-30T12:46:34.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd621980640 1 -- 172.21.15.12:0/2374652072 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61c0cea50 con 0x7fd61c1538a0 2023-10-30T12:46:34.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.779+0000 7fd621980640 1 -- 172.21.15.12:0/2374652072 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61c0b2a50 con 0x7fd61c1538a0 2023-10-30T12:46:34.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.783+0000 7fd618ff9640 1 -- 172.21.15.12:0/2374652072 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd60400cca0 con 0x7fd61c1538a0 2023-10-30T12:46:34.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.783+0000 7fd618ff9640 1 --2- 172.21.15.12:0/2374652072 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd5fc0664d0 0x7fd5fc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:34.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.783+0000 7fd618ff9640 1 -- 172.21.15.12:0/2374652072 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fd604084080 con 0x7fd61c1538a0 2023-10-30T12:46:34.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.783+0000 7fd61affd640 1 --2- 172.21.15.12:0/2374652072 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd5fc0664d0 0x7fd5fc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:34.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.783+0000 7fd61affd640 1 --2- 172.21.15.12:0/2374652072 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd5fc0664d0 0x7fd5fc068990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd60c000f70 tx=0x7fd60c007e50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:34.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.787+0000 7fd618ff9640 1 -- 172.21.15.12:0/2374652072 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd6040b5090 con 0x7fd61c1538a0 2023-10-30T12:46:34.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.975+0000 7fd621980640 1 -- 172.21.15.12:0/2374652072 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd61c04e170 con 0x7fd61c1538a0 2023-10-30T12:46:34.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.975+0000 7fd618ff9640 1 -- 172.21.15.12:0/2374652072 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6538 (secure 0 0 0) 0x7fd6040513d0 con 0x7fd61c1538a0 2023-10-30T12:46:34.980 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:34.980 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:18.260197+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:34.981 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 56 2023-10-30T12:46:34.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.975+0000 7fd621980640 1 -- 172.21.15.12:0/2374652072 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd5fc0664d0 msgr2=0x7fd5fc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:34.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.975+0000 7fd621980640 1 --2- 172.21.15.12:0/2374652072 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd5fc0664d0 0x7fd5fc068990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd60c000f70 tx=0x7fd60c007e50 comp rx=0 tx=0).stop 2023-10-30T12:46:34.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.975+0000 7fd621980640 1 -- 172.21.15.12:0/2374652072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd61c1538a0 msgr2=0x7fd61c0b49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:34.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.975+0000 7fd621980640 1 --2- 172.21.15.12:0/2374652072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd61c1538a0 0x7fd61c0b49f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd60400e870 tx=0x7fd60400ec70 comp rx=0 tx=0).stop 2023-10-30T12:46:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.975+0000 7fd621980640 1 -- 172.21.15.12:0/2374652072 shutdown_connections 2023-10-30T12:46:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.975+0000 7fd621980640 1 --2- 172.21.15.12:0/2374652072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61c15d6f0 0x7fd61c0cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.975+0000 7fd621980640 1 --2- 172.21.15.12:0/2374652072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd61c1538a0 0x7fd61c0b49f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.975+0000 7fd621980640 1 --2- 172.21.15.12:0/2374652072 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd5fc0664d0 0x7fd5fc068990 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.975+0000 7fd621980640 1 --2- 172.21.15.12:0/2374652072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd61c0cf160 0x7fd61c14e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.975+0000 7fd621980640 1 -- 172.21.15.12:0/2374652072 >> 172.21.15.12:0/2374652072 conn(0x7fd61c06bfe0 msgr2=0x7fd61c0c48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.975+0000 7fd621980640 1 -- 172.21.15.12:0/2374652072 shutdown_connections 2023-10-30T12:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:34.975+0000 7fd621980640 1 -- 172.21.15.12:0/2374652072 wait complete. 2023-10-30T12:46:35.001 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:46:35.529 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:46:35.533 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:46:37.002 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:37.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.151+0000 7fbf3ec54640 1 Processor -- start 2023-10-30T12:46:37.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.151+0000 7fbf3ec54640 1 -- start start 2023-10-30T12:46:37.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.155+0000 7fbf3ec54640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbf381539a0 0x7fbf3815a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:37.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.155+0000 7fbf3ec54640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf38069e90 0x7fbf3806a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:37.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.155+0000 7fbf3ec54640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3815a8c0 con 0x7fbf38070a00 2023-10-30T12:46:37.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.155+0000 7fbf3ec54640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3806fbb0 con 0x7fbf381539a0 2023-10-30T12:46:37.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.155+0000 7fbf3ec54640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf38153d80 con 0x7fbf38069e90 2023-10-30T12:46:37.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.155+0000 7fbf3c9f0640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbf381539a0 0x7fbf3815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:37.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.155+0000 7fbf37fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbf38070a00 0x7fbf38070e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45184/0 (socket says 172.21.15.12:45184) 2023-10-30T12:46:37.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.155+0000 7fbf37fff640 1 -- 172.21.15.12:0/2601735853 learned_addr learned my addr 172.21.15.12:0/2601735853 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:46:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.155+0000 7fbf3d1f1640 1 --2- 172.21.15.12:0/2601735853 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf38069e90 0x7fbf3806a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.155+0000 7fbf3c9f0640 1 -- 172.21.15.12:0/2601735853 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf38069e90 msgr2=0x7fbf3806a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.155+0000 7fbf3c9f0640 1 --2- 172.21.15.12:0/2601735853 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf38069e90 0x7fbf3806a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.155+0000 7fbf3c9f0640 1 -- 172.21.15.12:0/2601735853 >> v1:172.21.15.12:6789/0 conn(0x7fbf38070a00 legacy=0x7fbf38070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:37.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.155+0000 7fbf3c9f0640 1 -- 172.21.15.12:0/2601735853 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf380c5c50 con 0x7fbf381539a0 2023-10-30T12:46:37.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.159+0000 7fbf3c9f0640 1 --2- 172.21.15.12:0/2601735853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf381539a0 0x7fbf3815a380 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0097c0 tx=0x7fbf2c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5562580d017d78f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:37.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.159+0000 7fbf377fe640 1 -- 172.21.15.12:0/2601735853 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c0ab600 con 0x7fbf381539a0 2023-10-30T12:46:37.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.159+0000 7fbf377fe640 1 -- 172.21.15.12:0/2601735853 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2c0a7050 con 0x7fbf381539a0 2023-10-30T12:46:37.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.159+0000 7fbf377fe640 1 -- 172.21.15.12:0/2601735853 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c0ab980 con 0x7fbf381539a0 2023-10-30T12:46:37.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.159+0000 7fbf3ec54640 1 -- 172.21.15.12:0/2601735853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf381539a0 msgr2=0x7fbf3815a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:37.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.159+0000 7fbf3ec54640 1 --2- 172.21.15.12:0/2601735853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf381539a0 0x7fbf3815a380 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0097c0 tx=0x7fbf2c01c3c0 comp rx=0 tx=0).stop 2023-10-30T12:46:37.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.159+0000 7fbf3ec54640 1 -- 172.21.15.12:0/2601735853 shutdown_connections 2023-10-30T12:46:37.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.159+0000 7fbf3ec54640 1 --2- 172.21.15.12:0/2601735853 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf38069e90 0x7fbf3806a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.159+0000 7fbf3ec54640 1 --2- 172.21.15.12:0/2601735853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf381539a0 0x7fbf3815a380 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.159+0000 7fbf3ec54640 1 -- 172.21.15.12:0/2601735853 >> 172.21.15.12:0/2601735853 conn(0x7fbf380c7910 msgr2=0x7fbf380c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.159+0000 7fbf3ec54640 1 -- 172.21.15.12:0/2601735853 shutdown_connections 2023-10-30T12:46:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.159+0000 7fbf3ec54640 1 -- 172.21.15.12:0/2601735853 wait complete. 2023-10-30T12:46:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.159+0000 7fbf3ec54640 1 Processor -- start 2023-10-30T12:46:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.159+0000 7fbf3ec54640 1 -- start start 2023-10-30T12:46:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3ec54640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf38069e90 0x7fbf380c1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3ec54640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf38070a00 0x7fbf38141750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3ec54640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf381539a0 0x7fbf38141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3ec54640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf380ca830 con 0x7fbf38069e90 2023-10-30T12:46:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3ec54640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf380c0b20 con 0x7fbf381539a0 2023-10-30T12:46:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3ec54640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf380caab0 con 0x7fbf38070a00 2023-10-30T12:46:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3c9f0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf38069e90 0x7fbf380c1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3c9f0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf38069e90 0x7fbf380c1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58822/0 (socket says 172.21.15.12:58822) 2023-10-30T12:46:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3c9f0640 1 -- 172.21.15.12:0/79948813 learned_addr learned my addr 172.21.15.12:0/79948813 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf37fff640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf38070a00 0x7fbf38141750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3d1f1640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf381539a0 0x7fbf38141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3c9f0640 1 -- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf38070a00 msgr2=0x7fbf38141750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3c9f0640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf38070a00 0x7fbf38141750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3c9f0640 1 -- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf381539a0 msgr2=0x7fbf38141c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3c9f0640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf381539a0 0x7fbf38141c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3c9f0640 1 -- 172.21.15.12:0/79948813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c009420 con 0x7fbf38069e90 2023-10-30T12:46:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf37fff640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf38070a00 0x7fbf38141750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:46:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3d1f1640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf381539a0 0x7fbf38141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:46:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3c9f0640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf38069e90 0x7fbf380c1610 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0ae040 tx=0x7fbf2c01ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf357fa640 1 -- 172.21.15.12:0/79948813 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c0b7650 con 0x7fbf38069e90 2023-10-30T12:46:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf357fa640 1 -- 172.21.15.12:0/79948813 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2c0abe90 con 0x7fbf38069e90 2023-10-30T12:46:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf357fa640 1 -- 172.21.15.12:0/79948813 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c0b7a10 con 0x7fbf38069e90 2023-10-30T12:46:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3ec54640 1 -- 172.21.15.12:0/79948813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf380c0dc0 con 0x7fbf38069e90 2023-10-30T12:46:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3ec54640 1 -- 172.21.15.12:0/79948813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf380c8c10 con 0x7fbf38069e90 2023-10-30T12:46:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf3ec54640 1 -- 172.21.15.12:0/79948813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3804e170 con 0x7fbf38069e90 2023-10-30T12:46:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.163+0000 7fbf357fa640 1 -- 172.21.15.12:0/79948813 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbf2c0b7b70 con 0x7fbf38069e90 2023-10-30T12:46:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.167+0000 7fbf357fa640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbf1c0664d0 0x7fbf1c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.167+0000 7fbf357fa640 1 -- 172.21.15.12:0/79948813 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fbf2c11ea10 con 0x7fbf38069e90 2023-10-30T12:46:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.167+0000 7fbf37fff640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbf1c0664d0 0x7fbf1c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.167+0000 7fbf37fff640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbf1c0664d0 0x7fbf1c068990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbf381578e0 tx=0x7fbf24009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:37.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.171+0000 7fbf357fa640 1 -- 172.21.15.12:0/79948813 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbf2c0ebd50 con 0x7fbf38069e90 2023-10-30T12:46:37.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.359+0000 7fbf3ec54640 1 -- 172.21.15.12:0/79948813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf381596b0 con 0x7fbf38069e90 2023-10-30T12:46:37.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.363+0000 7fbf357fa640 1 -- 172.21.15.12:0/79948813 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6538 (secure 0 0 0) 0x7fbf2c0b0090 con 0x7fbf38069e90 2023-10-30T12:46:37.366 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:37.366 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:18.260197+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:37.367 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 56 2023-10-30T12:46:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.363+0000 7fbf3ec54640 1 -- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbf1c0664d0 msgr2=0x7fbf1c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.363+0000 7fbf3ec54640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbf1c0664d0 0x7fbf1c068990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbf381578e0 tx=0x7fbf24009040 comp rx=0 tx=0).stop 2023-10-30T12:46:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.363+0000 7fbf3ec54640 1 -- 172.21.15.12:0/79948813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf38069e90 msgr2=0x7fbf380c1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.363+0000 7fbf3ec54640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf38069e90 0x7fbf380c1610 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0ae040 tx=0x7fbf2c01ac00 comp rx=0 tx=0).stop 2023-10-30T12:46:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.363+0000 7fbf3ec54640 1 -- 172.21.15.12:0/79948813 shutdown_connections 2023-10-30T12:46:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.363+0000 7fbf3ec54640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf381539a0 0x7fbf38141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.363+0000 7fbf3ec54640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbf1c0664d0 0x7fbf1c068990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.363+0000 7fbf3ec54640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf38070a00 0x7fbf38141750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:37.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.363+0000 7fbf3ec54640 1 --2- 172.21.15.12:0/79948813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf38069e90 0x7fbf380c1610 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:37.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.363+0000 7fbf3ec54640 1 -- 172.21.15.12:0/79948813 >> 172.21.15.12:0/79948813 conn(0x7fbf380c7910 msgr2=0x7fbf38158f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:37.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.363+0000 7fbf3ec54640 1 -- 172.21.15.12:0/79948813 shutdown_connections 2023-10-30T12:46:37.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:37.363+0000 7fbf3ec54640 1 -- 172.21.15.12:0/79948813 wait complete. 2023-10-30T12:46:37.388 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:46:39.388 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:39.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e4c29e640 1 Processor -- start 2023-10-30T12:46:39.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e4c29e640 1 -- start start 2023-10-30T12:46:39.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e4c29e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e44069e90 0x7f3e4415a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:39.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e4c29e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3e4406a270 0x7f3e440c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:39.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e4c29e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e440c6c10 con 0x7f3e4406cb80 2023-10-30T12:46:39.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e4c29e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e440c8570 con 0x7f3e4406a270 2023-10-30T12:46:39.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e4c29e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e440c6ef0 con 0x7f3e44069e90 2023-10-30T12:46:39.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e49839640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3e4406a270 0x7f3e440c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:39.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e49839640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3e4406a270 0x7f3e440c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58830/0 (socket says 172.21.15.12:58830) 2023-10-30T12:46:39.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e49839640 1 -- 172.21.15.12:0/4153045162 learned_addr learned my addr 172.21.15.12:0/4153045162 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:39.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e4a03a640 1 --2- 172.21.15.12:0/4153045162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e44069e90 0x7f3e4415a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:39.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e49839640 1 -- 172.21.15.12:0/4153045162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e44069e90 msgr2=0x7f3e4415a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:39.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e49839640 1 --2- 172.21.15.12:0/4153045162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e44069e90 0x7f3e4415a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:39.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e49839640 1 -- 172.21.15.12:0/4153045162 >> v1:172.21.15.154:6789/0 conn(0x7f3e4406cb80 legacy=0x7f3e4406d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:39.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e49839640 1 -- 172.21.15.12:0/4153045162 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e440c71d0 con 0x7f3e4406a270 2023-10-30T12:46:39.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.539+0000 7f3e4a03a640 1 --2- 172.21.15.12:0/4153045162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e44069e90 0x7f3e4415a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:46:39.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.543+0000 7f3e49839640 1 --2- 172.21.15.12:0/4153045162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4406a270 0x7f3e440c9cb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009870 tx=0x7f3e3401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64164a13d65b130b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:39.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.543+0000 7f3e49038640 1 -- 172.21.15.12:0/4153045162 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e340a7070 con 0x7f3e4406a270 2023-10-30T12:46:39.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.543+0000 7f3e49038640 1 -- 172.21.15.12:0/4153045162 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e340ab430 con 0x7f3e4406a270 2023-10-30T12:46:39.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.543+0000 7f3e49038640 1 -- 172.21.15.12:0/4153045162 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e340ab7c0 con 0x7f3e4406a270 2023-10-30T12:46:39.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.543+0000 7f3e4c29e640 1 -- 172.21.15.12:0/4153045162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4406a270 msgr2=0x7f3e440c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:39.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.543+0000 7f3e4c29e640 1 --2- 172.21.15.12:0/4153045162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4406a270 0x7f3e440c9cb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009870 tx=0x7f3e3401c540 comp rx=0 tx=0).stop 2023-10-30T12:46:39.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.543+0000 7f3e4c29e640 1 -- 172.21.15.12:0/4153045162 shutdown_connections 2023-10-30T12:46:39.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.543+0000 7f3e4c29e640 1 --2- 172.21.15.12:0/4153045162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4406a270 0x7f3e440c9cb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:39.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.543+0000 7f3e4c29e640 1 --2- 172.21.15.12:0/4153045162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e44069e90 0x7f3e4415a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.543+0000 7f3e4c29e640 1 -- 172.21.15.12:0/4153045162 >> 172.21.15.12:0/4153045162 conn(0x7f3e4406dfd0 msgr2=0x7f3e4406e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.543+0000 7f3e4c29e640 1 -- 172.21.15.12:0/4153045162 shutdown_connections 2023-10-30T12:46:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.543+0000 7f3e4c29e640 1 -- 172.21.15.12:0/4153045162 wait complete. 2023-10-30T12:46:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4c29e640 1 Processor -- start 2023-10-30T12:46:39.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4c29e640 1 -- start start 2023-10-30T12:46:39.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4c29e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e44069e90 0x7f3e4415d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:39.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4c29e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e4406a270 0x7f3e44149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:39.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4c29e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4406cb80 0x7f3e441f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:39.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4c29e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4415b310 con 0x7f3e4406cb80 2023-10-30T12:46:39.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4c29e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e441f5ae0 con 0x7f3e4406a270 2023-10-30T12:46:39.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4c29e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e440cb8f0 con 0x7f3e44069e90 2023-10-30T12:46:39.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4a83b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4406cb80 0x7f3e441f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:39.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4a83b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4406cb80 0x7f3e441f4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58832/0 (socket says 172.21.15.12:58832) 2023-10-30T12:46:39.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4a83b640 1 -- 172.21.15.12:0/3080128742 learned_addr learned my addr 172.21.15.12:0/3080128742 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:39.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4a03a640 1 --2- 172.21.15.12:0/3080128742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e44069e90 0x7f3e4415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:39.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e49839640 1 --2- 172.21.15.12:0/3080128742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e4406a270 0x7f3e44149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:39.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4a83b640 1 -- 172.21.15.12:0/3080128742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e44069e90 msgr2=0x7f3e4415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4a83b640 1 --2- 172.21.15.12:0/3080128742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e44069e90 0x7f3e4415d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4a83b640 1 -- 172.21.15.12:0/3080128742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e4406a270 msgr2=0x7f3e44149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4a83b640 1 --2- 172.21.15.12:0/3080128742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e4406a270 0x7f3e44149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4a83b640 1 -- 172.21.15.12:0/3080128742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e340094d0 con 0x7f3e4406cb80 2023-10-30T12:46:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4a83b640 1 --2- 172.21.15.12:0/3080128742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4406cb80 0x7f3e441f4cf0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00c730 tx=0x7f3e3c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e3affd640 1 -- 172.21.15.12:0/3080128742 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c003b80 con 0x7f3e4406cb80 2023-10-30T12:46:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e3affd640 1 -- 172.21.15.12:0/3080128742 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3c003ce0 con 0x7f3e4406cb80 2023-10-30T12:46:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4c29e640 1 -- 172.21.15.12:0/3080128742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e440cbbd0 con 0x7f3e4406cb80 2023-10-30T12:46:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e3affd640 1 -- 172.21.15.12:0/3080128742 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c005310 con 0x7f3e4406cb80 2023-10-30T12:46:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4c29e640 1 -- 172.21.15.12:0/3080128742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4415ce90 con 0x7f3e4406cb80 2023-10-30T12:46:39.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.547+0000 7f3e4c29e640 1 -- 172.21.15.12:0/3080128742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e440b1df0 con 0x7f3e4406cb80 2023-10-30T12:46:39.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.551+0000 7f3e3affd640 1 -- 172.21.15.12:0/3080128742 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3e3c005680 con 0x7f3e4406cb80 2023-10-30T12:46:39.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.551+0000 7f3e3affd640 1 --2- 172.21.15.12:0/3080128742 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3e280664a0 0x7f3e28068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:39.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.551+0000 7f3e3affd640 1 -- 172.21.15.12:0/3080128742 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f3e3c083ff0 con 0x7f3e4406cb80 2023-10-30T12:46:39.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.551+0000 7f3e4a03a640 1 --2- 172.21.15.12:0/3080128742 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3e280664a0 0x7f3e28068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:39.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.551+0000 7f3e4a03a640 1 --2- 172.21.15.12:0/3080128742 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3e280664a0 0x7f3e28068960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000c900 tx=0x7f3e40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:39.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.555+0000 7f3e3affd640 1 -- 172.21.15.12:0/3080128742 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3e3c051260 con 0x7f3e4406cb80 2023-10-30T12:46:39.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.747+0000 7f3e4c29e640 1 -- 172.21.15.12:0/3080128742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e4404e170 con 0x7f3e4406cb80 2023-10-30T12:46:39.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.747+0000 7f3e3affd640 1 -- 172.21.15.12:0/3080128742 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6538 (secure 0 0 0) 0x7f3e3c054880 con 0x7f3e4406cb80 2023-10-30T12:46:39.752 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:39.752 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:18.260197+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"c","rank":2,"incarnation":26,"state":"up:active","state_seq":44,"addr":"172.21.15.12:6837/3124135109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3124135109},{"type":"v1","addr":"172.21.15.12:6837","nonce":3124135109}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:39.753 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 56 2023-10-30T12:46:39.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.751+0000 7f3e4c29e640 1 -- 172.21.15.12:0/3080128742 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3e280664a0 msgr2=0x7f3e28068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:39.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.751+0000 7f3e4c29e640 1 --2- 172.21.15.12:0/3080128742 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3e280664a0 0x7f3e28068960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000c900 tx=0x7f3e40008040 comp rx=0 tx=0).stop 2023-10-30T12:46:39.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.751+0000 7f3e4c29e640 1 -- 172.21.15.12:0/3080128742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4406cb80 msgr2=0x7f3e441f4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:39.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.751+0000 7f3e4c29e640 1 --2- 172.21.15.12:0/3080128742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4406cb80 0x7f3e441f4cf0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00c730 tx=0x7f3e3c00cbf0 comp rx=0 tx=0).stop 2023-10-30T12:46:39.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.751+0000 7f3e4c29e640 1 -- 172.21.15.12:0/3080128742 shutdown_connections 2023-10-30T12:46:39.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.751+0000 7f3e4c29e640 1 --2- 172.21.15.12:0/3080128742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e4406cb80 0x7f3e441f4cf0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:39.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.751+0000 7f3e4c29e640 1 --2- 172.21.15.12:0/3080128742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e4406a270 0x7f3e44149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:39.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.751+0000 7f3e4c29e640 1 --2- 172.21.15.12:0/3080128742 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3e280664a0 0x7f3e28068960 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:39.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.751+0000 7f3e4c29e640 1 --2- 172.21.15.12:0/3080128742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e44069e90 0x7f3e4415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:39.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.751+0000 7f3e4c29e640 1 -- 172.21.15.12:0/3080128742 >> 172.21.15.12:0/3080128742 conn(0x7f3e4406dfd0 msgr2=0x7f3e4406b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:39.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.751+0000 7f3e4c29e640 1 -- 172.21.15.12:0/3080128742 shutdown_connections 2023-10-30T12:46:39.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:39.751+0000 7f3e4c29e640 1 -- 172.21.15.12:0/3080128742 wait complete. 2023-10-30T12:46:39.774 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:46:41.775 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:41.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c45d3640 1 Processor -- start 2023-10-30T12:46:41.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c45d3640 1 -- start start 2023-10-30T12:46:41.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c45d3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f68bc0c6b80 0x7f68bc0bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:41.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c45d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68bc13f4a0 0x7f68bc0c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:41.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c45d3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68bc0c0a50 con 0x7f68bc0c6f60 2023-10-30T12:46:41.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c45d3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68bc06f6e0 con 0x7f68bc0c6b80 2023-10-30T12:46:41.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c45d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc06f9c0 con 0x7f68bc13f4a0 2023-10-30T12:46:41.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c2b70640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68bc13f4a0 0x7f68bc0c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:41.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c236f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f68bc0c6b80 0x7f68bc0bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:41.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c1b6e640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f68bc0c6f60 0x7f68bc13ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:49300/0 (socket says 172.21.15.12:49300) 2023-10-30T12:46:41.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c1b6e640 1 -- 172.21.15.12:0/2395497401 learned_addr learned my addr 172.21.15.12:0/2395497401 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:46:41.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c2b70640 1 -- 172.21.15.12:0/2395497401 >> v2:172.21.15.154:3300/0 conn(0x7f68bc0c6b80 msgr2=0x7f68bc0bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:41.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c2b70640 1 --2- 172.21.15.12:0/2395497401 >> v2:172.21.15.154:3300/0 conn(0x7f68bc0c6b80 0x7f68bc0bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:41.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c2b70640 1 -- 172.21.15.12:0/2395497401 >> v1:172.21.15.154:6789/0 conn(0x7f68bc0c6f60 legacy=0x7f68bc13ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:41.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c2b70640 1 -- 172.21.15.12:0/2395497401 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc06fca0 con 0x7f68bc13f4a0 2023-10-30T12:46:41.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.923+0000 7f68c236f640 1 --2- 172.21.15.12:0/2395497401 >> v2:172.21.15.154:3300/0 conn(0x7f68bc0c6b80 0x7f68bc0bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:46:41.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.927+0000 7f68c2b70640 1 --2- 172.21.15.12:0/2395497401 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68bc13f4a0 0x7f68bc0c0400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009d90 tx=0x7f68b401eb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a04fda1733b18147 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:41.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.927+0000 7f68c136d640 1 -- 172.21.15.12:0/2395497401 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b40a7b40 con 0x7f68bc13f4a0 2023-10-30T12:46:41.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.927+0000 7f68c136d640 1 -- 172.21.15.12:0/2395497401 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b40a7ca0 con 0x7f68bc13f4a0 2023-10-30T12:46:41.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.927+0000 7f68c136d640 1 -- 172.21.15.12:0/2395497401 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b40af660 con 0x7f68bc13f4a0 2023-10-30T12:46:41.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.927+0000 7f68c45d3640 1 -- 172.21.15.12:0/2395497401 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68bc13f4a0 msgr2=0x7f68bc0c0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:41.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.927+0000 7f68c45d3640 1 --2- 172.21.15.12:0/2395497401 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68bc13f4a0 0x7f68bc0c0400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009d90 tx=0x7f68b401eb60 comp rx=0 tx=0).stop 2023-10-30T12:46:41.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.927+0000 7f68c45d3640 1 -- 172.21.15.12:0/2395497401 shutdown_connections 2023-10-30T12:46:41.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.927+0000 7f68c45d3640 1 --2- 172.21.15.12:0/2395497401 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68bc13f4a0 0x7f68bc0c0400 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:41.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.927+0000 7f68c45d3640 1 --2- 172.21.15.12:0/2395497401 >> v2:172.21.15.154:3300/0 conn(0x7f68bc0c6b80 0x7f68bc0bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:41.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.927+0000 7f68c45d3640 1 -- 172.21.15.12:0/2395497401 >> 172.21.15.12:0/2395497401 conn(0x7f68bc0c7910 msgr2=0x7f68bc0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:41.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.927+0000 7f68c45d3640 1 -- 172.21.15.12:0/2395497401 shutdown_connections 2023-10-30T12:46:41.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.927+0000 7f68c45d3640 1 -- 172.21.15.12:0/2395497401 wait complete. 2023-10-30T12:46:41.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c45d3640 1 Processor -- start 2023-10-30T12:46:41.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c45d3640 1 -- start start 2023-10-30T12:46:41.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c45d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68bc0c6b80 0x7f68bc0c23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:41.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c45d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68bc0c6f60 0x7f68bc155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:41.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c45d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68bc13f4a0 0x7f68bc0c3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:41.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c45d3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc0c28e0 con 0x7f68bc0c6b80 2023-10-30T12:46:41.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c45d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc0c9dd0 con 0x7f68bc0c6f60 2023-10-30T12:46:41.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c45d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc0ca0b0 con 0x7f68bc13f4a0 2023-10-30T12:46:41.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c236f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68bc0c6b80 0x7f68bc0c23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:41.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c236f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68bc0c6b80 0x7f68bc0c23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51840/0 (socket says 172.21.15.12:51840) 2023-10-30T12:46:41.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c236f640 1 -- 172.21.15.12:0/1025527429 learned_addr learned my addr 172.21.15.12:0/1025527429 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:41.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c1b6e640 1 --2- 172.21.15.12:0/1025527429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68bc0c6f60 0x7f68bc155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:41.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c2b70640 1 --2- 172.21.15.12:0/1025527429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68bc13f4a0 0x7f68bc0c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:41.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c236f640 1 -- 172.21.15.12:0/1025527429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68bc13f4a0 msgr2=0x7f68bc0c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:41.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c236f640 1 --2- 172.21.15.12:0/1025527429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68bc13f4a0 0x7f68bc0c3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:41.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c236f640 1 -- 172.21.15.12:0/1025527429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68bc0c6f60 msgr2=0x7f68bc155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:41.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c236f640 1 --2- 172.21.15.12:0/1025527429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68bc0c6f60 0x7f68bc155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:41.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c236f640 1 -- 172.21.15.12:0/1025527429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b8009420 con 0x7f68bc0c6b80 2023-10-30T12:46:41.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c236f640 1 --2- 172.21.15.12:0/1025527429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68bc0c6b80 0x7f68bc0c23a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f68b800ebe0 tx=0x7f68b800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:41.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68b2ffd640 1 -- 172.21.15.12:0/1025527429 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b800eee0 con 0x7f68bc0c6b80 2023-10-30T12:46:41.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68b2ffd640 1 -- 172.21.15.12:0/1025527429 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b800cc30 con 0x7f68bc0c6b80 2023-10-30T12:46:41.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c45d3640 1 -- 172.21.15.12:0/1025527429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b4009a70 con 0x7f68bc0c6b80 2023-10-30T12:46:41.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68b2ffd640 1 -- 172.21.15.12:0/1025527429 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8013470 con 0x7f68bc0c6b80 2023-10-30T12:46:41.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.931+0000 7f68c45d3640 1 -- 172.21.15.12:0/1025527429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68bc0ca670 con 0x7f68bc0c6b80 2023-10-30T12:46:41.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.935+0000 7f68c45d3640 1 -- 172.21.15.12:0/1025527429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68bc04e170 con 0x7f68bc0c6b80 2023-10-30T12:46:41.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.935+0000 7f68b2ffd640 1 -- 172.21.15.12:0/1025527429 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f68b80135d0 con 0x7f68bc0c6b80 2023-10-30T12:46:41.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.935+0000 7f68b2ffd640 1 --2- 172.21.15.12:0/1025527429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f68a00664d0 0x7f68a0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:41.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.935+0000 7f68b2ffd640 1 -- 172.21.15.12:0/1025527429 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f68b8084410 con 0x7f68bc0c6b80 2023-10-30T12:46:41.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.935+0000 7f68c1b6e640 1 --2- 172.21.15.12:0/1025527429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f68a00664d0 0x7f68a0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:41.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.935+0000 7f68c1b6e640 1 --2- 172.21.15.12:0/1025527429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f68a00664d0 0x7f68a0068990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f68bc13a440 tx=0x7f68ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:41.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:41.939+0000 7f68b2ffd640 1 -- 172.21.15.12:0/1025527429 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f68b80515d0 con 0x7f68bc0c6b80 2023-10-30T12:46:42.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.135+0000 7f68c45d3640 1 -- 172.21.15.12:0/1025527429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68bc0c2c40 con 0x7f68bc0c6b80 2023-10-30T12:46:42.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.135+0000 7f68b2ffd640 1 -- 172.21.15.12:0/1025527429 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+5743 (secure 0 0 0) 0x7f68b8054bf0 con 0x7f68bc0c6b80 2023-10-30T12:46:42.140 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:42.140 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:40.313895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:42.141 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 57 2023-10-30T12:46:42.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.139+0000 7f68c45d3640 1 -- 172.21.15.12:0/1025527429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f68a00664d0 msgr2=0x7f68a0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:42.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.139+0000 7f68c45d3640 1 --2- 172.21.15.12:0/1025527429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f68a00664d0 0x7f68a0068990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f68bc13a440 tx=0x7f68ac009040 comp rx=0 tx=0).stop 2023-10-30T12:46:42.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.139+0000 7f68c45d3640 1 -- 172.21.15.12:0/1025527429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68bc0c6b80 msgr2=0x7f68bc0c23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:42.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.139+0000 7f68c45d3640 1 --2- 172.21.15.12:0/1025527429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68bc0c6b80 0x7f68bc0c23a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f68b800ebe0 tx=0x7f68b800c520 comp rx=0 tx=0).stop 2023-10-30T12:46:42.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.139+0000 7f68c45d3640 1 -- 172.21.15.12:0/1025527429 shutdown_connections 2023-10-30T12:46:42.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.139+0000 7f68c45d3640 1 --2- 172.21.15.12:0/1025527429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f68a00664d0 0x7f68a0068990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:42.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.139+0000 7f68c45d3640 1 --2- 172.21.15.12:0/1025527429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f68bc13f4a0 0x7f68bc0c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:42.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.139+0000 7f68c45d3640 1 --2- 172.21.15.12:0/1025527429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68bc0c6f60 0x7f68bc155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:42.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.139+0000 7f68c45d3640 1 --2- 172.21.15.12:0/1025527429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68bc0c6b80 0x7f68bc0c23a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:42.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.139+0000 7f68c45d3640 1 -- 172.21.15.12:0/1025527429 >> 172.21.15.12:0/1025527429 conn(0x7f68bc0c7910 msgr2=0x7f68bc0bfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:42.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.139+0000 7f68c45d3640 1 -- 172.21.15.12:0/1025527429 shutdown_connections 2023-10-30T12:46:42.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.139+0000 7f68c45d3640 1 -- 172.21.15.12:0/1025527429 wait complete. 2023-10-30T12:46:42.162 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2023-10-30T12:46:42.162 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:46:42.162 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:42.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.303+0000 7f3fa1809640 1 Processor -- start 2023-10-30T12:46:42.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.303+0000 7f3fa1809640 1 -- start start 2023-10-30T12:46:42.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.307+0000 7f3fa1809640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3f9c153b10 0x7f3f9c15ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:42.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.307+0000 7f3fa1809640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f9c0c74f0 con 0x7f3f9c0c5e70 2023-10-30T12:46:42.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.307+0000 7f3fa1809640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f9c0c77b0 con 0x7f3f9c0b0dc0 2023-10-30T12:46:42.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.307+0000 7f3fa1809640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f9c0c1f20 con 0x7f3f9c153b10 2023-10-30T12:46:42.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.307+0000 7f3f9affd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3f9c153b10 0x7f3f9c15ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:42.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.307+0000 7f3f9b7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3f9c0c5e70 0x7f3f9c1641f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34514/0 (socket says 172.21.15.12:34514) 2023-10-30T12:46:42.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.307+0000 7f3f9b7fe640 1 -- 172.21.15.12:0/542746127 learned_addr learned my addr 172.21.15.12:0/542746127 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:46:42.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.307+0000 7f3f9affd640 1 -- 172.21.15.12:0/542746127 >> v1:172.21.15.154:6789/0 conn(0x7f3f9c0b0dc0 legacy=0x7f3f9c0c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:42.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.307+0000 7f3f9affd640 1 -- 172.21.15.12:0/542746127 >> v1:172.21.15.12:6789/0 conn(0x7f3f9c0c5e70 legacy=0x7f3f9c1641f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:42.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.307+0000 7f3f9affd640 1 -- 172.21.15.12:0/542746127 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9c159b80 con 0x7f3f9c153b10 2023-10-30T12:46:42.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3f9affd640 1 --2- 172.21.15.12:0/542746127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f9c153b10 0x7f3f9c15ce60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0097c0 tx=0x7f3f8c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850c7e8eda61d4b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:42.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3f9a7fc640 1 -- 172.21.15.12:0/542746127 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c0a8970 con 0x7f3f9c153b10 2023-10-30T12:46:42.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3f9a7fc640 1 -- 172.21.15.12:0/542746127 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8c0a8ad0 con 0x7f3f9c153b10 2023-10-30T12:46:42.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3f9a7fc640 1 -- 172.21.15.12:0/542746127 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c0a8e60 con 0x7f3f9c153b10 2023-10-30T12:46:42.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3fa1809640 1 -- 172.21.15.12:0/542746127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f9c153b10 msgr2=0x7f3f9c15ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:42.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3fa1809640 1 --2- 172.21.15.12:0/542746127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f9c153b10 0x7f3f9c15ce60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0097c0 tx=0x7f3f8c01ec70 comp rx=0 tx=0).stop 2023-10-30T12:46:42.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3fa1809640 1 -- 172.21.15.12:0/542746127 shutdown_connections 2023-10-30T12:46:42.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3fa1809640 1 --2- 172.21.15.12:0/542746127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f9c153b10 0x7f3f9c15ce60 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:42.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3fa1809640 1 -- 172.21.15.12:0/542746127 >> 172.21.15.12:0/542746127 conn(0x7f3f9c06d770 msgr2=0x7f3f9c06db80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:42.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3fa1809640 1 -- 172.21.15.12:0/542746127 shutdown_connections 2023-10-30T12:46:42.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3fa1809640 1 -- 172.21.15.12:0/542746127 wait complete. 2023-10-30T12:46:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3fa1809640 1 Processor -- start 2023-10-30T12:46:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3fa1809640 1 -- start start 2023-10-30T12:46:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3fa1809640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f9c0b0dc0 0x7f3f9c15c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3fa1809640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f9c0c5e70 0x7f3f9c149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3fa1809640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f9c153b10 0x7f3f9c155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3fa1809640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c15a9f0 con 0x7f3f9c0b0dc0 2023-10-30T12:46:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3fa1809640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c155290 con 0x7f3f9c0c5e70 2023-10-30T12:46:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3fa1809640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c154170 con 0x7f3f9c153b10 2023-10-30T12:46:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3f9affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f9c0b0dc0 0x7f3f9c15c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3f9affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f9c0b0dc0 0x7f3f9c15c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51860/0 (socket says 172.21.15.12:51860) 2023-10-30T12:46:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.311+0000 7f3f9affd640 1 -- 172.21.15.12:0/3496768017 learned_addr learned my addr 172.21.15.12:0/3496768017 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3f93fff640 1 --2- 172.21.15.12:0/3496768017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f9c0c5e70 0x7f3f9c149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3f9b7fe640 1 --2- 172.21.15.12:0/3496768017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f9c153b10 0x7f3f9c155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3f9affd640 1 -- 172.21.15.12:0/3496768017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f9c153b10 msgr2=0x7f3f9c155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3f9affd640 1 --2- 172.21.15.12:0/3496768017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f9c153b10 0x7f3f9c155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3f9affd640 1 -- 172.21.15.12:0/3496768017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f9c0c5e70 msgr2=0x7f3f9c149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3f9affd640 1 --2- 172.21.15.12:0/3496768017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f9c0c5e70 0x7f3f9c149d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3f9affd640 1 -- 172.21.15.12:0/3496768017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8c009420 con 0x7f3f9c0b0dc0 2023-10-30T12:46:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3f9affd640 1 --2- 172.21.15.12:0/3496768017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f9c0b0dc0 0x7f3f9c15c1e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0b3040 tx=0x7f3f8c01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3f98ff9640 1 -- 172.21.15.12:0/3496768017 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c0b9630 con 0x7f3f9c0b0dc0 2023-10-30T12:46:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3f98ff9640 1 -- 172.21.15.12:0/3496768017 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8c0b97b0 con 0x7f3f9c0b0dc0 2023-10-30T12:46:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3f98ff9640 1 -- 172.21.15.12:0/3496768017 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c0b9b60 con 0x7f3f9c0b0dc0 2023-10-30T12:46:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3fa1809640 1 -- 172.21.15.12:0/3496768017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9c154f60 con 0x7f3f9c0b0dc0 2023-10-30T12:46:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3fa1809640 1 -- 172.21.15.12:0/3496768017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9c1f2ec0 con 0x7f3f9c0b0dc0 2023-10-30T12:46:42.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3fa1809640 1 -- 172.21.15.12:0/3496768017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f9c074250 con 0x7f3f9c0b0dc0 2023-10-30T12:46:42.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3f98ff9640 1 -- 172.21.15.12:0/3496768017 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3f8c0b7040 con 0x7f3f9c0b0dc0 2023-10-30T12:46:42.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3f98ff9640 1 --2- 172.21.15.12:0/3496768017 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f7c0664a0 0x7f3f7c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:42.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.315+0000 7f3f98ff9640 1 -- 172.21.15.12:0/3496768017 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f3f8c122120 con 0x7f3f9c0b0dc0 2023-10-30T12:46:42.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.319+0000 7f3f93fff640 1 --2- 172.21.15.12:0/3496768017 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f7c0664a0 0x7f3f7c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:42.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.319+0000 7f3f93fff640 1 --2- 172.21.15.12:0/3496768017 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f7c0664a0 0x7f3f7c068960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0c0400 tx=0x7f3f84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:42.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.319+0000 7f3f98ff9640 1 -- 172.21.15.12:0/3496768017 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3f8c0ef250 con 0x7f3f9c0b0dc0 2023-10-30T12:46:42.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.515+0000 7f3fa1809640 1 -- 172.21.15.12:0/3496768017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f9c04e170 con 0x7f3f9c0b0dc0 2023-10-30T12:46:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.515+0000 7f3f98ff9640 1 -- 172.21.15.12:0/3496768017 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+5743 (secure 0 0 0) 0x7f3f8c0b5090 con 0x7f3f9c0b0dc0 2023-10-30T12:46:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:42.521 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:40.313895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 57 2023-10-30T12:46:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.515+0000 7f3fa1809640 1 -- 172.21.15.12:0/3496768017 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f7c0664a0 msgr2=0x7f3f7c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.515+0000 7f3fa1809640 1 --2- 172.21.15.12:0/3496768017 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f7c0664a0 0x7f3f7c068960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0c0400 tx=0x7f3f84009040 comp rx=0 tx=0).stop 2023-10-30T12:46:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.515+0000 7f3fa1809640 1 -- 172.21.15.12:0/3496768017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f9c0b0dc0 msgr2=0x7f3f9c15c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:42.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.515+0000 7f3fa1809640 1 --2- 172.21.15.12:0/3496768017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f9c0b0dc0 0x7f3f9c15c1e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0b3040 tx=0x7f3f8c01ba10 comp rx=0 tx=0).stop 2023-10-30T12:46:42.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.515+0000 7f3fa1809640 1 -- 172.21.15.12:0/3496768017 shutdown_connections 2023-10-30T12:46:42.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.515+0000 7f3fa1809640 1 --2- 172.21.15.12:0/3496768017 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f7c0664a0 0x7f3f7c068960 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:42.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.515+0000 7f3fa1809640 1 --2- 172.21.15.12:0/3496768017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f9c153b10 0x7f3f9c155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:42.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.515+0000 7f3fa1809640 1 --2- 172.21.15.12:0/3496768017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f9c0c5e70 0x7f3f9c149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:42.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.515+0000 7f3fa1809640 1 --2- 172.21.15.12:0/3496768017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f9c0b0dc0 0x7f3f9c15c1e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:42.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.515+0000 7f3fa1809640 1 -- 172.21.15.12:0/3496768017 >> 172.21.15.12:0/3496768017 conn(0x7f3f9c06d770 msgr2=0x7f3f9c1590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:42.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.519+0000 7f3fa1809640 1 -- 172.21.15.12:0/3496768017 shutdown_connections 2023-10-30T12:46:42.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:42.519+0000 7f3fa1809640 1 -- 172.21.15.12:0/3496768017 wait complete. 2023-10-30T12:46:42.543 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 57, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 57, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5000": { "addr": "172.21.15.154:6839/1635719967", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1635719967, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 1635719967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5000, "incarnation": 44, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 25 }, "gid_5124": { "addr": "172.21.15.12:6835/559649789", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 559649789, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 559649789, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5124, "incarnation": 36, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 39 }, "gid_5427": { "addr": "172.21.15.12:6836/2219276371", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 2219276371, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 2219276371, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5427, "incarnation": 57, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 50, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:46:40.313895+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5000, "mds_1": 5124, "mds_2": 5427 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:46:44.545 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.695+0000 7f51d7b3e640 1 Processor -- start 2023-10-30T12:46:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.695+0000 7f51d7b3e640 1 -- start start 2023-10-30T12:46:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.695+0000 7f51d7b3e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f51d0070740 0x7f51d00c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.695+0000 7f51d7b3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d015e700 0x7f51d006a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.695+0000 7f51d7b3e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d00c5810 con 0x7f51d01580e0 2023-10-30T12:46:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.695+0000 7f51d7b3e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d015eb90 con 0x7f51d0070740 2023-10-30T12:46:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.695+0000 7f51d7b3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d00c6bb0 con 0x7f51d015e700 2023-10-30T12:46:44.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.695+0000 7f51d50d9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f51d0070740 0x7f51d00c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:44.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.695+0000 7f51d60db640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d015e700 0x7f51d006a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:44.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.695+0000 7f51d60db640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d015e700 0x7f51d006a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:59360/0 (socket says 172.21.15.12:59360) 2023-10-30T12:46:44.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.695+0000 7f51d60db640 1 -- 172.21.15.12:0/1435138533 learned_addr learned my addr 172.21.15.12:0/1435138533 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:44.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.695+0000 7f51d58da640 1 --1- 172.21.15.12:0/1435138533 >> v1:172.21.15.154:6789/0 conn(0x7f51d01580e0 0x7f51d0069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:49310/0 (socket says 172.21.15.12:49310) 2023-10-30T12:46:44.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.699+0000 7f51d50d9640 1 -- 172.21.15.12:0/1435138533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d015e700 msgr2=0x7f51d006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:44.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.699+0000 7f51d50d9640 1 --2- 172.21.15.12:0/1435138533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d015e700 0x7f51d006a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:44.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.699+0000 7f51d50d9640 1 -- 172.21.15.12:0/1435138533 >> v1:172.21.15.154:6789/0 conn(0x7f51d01580e0 legacy=0x7f51d0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:44.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.699+0000 7f51d50d9640 1 -- 172.21.15.12:0/1435138533 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d00c6e70 con 0x7f51d0070740 2023-10-30T12:46:44.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.699+0000 7f51d60db640 1 --2- 172.21.15.12:0/1435138533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d015e700 0x7f51d006a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:46:44.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.699+0000 7f51d50d9640 1 --2- 172.21.15.12:0/1435138533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d0070740 0x7f51d00c6540 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f51c00097c0 tx=0x7f51c001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5d0fd61c01b747 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:44.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.699+0000 7f51d48d8640 1 -- 172.21.15.12:0/1435138533 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c00ab600 con 0x7f51d0070740 2023-10-30T12:46:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.699+0000 7f51d48d8640 1 -- 172.21.15.12:0/1435138533 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c00a7050 con 0x7f51d0070740 2023-10-30T12:46:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.699+0000 7f51d48d8640 1 -- 172.21.15.12:0/1435138533 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c00ab980 con 0x7f51d0070740 2023-10-30T12:46:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 -- 172.21.15.12:0/1435138533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d0070740 msgr2=0x7f51d00c6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 --2- 172.21.15.12:0/1435138533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d0070740 0x7f51d00c6540 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f51c00097c0 tx=0x7f51c001c480 comp rx=0 tx=0).stop 2023-10-30T12:46:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 -- 172.21.15.12:0/1435138533 shutdown_connections 2023-10-30T12:46:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 --2- 172.21.15.12:0/1435138533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d015e700 0x7f51d006a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 --2- 172.21.15.12:0/1435138533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d0070740 0x7f51d00c6540 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 -- 172.21.15.12:0/1435138533 >> 172.21.15.12:0/1435138533 conn(0x7f51d00c7910 msgr2=0x7f51d00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 -- 172.21.15.12:0/1435138533 shutdown_connections 2023-10-30T12:46:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 -- 172.21.15.12:0/1435138533 wait complete. 2023-10-30T12:46:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 Processor -- start 2023-10-30T12:46:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 -- start start 2023-10-30T12:46:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d0070740 0x7f51d0154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d01580e0 0x7f51d01fa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d015e700 0x7f51d0152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01fa620 con 0x7f51d0070740 2023-10-30T12:46:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01f6a70 con 0x7f51d01580e0 2023-10-30T12:46:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d7b3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d00c4ea0 con 0x7f51d015e700 2023-10-30T12:46:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d58da640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d0070740 0x7f51d0154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d58da640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d0070740 0x7f51d0154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51868/0 (socket says 172.21.15.12:51868) 2023-10-30T12:46:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d58da640 1 -- 172.21.15.12:0/3304235797 learned_addr learned my addr 172.21.15.12:0/3304235797 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d50d9640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d01580e0 0x7f51d01fa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:44.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d60db640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d015e700 0x7f51d0152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:44.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d58da640 1 -- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d015e700 msgr2=0x7f51d0152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:44.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.703+0000 7f51d58da640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d015e700 0x7f51d0152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:44.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51d58da640 1 -- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d01580e0 msgr2=0x7f51d01fa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:44.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51d58da640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d01580e0 0x7f51d01fa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:44.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51d58da640 1 -- 172.21.15.12:0/3304235797 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c8009b30 con 0x7f51d0070740 2023-10-30T12:46:44.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51d50d9640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d01580e0 0x7f51d01fa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:46:44.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51d60db640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d015e700 0x7f51d0152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:46:44.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51d58da640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d0070740 0x7f51d0154e60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c490 tx=0x7f51cc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:44.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51c67fc640 1 -- 172.21.15.12:0/3304235797 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0074e0 con 0x7f51d0070740 2023-10-30T12:46:44.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51c67fc640 1 -- 172.21.15.12:0/3304235797 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc007640 con 0x7f51d0070740 2023-10-30T12:46:44.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51c67fc640 1 -- 172.21.15.12:0/3304235797 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc004a10 con 0x7f51d0070740 2023-10-30T12:46:44.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51d7b3e640 1 -- 172.21.15.12:0/3304235797 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c0009420 con 0x7f51d0070740 2023-10-30T12:46:44.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51d7b3e640 1 -- 172.21.15.12:0/3304235797 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d00c5140 con 0x7f51d0070740 2023-10-30T12:46:44.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51d7b3e640 1 -- 172.21.15.12:0/3304235797 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d004e170 con 0x7f51d0070740 2023-10-30T12:46:44.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51c67fc640 1 -- 172.21.15.12:0/3304235797 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f51cc004b70 con 0x7f51d0070740 2023-10-30T12:46:44.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51c67fc640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51b40664a0 0x7f51b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:44.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51c67fc640 1 -- 172.21.15.12:0/3304235797 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f51cc083b10 con 0x7f51d0070740 2023-10-30T12:46:44.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.707+0000 7f51d50d9640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51b40664a0 0x7f51b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:44.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.711+0000 7f51d50d9640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51b40664a0 0x7f51b4068960 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f51c00ad000 tx=0x7f51c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:44.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.711+0000 7f51c67fc640 1 -- 172.21.15.12:0/3304235797 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f51cc050d60 con 0x7f51d0070740 2023-10-30T12:46:44.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.903+0000 7f51d7b3e640 1 -- 172.21.15.12:0/3304235797 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51d006aed0 con 0x7f51d0070740 2023-10-30T12:46:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.903+0000 7f51c67fc640 1 -- 172.21.15.12:0/3304235797 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+5743 (secure 0 0 0) 0x7f51cc054380 con 0x7f51d0070740 2023-10-30T12:46:44.909 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:44.910 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:40.313895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 57 2023-10-30T12:46:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.907+0000 7f51d7b3e640 1 -- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51b40664a0 msgr2=0x7f51b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.907+0000 7f51d7b3e640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51b40664a0 0x7f51b4068960 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f51c00ad000 tx=0x7f51c0008040 comp rx=0 tx=0).stop 2023-10-30T12:46:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.907+0000 7f51d7b3e640 1 -- 172.21.15.12:0/3304235797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d0070740 msgr2=0x7f51d0154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:44.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.907+0000 7f51d7b3e640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d0070740 0x7f51d0154e60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c490 tx=0x7f51cc00c950 comp rx=0 tx=0).stop 2023-10-30T12:46:44.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.907+0000 7f51d7b3e640 1 -- 172.21.15.12:0/3304235797 shutdown_connections 2023-10-30T12:46:44.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.907+0000 7f51d7b3e640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51b40664a0 0x7f51b4068960 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:44.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.907+0000 7f51d7b3e640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d015e700 0x7f51d0152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:44.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.907+0000 7f51d7b3e640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d01580e0 0x7f51d01fa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.907+0000 7f51d7b3e640 1 --2- 172.21.15.12:0/3304235797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d0070740 0x7f51d0154e60 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.907+0000 7f51d7b3e640 1 -- 172.21.15.12:0/3304235797 >> 172.21.15.12:0/3304235797 conn(0x7f51d00c7910 msgr2=0x7f51d0156c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.907+0000 7f51d7b3e640 1 -- 172.21.15.12:0/3304235797 shutdown_connections 2023-10-30T12:46:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:44.907+0000 7f51d7b3e640 1 -- 172.21.15.12:0/3304235797 wait complete. 2023-10-30T12:46:44.931 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:46:46.932 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:47.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.075+0000 7fdeee70f640 1 Processor -- start 2023-10-30T12:46:47.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.075+0000 7fdeee70f640 1 -- start start 2023-10-30T12:46:47.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.075+0000 7fdeee70f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdee81538a0 0x7fdee815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:47.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.075+0000 7fdeee70f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdee80ca0e0 0x7fdee815a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:47.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.075+0000 7fdeee70f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee80cb800 con 0x7fdee80cf160 2023-10-30T12:46:47.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.075+0000 7fdeee70f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee815dc30 con 0x7fdee81538a0 2023-10-30T12:46:47.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.075+0000 7fdeee70f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee80cb330 con 0x7fdee80ca0e0 2023-10-30T12:46:47.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.075+0000 7fdee7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdee81538a0 0x7fdee815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:47.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.075+0000 7fdee7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdee81538a0 0x7fdee815d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51870/0 (socket says 172.21.15.12:51870) 2023-10-30T12:46:47.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.075+0000 7fdee7fff640 1 -- 172.21.15.12:0/1640475117 learned_addr learned my addr 172.21.15.12:0/1640475117 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:47.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.075+0000 7fdee77fe640 1 --2- 172.21.15.12:0/1640475117 >> v2:172.21.15.154:3300/0 conn(0x7fdee80ca0e0 0x7fdee815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:47.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.075+0000 7fdee7fff640 1 -- 172.21.15.12:0/1640475117 >> v2:172.21.15.154:3300/0 conn(0x7fdee80ca0e0 msgr2=0x7fdee815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:47.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.075+0000 7fdee7fff640 1 --2- 172.21.15.12:0/1640475117 >> v2:172.21.15.154:3300/0 conn(0x7fdee80ca0e0 0x7fdee815a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:47.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.075+0000 7fdee7fff640 1 -- 172.21.15.12:0/1640475117 >> v1:172.21.15.154:6789/0 conn(0x7fdee80cf160 legacy=0x7fdee80cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:47.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.075+0000 7fdee7fff640 1 -- 172.21.15.12:0/1640475117 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80cad70 con 0x7fdee81538a0 2023-10-30T12:46:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.079+0000 7fdee7fff640 1 --2- 172.21.15.12:0/1640475117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee81538a0 0x7fdee815d6f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdee80b30a0 tx=0x7fded801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81da8bd4debc79aa server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.079+0000 7fdee6ffd640 1 -- 172.21.15.12:0/1640475117 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded80a89f0 con 0x7fdee81538a0 2023-10-30T12:46:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.079+0000 7fdee6ffd640 1 -- 172.21.15.12:0/1640475117 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded80a8b50 con 0x7fdee81538a0 2023-10-30T12:46:47.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.079+0000 7fdee6ffd640 1 -- 172.21.15.12:0/1640475117 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded80b0440 con 0x7fdee81538a0 2023-10-30T12:46:47.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.079+0000 7fdeee70f640 1 -- 172.21.15.12:0/1640475117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee81538a0 msgr2=0x7fdee815d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:47.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.079+0000 7fdeee70f640 1 --2- 172.21.15.12:0/1640475117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee81538a0 0x7fdee815d6f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdee80b30a0 tx=0x7fded801ecf0 comp rx=0 tx=0).stop 2023-10-30T12:46:47.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.079+0000 7fdeee70f640 1 -- 172.21.15.12:0/1640475117 shutdown_connections 2023-10-30T12:46:47.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.079+0000 7fdeee70f640 1 --2- 172.21.15.12:0/1640475117 >> v2:172.21.15.154:3300/0 conn(0x7fdee80ca0e0 0x7fdee815a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:47.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.079+0000 7fdeee70f640 1 --2- 172.21.15.12:0/1640475117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee81538a0 0x7fdee815d6f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:47.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.079+0000 7fdeee70f640 1 -- 172.21.15.12:0/1640475117 >> 172.21.15.12:0/1640475117 conn(0x7fdee806bfe0 msgr2=0x7fdee806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:47.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.079+0000 7fdeee70f640 1 -- 172.21.15.12:0/1640475117 shutdown_connections 2023-10-30T12:46:47.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.079+0000 7fdeee70f640 1 -- 172.21.15.12:0/1640475117 wait complete. 2023-10-30T12:46:47.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdeee70f640 1 Processor -- start 2023-10-30T12:46:47.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdeee70f640 1 -- start start 2023-10-30T12:46:47.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdeee70f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee80ca0e0 0x7fdee80c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:47.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdeee70f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdee80cf160 0x7fdee814ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:47.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdeee70f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdee81538a0 0x7fdee81ef210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:47.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdeee70f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee8151a80 con 0x7fdee80ca0e0 2023-10-30T12:46:47.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdeee70f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee81f3330 con 0x7fdee80cf160 2023-10-30T12:46:47.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdeee70f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80c9430 con 0x7fdee81538a0 2023-10-30T12:46:47.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdee7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee80ca0e0 0x7fdee80c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:47.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdee7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee80ca0e0 0x7fdee80c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51882/0 (socket says 172.21.15.12:51882) 2023-10-30T12:46:47.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdee7fff640 1 -- 172.21.15.12:0/1766325033 learned_addr learned my addr 172.21.15.12:0/1766325033 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:47.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdee77fe640 1 --2- 172.21.15.12:0/1766325033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdee80cf160 0x7fdee814ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:47.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdeeccac640 1 --2- 172.21.15.12:0/1766325033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdee81538a0 0x7fdee81ef210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:47.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdee7fff640 1 -- 172.21.15.12:0/1766325033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdee81538a0 msgr2=0x7fdee81ef210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:47.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdee7fff640 1 --2- 172.21.15.12:0/1766325033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdee81538a0 0x7fdee81ef210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:47.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdee7fff640 1 -- 172.21.15.12:0/1766325033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdee80cf160 msgr2=0x7fdee814ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:47.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdee7fff640 1 --2- 172.21.15.12:0/1766325033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdee80cf160 0x7fdee814ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:47.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdee7fff640 1 -- 172.21.15.12:0/1766325033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded80094d0 con 0x7fdee80ca0e0 2023-10-30T12:46:47.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdee7fff640 1 --2- 172.21.15.12:0/1766325033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee80ca0e0 0x7fdee80c8bf0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fded80b3040 tx=0x7fded801ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:47.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdee4ff9640 1 -- 172.21.15.12:0/1766325033 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded80ba600 con 0x7fdee80ca0e0 2023-10-30T12:46:47.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdee4ff9640 1 -- 172.21.15.12:0/1766325033 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded80ba760 con 0x7fdee80ca0e0 2023-10-30T12:46:47.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdee4ff9640 1 -- 172.21.15.12:0/1766325033 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded80bab10 con 0x7fdee80ca0e0 2023-10-30T12:46:47.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdeee70f640 1 -- 172.21.15.12:0/1766325033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee81f2720 con 0x7fdee80ca0e0 2023-10-30T12:46:47.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.083+0000 7fdeee70f640 1 -- 172.21.15.12:0/1766325033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee81f2ce0 con 0x7fdee80ca0e0 2023-10-30T12:46:47.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.087+0000 7fdeee70f640 1 -- 172.21.15.12:0/1766325033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee80b2a50 con 0x7fdee80ca0e0 2023-10-30T12:46:47.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.087+0000 7fdee4ff9640 1 -- 172.21.15.12:0/1766325033 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fded80bac70 con 0x7fdee80ca0e0 2023-10-30T12:46:47.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.087+0000 7fdee4ff9640 1 --2- 172.21.15.12:0/1766325033 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fded40664a0 0x7fded4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:47.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.087+0000 7fdee4ff9640 1 -- 172.21.15.12:0/1766325033 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fded8122570 con 0x7fdee80ca0e0 2023-10-30T12:46:47.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.087+0000 7fdee77fe640 1 --2- 172.21.15.12:0/1766325033 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fded40664a0 0x7fded4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:47.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.087+0000 7fdee77fe640 1 --2- 172.21.15.12:0/1766325033 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fded40664a0 0x7fded4068960 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdee80c5040 tx=0x7fdedc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:47.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.091+0000 7fdee4ff9640 1 -- 172.21.15.12:0/1766325033 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fded80ef740 con 0x7fdee80ca0e0 2023-10-30T12:46:47.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.279+0000 7fdeee70f640 1 -- 172.21.15.12:0/1766325033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdee804e170 con 0x7fdee80ca0e0 2023-10-30T12:46:47.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.279+0000 7fdee4ff9640 1 -- 172.21.15.12:0/1766325033 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5745 (secure 0 0 0) 0x7fded80f2d60 con 0x7fdee80ca0e0 2023-10-30T12:46:47.285 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:47.286 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:46.296351+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:resolve","state_seq":25,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:47.286 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 58 2023-10-30T12:46:47.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.283+0000 7fdeee70f640 1 -- 172.21.15.12:0/1766325033 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fded40664a0 msgr2=0x7fded4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:47.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.283+0000 7fdeee70f640 1 --2- 172.21.15.12:0/1766325033 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fded40664a0 0x7fded4068960 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdee80c5040 tx=0x7fdedc008040 comp rx=0 tx=0).stop 2023-10-30T12:46:47.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.283+0000 7fdeee70f640 1 -- 172.21.15.12:0/1766325033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee80ca0e0 msgr2=0x7fdee80c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:47.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.283+0000 7fdeee70f640 1 --2- 172.21.15.12:0/1766325033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee80ca0e0 0x7fdee80c8bf0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fded80b3040 tx=0x7fded801ba90 comp rx=0 tx=0).stop 2023-10-30T12:46:47.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.283+0000 7fdeee70f640 1 -- 172.21.15.12:0/1766325033 shutdown_connections 2023-10-30T12:46:47.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.283+0000 7fdeee70f640 1 --2- 172.21.15.12:0/1766325033 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fded40664a0 0x7fded4068960 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:47.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.283+0000 7fdeee70f640 1 --2- 172.21.15.12:0/1766325033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdee81538a0 0x7fdee81ef210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:47.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.283+0000 7fdeee70f640 1 --2- 172.21.15.12:0/1766325033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdee80cf160 0x7fdee814ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:47.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.283+0000 7fdeee70f640 1 --2- 172.21.15.12:0/1766325033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee80ca0e0 0x7fdee80c8bf0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.283+0000 7fdeee70f640 1 -- 172.21.15.12:0/1766325033 >> 172.21.15.12:0/1766325033 conn(0x7fdee806bfe0 msgr2=0x7fdee80c4500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.283+0000 7fdeee70f640 1 -- 172.21.15.12:0/1766325033 shutdown_connections 2023-10-30T12:46:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:47.283+0000 7fdeee70f640 1 -- 172.21.15.12:0/1766325033 wait complete. 2023-10-30T12:46:47.307 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:46:49.308 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:49.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51e2e6b640 1 Processor -- start 2023-10-30T12:46:49.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51e2e6b640 1 -- start start 2023-10-30T12:46:49.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51e2e6b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f51dc0c93d0 0x7f51dc06ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:49.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51e2e6b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51dc06afa0 con 0x7f51dc06e470 2023-10-30T12:46:49.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51e2e6b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51dc154150 con 0x7f51dc153d70 2023-10-30T12:46:49.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51e2e6b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc06d780 con 0x7f51dc0c93d0 2023-10-30T12:46:49.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51dbfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f51dc06e470 0x7f51dc0c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34522/0 (socket says 172.21.15.12:34522) 2023-10-30T12:46:49.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51dbfff640 1 -- 172.21.15.12:0/3403094265 learned_addr learned my addr 172.21.15.12:0/3403094265 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:46:49.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51e1408640 1 --2- 172.21.15.12:0/3403094265 >> v2:172.21.15.154:3300/0 conn(0x7f51dc0c93d0 0x7f51dc06ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:49.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51db7fe640 1 -- 172.21.15.12:0/3403094265 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417426181 0 0) 0x7f51dc06afa0 con 0x7f51dc06e470 2023-10-30T12:46:49.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51db7fe640 1 -- 172.21.15.12:0/3403094265 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51cc003530 con 0x7f51dc06e470 2023-10-30T12:46:49.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51db7fe640 1 -- 172.21.15.12:0/3403094265 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4278205436 0 0) 0x7f51cc003530 con 0x7f51dc06e470 2023-10-30T12:46:49.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51db7fe640 1 -- 172.21.15.12:0/3403094265 >> v2:172.21.15.154:3300/0 conn(0x7f51dc0c93d0 msgr2=0x7f51dc06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:49.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51db7fe640 1 --2- 172.21.15.12:0/3403094265 >> v2:172.21.15.154:3300/0 conn(0x7f51dc0c93d0 0x7f51dc06ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:49.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51db7fe640 1 -- 172.21.15.12:0/3403094265 >> v1:172.21.15.154:6789/0 conn(0x7f51dc153d70 legacy=0x7f51dc0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:49.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51db7fe640 1 -- 172.21.15.12:0/3403094265 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc06e030 con 0x7f51dc06e470 2023-10-30T12:46:49.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51db7fe640 1 -- 172.21.15.12:0/3403094265 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f51c8001680 con 0x7f51dc06e470 2023-10-30T12:46:49.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51db7fe640 1 -- 172.21.15.12:0/3403094265 >> v1:172.21.15.12:6789/0 conn(0x7f51dc06e470 legacy=0x7f51dc0c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:49.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51db7fe640 1 --2- 172.21.15.12:0/3403094265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51cc003980 0x7f51cc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:49.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51db7fe640 1 --2- 172.21.15.12:0/3403094265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51cc0047d0 0x7f51cc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:49.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51db7fe640 1 --2- 172.21.15.12:0/3403094265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51cc005190 0x7f51cc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:49.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51db7fe640 1 -- 172.21.15.12:0/3403094265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc06d780 con 0x7f51cc0047d0 2023-10-30T12:46:49.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51db7fe640 1 -- 172.21.15.12:0/3403094265 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc00c030 con 0x7f51cc003980 2023-10-30T12:46:49.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.459+0000 7f51db7fe640 1 -- 172.21.15.12:0/3403094265 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc00c1f0 con 0x7f51cc005190 2023-10-30T12:46:49.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.463+0000 7f51e1408640 1 --2- 172.21.15.12:0/3403094265 >> v2:172.21.15.154:3300/0 conn(0x7f51dc0c93d0 0x7f51dc06ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:46:49.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.463+0000 7f51dbfff640 1 --2- 172.21.15.12:0/3403094265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51cc0047d0 0x7f51cc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:49.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.463+0000 7f51e0c07640 1 --2- 172.21.15.12:0/3403094265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51cc003980 0x7f51cc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:49.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.463+0000 7f51e1408640 1 --2- 172.21.15.12:0/3403094265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51cc005190 0x7f51cc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:49.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.463+0000 7f51dbfff640 1 -- 172.21.15.12:0/3403094265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51cc005190 msgr2=0x7f51cc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:49.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.463+0000 7f51dbfff640 1 --2- 172.21.15.12:0/3403094265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51cc005190 0x7f51cc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:49.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.463+0000 7f51dbfff640 1 -- 172.21.15.12:0/3403094265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51cc003980 msgr2=0x7f51cc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:49.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.463+0000 7f51dbfff640 1 --2- 172.21.15.12:0/3403094265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51cc003980 0x7f51cc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:49.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.463+0000 7f51dbfff640 1 -- 172.21.15.12:0/3403094265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51cc003530 con 0x7f51cc0047d0 2023-10-30T12:46:49.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.463+0000 7f51e1408640 1 -- 172.21.15.12:0/3403094265 reap_dead start 2023-10-30T12:46:49.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.463+0000 7f51dbfff640 1 --2- 172.21.15.12:0/3403094265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51cc0047d0 0x7f51cc004c50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f51c800c240 tx=0x7f51c801f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66cb2533aafc37c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:49.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.463+0000 7f51db7fe640 1 -- 172.21.15.12:0/3403094265 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8005ae0 con 0x7f51cc0047d0 2023-10-30T12:46:49.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.463+0000 7f51db7fe640 1 -- 172.21.15.12:0/3403094265 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c80ab070 con 0x7f51cc0047d0 2023-10-30T12:46:49.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.463+0000 7f51db7fe640 1 -- 172.21.15.12:0/3403094265 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8007070 con 0x7f51cc0047d0 2023-10-30T12:46:49.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e2e6b640 1 -- 172.21.15.12:0/3403094265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51cc0047d0 msgr2=0x7f51cc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:49.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e2e6b640 1 --2- 172.21.15.12:0/3403094265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51cc0047d0 0x7f51cc004c50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f51c800c240 tx=0x7f51c801f330 comp rx=0 tx=0).stop 2023-10-30T12:46:49.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e2e6b640 1 -- 172.21.15.12:0/3403094265 shutdown_connections 2023-10-30T12:46:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e2e6b640 1 --2- 172.21.15.12:0/3403094265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51cc0047d0 0x7f51cc004c50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e2e6b640 1 -- 172.21.15.12:0/3403094265 >> 172.21.15.12:0/3403094265 conn(0x7f51dc071ea0 msgr2=0x7f51dc065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e2e6b640 1 -- 172.21.15.12:0/3403094265 shutdown_connections 2023-10-30T12:46:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e2e6b640 1 -- 172.21.15.12:0/3403094265 wait complete. 2023-10-30T12:46:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e2e6b640 1 Processor -- start 2023-10-30T12:46:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e2e6b640 1 -- start start 2023-10-30T12:46:49.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e2e6b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51cc0047d0 0x7f51dc14b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:49.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e2e6b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51dc0ccb70 0x7f51dc14ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:49.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e2e6b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51dc1f2560 0x7f51dc1f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:49.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e2e6b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0cc6e0 con 0x7f51dc1f2560 2023-10-30T12:46:49.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e2e6b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc155ee0 con 0x7f51dc0ccb70 2023-10-30T12:46:49.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e2e6b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc156180 con 0x7f51cc0047d0 2023-10-30T12:46:49.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e1408640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51dc1f2560 0x7f51dc1f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:49.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e1408640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51dc1f2560 0x7f51dc1f29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51906/0 (socket says 172.21.15.12:51906) 2023-10-30T12:46:49.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e1408640 1 -- 172.21.15.12:0/593043790 learned_addr learned my addr 172.21.15.12:0/593043790 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:49.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51e0c07640 1 --2- 172.21.15.12:0/593043790 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51cc0047d0 0x7f51dc14b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:49.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.467+0000 7f51dbfff640 1 --2- 172.21.15.12:0/593043790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51dc0ccb70 0x7f51dc14ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:49.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.471+0000 7f51e1408640 1 -- 172.21.15.12:0/593043790 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51cc0047d0 msgr2=0x7f51dc14b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:49.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.471+0000 7f51e1408640 1 --2- 172.21.15.12:0/593043790 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51cc0047d0 0x7f51dc14b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:49.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.471+0000 7f51e1408640 1 -- 172.21.15.12:0/593043790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51dc0ccb70 msgr2=0x7f51dc14ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:49.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.471+0000 7f51e1408640 1 --2- 172.21.15.12:0/593043790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51dc0ccb70 0x7f51dc14ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:49.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.471+0000 7f51e1408640 1 -- 172.21.15.12:0/593043790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0cbdb0 con 0x7f51dc1f2560 2023-10-30T12:46:49.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.471+0000 7f51e1408640 1 --2- 172.21.15.12:0/593043790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51dc1f2560 0x7f51dc1f29e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0c8440 tx=0x7f51d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:49.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.471+0000 7f51d97fa640 1 -- 172.21.15.12:0/593043790 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d400e070 con 0x7f51dc1f2560 2023-10-30T12:46:49.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.471+0000 7f51d97fa640 1 -- 172.21.15.12:0/593043790 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d4011e60 con 0x7f51dc1f2560 2023-10-30T12:46:49.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.471+0000 7f51e2e6b640 1 -- 172.21.15.12:0/593043790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51dc0cc090 con 0x7f51dc1f2560 2023-10-30T12:46:49.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.471+0000 7f51d97fa640 1 -- 172.21.15.12:0/593043790 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d40039a0 con 0x7f51dc1f2560 2023-10-30T12:46:49.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.471+0000 7f51e2e6b640 1 -- 172.21.15.12:0/593043790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc154330 con 0x7f51dc1f2560 2023-10-30T12:46:49.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.471+0000 7f51e2e6b640 1 -- 172.21.15.12:0/593043790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51dc04e170 con 0x7f51dc1f2560 2023-10-30T12:46:49.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.471+0000 7f51d97fa640 1 -- 172.21.15.12:0/593043790 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f51d4003b00 con 0x7f51dc1f2560 2023-10-30T12:46:49.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.471+0000 7f51d97fa640 1 --2- 172.21.15.12:0/593043790 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51c00664a0 0x7f51c0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:49.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.471+0000 7f51d97fa640 1 -- 172.21.15.12:0/593043790 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f51d4084980 con 0x7f51dc1f2560 2023-10-30T12:46:49.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.475+0000 7f51e0c07640 1 --2- 172.21.15.12:0/593043790 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51c00664a0 0x7f51c0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:49.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.475+0000 7f51e0c07640 1 --2- 172.21.15.12:0/593043790 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51c00664a0 0x7f51c0068960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f51d0000c10 tx=0x7f51d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:49.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.479+0000 7f51d97fa640 1 -- 172.21.15.12:0/593043790 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f51d4051b40 con 0x7f51dc1f2560 2023-10-30T12:46:49.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.671+0000 7f51e2e6b640 1 -- 172.21.15.12:0/593043790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51dc0cc820 con 0x7f51dc1f2560 2023-10-30T12:46:49.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.671+0000 7f51d97fa640 1 -- 172.21.15.12:0/593043790 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5745 (secure 0 0 0) 0x7f51d4055160 con 0x7f51dc1f2560 2023-10-30T12:46:49.675 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:49.676 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:46.296351+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:resolve","state_seq":25,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:49.677 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 58 2023-10-30T12:46:49.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.671+0000 7f51e2e6b640 1 -- 172.21.15.12:0/593043790 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51c00664a0 msgr2=0x7f51c0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:49.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.671+0000 7f51e2e6b640 1 --2- 172.21.15.12:0/593043790 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51c00664a0 0x7f51c0068960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f51d0000c10 tx=0x7f51d0009040 comp rx=0 tx=0).stop 2023-10-30T12:46:49.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.671+0000 7f51e2e6b640 1 -- 172.21.15.12:0/593043790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51dc1f2560 msgr2=0x7f51dc1f29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:49.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.671+0000 7f51e2e6b640 1 --2- 172.21.15.12:0/593043790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51dc1f2560 0x7f51dc1f29e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0c8440 tx=0x7f51d4009f90 comp rx=0 tx=0).stop 2023-10-30T12:46:49.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.671+0000 7f51e2e6b640 1 -- 172.21.15.12:0/593043790 shutdown_connections 2023-10-30T12:46:49.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.671+0000 7f51e2e6b640 1 --2- 172.21.15.12:0/593043790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51dc1f2560 0x7f51dc1f29e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:49.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.671+0000 7f51e2e6b640 1 --2- 172.21.15.12:0/593043790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51dc0ccb70 0x7f51dc14ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:49.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.671+0000 7f51e2e6b640 1 --2- 172.21.15.12:0/593043790 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51c00664a0 0x7f51c0068960 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:49.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.671+0000 7f51e2e6b640 1 --2- 172.21.15.12:0/593043790 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51cc0047d0 0x7f51dc14b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:49.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.671+0000 7f51e2e6b640 1 -- 172.21.15.12:0/593043790 >> 172.21.15.12:0/593043790 conn(0x7f51dc071ea0 msgr2=0x7f51dc1515a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:49.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.671+0000 7f51e2e6b640 1 -- 172.21.15.12:0/593043790 shutdown_connections 2023-10-30T12:46:49.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:49.671+0000 7f51e2e6b640 1 -- 172.21.15.12:0/593043790 wait complete. 2023-10-30T12:46:49.696 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:46:51.697 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:51.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.839+0000 7fd1ae209640 1 Processor -- start 2023-10-30T12:46:51.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.839+0000 7fd1ae209640 1 -- start start 2023-10-30T12:46:51.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1ae209640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80c05d0 0x7fd1a813ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:51.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1ae209640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a80c0a50 con 0x7fd1a80c6b80 2023-10-30T12:46:51.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1ae209640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a806f6e0 con 0x7fd1a80c6f60 2023-10-30T12:46:51.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1ae209640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a806f9c0 con 0x7fd1a80c05d0 2023-10-30T12:46:51.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a77fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd1a80c6b80 0x7fd1a80bfec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42432/0 (socket says 172.21.15.12:42432) 2023-10-30T12:46:51.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a77fe640 1 -- 172.21.15.12:0/1200933517 learned_addr learned my addr 172.21.15.12:0/1200933517 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:46:51.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a6ffd640 1 --2- 172.21.15.12:0/1200933517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80c05d0 0x7fd1a813ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:51.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a67fc640 1 -- 172.21.15.12:0/1200933517 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1187639353 0 0) 0x7fd1a80c0a50 con 0x7fd1a80c6b80 2023-10-30T12:46:51.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a67fc640 1 -- 172.21.15.12:0/1200933517 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd190005740 con 0x7fd1a80c6b80 2023-10-30T12:46:51.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a67fc640 1 -- 172.21.15.12:0/1200933517 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 448178378 0 0) 0x7fd190005740 con 0x7fd1a80c6b80 2023-10-30T12:46:51.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a67fc640 1 -- 172.21.15.12:0/1200933517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80c05d0 msgr2=0x7fd1a813ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:51.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a67fc640 1 --2- 172.21.15.12:0/1200933517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80c05d0 0x7fd1a813ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:51.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a67fc640 1 -- 172.21.15.12:0/1200933517 >> v1:172.21.15.154:6789/0 conn(0x7fd1a80c6f60 legacy=0x7fd1a81566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:51.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a67fc640 1 -- 172.21.15.12:0/1200933517 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a806fca0 con 0x7fd1a80c6b80 2023-10-30T12:46:51.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a67fc640 1 -- 172.21.15.12:0/1200933517 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fd198001680 con 0x7fd1a80c6b80 2023-10-30T12:46:51.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a67fc640 1 -- 172.21.15.12:0/1200933517 >> v1:172.21.15.12:6789/0 conn(0x7fd1a80c6b80 legacy=0x7fd1a80bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:51.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a67fc640 1 --2- 172.21.15.12:0/1200933517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd190005bc0 0x7fd190005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:51.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a67fc640 1 --2- 172.21.15.12:0/1200933517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80c05d0 0x7fd1900069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:51.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a67fc640 1 --2- 172.21.15.12:0/1200933517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd190006f20 0x7fd1900073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:51.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a67fc640 1 -- 172.21.15.12:0/1200933517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a806f9c0 con 0x7fd190006f20 2023-10-30T12:46:51.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a67fc640 1 -- 172.21.15.12:0/1200933517 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19000dc80 con 0x7fd190005bc0 2023-10-30T12:46:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a67fc640 1 -- 172.21.15.12:0/1200933517 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19000de40 con 0x7fd1a80c05d0 2023-10-30T12:46:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a7fff640 1 --2- 172.21.15.12:0/1200933517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd190006f20 0x7fd1900073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a6ffd640 1 --2- 172.21.15.12:0/1200933517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80c05d0 0x7fd1900069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a77fe640 1 --2- 172.21.15.12:0/1200933517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd190005bc0 0x7fd190005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:51.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a7fff640 1 -- 172.21.15.12:0/1200933517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80c05d0 msgr2=0x7fd1900069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a7fff640 1 --2- 172.21.15.12:0/1200933517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80c05d0 0x7fd1900069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a7fff640 1 -- 172.21.15.12:0/1200933517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd190005bc0 msgr2=0x7fd190005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a7fff640 1 --2- 172.21.15.12:0/1200933517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd190005bc0 0x7fd190005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:51.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.843+0000 7fd1a7fff640 1 -- 172.21.15.12:0/1200933517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd190005740 con 0x7fd190006f20 2023-10-30T12:46:51.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.847+0000 7fd1a7fff640 1 --2- 172.21.15.12:0/1200933517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd190006f20 0x7fd1900073a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0001e40 tx=0x7fd1a001c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31eeca8620a9d340 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:51.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.847+0000 7fd1a67fc640 1 -- 172.21.15.12:0/1200933517 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0003b80 con 0x7fd190006f20 2023-10-30T12:46:51.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.847+0000 7fd1a67fc640 1 -- 172.21.15.12:0/1200933517 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a0003ce0 con 0x7fd190006f20 2023-10-30T12:46:51.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.847+0000 7fd1a67fc640 1 -- 172.21.15.12:0/1200933517 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a00052c0 con 0x7fd190006f20 2023-10-30T12:46:51.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.847+0000 7fd1ae209640 1 -- 172.21.15.12:0/1200933517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd190006f20 msgr2=0x7fd1900073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:51.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.847+0000 7fd1ae209640 1 --2- 172.21.15.12:0/1200933517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd190006f20 0x7fd1900073a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0001e40 tx=0x7fd1a001c740 comp rx=0 tx=0).stop 2023-10-30T12:46:51.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.847+0000 7fd1a7fff640 1 -- 172.21.15.12:0/1200933517 reap_dead start 2023-10-30T12:46:51.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.847+0000 7fd1ae209640 1 -- 172.21.15.12:0/1200933517 shutdown_connections 2023-10-30T12:46:51.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.847+0000 7fd1ae209640 1 -- 172.21.15.12:0/1200933517 >> 172.21.15.12:0/1200933517 conn(0x7fd1a80c7910 msgr2=0x7fd1a80c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:51.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.847+0000 7fd1ae209640 1 -- 172.21.15.12:0/1200933517 shutdown_connections 2023-10-30T12:46:51.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.847+0000 7fd1ae209640 1 -- 172.21.15.12:0/1200933517 wait complete. 2023-10-30T12:46:51.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.847+0000 7fd1ae209640 1 Processor -- start 2023-10-30T12:46:51.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1ae209640 1 -- start start 2023-10-30T12:46:51.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1ae209640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd190006f20 0x7fd1a80c4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:51.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1ae209640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1a813a190 0x7fd1a80ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:51.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1ae209640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80ca6b0 0x7fd1a8154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:51.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1ae209640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a8153640 con 0x7fd1a813a190 2023-10-30T12:46:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1ae209640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a8155cb0 con 0x7fd190006f20 2023-10-30T12:46:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1ae209640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a80c4eb0 con 0x7fd1a80ca6b0 2023-10-30T12:46:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1a6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1a813a190 0x7fd1a80ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1a6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1a813a190 0x7fd1a80ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47994/0 (socket says 172.21.15.12:47994) 2023-10-30T12:46:51.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1a6ffd640 1 -- 172.21.15.12:0/1580420989 learned_addr learned my addr 172.21.15.12:0/1580420989 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:51.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1a7fff640 1 --2- 172.21.15.12:0/1580420989 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80ca6b0 0x7fd1a8154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:51.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1a77fe640 1 --2- 172.21.15.12:0/1580420989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd190006f20 0x7fd1a80c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:51.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1a6ffd640 1 -- 172.21.15.12:0/1580420989 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80ca6b0 msgr2=0x7fd1a8154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:51.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1a6ffd640 1 --2- 172.21.15.12:0/1580420989 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80ca6b0 0x7fd1a8154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:51.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1a6ffd640 1 -- 172.21.15.12:0/1580420989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd190006f20 msgr2=0x7fd1a80c4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:51.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1a6ffd640 1 --2- 172.21.15.12:0/1580420989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd190006f20 0x7fd1a80c4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:51.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1a6ffd640 1 -- 172.21.15.12:0/1580420989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a81ef6e0 con 0x7fd1a813a190 2023-10-30T12:46:51.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1a6ffd640 1 --2- 172.21.15.12:0/1580420989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1a813a190 0x7fd1a80ca170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0119d0 tx=0x7fd19c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:51.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd18ffff640 1 -- 172.21.15.12:0/1580420989 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c00fb50 con 0x7fd1a813a190 2023-10-30T12:46:51.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd18ffff640 1 -- 172.21.15.12:0/1580420989 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19c00fcb0 con 0x7fd1a813a190 2023-10-30T12:46:51.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1ae209640 1 -- 172.21.15.12:0/1580420989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a8155f00 con 0x7fd1a813a190 2023-10-30T12:46:51.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd18ffff640 1 -- 172.21.15.12:0/1580420989 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c014440 con 0x7fd1a813a190 2023-10-30T12:46:51.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1ae209640 1 -- 172.21.15.12:0/1580420989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a81564c0 con 0x7fd1a813a190 2023-10-30T12:46:51.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.851+0000 7fd1ae209640 1 -- 172.21.15.12:0/1580420989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a8156680 con 0x7fd1a813a190 2023-10-30T12:46:51.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.855+0000 7fd18ffff640 1 -- 172.21.15.12:0/1580420989 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd19c01d400 con 0x7fd1a813a190 2023-10-30T12:46:51.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.855+0000 7fd18ffff640 1 --2- 172.21.15.12:0/1580420989 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1940664a0 0x7fd194068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:51.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.855+0000 7fd18ffff640 1 -- 172.21.15.12:0/1580420989 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fd19c084910 con 0x7fd1a813a190 2023-10-30T12:46:51.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.855+0000 7fd1a77fe640 1 --2- 172.21.15.12:0/1580420989 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1940664a0 0x7fd194068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:51.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.855+0000 7fd1a77fe640 1 --2- 172.21.15.12:0/1580420989 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1940664a0 0x7fd194068960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd198000c10 tx=0x7fd198009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:51.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:51.859+0000 7fd18ffff640 1 -- 172.21.15.12:0/1580420989 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd19c051c20 con 0x7fd1a813a190 2023-10-30T12:46:52.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:52.047+0000 7fd1ae209640 1 -- 172.21.15.12:0/1580420989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1a8154530 con 0x7fd1a813a190 2023-10-30T12:46:52.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:52.047+0000 7fd18ffff640 1 -- 172.21.15.12:0/1580420989 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+5747 (secure 0 0 0) 0x7fd1a8154530 con 0x7fd1a813a190 2023-10-30T12:46:52.054 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:52.054 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:51.545295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:52.055 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 59 2023-10-30T12:46:52.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:52.051+0000 7fd1ae209640 1 -- 172.21.15.12:0/1580420989 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1940664a0 msgr2=0x7fd194068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:52.051+0000 7fd1ae209640 1 --2- 172.21.15.12:0/1580420989 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1940664a0 0x7fd194068960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd198000c10 tx=0x7fd198009040 comp rx=0 tx=0).stop 2023-10-30T12:46:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:52.051+0000 7fd1ae209640 1 -- 172.21.15.12:0/1580420989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1a813a190 msgr2=0x7fd1a80ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:52.051+0000 7fd1ae209640 1 --2- 172.21.15.12:0/1580420989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1a813a190 0x7fd1a80ca170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0119d0 tx=0x7fd19c011e90 comp rx=0 tx=0).stop 2023-10-30T12:46:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:52.051+0000 7fd1ae209640 1 -- 172.21.15.12:0/1580420989 shutdown_connections 2023-10-30T12:46:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:52.051+0000 7fd1ae209640 1 --2- 172.21.15.12:0/1580420989 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd1940664a0 0x7fd194068960 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:52.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:52.051+0000 7fd1ae209640 1 --2- 172.21.15.12:0/1580420989 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80ca6b0 0x7fd1a8154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:52.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:52.051+0000 7fd1ae209640 1 --2- 172.21.15.12:0/1580420989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1a813a190 0x7fd1a80ca170 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:52.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:52.051+0000 7fd1ae209640 1 --2- 172.21.15.12:0/1580420989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd190006f20 0x7fd1a80c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:52.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:52.051+0000 7fd1ae209640 1 -- 172.21.15.12:0/1580420989 >> 172.21.15.12:0/1580420989 conn(0x7fd1a80c7910 msgr2=0x7fd1a80bfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:52.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:52.051+0000 7fd1ae209640 1 -- 172.21.15.12:0/1580420989 shutdown_connections 2023-10-30T12:46:52.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:52.051+0000 7fd1ae209640 1 -- 172.21.15.12:0/1580420989 wait complete. 2023-10-30T12:46:52.075 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:46:54.076 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca32009640 1 Processor -- start 2023-10-30T12:46:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca32009640 1 -- start start 2023-10-30T12:46:54.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca32009640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca2c069e90 0x7fca2c15a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:54.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca32009640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fca2c06a270 0x7fca2c0c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:54.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca32009640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2c0c6c10 con 0x7fca2c06cb80 2023-10-30T12:46:54.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca32009640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2c0c8570 con 0x7fca2c06a270 2023-10-30T12:46:54.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca32009640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0c6ef0 con 0x7fca2c069e90 2023-10-30T12:46:54.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2bfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fca2c06cb80 0x7fca2c06d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42448/0 (socket says 172.21.15.12:42448) 2023-10-30T12:46:54.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2bfff640 1 -- 172.21.15.12:0/3162017272 learned_addr learned my addr 172.21.15.12:0/3162017272 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:46:54.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2b7fe640 1 --2- 172.21.15.12:0/3162017272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca2c069e90 0x7fca2c15a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:54.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2affd640 1 --2- 172.21.15.12:0/3162017272 >> v2:172.21.15.154:3300/0 conn(0x7fca2c06a270 0x7fca2c0c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:54.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 -- 172.21.15.12:0/3162017272 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330527572 0 0) 0x7fca2c0c6c10 con 0x7fca2c06cb80 2023-10-30T12:46:54.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 -- 172.21.15.12:0/3162017272 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca14003530 con 0x7fca2c06cb80 2023-10-30T12:46:54.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 -- 172.21.15.12:0/3162017272 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1991506695 0 0) 0x7fca14003530 con 0x7fca2c06cb80 2023-10-30T12:46:54.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 -- 172.21.15.12:0/3162017272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca2c069e90 msgr2=0x7fca2c15a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 --2- 172.21.15.12:0/3162017272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca2c069e90 0x7fca2c15a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 -- 172.21.15.12:0/3162017272 >> v2:172.21.15.154:3300/0 conn(0x7fca2c06a270 msgr2=0x7fca2c0c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 --2- 172.21.15.12:0/3162017272 >> v2:172.21.15.154:3300/0 conn(0x7fca2c06a270 0x7fca2c0c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 -- 172.21.15.12:0/3162017272 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c0c71d0 con 0x7fca2c06cb80 2023-10-30T12:46:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 -- 172.21.15.12:0/3162017272 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fca24001a00 con 0x7fca2c06cb80 2023-10-30T12:46:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2b7fe640 1 --2- 172.21.15.12:0/3162017272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca2c069e90 0x7fca2c15a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:46:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 -- 172.21.15.12:0/3162017272 >> v1:172.21.15.12:6789/0 conn(0x7fca2c06cb80 legacy=0x7fca2c06d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 --2- 172.21.15.12:0/3162017272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca14003980 0x7fca14003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 --2- 172.21.15.12:0/3162017272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca2c069e90 0x7fca140047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 --2- 172.21.15.12:0/3162017272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca14004d10 0x7fca14005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 -- 172.21.15.12:0/3162017272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0c8570 con 0x7fca14004d10 2023-10-30T12:46:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 -- 172.21.15.12:0/3162017272 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0c6ef0 con 0x7fca14003980 2023-10-30T12:46:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.227+0000 7fca2a7fc640 1 -- 172.21.15.12:0/3162017272 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1400b990 con 0x7fca2c069e90 2023-10-30T12:46:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.231+0000 7fca2bfff640 1 --2- 172.21.15.12:0/3162017272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca14004d10 0x7fca14005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.231+0000 7fca2affd640 1 --2- 172.21.15.12:0/3162017272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca2c069e90 0x7fca140047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.231+0000 7fca2b7fe640 1 --2- 172.21.15.12:0/3162017272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca14003980 0x7fca14003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.231+0000 7fca2bfff640 1 -- 172.21.15.12:0/3162017272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca2c069e90 msgr2=0x7fca140047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.231+0000 7fca2bfff640 1 --2- 172.21.15.12:0/3162017272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca2c069e90 0x7fca140047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.231+0000 7fca2bfff640 1 -- 172.21.15.12:0/3162017272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca14003980 msgr2=0x7fca14003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.231+0000 7fca2bfff640 1 --2- 172.21.15.12:0/3162017272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca14003980 0x7fca14003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.231+0000 7fca2bfff640 1 -- 172.21.15.12:0/3162017272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c009420 con 0x7fca14004d10 2023-10-30T12:46:54.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.231+0000 7fca2bfff640 1 --2- 172.21.15.12:0/3162017272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca14004d10 0x7fca14005190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fca2400a860 tx=0x7fca2401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3694fab10359c573 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:54.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.231+0000 7fca2a7fc640 1 -- 172.21.15.12:0/3162017272 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca240ad630 con 0x7fca14004d10 2023-10-30T12:46:54.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.231+0000 7fca2a7fc640 1 -- 172.21.15.12:0/3162017272 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca240a8070 con 0x7fca14004d10 2023-10-30T12:46:54.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.231+0000 7fca2a7fc640 1 -- 172.21.15.12:0/3162017272 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca240ad9f0 con 0x7fca14004d10 2023-10-30T12:46:54.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.231+0000 7fca32009640 1 -- 172.21.15.12:0/3162017272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca14004d10 msgr2=0x7fca14005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:54.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.231+0000 7fca32009640 1 --2- 172.21.15.12:0/3162017272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca14004d10 0x7fca14005190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fca2400a860 tx=0x7fca2401d360 comp rx=0 tx=0).stop 2023-10-30T12:46:54.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.231+0000 7fca2bfff640 1 -- 172.21.15.12:0/3162017272 reap_dead start 2023-10-30T12:46:54.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca32009640 1 -- 172.21.15.12:0/3162017272 shutdown_connections 2023-10-30T12:46:54.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca32009640 1 -- 172.21.15.12:0/3162017272 >> 172.21.15.12:0/3162017272 conn(0x7fca2c06dfd0 msgr2=0x7fca2c06e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:54.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca32009640 1 -- 172.21.15.12:0/3162017272 shutdown_connections 2023-10-30T12:46:54.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca32009640 1 -- 172.21.15.12:0/3162017272 wait complete. 2023-10-30T12:46:54.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca32009640 1 Processor -- start 2023-10-30T12:46:54.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca32009640 1 -- start start 2023-10-30T12:46:54.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca32009640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca14004d10 0x7fca2c159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:54.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca32009640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca2c1ef000 0x7fca2c154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:54.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca32009640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca2c158c40 0x7fca2c14aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:54.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca32009640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0c6650 con 0x7fca2c1ef000 2023-10-30T12:46:54.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca32009640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c157150 con 0x7fca2c158c40 2023-10-30T12:46:54.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca32009640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c154940 con 0x7fca14004d10 2023-10-30T12:46:54.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca2affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca2c1ef000 0x7fca2c154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:54.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca2b7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca14004d10 0x7fca2c159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:54.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca2affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca2c1ef000 0x7fca2c154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48002/0 (socket says 172.21.15.12:48002) 2023-10-30T12:46:54.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca2affd640 1 -- 172.21.15.12:0/2854014469 learned_addr learned my addr 172.21.15.12:0/2854014469 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:54.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca2bfff640 1 --2- 172.21.15.12:0/2854014469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca2c158c40 0x7fca2c14aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:54.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca2affd640 1 -- 172.21.15.12:0/2854014469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca14004d10 msgr2=0x7fca2c159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:54.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca2affd640 1 --2- 172.21.15.12:0/2854014469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca14004d10 0x7fca2c159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:54.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca2affd640 1 -- 172.21.15.12:0/2854014469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca2c158c40 msgr2=0x7fca2c14aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca2affd640 1 --2- 172.21.15.12:0/2854014469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca2c158c40 0x7fca2c14aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca2affd640 1 -- 172.21.15.12:0/2854014469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c156bd0 con 0x7fca2c1ef000 2023-10-30T12:46:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.235+0000 7fca2bfff640 1 --2- 172.21.15.12:0/2854014469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca2c158c40 0x7fca2c14aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:46:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.239+0000 7fca2affd640 1 --2- 172.21.15.12:0/2854014469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca2c1ef000 0x7fca2c154400 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fca200116c0 tx=0x7fca20011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.239+0000 7fca13fff640 1 -- 172.21.15.12:0/2854014469 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000f9c0 con 0x7fca2c1ef000 2023-10-30T12:46:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.239+0000 7fca13fff640 1 -- 172.21.15.12:0/2854014469 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2000fb20 con 0x7fca2c1ef000 2023-10-30T12:46:54.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.239+0000 7fca13fff640 1 -- 172.21.15.12:0/2854014469 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000fe90 con 0x7fca2c1ef000 2023-10-30T12:46:54.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.239+0000 7fca32009640 1 -- 172.21.15.12:0/2854014469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2c156e70 con 0x7fca2c1ef000 2023-10-30T12:46:54.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.239+0000 7fca32009640 1 -- 172.21.15.12:0/2854014469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2c1ee800 con 0x7fca2c1ef000 2023-10-30T12:46:54.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.239+0000 7fca32009640 1 -- 172.21.15.12:0/2854014469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2c1ee380 con 0x7fca2c1ef000 2023-10-30T12:46:54.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.239+0000 7fca13fff640 1 -- 172.21.15.12:0/2854014469 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fca20015460 con 0x7fca2c1ef000 2023-10-30T12:46:54.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.239+0000 7fca13fff640 1 --2- 172.21.15.12:0/2854014469 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fca180664a0 0x7fca18068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:54.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.239+0000 7fca13fff640 1 -- 172.21.15.12:0/2854014469 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fca20084620 con 0x7fca2c1ef000 2023-10-30T12:46:54.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.239+0000 7fca2b7fe640 1 --2- 172.21.15.12:0/2854014469 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fca180664a0 0x7fca18068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:54.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.243+0000 7fca2b7fe640 1 --2- 172.21.15.12:0/2854014469 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fca180664a0 0x7fca18068960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0093f0 tx=0x7fca1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:54.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.243+0000 7fca13fff640 1 -- 172.21.15.12:0/2854014469 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fca2c1ee380 con 0x7fca2c1ef000 2023-10-30T12:46:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.435+0000 7fca32009640 1 -- 172.21.15.12:0/2854014469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca2c1ee380 con 0x7fca2c1ef000 2023-10-30T12:46:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.435+0000 7fca13fff640 1 -- 172.21.15.12:0/2854014469 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+5747 (secure 0 0 0) 0x7fca2c1ee380 con 0x7fca2c1ef000 2023-10-30T12:46:54.442 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:54.442 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:51.545295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:54.442 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 59 2023-10-30T12:46:54.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.439+0000 7fca32009640 1 -- 172.21.15.12:0/2854014469 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fca180664a0 msgr2=0x7fca18068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:54.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.439+0000 7fca32009640 1 --2- 172.21.15.12:0/2854014469 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fca180664a0 0x7fca18068960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0093f0 tx=0x7fca1c008040 comp rx=0 tx=0).stop 2023-10-30T12:46:54.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.439+0000 7fca32009640 1 -- 172.21.15.12:0/2854014469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca2c1ef000 msgr2=0x7fca2c154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:54.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.439+0000 7fca32009640 1 --2- 172.21.15.12:0/2854014469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca2c1ef000 0x7fca2c154400 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fca200116c0 tx=0x7fca20011b80 comp rx=0 tx=0).stop 2023-10-30T12:46:54.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.439+0000 7fca32009640 1 -- 172.21.15.12:0/2854014469 shutdown_connections 2023-10-30T12:46:54.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.439+0000 7fca32009640 1 --2- 172.21.15.12:0/2854014469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca2c158c40 0x7fca2c14aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.439+0000 7fca32009640 1 --2- 172.21.15.12:0/2854014469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca2c1ef000 0x7fca2c154400 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.439+0000 7fca32009640 1 --2- 172.21.15.12:0/2854014469 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fca180664a0 0x7fca18068960 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.439+0000 7fca32009640 1 --2- 172.21.15.12:0/2854014469 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca14004d10 0x7fca2c159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.439+0000 7fca32009640 1 -- 172.21.15.12:0/2854014469 >> 172.21.15.12:0/2854014469 conn(0x7fca2c06dfd0 msgr2=0x7fca2c06ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.439+0000 7fca32009640 1 -- 172.21.15.12:0/2854014469 shutdown_connections 2023-10-30T12:46:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:54.439+0000 7fca32009640 1 -- 172.21.15.12:0/2854014469 wait complete. 2023-10-30T12:46:54.464 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 59, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 59, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5000": { "addr": "172.21.15.154:6839/1635719967", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1635719967, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 1635719967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5000, "incarnation": 44, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 25 }, "gid_5124": { "addr": "172.21.15.12:6835/559649789", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 559649789, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 559649789, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5124, "incarnation": 36, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 39 }, "gid_5427": { "addr": "172.21.15.12:6836/2219276371", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 2219276371, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 2219276371, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5427, "incarnation": 57, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:reconnect", "state_seq": 27 } }, "last_failure": 0, "last_failure_osd_epoch": 50, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:46:51.545295+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5000, "mds_1": 5124, "mds_2": 5427 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:46:56.465 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:56.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.615+0000 7f93c4a17640 1 Processor -- start 2023-10-30T12:46:56.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.615+0000 7f93c4a17640 1 -- start start 2023-10-30T12:46:56.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93c4a17640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f93c00c6b80 0x7f93c00bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:56.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93c4a17640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93c013f4a0 0x7f93c00c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:56.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93c4a17640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c00c0a50 con 0x7f93c00c6f60 2023-10-30T12:46:56.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93c4a17640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c006f6e0 con 0x7f93c00c6b80 2023-10-30T12:46:56.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93c4a17640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c006f9c0 con 0x7f93c013f4a0 2023-10-30T12:46:56.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93b7fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f93c00c6f60 0x7f93c013ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42460/0 (socket says 172.21.15.12:42460) 2023-10-30T12:46:56.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93b7fff640 1 -- 172.21.15.12:0/2978995685 learned_addr learned my addr 172.21.15.12:0/2978995685 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:46:56.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93be59c640 1 --2- 172.21.15.12:0/2978995685 >> v2:172.21.15.154:3300/0 conn(0x7f93c00c6b80 0x7f93c00bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:56.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bed9d640 1 --2- 172.21.15.12:0/2978995685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93c013f4a0 0x7f93c00c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:56.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 -- 172.21.15.12:0/2978995685 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814746776 0 0) 0x7f93c00c0a50 con 0x7f93c00c6f60 2023-10-30T12:46:56.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 -- 172.21.15.12:0/2978995685 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93ac005740 con 0x7f93c00c6f60 2023-10-30T12:46:56.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 -- 172.21.15.12:0/2978995685 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 998850363 0 0) 0x7f93ac005740 con 0x7f93c00c6f60 2023-10-30T12:46:56.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 -- 172.21.15.12:0/2978995685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93c013f4a0 msgr2=0x7f93c00c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:56.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 --2- 172.21.15.12:0/2978995685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93c013f4a0 0x7f93c00c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:56.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 -- 172.21.15.12:0/2978995685 >> v2:172.21.15.154:3300/0 conn(0x7f93c00c6b80 msgr2=0x7f93c00bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:56.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 --2- 172.21.15.12:0/2978995685 >> v2:172.21.15.154:3300/0 conn(0x7f93c00c6b80 0x7f93c00bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:56.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 -- 172.21.15.12:0/2978995685 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c006fca0 con 0x7f93c00c6f60 2023-10-30T12:46:56.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 -- 172.21.15.12:0/2978995685 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f93a8001680 con 0x7f93c00c6f60 2023-10-30T12:46:56.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 -- 172.21.15.12:0/2978995685 >> v1:172.21.15.12:6789/0 conn(0x7f93c00c6f60 legacy=0x7f93c013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:56.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 --2- 172.21.15.12:0/2978995685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93ac005b90 0x7f93ac005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:56.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 --2- 172.21.15.12:0/2978995685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93c013f4a0 0x7f93ac0069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:56.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 --2- 172.21.15.12:0/2978995685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93ac006f20 0x7f93ac0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:56.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 -- 172.21.15.12:0/2978995685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c006f6e0 con 0x7f93ac005b90 2023-10-30T12:46:56.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 -- 172.21.15.12:0/2978995685 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c006f9c0 con 0x7f93ac006f20 2023-10-30T12:46:56.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bdd9b640 1 -- 172.21.15.12:0/2978995685 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac00dba0 con 0x7f93c013f4a0 2023-10-30T12:46:56.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93be59c640 1 --2- 172.21.15.12:0/2978995685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93ac005b90 0x7f93ac005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:56.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93bed9d640 1 --2- 172.21.15.12:0/2978995685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93ac006f20 0x7f93ac0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:56.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93b7fff640 1 --2- 172.21.15.12:0/2978995685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93c013f4a0 0x7f93ac0069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:56.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93be59c640 1 -- 172.21.15.12:0/2978995685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93c013f4a0 msgr2=0x7f93ac0069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:56.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93be59c640 1 --2- 172.21.15.12:0/2978995685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93c013f4a0 0x7f93ac0069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:56.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93be59c640 1 -- 172.21.15.12:0/2978995685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93ac006f20 msgr2=0x7f93ac0073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:56.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93be59c640 1 --2- 172.21.15.12:0/2978995685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93ac006f20 0x7f93ac0073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:56.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.619+0000 7f93be59c640 1 -- 172.21.15.12:0/2978995685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac005740 con 0x7f93ac005b90 2023-10-30T12:46:56.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.623+0000 7f93be59c640 1 --2- 172.21.15.12:0/2978995685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93ac005b90 0x7f93ac005fb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009ae0 tx=0x7f93b001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1f87cc8d9d536 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:56.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.623+0000 7f93bdd9b640 1 -- 172.21.15.12:0/2978995685 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b00a5570 con 0x7f93ac005b90 2023-10-30T12:46:56.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.623+0000 7f93bdd9b640 1 -- 172.21.15.12:0/2978995685 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b00a56d0 con 0x7f93ac005b90 2023-10-30T12:46:56.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.623+0000 7f93bdd9b640 1 -- 172.21.15.12:0/2978995685 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b00a5a80 con 0x7f93ac005b90 2023-10-30T12:46:56.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.623+0000 7f93c4a17640 1 -- 172.21.15.12:0/2978995685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93ac005b90 msgr2=0x7f93ac005fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:56.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.623+0000 7f93c4a17640 1 --2- 172.21.15.12:0/2978995685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93ac005b90 0x7f93ac005fb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009ae0 tx=0x7f93b001ca90 comp rx=0 tx=0).stop 2023-10-30T12:46:56.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.623+0000 7f93bed9d640 1 -- 172.21.15.12:0/2978995685 reap_dead start 2023-10-30T12:46:56.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.623+0000 7f93c4a17640 1 -- 172.21.15.12:0/2978995685 shutdown_connections 2023-10-30T12:46:56.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.623+0000 7f93c4a17640 1 -- 172.21.15.12:0/2978995685 >> 172.21.15.12:0/2978995685 conn(0x7f93c00c7910 msgr2=0x7f93c00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:56.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.623+0000 7f93c4a17640 1 -- 172.21.15.12:0/2978995685 shutdown_connections 2023-10-30T12:46:56.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.623+0000 7f93c4a17640 1 -- 172.21.15.12:0/2978995685 wait complete. 2023-10-30T12:46:56.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93c4a17640 1 Processor -- start 2023-10-30T12:46:56.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93c4a17640 1 -- start start 2023-10-30T12:46:56.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93c4a17640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93ac005b90 0x7f93c01550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:56.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93c4a17640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c00c1100 0x7f93c01f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:56.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93c4a17640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c01f6070 0x7f93c00aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:56.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93c4a17640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c0155610 con 0x7f93c00c1100 2023-10-30T12:46:56.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93c4a17640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c01f5900 con 0x7f93c01f6070 2023-10-30T12:46:56.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93c4a17640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c01f5be0 con 0x7f93ac005b90 2023-10-30T12:46:56.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93b7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c00c1100 0x7f93c01f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:56.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93b7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c00c1100 0x7f93c01f2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48014/0 (socket says 172.21.15.12:48014) 2023-10-30T12:46:56.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93b7fff640 1 -- 172.21.15.12:0/1773790347 learned_addr learned my addr 172.21.15.12:0/1773790347 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:56.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93be59c640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93ac005b90 0x7f93c01550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:56.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93bed9d640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c01f6070 0x7f93c00aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:56.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93b7fff640 1 -- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93ac005b90 msgr2=0x7f93c01550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:56.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93b7fff640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93ac005b90 0x7f93c01550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:56.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93b7fff640 1 -- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c01f6070 msgr2=0x7f93c00aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:56.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93b7fff640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c01f6070 0x7f93c00aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:56.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93b7fff640 1 -- 172.21.15.12:0/1773790347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c01f32e0 con 0x7f93c00c1100 2023-10-30T12:46:56.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93be59c640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93ac005b90 0x7f93c01550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:46:56.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93bed9d640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c01f6070 0x7f93c00aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:46:56.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93b7fff640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c00c1100 0x7f93c01f2da0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f93a800e880 tx=0x7f93a800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:56.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93b6ffd640 1 -- 172.21.15.12:0/1773790347 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a800c8d0 con 0x7f93c00c1100 2023-10-30T12:46:56.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93b6ffd640 1 -- 172.21.15.12:0/1773790347 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a800ca50 con 0x7f93c00c1100 2023-10-30T12:46:56.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93c4a17640 1 -- 172.21.15.12:0/1773790347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c0155a60 con 0x7f93c00c1100 2023-10-30T12:46:56.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93b6ffd640 1 -- 172.21.15.12:0/1773790347 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a800ce00 con 0x7f93c00c1100 2023-10-30T12:46:56.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.627+0000 7f93c4a17640 1 -- 172.21.15.12:0/1773790347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c0156020 con 0x7f93c00c1100 2023-10-30T12:46:56.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.631+0000 7f93c4a17640 1 -- 172.21.15.12:0/1773790347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c004e170 con 0x7f93c00c1100 2023-10-30T12:46:56.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.631+0000 7f93b6ffd640 1 -- 172.21.15.12:0/1773790347 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f93a801d460 con 0x7f93c00c1100 2023-10-30T12:46:56.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.631+0000 7f93b6ffd640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f93a00664a0 0x7f93a0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:56.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.631+0000 7f93b6ffd640 1 -- 172.21.15.12:0/1773790347 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f93a8084210 con 0x7f93c00c1100 2023-10-30T12:46:56.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.631+0000 7f93be59c640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f93a00664a0 0x7f93a0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:56.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.631+0000 7f93be59c640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f93a00664a0 0x7f93a0068960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f93b00af000 tx=0x7f93b00046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:56.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.635+0000 7f93b6ffd640 1 -- 172.21.15.12:0/1773790347 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f93a8051500 con 0x7f93c00c1100 2023-10-30T12:46:56.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.827+0000 7f93c4a17640 1 -- 172.21.15.12:0/1773790347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93c01f5d20 con 0x7f93c00c1100 2023-10-30T12:46:56.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.827+0000 7f93b6ffd640 1 -- 172.21.15.12:0/1773790347 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+5747 (secure 0 0 0) 0x7f93a8054b20 con 0x7f93c00c1100 2023-10-30T12:46:56.833 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:56.833 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:51.545295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:56.834 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 59 2023-10-30T12:46:56.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.831+0000 7f93c4a17640 1 -- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f93a00664a0 msgr2=0x7f93a0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:56.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.831+0000 7f93c4a17640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f93a00664a0 0x7f93a0068960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f93b00af000 tx=0x7f93b00046d0 comp rx=0 tx=0).stop 2023-10-30T12:46:56.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.831+0000 7f93c4a17640 1 -- 172.21.15.12:0/1773790347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c00c1100 msgr2=0x7f93c01f2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:56.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.831+0000 7f93c4a17640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c00c1100 0x7f93c01f2da0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f93a800e880 tx=0x7f93a800ed40 comp rx=0 tx=0).stop 2023-10-30T12:46:56.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.831+0000 7f93c4a17640 1 -- 172.21.15.12:0/1773790347 shutdown_connections 2023-10-30T12:46:56.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.831+0000 7f93c4a17640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c01f6070 0x7f93c00aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:56.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.831+0000 7f93c4a17640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c00c1100 0x7f93c01f2da0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:56.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.831+0000 7f93c4a17640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f93a00664a0 0x7f93a0068960 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:56.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.831+0000 7f93c4a17640 1 --2- 172.21.15.12:0/1773790347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93ac005b90 0x7f93c01550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:56.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.831+0000 7f93c4a17640 1 -- 172.21.15.12:0/1773790347 >> 172.21.15.12:0/1773790347 conn(0x7f93c00c7910 msgr2=0x7f93c0069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:56.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.831+0000 7f93c4a17640 1 -- 172.21.15.12:0/1773790347 shutdown_connections 2023-10-30T12:46:56.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:56.831+0000 7f93c4a17640 1 -- 172.21.15.12:0/1773790347 wait complete. 2023-10-30T12:46:56.854 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:46:58.855 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:46:59.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.003+0000 7f61d87d3640 1 Processor -- start 2023-10-30T12:46:59.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.003+0000 7f61d87d3640 1 -- start start 2023-10-30T12:46:59.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.007+0000 7f61d87d3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f61d00c6b80 0x7f61d00bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:59.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.007+0000 7f61d87d3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f61d013f4a0 0x7f61d00c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:59.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.007+0000 7f61d87d3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d00c0a50 con 0x7f61d00c6f60 2023-10-30T12:46:59.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.007+0000 7f61d87d3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d006f6e0 con 0x7f61d013f4a0 2023-10-30T12:46:59.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.007+0000 7f61d87d3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d006f9c0 con 0x7f61d00c6b80 2023-10-30T12:46:59.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.007+0000 7f61d6d70640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f61d013f4a0 0x7f61d00c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:59.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.007+0000 7f61d6d70640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f61d013f4a0 0x7f61d00c0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48028/0 (socket says 172.21.15.12:48028) 2023-10-30T12:46:59.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.007+0000 7f61d6d70640 1 -- 172.21.15.12:0/1073288189 learned_addr learned my addr 172.21.15.12:0/1073288189 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:59.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.007+0000 7f61d5d6e640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f61d00c6f60 0x7f61d013ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42466/0 (socket says 172.21.15.12:42466) 2023-10-30T12:46:59.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.007+0000 7f61d656f640 1 --2- 172.21.15.12:0/1073288189 >> v2:172.21.15.154:3300/0 conn(0x7f61d00c6b80 0x7f61d00bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:59.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.007+0000 7f61d6d70640 1 -- 172.21.15.12:0/1073288189 >> v2:172.21.15.154:3300/0 conn(0x7f61d00c6b80 msgr2=0x7f61d00bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:46:59.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.007+0000 7f61d6d70640 1 --2- 172.21.15.12:0/1073288189 >> v2:172.21.15.154:3300/0 conn(0x7f61d00c6b80 0x7f61d00bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:59.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.007+0000 7f61d6d70640 1 -- 172.21.15.12:0/1073288189 >> v1:172.21.15.12:6789/0 conn(0x7f61d00c6f60 legacy=0x7f61d013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:59.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.007+0000 7f61d6d70640 1 -- 172.21.15.12:0/1073288189 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d006fca0 con 0x7f61d013f4a0 2023-10-30T12:46:59.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.007+0000 7f61d556d640 1 -- 172.21.15.12:0/1073288189 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792270559 0 0) 0x7f61d00c0a50 con 0x7f61d00c6f60 2023-10-30T12:46:59.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.011+0000 7f61d6d70640 1 --2- 172.21.15.12:0/1073288189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61d013f4a0 0x7f61d00c0400 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f61d006f1b0 tx=0x7f61c801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5994da1e90a60acd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:59.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.011+0000 7f61d556d640 1 -- 172.21.15.12:0/1073288189 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c80a57a0 con 0x7f61d013f4a0 2023-10-30T12:46:59.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.011+0000 7f61d556d640 1 -- 172.21.15.12:0/1073288189 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c80a5900 con 0x7f61d013f4a0 2023-10-30T12:46:59.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.011+0000 7f61d556d640 1 -- 172.21.15.12:0/1073288189 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c80a5c90 con 0x7f61d013f4a0 2023-10-30T12:46:59.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.011+0000 7f61d87d3640 1 -- 172.21.15.12:0/1073288189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61d013f4a0 msgr2=0x7f61d00c0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:59.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.011+0000 7f61d87d3640 1 --2- 172.21.15.12:0/1073288189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61d013f4a0 0x7f61d00c0400 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f61d006f1b0 tx=0x7f61c801c9d0 comp rx=0 tx=0).stop 2023-10-30T12:46:59.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.011+0000 7f61d87d3640 1 -- 172.21.15.12:0/1073288189 shutdown_connections 2023-10-30T12:46:59.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.011+0000 7f61d87d3640 1 --2- 172.21.15.12:0/1073288189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61d013f4a0 0x7f61d00c0400 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:59.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.011+0000 7f61d87d3640 1 --2- 172.21.15.12:0/1073288189 >> v2:172.21.15.154:3300/0 conn(0x7f61d00c6b80 0x7f61d00bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:59.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.011+0000 7f61d87d3640 1 -- 172.21.15.12:0/1073288189 >> 172.21.15.12:0/1073288189 conn(0x7f61d00c7910 msgr2=0x7f61d00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:59.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.011+0000 7f61d87d3640 1 -- 172.21.15.12:0/1073288189 shutdown_connections 2023-10-30T12:46:59.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.011+0000 7f61d87d3640 1 -- 172.21.15.12:0/1073288189 wait complete. 2023-10-30T12:46:59.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.011+0000 7f61d87d3640 1 Processor -- start 2023-10-30T12:46:59.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.011+0000 7f61d87d3640 1 -- start start 2023-10-30T12:46:59.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d87d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61d00c6b80 0x7f61d0159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:59.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d87d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61d00c6f60 0x7f61d015f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:59.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d87d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61d013f4a0 0x7f61d0134d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:59.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d87d3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d0159990 con 0x7f61d00c6f60 2023-10-30T12:46:59.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d87d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d0132380 con 0x7f61d00c6b80 2023-10-30T12:46:59.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d87d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d01577b0 con 0x7f61d013f4a0 2023-10-30T12:46:59.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d5d6e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61d00c6f60 0x7f61d015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:59.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d5d6e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61d00c6f60 0x7f61d015f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48032/0 (socket says 172.21.15.12:48032) 2023-10-30T12:46:59.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d5d6e640 1 -- 172.21.15.12:0/2746619146 learned_addr learned my addr 172.21.15.12:0/2746619146 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:46:59.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d6d70640 1 --2- 172.21.15.12:0/2746619146 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61d013f4a0 0x7f61d0134d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:59.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d656f640 1 --2- 172.21.15.12:0/2746619146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61d00c6b80 0x7f61d0159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:59.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d5d6e640 1 -- 172.21.15.12:0/2746619146 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61d013f4a0 msgr2=0x7f61d0134d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:59.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d5d6e640 1 --2- 172.21.15.12:0/2746619146 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61d013f4a0 0x7f61d0134d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:59.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d5d6e640 1 -- 172.21.15.12:0/2746619146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61d00c6b80 msgr2=0x7f61d0159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:59.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d5d6e640 1 --2- 172.21.15.12:0/2746619146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61d00c6b80 0x7f61d0159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:59.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d5d6e640 1 -- 172.21.15.12:0/2746619146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c8009b30 con 0x7f61d00c6f60 2023-10-30T12:46:59.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d5d6e640 1 --2- 172.21.15.12:0/2746619146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61d00c6f60 0x7f61d015f280 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f61c00011f0 tx=0x7f61c000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:59.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61c6ffd640 1 -- 172.21.15.12:0/2746619146 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00074e0 con 0x7f61d00c6f60 2023-10-30T12:46:59.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61c6ffd640 1 -- 172.21.15.12:0/2746619146 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c0007640 con 0x7f61d00c6f60 2023-10-30T12:46:59.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61c6ffd640 1 -- 172.21.15.12:0/2746619146 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0004a10 con 0x7f61d00c6f60 2023-10-30T12:46:59.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d87d3640 1 -- 172.21.15.12:0/2746619146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d01318e0 con 0x7f61d00c6f60 2023-10-30T12:46:59.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d87d3640 1 -- 172.21.15.12:0/2746619146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d0157d30 con 0x7f61d00c6f60 2023-10-30T12:46:59.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.015+0000 7f61d87d3640 1 -- 172.21.15.12:0/2746619146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d0157f20 con 0x7f61d00c6f60 2023-10-30T12:46:59.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.019+0000 7f61c6ffd640 1 -- 172.21.15.12:0/2746619146 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f61c0004b70 con 0x7f61d00c6f60 2023-10-30T12:46:59.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.019+0000 7f61c6ffd640 1 --2- 172.21.15.12:0/2746619146 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f61b40664d0 0x7f61b4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:46:59.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.019+0000 7f61c6ffd640 1 -- 172.21.15.12:0/2746619146 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f61c0083280 con 0x7f61d00c6f60 2023-10-30T12:46:59.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.019+0000 7f61d656f640 1 --2- 172.21.15.12:0/2746619146 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f61b40664d0 0x7f61b4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:46:59.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.019+0000 7f61d656f640 1 --2- 172.21.15.12:0/2746619146 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f61b40664d0 0x7f61b4068990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00b9d0 tx=0x7f61cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:46:59.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.023+0000 7f61c6ffd640 1 -- 172.21.15.12:0/2746619146 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f61c00504d0 con 0x7f61d00c6f60 2023-10-30T12:46:59.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.211+0000 7f61d87d3640 1 -- 172.21.15.12:0/2746619146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61d004e170 con 0x7f61d00c6f60 2023-10-30T12:46:59.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.211+0000 7f61c6ffd640 1 -- 172.21.15.12:0/2746619146 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5744 (secure 0 0 0) 0x7f61c0053af0 con 0x7f61d00c6f60 2023-10-30T12:46:59.217 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:46:59.217 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:56.599607+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:46:59.217 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 60 2023-10-30T12:46:59.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.215+0000 7f61d87d3640 1 -- 172.21.15.12:0/2746619146 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f61b40664d0 msgr2=0x7f61b4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:59.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.215+0000 7f61d87d3640 1 --2- 172.21.15.12:0/2746619146 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f61b40664d0 0x7f61b4068990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00b9d0 tx=0x7f61cc008040 comp rx=0 tx=0).stop 2023-10-30T12:46:59.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.215+0000 7f61d87d3640 1 -- 172.21.15.12:0/2746619146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61d00c6f60 msgr2=0x7f61d015f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:46:59.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.215+0000 7f61d87d3640 1 --2- 172.21.15.12:0/2746619146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61d00c6f60 0x7f61d015f280 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f61c00011f0 tx=0x7f61c000c9b0 comp rx=0 tx=0).stop 2023-10-30T12:46:59.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.215+0000 7f61d87d3640 1 -- 172.21.15.12:0/2746619146 shutdown_connections 2023-10-30T12:46:59.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.215+0000 7f61d87d3640 1 --2- 172.21.15.12:0/2746619146 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f61b40664d0 0x7f61b4068990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:59.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.215+0000 7f61d87d3640 1 --2- 172.21.15.12:0/2746619146 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61d013f4a0 0x7f61d0134d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:59.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.215+0000 7f61d87d3640 1 --2- 172.21.15.12:0/2746619146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61d00c6f60 0x7f61d015f280 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:59.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.215+0000 7f61d87d3640 1 --2- 172.21.15.12:0/2746619146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61d00c6b80 0x7f61d0159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:46:59.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.215+0000 7f61d87d3640 1 -- 172.21.15.12:0/2746619146 >> 172.21.15.12:0/2746619146 conn(0x7f61d00c7910 msgr2=0x7f61d00c10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:46:59.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.215+0000 7f61d87d3640 1 -- 172.21.15.12:0/2746619146 shutdown_connections 2023-10-30T12:46:59.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:46:59.215+0000 7f61d87d3640 1 -- 172.21.15.12:0/2746619146 wait complete. 2023-10-30T12:46:59.238 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:47:01.239 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:01.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.386+0000 7f6f8dff8640 1 Processor -- start 2023-10-30T12:47:01.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.386+0000 7f6f8dff8640 1 -- start start 2023-10-30T12:47:01.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.390+0000 7f6f8dff8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6f8815a410 0x7f6f8806a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:01.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.390+0000 7f6f8dff8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6f8806cb80 0x7f6f8806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:01.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.390+0000 7f6f8dff8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8806d530 con 0x7f6f88069e90 2023-10-30T12:47:01.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.390+0000 7f6f8dff8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f880c8570 con 0x7f6f8815a410 2023-10-30T12:47:01.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.390+0000 7f6f8dff8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f8815a850 con 0x7f6f8806cb80 2023-10-30T12:47:01.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.390+0000 7f6f86ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6f8815a410 0x7f6f8806a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:01.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.390+0000 7f6f877fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f6f88069e90 0x7f6f880c9cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49002/0 (socket says 172.21.15.12:49002) 2023-10-30T12:47:01.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.390+0000 7f6f877fe640 1 -- 172.21.15.12:0/538096 learned_addr learned my addr 172.21.15.12:0/538096 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:47:01.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.390+0000 7f6f87fff640 1 --2- 172.21.15.12:0/538096 >> v2:172.21.15.154:3300/0 conn(0x7f6f8806cb80 0x7f6f8806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:01.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.390+0000 7f6f86ffd640 1 -- 172.21.15.12:0/538096 >> v2:172.21.15.154:3300/0 conn(0x7f6f8806cb80 msgr2=0x7f6f8806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:01.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.390+0000 7f6f86ffd640 1 --2- 172.21.15.12:0/538096 >> v2:172.21.15.154:3300/0 conn(0x7f6f8806cb80 0x7f6f8806cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:01.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.390+0000 7f6f86ffd640 1 -- 172.21.15.12:0/538096 >> v1:172.21.15.12:6789/0 conn(0x7f6f88069e90 legacy=0x7f6f880c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:01.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.390+0000 7f6f86ffd640 1 -- 172.21.15.12:0/538096 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f880c6c10 con 0x7f6f8815a410 2023-10-30T12:47:01.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.390+0000 7f6f867fc640 1 -- 172.21.15.12:0/538096 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3318601896 0 0) 0x7f6f8806d530 con 0x7f6f88069e90 2023-10-30T12:47:01.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f86ffd640 1 --2- 172.21.15.12:0/538096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f8815a410 0x7f6f8806a270 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6f880ca5c0 tx=0x7f6f7c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb209a77a67b905c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:01.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f867fc640 1 -- 172.21.15.12:0/538096 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c0ab600 con 0x7f6f8815a410 2023-10-30T12:47:01.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f867fc640 1 -- 172.21.15.12:0/538096 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c0a7050 con 0x7f6f8815a410 2023-10-30T12:47:01.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f867fc640 1 -- 172.21.15.12:0/538096 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c0ab980 con 0x7f6f8815a410 2023-10-30T12:47:01.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 -- 172.21.15.12:0/538096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f8815a410 msgr2=0x7f6f8806a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:01.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 --2- 172.21.15.12:0/538096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f8815a410 0x7f6f8806a270 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6f880ca5c0 tx=0x7f6f7c01c460 comp rx=0 tx=0).stop 2023-10-30T12:47:01.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 -- 172.21.15.12:0/538096 shutdown_connections 2023-10-30T12:47:01.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 --2- 172.21.15.12:0/538096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f8815a410 0x7f6f8806a270 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:01.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 --2- 172.21.15.12:0/538096 >> v2:172.21.15.154:3300/0 conn(0x7f6f8806cb80 0x7f6f8806cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:01.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 -- 172.21.15.12:0/538096 >> 172.21.15.12:0/538096 conn(0x7f6f8806dfd0 msgr2=0x7f6f8806e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:01.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 -- 172.21.15.12:0/538096 shutdown_connections 2023-10-30T12:47:01.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 -- 172.21.15.12:0/538096 wait complete. 2023-10-30T12:47:01.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 Processor -- start 2023-10-30T12:47:01.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 -- start start 2023-10-30T12:47:01.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f88069e90 0x7f6f88071b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:01.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f8806cb80 0x7f6f8814c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:01.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f8815a410 0x7f6f881409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:01.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8814d5c0 con 0x7f6f8815a410 2023-10-30T12:47:01.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f88146510 con 0x7f6f88069e90 2023-10-30T12:47:01.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.394+0000 7f6f8dff8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f881467f0 con 0x7f6f8806cb80 2023-10-30T12:47:01.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f87fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f8815a410 0x7f6f881409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:01.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f87fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f8815a410 0x7f6f881409a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34540/0 (socket says 172.21.15.12:34540) 2023-10-30T12:47:01.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f87fff640 1 -- 172.21.15.12:0/3533466679 learned_addr learned my addr 172.21.15.12:0/3533466679 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:01.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f877fe640 1 --2- 172.21.15.12:0/3533466679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f88069e90 0x7f6f88071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:01.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f86ffd640 1 --2- 172.21.15.12:0/3533466679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f8806cb80 0x7f6f8814c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:01.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f87fff640 1 -- 172.21.15.12:0/3533466679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f8806cb80 msgr2=0x7f6f8814c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:01.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f87fff640 1 --2- 172.21.15.12:0/3533466679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f8806cb80 0x7f6f8814c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:01.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f87fff640 1 -- 172.21.15.12:0/3533466679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f88069e90 msgr2=0x7f6f88071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:01.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f87fff640 1 --2- 172.21.15.12:0/3533466679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f88069e90 0x7f6f88071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:01.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f87fff640 1 -- 172.21.15.12:0/3533466679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c009420 con 0x7f6f8815a410 2023-10-30T12:47:01.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f87fff640 1 --2- 172.21.15.12:0/3533466679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f8815a410 0x7f6f881409a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6f80007e40 tx=0x7f6f8000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:01.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f6ffff640 1 -- 172.21.15.12:0/3533466679 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80014620 con 0x7f6f8815a410 2023-10-30T12:47:01.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f6ffff640 1 -- 172.21.15.12:0/3533466679 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8000ee90 con 0x7f6f8815a410 2023-10-30T12:47:01.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f6ffff640 1 -- 172.21.15.12:0/3533466679 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f800149e0 con 0x7f6f8815a410 2023-10-30T12:47:01.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f8dff8640 1 -- 172.21.15.12:0/3533466679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f88147920 con 0x7f6f8815a410 2023-10-30T12:47:01.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f8dff8640 1 -- 172.21.15.12:0/3533466679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f88147be0 con 0x7f6f8815a410 2023-10-30T12:47:01.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f8dff8640 1 -- 172.21.15.12:0/3533466679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f88147dd0 con 0x7f6f8815a410 2023-10-30T12:47:01.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.398+0000 7f6f6ffff640 1 -- 172.21.15.12:0/3533466679 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6f80014b40 con 0x7f6f8815a410 2023-10-30T12:47:01.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.402+0000 7f6f6ffff640 1 --2- 172.21.15.12:0/3533466679 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6f740664a0 0x7f6f74068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:01.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.402+0000 7f6f6ffff640 1 -- 172.21.15.12:0/3533466679 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f6f80084a40 con 0x7f6f8815a410 2023-10-30T12:47:01.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.402+0000 7f6f877fe640 1 --2- 172.21.15.12:0/3533466679 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6f740664a0 0x7f6f74068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:01.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.402+0000 7f6f877fe640 1 --2- 172.21.15.12:0/3533466679 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6f740664a0 0x7f6f74068960 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f78007fd0 tx=0x7f6f78007d60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:01.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.406+0000 7f6f6ffff640 1 -- 172.21.15.12:0/3533466679 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6f80051c20 con 0x7f6f8815a410 2023-10-30T12:47:01.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.594+0000 7f6f8dff8640 1 -- 172.21.15.12:0/3533466679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f8804e170 con 0x7f6f8815a410 2023-10-30T12:47:01.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.598+0000 7f6f6ffff640 1 -- 172.21.15.12:0/3533466679 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5744 (secure 0 0 0) 0x7f6f80055240 con 0x7f6f8815a410 2023-10-30T12:47:01.603 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:01.603 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:56.599607+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:01.603 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 60 2023-10-30T12:47:01.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.598+0000 7f6f8dff8640 1 -- 172.21.15.12:0/3533466679 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6f740664a0 msgr2=0x7f6f74068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:01.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.598+0000 7f6f8dff8640 1 --2- 172.21.15.12:0/3533466679 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6f740664a0 0x7f6f74068960 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f78007fd0 tx=0x7f6f78007d60 comp rx=0 tx=0).stop 2023-10-30T12:47:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.598+0000 7f6f8dff8640 1 -- 172.21.15.12:0/3533466679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f8815a410 msgr2=0x7f6f881409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.598+0000 7f6f8dff8640 1 --2- 172.21.15.12:0/3533466679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f8815a410 0x7f6f881409a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6f80007e40 tx=0x7f6f8000e4b0 comp rx=0 tx=0).stop 2023-10-30T12:47:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.598+0000 7f6f8dff8640 1 -- 172.21.15.12:0/3533466679 shutdown_connections 2023-10-30T12:47:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.598+0000 7f6f8dff8640 1 --2- 172.21.15.12:0/3533466679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f8815a410 0x7f6f881409a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.598+0000 7f6f8dff8640 1 --2- 172.21.15.12:0/3533466679 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6f740664a0 0x7f6f74068960 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:01.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.598+0000 7f6f8dff8640 1 --2- 172.21.15.12:0/3533466679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f8806cb80 0x7f6f8814c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:01.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.598+0000 7f6f8dff8640 1 --2- 172.21.15.12:0/3533466679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f88069e90 0x7f6f88071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:01.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.598+0000 7f6f8dff8640 1 -- 172.21.15.12:0/3533466679 >> 172.21.15.12:0/3533466679 conn(0x7f6f8806dfd0 msgr2=0x7f6f8806afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:01.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.598+0000 7f6f8dff8640 1 -- 172.21.15.12:0/3533466679 shutdown_connections 2023-10-30T12:47:01.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:01.598+0000 7f6f8dff8640 1 -- 172.21.15.12:0/3533466679 wait complete. 2023-10-30T12:47:01.624 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:47:03.625 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:03.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.774+0000 7fefb95b8640 1 Processor -- start 2023-10-30T12:47:03.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.774+0000 7fefb95b8640 1 -- start start 2023-10-30T12:47:03.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb95b8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fefb40cf160 0x7fefb40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:03.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb95b8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb40cb800 con 0x7fefb41538a0 2023-10-30T12:47:03.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb95b8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb40cb330 con 0x7fefb415d6f0 2023-10-30T12:47:03.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb95b8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb40cad70 con 0x7fefb40cf160 2023-10-30T12:47:03.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb2ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fefb41538a0 0x7fefb40ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49016/0 (socket says 172.21.15.12:49016) 2023-10-30T12:47:03.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb2ffd640 1 -- 172.21.15.12:0/753518866 learned_addr learned my addr 172.21.15.12:0/753518866 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:47:03.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb37fe640 1 --2- 172.21.15.12:0/753518866 >> v2:172.21.15.154:3300/0 conn(0x7fefb40cf160 0x7fefb40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:03.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb27fc640 1 -- 172.21.15.12:0/753518866 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009952515 0 0) 0x7fefb40cb800 con 0x7fefb41538a0 2023-10-30T12:47:03.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb27fc640 1 -- 172.21.15.12:0/753518866 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefa0003530 con 0x7fefb41538a0 2023-10-30T12:47:03.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb27fc640 1 -- 172.21.15.12:0/753518866 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1604277596 0 0) 0x7fefa0003530 con 0x7fefb41538a0 2023-10-30T12:47:03.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb27fc640 1 -- 172.21.15.12:0/753518866 >> v2:172.21.15.154:3300/0 conn(0x7fefb40cf160 msgr2=0x7fefb40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:03.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb27fc640 1 --2- 172.21.15.12:0/753518866 >> v2:172.21.15.154:3300/0 conn(0x7fefb40cf160 0x7fefb40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:03.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb27fc640 1 -- 172.21.15.12:0/753518866 >> v1:172.21.15.154:6789/0 conn(0x7fefb415d6f0 legacy=0x7fefb415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:03.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb27fc640 1 -- 172.21.15.12:0/753518866 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb40cb050 con 0x7fefb41538a0 2023-10-30T12:47:03.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb27fc640 1 -- 172.21.15.12:0/753518866 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fefa4001680 con 0x7fefb41538a0 2023-10-30T12:47:03.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb27fc640 1 -- 172.21.15.12:0/753518866 >> v1:172.21.15.12:6789/0 conn(0x7fefb41538a0 legacy=0x7fefb40ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:03.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb27fc640 1 --2- 172.21.15.12:0/753518866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa0003980 0x7fefa0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:03.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb27fc640 1 --2- 172.21.15.12:0/753518866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefa00047d0 0x7fefa0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:03.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb27fc640 1 --2- 172.21.15.12:0/753518866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa0005190 0x7fefa000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:03.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb27fc640 1 -- 172.21.15.12:0/753518866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40cad70 con 0x7fefa0005190 2023-10-30T12:47:03.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb27fc640 1 -- 172.21.15.12:0/753518866 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa000c040 con 0x7fefa00047d0 2023-10-30T12:47:03.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb27fc640 1 -- 172.21.15.12:0/753518866 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa000c200 con 0x7fefa0003980 2023-10-30T12:47:03.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb37fe640 1 --2- 172.21.15.12:0/753518866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa0005190 0x7fefa000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:03.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb2ffd640 1 --2- 172.21.15.12:0/753518866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa0003980 0x7fefa0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:03.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefabfff640 1 --2- 172.21.15.12:0/753518866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefa00047d0 0x7fefa0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:03.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb37fe640 1 -- 172.21.15.12:0/753518866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa0003980 msgr2=0x7fefa0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:03.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb37fe640 1 --2- 172.21.15.12:0/753518866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa0003980 0x7fefa0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:03.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb37fe640 1 -- 172.21.15.12:0/753518866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefa00047d0 msgr2=0x7fefa0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:03.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb37fe640 1 --2- 172.21.15.12:0/753518866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefa00047d0 0x7fefa0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:03.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.778+0000 7fefb37fe640 1 -- 172.21.15.12:0/753518866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa0003530 con 0x7fefa0005190 2023-10-30T12:47:03.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.782+0000 7fefb37fe640 1 -- 172.21.15.12:0/753518866 reap_dead start 2023-10-30T12:47:03.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.782+0000 7fefb37fe640 1 --2- 172.21.15.12:0/753518866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa0005190 0x7fefa000b990 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fefac009fc0 tx=0x7fefac01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6097b0658b4593c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:03.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.782+0000 7fefb27fc640 1 -- 172.21.15.12:0/753518866 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac0a5cb0 con 0x7fefa0005190 2023-10-30T12:47:03.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.782+0000 7fefb27fc640 1 -- 172.21.15.12:0/753518866 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefac00f040 con 0x7fefa0005190 2023-10-30T12:47:03.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.782+0000 7fefb27fc640 1 -- 172.21.15.12:0/753518866 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac003be0 con 0x7fefa0005190 2023-10-30T12:47:03.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.782+0000 7fefb95b8640 1 -- 172.21.15.12:0/753518866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa0005190 msgr2=0x7fefa000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:03.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.782+0000 7fefb95b8640 1 --2- 172.21.15.12:0/753518866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa0005190 0x7fefa000b990 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fefac009fc0 tx=0x7fefac01cca0 comp rx=0 tx=0).stop 2023-10-30T12:47:03.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.782+0000 7fefb95b8640 1 -- 172.21.15.12:0/753518866 shutdown_connections 2023-10-30T12:47:03.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.782+0000 7fefb95b8640 1 --2- 172.21.15.12:0/753518866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa0005190 0x7fefa000b990 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:03.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.782+0000 7fefb95b8640 1 -- 172.21.15.12:0/753518866 >> 172.21.15.12:0/753518866 conn(0x7fefb406bfe0 msgr2=0x7fefb406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:03.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.782+0000 7fefb95b8640 1 -- 172.21.15.12:0/753518866 shutdown_connections 2023-10-30T12:47:03.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.782+0000 7fefb95b8640 1 -- 172.21.15.12:0/753518866 wait complete. 2023-10-30T12:47:03.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.782+0000 7fefb95b8640 1 Processor -- start 2023-10-30T12:47:03.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.782+0000 7fefb95b8640 1 -- start start 2023-10-30T12:47:03.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb95b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa0005190 0x7fefb40c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:03.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb95b8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefb40ce6d0 0x7fefb4151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:03.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb95b8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefb40c9210 0x7fefb415b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:03.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb95b8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb4151b10 con 0x7fefa0005190 2023-10-30T12:47:03.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb95b8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40ceef0 con 0x7fefb40ce6d0 2023-10-30T12:47:03.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb95b8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40cec10 con 0x7fefb40c9210 2023-10-30T12:47:03.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa0005190 0x7fefb40c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:03.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa0005190 0x7fefb40c64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34570/0 (socket says 172.21.15.12:34570) 2023-10-30T12:47:03.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb2ffd640 1 -- 172.21.15.12:0/407777591 learned_addr learned my addr 172.21.15.12:0/407777591 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:03.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb37fe640 1 --2- 172.21.15.12:0/407777591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefb40c9210 0x7fefb415b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:03.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefabfff640 1 --2- 172.21.15.12:0/407777591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefb40ce6d0 0x7fefb4151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:03.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb2ffd640 1 -- 172.21.15.12:0/407777591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefb40c9210 msgr2=0x7fefb415b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:03.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb2ffd640 1 --2- 172.21.15.12:0/407777591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefb40c9210 0x7fefb415b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:03.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb2ffd640 1 -- 172.21.15.12:0/407777591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefb40ce6d0 msgr2=0x7fefb4151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:03.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb2ffd640 1 --2- 172.21.15.12:0/407777591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefb40ce6d0 0x7fefb4151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:03.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb2ffd640 1 -- 172.21.15.12:0/407777591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb41511e0 con 0x7fefa0005190 2023-10-30T12:47:03.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefabfff640 1 --2- 172.21.15.12:0/407777591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefb40ce6d0 0x7fefb4151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:03.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb2ffd640 1 --2- 172.21.15.12:0/407777591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa0005190 0x7fefb40c64d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fefa400e780 tx=0x7fefa400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:03.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefab7fe640 1 -- 172.21.15.12:0/407777591 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa400c6c0 con 0x7fefa0005190 2023-10-30T12:47:03.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefab7fe640 1 -- 172.21.15.12:0/407777591 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa400c820 con 0x7fefa0005190 2023-10-30T12:47:03.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefab7fe640 1 -- 172.21.15.12:0/407777591 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa400cbd0 con 0x7fefa0005190 2023-10-30T12:47:03.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb95b8640 1 -- 172.21.15.12:0/407777591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb41514c0 con 0x7fefa0005190 2023-10-30T12:47:03.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb95b8640 1 -- 172.21.15.12:0/407777591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb40c4400 con 0x7fefa0005190 2023-10-30T12:47:03.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.786+0000 7fefb95b8640 1 -- 172.21.15.12:0/407777591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb404e170 con 0x7fefa0005190 2023-10-30T12:47:03.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.790+0000 7fefab7fe640 1 -- 172.21.15.12:0/407777591 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fefa400cd30 con 0x7fefa0005190 2023-10-30T12:47:03.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.790+0000 7fefab7fe640 1 --2- 172.21.15.12:0/407777591 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fef940664d0 0x7fef94068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.790+0000 7fefab7fe640 1 -- 172.21.15.12:0/407777591 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fefa4083ee0 con 0x7fefa0005190 2023-10-30T12:47:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.790+0000 7fefabfff640 1 --2- 172.21.15.12:0/407777591 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fef940664d0 0x7fef94068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.790+0000 7fefabfff640 1 --2- 172.21.15.12:0/407777591 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fef940664d0 0x7fef94068990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fefb414ce60 tx=0x7fef9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.794+0000 7fefab7fe640 1 -- 172.21.15.12:0/407777591 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fefa4051140 con 0x7fefa0005190 2023-10-30T12:47:03.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.986+0000 7fefb95b8640 1 -- 172.21.15.12:0/407777591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefb40b5080 con 0x7fefa0005190 2023-10-30T12:47:03.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.986+0000 7fefab7fe640 1 -- 172.21.15.12:0/407777591 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5744 (secure 0 0 0) 0x7fefa4054760 con 0x7fefa0005190 2023-10-30T12:47:03.994 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:03.994 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:46:56.599607+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:03.994 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 60 2023-10-30T12:47:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.990+0000 7fefb95b8640 1 -- 172.21.15.12:0/407777591 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fef940664d0 msgr2=0x7fef94068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.990+0000 7fefb95b8640 1 --2- 172.21.15.12:0/407777591 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fef940664d0 0x7fef94068990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fefb414ce60 tx=0x7fef9c009040 comp rx=0 tx=0).stop 2023-10-30T12:47:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.990+0000 7fefb95b8640 1 -- 172.21.15.12:0/407777591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa0005190 msgr2=0x7fefb40c64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.990+0000 7fefb95b8640 1 --2- 172.21.15.12:0/407777591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa0005190 0x7fefb40c64d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fefa400e780 tx=0x7fefa400eb80 comp rx=0 tx=0).stop 2023-10-30T12:47:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.990+0000 7fefb95b8640 1 -- 172.21.15.12:0/407777591 shutdown_connections 2023-10-30T12:47:03.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.990+0000 7fefb95b8640 1 --2- 172.21.15.12:0/407777591 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fef940664d0 0x7fef94068990 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:03.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.990+0000 7fefb95b8640 1 --2- 172.21.15.12:0/407777591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefb40c9210 0x7fefb415b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:03.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.990+0000 7fefb95b8640 1 --2- 172.21.15.12:0/407777591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefb40ce6d0 0x7fefb4151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:03.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.990+0000 7fefb95b8640 1 --2- 172.21.15.12:0/407777591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefa0005190 0x7fefb40c64d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:03.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.990+0000 7fefb95b8640 1 -- 172.21.15.12:0/407777591 >> 172.21.15.12:0/407777591 conn(0x7fefb406bfe0 msgr2=0x7fefb40c55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:03.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.990+0000 7fefb95b8640 1 -- 172.21.15.12:0/407777591 shutdown_connections 2023-10-30T12:47:03.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:03.990+0000 7fefb95b8640 1 -- 172.21.15.12:0/407777591 wait complete. 2023-10-30T12:47:04.015 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:47:05.592 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:47:05.597 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:47:06.017 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:06.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc53e50f640 1 Processor -- start 2023-10-30T12:47:06.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc53e50f640 1 -- start start 2023-10-30T12:47:06.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc53e50f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc53806f7a0 0x7fc53806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc53e50f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc538066cf0 con 0x7fc538071110 2023-10-30T12:47:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc53e50f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5380c70d0 con 0x7fc5380c6bd0 2023-10-30T12:47:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc53e50f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5380700c0 con 0x7fc53806f7a0 2023-10-30T12:47:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc53caac640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc538071110 0x7fc538158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49018/0 (socket says 172.21.15.12:49018) 2023-10-30T12:47:06.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc53caac640 1 -- 172.21.15.12:0/1954759332 learned_addr learned my addr 172.21.15.12:0/1954759332 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:47:06.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc537fff640 1 --2- 172.21.15.12:0/1954759332 >> v2:172.21.15.154:3300/0 conn(0x7fc53806f7a0 0x7fc53806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc536ffd640 1 -- 172.21.15.12:0/1954759332 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470519838 0 0) 0x7fc538066cf0 con 0x7fc538071110 2023-10-30T12:47:06.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc536ffd640 1 -- 172.21.15.12:0/1954759332 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc520003530 con 0x7fc538071110 2023-10-30T12:47:06.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc536ffd640 1 -- 172.21.15.12:0/1954759332 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192206721 0 0) 0x7fc5380c70d0 con 0x7fc5380c6bd0 2023-10-30T12:47:06.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc536ffd640 1 -- 172.21.15.12:0/1954759332 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc538066cf0 con 0x7fc5380c6bd0 2023-10-30T12:47:06.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc536ffd640 1 -- 172.21.15.12:0/1954759332 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2902611421 0 0) 0x7fc520003530 con 0x7fc538071110 2023-10-30T12:47:06.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc537fff640 1 -- 172.21.15.12:0/1954759332 >> v1:172.21.15.154:6789/0 conn(0x7fc5380c6bd0 legacy=0x7fc538069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc537fff640 1 -- 172.21.15.12:0/1954759332 >> v1:172.21.15.12:6789/0 conn(0x7fc538071110 legacy=0x7fc538158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.166+0000 7fc537fff640 1 -- 172.21.15.12:0/1954759332 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5380c7390 con 0x7fc53806f7a0 2023-10-30T12:47:06.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.170+0000 7fc537fff640 1 --2- 172.21.15.12:0/1954759332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc53806f7a0 0x7fc53806fb80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc5280097c0 tx=0x7fc52801eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3be100e9728280b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.170+0000 7fc536ffd640 1 -- 172.21.15.12:0/1954759332 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc530001a00 con 0x7fc53806f7a0 2023-10-30T12:47:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.170+0000 7fc536ffd640 1 -- 172.21.15.12:0/1954759332 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5280a8610 con 0x7fc53806f7a0 2023-10-30T12:47:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.170+0000 7fc536ffd640 1 -- 172.21.15.12:0/1954759332 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5280a89a0 con 0x7fc53806f7a0 2023-10-30T12:47:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.170+0000 7fc53e50f640 1 -- 172.21.15.12:0/1954759332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc53806f7a0 msgr2=0x7fc53806fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.170+0000 7fc53e50f640 1 --2- 172.21.15.12:0/1954759332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc53806f7a0 0x7fc53806fb80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc5280097c0 tx=0x7fc52801eb90 comp rx=0 tx=0).stop 2023-10-30T12:47:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.170+0000 7fc53e50f640 1 -- 172.21.15.12:0/1954759332 shutdown_connections 2023-10-30T12:47:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.170+0000 7fc53e50f640 1 --2- 172.21.15.12:0/1954759332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc53806f7a0 0x7fc53806fb80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.170+0000 7fc53e50f640 1 -- 172.21.15.12:0/1954759332 >> 172.21.15.12:0/1954759332 conn(0x7fc5380c7910 msgr2=0x7fc5380c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.170+0000 7fc53e50f640 1 -- 172.21.15.12:0/1954759332 shutdown_connections 2023-10-30T12:47:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.170+0000 7fc53e50f640 1 -- 172.21.15.12:0/1954759332 wait complete. 2023-10-30T12:47:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc53e50f640 1 Processor -- start 2023-10-30T12:47:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc53e50f640 1 -- start start 2023-10-30T12:47:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc53e50f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc53806f7a0 0x7fc538155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc53e50f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc538071110 0x7fc5381549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc53e50f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5380c6bd0 0x7fc538150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc53e50f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53814fcf0 con 0x7fc53806f7a0 2023-10-30T12:47:06.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc53e50f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc538155c00 con 0x7fc538071110 2023-10-30T12:47:06.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc53e50f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53814f0e0 con 0x7fc5380c6bd0 2023-10-30T12:47:06.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc537fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc53806f7a0 0x7fc538155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc537fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc53806f7a0 0x7fc538155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34580/0 (socket says 172.21.15.12:34580) 2023-10-30T12:47:06.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc537fff640 1 -- 172.21.15.12:0/2917135953 learned_addr learned my addr 172.21.15.12:0/2917135953 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:06.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc5377fe640 1 --2- 172.21.15.12:0/2917135953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc538071110 0x7fc5381549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc53caac640 1 --2- 172.21.15.12:0/2917135953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5380c6bd0 0x7fc538150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc537fff640 1 -- 172.21.15.12:0/2917135953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5380c6bd0 msgr2=0x7fc538150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc537fff640 1 --2- 172.21.15.12:0/2917135953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5380c6bd0 0x7fc538150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc537fff640 1 -- 172.21.15.12:0/2917135953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc538071110 msgr2=0x7fc5381549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc537fff640 1 --2- 172.21.15.12:0/2917135953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc538071110 0x7fc5381549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc537fff640 1 -- 172.21.15.12:0/2917135953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc528009420 con 0x7fc53806f7a0 2023-10-30T12:47:06.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc537fff640 1 --2- 172.21.15.12:0/2917135953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc53806f7a0 0x7fc538155550 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc5280097c0 tx=0x7fc52801b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:06.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc534ff9640 1 -- 172.21.15.12:0/2917135953 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528004220 con 0x7fc53806f7a0 2023-10-30T12:47:06.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc534ff9640 1 -- 172.21.15.12:0/2917135953 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc528004380 con 0x7fc53806f7a0 2023-10-30T12:47:06.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc53e50f640 1 -- 172.21.15.12:0/2917135953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53814f3c0 con 0x7fc53806f7a0 2023-10-30T12:47:06.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc534ff9640 1 -- 172.21.15.12:0/2917135953 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5280b0890 con 0x7fc53806f7a0 2023-10-30T12:47:06.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.174+0000 7fc53e50f640 1 -- 172.21.15.12:0/2917135953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53814f980 con 0x7fc53806f7a0 2023-10-30T12:47:06.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.178+0000 7fc53e50f640 1 -- 172.21.15.12:0/2917135953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc53804e170 con 0x7fc53806f7a0 2023-10-30T12:47:06.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.178+0000 7fc534ff9640 1 -- 172.21.15.12:0/2917135953 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc5280b7040 con 0x7fc53806f7a0 2023-10-30T12:47:06.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.178+0000 7fc534ff9640 1 --2- 172.21.15.12:0/2917135953 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc5240664a0 0x7fc524068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.178+0000 7fc534ff9640 1 -- 172.21.15.12:0/2917135953 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fc528120960 con 0x7fc53806f7a0 2023-10-30T12:47:06.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.178+0000 7fc5377fe640 1 --2- 172.21.15.12:0/2917135953 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc5240664a0 0x7fc524068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.178+0000 7fc5377fe640 1 --2- 172.21.15.12:0/2917135953 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc5240664a0 0x7fc524068960 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc5381e70c0 tx=0x7fc52c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:06.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.182+0000 7fc534ff9640 1 -- 172.21.15.12:0/2917135953 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc5280eda90 con 0x7fc53806f7a0 2023-10-30T12:47:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.378+0000 7fc53e50f640 1 -- 172.21.15.12:0/2917135953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5380caa70 con 0x7fc53806f7a0 2023-10-30T12:47:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.378+0000 7fc534ff9640 1 -- 172.21.15.12:0/2917135953 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5744 (secure 0 0 0) 0x7fc5280b5020 con 0x7fc53806f7a0 2023-10-30T12:47:06.383 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:06.383 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:03.591793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:06.383 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 61 2023-10-30T12:47:06.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.378+0000 7fc53e50f640 1 -- 172.21.15.12:0/2917135953 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc5240664a0 msgr2=0x7fc524068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.378+0000 7fc53e50f640 1 --2- 172.21.15.12:0/2917135953 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc5240664a0 0x7fc524068960 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc5381e70c0 tx=0x7fc52c009040 comp rx=0 tx=0).stop 2023-10-30T12:47:06.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.378+0000 7fc53e50f640 1 -- 172.21.15.12:0/2917135953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc53806f7a0 msgr2=0x7fc538155550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.378+0000 7fc53e50f640 1 --2- 172.21.15.12:0/2917135953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc53806f7a0 0x7fc538155550 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc5280097c0 tx=0x7fc52801b930 comp rx=0 tx=0).stop 2023-10-30T12:47:06.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.378+0000 7fc53e50f640 1 -- 172.21.15.12:0/2917135953 shutdown_connections 2023-10-30T12:47:06.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.378+0000 7fc53e50f640 1 --2- 172.21.15.12:0/2917135953 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc5240664a0 0x7fc524068960 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.378+0000 7fc53e50f640 1 --2- 172.21.15.12:0/2917135953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5380c6bd0 0x7fc538150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.378+0000 7fc53e50f640 1 --2- 172.21.15.12:0/2917135953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc538071110 0x7fc5381549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.378+0000 7fc53e50f640 1 --2- 172.21.15.12:0/2917135953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc53806f7a0 0x7fc538155550 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.378+0000 7fc53e50f640 1 -- 172.21.15.12:0/2917135953 >> 172.21.15.12:0/2917135953 conn(0x7fc5380c7910 msgr2=0x7fc5380c9200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:06.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.378+0000 7fc53e50f640 1 -- 172.21.15.12:0/2917135953 shutdown_connections 2023-10-30T12:47:06.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:06.378+0000 7fc53e50f640 1 -- 172.21.15.12:0/2917135953 wait complete. 2023-10-30T12:47:06.404 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=2, replacing gid=4971 2023-10-30T12:47:06.404 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.c 2023-10-30T12:47:07.405 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2023-10-30T12:47:07.405 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-30T12:47:07.405 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-30T12:47:07.409 INFO:tasks.ceph.mds.c:Started 2023-10-30T12:47:07.410 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:07.507 INFO:tasks.ceph.mds.c.smithi012.stdout:starting mds.c at 2023-10-30T12:47:07.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.558+0000 7f40291aa640 1 Processor -- start 2023-10-30T12:47:07.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.558+0000 7f40291aa640 1 -- start start 2023-10-30T12:47:07.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.558+0000 7f40291aa640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f40241538a0 0x7f402415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:07.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.558+0000 7f40291aa640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f40240cf160 0x7f40240cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:07.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.558+0000 7f40291aa640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40240cb800 con 0x7f40240ca0e0 2023-10-30T12:47:07.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.558+0000 7f40291aa640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402415dc30 con 0x7f40240cf160 2023-10-30T12:47:07.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.558+0000 7f40291aa640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40240cb330 con 0x7f40241538a0 2023-10-30T12:47:07.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.558+0000 7f402359e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f40240cf160 0x7f40240cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:07.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.558+0000 7f402259c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f40240ca0e0 0x7f402415a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49032/0 (socket says 172.21.15.12:49032) 2023-10-30T12:47:07.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.558+0000 7f402259c640 1 -- 172.21.15.12:0/3109064876 learned_addr learned my addr 172.21.15.12:0/3109064876 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:47:07.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.558+0000 7f4022d9d640 1 --2- 172.21.15.12:0/3109064876 >> v2:172.21.15.154:3300/0 conn(0x7f40241538a0 0x7f402415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:07.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.562+0000 7f4021d9b640 1 -- 172.21.15.12:0/3109064876 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925784932 0 0) 0x7f40240cb800 con 0x7f40240ca0e0 2023-10-30T12:47:07.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.562+0000 7f402359e640 1 -- 172.21.15.12:0/3109064876 >> v2:172.21.15.154:3300/0 conn(0x7f40241538a0 msgr2=0x7f402415d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:07.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.562+0000 7f402359e640 1 --2- 172.21.15.12:0/3109064876 >> v2:172.21.15.154:3300/0 conn(0x7f40241538a0 0x7f402415d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:07.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.562+0000 7f402359e640 1 -- 172.21.15.12:0/3109064876 >> v1:172.21.15.12:6789/0 conn(0x7f40240ca0e0 legacy=0x7f402415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:07.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.562+0000 7f402359e640 1 -- 172.21.15.12:0/3109064876 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40240cad70 con 0x7f40240cf160 2023-10-30T12:47:07.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.562+0000 7f402359e640 1 --2- 172.21.15.12:0/3109064876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40240cf160 0x7f40240cf590 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f40240cfd10 tx=0x7f401c01f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82393ce4eeee3cc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:07.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.562+0000 7f4021d9b640 1 -- 172.21.15.12:0/3109064876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c0b0440 con 0x7f40240cf160 2023-10-30T12:47:07.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.562+0000 7f4021d9b640 1 -- 172.21.15.12:0/3109064876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401c0b05a0 con 0x7f40240cf160 2023-10-30T12:47:07.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.562+0000 7f4021d9b640 1 -- 172.21.15.12:0/3109064876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c0b0930 con 0x7f40240cf160 2023-10-30T12:47:07.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.562+0000 7f40291aa640 1 -- 172.21.15.12:0/3109064876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40240cf160 msgr2=0x7f40240cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:07.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.562+0000 7f40291aa640 1 --2- 172.21.15.12:0/3109064876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40240cf160 0x7f40240cf590 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f40240cfd10 tx=0x7f401c01f0f0 comp rx=0 tx=0).stop 2023-10-30T12:47:07.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.562+0000 7f40291aa640 1 -- 172.21.15.12:0/3109064876 shutdown_connections 2023-10-30T12:47:07.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.562+0000 7f40291aa640 1 --2- 172.21.15.12:0/3109064876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40240cf160 0x7f40240cf590 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:07.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.562+0000 7f40291aa640 1 --2- 172.21.15.12:0/3109064876 >> v2:172.21.15.154:3300/0 conn(0x7f40241538a0 0x7f402415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:07.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.562+0000 7f40291aa640 1 -- 172.21.15.12:0/3109064876 >> 172.21.15.12:0/3109064876 conn(0x7f402406bfe0 msgr2=0x7f402406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:07.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f40291aa640 1 -- 172.21.15.12:0/3109064876 shutdown_connections 2023-10-30T12:47:07.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f40291aa640 1 -- 172.21.15.12:0/3109064876 wait complete. 2023-10-30T12:47:07.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f40291aa640 1 Processor -- start 2023-10-30T12:47:07.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f40291aa640 1 -- start start 2023-10-30T12:47:07.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f40291aa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40240ca0e0 0x7f40240b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:07.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f40291aa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40240cf160 0x7f40241490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:07.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f40291aa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40241538a0 0x7f40241efe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:07.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f40291aa640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40240b4940 con 0x7f40240cf160 2023-10-30T12:47:07.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f40291aa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4024148b30 con 0x7f40241538a0 2023-10-30T12:47:07.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f40291aa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40240b3a50 con 0x7f40240ca0e0 2023-10-30T12:47:07.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f402259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40240cf160 0x7f40241490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:07.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f402259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40240cf160 0x7f40241490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34600/0 (socket says 172.21.15.12:34600) 2023-10-30T12:47:07.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f402259c640 1 -- 172.21.15.12:0/1764392111 learned_addr learned my addr 172.21.15.12:0/1764392111 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:07.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f402359e640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40241538a0 0x7f40241efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:07.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f4022d9d640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40240ca0e0 0x7f40240b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:07.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f402259c640 1 -- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40240ca0e0 msgr2=0x7f40240b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:07.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f402259c640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40240ca0e0 0x7f40240b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:07.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f402259c640 1 -- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40241538a0 msgr2=0x7f40241efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:07.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f402259c640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40241538a0 0x7f40241efe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:07.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f402259c640 1 -- 172.21.15.12:0/1764392111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c009a70 con 0x7f40240cf160 2023-10-30T12:47:07.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f402359e640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40241538a0 0x7f40241efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:07.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.566+0000 7f4022d9d640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40240ca0e0 0x7f40240b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:07.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.570+0000 7f402259c640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40240cf160 0x7f40241490b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f401800c4b0 tx=0x7f401800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.570+0000 7f400b7fe640 1 -- 172.21.15.12:0/1764392111 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40180074e0 con 0x7f40240cf160 2023-10-30T12:47:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.570+0000 7f400b7fe640 1 -- 172.21.15.12:0/1764392111 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4018007660 con 0x7f40240cf160 2023-10-30T12:47:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.570+0000 7f400b7fe640 1 -- 172.21.15.12:0/1764392111 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018004a30 con 0x7f40240cf160 2023-10-30T12:47:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.570+0000 7f40291aa640 1 -- 172.21.15.12:0/1764392111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40240b3d30 con 0x7f40240cf160 2023-10-30T12:47:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.570+0000 7f40291aa640 1 -- 172.21.15.12:0/1764392111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40240b42f0 con 0x7f40240cf160 2023-10-30T12:47:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.570+0000 7f40291aa640 1 -- 172.21.15.12:0/1764392111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40240b2a50 con 0x7f40240cf160 2023-10-30T12:47:07.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.570+0000 7f400b7fe640 1 -- 172.21.15.12:0/1764392111 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4018004b90 con 0x7f40240cf160 2023-10-30T12:47:07.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.570+0000 7f400b7fe640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f40100664a0 0x7f4010068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:07.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.570+0000 7f400b7fe640 1 -- 172.21.15.12:0/1764392111 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f4018083350 con 0x7f40240cf160 2023-10-30T12:47:07.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.574+0000 7f4022d9d640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f40100664a0 0x7f4010068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:07.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.574+0000 7f4022d9d640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f40100664a0 0x7f4010068960 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4014009bb0 tx=0x7f4014008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:07.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.574+0000 7f400b7fe640 1 -- 172.21.15.12:0/1764392111 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4018053f10 con 0x7f40240cf160 2023-10-30T12:47:07.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.762+0000 7f40291aa640 1 -- 172.21.15.12:0/1764392111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40240cb020 con 0x7f40240cf160 2023-10-30T12:47:07.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.766+0000 7f400b7fe640 1 -- 172.21.15.12:0/1764392111 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5744 (secure 0 0 0) 0x7f4018018090 con 0x7f40240cf160 2023-10-30T12:47:07.770 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:07.770 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:03.591793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:07.771 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 61 2023-10-30T12:47:07.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.766+0000 7f40291aa640 1 -- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f40100664a0 msgr2=0x7f4010068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:07.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.766+0000 7f40291aa640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f40100664a0 0x7f4010068960 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4014009bb0 tx=0x7f4014008040 comp rx=0 tx=0).stop 2023-10-30T12:47:07.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.766+0000 7f40291aa640 1 -- 172.21.15.12:0/1764392111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40240cf160 msgr2=0x7f40241490b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:07.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.766+0000 7f40291aa640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40240cf160 0x7f40241490b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f401800c4b0 tx=0x7f401800c970 comp rx=0 tx=0).stop 2023-10-30T12:47:07.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.766+0000 7f40291aa640 1 -- 172.21.15.12:0/1764392111 shutdown_connections 2023-10-30T12:47:07.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.766+0000 7f40291aa640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40241538a0 0x7f40241efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:07.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.766+0000 7f40291aa640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40240cf160 0x7f40241490b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:07.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.766+0000 7f40291aa640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f40100664a0 0x7f4010068960 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:07.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.766+0000 7f40291aa640 1 --2- 172.21.15.12:0/1764392111 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40240ca0e0 0x7f40240b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:07.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.766+0000 7f40291aa640 1 -- 172.21.15.12:0/1764392111 >> 172.21.15.12:0/1764392111 conn(0x7f402406bfe0 msgr2=0x7f4024065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:07.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.766+0000 7f40291aa640 1 -- 172.21.15.12:0/1764392111 shutdown_connections 2023-10-30T12:47:07.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:07.766+0000 7f40291aa640 1 -- 172.21.15.12:0/1764392111 wait complete. 2023-10-30T12:47:07.792 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2023-10-30T12:47:09.793 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:09.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.934+0000 7f9f2d688640 1 Processor -- start 2023-10-30T12:47:09.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.934+0000 7f9f2d688640 1 -- start start 2023-10-30T12:47:09.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.934+0000 7f9f2d688640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9f280cefa0 0x7f9f280cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:09.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.934+0000 7f9f2d688640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2815a6c0 con 0x7f9f281538f0 2023-10-30T12:47:09.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.934+0000 7f9f2d688640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f280ca6a0 con 0x7f9f2815cd50 2023-10-30T12:47:09.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.934+0000 7f9f2d688640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f280cf8f0 con 0x7f9f280cefa0 2023-10-30T12:47:09.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f27fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9f281538f0 0x7f9f28159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49034/0 (socket says 172.21.15.12:49034) 2023-10-30T12:47:09.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f27fff640 1 -- 172.21.15.12:0/2868578912 learned_addr learned my addr 172.21.15.12:0/2868578912 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:47:09.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f2ce87640 1 --2- 172.21.15.12:0/2868578912 >> v2:172.21.15.154:3300/0 conn(0x7f9f280cefa0 0x7f9f280cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:09.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f26ffd640 1 -- 172.21.15.12:0/2868578912 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596636835 0 0) 0x7f9f2815a6c0 con 0x7f9f281538f0 2023-10-30T12:47:09.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f26ffd640 1 -- 172.21.15.12:0/2868578912 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f10003530 con 0x7f9f281538f0 2023-10-30T12:47:09.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f26ffd640 1 -- 172.21.15.12:0/2868578912 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3076515852 0 0) 0x7f9f10003530 con 0x7f9f281538f0 2023-10-30T12:47:09.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f26ffd640 1 -- 172.21.15.12:0/2868578912 >> v2:172.21.15.154:3300/0 conn(0x7f9f280cefa0 msgr2=0x7f9f280cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:09.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f26ffd640 1 --2- 172.21.15.12:0/2868578912 >> v2:172.21.15.154:3300/0 conn(0x7f9f280cefa0 0x7f9f280cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f26ffd640 1 -- 172.21.15.12:0/2868578912 >> v1:172.21.15.154:6789/0 conn(0x7f9f2815cd50 legacy=0x7f9f280c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:09.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f26ffd640 1 -- 172.21.15.12:0/2868578912 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2815d300 con 0x7f9f281538f0 2023-10-30T12:47:09.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f26ffd640 1 -- 172.21.15.12:0/2868578912 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f9f18001680 con 0x7f9f281538f0 2023-10-30T12:47:09.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f2ce87640 1 --2- 172.21.15.12:0/2868578912 >> v2:172.21.15.154:3300/0 conn(0x7f9f280cefa0 0x7f9f280cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:47:09.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f26ffd640 1 -- 172.21.15.12:0/2868578912 >> v1:172.21.15.12:6789/0 conn(0x7f9f281538f0 legacy=0x7f9f28159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:09.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f26ffd640 1 --2- 172.21.15.12:0/2868578912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10003980 0x7f9f10003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:09.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f26ffd640 1 --2- 172.21.15.12:0/2868578912 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f100047d0 0x7f9f10004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:09.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f26ffd640 1 --2- 172.21.15.12:0/2868578912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f10005190 0x7f9f1000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:09.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f26ffd640 1 -- 172.21.15.12:0/2868578912 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280cf8f0 con 0x7f9f10005190 2023-10-30T12:47:09.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f26ffd640 1 -- 172.21.15.12:0/2868578912 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1000c040 con 0x7f9f10003980 2023-10-30T12:47:09.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f26ffd640 1 -- 172.21.15.12:0/2868578912 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1000c200 con 0x7f9f100047d0 2023-10-30T12:47:09.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f2ce87640 1 --2- 172.21.15.12:0/2868578912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f10005190 0x7f9f1000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:09.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f27fff640 1 --2- 172.21.15.12:0/2868578912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10003980 0x7f9f10003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:09.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f277fe640 1 --2- 172.21.15.12:0/2868578912 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f100047d0 0x7f9f10004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:09.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f2ce87640 1 -- 172.21.15.12:0/2868578912 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f100047d0 msgr2=0x7f9f10004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:09.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f2ce87640 1 --2- 172.21.15.12:0/2868578912 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f100047d0 0x7f9f10004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f2ce87640 1 -- 172.21.15.12:0/2868578912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10003980 msgr2=0x7f9f10003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:09.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f2ce87640 1 --2- 172.21.15.12:0/2868578912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10003980 0x7f9f10003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f2ce87640 1 -- 172.21.15.12:0/2868578912 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f10003530 con 0x7f9f10005190 2023-10-30T12:47:09.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.938+0000 7f9f27fff640 1 --2- 172.21.15.12:0/2868578912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10003980 0x7f9f10003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:09.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.942+0000 7f9f2ce87640 1 -- 172.21.15.12:0/2868578912 reap_dead start 2023-10-30T12:47:09.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.942+0000 7f9f2ce87640 1 --2- 172.21.15.12:0/2868578912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f10005190 0x7f9f1000b990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000e350 tx=0x7f9f2001ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63058d92f70ed34f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:09.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.942+0000 7f9f26ffd640 1 -- 172.21.15.12:0/2868578912 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f200a5e70 con 0x7f9f10005190 2023-10-30T12:47:09.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.942+0000 7f9f26ffd640 1 -- 172.21.15.12:0/2868578912 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2000f040 con 0x7f9f10005190 2023-10-30T12:47:09.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.942+0000 7f9f26ffd640 1 -- 172.21.15.12:0/2868578912 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20003be0 con 0x7f9f10005190 2023-10-30T12:47:09.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.942+0000 7f9f2d688640 1 -- 172.21.15.12:0/2868578912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f10005190 msgr2=0x7f9f1000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:09.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.942+0000 7f9f2d688640 1 --2- 172.21.15.12:0/2868578912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f10005190 0x7f9f1000b990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000e350 tx=0x7f9f2001ce60 comp rx=0 tx=0).stop 2023-10-30T12:47:09.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.942+0000 7f9f2d688640 1 -- 172.21.15.12:0/2868578912 shutdown_connections 2023-10-30T12:47:09.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.942+0000 7f9f2d688640 1 --2- 172.21.15.12:0/2868578912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f10005190 0x7f9f1000b990 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.942+0000 7f9f2d688640 1 -- 172.21.15.12:0/2868578912 >> 172.21.15.12:0/2868578912 conn(0x7f9f2806a640 msgr2=0x7f9f28065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:09.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.942+0000 7f9f2d688640 1 -- 172.21.15.12:0/2868578912 shutdown_connections 2023-10-30T12:47:09.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.942+0000 7f9f2d688640 1 -- 172.21.15.12:0/2868578912 wait complete. 2023-10-30T12:47:09.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.942+0000 7f9f2d688640 1 Processor -- start 2023-10-30T12:47:09.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.942+0000 7f9f2d688640 1 -- start start 2023-10-30T12:47:09.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2d688640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10005190 0x7f9f28148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:09.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2d688640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f280b4ef0 0x7f9f28151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:09.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2d688640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f281ee5e0 0x7f9f281eea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:09.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2d688640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280b3930 con 0x7f9f281ee5e0 2023-10-30T12:47:09.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2d688640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f28152b70 con 0x7f9f10005190 2023-10-30T12:47:09.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2d688640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f281f2a40 con 0x7f9f280b4ef0 2023-10-30T12:47:09.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2ce87640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f281ee5e0 0x7f9f281eea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:09.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2ce87640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f281ee5e0 0x7f9f281eea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34622/0 (socket says 172.21.15.12:34622) 2023-10-30T12:47:09.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2ce87640 1 -- 172.21.15.12:0/1668212330 learned_addr learned my addr 172.21.15.12:0/1668212330 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:09.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f277fe640 1 --2- 172.21.15.12:0/1668212330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f280b4ef0 0x7f9f28151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:09.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f27fff640 1 --2- 172.21.15.12:0/1668212330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10005190 0x7f9f28148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:09.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2ce87640 1 -- 172.21.15.12:0/1668212330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f280b4ef0 msgr2=0x7f9f28151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:09.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2ce87640 1 --2- 172.21.15.12:0/1668212330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f280b4ef0 0x7f9f28151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2ce87640 1 -- 172.21.15.12:0/1668212330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10005190 msgr2=0x7f9f28148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:09.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2ce87640 1 --2- 172.21.15.12:0/1668212330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10005190 0x7f9f28148d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2ce87640 1 -- 172.21.15.12:0/1668212330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f28152890 con 0x7f9f281ee5e0 2023-10-30T12:47:09.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f27fff640 1 --2- 172.21.15.12:0/1668212330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10005190 0x7f9f28148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:09.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2ce87640 1 --2- 172.21.15.12:0/1668212330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f281ee5e0 0x7f9f281eea60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9f2001d700 tx=0x7f9f20009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:09.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f24ff9640 1 -- 172.21.15.12:0/1668212330 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f200bd070 con 0x7f9f281ee5e0 2023-10-30T12:47:09.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f24ff9640 1 -- 172.21.15.12:0/1668212330 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f200aeed0 con 0x7f9f281ee5e0 2023-10-30T12:47:09.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f24ff9640 1 -- 172.21.15.12:0/1668212330 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20003bc0 con 0x7f9f281ee5e0 2023-10-30T12:47:09.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2d688640 1 -- 172.21.15.12:0/1668212330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f281f1c80 con 0x7f9f281ee5e0 2023-10-30T12:47:09.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2d688640 1 -- 172.21.15.12:0/1668212330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f281f2240 con 0x7f9f281ee5e0 2023-10-30T12:47:09.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.946+0000 7f9f2d688640 1 -- 172.21.15.12:0/1668212330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2804e170 con 0x7f9f281ee5e0 2023-10-30T12:47:09.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.950+0000 7f9f24ff9640 1 -- 172.21.15.12:0/1668212330 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9f20003d20 con 0x7f9f281ee5e0 2023-10-30T12:47:09.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.950+0000 7f9f24ff9640 1 --2- 172.21.15.12:0/1668212330 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9f140664f0 0x7f9f140689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:09.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.950+0000 7f9f24ff9640 1 -- 172.21.15.12:0/1668212330 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f9f2011f4e0 con 0x7f9f281ee5e0 2023-10-30T12:47:09.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.950+0000 7f9f27fff640 1 --2- 172.21.15.12:0/1668212330 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9f140664f0 0x7f9f140689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:09.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.950+0000 7f9f27fff640 1 --2- 172.21.15.12:0/1668212330 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9f140664f0 0x7f9f140689b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9f18005b40 tx=0x7f9f18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:09.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:09.954+0000 7f9f24ff9640 1 -- 172.21.15.12:0/1668212330 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9f200ec720 con 0x7f9f281ee5e0 2023-10-30T12:47:10.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:10.142+0000 7f9f2d688640 1 -- 172.21.15.12:0/1668212330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f280b2290 con 0x7f9f281ee5e0 2023-10-30T12:47:10.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:10.142+0000 7f9f24ff9640 1 -- 172.21.15.12:0/1668212330 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+6535 (secure 0 0 0) 0x7f9f20009de0 con 0x7f9f281ee5e0 2023-10-30T12:47:10.150 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:10.150 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:03.591793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:10.150 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 62 2023-10-30T12:47:10.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:10.146+0000 7f9f2d688640 1 -- 172.21.15.12:0/1668212330 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9f140664f0 msgr2=0x7f9f140689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:10.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:10.146+0000 7f9f2d688640 1 --2- 172.21.15.12:0/1668212330 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9f140664f0 0x7f9f140689b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9f18005b40 tx=0x7f9f18009040 comp rx=0 tx=0).stop 2023-10-30T12:47:10.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:10.146+0000 7f9f2d688640 1 -- 172.21.15.12:0/1668212330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f281ee5e0 msgr2=0x7f9f281eea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:10.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:10.146+0000 7f9f2d688640 1 --2- 172.21.15.12:0/1668212330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f281ee5e0 0x7f9f281eea60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9f2001d700 tx=0x7f9f20009750 comp rx=0 tx=0).stop 2023-10-30T12:47:10.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:10.146+0000 7f9f2d688640 1 -- 172.21.15.12:0/1668212330 shutdown_connections 2023-10-30T12:47:10.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:10.146+0000 7f9f2d688640 1 --2- 172.21.15.12:0/1668212330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f281ee5e0 0x7f9f281eea60 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:10.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:10.146+0000 7f9f2d688640 1 --2- 172.21.15.12:0/1668212330 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9f140664f0 0x7f9f140689b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:10.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:10.146+0000 7f9f2d688640 1 --2- 172.21.15.12:0/1668212330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f280b4ef0 0x7f9f28151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:10.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:10.146+0000 7f9f2d688640 1 --2- 172.21.15.12:0/1668212330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f10005190 0x7f9f28148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:10.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:10.146+0000 7f9f2d688640 1 -- 172.21.15.12:0/1668212330 >> 172.21.15.12:0/1668212330 conn(0x7f9f2806a640 msgr2=0x7f9f280c5270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:10.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:10.146+0000 7f9f2d688640 1 -- 172.21.15.12:0/1668212330 shutdown_connections 2023-10-30T12:47:10.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:10.146+0000 7f9f2d688640 1 -- 172.21.15.12:0/1668212330 wait complete. 2023-10-30T12:47:10.170 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2023-10-30T12:47:10.171 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 18 secs before thrashing 2023-10-30T12:47:28.172 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:28.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.334+0000 7f80889d3640 1 Processor -- start 2023-10-30T12:47:28.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.334+0000 7f80889d3640 1 -- start start 2023-10-30T12:47:28.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.338+0000 7f80889d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80841538a0 0x7f808415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.338+0000 7f80889d3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840cb800 con 0x7f80840cf160 2023-10-30T12:47:28.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.338+0000 7f80889d3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f808415dc30 con 0x7f80840ca0e0 2023-10-30T12:47:28.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.338+0000 7f80889d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80840cb330 con 0x7f80841538a0 2023-10-30T12:47:28.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.338+0000 7f8082d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f80840cf160 0x7f8084164070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42570/0 (socket says 172.21.15.12:42570) 2023-10-30T12:47:28.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.338+0000 7f8082d9d640 1 -- 172.21.15.12:0/1076481638 learned_addr learned my addr 172.21.15.12:0/1076481638 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:47:28.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.338+0000 7f808259c640 1 --2- 172.21.15.12:0/1076481638 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80841538a0 0x7f808415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.338+0000 7f808159a640 1 -- 172.21.15.12:0/1076481638 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1548203714 0 0) 0x7f80840cb800 con 0x7f80840cf160 2023-10-30T12:47:28.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.338+0000 7f808159a640 1 -- 172.21.15.12:0/1076481638 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f806c003530 con 0x7f80840cf160 2023-10-30T12:47:28.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.338+0000 7f808259c640 1 -- 172.21.15.12:0/1076481638 >> v1:172.21.15.154:6789/0 conn(0x7f80840ca0e0 legacy=0x7f808415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.338+0000 7f808259c640 1 -- 172.21.15.12:0/1076481638 >> v1:172.21.15.12:6789/0 conn(0x7f80840cf160 legacy=0x7f8084164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.338+0000 7f808259c640 1 -- 172.21.15.12:0/1076481638 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80840cad70 con 0x7f80841538a0 2023-10-30T12:47:28.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.342+0000 7f808259c640 1 --2- 172.21.15.12:0/1076481638 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80841538a0 0x7f808415d6f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f80740097f0 tx=0x7f807401ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c395be3aa9a44585 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:28.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.342+0000 7f808159a640 1 -- 172.21.15.12:0/1076481638 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80740a89b0 con 0x7f80841538a0 2023-10-30T12:47:28.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.342+0000 7f808159a640 1 -- 172.21.15.12:0/1076481638 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80740a8b10 con 0x7f80841538a0 2023-10-30T12:47:28.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.342+0000 7f808159a640 1 -- 172.21.15.12:0/1076481638 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80740a8ea0 con 0x7f80841538a0 2023-10-30T12:47:28.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.342+0000 7f80889d3640 1 -- 172.21.15.12:0/1076481638 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80841538a0 msgr2=0x7f808415d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.342+0000 7f80889d3640 1 --2- 172.21.15.12:0/1076481638 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80841538a0 0x7f808415d6f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f80740097f0 tx=0x7f807401ecb0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.342+0000 7f80889d3640 1 -- 172.21.15.12:0/1076481638 shutdown_connections 2023-10-30T12:47:28.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.342+0000 7f80889d3640 1 --2- 172.21.15.12:0/1076481638 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80841538a0 0x7f808415d6f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.342+0000 7f80889d3640 1 -- 172.21.15.12:0/1076481638 >> 172.21.15.12:0/1076481638 conn(0x7f808406bfe0 msgr2=0x7f808406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:28.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.342+0000 7f80889d3640 1 -- 172.21.15.12:0/1076481638 shutdown_connections 2023-10-30T12:47:28.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.342+0000 7f80889d3640 1 -- 172.21.15.12:0/1076481638 wait complete. 2023-10-30T12:47:28.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f80889d3640 1 Processor -- start 2023-10-30T12:47:28.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f80889d3640 1 -- start start 2023-10-30T12:47:28.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f80889d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80840ca0e0 0x7f8084140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f80889d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80840cf160 0x7f80841526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f80889d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80841538a0 0x7f8084152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f80889d3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084157b50 con 0x7f80841538a0 2023-10-30T12:47:28.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f80889d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80840cd4b0 con 0x7f80840cf160 2023-10-30T12:47:28.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f80889d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80841f3350 con 0x7f80840ca0e0 2023-10-30T12:47:28.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f8082d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80841538a0 0x7f8084152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f8082d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80841538a0 0x7f8084152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34572/0 (socket says 172.21.15.12:34572) 2023-10-30T12:47:28.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f8082d9d640 1 -- 172.21.15.12:0/2924816697 learned_addr learned my addr 172.21.15.12:0/2924816697 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:28.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f808259c640 1 --2- 172.21.15.12:0/2924816697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80840ca0e0 0x7f8084140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f8081d9b640 1 --2- 172.21.15.12:0/2924816697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80840cf160 0x7f80841526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f8082d9d640 1 -- 172.21.15.12:0/2924816697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80840ca0e0 msgr2=0x7f8084140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f8082d9d640 1 --2- 172.21.15.12:0/2924816697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80840ca0e0 0x7f8084140da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f8082d9d640 1 -- 172.21.15.12:0/2924816697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80840cf160 msgr2=0x7f80841526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f8082d9d640 1 --2- 172.21.15.12:0/2924816697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80840cf160 0x7f80841526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f8082d9d640 1 -- 172.21.15.12:0/2924816697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8074009420 con 0x7f80841538a0 2023-10-30T12:47:28.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.346+0000 7f8082d9d640 1 --2- 172.21.15.12:0/2924816697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80841538a0 0x7f8084152c30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f807c00caa0 tx=0x7f807c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:28.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.350+0000 7f8072ffd640 1 -- 172.21.15.12:0/2924816697 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c00ac00 con 0x7f80841538a0 2023-10-30T12:47:28.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.350+0000 7f8072ffd640 1 -- 172.21.15.12:0/2924816697 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c00ad60 con 0x7f80841538a0 2023-10-30T12:47:28.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.350+0000 7f8072ffd640 1 -- 172.21.15.12:0/2924816697 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c012620 con 0x7f80841538a0 2023-10-30T12:47:28.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.350+0000 7f80889d3640 1 -- 172.21.15.12:0/2924816697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8084158400 con 0x7f80841538a0 2023-10-30T12:47:28.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.350+0000 7f80889d3640 1 -- 172.21.15.12:0/2924816697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80841f2a20 con 0x7f80841538a0 2023-10-30T12:47:28.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.350+0000 7f80889d3640 1 -- 172.21.15.12:0/2924816697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808404e170 con 0x7f80841538a0 2023-10-30T12:47:28.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.350+0000 7f8072ffd640 1 -- 172.21.15.12:0/2924816697 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f807c007070 con 0x7f80841538a0 2023-10-30T12:47:28.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.350+0000 7f8072ffd640 1 --2- 172.21.15.12:0/2924816697 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80640664a0 0x7f8064068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.350+0000 7f8072ffd640 1 -- 172.21.15.12:0/2924816697 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f807c0842b0 con 0x7f80841538a0 2023-10-30T12:47:28.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.350+0000 7f808259c640 1 --2- 172.21.15.12:0/2924816697 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80640664a0 0x7f8064068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.354+0000 7f808259c640 1 --2- 172.21.15.12:0/2924816697 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80640664a0 0x7f8064068960 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f80740b2000 tx=0x7f80740aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:28.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.358+0000 7f8072ffd640 1 -- 172.21.15.12:0/2924816697 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f807c051730 con 0x7f80841538a0 2023-10-30T12:47:28.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.598+0000 7f80889d3640 1 -- 172.21.15.12:0/2924816697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8084151930 con 0x7f80841538a0 2023-10-30T12:47:28.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.602+0000 7f8072ffd640 1 -- 172.21.15.12:0/2924816697 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+6535 (secure 0 0 0) 0x7f807c054d50 con 0x7f80841538a0 2023-10-30T12:47:28.608 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:28.608 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:03.591793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:28.609 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 62 2023-10-30T12:47:28.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.602+0000 7f80889d3640 1 -- 172.21.15.12:0/2924816697 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80640664a0 msgr2=0x7f8064068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.602+0000 7f80889d3640 1 --2- 172.21.15.12:0/2924816697 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80640664a0 0x7f8064068960 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f80740b2000 tx=0x7f80740aa070 comp rx=0 tx=0).stop 2023-10-30T12:47:28.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.602+0000 7f80889d3640 1 -- 172.21.15.12:0/2924816697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80841538a0 msgr2=0x7f8084152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.602+0000 7f80889d3640 1 --2- 172.21.15.12:0/2924816697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80841538a0 0x7f8084152c30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f807c00caa0 tx=0x7f807c00cf60 comp rx=0 tx=0).stop 2023-10-30T12:47:28.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.602+0000 7f80889d3640 1 -- 172.21.15.12:0/2924816697 shutdown_connections 2023-10-30T12:47:28.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.602+0000 7f80889d3640 1 --2- 172.21.15.12:0/2924816697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80841538a0 0x7f8084152c30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.602+0000 7f80889d3640 1 --2- 172.21.15.12:0/2924816697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80840cf160 0x7f80841526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.602+0000 7f80889d3640 1 --2- 172.21.15.12:0/2924816697 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f80640664a0 0x7f8064068960 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.602+0000 7f80889d3640 1 --2- 172.21.15.12:0/2924816697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80840ca0e0 0x7f8084140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.602+0000 7f80889d3640 1 -- 172.21.15.12:0/2924816697 >> 172.21.15.12:0/2924816697 conn(0x7f808406bfe0 msgr2=0x7f80840cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:28.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.602+0000 7f80889d3640 1 -- 172.21.15.12:0/2924816697 shutdown_connections 2023-10-30T12:47:28.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.602+0000 7f80889d3640 1 -- 172.21.15.12:0/2924816697 wait complete. 2023-10-30T12:47:28.639 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=0) 2023-10-30T12:47:28.639 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-30T12:47:28.640 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:47:28.662 INFO:tasks.ceph.mds.d:Stopped 2023-10-30T12:47:28.662 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:28.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.802+0000 7ff9899c4640 1 Processor -- start 2023-10-30T12:47:28.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.802+0000 7ff9899c4640 1 -- start start 2023-10-30T12:47:28.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.802+0000 7ff9899c4640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff98415d6f0 0x7ff98415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.802+0000 7ff9899c4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9840cf160 0x7ff9840cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.802+0000 7ff9899c4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9840cb800 con 0x7ff9841538a0 2023-10-30T12:47:28.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.802+0000 7ff9899c4640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9840cb330 con 0x7ff98415d6f0 2023-10-30T12:47:28.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.802+0000 7ff9899c4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9840cad70 con 0x7ff9840cf160 2023-10-30T12:47:28.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.802+0000 7ff9827fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff98415d6f0 0x7ff98415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.802+0000 7ff9827fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff98415d6f0 0x7ff98415a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34576/0 (socket says 172.21.15.12:34576) 2023-10-30T12:47:28.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.802+0000 7ff9827fc640 1 -- 172.21.15.12:0/914286526 learned_addr learned my addr 172.21.15.12:0/914286526 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:28.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.802+0000 7ff9837fe640 1 --2- 172.21.15.12:0/914286526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9840cf160 0x7ff9840cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.802+0000 7ff9827fc640 1 -- 172.21.15.12:0/914286526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9840cf160 msgr2=0x7ff9840cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:28.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.802+0000 7ff9827fc640 1 --2- 172.21.15.12:0/914286526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9840cf160 0x7ff9840cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.802+0000 7ff9827fc640 1 -- 172.21.15.12:0/914286526 >> v1:172.21.15.154:6789/0 conn(0x7ff9841538a0 legacy=0x7ff9840ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.802+0000 7ff9827fc640 1 -- 172.21.15.12:0/914286526 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9840cb050 con 0x7ff98415d6f0 2023-10-30T12:47:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.806+0000 7ff9827fc640 1 --2- 172.21.15.12:0/914286526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff98415d6f0 0x7ff98415a090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff978009870 tx=0x7ff97801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f744d3f7b473a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:28.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.806+0000 7ff981ffb640 1 -- 172.21.15.12:0/914286526 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9780a8a30 con 0x7ff98415d6f0 2023-10-30T12:47:28.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.806+0000 7ff981ffb640 1 -- 172.21.15.12:0/914286526 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9780a8b90 con 0x7ff98415d6f0 2023-10-30T12:47:28.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.806+0000 7ff981ffb640 1 -- 172.21.15.12:0/914286526 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9780b0440 con 0x7ff98415d6f0 2023-10-30T12:47:28.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.806+0000 7ff9899c4640 1 -- 172.21.15.12:0/914286526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff98415d6f0 msgr2=0x7ff98415a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.806+0000 7ff9899c4640 1 --2- 172.21.15.12:0/914286526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff98415d6f0 0x7ff98415a090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff978009870 tx=0x7ff97801ed30 comp rx=0 tx=0).stop 2023-10-30T12:47:28.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.806+0000 7ff9899c4640 1 -- 172.21.15.12:0/914286526 shutdown_connections 2023-10-30T12:47:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.806+0000 7ff9899c4640 1 --2- 172.21.15.12:0/914286526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9840cf160 0x7ff9840cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.806+0000 7ff9899c4640 1 --2- 172.21.15.12:0/914286526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff98415d6f0 0x7ff98415a090 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.806+0000 7ff9899c4640 1 -- 172.21.15.12:0/914286526 >> 172.21.15.12:0/914286526 conn(0x7ff98406bfe0 msgr2=0x7ff98406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.806+0000 7ff9899c4640 1 -- 172.21.15.12:0/914286526 shutdown_connections 2023-10-30T12:47:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.806+0000 7ff9899c4640 1 -- 172.21.15.12:0/914286526 wait complete. 2023-10-30T12:47:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.806+0000 7ff9899c4640 1 Processor -- start 2023-10-30T12:47:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff9899c4640 1 -- start start 2023-10-30T12:47:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff9899c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9840cf160 0x7ff9841e48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff9899c4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9841538a0 0x7ff9841f0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff9899c4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff98415d6f0 0x7ff9841f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff9899c4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9841e3d10 con 0x7ff9840cf160 2023-10-30T12:47:28.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff9899c4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9841f2e30 con 0x7ff98415d6f0 2023-10-30T12:47:28.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff9899c4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9841f3110 con 0x7ff9841538a0 2023-10-30T12:47:28.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff982ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9840cf160 0x7ff9841e48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff982ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9840cf160 0x7ff9841e48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34586/0 (socket says 172.21.15.12:34586) 2023-10-30T12:47:28.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff982ffd640 1 -- 172.21.15.12:0/2421120722 learned_addr learned my addr 172.21.15.12:0/2421120722 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:28.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff9827fc640 1 --2- 172.21.15.12:0/2421120722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9841538a0 0x7ff9841f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff9837fe640 1 --2- 172.21.15.12:0/2421120722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff98415d6f0 0x7ff9841f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff982ffd640 1 -- 172.21.15.12:0/2421120722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9841538a0 msgr2=0x7ff9841f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff982ffd640 1 --2- 172.21.15.12:0/2421120722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9841538a0 0x7ff9841f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff982ffd640 1 -- 172.21.15.12:0/2421120722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff98415d6f0 msgr2=0x7ff9841f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff982ffd640 1 --2- 172.21.15.12:0/2421120722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff98415d6f0 0x7ff9841f35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff982ffd640 1 -- 172.21.15.12:0/2421120722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9780094d0 con 0x7ff9840cf160 2023-10-30T12:47:28.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff9827fc640 1 --2- 172.21.15.12:0/2421120722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9841538a0 0x7ff9841f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:28.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff982ffd640 1 --2- 172.21.15.12:0/2421120722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9840cf160 0x7ff9841e48d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff97400c530 tx=0x7ff97400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:28.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff96bfff640 1 -- 172.21.15.12:0/2421120722 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff974005220 con 0x7ff9840cf160 2023-10-30T12:47:28.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff96bfff640 1 -- 172.21.15.12:0/2421120722 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff974005380 con 0x7ff9840cf160 2023-10-30T12:47:28.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff9899c4640 1 -- 172.21.15.12:0/2421120722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98414d4c0 con 0x7ff9840cf160 2023-10-30T12:47:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff96bfff640 1 -- 172.21.15.12:0/2421120722 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff974007270 con 0x7ff9840cf160 2023-10-30T12:47:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff9899c4640 1 -- 172.21.15.12:0/2421120722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9841e55d0 con 0x7ff9840cf160 2023-10-30T12:47:28.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.810+0000 7ff9899c4640 1 -- 172.21.15.12:0/2421120722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9840b2a50 con 0x7ff9840cf160 2023-10-30T12:47:28.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.814+0000 7ff96bfff640 1 -- 172.21.15.12:0/2421120722 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff9740073d0 con 0x7ff9840cf160 2023-10-30T12:47:28.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.814+0000 7ff96bfff640 1 --2- 172.21.15.12:0/2421120722 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff9700664a0 0x7ff970068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.814+0000 7ff96bfff640 1 -- 172.21.15.12:0/2421120722 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7ff9740839a0 con 0x7ff9840cf160 2023-10-30T12:47:28.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.814+0000 7ff9827fc640 1 --2- 172.21.15.12:0/2421120722 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff9700664a0 0x7ff970068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.814+0000 7ff9827fc640 1 --2- 172.21.15.12:0/2421120722 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff9700664a0 0x7ff970068960 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff9780b2000 tx=0x7ff978008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:28.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:28.818+0000 7ff96bfff640 1 -- 172.21.15.12:0/2421120722 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff974050c00 con 0x7ff9840cf160 2023-10-30T12:47:29.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:29.014+0000 7ff9899c4640 1 -- 172.21.15.12:0/2421120722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff98404e170 con 0x7ff9840cf160 2023-10-30T12:47:29.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:29.014+0000 7ff96bfff640 1 -- 172.21.15.12:0/2421120722 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+6535 (secure 0 0 0) 0x7ff974054220 con 0x7ff9840cf160 2023-10-30T12:47:29.022 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:29.022 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:03.591793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:29.022 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 62 2023-10-30T12:47:29.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:29.018+0000 7ff9899c4640 1 -- 172.21.15.12:0/2421120722 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff9700664a0 msgr2=0x7ff970068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:29.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:29.018+0000 7ff9899c4640 1 --2- 172.21.15.12:0/2421120722 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff9700664a0 0x7ff970068960 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff9780b2000 tx=0x7ff978008040 comp rx=0 tx=0).stop 2023-10-30T12:47:29.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:29.018+0000 7ff9899c4640 1 -- 172.21.15.12:0/2421120722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9840cf160 msgr2=0x7ff9841e48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:29.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:29.018+0000 7ff9899c4640 1 --2- 172.21.15.12:0/2421120722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9840cf160 0x7ff9841e48d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff97400c530 tx=0x7ff97400c9f0 comp rx=0 tx=0).stop 2023-10-30T12:47:29.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:29.018+0000 7ff9899c4640 1 -- 172.21.15.12:0/2421120722 shutdown_connections 2023-10-30T12:47:29.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:29.018+0000 7ff9899c4640 1 --2- 172.21.15.12:0/2421120722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff98415d6f0 0x7ff9841f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:29.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:29.018+0000 7ff9899c4640 1 --2- 172.21.15.12:0/2421120722 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff9700664a0 0x7ff970068960 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:29.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:29.018+0000 7ff9899c4640 1 --2- 172.21.15.12:0/2421120722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9841538a0 0x7ff9841f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:29.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:29.018+0000 7ff9899c4640 1 --2- 172.21.15.12:0/2421120722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9840cf160 0x7ff9841e48d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:29.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:29.018+0000 7ff9899c4640 1 -- 172.21.15.12:0/2421120722 >> 172.21.15.12:0/2421120722 conn(0x7ff98406bfe0 msgr2=0x7ff9840ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:29.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:29.018+0000 7ff9899c4640 1 -- 172.21.15.12:0/2421120722 shutdown_connections 2023-10-30T12:47:29.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:29.018+0000 7ff9899c4640 1 -- 172.21.15.12:0/2421120722 wait complete. 2023-10-30T12:47:29.044 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:47:31.045 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:31.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.190+0000 7f2371365640 1 Processor -- start 2023-10-30T12:47:31.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.190+0000 7f2371365640 1 -- start start 2023-10-30T12:47:31.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.194+0000 7f2371365640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f236c15d6f0 0x7f236c15a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.194+0000 7f2371365640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c0cb800 con 0x7f236c1538a0 2023-10-30T12:47:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.194+0000 7f2371365640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c0cb330 con 0x7f236c0cf160 2023-10-30T12:47:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.194+0000 7f2371365640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c0cad70 con 0x7f236c15d6f0 2023-10-30T12:47:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.194+0000 7f236a7fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f236c15d6f0 0x7f236c15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.194+0000 7f236affd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f236c1538a0 0x7f236c0ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48198/0 (socket says 172.21.15.12:48198) 2023-10-30T12:47:31.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.194+0000 7f236affd640 1 -- 172.21.15.12:0/4154239346 learned_addr learned my addr 172.21.15.12:0/4154239346 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:47:31.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.194+0000 7f236a7fc640 1 -- 172.21.15.12:0/4154239346 >> v1:172.21.15.154:6789/0 conn(0x7f236c0cf160 legacy=0x7f236c164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:31.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.194+0000 7f236a7fc640 1 -- 172.21.15.12:0/4154239346 >> v1:172.21.15.12:6789/0 conn(0x7f236c1538a0 legacy=0x7f236c0ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:31.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.194+0000 7f236a7fc640 1 -- 172.21.15.12:0/4154239346 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0cb050 con 0x7f236c15d6f0 2023-10-30T12:47:31.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.198+0000 7f236a7fc640 1 --2- 172.21.15.12:0/4154239346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f236c15d6f0 0x7f236c15a090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f23600097c0 tx=0x7f236001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870ebc9014c287b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:31.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.198+0000 7f2369ffb640 1 -- 172.21.15.12:0/4154239346 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600a8970 con 0x7f236c15d6f0 2023-10-30T12:47:31.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.198+0000 7f2369ffb640 1 -- 172.21.15.12:0/4154239346 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23600a8ad0 con 0x7f236c15d6f0 2023-10-30T12:47:31.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.198+0000 7f2369ffb640 1 -- 172.21.15.12:0/4154239346 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600a8e60 con 0x7f236c15d6f0 2023-10-30T12:47:31.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.198+0000 7f2371365640 1 -- 172.21.15.12:0/4154239346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f236c15d6f0 msgr2=0x7f236c15a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:31.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.198+0000 7f2371365640 1 --2- 172.21.15.12:0/4154239346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f236c15d6f0 0x7f236c15a090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f23600097c0 tx=0x7f236001ec70 comp rx=0 tx=0).stop 2023-10-30T12:47:31.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.198+0000 7f2371365640 1 -- 172.21.15.12:0/4154239346 shutdown_connections 2023-10-30T12:47:31.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.198+0000 7f2371365640 1 --2- 172.21.15.12:0/4154239346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f236c15d6f0 0x7f236c15a090 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.198+0000 7f2371365640 1 -- 172.21.15.12:0/4154239346 >> 172.21.15.12:0/4154239346 conn(0x7f236c06bfe0 msgr2=0x7f236c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:31.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.198+0000 7f2371365640 1 -- 172.21.15.12:0/4154239346 shutdown_connections 2023-10-30T12:47:31.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.198+0000 7f2371365640 1 -- 172.21.15.12:0/4154239346 wait complete. 2023-10-30T12:47:31.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.198+0000 7f2371365640 1 Processor -- start 2023-10-30T12:47:31.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.198+0000 7f2371365640 1 -- start start 2023-10-30T12:47:31.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f2371365640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f236c0cf160 0x7f236c148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:31.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f2371365640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f236c1538a0 0x7f236c1492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:31.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f2371365640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f236c15d6f0 0x7f236c1503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:31.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f2371365640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c1ef170 con 0x7f236c1538a0 2023-10-30T12:47:31.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f2371365640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c1ef360 con 0x7f236c0cf160 2023-10-30T12:47:31.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f2371365640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c1ef620 con 0x7f236c15d6f0 2023-10-30T12:47:31.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f236a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f236c1538a0 0x7f236c1492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:31.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f236a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f236c1538a0 0x7f236c1492a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50134/0 (socket says 172.21.15.12:50134) 2023-10-30T12:47:31.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f236a7fc640 1 -- 172.21.15.12:0/3097423595 learned_addr learned my addr 172.21.15.12:0/3097423595 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:31.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f236affd640 1 --2- 172.21.15.12:0/3097423595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f236c0cf160 0x7f236c148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:31.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f236b7fe640 1 --2- 172.21.15.12:0/3097423595 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f236c15d6f0 0x7f236c1503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:31.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f236a7fc640 1 -- 172.21.15.12:0/3097423595 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f236c15d6f0 msgr2=0x7f236c1503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:31.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f236a7fc640 1 --2- 172.21.15.12:0/3097423595 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f236c15d6f0 0x7f236c1503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f236a7fc640 1 -- 172.21.15.12:0/3097423595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f236c0cf160 msgr2=0x7f236c148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:31.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f236a7fc640 1 --2- 172.21.15.12:0/3097423595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f236c0cf160 0x7f236c148d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f236a7fc640 1 -- 172.21.15.12:0/3097423595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2360009420 con 0x7f236c1538a0 2023-10-30T12:47:31.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f236a7fc640 1 --2- 172.21.15.12:0/3097423595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f236c1538a0 0x7f236c1492a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f23600b3040 tx=0x7f236001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:31.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f23537fe640 1 -- 172.21.15.12:0/3097423595 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600b9630 con 0x7f236c1538a0 2023-10-30T12:47:31.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f23537fe640 1 -- 172.21.15.12:0/3097423595 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23600b97b0 con 0x7f236c1538a0 2023-10-30T12:47:31.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f2371365640 1 -- 172.21.15.12:0/3097423595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c1ef900 con 0x7f236c1538a0 2023-10-30T12:47:31.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f23537fe640 1 -- 172.21.15.12:0/3097423595 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600b9b60 con 0x7f236c1538a0 2023-10-30T12:47:31.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f2371365640 1 -- 172.21.15.12:0/3097423595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c1efec0 con 0x7f236c1538a0 2023-10-30T12:47:31.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f2371365640 1 -- 172.21.15.12:0/3097423595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c1f00b0 con 0x7f236c1538a0 2023-10-30T12:47:31.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.202+0000 7f23537fe640 1 -- 172.21.15.12:0/3097423595 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f23600b7040 con 0x7f236c1538a0 2023-10-30T12:47:31.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.206+0000 7f23537fe640 1 --2- 172.21.15.12:0/3097423595 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f23580664a0 0x7f2358068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:31.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.206+0000 7f23537fe640 1 -- 172.21.15.12:0/3097423595 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f2360122230 con 0x7f236c1538a0 2023-10-30T12:47:31.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.206+0000 7f236affd640 1 --2- 172.21.15.12:0/3097423595 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f23580664a0 0x7f2358068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:31.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.206+0000 7f236affd640 1 --2- 172.21.15.12:0/3097423595 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f23580664a0 0x7f2358068960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f235c005890 tx=0x7f235c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:31.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.210+0000 7f23537fe640 1 -- 172.21.15.12:0/3097423595 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f23600ef360 con 0x7f236c1538a0 2023-10-30T12:47:31.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.406+0000 7f2371365640 1 -- 172.21.15.12:0/3097423595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f236c04e170 con 0x7f236c1538a0 2023-10-30T12:47:31.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.406+0000 7f23537fe640 1 -- 172.21.15.12:0/3097423595 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+6535 (secure 0 0 0) 0x7f23600b5020 con 0x7f236c1538a0 2023-10-30T12:47:31.412 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:31.412 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:03.591793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:31.413 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 62 2023-10-30T12:47:31.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.406+0000 7f2371365640 1 -- 172.21.15.12:0/3097423595 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f23580664a0 msgr2=0x7f2358068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:31.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.406+0000 7f2371365640 1 --2- 172.21.15.12:0/3097423595 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f23580664a0 0x7f2358068960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f235c005890 tx=0x7f235c009040 comp rx=0 tx=0).stop 2023-10-30T12:47:31.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.406+0000 7f2371365640 1 -- 172.21.15.12:0/3097423595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f236c1538a0 msgr2=0x7f236c1492a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:31.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.406+0000 7f2371365640 1 --2- 172.21.15.12:0/3097423595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f236c1538a0 0x7f236c1492a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f23600b3040 tx=0x7f236001ba10 comp rx=0 tx=0).stop 2023-10-30T12:47:31.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.406+0000 7f2371365640 1 -- 172.21.15.12:0/3097423595 shutdown_connections 2023-10-30T12:47:31.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.406+0000 7f2371365640 1 --2- 172.21.15.12:0/3097423595 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f23580664a0 0x7f2358068960 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.406+0000 7f2371365640 1 --2- 172.21.15.12:0/3097423595 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f236c15d6f0 0x7f236c1503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.406+0000 7f2371365640 1 --2- 172.21.15.12:0/3097423595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f236c1538a0 0x7f236c1492a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.406+0000 7f2371365640 1 --2- 172.21.15.12:0/3097423595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f236c0cf160 0x7f236c148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.406+0000 7f2371365640 1 -- 172.21.15.12:0/3097423595 >> 172.21.15.12:0/3097423595 conn(0x7f236c06bfe0 msgr2=0x7f236c0ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:31.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.406+0000 7f2371365640 1 -- 172.21.15.12:0/3097423595 shutdown_connections 2023-10-30T12:47:31.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:31.410+0000 7f2371365640 1 -- 172.21.15.12:0/3097423595 wait complete. 2023-10-30T12:47:31.433 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:47:33.434 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.582+0000 7f55c5bf5640 1 Processor -- start 2023-10-30T12:47:33.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.582+0000 7f55c5bf5640 1 -- start start 2023-10-30T12:47:33.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55c5bf5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f55c00c6bd0 0x7f55c006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:33.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55c5bf5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f55c0071110 0x7f55c0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:33.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55c5bf5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c0066cf0 con 0x7f55c006f7a0 2023-10-30T12:47:33.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55c5bf5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c00700c0 con 0x7f55c0071110 2023-10-30T12:47:33.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55c5bf5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c00c70d0 con 0x7f55c00c6bd0 2023-10-30T12:47:33.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55bffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f55c0071110 0x7f55c0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:33.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55bffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f55c0071110 0x7f55c0158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50138/0 (socket says 172.21.15.12:50138) 2023-10-30T12:47:33.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55bffff640 1 -- 172.21.15.12:0/856495228 learned_addr learned my addr 172.21.15.12:0/856495228 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:33.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55beffd640 1 --2- 172.21.15.12:0/856495228 >> v2:172.21.15.154:3300/0 conn(0x7f55c00c6bd0 0x7f55c006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:33.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55bffff640 1 -- 172.21.15.12:0/856495228 >> v2:172.21.15.154:3300/0 conn(0x7f55c00c6bd0 msgr2=0x7f55c006fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:33.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55bffff640 1 --2- 172.21.15.12:0/856495228 >> v2:172.21.15.154:3300/0 conn(0x7f55c00c6bd0 0x7f55c006fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55bffff640 1 -- 172.21.15.12:0/856495228 >> v1:172.21.15.154:6789/0 conn(0x7f55c006f7a0 legacy=0x7f55c0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:33.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55bffff640 1 -- 172.21.15.12:0/856495228 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c00c7390 con 0x7f55c0071110 2023-10-30T12:47:33.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55bffff640 1 --2- 172.21.15.12:0/856495228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c0071110 0x7f55c0158110 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f55c006f1c0 tx=0x7f55b801f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74218b62bc405f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:33.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55be7fc640 1 -- 172.21.15.12:0/856495228 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b80af400 con 0x7f55c0071110 2023-10-30T12:47:33.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55be7fc640 1 -- 172.21.15.12:0/856495228 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b80ab050 con 0x7f55c0071110 2023-10-30T12:47:33.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.586+0000 7f55be7fc640 1 -- 172.21.15.12:0/856495228 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b80af780 con 0x7f55c0071110 2023-10-30T12:47:33.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 -- 172.21.15.12:0/856495228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c0071110 msgr2=0x7f55c0158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:33.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 --2- 172.21.15.12:0/856495228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c0071110 0x7f55c0158110 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f55c006f1c0 tx=0x7f55b801f1c0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 -- 172.21.15.12:0/856495228 shutdown_connections 2023-10-30T12:47:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 --2- 172.21.15.12:0/856495228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c0071110 0x7f55c0158110 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 --2- 172.21.15.12:0/856495228 >> v2:172.21.15.154:3300/0 conn(0x7f55c00c6bd0 0x7f55c006fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 -- 172.21.15.12:0/856495228 >> 172.21.15.12:0/856495228 conn(0x7f55c00c7910 msgr2=0x7f55c00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 -- 172.21.15.12:0/856495228 shutdown_connections 2023-10-30T12:47:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 -- 172.21.15.12:0/856495228 wait complete. 2023-10-30T12:47:33.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 Processor -- start 2023-10-30T12:47:33.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 -- start start 2023-10-30T12:47:33.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55c006f7a0 0x7f55c01566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:33.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c0071110 0x7f55c00c0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:33.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55c00c6bd0 0x7f55c014f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:33.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c01f2870 con 0x7f55c0071110 2023-10-30T12:47:33.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c0158430 con 0x7f55c00c6bd0 2023-10-30T12:47:33.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55c5bf5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c01f1c60 con 0x7f55c006f7a0 2023-10-30T12:47:33.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55beffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c0071110 0x7f55c00c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:33.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55beffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c0071110 0x7f55c00c0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50148/0 (socket says 172.21.15.12:50148) 2023-10-30T12:47:33.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55beffd640 1 -- 172.21.15.12:0/858542287 learned_addr learned my addr 172.21.15.12:0/858542287 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:33.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55bffff640 1 --2- 172.21.15.12:0/858542287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55c00c6bd0 0x7f55c014f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:33.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.590+0000 7f55bf7fe640 1 --2- 172.21.15.12:0/858542287 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55c006f7a0 0x7f55c01566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:33.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.594+0000 7f55beffd640 1 -- 172.21.15.12:0/858542287 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55c006f7a0 msgr2=0x7f55c01566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:33.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.594+0000 7f55beffd640 1 --2- 172.21.15.12:0/858542287 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55c006f7a0 0x7f55c01566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.594+0000 7f55beffd640 1 -- 172.21.15.12:0/858542287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55c00c6bd0 msgr2=0x7f55c014f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:33.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.594+0000 7f55beffd640 1 --2- 172.21.15.12:0/858542287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55c00c6bd0 0x7f55c014f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.594+0000 7f55beffd640 1 -- 172.21.15.12:0/858542287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b8009b30 con 0x7f55c0071110 2023-10-30T12:47:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.594+0000 7f55beffd640 1 --2- 172.21.15.12:0/858542287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c0071110 0x7f55c00c0120 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f55b400d7d0 tx=0x7f55b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.594+0000 7f55a7fff640 1 -- 172.21.15.12:0/858542287 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4018070 con 0x7f55c0071110 2023-10-30T12:47:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.594+0000 7f55a7fff640 1 -- 172.21.15.12:0/858542287 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b4004020 con 0x7f55c0071110 2023-10-30T12:47:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.594+0000 7f55a7fff640 1 -- 172.21.15.12:0/858542287 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b40041d0 con 0x7f55c0071110 2023-10-30T12:47:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.594+0000 7f55c5bf5640 1 -- 172.21.15.12:0/858542287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c01f1f40 con 0x7f55c0071110 2023-10-30T12:47:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.594+0000 7f55c5bf5640 1 -- 172.21.15.12:0/858542287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c01f2500 con 0x7f55c0071110 2023-10-30T12:47:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.594+0000 7f55c5bf5640 1 -- 172.21.15.12:0/858542287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c01585f0 con 0x7f55c0071110 2023-10-30T12:47:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.594+0000 7f55a7fff640 1 -- 172.21.15.12:0/858542287 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f55b4004330 con 0x7f55c0071110 2023-10-30T12:47:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.594+0000 7f55a7fff640 1 --2- 172.21.15.12:0/858542287 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f55ac0664f0 0x7f55ac0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.594+0000 7f55a7fff640 1 -- 172.21.15.12:0/858542287 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f55b4083e20 con 0x7f55c0071110 2023-10-30T12:47:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.598+0000 7f55bf7fe640 1 --2- 172.21.15.12:0/858542287 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f55ac0664f0 0x7f55ac0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.598+0000 7f55bf7fe640 1 --2- 172.21.15.12:0/858542287 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f55ac0664f0 0x7f55ac0689b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f55b0007fd0 tx=0x7f55b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.598+0000 7f55a7fff640 1 -- 172.21.15.12:0/858542287 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f55b4051110 con 0x7f55c0071110 2023-10-30T12:47:33.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.802+0000 7f55c5bf5640 1 -- 172.21.15.12:0/858542287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55c004e170 con 0x7f55c0071110 2023-10-30T12:47:33.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.802+0000 7f55a7fff640 1 -- 172.21.15.12:0/858542287 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+6535 (secure 0 0 0) 0x7f55b4054730 con 0x7f55c0071110 2023-10-30T12:47:33.809 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:33.810 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:03.591793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:33.810 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 62 2023-10-30T12:47:33.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.806+0000 7f55c5bf5640 1 -- 172.21.15.12:0/858542287 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f55ac0664f0 msgr2=0x7f55ac0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:33.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.806+0000 7f55c5bf5640 1 --2- 172.21.15.12:0/858542287 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f55ac0664f0 0x7f55ac0689b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f55b0007fd0 tx=0x7f55b0009040 comp rx=0 tx=0).stop 2023-10-30T12:47:33.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.806+0000 7f55c5bf5640 1 -- 172.21.15.12:0/858542287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c0071110 msgr2=0x7f55c00c0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:33.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.806+0000 7f55c5bf5640 1 --2- 172.21.15.12:0/858542287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c0071110 0x7f55c00c0120 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f55b400d7d0 tx=0x7f55b400dc90 comp rx=0 tx=0).stop 2023-10-30T12:47:33.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.806+0000 7f55c5bf5640 1 -- 172.21.15.12:0/858542287 shutdown_connections 2023-10-30T12:47:33.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.806+0000 7f55c5bf5640 1 --2- 172.21.15.12:0/858542287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55c00c6bd0 0x7f55c014f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.806+0000 7f55c5bf5640 1 --2- 172.21.15.12:0/858542287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c0071110 0x7f55c00c0120 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.806+0000 7f55c5bf5640 1 --2- 172.21.15.12:0/858542287 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f55ac0664f0 0x7f55ac0689b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.806+0000 7f55c5bf5640 1 --2- 172.21.15.12:0/858542287 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55c006f7a0 0x7f55c01566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.806+0000 7f55c5bf5640 1 -- 172.21.15.12:0/858542287 >> 172.21.15.12:0/858542287 conn(0x7f55c00c7910 msgr2=0x7f55c00c2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:33.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.806+0000 7f55c5bf5640 1 -- 172.21.15.12:0/858542287 shutdown_connections 2023-10-30T12:47:33.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:33.806+0000 7f55c5bf5640 1 -- 172.21.15.12:0/858542287 wait complete. 2023-10-30T12:47:33.833 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:47:35.731 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:47:35.736 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:47:35.834 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:35.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.982+0000 7f901993e640 1 Processor -- start 2023-10-30T12:47:35.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.982+0000 7f901993e640 1 -- start start 2023-10-30T12:47:35.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.982+0000 7f901993e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90141538a0 0x7f901415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:35.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.982+0000 7f901993e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90140cb800 con 0x7f90140ca0e0 2023-10-30T12:47:35.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.982+0000 7f901993e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901415dc30 con 0x7f90140cf160 2023-10-30T12:47:35.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.982+0000 7f901993e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90140cb330 con 0x7f90141538a0 2023-10-30T12:47:35.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.982+0000 7f9012ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90141538a0 0x7f901415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:35.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.982+0000 7f9012ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90141538a0 0x7f901415d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50162/0 (socket says 172.21.15.12:50162) 2023-10-30T12:47:35.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.982+0000 7f9012ffd640 1 -- 172.21.15.12:0/219174831 learned_addr learned my addr 172.21.15.12:0/219174831 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:35.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.982+0000 7f9012ffd640 1 -- 172.21.15.12:0/219174831 >> v1:172.21.15.154:6789/0 conn(0x7f90140cf160 legacy=0x7f9014164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:35.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.982+0000 7f9012ffd640 1 -- 172.21.15.12:0/219174831 >> v1:172.21.15.12:6789/0 conn(0x7f90140ca0e0 legacy=0x7f901415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:35.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.982+0000 7f9012ffd640 1 -- 172.21.15.12:0/219174831 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90140cad70 con 0x7f90141538a0 2023-10-30T12:47:35.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.986+0000 7f9012ffd640 1 --2- 172.21.15.12:0/219174831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90141538a0 0x7f901415d6f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9004009870 tx=0x7f900401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639ce35413af60b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:35.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.986+0000 7f90127fc640 1 -- 172.21.15.12:0/219174831 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90040a8a30 con 0x7f90141538a0 2023-10-30T12:47:35.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.986+0000 7f90127fc640 1 -- 172.21.15.12:0/219174831 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90040a8b90 con 0x7f90141538a0 2023-10-30T12:47:35.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.986+0000 7f90127fc640 1 -- 172.21.15.12:0/219174831 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90040b0440 con 0x7f90141538a0 2023-10-30T12:47:35.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.986+0000 7f901993e640 1 -- 172.21.15.12:0/219174831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90141538a0 msgr2=0x7f901415d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:35.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.986+0000 7f901993e640 1 --2- 172.21.15.12:0/219174831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90141538a0 0x7f901415d6f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9004009870 tx=0x7f900401ed30 comp rx=0 tx=0).stop 2023-10-30T12:47:35.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.986+0000 7f901993e640 1 -- 172.21.15.12:0/219174831 shutdown_connections 2023-10-30T12:47:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.986+0000 7f901993e640 1 --2- 172.21.15.12:0/219174831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90141538a0 0x7f901415d6f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.986+0000 7f901993e640 1 -- 172.21.15.12:0/219174831 >> 172.21.15.12:0/219174831 conn(0x7f901406bfe0 msgr2=0x7f901406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.986+0000 7f901993e640 1 -- 172.21.15.12:0/219174831 shutdown_connections 2023-10-30T12:47:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.986+0000 7f901993e640 1 -- 172.21.15.12:0/219174831 wait complete. 2023-10-30T12:47:35.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f901993e640 1 Processor -- start 2023-10-30T12:47:35.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f901993e640 1 -- start start 2023-10-30T12:47:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f901993e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90140ca0e0 0x7f90141e2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f901993e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90140cf160 0x7f9014157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f901993e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90141538a0 0x7f90140b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f901993e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90140c9670 con 0x7f90141538a0 2023-10-30T12:47:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f901993e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9014158470 con 0x7f90140ca0e0 2023-10-30T12:47:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f901993e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90141e5340 con 0x7f90140cf160 2023-10-30T12:47:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f90137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90141538a0 0x7f90140b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f90137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90141538a0 0x7f90140b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50170/0 (socket says 172.21.15.12:50170) 2023-10-30T12:47:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f90137fe640 1 -- 172.21.15.12:0/3627919855 learned_addr learned my addr 172.21.15.12:0/3627919855 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f9012ffd640 1 --2- 172.21.15.12:0/3627919855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90140ca0e0 0x7f90141e2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:36.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f900bfff640 1 --2- 172.21.15.12:0/3627919855 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90140cf160 0x7f9014157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:36.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f90137fe640 1 -- 172.21.15.12:0/3627919855 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90140cf160 msgr2=0x7f9014157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:36.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f90137fe640 1 --2- 172.21.15.12:0/3627919855 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90140cf160 0x7f9014157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:36.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f90137fe640 1 -- 172.21.15.12:0/3627919855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90140ca0e0 msgr2=0x7f90141e2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:36.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f90137fe640 1 --2- 172.21.15.12:0/3627919855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90140ca0e0 0x7f90141e2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f90137fe640 1 -- 172.21.15.12:0/3627919855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90040094d0 con 0x7f90141538a0 2023-10-30T12:47:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f90137fe640 1 --2- 172.21.15.12:0/3627919855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90141538a0 0x7f90140b4450 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f900c00c6f0 tx=0x7f900c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f9010ff9640 1 -- 172.21.15.12:0/3627919855 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c003980 con 0x7f90141538a0 2023-10-30T12:47:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f9010ff9640 1 -- 172.21.15.12:0/3627919855 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900c003ae0 con 0x7f90141538a0 2023-10-30T12:47:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f901993e640 1 -- 172.21.15.12:0/3627919855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90141e4eb0 con 0x7f90141538a0 2023-10-30T12:47:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f9010ff9640 1 -- 172.21.15.12:0/3627919855 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c003e90 con 0x7f90141538a0 2023-10-30T12:47:36.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.990+0000 7f901993e640 1 -- 172.21.15.12:0/3627919855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90141e42a0 con 0x7f90141538a0 2023-10-30T12:47:36.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.994+0000 7f901993e640 1 -- 172.21.15.12:0/3627919855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90141e5040 con 0x7f90141538a0 2023-10-30T12:47:36.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.994+0000 7f9010ff9640 1 -- 172.21.15.12:0/3627919855 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f900c00a850 con 0x7f90141538a0 2023-10-30T12:47:36.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.994+0000 7f9010ff9640 1 --2- 172.21.15.12:0/3627919855 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8ff40664a0 0x7f8ff4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:36.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.994+0000 7f9010ff9640 1 -- 172.21.15.12:0/3627919855 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f900c083d00 con 0x7f90141538a0 2023-10-30T12:47:36.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.994+0000 7f9012ffd640 1 --2- 172.21.15.12:0/3627919855 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8ff40664a0 0x7f8ff4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:36.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.994+0000 7f9012ffd640 1 --2- 172.21.15.12:0/3627919855 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8ff40664a0 0x7f8ff4068960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f90040b2000 tx=0x7f9004009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:36.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:35.998+0000 7f9010ff9640 1 -- 172.21.15.12:0/3627919855 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f900c050e30 con 0x7f90141538a0 2023-10-30T12:47:36.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:36.190+0000 7f901993e640 1 -- 172.21.15.12:0/3627919855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f901415dc30 con 0x7f90141538a0 2023-10-30T12:47:36.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:36.190+0000 7f9010ff9640 1 -- 172.21.15.12:0/3627919855 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+6535 (secure 0 0 0) 0x7f901415dc30 con 0x7f90141538a0 2023-10-30T12:47:36.196 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:36.196 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:03.591793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:36.197 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 62 2023-10-30T12:47:36.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:36.194+0000 7f901993e640 1 -- 172.21.15.12:0/3627919855 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8ff40664a0 msgr2=0x7f8ff4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:36.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:36.194+0000 7f901993e640 1 --2- 172.21.15.12:0/3627919855 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8ff40664a0 0x7f8ff4068960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f90040b2000 tx=0x7f9004009040 comp rx=0 tx=0).stop 2023-10-30T12:47:36.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:36.194+0000 7f901993e640 1 -- 172.21.15.12:0/3627919855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90141538a0 msgr2=0x7f90140b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:36.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:36.194+0000 7f901993e640 1 --2- 172.21.15.12:0/3627919855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90141538a0 0x7f90140b4450 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f900c00c6f0 tx=0x7f900c00cbb0 comp rx=0 tx=0).stop 2023-10-30T12:47:36.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:36.194+0000 7f901993e640 1 -- 172.21.15.12:0/3627919855 shutdown_connections 2023-10-30T12:47:36.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:36.194+0000 7f901993e640 1 --2- 172.21.15.12:0/3627919855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90141538a0 0x7f90140b4450 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:36.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:36.194+0000 7f901993e640 1 --2- 172.21.15.12:0/3627919855 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8ff40664a0 0x7f8ff4068960 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:36.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:36.194+0000 7f901993e640 1 --2- 172.21.15.12:0/3627919855 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90140cf160 0x7f9014157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:36.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:36.194+0000 7f901993e640 1 --2- 172.21.15.12:0/3627919855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90140ca0e0 0x7f90141e2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:36.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:36.194+0000 7f901993e640 1 -- 172.21.15.12:0/3627919855 >> 172.21.15.12:0/3627919855 conn(0x7f901406bfe0 msgr2=0x7f90140c54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:36.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:36.194+0000 7f901993e640 1 -- 172.21.15.12:0/3627919855 shutdown_connections 2023-10-30T12:47:36.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:36.194+0000 7f901993e640 1 -- 172.21.15.12:0/3627919855 wait complete. 2023-10-30T12:47:36.218 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:47:38.219 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:38.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.358+0000 7f49c4b9d640 1 Processor -- start 2023-10-30T12:47:38.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.358+0000 7f49c4b9d640 1 -- start start 2023-10-30T12:47:38.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.358+0000 7f49c4b9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49c00c6bd0 0x7f49c006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:38.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.358+0000 7f49c4b9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f49c0071110 0x7f49c0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:38.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.358+0000 7f49c4b9d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c0066cf0 con 0x7f49c006f7a0 2023-10-30T12:47:38.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.358+0000 7f49c4b9d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c00700c0 con 0x7f49c0071110 2023-10-30T12:47:38.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.358+0000 7f49c4b9d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00c70d0 con 0x7f49c00c6bd0 2023-10-30T12:47:38.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.362+0000 7f49bed9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f49c0071110 0x7f49c0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:38.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.362+0000 7f49bed9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f49c0071110 0x7f49c0158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50178/0 (socket says 172.21.15.12:50178) 2023-10-30T12:47:38.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.362+0000 7f49bed9d640 1 -- 172.21.15.12:0/1750757892 learned_addr learned my addr 172.21.15.12:0/1750757892 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:38.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.362+0000 7f49bdd9b640 1 --2- 172.21.15.12:0/1750757892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49c00c6bd0 0x7f49c006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:38.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.362+0000 7f49bed9d640 1 -- 172.21.15.12:0/1750757892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49c00c6bd0 msgr2=0x7f49c006fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:38.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.362+0000 7f49bed9d640 1 --2- 172.21.15.12:0/1750757892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49c00c6bd0 0x7f49c006fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.362+0000 7f49bed9d640 1 -- 172.21.15.12:0/1750757892 >> v1:172.21.15.12:6789/0 conn(0x7f49c006f7a0 legacy=0x7f49c0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:38.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.362+0000 7f49bed9d640 1 -- 172.21.15.12:0/1750757892 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00c7390 con 0x7f49c0071110 2023-10-30T12:47:38.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.362+0000 7f49bed9d640 1 --2- 172.21.15.12:0/1750757892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49c0071110 0x7f49c0158110 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009e50 tx=0x7f49b801f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ce7052452a7fac server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:38.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.362+0000 7f49bd59a640 1 -- 172.21.15.12:0/1750757892 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b80af400 con 0x7f49c0071110 2023-10-30T12:47:38.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.362+0000 7f49bd59a640 1 -- 172.21.15.12:0/1750757892 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b80ab050 con 0x7f49c0071110 2023-10-30T12:47:38.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.362+0000 7f49bd59a640 1 -- 172.21.15.12:0/1750757892 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b80af780 con 0x7f49c0071110 2023-10-30T12:47:38.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 -- 172.21.15.12:0/1750757892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49c0071110 msgr2=0x7f49c0158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:38.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 --2- 172.21.15.12:0/1750757892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49c0071110 0x7f49c0158110 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009e50 tx=0x7f49b801f1f0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 -- 172.21.15.12:0/1750757892 shutdown_connections 2023-10-30T12:47:38.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 --2- 172.21.15.12:0/1750757892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49c0071110 0x7f49c0158110 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 --2- 172.21.15.12:0/1750757892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49c00c6bd0 0x7f49c006fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 -- 172.21.15.12:0/1750757892 >> 172.21.15.12:0/1750757892 conn(0x7f49c00c7910 msgr2=0x7f49c00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 -- 172.21.15.12:0/1750757892 shutdown_connections 2023-10-30T12:47:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 -- 172.21.15.12:0/1750757892 wait complete. 2023-10-30T12:47:38.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 Processor -- start 2023-10-30T12:47:38.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 -- start start 2023-10-30T12:47:38.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49c006f7a0 0x7f49c00c3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:38.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49c0071110 0x7f49c00ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:38.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49c00c6bd0 0x7f49c00aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:38.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00af8f0 con 0x7f49c00c6bd0 2023-10-30T12:47:38.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c01ef380 con 0x7f49c0071110 2023-10-30T12:47:38.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49c4b9d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c01ef8b0 con 0x7f49c006f7a0 2023-10-30T12:47:38.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49bed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49c00c6bd0 0x7f49c00aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:38.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49bed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49c00c6bd0 0x7f49c00aeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50190/0 (socket says 172.21.15.12:50190) 2023-10-30T12:47:38.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49bed9d640 1 -- 172.21.15.12:0/1104029865 learned_addr learned my addr 172.21.15.12:0/1104029865 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:38.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49be59c640 1 --2- 172.21.15.12:0/1104029865 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49c006f7a0 0x7f49c00c3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:38.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.366+0000 7f49bdd9b640 1 --2- 172.21.15.12:0/1104029865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49c0071110 0x7f49c00ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:38.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.370+0000 7f49bed9d640 1 -- 172.21.15.12:0/1104029865 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49c006f7a0 msgr2=0x7f49c00c3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:38.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.370+0000 7f49bed9d640 1 --2- 172.21.15.12:0/1104029865 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49c006f7a0 0x7f49c00c3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.370+0000 7f49bed9d640 1 -- 172.21.15.12:0/1104029865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49c0071110 msgr2=0x7f49c00ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:38.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.370+0000 7f49bed9d640 1 --2- 172.21.15.12:0/1104029865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49c0071110 0x7f49c00ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.370+0000 7f49bed9d640 1 -- 172.21.15.12:0/1104029865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b8009b30 con 0x7f49c00c6bd0 2023-10-30T12:47:38.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.370+0000 7f49bed9d640 1 --2- 172.21.15.12:0/1104029865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49c00c6bd0 0x7f49c00aeb80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f49b801ce20 tx=0x7f49b801bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.370+0000 7f49aeffd640 1 -- 172.21.15.12:0/1104029865 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b80afde0 con 0x7f49c00c6bd0 2023-10-30T12:47:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.370+0000 7f49aeffd640 1 -- 172.21.15.12:0/1104029865 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b8009510 con 0x7f49c00c6bd0 2023-10-30T12:47:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.370+0000 7f49c4b9d640 1 -- 172.21.15.12:0/1104029865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c014f270 con 0x7f49c00c6bd0 2023-10-30T12:47:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.370+0000 7f49aeffd640 1 -- 172.21.15.12:0/1104029865 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b80ba660 con 0x7f49c00c6bd0 2023-10-30T12:47:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.370+0000 7f49c4b9d640 1 -- 172.21.15.12:0/1104029865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c014f510 con 0x7f49c00c6bd0 2023-10-30T12:47:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.370+0000 7f49c4b9d640 1 -- 172.21.15.12:0/1104029865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c00ae330 con 0x7f49c00c6bd0 2023-10-30T12:47:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.370+0000 7f49aeffd640 1 -- 172.21.15.12:0/1104029865 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f49b80b6040 con 0x7f49c00c6bd0 2023-10-30T12:47:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.370+0000 7f49aeffd640 1 --2- 172.21.15.12:0/1104029865 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f49a00664d0 0x7f49a0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.370+0000 7f49aeffd640 1 -- 172.21.15.12:0/1104029865 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f49b8122210 con 0x7f49c00c6bd0 2023-10-30T12:47:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.374+0000 7f49be59c640 1 --2- 172.21.15.12:0/1104029865 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f49a00664d0 0x7f49a0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.374+0000 7f49be59c640 1 --2- 172.21.15.12:0/1104029865 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f49a00664d0 0x7f49a0068990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f49b0000f70 tx=0x7f49b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:38.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.374+0000 7f49aeffd640 1 -- 172.21.15.12:0/1104029865 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f49b80ef530 con 0x7f49c00c6bd0 2023-10-30T12:47:38.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.570+0000 7f49c4b9d640 1 -- 172.21.15.12:0/1104029865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49c004e170 con 0x7f49c00c6bd0 2023-10-30T12:47:38.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.570+0000 7f49aeffd640 1 -- 172.21.15.12:0/1104029865 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+6535 (secure 0 0 0) 0x7f49b80f2b50 con 0x7f49c00c6bd0 2023-10-30T12:47:38.575 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:38.576 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:03.591793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:38.576 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 62 2023-10-30T12:47:38.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.570+0000 7f49c4b9d640 1 -- 172.21.15.12:0/1104029865 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f49a00664d0 msgr2=0x7f49a0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:38.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.570+0000 7f49c4b9d640 1 --2- 172.21.15.12:0/1104029865 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f49a00664d0 0x7f49a0068990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f49b0000f70 tx=0x7f49b0009040 comp rx=0 tx=0).stop 2023-10-30T12:47:38.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.570+0000 7f49c4b9d640 1 -- 172.21.15.12:0/1104029865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49c00c6bd0 msgr2=0x7f49c00aeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:38.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.570+0000 7f49c4b9d640 1 --2- 172.21.15.12:0/1104029865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49c00c6bd0 0x7f49c00aeb80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f49b801ce20 tx=0x7f49b801bf90 comp rx=0 tx=0).stop 2023-10-30T12:47:38.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.570+0000 7f49c4b9d640 1 -- 172.21.15.12:0/1104029865 shutdown_connections 2023-10-30T12:47:38.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.570+0000 7f49c4b9d640 1 --2- 172.21.15.12:0/1104029865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49c00c6bd0 0x7f49c00aeb80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.570+0000 7f49c4b9d640 1 --2- 172.21.15.12:0/1104029865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49c0071110 0x7f49c00ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.570+0000 7f49c4b9d640 1 --2- 172.21.15.12:0/1104029865 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f49a00664d0 0x7f49a0068990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.570+0000 7f49c4b9d640 1 --2- 172.21.15.12:0/1104029865 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49c006f7a0 0x7f49c00c3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.570+0000 7f49c4b9d640 1 -- 172.21.15.12:0/1104029865 >> 172.21.15.12:0/1104029865 conn(0x7f49c00c7910 msgr2=0x7f49c00c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:38.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.570+0000 7f49c4b9d640 1 -- 172.21.15.12:0/1104029865 shutdown_connections 2023-10-30T12:47:38.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:38.570+0000 7f49c4b9d640 1 -- 172.21.15.12:0/1104029865 wait complete. 2023-10-30T12:47:38.601 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:47:40.602 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:40.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.750+0000 7f98b0f3e640 1 Processor -- start 2023-10-30T12:47:40.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.750+0000 7f98b0f3e640 1 -- start start 2023-10-30T12:47:40.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98b0f3e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f98ac069e90 0x7f98ac15a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98b0f3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98ac06cb80 0x7f98ac06d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98b0f3e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac06d550 con 0x7f98ac06a270 2023-10-30T12:47:40.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98b0f3e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ac0c8570 con 0x7f98ac069e90 2023-10-30T12:47:40.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98b0f3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0c6c10 con 0x7f98ac06cb80 2023-10-30T12:47:40.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a3fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f98ac06a270 0x7f98ac0c9cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48230/0 (socket says 172.21.15.12:48230) 2023-10-30T12:47:40.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a3fff640 1 -- 172.21.15.12:0/2847314044 learned_addr learned my addr 172.21.15.12:0/2847314044 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:47:40.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98aad9d640 1 --2- 172.21.15.12:0/2847314044 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98ac06cb80 0x7f98ac06d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98aa59c640 1 --2- 172.21.15.12:0/2847314044 >> v2:172.21.15.154:3300/0 conn(0x7f98ac069e90 0x7f98ac15a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 -- 172.21.15.12:0/2847314044 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397518973 0 0) 0x7f98ac06d550 con 0x7f98ac06a270 2023-10-30T12:47:40.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 -- 172.21.15.12:0/2847314044 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9898003530 con 0x7f98ac06a270 2023-10-30T12:47:40.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 -- 172.21.15.12:0/2847314044 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3472107983 0 0) 0x7f9898003530 con 0x7f98ac06a270 2023-10-30T12:47:40.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 -- 172.21.15.12:0/2847314044 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98ac06cb80 msgr2=0x7f98ac06d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:40.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 --2- 172.21.15.12:0/2847314044 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98ac06cb80 0x7f98ac06d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 -- 172.21.15.12:0/2847314044 >> v2:172.21.15.154:3300/0 conn(0x7f98ac069e90 msgr2=0x7f98ac15a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:40.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 --2- 172.21.15.12:0/2847314044 >> v2:172.21.15.154:3300/0 conn(0x7f98ac069e90 0x7f98ac15a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 -- 172.21.15.12:0/2847314044 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac0c6ef0 con 0x7f98ac06a270 2023-10-30T12:47:40.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 -- 172.21.15.12:0/2847314044 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f9894001680 con 0x7f98ac06a270 2023-10-30T12:47:40.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 -- 172.21.15.12:0/2847314044 >> v1:172.21.15.12:6789/0 conn(0x7f98ac06a270 legacy=0x7f98ac0c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:40.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 --2- 172.21.15.12:0/2847314044 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98ac06cb80 0x7f9898003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 --2- 172.21.15.12:0/2847314044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9898003f00 0x7f98980047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 --2- 172.21.15.12:0/2847314044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9898004d10 0x7f9898005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 -- 172.21.15.12:0/2847314044 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0c8570 con 0x7f9898003f00 2023-10-30T12:47:40.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 -- 172.21.15.12:0/2847314044 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0c6c10 con 0x7f9898004d10 2023-10-30T12:47:40.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a9d9b640 1 -- 172.21.15.12:0/2847314044 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989800b990 con 0x7f98ac06cb80 2023-10-30T12:47:40.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a3fff640 1 --2- 172.21.15.12:0/2847314044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9898003f00 0x7f98980047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98aa59c640 1 --2- 172.21.15.12:0/2847314044 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98ac06cb80 0x7f9898003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98aad9d640 1 --2- 172.21.15.12:0/2847314044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9898004d10 0x7f9898005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a3fff640 1 -- 172.21.15.12:0/2847314044 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98ac06cb80 msgr2=0x7f9898003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:40.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a3fff640 1 --2- 172.21.15.12:0/2847314044 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98ac06cb80 0x7f9898003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a3fff640 1 -- 172.21.15.12:0/2847314044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9898004d10 msgr2=0x7f9898005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:40.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a3fff640 1 --2- 172.21.15.12:0/2847314044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9898004d10 0x7f9898005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98a3fff640 1 -- 172.21.15.12:0/2847314044 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9898003530 con 0x7f9898003f00 2023-10-30T12:47:40.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.754+0000 7f98aad9d640 1 --2- 172.21.15.12:0/2847314044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9898004d10 0x7f9898005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:40.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.758+0000 7f98a3fff640 1 --2- 172.21.15.12:0/2847314044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9898003f00 0x7f98980047d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f989400c240 tx=0x7f989401f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f87730fed09db36 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:40.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.758+0000 7f98a9d9b640 1 -- 172.21.15.12:0/2847314044 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894005ae0 con 0x7f9898003f00 2023-10-30T12:47:40.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.758+0000 7f98a9d9b640 1 -- 172.21.15.12:0/2847314044 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98940ab070 con 0x7f9898003f00 2023-10-30T12:47:40.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.758+0000 7f98a9d9b640 1 -- 172.21.15.12:0/2847314044 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894007070 con 0x7f9898003f00 2023-10-30T12:47:40.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.758+0000 7f98b0f3e640 1 -- 172.21.15.12:0/2847314044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9898003f00 msgr2=0x7f98980047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:40.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.758+0000 7f98b0f3e640 1 --2- 172.21.15.12:0/2847314044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9898003f00 0x7f98980047d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f989400c240 tx=0x7f989401f370 comp rx=0 tx=0).stop 2023-10-30T12:47:40.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.758+0000 7f98aad9d640 1 -- 172.21.15.12:0/2847314044 reap_dead start 2023-10-30T12:47:40.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.758+0000 7f98b0f3e640 1 -- 172.21.15.12:0/2847314044 shutdown_connections 2023-10-30T12:47:40.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.758+0000 7f98b0f3e640 1 -- 172.21.15.12:0/2847314044 >> 172.21.15.12:0/2847314044 conn(0x7f98ac06dfd0 msgr2=0x7f98ac06e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:40.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.758+0000 7f98b0f3e640 1 -- 172.21.15.12:0/2847314044 shutdown_connections 2023-10-30T12:47:40.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.758+0000 7f98b0f3e640 1 -- 172.21.15.12:0/2847314044 wait complete. 2023-10-30T12:47:40.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.758+0000 7f98b0f3e640 1 Processor -- start 2023-10-30T12:47:40.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.758+0000 7f98b0f3e640 1 -- start start 2023-10-30T12:47:40.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98b0f3e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9898003f00 0x7f98ac0c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98b0f3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98ac0cb600 0x7f98ac0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98b0f3e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac14b950 0x7f98ac14bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98b0f3e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0717e0 con 0x7f9898003f00 2023-10-30T12:47:40.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98b0f3e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac154870 con 0x7f98ac14b950 2023-10-30T12:47:40.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98b0f3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac14c310 con 0x7f98ac0cb600 2023-10-30T12:47:40.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98aa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9898003f00 0x7f98ac0c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98aa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9898003f00 0x7f98ac0c48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50202/0 (socket says 172.21.15.12:50202) 2023-10-30T12:47:40.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98aa59c640 1 -- 172.21.15.12:0/586226921 learned_addr learned my addr 172.21.15.12:0/586226921 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:40.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98a3fff640 1 --2- 172.21.15.12:0/586226921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98ac0cb600 0x7f98ac0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98aad9d640 1 --2- 172.21.15.12:0/586226921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac14b950 0x7f98ac14bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98aa59c640 1 -- 172.21.15.12:0/586226921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98ac0cb600 msgr2=0x7f98ac0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:40.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98aa59c640 1 --2- 172.21.15.12:0/586226921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98ac0cb600 0x7f98ac0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98aa59c640 1 -- 172.21.15.12:0/586226921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac14b950 msgr2=0x7f98ac14bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:40.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98aa59c640 1 --2- 172.21.15.12:0/586226921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac14b950 0x7f98ac14bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98aa59c640 1 -- 172.21.15.12:0/586226921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac155ff0 con 0x7f9898003f00 2023-10-30T12:47:40.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98aad9d640 1 --2- 172.21.15.12:0/586226921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac14b950 0x7f98ac14bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:47:40.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98aa59c640 1 --2- 172.21.15.12:0/586226921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9898003f00 0x7f98ac0c48f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f989c011aa0 tx=0x7f989c011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:40.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98a2ffd640 1 -- 172.21.15.12:0/586226921 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c00fb80 con 0x7f9898003f00 2023-10-30T12:47:40.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98a2ffd640 1 -- 172.21.15.12:0/586226921 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c00fce0 con 0x7f9898003f00 2023-10-30T12:47:40.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98a2ffd640 1 -- 172.21.15.12:0/586226921 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c014680 con 0x7f9898003f00 2023-10-30T12:47:40.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98b0f3e640 1 -- 172.21.15.12:0/586226921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac154ae0 con 0x7f9898003f00 2023-10-30T12:47:40.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98b0f3e640 1 -- 172.21.15.12:0/586226921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac0c2970 con 0x7f9898003f00 2023-10-30T12:47:40.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.762+0000 7f98b0f3e640 1 -- 172.21.15.12:0/586226921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ac04e170 con 0x7f9898003f00 2023-10-30T12:47:40.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.766+0000 7f98a2ffd640 1 -- 172.21.15.12:0/586226921 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f989c0147e0 con 0x7f9898003f00 2023-10-30T12:47:40.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.766+0000 7f98a2ffd640 1 --2- 172.21.15.12:0/586226921 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f988c0664f0 0x7f988c0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.766+0000 7f98a2ffd640 1 -- 172.21.15.12:0/586226921 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f989c084ab0 con 0x7f9898003f00 2023-10-30T12:47:40.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.766+0000 7f98a3fff640 1 --2- 172.21.15.12:0/586226921 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f988c0664f0 0x7f988c0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.766+0000 7f98a3fff640 1 --2- 172.21.15.12:0/586226921 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f988c0664f0 0x7f988c0689b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f98940ae000 tx=0x7f9894001a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:40.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.770+0000 7f98a2ffd640 1 -- 172.21.15.12:0/586226921 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f989c051c60 con 0x7f9898003f00 2023-10-30T12:47:40.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.974+0000 7f98b0f3e640 1 -- 172.21.15.12:0/586226921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98ac0c5f10 con 0x7f9898003f00 2023-10-30T12:47:40.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.974+0000 7f98a2ffd640 1 -- 172.21.15.12:0/586226921 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6536 (secure 0 0 0) 0x7f989c055280 con 0x7f9898003f00 2023-10-30T12:47:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:40.980 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:39.347552+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:40.980 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 63 2023-10-30T12:47:40.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.974+0000 7f98b0f3e640 1 -- 172.21.15.12:0/586226921 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f988c0664f0 msgr2=0x7f988c0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:40.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.974+0000 7f98b0f3e640 1 --2- 172.21.15.12:0/586226921 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f988c0664f0 0x7f988c0689b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f98940ae000 tx=0x7f9894001a80 comp rx=0 tx=0).stop 2023-10-30T12:47:40.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.974+0000 7f98b0f3e640 1 -- 172.21.15.12:0/586226921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9898003f00 msgr2=0x7f98ac0c48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:40.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.974+0000 7f98b0f3e640 1 --2- 172.21.15.12:0/586226921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9898003f00 0x7f98ac0c48f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f989c011aa0 tx=0x7f989c011f60 comp rx=0 tx=0).stop 2023-10-30T12:47:40.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.974+0000 7f98b0f3e640 1 -- 172.21.15.12:0/586226921 shutdown_connections 2023-10-30T12:47:40.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.974+0000 7f98b0f3e640 1 --2- 172.21.15.12:0/586226921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac14b950 0x7f98ac14bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.974+0000 7f98b0f3e640 1 --2- 172.21.15.12:0/586226921 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f988c0664f0 0x7f988c0689b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.974+0000 7f98b0f3e640 1 --2- 172.21.15.12:0/586226921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98ac0cb600 0x7f98ac0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.974+0000 7f98b0f3e640 1 --2- 172.21.15.12:0/586226921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9898003f00 0x7f98ac0c48f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.974+0000 7f98b0f3e640 1 -- 172.21.15.12:0/586226921 >> 172.21.15.12:0/586226921 conn(0x7f98ac06dfd0 msgr2=0x7f98ac153940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:40.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.974+0000 7f98b0f3e640 1 -- 172.21.15.12:0/586226921 shutdown_connections 2023-10-30T12:47:40.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:40.974+0000 7f98b0f3e640 1 -- 172.21.15.12:0/586226921 wait complete. 2023-10-30T12:47:41.000 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:47:43.001 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:43.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.162+0000 7fe98f2dc640 1 Processor -- start 2023-10-30T12:47:43.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.162+0000 7fe98f2dc640 1 -- start start 2023-10-30T12:47:43.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.162+0000 7fe98f2dc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe9881538f0 0x7fe988159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:43.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.162+0000 7fe98f2dc640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9880cefa0 0x7fe9880cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:43.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.162+0000 7fe98f2dc640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9880ca6a0 con 0x7fe98815cd50 2023-10-30T12:47:43.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.162+0000 7fe98f2dc640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98815d300 con 0x7fe9881538f0 2023-10-30T12:47:43.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.162+0000 7fe98f2dc640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98815d5e0 con 0x7fe9880cefa0 2023-10-30T12:47:43.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.162+0000 7fe98e2da640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe9881538f0 0x7fe988159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:43.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.162+0000 7fe98e2da640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe9881538f0 0x7fe988159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40616/0 (socket says 172.21.15.12:40616) 2023-10-30T12:47:43.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.162+0000 7fe98e2da640 1 -- 172.21.15.12:0/1738505422 learned_addr learned my addr 172.21.15.12:0/1738505422 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:43.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.162+0000 7fe98dad9640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe98815cd50 0x7fe9880c9f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44952/0 (socket says 172.21.15.12:44952) 2023-10-30T12:47:43.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.166+0000 7fe98eadb640 1 --2- 172.21.15.12:0/1738505422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9880cefa0 0x7fe9880cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:43.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.166+0000 7fe98e2da640 1 -- 172.21.15.12:0/1738505422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9880cefa0 msgr2=0x7fe9880cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:43.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.166+0000 7fe98e2da640 1 --2- 172.21.15.12:0/1738505422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9880cefa0 0x7fe9880cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:43.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.166+0000 7fe98e2da640 1 -- 172.21.15.12:0/1738505422 >> v1:172.21.15.12:6789/0 conn(0x7fe98815cd50 legacy=0x7fe9880c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:43.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.166+0000 7fe98e2da640 1 -- 172.21.15.12:0/1738505422 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98815d8c0 con 0x7fe9881538f0 2023-10-30T12:47:43.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.166+0000 7fe98d2d8640 1 -- 172.21.15.12:0/1738505422 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371997284 0 0) 0x7fe9880ca6a0 con 0x7fe98815cd50 2023-10-30T12:47:43.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.166+0000 7fe98eadb640 1 --2- 172.21.15.12:0/1738505422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9880cefa0 0x7fe9880cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:43.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.166+0000 7fe98e2da640 1 --2- 172.21.15.12:0/1738505422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9881538f0 0x7fe988159fb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe984009870 tx=0x7fe98401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18c5b68055b56f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:43.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.166+0000 7fe98d2d8640 1 -- 172.21.15.12:0/1738505422 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9840a8a30 con 0x7fe9881538f0 2023-10-30T12:47:43.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.166+0000 7fe98d2d8640 1 -- 172.21.15.12:0/1738505422 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9840a8b90 con 0x7fe9881538f0 2023-10-30T12:47:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.166+0000 7fe98d2d8640 1 -- 172.21.15.12:0/1738505422 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9840b0440 con 0x7fe9881538f0 2023-10-30T12:47:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.166+0000 7fe98f2dc640 1 -- 172.21.15.12:0/1738505422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9881538f0 msgr2=0x7fe988159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.166+0000 7fe98f2dc640 1 --2- 172.21.15.12:0/1738505422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9881538f0 0x7fe988159fb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe984009870 tx=0x7fe98401ed30 comp rx=0 tx=0).stop 2023-10-30T12:47:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98f2dc640 1 -- 172.21.15.12:0/1738505422 shutdown_connections 2023-10-30T12:47:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98f2dc640 1 --2- 172.21.15.12:0/1738505422 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9880cefa0 0x7fe9880cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98f2dc640 1 --2- 172.21.15.12:0/1738505422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9881538f0 0x7fe988159fb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98f2dc640 1 -- 172.21.15.12:0/1738505422 >> 172.21.15.12:0/1738505422 conn(0x7fe98806a640 msgr2=0x7fe988065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98f2dc640 1 -- 172.21.15.12:0/1738505422 shutdown_connections 2023-10-30T12:47:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98f2dc640 1 -- 172.21.15.12:0/1738505422 wait complete. 2023-10-30T12:47:43.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98f2dc640 1 Processor -- start 2023-10-30T12:47:43.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98f2dc640 1 -- start start 2023-10-30T12:47:43.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98f2dc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9880cefa0 0x7fe9881f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:43.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98f2dc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9881538f0 0x7fe988147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:43.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98f2dc640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98815cd50 0x7fe9880b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:43.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98f2dc640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9881f34b0 con 0x7fe9880cefa0 2023-10-30T12:47:43.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98f2dc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98815a920 con 0x7fe9881538f0 2023-10-30T12:47:43.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98f2dc640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9881ef100 con 0x7fe98815cd50 2023-10-30T12:47:43.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98e2da640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9880cefa0 0x7fe9881f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:43.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98e2da640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9880cefa0 0x7fe9881f2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40628/0 (socket says 172.21.15.12:40628) 2023-10-30T12:47:43.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98e2da640 1 -- 172.21.15.12:0/2806150656 learned_addr learned my addr 172.21.15.12:0/2806150656 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:43.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98dad9640 1 --2- 172.21.15.12:0/2806150656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9881538f0 0x7fe988147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:43.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98eadb640 1 --2- 172.21.15.12:0/2806150656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98815cd50 0x7fe9880b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:43.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98e2da640 1 -- 172.21.15.12:0/2806150656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98815cd50 msgr2=0x7fe9880b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:43.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98e2da640 1 --2- 172.21.15.12:0/2806150656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98815cd50 0x7fe9880b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:43.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98e2da640 1 -- 172.21.15.12:0/2806150656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9881538f0 msgr2=0x7fe988147ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:43.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98e2da640 1 --2- 172.21.15.12:0/2806150656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9881538f0 0x7fe988147ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:43.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98e2da640 1 -- 172.21.15.12:0/2806150656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9840094d0 con 0x7fe9880cefa0 2023-10-30T12:47:43.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.170+0000 7fe98e2da640 1 --2- 172.21.15.12:0/2806150656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9880cefa0 0x7fe9881f2f70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe9840b3040 tx=0x7fe98401bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:43.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.174+0000 7fe97effd640 1 -- 172.21.15.12:0/2806150656 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9840be070 con 0x7fe9880cefa0 2023-10-30T12:47:43.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.174+0000 7fe97effd640 1 -- 172.21.15.12:0/2806150656 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9840ba400 con 0x7fe9880cefa0 2023-10-30T12:47:43.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.174+0000 7fe97effd640 1 -- 172.21.15.12:0/2806150656 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9840ba7d0 con 0x7fe9880cefa0 2023-10-30T12:47:43.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.174+0000 7fe98f2dc640 1 -- 172.21.15.12:0/2806150656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98815ab70 con 0x7fe9880cefa0 2023-10-30T12:47:43.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.174+0000 7fe98f2dc640 1 -- 172.21.15.12:0/2806150656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9881ef680 con 0x7fe9880cefa0 2023-10-30T12:47:43.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.174+0000 7fe98f2dc640 1 -- 172.21.15.12:0/2806150656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98804e170 con 0x7fe9880cefa0 2023-10-30T12:47:43.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.174+0000 7fe97effd640 1 -- 172.21.15.12:0/2806150656 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe9840c3410 con 0x7fe9880cefa0 2023-10-30T12:47:43.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.174+0000 7fe97effd640 1 --2- 172.21.15.12:0/2806150656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe96c0664a0 0x7fe96c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:43.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.174+0000 7fe97effd640 1 -- 172.21.15.12:0/2806150656 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fe984121d30 con 0x7fe9880cefa0 2023-10-30T12:47:43.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.174+0000 7fe98dad9640 1 --2- 172.21.15.12:0/2806150656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe96c0664a0 0x7fe96c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:43.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.178+0000 7fe98dad9640 1 --2- 172.21.15.12:0/2806150656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe96c0664a0 0x7fe96c068960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe9880b5270 tx=0x7fe978007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:43.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.178+0000 7fe97effd640 1 -- 172.21.15.12:0/2806150656 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe9840eeef0 con 0x7fe9880cefa0 2023-10-30T12:47:43.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.370+0000 7fe98f2dc640 1 -- 172.21.15.12:0/2806150656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9880b2290 con 0x7fe9880cefa0 2023-10-30T12:47:43.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.370+0000 7fe97effd640 1 -- 172.21.15.12:0/2806150656 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6536 (secure 0 0 0) 0x7fe9840b5090 con 0x7fe9880cefa0 2023-10-30T12:47:43.376 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:43.377 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:39.347552+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5000,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5000":{"gid":5000,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6839/1635719967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1635719967},{"type":"v1","addr":"172.21.15.154:6839","nonce":1635719967}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:43.377 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 63 2023-10-30T12:47:43.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.374+0000 7fe98f2dc640 1 -- 172.21.15.12:0/2806150656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe96c0664a0 msgr2=0x7fe96c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:43.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.374+0000 7fe98f2dc640 1 --2- 172.21.15.12:0/2806150656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe96c0664a0 0x7fe96c068960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe9880b5270 tx=0x7fe978007c10 comp rx=0 tx=0).stop 2023-10-30T12:47:43.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.374+0000 7fe98f2dc640 1 -- 172.21.15.12:0/2806150656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9880cefa0 msgr2=0x7fe9881f2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:43.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.374+0000 7fe98f2dc640 1 --2- 172.21.15.12:0/2806150656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9880cefa0 0x7fe9881f2f70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe9840b3040 tx=0x7fe98401bad0 comp rx=0 tx=0).stop 2023-10-30T12:47:43.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.374+0000 7fe98f2dc640 1 -- 172.21.15.12:0/2806150656 shutdown_connections 2023-10-30T12:47:43.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.374+0000 7fe98f2dc640 1 --2- 172.21.15.12:0/2806150656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe96c0664a0 0x7fe96c068960 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:43.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.374+0000 7fe98f2dc640 1 --2- 172.21.15.12:0/2806150656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98815cd50 0x7fe9880b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:43.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.374+0000 7fe98f2dc640 1 --2- 172.21.15.12:0/2806150656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9881538f0 0x7fe988147ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:43.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.374+0000 7fe98f2dc640 1 --2- 172.21.15.12:0/2806150656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9880cefa0 0x7fe9881f2f70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:43.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.374+0000 7fe98f2dc640 1 -- 172.21.15.12:0/2806150656 >> 172.21.15.12:0/2806150656 conn(0x7fe98806a640 msgr2=0x7fe9880cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:43.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.374+0000 7fe98f2dc640 1 -- 172.21.15.12:0/2806150656 shutdown_connections 2023-10-30T12:47:43.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:43.374+0000 7fe98f2dc640 1 -- 172.21.15.12:0/2806150656 wait complete. 2023-10-30T12:47:43.397 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:47:45.398 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:45.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.546+0000 7fbced3a1640 1 Processor -- start 2023-10-30T12:47:45.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.546+0000 7fbced3a1640 1 -- start start 2023-10-30T12:47:45.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbced3a1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbce806f7a0 0x7fbce806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbced3a1640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce8066cf0 con 0x7fbce8071110 2023-10-30T12:47:45.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbced3a1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce80c70d0 con 0x7fbce80c6bd0 2023-10-30T12:47:45.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbced3a1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce80700c0 con 0x7fbce806f7a0 2023-10-30T12:47:45.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce77fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbce8071110 0x7fbce8158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44964/0 (socket says 172.21.15.12:44964) 2023-10-30T12:47:45.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce77fe640 1 -- 172.21.15.12:0/4268180868 learned_addr learned my addr 172.21.15.12:0/4268180868 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:47:45.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce6ffd640 1 --2- 172.21.15.12:0/4268180868 >> v2:172.21.15.154:3300/0 conn(0x7fbce806f7a0 0x7fbce806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce67fc640 1 -- 172.21.15.12:0/4268180868 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2128114689 0 0) 0x7fbce8066cf0 con 0x7fbce8071110 2023-10-30T12:47:45.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce67fc640 1 -- 172.21.15.12:0/4268180868 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcd4003530 con 0x7fbce8071110 2023-10-30T12:47:45.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce67fc640 1 -- 172.21.15.12:0/4268180868 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3116183327 0 0) 0x7fbcd4003530 con 0x7fbce8071110 2023-10-30T12:47:45.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce67fc640 1 -- 172.21.15.12:0/4268180868 >> v2:172.21.15.154:3300/0 conn(0x7fbce806f7a0 msgr2=0x7fbce806fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:45.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce67fc640 1 --2- 172.21.15.12:0/4268180868 >> v2:172.21.15.154:3300/0 conn(0x7fbce806f7a0 0x7fbce806fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce67fc640 1 -- 172.21.15.12:0/4268180868 >> v1:172.21.15.154:6789/0 conn(0x7fbce80c6bd0 legacy=0x7fbce8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:45.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce67fc640 1 -- 172.21.15.12:0/4268180868 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce80c7390 con 0x7fbce8071110 2023-10-30T12:47:45.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce67fc640 1 -- 172.21.15.12:0/4268180868 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fbce0001a00 con 0x7fbce8071110 2023-10-30T12:47:45.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce67fc640 1 -- 172.21.15.12:0/4268180868 >> v1:172.21.15.12:6789/0 conn(0x7fbce8071110 legacy=0x7fbce8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:45.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce67fc640 1 --2- 172.21.15.12:0/4268180868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcd4003980 0x7fbcd4003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce67fc640 1 --2- 172.21.15.12:0/4268180868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcd4004380 0x7fbcd4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce67fc640 1 --2- 172.21.15.12:0/4268180868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd4005050 0x7fbcd400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce67fc640 1 -- 172.21.15.12:0/4268180868 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce80700c0 con 0x7fbcd4005050 2023-10-30T12:47:45.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce67fc640 1 -- 172.21.15.12:0/4268180868 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd400bf00 con 0x7fbcd4004380 2023-10-30T12:47:45.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce67fc640 1 -- 172.21.15.12:0/4268180868 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd400c0c0 con 0x7fbcd4003980 2023-10-30T12:47:45.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce77fe640 1 --2- 172.21.15.12:0/4268180868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd4005050 0x7fbcd400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce6ffd640 1 --2- 172.21.15.12:0/4268180868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcd4003980 0x7fbcd4003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbcdffff640 1 --2- 172.21.15.12:0/4268180868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcd4004380 0x7fbcd4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce77fe640 1 -- 172.21.15.12:0/4268180868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcd4003980 msgr2=0x7fbcd4003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:45.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce77fe640 1 --2- 172.21.15.12:0/4268180868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcd4003980 0x7fbcd4003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce77fe640 1 -- 172.21.15.12:0/4268180868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcd4004380 msgr2=0x7fbcd4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:45.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce77fe640 1 --2- 172.21.15.12:0/4268180868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcd4004380 0x7fbcd4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.550+0000 7fbce77fe640 1 -- 172.21.15.12:0/4268180868 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd4003530 con 0x7fbcd4005050 2023-10-30T12:47:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.554+0000 7fbce77fe640 1 -- 172.21.15.12:0/4268180868 reap_dead start 2023-10-30T12:47:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.554+0000 7fbce77fe640 1 --2- 172.21.15.12:0/4268180868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd4005050 0x7fbcd400b990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbce000a830 tx=0x7fbce001d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a92cbf76cd42a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.554+0000 7fbce67fc640 1 -- 172.21.15.12:0/4268180868 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce00ad630 con 0x7fbcd4005050 2023-10-30T12:47:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.554+0000 7fbce67fc640 1 -- 172.21.15.12:0/4268180868 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce00a8070 con 0x7fbcd4005050 2023-10-30T12:47:45.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.554+0000 7fbce67fc640 1 -- 172.21.15.12:0/4268180868 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce00ad9f0 con 0x7fbcd4005050 2023-10-30T12:47:45.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.554+0000 7fbced3a1640 1 -- 172.21.15.12:0/4268180868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd4005050 msgr2=0x7fbcd400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:45.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.554+0000 7fbced3a1640 1 --2- 172.21.15.12:0/4268180868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd4005050 0x7fbcd400b990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbce000a830 tx=0x7fbce001d340 comp rx=0 tx=0).stop 2023-10-30T12:47:45.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.554+0000 7fbced3a1640 1 -- 172.21.15.12:0/4268180868 shutdown_connections 2023-10-30T12:47:45.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.554+0000 7fbced3a1640 1 --2- 172.21.15.12:0/4268180868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd4005050 0x7fbcd400b990 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.554+0000 7fbced3a1640 1 -- 172.21.15.12:0/4268180868 >> 172.21.15.12:0/4268180868 conn(0x7fbce80c7910 msgr2=0x7fbce80c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:45.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.554+0000 7fbced3a1640 1 -- 172.21.15.12:0/4268180868 shutdown_connections 2023-10-30T12:47:45.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.554+0000 7fbced3a1640 1 -- 172.21.15.12:0/4268180868 wait complete. 2023-10-30T12:47:45.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.554+0000 7fbced3a1640 1 Processor -- start 2023-10-30T12:47:45.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.554+0000 7fbced3a1640 1 -- start start 2023-10-30T12:47:45.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbced3a1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd4005050 0x7fbce814ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbced3a1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbce814cf40 0x7fbce814b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbced3a1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbce80c3d10 0x7fbce814b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbced3a1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce814af60 con 0x7fbcd4005050 2023-10-30T12:47:45.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbced3a1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce81eecd0 con 0x7fbce814cf40 2023-10-30T12:47:45.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbced3a1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce81eea30 con 0x7fbce80c3d10 2023-10-30T12:47:45.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbce6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd4005050 0x7fbce814ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbce6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd4005050 0x7fbce814ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40652/0 (socket says 172.21.15.12:40652) 2023-10-30T12:47:45.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbce6ffd640 1 -- 172.21.15.12:0/3345127983 learned_addr learned my addr 172.21.15.12:0/3345127983 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:45.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbce77fe640 1 --2- 172.21.15.12:0/3345127983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbce80c3d10 0x7fbce814b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbcdffff640 1 --2- 172.21.15.12:0/3345127983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbce814cf40 0x7fbce814b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbce6ffd640 1 -- 172.21.15.12:0/3345127983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbce80c3d10 msgr2=0x7fbce814b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:45.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbce6ffd640 1 --2- 172.21.15.12:0/3345127983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbce80c3d10 0x7fbce814b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbce6ffd640 1 -- 172.21.15.12:0/3345127983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbce814cf40 msgr2=0x7fbce814b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:45.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbce6ffd640 1 --2- 172.21.15.12:0/3345127983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbce814cf40 0x7fbce814b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbce6ffd640 1 -- 172.21.15.12:0/3345127983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce814a5c0 con 0x7fbcd4005050 2023-10-30T12:47:45.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbcdffff640 1 --2- 172.21.15.12:0/3345127983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbce814cf40 0x7fbce814b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:45.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbce6ffd640 1 --2- 172.21.15.12:0/3345127983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd4005050 0x7fbce814ede0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80115c0 tx=0x7fbcd8011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:45.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbcdf7fe640 1 -- 172.21.15.12:0/3345127983 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8015870 con 0x7fbcd4005050 2023-10-30T12:47:45.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbcdf7fe640 1 -- 172.21.15.12:0/3345127983 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd80159d0 con 0x7fbcd4005050 2023-10-30T12:47:45.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbcdf7fe640 1 -- 172.21.15.12:0/3345127983 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8015d40 con 0x7fbcd4005050 2023-10-30T12:47:45.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbced3a1640 1 -- 172.21.15.12:0/3345127983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce814a8a0 con 0x7fbcd4005050 2023-10-30T12:47:45.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbced3a1640 1 -- 172.21.15.12:0/3345127983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce80c91b0 con 0x7fbcd4005050 2023-10-30T12:47:45.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.558+0000 7fbced3a1640 1 -- 172.21.15.12:0/3345127983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce804e170 con 0x7fbcd4005050 2023-10-30T12:47:45.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.562+0000 7fbcdf7fe640 1 -- 172.21.15.12:0/3345127983 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbcd800f460 con 0x7fbcd4005050 2023-10-30T12:47:45.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.562+0000 7fbcdf7fe640 1 --2- 172.21.15.12:0/3345127983 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbcc80664d0 0x7fbcc8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.562+0000 7fbcdf7fe640 1 -- 172.21.15.12:0/3345127983 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7fbcd80843c0 con 0x7fbcd4005050 2023-10-30T12:47:45.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.562+0000 7fbcdffff640 1 --2- 172.21.15.12:0/3345127983 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbcc80664d0 0x7fbcc8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.562+0000 7fbcdffff640 1 --2- 172.21.15.12:0/3345127983 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbcc80664d0 0x7fbcc8068990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbce815bd90 tx=0x7fbcd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:45.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.566+0000 7fbcdf7fe640 1 -- 172.21.15.12:0/3345127983 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbcd80515a0 con 0x7fbcd4005050 2023-10-30T12:47:45.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.754+0000 7fbced3a1640 1 -- 172.21.15.12:0/3345127983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbce80c6610 con 0x7fbcd4005050 2023-10-30T12:47:45.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.754+0000 7fbcdf7fe640 1 -- 172.21.15.12:0/3345127983 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5738 (secure 0 0 0) 0x7fbcd8054bc0 con 0x7fbcd4005050 2023-10-30T12:47:45.762 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:45.762 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:45.331687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:45.762 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 64 2023-10-30T12:47:45.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.758+0000 7fbced3a1640 1 -- 172.21.15.12:0/3345127983 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbcc80664d0 msgr2=0x7fbcc8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:45.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.758+0000 7fbced3a1640 1 --2- 172.21.15.12:0/3345127983 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbcc80664d0 0x7fbcc8068990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbce815bd90 tx=0x7fbcd0009040 comp rx=0 tx=0).stop 2023-10-30T12:47:45.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.758+0000 7fbced3a1640 1 -- 172.21.15.12:0/3345127983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd4005050 msgr2=0x7fbce814ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:45.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.758+0000 7fbced3a1640 1 --2- 172.21.15.12:0/3345127983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd4005050 0x7fbce814ede0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80115c0 tx=0x7fbcd8011a80 comp rx=0 tx=0).stop 2023-10-30T12:47:45.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.758+0000 7fbced3a1640 1 -- 172.21.15.12:0/3345127983 shutdown_connections 2023-10-30T12:47:45.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.758+0000 7fbced3a1640 1 --2- 172.21.15.12:0/3345127983 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbcc80664d0 0x7fbcc8068990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.758+0000 7fbced3a1640 1 --2- 172.21.15.12:0/3345127983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbce80c3d10 0x7fbce814b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.758+0000 7fbced3a1640 1 --2- 172.21.15.12:0/3345127983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbce814cf40 0x7fbce814b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.758+0000 7fbced3a1640 1 --2- 172.21.15.12:0/3345127983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd4005050 0x7fbce814ede0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.758+0000 7fbced3a1640 1 -- 172.21.15.12:0/3345127983 >> 172.21.15.12:0/3345127983 conn(0x7fbce80c7910 msgr2=0x7fbce806bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.758+0000 7fbced3a1640 1 -- 172.21.15.12:0/3345127983 shutdown_connections 2023-10-30T12:47:45.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.758+0000 7fbced3a1640 1 -- 172.21.15.12:0/3345127983 wait complete. 2023-10-30T12:47:45.783 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2023-10-30T12:47:45.783 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:47:45.783 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:45.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8d032640 1 Processor -- start 2023-10-30T12:47:45.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8d032640 1 -- start start 2023-10-30T12:47:45.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8d032640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4a8806f7a0 0x7f4a88157610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8d032640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a8806fb80 0x7f4a88151490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8d032640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8806ad70 con 0x7f4a881519d0 2023-10-30T12:47:45.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8d032640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a88070490 con 0x7f4a8806f7a0 2023-10-30T12:47:45.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8d032640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8806a1f0 con 0x7f4a8806fb80 2023-10-30T12:47:45.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8759e640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4a881519d0 0x7f4a880708b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44970/0 (socket says 172.21.15.12:44970) 2023-10-30T12:47:45.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8759e640 1 -- 172.21.15.12:0/70565983 learned_addr learned my addr 172.21.15.12:0/70565983 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:47:45.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a86d9d640 1 --2- 172.21.15.12:0/70565983 >> v2:172.21.15.154:3300/0 conn(0x7f4a8806f7a0 0x7f4a88157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a7ffff640 1 --2- 172.21.15.12:0/70565983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a8806fb80 0x7f4a88151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 -- 172.21.15.12:0/70565983 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980912822 0 0) 0x7f4a8806ad70 con 0x7f4a881519d0 2023-10-30T12:47:45.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 -- 172.21.15.12:0/70565983 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a74003530 con 0x7f4a881519d0 2023-10-30T12:47:45.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 -- 172.21.15.12:0/70565983 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3912763235 0 0) 0x7f4a74003530 con 0x7f4a881519d0 2023-10-30T12:47:45.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 -- 172.21.15.12:0/70565983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a8806fb80 msgr2=0x7f4a88151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:45.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 --2- 172.21.15.12:0/70565983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a8806fb80 0x7f4a88151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 -- 172.21.15.12:0/70565983 >> v2:172.21.15.154:3300/0 conn(0x7f4a8806f7a0 msgr2=0x7f4a88157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:45.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 --2- 172.21.15.12:0/70565983 >> v2:172.21.15.154:3300/0 conn(0x7f4a8806f7a0 0x7f4a88157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 -- 172.21.15.12:0/70565983 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8806a4d0 con 0x7f4a881519d0 2023-10-30T12:47:45.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 -- 172.21.15.12:0/70565983 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f4a80001a00 con 0x7f4a881519d0 2023-10-30T12:47:45.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 -- 172.21.15.12:0/70565983 >> v1:172.21.15.12:6789/0 conn(0x7f4a881519d0 legacy=0x7f4a880708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:45.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 --2- 172.21.15.12:0/70565983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a74003980 0x7f4a74003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 --2- 172.21.15.12:0/70565983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a740047d0 0x7f4a74004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 --2- 172.21.15.12:0/70565983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a8806fb80 0x7f4a74005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 -- 172.21.15.12:0/70565983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a88070490 con 0x7f4a74003980 2023-10-30T12:47:45.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 -- 172.21.15.12:0/70565983 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8806a1f0 con 0x7f4a740047d0 2023-10-30T12:47:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.922+0000 7f4a8659c640 1 -- 172.21.15.12:0/70565983 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7400b990 con 0x7f4a8806fb80 2023-10-30T12:47:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.926+0000 7f4a86d9d640 1 --2- 172.21.15.12:0/70565983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a74003980 0x7f4a74003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.926+0000 7f4a7ffff640 1 --2- 172.21.15.12:0/70565983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a740047d0 0x7f4a74004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.926+0000 7f4a8759e640 1 --2- 172.21.15.12:0/70565983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a8806fb80 0x7f4a74005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.926+0000 7f4a86d9d640 1 -- 172.21.15.12:0/70565983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a8806fb80 msgr2=0x7f4a74005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:45.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.926+0000 7f4a86d9d640 1 --2- 172.21.15.12:0/70565983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a8806fb80 0x7f4a74005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.926+0000 7f4a86d9d640 1 -- 172.21.15.12:0/70565983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a740047d0 msgr2=0x7f4a74004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.926+0000 7f4a86d9d640 1 --2- 172.21.15.12:0/70565983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a740047d0 0x7f4a74004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.926+0000 7f4a86d9d640 1 -- 172.21.15.12:0/70565983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a74003530 con 0x7f4a74003980 2023-10-30T12:47:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.926+0000 7f4a7ffff640 1 --2- 172.21.15.12:0/70565983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a740047d0 0x7f4a74004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:45.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.926+0000 7f4a86d9d640 1 --2- 172.21.15.12:0/70565983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a74003980 0x7f4a74003da0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009ae0 tx=0x7f4a7801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1316afb020b2718 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:45.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.926+0000 7f4a8659c640 1 -- 172.21.15.12:0/70565983 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a780b0400 con 0x7f4a74003980 2023-10-30T12:47:45.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.926+0000 7f4a8659c640 1 -- 172.21.15.12:0/70565983 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a780b0560 con 0x7f4a74003980 2023-10-30T12:47:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.926+0000 7f4a8659c640 1 -- 172.21.15.12:0/70565983 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a780b0910 con 0x7f4a74003980 2023-10-30T12:47:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.926+0000 7f4a8d032640 1 -- 172.21.15.12:0/70565983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a74003980 msgr2=0x7f4a74003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.926+0000 7f4a8d032640 1 --2- 172.21.15.12:0/70565983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a74003980 0x7f4a74003da0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009ae0 tx=0x7f4a7801ca90 comp rx=0 tx=0).stop 2023-10-30T12:47:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8759e640 1 -- 172.21.15.12:0/70565983 reap_dead start 2023-10-30T12:47:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8d032640 1 -- 172.21.15.12:0/70565983 shutdown_connections 2023-10-30T12:47:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8d032640 1 -- 172.21.15.12:0/70565983 >> 172.21.15.12:0/70565983 conn(0x7f4a880c7910 msgr2=0x7f4a880c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8d032640 1 -- 172.21.15.12:0/70565983 shutdown_connections 2023-10-30T12:47:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8d032640 1 -- 172.21.15.12:0/70565983 wait complete. 2023-10-30T12:47:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8d032640 1 Processor -- start 2023-10-30T12:47:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8d032640 1 -- start start 2023-10-30T12:47:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8d032640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a74003980 0x7f4a880ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8d032640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a8815d970 0x7f4a88143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8d032640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a881436c0 0x7f4a88161250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8d032640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a880c8ca0 con 0x7f4a881436c0 2023-10-30T12:47:45.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8d032640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a880c8f50 con 0x7f4a8815d970 2023-10-30T12:47:45.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8d032640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a880c9d60 con 0x7f4a74003980 2023-10-30T12:47:45.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a881436c0 0x7f4a88161250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a881436c0 0x7f4a88161250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40676/0 (socket says 172.21.15.12:40676) 2023-10-30T12:47:45.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8759e640 1 -- 172.21.15.12:0/3482317988 learned_addr learned my addr 172.21.15.12:0/3482317988 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:45.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a7ffff640 1 --2- 172.21.15.12:0/3482317988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a8815d970 0x7f4a88143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a86d9d640 1 --2- 172.21.15.12:0/3482317988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a74003980 0x7f4a880ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8759e640 1 -- 172.21.15.12:0/3482317988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a74003980 msgr2=0x7f4a880ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:45.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8759e640 1 --2- 172.21.15.12:0/3482317988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a74003980 0x7f4a880ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8759e640 1 -- 172.21.15.12:0/3482317988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a8815d970 msgr2=0x7f4a88143180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:45.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8759e640 1 --2- 172.21.15.12:0/3482317988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a8815d970 0x7f4a88143180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:45.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.930+0000 7f4a8759e640 1 -- 172.21.15.12:0/3482317988 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a880c91d0 con 0x7f4a881436c0 2023-10-30T12:47:45.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.934+0000 7f4a8759e640 1 --2- 172.21.15.12:0/3482317988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a881436c0 0x7f4a88161250 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4a880c2580 tx=0x7f4a8000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:45.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.934+0000 7f4a7f7fe640 1 -- 172.21.15.12:0/3482317988 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8000eb90 con 0x7f4a881436c0 2023-10-30T12:47:45.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.934+0000 7f4a7f7fe640 1 -- 172.21.15.12:0/3482317988 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8000ed10 con 0x7f4a881436c0 2023-10-30T12:47:45.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.934+0000 7f4a7f7fe640 1 -- 172.21.15.12:0/3482317988 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a80014670 con 0x7f4a881436c0 2023-10-30T12:47:45.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.934+0000 7f4a8d032640 1 -- 172.21.15.12:0/3482317988 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a881f6110 con 0x7f4a881436c0 2023-10-30T12:47:45.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.934+0000 7f4a8d032640 1 -- 172.21.15.12:0/3482317988 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a880ca020 con 0x7f4a881436c0 2023-10-30T12:47:45.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.934+0000 7f4a8d032640 1 -- 172.21.15.12:0/3482317988 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a881f6630 con 0x7f4a881436c0 2023-10-30T12:47:45.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.934+0000 7f4a7f7fe640 1 -- 172.21.15.12:0/3482317988 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4a800147d0 con 0x7f4a881436c0 2023-10-30T12:47:45.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.934+0000 7f4a7f7fe640 1 --2- 172.21.15.12:0/3482317988 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4a680664d0 0x7f4a68068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:45.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.934+0000 7f4a7f7fe640 1 -- 172.21.15.12:0/3482317988 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f4a80083d80 con 0x7f4a881436c0 2023-10-30T12:47:45.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.938+0000 7f4a86d9d640 1 --2- 172.21.15.12:0/3482317988 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4a680664d0 0x7f4a68068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:45.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.938+0000 7f4a86d9d640 1 --2- 172.21.15.12:0/3482317988 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4a680664d0 0x7f4a68068990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4a780af000 tx=0x7f4a780b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:45.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:45.942+0000 7f4a7f7fe640 1 -- 172.21.15.12:0/3482317988 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4a80050f80 con 0x7f4a881436c0 2023-10-30T12:47:46.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:46.138+0000 7f4a8d032640 1 -- 172.21.15.12:0/3482317988 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a880b0d70 con 0x7f4a881436c0 2023-10-30T12:47:46.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:46.138+0000 7f4a7f7fe640 1 -- 172.21.15.12:0/3482317988 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5738 (secure 0 0 0) 0x7f4a800545a0 con 0x7f4a881436c0 2023-10-30T12:47:46.143 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:46.144 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:45.331687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:46.144 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 64 2023-10-30T12:47:46.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:46.138+0000 7f4a8d032640 1 -- 172.21.15.12:0/3482317988 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4a680664d0 msgr2=0x7f4a68068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:46.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:46.138+0000 7f4a8d032640 1 --2- 172.21.15.12:0/3482317988 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4a680664d0 0x7f4a68068990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4a780af000 tx=0x7f4a780b2040 comp rx=0 tx=0).stop 2023-10-30T12:47:46.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:46.138+0000 7f4a8d032640 1 -- 172.21.15.12:0/3482317988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a881436c0 msgr2=0x7f4a88161250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:46.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:46.138+0000 7f4a8d032640 1 --2- 172.21.15.12:0/3482317988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a881436c0 0x7f4a88161250 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4a880c2580 tx=0x7f4a8000e530 comp rx=0 tx=0).stop 2023-10-30T12:47:46.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:46.138+0000 7f4a8d032640 1 -- 172.21.15.12:0/3482317988 shutdown_connections 2023-10-30T12:47:46.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:46.138+0000 7f4a8d032640 1 --2- 172.21.15.12:0/3482317988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a881436c0 0x7f4a88161250 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:46.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:46.138+0000 7f4a8d032640 1 --2- 172.21.15.12:0/3482317988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a8815d970 0x7f4a88143180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:46.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:46.138+0000 7f4a8d032640 1 --2- 172.21.15.12:0/3482317988 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4a680664d0 0x7f4a68068990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:46.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:46.138+0000 7f4a8d032640 1 --2- 172.21.15.12:0/3482317988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a74003980 0x7f4a880ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:46.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:46.138+0000 7f4a8d032640 1 -- 172.21.15.12:0/3482317988 >> 172.21.15.12:0/3482317988 conn(0x7f4a880c7910 msgr2=0x7f4a88071bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:46.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:46.138+0000 7f4a8d032640 1 -- 172.21.15.12:0/3482317988 shutdown_connections 2023-10-30T12:47:46.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:46.138+0000 7f4a8d032640 1 -- 172.21.15.12:0/3482317988 wait complete. 2023-10-30T12:47:46.165 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 64, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 64, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5124": { "addr": "172.21.15.12:6835/559649789", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 559649789, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 559649789, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5124, "incarnation": 36, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 39 }, "gid_5427": { "addr": "172.21.15.12:6836/2219276371", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 2219276371, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 2219276371, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138322906710015, "flags": 0, "gid": 5427, "incarnation": 57, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 31 }, "gid_5571": { "addr": "172.21.15.12:6837/2752088514", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2752088514, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 2752088514, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5571, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 51, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:47:45.331687+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5571, "mds_1": 5124, "mds_2": 5427 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:47:48.166 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f339e71d640 1 Processor -- start 2023-10-30T12:47:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f339e71d640 1 -- start start 2023-10-30T12:47:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f339e71d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f33981538a0 0x7f339815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f339e71d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f33980cf160 0x7f33980cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f339e71d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33980cb800 con 0x7f33980ca0e0 2023-10-30T12:47:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f339e71d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339815dc30 con 0x7f33980cf160 2023-10-30T12:47:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f339e71d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33980cb330 con 0x7f33981538a0 2023-10-30T12:47:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f339ccba640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f33980cf160 0x7f33980cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f339ccba640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f33980cf160 0x7f33980cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40684/0 (socket says 172.21.15.12:40684) 2023-10-30T12:47:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f339ccba640 1 -- 172.21.15.12:0/389482663 learned_addr learned my addr 172.21.15.12:0/389482663 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f3397fff640 1 --2- 172.21.15.12:0/389482663 >> v2:172.21.15.154:3300/0 conn(0x7f33981538a0 0x7f339815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f339ccba640 1 -- 172.21.15.12:0/389482663 >> v2:172.21.15.154:3300/0 conn(0x7f33981538a0 msgr2=0x7f339815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f339ccba640 1 --2- 172.21.15.12:0/389482663 >> v2:172.21.15.154:3300/0 conn(0x7f33981538a0 0x7f339815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f339ccba640 1 -- 172.21.15.12:0/389482663 >> v1:172.21.15.154:6789/0 conn(0x7f33980ca0e0 legacy=0x7f339815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f339ccba640 1 -- 172.21.15.12:0/389482663 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33980cad70 con 0x7f33980cf160 2023-10-30T12:47:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.306+0000 7f3397fff640 1 --2- 172.21.15.12:0/389482663 >> v2:172.21.15.154:3300/0 conn(0x7f33981538a0 0x7f339815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.310+0000 7f339ccba640 1 --2- 172.21.15.12:0/389482663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33980cf160 0x7f33980cf590 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f33980cfd10 tx=0x7f339001f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adbbf2c365b72c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.310+0000 7f3396ffd640 1 -- 172.21.15.12:0/389482663 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33900af400 con 0x7f33980cf160 2023-10-30T12:47:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.310+0000 7f3396ffd640 1 -- 172.21.15.12:0/389482663 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33900ab050 con 0x7f33980cf160 2023-10-30T12:47:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.310+0000 7f3396ffd640 1 -- 172.21.15.12:0/389482663 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33900af780 con 0x7f33980cf160 2023-10-30T12:47:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.310+0000 7f339e71d640 1 -- 172.21.15.12:0/389482663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33980cf160 msgr2=0x7f33980cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.310+0000 7f339e71d640 1 --2- 172.21.15.12:0/389482663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33980cf160 0x7f33980cf590 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f33980cfd10 tx=0x7f339001f1c0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.310+0000 7f339e71d640 1 -- 172.21.15.12:0/389482663 shutdown_connections 2023-10-30T12:47:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.310+0000 7f339e71d640 1 --2- 172.21.15.12:0/389482663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33980cf160 0x7f33980cf590 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.310+0000 7f339e71d640 1 --2- 172.21.15.12:0/389482663 >> v2:172.21.15.154:3300/0 conn(0x7f33981538a0 0x7f339815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.310+0000 7f339e71d640 1 -- 172.21.15.12:0/389482663 >> 172.21.15.12:0/389482663 conn(0x7f339806bfe0 msgr2=0x7f339806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.310+0000 7f339e71d640 1 -- 172.21.15.12:0/389482663 shutdown_connections 2023-10-30T12:47:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.310+0000 7f339e71d640 1 -- 172.21.15.12:0/389482663 wait complete. 2023-10-30T12:47:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f339e71d640 1 Processor -- start 2023-10-30T12:47:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f339e71d640 1 -- start start 2023-10-30T12:47:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f339e71d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33980ca0e0 0x7f3398143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f339e71d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33980cf160 0x7f33981f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f339e71d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33981538a0 0x7f33980c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f339e71d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3398144510 con 0x7f33980ca0e0 2023-10-30T12:47:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f339e71d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33981f3800 con 0x7f33981538a0 2023-10-30T12:47:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f339e71d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33980b34d0 con 0x7f33980cf160 2023-10-30T12:47:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f3397fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33980ca0e0 0x7f3398143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:48.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f3397fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33980ca0e0 0x7f3398143fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40694/0 (socket says 172.21.15.12:40694) 2023-10-30T12:47:48.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f3397fff640 1 -- 172.21.15.12:0/1270321962 learned_addr learned my addr 172.21.15.12:0/1270321962 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:48.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f33977fe640 1 --2- 172.21.15.12:0/1270321962 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33980cf160 0x7f33981f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:48.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f339ccba640 1 --2- 172.21.15.12:0/1270321962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33981538a0 0x7f33980c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:48.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f3397fff640 1 -- 172.21.15.12:0/1270321962 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33980cf160 msgr2=0x7f33981f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f3397fff640 1 --2- 172.21.15.12:0/1270321962 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33980cf160 0x7f33981f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f3397fff640 1 -- 172.21.15.12:0/1270321962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33981538a0 msgr2=0x7f33980c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f3397fff640 1 --2- 172.21.15.12:0/1270321962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33981538a0 0x7f33980c4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f3397fff640 1 -- 172.21.15.12:0/1270321962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3390009b30 con 0x7f33980ca0e0 2023-10-30T12:47:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f339ccba640 1 --2- 172.21.15.12:0/1270321962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33981538a0 0x7f33980c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:48.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f3397fff640 1 --2- 172.21.15.12:0/1270321962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33980ca0e0 0x7f3398143fd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f338800e810 tx=0x7f338800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:48.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f3394ff9640 1 -- 172.21.15.12:0/1270321962 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338800c870 con 0x7f33980ca0e0 2023-10-30T12:47:48.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f3394ff9640 1 -- 172.21.15.12:0/1270321962 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338800c9d0 con 0x7f33980ca0e0 2023-10-30T12:47:48.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f3394ff9640 1 -- 172.21.15.12:0/1270321962 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338800cd60 con 0x7f33980ca0e0 2023-10-30T12:47:48.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f339e71d640 1 -- 172.21.15.12:0/1270321962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33981ef340 con 0x7f33980ca0e0 2023-10-30T12:47:48.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f339e71d640 1 -- 172.21.15.12:0/1270321962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33981ef8c0 con 0x7f33980ca0e0 2023-10-30T12:47:48.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.314+0000 7f339e71d640 1 -- 172.21.15.12:0/1270321962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33980b2a50 con 0x7f33980ca0e0 2023-10-30T12:47:48.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.318+0000 7f3394ff9640 1 -- 172.21.15.12:0/1270321962 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3388013410 con 0x7f33980ca0e0 2023-10-30T12:47:48.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.318+0000 7f3394ff9640 1 --2- 172.21.15.12:0/1270321962 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f33840664a0 0x7f3384068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:48.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.318+0000 7f3394ff9640 1 -- 172.21.15.12:0/1270321962 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f3388083fa0 con 0x7f33980ca0e0 2023-10-30T12:47:48.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.318+0000 7f33977fe640 1 --2- 172.21.15.12:0/1270321962 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f33840664a0 0x7f3384068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.318+0000 7f33977fe640 1 --2- 172.21.15.12:0/1270321962 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f33840664a0 0x7f3384068960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f339815dc00 tx=0x7f338c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.322+0000 7f3394ff9640 1 -- 172.21.15.12:0/1270321962 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3388051070 con 0x7f33980ca0e0 2023-10-30T12:47:48.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.550+0000 7f339e71d640 1 -- 172.21.15.12:0/1270321962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f339804e170 con 0x7f33980ca0e0 2023-10-30T12:47:48.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.550+0000 7f3394ff9640 1 -- 172.21.15.12:0/1270321962 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5738 (secure 0 0 0) 0x7f3388021090 con 0x7f33980ca0e0 2023-10-30T12:47:48.557 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:48.557 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:45.331687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:48.558 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 64 2023-10-30T12:47:48.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.554+0000 7f339e71d640 1 -- 172.21.15.12:0/1270321962 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f33840664a0 msgr2=0x7f3384068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:48.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.554+0000 7f339e71d640 1 --2- 172.21.15.12:0/1270321962 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f33840664a0 0x7f3384068960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f339815dc00 tx=0x7f338c009040 comp rx=0 tx=0).stop 2023-10-30T12:47:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.554+0000 7f339e71d640 1 -- 172.21.15.12:0/1270321962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33980ca0e0 msgr2=0x7f3398143fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.554+0000 7f339e71d640 1 --2- 172.21.15.12:0/1270321962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33980ca0e0 0x7f3398143fd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f338800e810 tx=0x7f338800ecd0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.554+0000 7f339e71d640 1 -- 172.21.15.12:0/1270321962 shutdown_connections 2023-10-30T12:47:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.554+0000 7f339e71d640 1 --2- 172.21.15.12:0/1270321962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33981538a0 0x7f33980c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.554+0000 7f339e71d640 1 --2- 172.21.15.12:0/1270321962 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f33840664a0 0x7f3384068960 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.554+0000 7f339e71d640 1 --2- 172.21.15.12:0/1270321962 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33980cf160 0x7f33981f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.554+0000 7f339e71d640 1 --2- 172.21.15.12:0/1270321962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33980ca0e0 0x7f3398143fd0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.554+0000 7f339e71d640 1 -- 172.21.15.12:0/1270321962 >> 172.21.15.12:0/1270321962 conn(0x7f339806bfe0 msgr2=0x7f33981513b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.554+0000 7f339e71d640 1 -- 172.21.15.12:0/1270321962 shutdown_connections 2023-10-30T12:47:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:48.554+0000 7f339e71d640 1 -- 172.21.15.12:0/1270321962 wait complete. 2023-10-30T12:47:48.579 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:47:50.579 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:50.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.722+0000 7f3c3d71d640 1 Processor -- start 2023-10-30T12:47:50.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.722+0000 7f3c3d71d640 1 -- start start 2023-10-30T12:47:50.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.726+0000 7f3c3d71d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3c381538a0 0x7f3c3815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:50.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.726+0000 7f3c3d71d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3c380ca0e0 0x7f3c3815a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:50.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.726+0000 7f3c3d71d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c380cb800 con 0x7f3c380cf160 2023-10-30T12:47:50.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.726+0000 7f3c3d71d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c3815dc30 con 0x7f3c380ca0e0 2023-10-30T12:47:50.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.726+0000 7f3c3d71d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c380cb330 con 0x7f3c381538a0 2023-10-30T12:47:50.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.726+0000 7f3c367fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3c380ca0e0 0x7f3c3815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:50.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.726+0000 7f3c377fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3c380cf160 0x7f3c380cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44976/0 (socket says 172.21.15.12:44976) 2023-10-30T12:47:50.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.726+0000 7f3c377fe640 1 -- 172.21.15.12:0/4185895605 learned_addr learned my addr 172.21.15.12:0/4185895605 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:47:50.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.726+0000 7f3c36ffd640 1 --2- 172.21.15.12:0/4185895605 >> v2:172.21.15.154:3300/0 conn(0x7f3c381538a0 0x7f3c3815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:50.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.726+0000 7f3c367fc640 1 -- 172.21.15.12:0/4185895605 >> v2:172.21.15.154:3300/0 conn(0x7f3c381538a0 msgr2=0x7f3c3815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:50.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.726+0000 7f3c367fc640 1 --2- 172.21.15.12:0/4185895605 >> v2:172.21.15.154:3300/0 conn(0x7f3c381538a0 0x7f3c3815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.726+0000 7f3c367fc640 1 -- 172.21.15.12:0/4185895605 >> v1:172.21.15.12:6789/0 conn(0x7f3c380cf160 legacy=0x7f3c380cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:50.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.726+0000 7f3c367fc640 1 -- 172.21.15.12:0/4185895605 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c380cad70 con 0x7f3c380ca0e0 2023-10-30T12:47:50.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.726+0000 7f3c35ffb640 1 -- 172.21.15.12:0/4185895605 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435582650 0 0) 0x7f3c380cb800 con 0x7f3c380cf160 2023-10-30T12:47:50.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.730+0000 7f3c367fc640 1 --2- 172.21.15.12:0/4185895605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c380ca0e0 0x7f3c3815a090 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3c380cfd10 tx=0x7f3c2c01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bae839b8dcc648f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:50.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.730+0000 7f3c35ffb640 1 -- 172.21.15.12:0/4185895605 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c0a8950 con 0x7f3c380ca0e0 2023-10-30T12:47:50.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.730+0000 7f3c35ffb640 1 -- 172.21.15.12:0/4185895605 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c0a8ab0 con 0x7f3c380ca0e0 2023-10-30T12:47:50.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.730+0000 7f3c35ffb640 1 -- 172.21.15.12:0/4185895605 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c0a8e40 con 0x7f3c380ca0e0 2023-10-30T12:47:50.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.730+0000 7f3c3d71d640 1 -- 172.21.15.12:0/4185895605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c380ca0e0 msgr2=0x7f3c3815a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:50.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.730+0000 7f3c3d71d640 1 --2- 172.21.15.12:0/4185895605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c380ca0e0 0x7f3c3815a090 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3c380cfd10 tx=0x7f3c2c01ec50 comp rx=0 tx=0).stop 2023-10-30T12:47:50.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.730+0000 7f3c3d71d640 1 -- 172.21.15.12:0/4185895605 shutdown_connections 2023-10-30T12:47:50.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.730+0000 7f3c3d71d640 1 --2- 172.21.15.12:0/4185895605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c380ca0e0 0x7f3c3815a090 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.730+0000 7f3c3d71d640 1 --2- 172.21.15.12:0/4185895605 >> v2:172.21.15.154:3300/0 conn(0x7f3c381538a0 0x7f3c3815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.730+0000 7f3c3d71d640 1 -- 172.21.15.12:0/4185895605 >> 172.21.15.12:0/4185895605 conn(0x7f3c3806bfe0 msgr2=0x7f3c3806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:50.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c3d71d640 1 -- 172.21.15.12:0/4185895605 shutdown_connections 2023-10-30T12:47:50.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c3d71d640 1 -- 172.21.15.12:0/4185895605 wait complete. 2023-10-30T12:47:50.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c3d71d640 1 Processor -- start 2023-10-30T12:47:50.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c3d71d640 1 -- start start 2023-10-30T12:47:50.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c3d71d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c380ca0e0 0x7f3c380b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:50.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c3d71d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c380cf160 0x7f3c38158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:50.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c3d71d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c381538a0 0x7f3c38158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c3d71d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c380b3b10 con 0x7f3c381538a0 2023-10-30T12:47:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c3d71d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c381e35b0 con 0x7f3c380cf160 2023-10-30T12:47:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c3d71d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c380ce8f0 con 0x7f3c380ca0e0 2023-10-30T12:47:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c377fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c381538a0 0x7f3c38158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c377fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c381538a0 0x7f3c38158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40708/0 (socket says 172.21.15.12:40708) 2023-10-30T12:47:50.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c377fe640 1 -- 172.21.15.12:0/528186405 learned_addr learned my addr 172.21.15.12:0/528186405 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:50.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c367fc640 1 --2- 172.21.15.12:0/528186405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c380cf160 0x7f3c38158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:50.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c36ffd640 1 --2- 172.21.15.12:0/528186405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c380ca0e0 0x7f3c380b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:50.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c377fe640 1 -- 172.21.15.12:0/528186405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c380ca0e0 msgr2=0x7f3c380b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:50.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.734+0000 7f3c377fe640 1 --2- 172.21.15.12:0/528186405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c380ca0e0 0x7f3c380b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.738+0000 7f3c377fe640 1 -- 172.21.15.12:0/528186405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c380cf160 msgr2=0x7f3c38158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:50.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.738+0000 7f3c377fe640 1 --2- 172.21.15.12:0/528186405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c380cf160 0x7f3c38158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.738+0000 7f3c377fe640 1 -- 172.21.15.12:0/528186405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c009420 con 0x7f3c381538a0 2023-10-30T12:47:50.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.738+0000 7f3c377fe640 1 --2- 172.21.15.12:0/528186405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c381538a0 0x7f3c38158870 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3c30001570 tx=0x7f3c3000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:50.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.738+0000 7f3c1f7fe640 1 -- 172.21.15.12:0/528186405 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3000ce40 con 0x7f3c381538a0 2023-10-30T12:47:50.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.738+0000 7f3c1f7fe640 1 -- 172.21.15.12:0/528186405 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3000a5e0 con 0x7f3c381538a0 2023-10-30T12:47:50.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.738+0000 7f3c3d71d640 1 -- 172.21.15.12:0/528186405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c380b4a40 con 0x7f3c381538a0 2023-10-30T12:47:50.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.738+0000 7f3c1f7fe640 1 -- 172.21.15.12:0/528186405 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3000a990 con 0x7f3c381538a0 2023-10-30T12:47:50.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.738+0000 7f3c3d71d640 1 -- 172.21.15.12:0/528186405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c380b4ce0 con 0x7f3c381538a0 2023-10-30T12:47:50.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.738+0000 7f3c3d71d640 1 -- 172.21.15.12:0/528186405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c380b2a50 con 0x7f3c381538a0 2023-10-30T12:47:50.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.738+0000 7f3c1f7fe640 1 -- 172.21.15.12:0/528186405 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3c3000aaf0 con 0x7f3c381538a0 2023-10-30T12:47:50.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.738+0000 7f3c1f7fe640 1 --2- 172.21.15.12:0/528186405 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3c240664a0 0x7f3c24068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:50.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.742+0000 7f3c1f7fe640 1 -- 172.21.15.12:0/528186405 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f3c300837e0 con 0x7f3c381538a0 2023-10-30T12:47:50.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.742+0000 7f3c36ffd640 1 --2- 172.21.15.12:0/528186405 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3c240664a0 0x7f3c24068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:50.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.742+0000 7f3c36ffd640 1 --2- 172.21.15.12:0/528186405 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3c240664a0 0x7f3c24068960 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009ac0 tx=0x7f3c28009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:50.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.742+0000 7f3c1f7fe640 1 -- 172.21.15.12:0/528186405 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3c300509f0 con 0x7f3c381538a0 2023-10-30T12:47:50.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.950+0000 7f3c3d71d640 1 -- 172.21.15.12:0/528186405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c38158050 con 0x7f3c381538a0 2023-10-30T12:47:50.955 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:50.955 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:45.331687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:50.956 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 64 2023-10-30T12:47:50.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.950+0000 7f3c1f7fe640 1 -- 172.21.15.12:0/528186405 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5738 (secure 0 0 0) 0x7f3c38158050 con 0x7f3c381538a0 2023-10-30T12:47:50.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.950+0000 7f3c3d71d640 1 -- 172.21.15.12:0/528186405 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3c240664a0 msgr2=0x7f3c24068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:50.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.950+0000 7f3c3d71d640 1 --2- 172.21.15.12:0/528186405 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3c240664a0 0x7f3c24068960 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009ac0 tx=0x7f3c28009190 comp rx=0 tx=0).stop 2023-10-30T12:47:50.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.950+0000 7f3c3d71d640 1 -- 172.21.15.12:0/528186405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c381538a0 msgr2=0x7f3c38158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:50.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.950+0000 7f3c3d71d640 1 --2- 172.21.15.12:0/528186405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c381538a0 0x7f3c38158870 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3c30001570 tx=0x7f3c3000cc50 comp rx=0 tx=0).stop 2023-10-30T12:47:50.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.950+0000 7f3c3d71d640 1 -- 172.21.15.12:0/528186405 shutdown_connections 2023-10-30T12:47:50.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.950+0000 7f3c3d71d640 1 --2- 172.21.15.12:0/528186405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c381538a0 0x7f3c38158870 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.950+0000 7f3c3d71d640 1 --2- 172.21.15.12:0/528186405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c380cf160 0x7f3c38158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.950+0000 7f3c3d71d640 1 --2- 172.21.15.12:0/528186405 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3c240664a0 0x7f3c24068960 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.950+0000 7f3c3d71d640 1 --2- 172.21.15.12:0/528186405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c380ca0e0 0x7f3c380b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.950+0000 7f3c3d71d640 1 -- 172.21.15.12:0/528186405 >> 172.21.15.12:0/528186405 conn(0x7f3c3806bfe0 msgr2=0x7f3c381e5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:50.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.950+0000 7f3c3d71d640 1 -- 172.21.15.12:0/528186405 shutdown_connections 2023-10-30T12:47:50.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:50.950+0000 7f3c3d71d640 1 -- 172.21.15.12:0/528186405 wait complete. 2023-10-30T12:47:50.992 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:47:52.993 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:53.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.134+0000 7fe72b691640 1 Processor -- start 2023-10-30T12:47:53.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.134+0000 7fe72b691640 1 -- start start 2023-10-30T12:47:53.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.134+0000 7fe72b691640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe72406f7a0 0x7fe72406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:53.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.134+0000 7fe72b691640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe724066cf0 con 0x7fe7240c6bd0 2023-10-30T12:47:53.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.134+0000 7fe72b691640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7240c70d0 con 0x7fe724071110 2023-10-30T12:47:53.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.134+0000 7fe72b691640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7240700c0 con 0x7fe72406f7a0 2023-10-30T12:47:53.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.134+0000 7fe728c2c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe7240c6bd0 0x7fe724069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42320/0 (socket says 172.21.15.12:42320) 2023-10-30T12:47:53.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.134+0000 7fe728c2c640 1 -- 172.21.15.12:0/2111524337 learned_addr learned my addr 172.21.15.12:0/2111524337 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:47:53.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.134+0000 7fe72942d640 1 --2- 172.21.15.12:0/2111524337 >> v2:172.21.15.154:3300/0 conn(0x7fe72406f7a0 0x7fe72406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:53.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.134+0000 7fe71bfff640 1 -- 172.21.15.12:0/2111524337 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251886408 0 0) 0x7fe724066cf0 con 0x7fe7240c6bd0 2023-10-30T12:47:53.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.134+0000 7fe71bfff640 1 -- 172.21.15.12:0/2111524337 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe710003530 con 0x7fe7240c6bd0 2023-10-30T12:47:53.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.138+0000 7fe71bfff640 1 -- 172.21.15.12:0/2111524337 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2236025251 0 0) 0x7fe710003530 con 0x7fe7240c6bd0 2023-10-30T12:47:53.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.138+0000 7fe72942d640 1 -- 172.21.15.12:0/2111524337 >> v1:172.21.15.154:6789/0 conn(0x7fe724071110 legacy=0x7fe724158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:53.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.138+0000 7fe72942d640 1 -- 172.21.15.12:0/2111524337 >> v1:172.21.15.12:6789/0 conn(0x7fe7240c6bd0 legacy=0x7fe724069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:53.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.138+0000 7fe72942d640 1 -- 172.21.15.12:0/2111524337 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7240c7390 con 0x7fe72406f7a0 2023-10-30T12:47:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.138+0000 7fe72942d640 1 --2- 172.21.15.12:0/2111524337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72406f7a0 0x7fe72406fb80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe7200097c0 tx=0x7fe72001eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64773b320dc361c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:53.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.138+0000 7fe71bfff640 1 -- 172.21.15.12:0/2111524337 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714001680 con 0x7fe72406f7a0 2023-10-30T12:47:53.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.138+0000 7fe71bfff640 1 -- 172.21.15.12:0/2111524337 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7200a8610 con 0x7fe72406f7a0 2023-10-30T12:47:53.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.138+0000 7fe71bfff640 1 -- 172.21.15.12:0/2111524337 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7200a8980 con 0x7fe72406f7a0 2023-10-30T12:47:53.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72b691640 1 -- 172.21.15.12:0/2111524337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72406f7a0 msgr2=0x7fe72406fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:53.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72b691640 1 --2- 172.21.15.12:0/2111524337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72406f7a0 0x7fe72406fb80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe7200097c0 tx=0x7fe72001eb90 comp rx=0 tx=0).stop 2023-10-30T12:47:53.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72b691640 1 -- 172.21.15.12:0/2111524337 shutdown_connections 2023-10-30T12:47:53.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72b691640 1 --2- 172.21.15.12:0/2111524337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72406f7a0 0x7fe72406fb80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:53.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72b691640 1 -- 172.21.15.12:0/2111524337 >> 172.21.15.12:0/2111524337 conn(0x7fe7240c7910 msgr2=0x7fe7240c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:53.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72b691640 1 -- 172.21.15.12:0/2111524337 shutdown_connections 2023-10-30T12:47:53.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72b691640 1 -- 172.21.15.12:0/2111524337 wait complete. 2023-10-30T12:47:53.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72b691640 1 Processor -- start 2023-10-30T12:47:53.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72b691640 1 -- start start 2023-10-30T12:47:53.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72b691640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72406f7a0 0x7fe7240aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:53.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72b691640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe724071110 0x7fe7241ef420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:53.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72b691640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7240c6bd0 0x7fe7241ef960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:53.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72b691640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe724156cc0 con 0x7fe724071110 2023-10-30T12:47:53.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72b691640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe724156fa0 con 0x7fe72406f7a0 2023-10-30T12:47:53.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72b691640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe724157280 con 0x7fe7240c6bd0 2023-10-30T12:47:53.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe728c2c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe724071110 0x7fe7241ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:53.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe728c2c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe724071110 0x7fe7241ef420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36912/0 (socket says 172.21.15.12:36912) 2023-10-30T12:47:53.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe728c2c640 1 -- 172.21.15.12:0/3634190162 learned_addr learned my addr 172.21.15.12:0/3634190162 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:53.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe72942d640 1 --2- 172.21.15.12:0/3634190162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72406f7a0 0x7fe7240aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:53.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.142+0000 7fe729c2e640 1 --2- 172.21.15.12:0/3634190162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7240c6bd0 0x7fe7241ef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:53.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.146+0000 7fe728c2c640 1 -- 172.21.15.12:0/3634190162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7240c6bd0 msgr2=0x7fe7241ef960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.146+0000 7fe728c2c640 1 --2- 172.21.15.12:0/3634190162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7240c6bd0 0x7fe7241ef960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.146+0000 7fe728c2c640 1 -- 172.21.15.12:0/3634190162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72406f7a0 msgr2=0x7fe7240aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.146+0000 7fe728c2c640 1 --2- 172.21.15.12:0/3634190162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72406f7a0 0x7fe7240aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.146+0000 7fe728c2c640 1 -- 172.21.15.12:0/3634190162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe720009420 con 0x7fe724071110 2023-10-30T12:47:53.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.146+0000 7fe728c2c640 1 --2- 172.21.15.12:0/3634190162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe724071110 0x7fe7241ef420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe71400c730 tx=0x7fe71400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:53.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.146+0000 7fe719ffb640 1 -- 172.21.15.12:0/3634190162 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714005ae0 con 0x7fe724071110 2023-10-30T12:47:53.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.146+0000 7fe719ffb640 1 -- 172.21.15.12:0/3634190162 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe714005c40 con 0x7fe724071110 2023-10-30T12:47:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.146+0000 7fe719ffb640 1 -- 172.21.15.12:0/3634190162 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714007070 con 0x7fe724071110 2023-10-30T12:47:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.146+0000 7fe72b691640 1 -- 172.21.15.12:0/3634190162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe724157560 con 0x7fe724071110 2023-10-30T12:47:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.146+0000 7fe72b691640 1 -- 172.21.15.12:0/3634190162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7241583a0 con 0x7fe724071110 2023-10-30T12:47:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.146+0000 7fe72b691640 1 -- 172.21.15.12:0/3634190162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe72404e170 con 0x7fe724071110 2023-10-30T12:47:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.146+0000 7fe719ffb640 1 -- 172.21.15.12:0/3634190162 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe7140073e0 con 0x7fe724071110 2023-10-30T12:47:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.146+0000 7fe719ffb640 1 --2- 172.21.15.12:0/3634190162 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe7080664d0 0x7fe708068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.146+0000 7fe719ffb640 1 -- 172.21.15.12:0/3634190162 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7fe714083f80 con 0x7fe724071110 2023-10-30T12:47:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.150+0000 7fe72942d640 1 --2- 172.21.15.12:0/3634190162 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe7080664d0 0x7fe708068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.150+0000 7fe72942d640 1 --2- 172.21.15.12:0/3634190162 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe7080664d0 0x7fe708068990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe7200b2000 tx=0x7fe7200aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:53.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.154+0000 7fe719ffb640 1 -- 172.21.15.12:0/3634190162 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe714051190 con 0x7fe724071110 2023-10-30T12:47:53.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.374+0000 7fe72b691640 1 -- 172.21.15.12:0/3634190162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe724151590 con 0x7fe724071110 2023-10-30T12:47:53.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.374+0000 7fe719ffb640 1 -- 172.21.15.12:0/3634190162 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5738 (secure 0 0 0) 0x7fe7140547b0 con 0x7fe724071110 2023-10-30T12:47:53.379 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:53.379 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:45.331687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:53.380 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 64 2023-10-30T12:47:53.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.374+0000 7fe72b691640 1 -- 172.21.15.12:0/3634190162 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe7080664d0 msgr2=0x7fe708068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:53.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.374+0000 7fe72b691640 1 --2- 172.21.15.12:0/3634190162 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe7080664d0 0x7fe708068990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe7200b2000 tx=0x7fe7200aa070 comp rx=0 tx=0).stop 2023-10-30T12:47:53.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.374+0000 7fe72b691640 1 -- 172.21.15.12:0/3634190162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe724071110 msgr2=0x7fe7241ef420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:53.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.374+0000 7fe72b691640 1 --2- 172.21.15.12:0/3634190162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe724071110 0x7fe7241ef420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe71400c730 tx=0x7fe71400cbf0 comp rx=0 tx=0).stop 2023-10-30T12:47:53.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.374+0000 7fe72b691640 1 -- 172.21.15.12:0/3634190162 shutdown_connections 2023-10-30T12:47:53.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.374+0000 7fe72b691640 1 --2- 172.21.15.12:0/3634190162 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe7080664d0 0x7fe708068990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:53.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.374+0000 7fe72b691640 1 --2- 172.21.15.12:0/3634190162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7240c6bd0 0x7fe7241ef960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:53.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.374+0000 7fe72b691640 1 --2- 172.21.15.12:0/3634190162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe724071110 0x7fe7241ef420 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:53.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.374+0000 7fe72b691640 1 --2- 172.21.15.12:0/3634190162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72406f7a0 0x7fe7240aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:53.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.374+0000 7fe72b691640 1 -- 172.21.15.12:0/3634190162 >> 172.21.15.12:0/3634190162 conn(0x7fe7240c7910 msgr2=0x7fe7240c93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:53.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.374+0000 7fe72b691640 1 -- 172.21.15.12:0/3634190162 shutdown_connections 2023-10-30T12:47:53.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:53.374+0000 7fe72b691640 1 -- 172.21.15.12:0/3634190162 wait complete. 2023-10-30T12:47:53.408 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:47:55.409 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:55.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.566+0000 7ff5d9394640 1 Processor -- start 2023-10-30T12:47:55.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.566+0000 7ff5d9394640 1 -- start start 2023-10-30T12:47:55.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.566+0000 7ff5d9394640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff5d415d6f0 0x7ff5d415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:55.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.566+0000 7ff5d9394640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff5d40cf160 0x7ff5d40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:55.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.566+0000 7ff5d9394640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d40cb800 con 0x7ff5d41538a0 2023-10-30T12:47:55.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.566+0000 7ff5d9394640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d40cb330 con 0x7ff5d415d6f0 2023-10-30T12:47:55.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.566+0000 7ff5d9394640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d40cad70 con 0x7ff5d40cf160 2023-10-30T12:47:55.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.566+0000 7ff5d27fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff5d415d6f0 0x7ff5d415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:55.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.566+0000 7ff5d2ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff5d41538a0 0x7ff5d40ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42324/0 (socket says 172.21.15.12:42324) 2023-10-30T12:47:55.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.566+0000 7ff5d27fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff5d415d6f0 0x7ff5d415a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36926/0 (socket says 172.21.15.12:36926) 2023-10-30T12:47:55.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.570+0000 7ff5d2ffd640 1 -- 172.21.15.12:0/1816850899 learned_addr learned my addr 172.21.15.12:0/1816850899 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:47:55.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.570+0000 7ff5d37fe640 1 --2- 172.21.15.12:0/1816850899 >> v2:172.21.15.154:3300/0 conn(0x7ff5d40cf160 0x7ff5d40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:55.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.570+0000 7ff5d27fc640 1 -- 172.21.15.12:0/1816850899 >> v2:172.21.15.154:3300/0 conn(0x7ff5d40cf160 msgr2=0x7ff5d40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:55.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.570+0000 7ff5d27fc640 1 --2- 172.21.15.12:0/1816850899 >> v2:172.21.15.154:3300/0 conn(0x7ff5d40cf160 0x7ff5d40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:55.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.570+0000 7ff5d27fc640 1 -- 172.21.15.12:0/1816850899 >> v1:172.21.15.12:6789/0 conn(0x7ff5d41538a0 legacy=0x7ff5d40ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:55.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.570+0000 7ff5d27fc640 1 -- 172.21.15.12:0/1816850899 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d40cb050 con 0x7ff5d415d6f0 2023-10-30T12:47:55.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.574+0000 7ff5d27fc640 1 --2- 172.21.15.12:0/1816850899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d415d6f0 0x7ff5d415a090 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40b5330 tx=0x7ff5c801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7032a5e059b2216 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:55.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.574+0000 7ff5d1ffb640 1 -- 172.21.15.12:0/1816850899 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80a8950 con 0x7ff5d415d6f0 2023-10-30T12:47:55.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.574+0000 7ff5d1ffb640 1 -- 172.21.15.12:0/1816850899 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c80a8ab0 con 0x7ff5d415d6f0 2023-10-30T12:47:55.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.574+0000 7ff5d1ffb640 1 -- 172.21.15.12:0/1816850899 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80a8e40 con 0x7ff5d415d6f0 2023-10-30T12:47:55.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.574+0000 7ff5d9394640 1 -- 172.21.15.12:0/1816850899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d415d6f0 msgr2=0x7ff5d415a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:55.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.574+0000 7ff5d9394640 1 --2- 172.21.15.12:0/1816850899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d415d6f0 0x7ff5d415a090 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40b5330 tx=0x7ff5c801ec50 comp rx=0 tx=0).stop 2023-10-30T12:47:55.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.574+0000 7ff5d9394640 1 -- 172.21.15.12:0/1816850899 shutdown_connections 2023-10-30T12:47:55.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.574+0000 7ff5d9394640 1 --2- 172.21.15.12:0/1816850899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d415d6f0 0x7ff5d415a090 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:55.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.574+0000 7ff5d9394640 1 --2- 172.21.15.12:0/1816850899 >> v2:172.21.15.154:3300/0 conn(0x7ff5d40cf160 0x7ff5d40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:55.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.574+0000 7ff5d9394640 1 -- 172.21.15.12:0/1816850899 >> 172.21.15.12:0/1816850899 conn(0x7ff5d406bfe0 msgr2=0x7ff5d406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:55.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.574+0000 7ff5d9394640 1 -- 172.21.15.12:0/1816850899 shutdown_connections 2023-10-30T12:47:55.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.574+0000 7ff5d9394640 1 -- 172.21.15.12:0/1816850899 wait complete. 2023-10-30T12:47:55.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.574+0000 7ff5d9394640 1 Processor -- start 2023-10-30T12:47:55.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.574+0000 7ff5d9394640 1 -- start start 2023-10-30T12:47:55.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d9394640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d40cf160 0x7ff5d4158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:55.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d9394640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d41538a0 0x7ff5d40b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:55.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d9394640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d415d6f0 0x7ff5d40c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:55.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d9394640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d4158d60 con 0x7ff5d41538a0 2023-10-30T12:47:55.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d9394640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d41e5280 con 0x7ff5d40cf160 2023-10-30T12:47:55.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d9394640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d40c9650 con 0x7ff5d415d6f0 2023-10-30T12:47:55.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d41538a0 0x7ff5d40b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:55.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d41538a0 0x7ff5d40b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36930/0 (socket says 172.21.15.12:36930) 2023-10-30T12:47:55.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d27fc640 1 -- 172.21.15.12:0/3500384434 learned_addr learned my addr 172.21.15.12:0/3500384434 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:55.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d2ffd640 1 --2- 172.21.15.12:0/3500384434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d40cf160 0x7ff5d4158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:55.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d37fe640 1 --2- 172.21.15.12:0/3500384434 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d415d6f0 0x7ff5d40c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:55.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d27fc640 1 -- 172.21.15.12:0/3500384434 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d415d6f0 msgr2=0x7ff5d40c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:55.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d27fc640 1 --2- 172.21.15.12:0/3500384434 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d415d6f0 0x7ff5d40c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:55.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d27fc640 1 -- 172.21.15.12:0/3500384434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d40cf160 msgr2=0x7ff5d4158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:55.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d27fc640 1 --2- 172.21.15.12:0/3500384434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d40cf160 0x7ff5d4158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:55.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d27fc640 1 -- 172.21.15.12:0/3500384434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c8009420 con 0x7ff5d41538a0 2023-10-30T12:47:55.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d27fc640 1 --2- 172.21.15.12:0/3500384434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d41538a0 0x7ff5d40b4960 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80b3040 tx=0x7ff5c801b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:55.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5bb7fe640 1 -- 172.21.15.12:0/3500384434 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80b9630 con 0x7ff5d41538a0 2023-10-30T12:47:55.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5bb7fe640 1 -- 172.21.15.12:0/3500384434 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c80b97b0 con 0x7ff5d41538a0 2023-10-30T12:47:55.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d9394640 1 -- 172.21.15.12:0/3500384434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d40c6ec0 con 0x7ff5d41538a0 2023-10-30T12:47:55.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5bb7fe640 1 -- 172.21.15.12:0/3500384434 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80b9b60 con 0x7ff5d41538a0 2023-10-30T12:47:55.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d9394640 1 -- 172.21.15.12:0/3500384434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d4159050 con 0x7ff5d41538a0 2023-10-30T12:47:55.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.578+0000 7ff5d9394640 1 -- 172.21.15.12:0/3500384434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d40b2a50 con 0x7ff5d41538a0 2023-10-30T12:47:55.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.582+0000 7ff5bb7fe640 1 -- 172.21.15.12:0/3500384434 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff5c80b7040 con 0x7ff5d41538a0 2023-10-30T12:47:55.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.582+0000 7ff5bb7fe640 1 --2- 172.21.15.12:0/3500384434 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5c00664a0 0x7ff5c0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:55.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.582+0000 7ff5bb7fe640 1 -- 172.21.15.12:0/3500384434 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7ff5c8121f60 con 0x7ff5d41538a0 2023-10-30T12:47:55.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.582+0000 7ff5d2ffd640 1 --2- 172.21.15.12:0/3500384434 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5c00664a0 0x7ff5c0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:55.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.582+0000 7ff5d2ffd640 1 --2- 172.21.15.12:0/3500384434 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5c00664a0 0x7ff5c0068960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4005890 tx=0x7ff5c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:55.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.590+0000 7ff5bb7fe640 1 -- 172.21.15.12:0/3500384434 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff5c80ef180 con 0x7ff5d41538a0 2023-10-30T12:47:55.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.810+0000 7ff5d9394640 1 -- 172.21.15.12:0/3500384434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5d404e170 con 0x7ff5d41538a0 2023-10-30T12:47:55.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.810+0000 7ff5bb7fe640 1 -- 172.21.15.12:0/3500384434 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5740 (secure 0 0 0) 0x7ff5c80f27a0 con 0x7ff5d41538a0 2023-10-30T12:47:55.817 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:55.817 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:53.545932+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:resolve","state_seq":13,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:55.817 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 65 2023-10-30T12:47:55.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.814+0000 7ff5d9394640 1 -- 172.21.15.12:0/3500384434 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5c00664a0 msgr2=0x7ff5c0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:55.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.814+0000 7ff5d9394640 1 --2- 172.21.15.12:0/3500384434 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5c00664a0 0x7ff5c0068960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4005890 tx=0x7ff5c4009040 comp rx=0 tx=0).stop 2023-10-30T12:47:55.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.814+0000 7ff5d9394640 1 -- 172.21.15.12:0/3500384434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d41538a0 msgr2=0x7ff5d40b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:55.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.814+0000 7ff5d9394640 1 --2- 172.21.15.12:0/3500384434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d41538a0 0x7ff5d40b4960 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80b3040 tx=0x7ff5c801b9f0 comp rx=0 tx=0).stop 2023-10-30T12:47:55.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.814+0000 7ff5d9394640 1 -- 172.21.15.12:0/3500384434 shutdown_connections 2023-10-30T12:47:55.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.814+0000 7ff5d9394640 1 --2- 172.21.15.12:0/3500384434 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5c00664a0 0x7ff5c0068960 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:55.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.814+0000 7ff5d9394640 1 --2- 172.21.15.12:0/3500384434 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d415d6f0 0x7ff5d40c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:55.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.814+0000 7ff5d9394640 1 --2- 172.21.15.12:0/3500384434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d41538a0 0x7ff5d40b4960 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:55.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.814+0000 7ff5d9394640 1 --2- 172.21.15.12:0/3500384434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d40cf160 0x7ff5d4158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:55.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.814+0000 7ff5d9394640 1 -- 172.21.15.12:0/3500384434 >> 172.21.15.12:0/3500384434 conn(0x7ff5d406bfe0 msgr2=0x7ff5d41488e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:55.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.814+0000 7ff5d9394640 1 -- 172.21.15.12:0/3500384434 shutdown_connections 2023-10-30T12:47:55.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:55.814+0000 7ff5d9394640 1 -- 172.21.15.12:0/3500384434 wait complete. 2023-10-30T12:47:55.838 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:47:57.839 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.990+0000 7fa561372640 1 Processor -- start 2023-10-30T12:47:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.990+0000 7fa561372640 1 -- start start 2023-10-30T12:47:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.994+0000 7fa561372640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa55c0c8cf0 0x7fa55c0c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.994+0000 7fa561372640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa55c0ca340 0x7fa55c15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.994+0000 7fa561372640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55c15ce30 con 0x7fa55c0c90d0 2023-10-30T12:47:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.994+0000 7fa561372640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55c159880 con 0x7fa55c0ca340 2023-10-30T12:47:58.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.994+0000 7fa561372640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55c159ae0 con 0x7fa55c0c8cf0 2023-10-30T12:47:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.994+0000 7fa55b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa55c0ca340 0x7fa55c15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.994+0000 7fa55b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa55c0ca340 0x7fa55c15d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36938/0 (socket says 172.21.15.12:36938) 2023-10-30T12:47:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.994+0000 7fa55b7fe640 1 -- 172.21.15.12:0/770828040 learned_addr learned my addr 172.21.15.12:0/770828040 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.994+0000 7fa55affd640 1 --2- 172.21.15.12:0/770828040 >> v2:172.21.15.154:3300/0 conn(0x7fa55c0c8cf0 0x7fa55c0c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.994+0000 7fa55b7fe640 1 -- 172.21.15.12:0/770828040 >> v2:172.21.15.154:3300/0 conn(0x7fa55c0c8cf0 msgr2=0x7fa55c0c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.994+0000 7fa55b7fe640 1 --2- 172.21.15.12:0/770828040 >> v2:172.21.15.154:3300/0 conn(0x7fa55c0c8cf0 0x7fa55c0c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.994+0000 7fa55b7fe640 1 -- 172.21.15.12:0/770828040 >> v1:172.21.15.154:6789/0 conn(0x7fa55c0c90d0 legacy=0x7fa55c0cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.994+0000 7fa55b7fe640 1 -- 172.21.15.12:0/770828040 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c159dc0 con 0x7fa55c0ca340 2023-10-30T12:47:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.998+0000 7fa55b7fe640 1 --2- 172.21.15.12:0/770828040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55c0ca340 0x7fa55c15d2c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa55c08ca20 tx=0x7fa55401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3420d441de466b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.998+0000 7fa559ffb640 1 -- 172.21.15.12:0/770828040 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5540a5860 con 0x7fa55c0ca340 2023-10-30T12:47:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.998+0000 7fa559ffb640 1 -- 172.21.15.12:0/770828040 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5540a59c0 con 0x7fa55c0ca340 2023-10-30T12:47:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.998+0000 7fa559ffb640 1 -- 172.21.15.12:0/770828040 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5540a5d50 con 0x7fa55c0ca340 2023-10-30T12:47:58.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.998+0000 7fa561372640 1 -- 172.21.15.12:0/770828040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55c0ca340 msgr2=0x7fa55c15d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:58.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.998+0000 7fa561372640 1 --2- 172.21.15.12:0/770828040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55c0ca340 0x7fa55c15d2c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa55c08ca20 tx=0x7fa55401c9a0 comp rx=0 tx=0).stop 2023-10-30T12:47:58.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.998+0000 7fa561372640 1 -- 172.21.15.12:0/770828040 shutdown_connections 2023-10-30T12:47:58.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.998+0000 7fa561372640 1 --2- 172.21.15.12:0/770828040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55c0ca340 0x7fa55c15d2c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:58.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.998+0000 7fa561372640 1 --2- 172.21.15.12:0/770828040 >> v2:172.21.15.154:3300/0 conn(0x7fa55c0c8cf0 0x7fa55c0c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:58.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.998+0000 7fa561372640 1 -- 172.21.15.12:0/770828040 >> 172.21.15.12:0/770828040 conn(0x7fa55c06be40 msgr2=0x7fa55c06c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.998+0000 7fa561372640 1 -- 172.21.15.12:0/770828040 shutdown_connections 2023-10-30T12:47:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.998+0000 7fa561372640 1 -- 172.21.15.12:0/770828040 wait complete. 2023-10-30T12:47:58.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.998+0000 7fa561372640 1 Processor -- start 2023-10-30T12:47:58.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:57.998+0000 7fa561372640 1 -- start start 2023-10-30T12:47:58.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa561372640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55c0c8cf0 0x7fa55c155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:58.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa561372640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa55c0c90d0 0x7fa55c0c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:58.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa561372640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa55c0ca340 0x7fa55c1541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:58.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa561372640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c1f1c40 con 0x7fa55c0c8cf0 2023-10-30T12:47:58.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa561372640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c0c6860 con 0x7fa55c0c90d0 2023-10-30T12:47:58.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa561372640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c0c6a90 con 0x7fa55c0ca340 2023-10-30T12:47:58.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa55affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55c0c8cf0 0x7fa55c155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:58.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa55affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55c0c8cf0 0x7fa55c155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36944/0 (socket says 172.21.15.12:36944) 2023-10-30T12:47:58.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa55affd640 1 -- 172.21.15.12:0/1644028953 learned_addr learned my addr 172.21.15.12:0/1644028953 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:47:58.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa55a7fc640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa55c0c90d0 0x7fa55c0c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:58.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa55b7fe640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa55c0ca340 0x7fa55c1541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:58.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa55affd640 1 -- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa55c0ca340 msgr2=0x7fa55c1541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:58.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa55affd640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa55c0ca340 0x7fa55c1541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:58.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa55affd640 1 -- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa55c0c90d0 msgr2=0x7fa55c0c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:58.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa55affd640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa55c0c90d0 0x7fa55c0c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:58.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa55affd640 1 -- 172.21.15.12:0/1644028953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa554009b30 con 0x7fa55c0c8cf0 2023-10-30T12:47:58.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa55b7fe640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa55c0ca340 0x7fa55c1541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:47:58.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa55a7fc640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa55c0c90d0 0x7fa55c0c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:58.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa55affd640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55c0c8cf0 0x7fa55c155650 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00d5f0 tx=0x7fa54c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:58.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa5437fe640 1 -- 172.21.15.12:0/1644028953 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c0125f0 con 0x7fa55c0c8cf0 2023-10-30T12:47:58.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa5437fe640 1 -- 172.21.15.12:0/1644028953 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54c012750 con 0x7fa55c0c8cf0 2023-10-30T12:47:58.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa5437fe640 1 -- 172.21.15.12:0/1644028953 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c012ae0 con 0x7fa55c0c8cf0 2023-10-30T12:47:58.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa561372640 1 -- 172.21.15.12:0/1644028953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55c0c6d50 con 0x7fa55c0c8cf0 2023-10-30T12:47:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa561372640 1 -- 172.21.15.12:0/1644028953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c0c7310 con 0x7fa55c0c8cf0 2023-10-30T12:47:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.002+0000 7fa561372640 1 -- 172.21.15.12:0/1644028953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55c1586e0 con 0x7fa55c0c8cf0 2023-10-30T12:47:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.006+0000 7fa5437fe640 1 -- 172.21.15.12:0/1644028953 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa54c012c40 con 0x7fa55c0c8cf0 2023-10-30T12:47:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.006+0000 7fa5437fe640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa5480664a0 0x7fa548068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.006+0000 7fa5437fe640 1 -- 172.21.15.12:0/1644028953 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7fa54c0846a0 con 0x7fa55c0c8cf0 2023-10-30T12:47:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.006+0000 7fa55a7fc640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa5480664a0 0x7fa548068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.006+0000 7fa55a7fc640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa5480664a0 0x7fa548068960 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0cc720 tx=0x7fa550009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.010+0000 7fa5437fe640 1 -- 172.21.15.12:0/1644028953 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa54c051800 con 0x7fa55c0c8cf0 2023-10-30T12:47:58.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.230+0000 7fa561372640 1 -- 172.21.15.12:0/1644028953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa55c04e170 con 0x7fa55c0c8cf0 2023-10-30T12:47:58.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.230+0000 7fa5437fe640 1 -- 172.21.15.12:0/1644028953 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5740 (secure 0 0 0) 0x7fa54c054e20 con 0x7fa55c0c8cf0 2023-10-30T12:47:58.235 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:47:58.236 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:53.545932+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:resolve","state_seq":13,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:58.236 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 65 2023-10-30T12:47:58.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.230+0000 7fa561372640 1 -- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa5480664a0 msgr2=0x7fa548068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:58.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.230+0000 7fa561372640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa5480664a0 0x7fa548068960 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0cc720 tx=0x7fa550009040 comp rx=0 tx=0).stop 2023-10-30T12:47:58.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.230+0000 7fa561372640 1 -- 172.21.15.12:0/1644028953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55c0c8cf0 msgr2=0x7fa55c155650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:58.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.230+0000 7fa561372640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55c0c8cf0 0x7fa55c155650 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00d5f0 tx=0x7fa54c00dab0 comp rx=0 tx=0).stop 2023-10-30T12:47:58.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.230+0000 7fa561372640 1 -- 172.21.15.12:0/1644028953 shutdown_connections 2023-10-30T12:47:58.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.230+0000 7fa561372640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa5480664a0 0x7fa548068960 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:58.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.230+0000 7fa561372640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa55c0ca340 0x7fa55c1541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:58.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.230+0000 7fa561372640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa55c0c90d0 0x7fa55c0c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:58.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.230+0000 7fa561372640 1 --2- 172.21.15.12:0/1644028953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55c0c8cf0 0x7fa55c155650 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:58.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.230+0000 7fa561372640 1 -- 172.21.15.12:0/1644028953 >> 172.21.15.12:0/1644028953 conn(0x7fa55c06be40 msgr2=0x7fa55c158e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:58.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.230+0000 7fa561372640 1 -- 172.21.15.12:0/1644028953 shutdown_connections 2023-10-30T12:47:58.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:47:58.230+0000 7fa561372640 1 -- 172.21.15.12:0/1644028953 wait complete. 2023-10-30T12:47:58.256 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 65, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 65, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5124": { "addr": "172.21.15.12:6835/559649789", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 559649789, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 559649789, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5124, "incarnation": 36, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 39 }, "gid_5427": { "addr": "172.21.15.12:6836/2219276371", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 2219276371, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 2219276371, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138322906710015, "flags": 0, "gid": 5427, "incarnation": 57, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 31 }, "gid_5571": { "addr": "172.21.15.12:6837/2752088514", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2752088514, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 2752088514, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5571, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:resolve", "state_seq": 13 } }, "last_failure": 0, "last_failure_osd_epoch": 51, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:47:53.545932+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5571, "mds_1": 5124, "mds_2": 5427 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:48:00.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:48:00.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.414+0000 7fc45269a640 1 Processor -- start 2023-10-30T12:48:00.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.414+0000 7fc45269a640 1 -- start start 2023-10-30T12:48:00.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.414+0000 7fc45269a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc44c071110 0x7fc44c06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:00.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.414+0000 7fc45269a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c066cf0 con 0x7fc44c06f7a0 2023-10-30T12:48:00.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.414+0000 7fc45269a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c0c70d0 con 0x7fc44c0c6bd0 2023-10-30T12:48:00.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.414+0000 7fc45269a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44c0700c0 con 0x7fc44c071110 2023-10-30T12:48:00.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.414+0000 7fc450c37640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc44c071110 0x7fc44c06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:00.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.414+0000 7fc450c37640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc44c071110 0x7fc44c06fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36946/0 (socket says 172.21.15.12:36946) 2023-10-30T12:48:00.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.414+0000 7fc450c37640 1 -- 172.21.15.12:0/1040894266 learned_addr learned my addr 172.21.15.12:0/1040894266 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:48:00.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.414+0000 7fc44bfff640 1 --1- 172.21.15.12:0/1040894266 >> v1:172.21.15.12:6789/0 conn(0x7fc44c06f7a0 0x7fc44c069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42330/0 (socket says 172.21.15.12:42330) 2023-10-30T12:48:00.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.414+0000 7fc450c37640 1 -- 172.21.15.12:0/1040894266 >> v1:172.21.15.154:6789/0 conn(0x7fc44c0c6bd0 legacy=0x7fc44c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:00.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.414+0000 7fc450c37640 1 -- 172.21.15.12:0/1040894266 >> v1:172.21.15.12:6789/0 conn(0x7fc44c06f7a0 legacy=0x7fc44c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:00.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.414+0000 7fc450c37640 1 -- 172.21.15.12:0/1040894266 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c0c7390 con 0x7fc44c071110 2023-10-30T12:48:00.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.414+0000 7fc44affd640 1 -- 172.21.15.12:0/1040894266 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030339403 0 0) 0x7fc44c066cf0 con 0x7fc44c06f7a0 2023-10-30T12:48:00.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.418+0000 7fc450c37640 1 --2- 172.21.15.12:0/1040894266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc44c071110 0x7fc44c06fb80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc444009e50 tx=0x7fc44401f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885749e6940ea786 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:00.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.418+0000 7fc44affd640 1 -- 172.21.15.12:0/1040894266 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4440af400 con 0x7fc44c071110 2023-10-30T12:48:00.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.418+0000 7fc44affd640 1 -- 172.21.15.12:0/1040894266 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4440ab050 con 0x7fc44c071110 2023-10-30T12:48:00.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.418+0000 7fc44affd640 1 -- 172.21.15.12:0/1040894266 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4440af780 con 0x7fc44c071110 2023-10-30T12:48:00.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.418+0000 7fc45269a640 1 -- 172.21.15.12:0/1040894266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc44c071110 msgr2=0x7fc44c06fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:00.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.418+0000 7fc45269a640 1 --2- 172.21.15.12:0/1040894266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc44c071110 0x7fc44c06fb80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc444009e50 tx=0x7fc44401f1f0 comp rx=0 tx=0).stop 2023-10-30T12:48:00.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.418+0000 7fc45269a640 1 -- 172.21.15.12:0/1040894266 shutdown_connections 2023-10-30T12:48:00.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.418+0000 7fc45269a640 1 --2- 172.21.15.12:0/1040894266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc44c071110 0x7fc44c06fb80 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:00.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.418+0000 7fc45269a640 1 -- 172.21.15.12:0/1040894266 >> 172.21.15.12:0/1040894266 conn(0x7fc44c0c7910 msgr2=0x7fc44c0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:00.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.418+0000 7fc45269a640 1 -- 172.21.15.12:0/1040894266 shutdown_connections 2023-10-30T12:48:00.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.418+0000 7fc45269a640 1 -- 172.21.15.12:0/1040894266 wait complete. 2023-10-30T12:48:00.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc45269a640 1 Processor -- start 2023-10-30T12:48:00.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc45269a640 1 -- start start 2023-10-30T12:48:00.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc45269a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c06f7a0 0x7fc44c0ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:00.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc45269a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc44c071110 0x7fc44c1e2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:00.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc45269a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc44c0c6bd0 0x7fc44c1468a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:00.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc45269a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1e3c20 con 0x7fc44c0c6bd0 2023-10-30T12:48:00.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc45269a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0c1aa0 con 0x7fc44c071110 2023-10-30T12:48:00.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc45269a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c157f50 con 0x7fc44c06f7a0 2023-10-30T12:48:00.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc450c37640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc44c0c6bd0 0x7fc44c1468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:00.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc450c37640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc44c0c6bd0 0x7fc44c1468a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36958/0 (socket says 172.21.15.12:36958) 2023-10-30T12:48:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc450c37640 1 -- 172.21.15.12:0/3646987057 learned_addr learned my addr 172.21.15.12:0/3646987057 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:48:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc44bfff640 1 --2- 172.21.15.12:0/3646987057 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c06f7a0 0x7fc44c0ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc44b7fe640 1 --2- 172.21.15.12:0/3646987057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc44c071110 0x7fc44c1e2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc450c37640 1 -- 172.21.15.12:0/3646987057 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c06f7a0 msgr2=0x7fc44c0ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc450c37640 1 --2- 172.21.15.12:0/3646987057 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c06f7a0 0x7fc44c0ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc450c37640 1 -- 172.21.15.12:0/3646987057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc44c071110 msgr2=0x7fc44c1e2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc450c37640 1 --2- 172.21.15.12:0/3646987057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc44c071110 0x7fc44c1e2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc450c37640 1 -- 172.21.15.12:0/3646987057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc444009b30 con 0x7fc44c0c6bd0 2023-10-30T12:48:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc450c37640 1 --2- 172.21.15.12:0/3646987057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc44c0c6bd0 0x7fc44c1468a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc4440b2040 tx=0x7fc44401bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:00.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc448ff9640 1 -- 172.21.15.12:0/3646987057 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4440afde0 con 0x7fc44c0c6bd0 2023-10-30T12:48:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc448ff9640 1 -- 172.21.15.12:0/3646987057 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc444009510 con 0x7fc44c0c6bd0 2023-10-30T12:48:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc448ff9640 1 -- 172.21.15.12:0/3646987057 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4440ba660 con 0x7fc44c0c6bd0 2023-10-30T12:48:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc45269a640 1 -- 172.21.15.12:0/3646987057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c0c1cf0 con 0x7fc44c0c6bd0 2023-10-30T12:48:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.422+0000 7fc45269a640 1 -- 172.21.15.12:0/3646987057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c1ef9e0 con 0x7fc44c0c6bd0 2023-10-30T12:48:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.426+0000 7fc45269a640 1 -- 172.21.15.12:0/3646987057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44c1ef560 con 0x7fc44c0c6bd0 2023-10-30T12:48:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.426+0000 7fc448ff9640 1 -- 172.21.15.12:0/3646987057 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc4440b6040 con 0x7fc44c0c6bd0 2023-10-30T12:48:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.426+0000 7fc448ff9640 1 --2- 172.21.15.12:0/3646987057 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4380664f0 0x7fc4380689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.426+0000 7fc448ff9640 1 -- 172.21.15.12:0/3646987057 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7fc444122510 con 0x7fc44c0c6bd0 2023-10-30T12:48:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.426+0000 7fc44bfff640 1 --2- 172.21.15.12:0/3646987057 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4380664f0 0x7fc4380689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.426+0000 7fc44bfff640 1 --2- 172.21.15.12:0/3646987057 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4380664f0 0x7fc4380689b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc43c000f70 tx=0x7fc43c007d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:00.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.430+0000 7fc448ff9640 1 -- 172.21.15.12:0/3646987057 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc44c1ef560 con 0x7fc44c0c6bd0 2023-10-30T12:48:00.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.622+0000 7fc45269a640 1 -- 172.21.15.12:0/3646987057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc44c1ef560 con 0x7fc44c0c6bd0 2023-10-30T12:48:00.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.622+0000 7fc448ff9640 1 -- 172.21.15.12:0/3646987057 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+5742 (secure 0 0 0) 0x7fc44c1ef560 con 0x7fc44c0c6bd0 2023-10-30T12:48:00.628 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:48:00.628 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:58.620065+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:48:00.629 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 66 2023-10-30T12:48:00.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.622+0000 7fc45269a640 1 -- 172.21.15.12:0/3646987057 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4380664f0 msgr2=0x7fc4380689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:00.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.622+0000 7fc45269a640 1 --2- 172.21.15.12:0/3646987057 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4380664f0 0x7fc4380689b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc43c000f70 tx=0x7fc43c007d10 comp rx=0 tx=0).stop 2023-10-30T12:48:00.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.622+0000 7fc45269a640 1 -- 172.21.15.12:0/3646987057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc44c0c6bd0 msgr2=0x7fc44c1468a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:00.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.622+0000 7fc45269a640 1 --2- 172.21.15.12:0/3646987057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc44c0c6bd0 0x7fc44c1468a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc4440b2040 tx=0x7fc44401bf90 comp rx=0 tx=0).stop 2023-10-30T12:48:00.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.622+0000 7fc45269a640 1 -- 172.21.15.12:0/3646987057 shutdown_connections 2023-10-30T12:48:00.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.622+0000 7fc45269a640 1 --2- 172.21.15.12:0/3646987057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc44c0c6bd0 0x7fc44c1468a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:00.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.622+0000 7fc45269a640 1 --2- 172.21.15.12:0/3646987057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc44c071110 0x7fc44c1e2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:00.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.622+0000 7fc45269a640 1 --2- 172.21.15.12:0/3646987057 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4380664f0 0x7fc4380689b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:00.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.622+0000 7fc45269a640 1 --2- 172.21.15.12:0/3646987057 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c06f7a0 0x7fc44c0ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:00.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.622+0000 7fc45269a640 1 -- 172.21.15.12:0/3646987057 >> 172.21.15.12:0/3646987057 conn(0x7fc44c0c7910 msgr2=0x7fc44c0c9260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:00.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.622+0000 7fc45269a640 1 -- 172.21.15.12:0/3646987057 shutdown_connections 2023-10-30T12:48:00.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:00.622+0000 7fc45269a640 1 -- 172.21.15.12:0/3646987057 wait complete. 2023-10-30T12:48:00.649 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:48:02.650 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:48:02.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.798+0000 7f4b9c2d6640 1 Processor -- start 2023-10-30T12:48:02.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.798+0000 7f4b9c2d6640 1 -- start start 2023-10-30T12:48:02.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b9c2d6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4b9415d6f0 0x7f4b9415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:02.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b9c2d6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b940cf160 0x7f4b940cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:02.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b9c2d6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b940cb800 con 0x7f4b941538a0 2023-10-30T12:48:02.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b9c2d6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b940cb330 con 0x7f4b9415d6f0 2023-10-30T12:48:02.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b9c2d6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b940cad70 con 0x7f4b940cf160 2023-10-30T12:48:02.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b99871640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4b9415d6f0 0x7f4b9415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:02.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b99871640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4b9415d6f0 0x7f4b9415a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49554/0 (socket says 172.21.15.12:49554) 2023-10-30T12:48:02.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b99871640 1 -- 172.21.15.12:0/2407436404 learned_addr learned my addr 172.21.15.12:0/2407436404 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:48:02.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b9a072640 1 --1- 172.21.15.12:0/2407436404 >> v1:172.21.15.12:6789/0 conn(0x7f4b941538a0 0x7f4b940ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:53132/0 (socket says 172.21.15.12:53132) 2023-10-30T12:48:02.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b9a873640 1 --2- 172.21.15.12:0/2407436404 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b940cf160 0x7f4b940cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:02.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b99871640 1 -- 172.21.15.12:0/2407436404 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b940cf160 msgr2=0x7f4b940cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:48:02.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b99871640 1 --2- 172.21.15.12:0/2407436404 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b940cf160 0x7f4b940cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:02.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b99871640 1 -- 172.21.15.12:0/2407436404 >> v1:172.21.15.12:6789/0 conn(0x7f4b941538a0 legacy=0x7f4b940ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:02.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b99871640 1 -- 172.21.15.12:0/2407436404 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b940cb050 con 0x7f4b9415d6f0 2023-10-30T12:48:02.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b99070640 1 -- 172.21.15.12:0/2407436404 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557185801 0 0) 0x7f4b940cb800 con 0x7f4b941538a0 2023-10-30T12:48:02.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.802+0000 7f4b99871640 1 --2- 172.21.15.12:0/2407436404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b9415d6f0 0x7f4b9415a090 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009870 tx=0x7f4b8401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5e1cb707e8be41 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:02.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b99070640 1 -- 172.21.15.12:0/2407436404 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b840a8a30 con 0x7f4b9415d6f0 2023-10-30T12:48:02.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b99070640 1 -- 172.21.15.12:0/2407436404 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b840a8b90 con 0x7f4b9415d6f0 2023-10-30T12:48:02.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b99070640 1 -- 172.21.15.12:0/2407436404 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b840b0440 con 0x7f4b9415d6f0 2023-10-30T12:48:02.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 -- 172.21.15.12:0/2407436404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b9415d6f0 msgr2=0x7f4b9415a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:02.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 --2- 172.21.15.12:0/2407436404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b9415d6f0 0x7f4b9415a090 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009870 tx=0x7f4b8401ed30 comp rx=0 tx=0).stop 2023-10-30T12:48:02.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 -- 172.21.15.12:0/2407436404 shutdown_connections 2023-10-30T12:48:02.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 --2- 172.21.15.12:0/2407436404 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b940cf160 0x7f4b940cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:02.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 --2- 172.21.15.12:0/2407436404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b9415d6f0 0x7f4b9415a090 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:02.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 -- 172.21.15.12:0/2407436404 >> 172.21.15.12:0/2407436404 conn(0x7f4b9406bfe0 msgr2=0x7f4b9406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:02.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 -- 172.21.15.12:0/2407436404 shutdown_connections 2023-10-30T12:48:02.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 -- 172.21.15.12:0/2407436404 wait complete. 2023-10-30T12:48:02.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 Processor -- start 2023-10-30T12:48:02.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 -- start start 2023-10-30T12:48:02.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b940cf160 0x7f4b940b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:02.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b941538a0 0x7f4b9415bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:02.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b9415d6f0 0x7f4b9414c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:02.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b940b3b20 con 0x7f4b941538a0 2023-10-30T12:48:02.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b940b40e0 con 0x7f4b940cf160 2023-10-30T12:48:02.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.806+0000 7f4b9c2d6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b940b4ab0 con 0x7f4b9415d6f0 2023-10-30T12:48:02.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b99871640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b941538a0 0x7f4b9415bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:02.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b99871640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b941538a0 0x7f4b9415bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49570/0 (socket says 172.21.15.12:49570) 2023-10-30T12:48:02.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b99871640 1 -- 172.21.15.12:0/279167876 learned_addr learned my addr 172.21.15.12:0/279167876 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:48:02.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b9a072640 1 --2- 172.21.15.12:0/279167876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b940cf160 0x7f4b940b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:02.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b9a873640 1 --2- 172.21.15.12:0/279167876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b9415d6f0 0x7f4b9414c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b99871640 1 -- 172.21.15.12:0/279167876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b9415d6f0 msgr2=0x7f4b9414c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b99871640 1 --2- 172.21.15.12:0/279167876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b9415d6f0 0x7f4b9414c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b99871640 1 -- 172.21.15.12:0/279167876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b940cf160 msgr2=0x7f4b940b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b99871640 1 --2- 172.21.15.12:0/279167876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b940cf160 0x7f4b940b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b99871640 1 -- 172.21.15.12:0/279167876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840094d0 con 0x7f4b941538a0 2023-10-30T12:48:02.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b99871640 1 --2- 172.21.15.12:0/279167876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b941538a0 0x7f4b9415bd70 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4b8401c960 tx=0x7f4b8401d570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:02.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b8affd640 1 -- 172.21.15.12:0/279167876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b840a8a30 con 0x7f4b941538a0 2023-10-30T12:48:02.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b8affd640 1 -- 172.21.15.12:0/279167876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b84004020 con 0x7f4b941538a0 2023-10-30T12:48:02.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b8affd640 1 -- 172.21.15.12:0/279167876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b840043b0 con 0x7f4b941538a0 2023-10-30T12:48:02.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b9c2d6640 1 -- 172.21.15.12:0/279167876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9414f750 con 0x7f4b941538a0 2023-10-30T12:48:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b9c2d6640 1 -- 172.21.15.12:0/279167876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b940b3d70 con 0x7f4b941538a0 2023-10-30T12:48:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b9c2d6640 1 -- 172.21.15.12:0/279167876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b940b3f30 con 0x7f4b941538a0 2023-10-30T12:48:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b8affd640 1 -- 172.21.15.12:0/279167876 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4b840b7040 con 0x7f4b941538a0 2023-10-30T12:48:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b8affd640 1 --2- 172.21.15.12:0/279167876 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4b780664a0 0x7f4b78068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.810+0000 7f4b8affd640 1 -- 172.21.15.12:0/279167876 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f4b841215d0 con 0x7f4b941538a0 2023-10-30T12:48:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.814+0000 7f4b9a072640 1 --2- 172.21.15.12:0/279167876 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4b780664a0 0x7f4b78068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:02.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.814+0000 7f4b9a072640 1 --2- 172.21.15.12:0/279167876 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4b780664a0 0x7f4b78068960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4b90000f70 tx=0x7f4b90007b60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:02.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:02.814+0000 7f4b8affd640 1 -- 172.21.15.12:0/279167876 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4b840ee7e0 con 0x7f4b941538a0 2023-10-30T12:48:03.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:03.006+0000 7f4b9c2d6640 1 -- 172.21.15.12:0/279167876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b940b2a50 con 0x7f4b941538a0 2023-10-30T12:48:03.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:03.006+0000 7f4b8affd640 1 -- 172.21.15.12:0/279167876 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+5742 (secure 0 0 0) 0x7f4b840f1e00 con 0x7f4b941538a0 2023-10-30T12:48:03.011 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:48:03.011 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:47:58.620065+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:48:03.012 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 66 2023-10-30T12:48:03.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:03.006+0000 7f4b9c2d6640 1 -- 172.21.15.12:0/279167876 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4b780664a0 msgr2=0x7f4b78068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:03.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:03.006+0000 7f4b9c2d6640 1 --2- 172.21.15.12:0/279167876 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4b780664a0 0x7f4b78068960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4b90000f70 tx=0x7f4b90007b60 comp rx=0 tx=0).stop 2023-10-30T12:48:03.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:03.006+0000 7f4b9c2d6640 1 -- 172.21.15.12:0/279167876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b941538a0 msgr2=0x7f4b9415bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:03.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:03.006+0000 7f4b9c2d6640 1 --2- 172.21.15.12:0/279167876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b941538a0 0x7f4b9415bd70 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4b8401c960 tx=0x7f4b8401d570 comp rx=0 tx=0).stop 2023-10-30T12:48:03.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:03.006+0000 7f4b9c2d6640 1 -- 172.21.15.12:0/279167876 shutdown_connections 2023-10-30T12:48:03.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:03.006+0000 7f4b9c2d6640 1 --2- 172.21.15.12:0/279167876 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4b780664a0 0x7f4b78068960 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:03.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:03.006+0000 7f4b9c2d6640 1 --2- 172.21.15.12:0/279167876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b9415d6f0 0x7f4b9414c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:03.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:03.006+0000 7f4b9c2d6640 1 --2- 172.21.15.12:0/279167876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b941538a0 0x7f4b9415bd70 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:03.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:03.006+0000 7f4b9c2d6640 1 --2- 172.21.15.12:0/279167876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b940cf160 0x7f4b940b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:03.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:03.006+0000 7f4b9c2d6640 1 -- 172.21.15.12:0/279167876 >> 172.21.15.12:0/279167876 conn(0x7f4b9406bfe0 msgr2=0x7f4b941e2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:03.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:03.006+0000 7f4b9c2d6640 1 -- 172.21.15.12:0/279167876 shutdown_connections 2023-10-30T12:48:03.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:03.006+0000 7f4b9c2d6640 1 -- 172.21.15.12:0/279167876 wait complete. 2023-10-30T12:48:03.032 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:48:05.033 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:48:05.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.182+0000 7f19db9b8640 1 Processor -- start 2023-10-30T12:48:05.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.182+0000 7f19db9b8640 1 -- start start 2023-10-30T12:48:05.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.182+0000 7f19db9b8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19d41580e0 0x7f19d4070740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:05.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.182+0000 7f19db9b8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f19d40c6540 0x7f19d415e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:05.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.182+0000 7f19db9b8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d40c5810 con 0x7f19d4069e90 2023-10-30T12:48:05.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.182+0000 7f19db9b8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d40c6990 con 0x7f19d41580e0 2023-10-30T12:48:05.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.182+0000 7f19db9b8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d40c6bb0 con 0x7f19d40c6540 2023-10-30T12:48:05.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.182+0000 7f19d9754640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19d41580e0 0x7f19d4070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:05.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.182+0000 7f19d9754640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19d41580e0 0x7f19d4070740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49580/0 (socket says 172.21.15.12:49580) 2023-10-30T12:48:05.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.182+0000 7f19d9754640 1 -- 172.21.15.12:0/1614084429 learned_addr learned my addr 172.21.15.12:0/1614084429 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:48:05.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.182+0000 7f19d8f53640 1 --2- 172.21.15.12:0/1614084429 >> v2:172.21.15.154:3300/0 conn(0x7f19d40c6540 0x7f19d415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:05.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.186+0000 7f19d9754640 1 -- 172.21.15.12:0/1614084429 >> v2:172.21.15.154:3300/0 conn(0x7f19d40c6540 msgr2=0x7f19d415e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:48:05.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.186+0000 7f19d9754640 1 --2- 172.21.15.12:0/1614084429 >> v2:172.21.15.154:3300/0 conn(0x7f19d40c6540 0x7f19d415e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:05.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.186+0000 7f19d9754640 1 -- 172.21.15.12:0/1614084429 >> v1:172.21.15.154:6789/0 conn(0x7f19d4069e90 legacy=0x7f19d406a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:05.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.186+0000 7f19d9754640 1 -- 172.21.15.12:0/1614084429 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d40c6e70 con 0x7f19d41580e0 2023-10-30T12:48:05.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.186+0000 7f19d9754640 1 --2- 172.21.15.12:0/1614084429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d41580e0 0x7f19d4070740 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f19d406b1a0 tx=0x7f19d001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b960bbec1648d59 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:05.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.186+0000 7f19cbfff640 1 -- 172.21.15.12:0/1614084429 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d00aa660 con 0x7f19d41580e0 2023-10-30T12:48:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.186+0000 7f19cbfff640 1 -- 172.21.15.12:0/1614084429 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d00aa7c0 con 0x7f19d41580e0 2023-10-30T12:48:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.186+0000 7f19cbfff640 1 -- 172.21.15.12:0/1614084429 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d00aab50 con 0x7f19d41580e0 2023-10-30T12:48:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.186+0000 7f19db9b8640 1 -- 172.21.15.12:0/1614084429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d41580e0 msgr2=0x7f19d4070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.186+0000 7f19db9b8640 1 --2- 172.21.15.12:0/1614084429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d41580e0 0x7f19d4070740 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f19d406b1a0 tx=0x7f19d001c500 comp rx=0 tx=0).stop 2023-10-30T12:48:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.186+0000 7f19db9b8640 1 -- 172.21.15.12:0/1614084429 shutdown_connections 2023-10-30T12:48:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.186+0000 7f19db9b8640 1 --2- 172.21.15.12:0/1614084429 >> v2:172.21.15.154:3300/0 conn(0x7f19d40c6540 0x7f19d415e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.186+0000 7f19db9b8640 1 --2- 172.21.15.12:0/1614084429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d41580e0 0x7f19d4070740 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.186+0000 7f19db9b8640 1 -- 172.21.15.12:0/1614084429 >> 172.21.15.12:0/1614084429 conn(0x7f19d40c7910 msgr2=0x7f19d40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19db9b8640 1 -- 172.21.15.12:0/1614084429 shutdown_connections 2023-10-30T12:48:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19db9b8640 1 -- 172.21.15.12:0/1614084429 wait complete. 2023-10-30T12:48:05.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19db9b8640 1 Processor -- start 2023-10-30T12:48:05.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19db9b8640 1 -- start start 2023-10-30T12:48:05.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19db9b8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19d4069e90 0x7f19d414d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:05.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19db9b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d40c6540 0x7f19d414dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:05.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19db9b8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19d41580e0 0x7f19d414a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:05.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19db9b8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d414b9a0 con 0x7f19d40c6540 2023-10-30T12:48:05.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19db9b8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40c1ce0 con 0x7f19d41580e0 2023-10-30T12:48:05.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19db9b8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d414bbd0 con 0x7f19d4069e90 2023-10-30T12:48:05.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19d8f53640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d40c6540 0x7f19d414dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:05.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19d8f53640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d40c6540 0x7f19d414dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49590/0 (socket says 172.21.15.12:49590) 2023-10-30T12:48:05.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19d8f53640 1 -- 172.21.15.12:0/1054198225 learned_addr learned my addr 172.21.15.12:0/1054198225 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:48:05.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19d9f55640 1 --2- 172.21.15.12:0/1054198225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19d41580e0 0x7f19d414a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:05.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19d9754640 1 --2- 172.21.15.12:0/1054198225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19d4069e90 0x7f19d414d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:05.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19d8f53640 1 -- 172.21.15.12:0/1054198225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19d4069e90 msgr2=0x7f19d414d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:05.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19d8f53640 1 --2- 172.21.15.12:0/1054198225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19d4069e90 0x7f19d414d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:05.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19d8f53640 1 -- 172.21.15.12:0/1054198225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19d41580e0 msgr2=0x7f19d414a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:05.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19d8f53640 1 --2- 172.21.15.12:0/1054198225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19d41580e0 0x7f19d414a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:05.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19d8f53640 1 -- 172.21.15.12:0/1054198225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d00094d0 con 0x7f19d40c6540 2023-10-30T12:48:05.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19d9754640 1 --2- 172.21.15.12:0/1054198225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19d4069e90 0x7f19d414d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:48:05.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.190+0000 7f19d8f53640 1 --2- 172.21.15.12:0/1054198225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d40c6540 0x7f19d414dc60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f19c400d800 tx=0x7f19c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:05.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.194+0000 7f19c9ffb640 1 -- 172.21.15.12:0/1054198225 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c400b6c0 con 0x7f19d40c6540 2023-10-30T12:48:05.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.194+0000 7f19c9ffb640 1 -- 172.21.15.12:0/1054198225 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c400b820 con 0x7f19d40c6540 2023-10-30T12:48:05.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.194+0000 7f19c9ffb640 1 -- 172.21.15.12:0/1054198225 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c400bb90 con 0x7f19d40c6540 2023-10-30T12:48:05.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.194+0000 7f19db9b8640 1 -- 172.21.15.12:0/1054198225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d414be90 con 0x7f19d40c6540 2023-10-30T12:48:05.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.194+0000 7f19db9b8640 1 -- 172.21.15.12:0/1054198225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d40c13b0 con 0x7f19d40c6540 2023-10-30T12:48:05.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.194+0000 7f19db9b8640 1 -- 172.21.15.12:0/1054198225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d404e170 con 0x7f19d40c6540 2023-10-30T12:48:05.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.194+0000 7f19c9ffb640 1 -- 172.21.15.12:0/1054198225 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f19c400bcf0 con 0x7f19d40c6540 2023-10-30T12:48:05.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.194+0000 7f19c9ffb640 1 --2- 172.21.15.12:0/1054198225 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19b80664a0 0x7f19b8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:05.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.194+0000 7f19c9ffb640 1 -- 172.21.15.12:0/1054198225 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f19c4084620 con 0x7f19d40c6540 2023-10-30T12:48:05.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.194+0000 7f19d9754640 1 --2- 172.21.15.12:0/1054198225 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19b80664a0 0x7f19b8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:05.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.198+0000 7f19d9754640 1 --2- 172.21.15.12:0/1054198225 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19b80664a0 0x7f19b8068960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f19d00ae000 tx=0x7f19d00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:05.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.198+0000 7f19c9ffb640 1 -- 172.21.15.12:0/1054198225 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f19c4051780 con 0x7f19d40c6540 2023-10-30T12:48:05.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.394+0000 7f19db9b8640 1 -- 172.21.15.12:0/1054198225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19d40c5290 con 0x7f19d40c6540 2023-10-30T12:48:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.394+0000 7f19c9ffb640 1 -- 172.21.15.12:0/1054198225 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5739 (secure 0 0 0) 0x7f19c4054da0 con 0x7f19d40c6540 2023-10-30T12:48:05.400 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:48:05.401 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:48:04.589766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:48:05.401 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 67 2023-10-30T12:48:05.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.398+0000 7f19db9b8640 1 -- 172.21.15.12:0/1054198225 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19b80664a0 msgr2=0x7f19b8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:05.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.398+0000 7f19db9b8640 1 --2- 172.21.15.12:0/1054198225 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19b80664a0 0x7f19b8068960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f19d00ae000 tx=0x7f19d00a7000 comp rx=0 tx=0).stop 2023-10-30T12:48:05.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.398+0000 7f19db9b8640 1 -- 172.21.15.12:0/1054198225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d40c6540 msgr2=0x7f19d414dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:05.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.398+0000 7f19db9b8640 1 --2- 172.21.15.12:0/1054198225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d40c6540 0x7f19d414dc60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f19c400d800 tx=0x7f19c400dcc0 comp rx=0 tx=0).stop 2023-10-30T12:48:05.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.398+0000 7f19db9b8640 1 -- 172.21.15.12:0/1054198225 shutdown_connections 2023-10-30T12:48:05.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.398+0000 7f19db9b8640 1 --2- 172.21.15.12:0/1054198225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19d41580e0 0x7f19d414a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:05.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.398+0000 7f19db9b8640 1 --2- 172.21.15.12:0/1054198225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d40c6540 0x7f19d414dc60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:05.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.398+0000 7f19db9b8640 1 --2- 172.21.15.12:0/1054198225 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19b80664a0 0x7f19b8068960 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:05.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.398+0000 7f19db9b8640 1 --2- 172.21.15.12:0/1054198225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19d4069e90 0x7f19d414d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:05.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.398+0000 7f19db9b8640 1 -- 172.21.15.12:0/1054198225 >> 172.21.15.12:0/1054198225 conn(0x7f19d40c7910 msgr2=0x7f19d4154e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:05.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.398+0000 7f19db9b8640 1 -- 172.21.15.12:0/1054198225 shutdown_connections 2023-10-30T12:48:05.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:05.398+0000 7f19db9b8640 1 -- 172.21.15.12:0/1054198225 wait complete. 2023-10-30T12:48:05.421 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:48:05.862 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:48:05.866 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:48:07.422 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:48:07.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.570+0000 7fb646580640 1 Processor -- start 2023-10-30T12:48:07.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.570+0000 7fb646580640 1 -- start start 2023-10-30T12:48:07.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb646580640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6400c6b80 0x7fb6400bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb646580640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb6400c6f60 0x7fb6400c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb646580640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb6400c8430 0x7fb64013ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb646580640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6400c0a50 con 0x7fb6400c8430 2023-10-30T12:48:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb646580640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64006f6e0 con 0x7fb6400c6f60 2023-10-30T12:48:07.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb646580640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64006f9c0 con 0x7fb6400c6b80 2023-10-30T12:48:07.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb644b1d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb6400c8430 0x7fb64013ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:07.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb644b1d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb6400c8430 0x7fb64013ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49598/0 (socket says 172.21.15.12:49598) 2023-10-30T12:48:07.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb644b1d640 1 -- 172.21.15.12:0/3685023719 learned_addr learned my addr 172.21.15.12:0/3685023719 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:48:07.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb63ffff640 1 --2- 172.21.15.12:0/3685023719 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6400c6b80 0x7fb6400bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:07.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb63f7fe640 1 --2- 172.21.15.12:0/3685023719 >> v2:172.21.15.154:3300/0 conn(0x7fb6400c6f60 0x7fb6400c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:07.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb644b1d640 1 -- 172.21.15.12:0/3685023719 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6400c6b80 msgr2=0x7fb6400bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:48:07.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb644b1d640 1 --2- 172.21.15.12:0/3685023719 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6400c6b80 0x7fb6400bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:07.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb644b1d640 1 -- 172.21.15.12:0/3685023719 >> v2:172.21.15.154:3300/0 conn(0x7fb6400c6f60 msgr2=0x7fb6400c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:48:07.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb644b1d640 1 --2- 172.21.15.12:0/3685023719 >> v2:172.21.15.154:3300/0 conn(0x7fb6400c6f60 0x7fb6400c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:07.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.574+0000 7fb644b1d640 1 -- 172.21.15.12:0/3685023719 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64006fca0 con 0x7fb6400c8430 2023-10-30T12:48:07.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.578+0000 7fb644b1d640 1 --2- 172.21.15.12:0/3685023719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6400c8430 0x7fb64013ed90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb64006c830 tx=0x7fb63801ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85ad9c5c7b7161f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:07.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.578+0000 7fb63effd640 1 -- 172.21.15.12:0/3685023719 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63800b2d0 con 0x7fb6400c8430 2023-10-30T12:48:07.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.578+0000 7fb63effd640 1 -- 172.21.15.12:0/3685023719 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63800b430 con 0x7fb6400c8430 2023-10-30T12:48:07.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.578+0000 7fb63effd640 1 -- 172.21.15.12:0/3685023719 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6380a7b90 con 0x7fb6400c8430 2023-10-30T12:48:07.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.578+0000 7fb646580640 1 -- 172.21.15.12:0/3685023719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6400c8430 msgr2=0x7fb64013ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:07.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.578+0000 7fb646580640 1 --2- 172.21.15.12:0/3685023719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6400c8430 0x7fb64013ed90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb64006c830 tx=0x7fb63801ebd0 comp rx=0 tx=0).stop 2023-10-30T12:48:07.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.578+0000 7fb646580640 1 -- 172.21.15.12:0/3685023719 shutdown_connections 2023-10-30T12:48:07.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.578+0000 7fb646580640 1 --2- 172.21.15.12:0/3685023719 >> v2:172.21.15.154:3300/0 conn(0x7fb6400c6f60 0x7fb6400c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:07.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.578+0000 7fb646580640 1 --2- 172.21.15.12:0/3685023719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6400c8430 0x7fb64013ed90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:07.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.578+0000 7fb646580640 1 --2- 172.21.15.12:0/3685023719 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6400c6b80 0x7fb6400bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:07.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.578+0000 7fb646580640 1 -- 172.21.15.12:0/3685023719 >> 172.21.15.12:0/3685023719 conn(0x7fb6400c7910 msgr2=0x7fb6400c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:07.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.578+0000 7fb646580640 1 -- 172.21.15.12:0/3685023719 shutdown_connections 2023-10-30T12:48:07.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.578+0000 7fb646580640 1 -- 172.21.15.12:0/3685023719 wait complete. 2023-10-30T12:48:07.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.578+0000 7fb646580640 1 Processor -- start 2023-10-30T12:48:07.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.578+0000 7fb646580640 1 -- start start 2023-10-30T12:48:07.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb646580640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6400c6b80 0x7fb640150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:07.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb646580640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6400c6f60 0x7fb64012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:07.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb646580640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6400c8430 0x7fb640130260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:07.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb646580640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400b0a40 con 0x7fb6400c6b80 2023-10-30T12:48:07.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb646580640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400c18c0 con 0x7fb6400c8430 2023-10-30T12:48:07.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb646580640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400c1ba0 con 0x7fb6400c6f60 2023-10-30T12:48:07.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb63ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6400c6b80 0x7fb640150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:07.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb63ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6400c6b80 0x7fb640150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49614/0 (socket says 172.21.15.12:49614) 2023-10-30T12:48:07.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb63ffff640 1 -- 172.21.15.12:0/2581484371 learned_addr learned my addr 172.21.15.12:0/2581484371 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:48:07.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb63f7fe640 1 --2- 172.21.15.12:0/2581484371 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6400c6f60 0x7fb64012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:07.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb644b1d640 1 --2- 172.21.15.12:0/2581484371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6400c8430 0x7fb640130260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:07.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb63ffff640 1 -- 172.21.15.12:0/2581484371 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6400c6f60 msgr2=0x7fb64012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:07.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb63ffff640 1 --2- 172.21.15.12:0/2581484371 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6400c6f60 0x7fb64012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:07.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb63ffff640 1 -- 172.21.15.12:0/2581484371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6400c8430 msgr2=0x7fb640130260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:07.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb63ffff640 1 --2- 172.21.15.12:0/2581484371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6400c8430 0x7fb640130260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:07.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb63ffff640 1 -- 172.21.15.12:0/2581484371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63800bd40 con 0x7fb6400c6b80 2023-10-30T12:48:07.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb644b1d640 1 --2- 172.21.15.12:0/2581484371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6400c8430 0x7fb640130260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:48:07.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb63ffff640 1 --2- 172.21.15.12:0/2581484371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6400c6b80 0x7fb640150540 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb63000e8e0 tx=0x7fb63000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:07.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb63cff9640 1 -- 172.21.15.12:0/2581484371 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63000ca00 con 0x7fb6400c6b80 2023-10-30T12:48:07.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb63cff9640 1 -- 172.21.15.12:0/2581484371 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63000cb60 con 0x7fb6400c6b80 2023-10-30T12:48:07.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb646580640 1 -- 172.21.15.12:0/2581484371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64014d690 con 0x7fb6400c6b80 2023-10-30T12:48:07.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb63cff9640 1 -- 172.21.15.12:0/2581484371 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630013470 con 0x7fb6400c6b80 2023-10-30T12:48:07.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb646580640 1 -- 172.21.15.12:0/2581484371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb640133000 con 0x7fb6400c6b80 2023-10-30T12:48:07.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb646580640 1 -- 172.21.15.12:0/2581484371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb64004e170 con 0x7fb6400c6b80 2023-10-30T12:48:07.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.582+0000 7fb63cff9640 1 -- 172.21.15.12:0/2581484371 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb6300135d0 con 0x7fb6400c6b80 2023-10-30T12:48:07.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.586+0000 7fb63cff9640 1 --2- 172.21.15.12:0/2581484371 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb62c0664d0 0x7fb62c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:07.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.586+0000 7fb63cff9640 1 -- 172.21.15.12:0/2581484371 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7fb6300842a0 con 0x7fb6400c6b80 2023-10-30T12:48:07.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.586+0000 7fb63f7fe640 1 --2- 172.21.15.12:0/2581484371 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb62c0664d0 0x7fb62c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:07.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.586+0000 7fb63f7fe640 1 --2- 172.21.15.12:0/2581484371 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb62c0664d0 0x7fb62c068990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb6400c8320 tx=0x7fb634009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:07.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.590+0000 7fb63cff9640 1 -- 172.21.15.12:0/2581484371 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb630051540 con 0x7fb6400c6b80 2023-10-30T12:48:07.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.782+0000 7fb646580640 1 -- 172.21.15.12:0/2581484371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6400c8070 con 0x7fb6400c6b80 2023-10-30T12:48:07.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.782+0000 7fb63cff9640 1 -- 172.21.15.12:0/2581484371 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5739 (secure 0 0 0) 0x7fb630054b60 con 0x7fb6400c6b80 2023-10-30T12:48:07.787 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:48:07.788 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:48:04.589766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:48:07.788 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 67 2023-10-30T12:48:07.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.782+0000 7fb646580640 1 -- 172.21.15.12:0/2581484371 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb62c0664d0 msgr2=0x7fb62c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:07.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.782+0000 7fb646580640 1 --2- 172.21.15.12:0/2581484371 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb62c0664d0 0x7fb62c068990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb6400c8320 tx=0x7fb634009190 comp rx=0 tx=0).stop 2023-10-30T12:48:07.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.782+0000 7fb646580640 1 -- 172.21.15.12:0/2581484371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6400c6b80 msgr2=0x7fb640150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:07.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.782+0000 7fb646580640 1 --2- 172.21.15.12:0/2581484371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6400c6b80 0x7fb640150540 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb63000e8e0 tx=0x7fb63000eda0 comp rx=0 tx=0).stop 2023-10-30T12:48:07.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.782+0000 7fb646580640 1 -- 172.21.15.12:0/2581484371 shutdown_connections 2023-10-30T12:48:07.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.782+0000 7fb646580640 1 --2- 172.21.15.12:0/2581484371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6400c8430 0x7fb640130260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:07.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.782+0000 7fb646580640 1 --2- 172.21.15.12:0/2581484371 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb62c0664d0 0x7fb62c068990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:07.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.782+0000 7fb646580640 1 --2- 172.21.15.12:0/2581484371 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6400c6f60 0x7fb64012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:07.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.782+0000 7fb646580640 1 --2- 172.21.15.12:0/2581484371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6400c6b80 0x7fb640150540 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:07.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.782+0000 7fb646580640 1 -- 172.21.15.12:0/2581484371 >> 172.21.15.12:0/2581484371 conn(0x7fb6400c7910 msgr2=0x7fb640134910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:07.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.782+0000 7fb646580640 1 -- 172.21.15.12:0/2581484371 shutdown_connections 2023-10-30T12:48:07.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:07.782+0000 7fb646580640 1 -- 172.21.15.12:0/2581484371 wait complete. 2023-10-30T12:48:07.808 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:48:09.809 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:48:09.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51dc1d3640 1 Processor -- start 2023-10-30T12:48:09.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51dc1d3640 1 -- start start 2023-10-30T12:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51dc1d3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f51d41538a0 0x7f51d415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51dc1d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d40ca0e0 0x7f51d415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51dc1d3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d40cb800 con 0x7f51d40cf160 2023-10-30T12:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51dc1d3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d415dc30 con 0x7f51d41538a0 2023-10-30T12:48:09.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51dc1d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d40cb330 con 0x7f51d40ca0e0 2023-10-30T12:48:09.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51d9f6f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f51d41538a0 0x7f51d415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:09.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51d9f6f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f51d41538a0 0x7f51d415d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49618/0 (socket says 172.21.15.12:49618) 2023-10-30T12:48:09.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51d9f6f640 1 -- 172.21.15.12:0/2995036253 learned_addr learned my addr 172.21.15.12:0/2995036253 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:48:09.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51d976e640 1 --2- 172.21.15.12:0/2995036253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d40ca0e0 0x7f51d415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:09.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51d9f6f640 1 -- 172.21.15.12:0/2995036253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d40ca0e0 msgr2=0x7f51d415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:48:09.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51d9f6f640 1 --2- 172.21.15.12:0/2995036253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d40ca0e0 0x7f51d415a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:09.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51d9f6f640 1 -- 172.21.15.12:0/2995036253 >> v1:172.21.15.12:6789/0 conn(0x7f51d40cf160 legacy=0x7f51d40cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:09.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51d9f6f640 1 -- 172.21.15.12:0/2995036253 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d40cad70 con 0x7f51d41538a0 2023-10-30T12:48:09.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51d8f6d640 1 -- 172.21.15.12:0/2995036253 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790900503 0 0) 0x7f51d40cb800 con 0x7f51d40cf160 2023-10-30T12:48:09.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.958+0000 7f51d976e640 1 --2- 172.21.15.12:0/2995036253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d40ca0e0 0x7f51d415a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:48:09.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.962+0000 7f51d9f6f640 1 --2- 172.21.15.12:0/2995036253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d41538a0 0x7f51d415d6f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009870 tx=0x7f51d001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19af0f1281512ed8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:09.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.962+0000 7f51d8f6d640 1 -- 172.21.15.12:0/2995036253 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d00a8a30 con 0x7f51d41538a0 2023-10-30T12:48:09.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.962+0000 7f51d8f6d640 1 -- 172.21.15.12:0/2995036253 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d00a8b90 con 0x7f51d41538a0 2023-10-30T12:48:09.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.962+0000 7f51d8f6d640 1 -- 172.21.15.12:0/2995036253 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d00b0440 con 0x7f51d41538a0 2023-10-30T12:48:09.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.962+0000 7f51dc1d3640 1 -- 172.21.15.12:0/2995036253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d41538a0 msgr2=0x7f51d415d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:09.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.962+0000 7f51dc1d3640 1 --2- 172.21.15.12:0/2995036253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d41538a0 0x7f51d415d6f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009870 tx=0x7f51d001ed30 comp rx=0 tx=0).stop 2023-10-30T12:48:09.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.962+0000 7f51dc1d3640 1 -- 172.21.15.12:0/2995036253 shutdown_connections 2023-10-30T12:48:09.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.962+0000 7f51dc1d3640 1 --2- 172.21.15.12:0/2995036253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d40ca0e0 0x7f51d415a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:09.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.962+0000 7f51dc1d3640 1 --2- 172.21.15.12:0/2995036253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d41538a0 0x7f51d415d6f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:09.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.962+0000 7f51dc1d3640 1 -- 172.21.15.12:0/2995036253 >> 172.21.15.12:0/2995036253 conn(0x7f51d406bfe0 msgr2=0x7f51d406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:09.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.962+0000 7f51dc1d3640 1 -- 172.21.15.12:0/2995036253 shutdown_connections 2023-10-30T12:48:09.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.962+0000 7f51dc1d3640 1 -- 172.21.15.12:0/2995036253 wait complete. 2023-10-30T12:48:09.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51dc1d3640 1 Processor -- start 2023-10-30T12:48:09.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51dc1d3640 1 -- start start 2023-10-30T12:48:09.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51dc1d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d40ca0e0 0x7f51d40b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:09.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51dc1d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d40cf160 0x7f51d41f0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:09.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51dc1d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d41538a0 0x7f51d41f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:09.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51dc1d3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d41e4fa0 con 0x7f51d40ca0e0 2023-10-30T12:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51dc1d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d41e4130 con 0x7f51d41538a0 2023-10-30T12:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51dc1d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d40b5160 con 0x7f51d40cf160 2023-10-30T12:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51d9f6f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d40ca0e0 0x7f51d40b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51d9f6f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d40ca0e0 0x7f51d40b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49630/0 (socket says 172.21.15.12:49630) 2023-10-30T12:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51d9f6f640 1 -- 172.21.15.12:0/3579971399 learned_addr learned my addr 172.21.15.12:0/3579971399 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51d976e640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d40cf160 0x7f51d41f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51d9f6f640 1 -- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d40cf160 msgr2=0x7f51d41f0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51d9f6f640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d40cf160 0x7f51d41f0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51d9f6f640 1 -- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d41538a0 msgr2=0x7f51d41f0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51da770640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d41538a0 0x7f51d41f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51d9f6f640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d41538a0 0x7f51d41f0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51d9f6f640 1 -- 172.21.15.12:0/3579971399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d00094d0 con 0x7f51d40ca0e0 2023-10-30T12:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51d976e640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d40cf160 0x7f51d41f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51da770640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d41538a0 0x7f51d41f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51d9f6f640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d40ca0e0 0x7f51d40b4450 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009960 tx=0x7f51d001d570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51ca7fc640 1 -- 172.21.15.12:0/3579971399 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d00a8a30 con 0x7f51d40ca0e0 2023-10-30T12:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51ca7fc640 1 -- 172.21.15.12:0/3579971399 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d0004020 con 0x7f51d40ca0e0 2023-10-30T12:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51ca7fc640 1 -- 172.21.15.12:0/3579971399 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0004390 con 0x7f51d40ca0e0 2023-10-30T12:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51dc1d3640 1 -- 172.21.15.12:0/3579971399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d415aac0 con 0x7f51d40ca0e0 2023-10-30T12:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51dc1d3640 1 -- 172.21.15.12:0/3579971399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d41e4340 con 0x7f51d40ca0e0 2023-10-30T12:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.966+0000 7f51dc1d3640 1 -- 172.21.15.12:0/3579971399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d40b2a50 con 0x7f51d40ca0e0 2023-10-30T12:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.970+0000 7f51ca7fc640 1 -- 172.21.15.12:0/3579971399 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f51d00b7040 con 0x7f51d40ca0e0 2023-10-30T12:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.970+0000 7f51ca7fc640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51b80664a0 0x7f51b8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.970+0000 7f51ca7fc640 1 -- 172.21.15.12:0/3579971399 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f51d01214f0 con 0x7f51d40ca0e0 2023-10-30T12:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.970+0000 7f51d976e640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51b80664a0 0x7f51b8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:09.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.970+0000 7f51d976e640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51b80664a0 0x7f51b8068960 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f51d40b1da0 tx=0x7f51c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:09.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:09.974+0000 7f51ca7fc640 1 -- 172.21.15.12:0/3579971399 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f51d00ee6e0 con 0x7f51d40ca0e0 2023-10-30T12:48:10.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:10.166+0000 7f51dc1d3640 1 -- 172.21.15.12:0/3579971399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51d415aac0 con 0x7f51d40ca0e0 2023-10-30T12:48:10.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:10.166+0000 7f51ca7fc640 1 -- 172.21.15.12:0/3579971399 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5739 (secure 0 0 0) 0x7f51d415aac0 con 0x7f51d40ca0e0 2023-10-30T12:48:10.174 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:48:10.174 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:48:04.589766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:48:10.174 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 67 2023-10-30T12:48:10.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:10.170+0000 7f51dc1d3640 1 -- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51b80664a0 msgr2=0x7f51b8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:10.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:10.170+0000 7f51dc1d3640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51b80664a0 0x7f51b8068960 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f51d40b1da0 tx=0x7f51c4008040 comp rx=0 tx=0).stop 2023-10-30T12:48:10.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:10.170+0000 7f51dc1d3640 1 -- 172.21.15.12:0/3579971399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d40ca0e0 msgr2=0x7f51d40b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:10.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:10.170+0000 7f51dc1d3640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d40ca0e0 0x7f51d40b4450 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009960 tx=0x7f51d001d570 comp rx=0 tx=0).stop 2023-10-30T12:48:10.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:10.170+0000 7f51dc1d3640 1 -- 172.21.15.12:0/3579971399 shutdown_connections 2023-10-30T12:48:10.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:10.170+0000 7f51dc1d3640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51d41538a0 0x7f51d41f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:10.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:10.170+0000 7f51dc1d3640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51b80664a0 0x7f51b8068960 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:10.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:10.170+0000 7f51dc1d3640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51d40cf160 0x7f51d41f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:10.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:10.170+0000 7f51dc1d3640 1 --2- 172.21.15.12:0/3579971399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51d40ca0e0 0x7f51d40b4450 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:10.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:10.170+0000 7f51dc1d3640 1 -- 172.21.15.12:0/3579971399 >> 172.21.15.12:0/3579971399 conn(0x7f51d406bfe0 msgr2=0x7f51d4152680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:10.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:10.170+0000 7f51dc1d3640 1 -- 172.21.15.12:0/3579971399 shutdown_connections 2023-10-30T12:48:10.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:10.170+0000 7f51dc1d3640 1 -- 172.21.15.12:0/3579971399 wait complete. 2023-10-30T12:48:10.195 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 67, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 67, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5124": { "addr": "172.21.15.12:6835/559649789", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 559649789, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 559649789, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5124, "incarnation": 36, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 39 }, "gid_5427": { "addr": "172.21.15.12:6836/2219276371", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 2219276371, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 2219276371, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138322906710015, "flags": 0, "gid": 5427, "incarnation": 57, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 31 }, "gid_5571": { "addr": "172.21.15.12:6837/2752088514", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2752088514, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 2752088514, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5571, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:rejoin", "state_seq": 17 } }, "last_failure": 0, "last_failure_osd_epoch": 51, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:48:04.589766+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5571, "mds_1": 5124, "mds_2": 5427 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:48:12.196 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:48:12.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.350+0000 7f8674f3e640 1 Processor -- start 2023-10-30T12:48:12.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.350+0000 7f8674f3e640 1 -- start start 2023-10-30T12:48:12.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.350+0000 7f8674f3e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8670153d70 0x7f867006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:12.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.350+0000 7f8674f3e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f867006e470 0x7f86700cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:12.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.350+0000 7f8674f3e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867006afa0 con 0x7f86700c8490 2023-10-30T12:48:12.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.350+0000 7f8674f3e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867006d780 con 0x7f867006e470 2023-10-30T12:48:12.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.350+0000 7f8674f3e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8670154150 con 0x7f8670153d70 2023-10-30T12:48:12.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.350+0000 7f8667fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f867006e470 0x7f86700cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:12.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.350+0000 7f8667fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f867006e470 0x7f86700cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45148/0 (socket says 172.21.15.12:45148) 2023-10-30T12:48:12.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.350+0000 7f8667fff640 1 -- 172.21.15.12:0/2650324190 learned_addr learned my addr 172.21.15.12:0/2650324190 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:48:12.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.350+0000 7f866e59c640 1 --2- 172.21.15.12:0/2650324190 >> v2:172.21.15.154:3300/0 conn(0x7f8670153d70 0x7f867006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:12.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f8667fff640 1 -- 172.21.15.12:0/2650324190 >> v2:172.21.15.154:3300/0 conn(0x7f8670153d70 msgr2=0x7f867006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:48:12.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f8667fff640 1 --2- 172.21.15.12:0/2650324190 >> v2:172.21.15.154:3300/0 conn(0x7f8670153d70 0x7f867006ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:12.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f8667fff640 1 -- 172.21.15.12:0/2650324190 >> v1:172.21.15.154:6789/0 conn(0x7f86700c8490 legacy=0x7f86700c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:12.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f8667fff640 1 -- 172.21.15.12:0/2650324190 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867006e030 con 0x7f867006e470 2023-10-30T12:48:12.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f8667fff640 1 --2- 172.21.15.12:0/2650324190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f867006e470 0x7f86700cce00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f867006d180 tx=0x7f865801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2823de0e7500482a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:12.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f866dd9b640 1 -- 172.21.15.12:0/2650324190 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86580aa430 con 0x7f867006e470 2023-10-30T12:48:12.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f866dd9b640 1 -- 172.21.15.12:0/2650324190 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86580aa590 con 0x7f867006e470 2023-10-30T12:48:12.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f866dd9b640 1 -- 172.21.15.12:0/2650324190 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86580aa920 con 0x7f867006e470 2023-10-30T12:48:12.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f8674f3e640 1 -- 172.21.15.12:0/2650324190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f867006e470 msgr2=0x7f86700cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:12.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f8674f3e640 1 --2- 172.21.15.12:0/2650324190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f867006e470 0x7f86700cce00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f867006d180 tx=0x7f865801c4d0 comp rx=0 tx=0).stop 2023-10-30T12:48:12.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f8674f3e640 1 -- 172.21.15.12:0/2650324190 shutdown_connections 2023-10-30T12:48:12.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f8674f3e640 1 --2- 172.21.15.12:0/2650324190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f867006e470 0x7f86700cce00 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:12.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f8674f3e640 1 --2- 172.21.15.12:0/2650324190 >> v2:172.21.15.154:3300/0 conn(0x7f8670153d70 0x7f867006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:12.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f8674f3e640 1 -- 172.21.15.12:0/2650324190 >> 172.21.15.12:0/2650324190 conn(0x7f8670071ea0 msgr2=0x7f8670065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:12.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f8674f3e640 1 -- 172.21.15.12:0/2650324190 shutdown_connections 2023-10-30T12:48:12.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.354+0000 7f8674f3e640 1 -- 172.21.15.12:0/2650324190 wait complete. 2023-10-30T12:48:12.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8674f3e640 1 Processor -- start 2023-10-30T12:48:12.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8674f3e640 1 -- start start 2023-10-30T12:48:12.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8674f3e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f867006e470 0x7f8670142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:12.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8674f3e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86700c8490 0x7f8670142b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:12.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8674f3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8670153d70 0x7f8670141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:12.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8674f3e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86701459c0 con 0x7f86700c8490 2023-10-30T12:48:12.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8674f3e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8670145490 con 0x7f867006e470 2023-10-30T12:48:12.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8674f3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8670146e00 con 0x7f8670153d70 2023-10-30T12:48:12.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8667fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86700c8490 0x7f8670142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:12.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8667fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86700c8490 0x7f8670142b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45150/0 (socket says 172.21.15.12:45150) 2023-10-30T12:48:12.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8667fff640 1 -- 172.21.15.12:0/2565345136 learned_addr learned my addr 172.21.15.12:0/2565345136 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:48:12.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f866e59c640 1 --2- 172.21.15.12:0/2565345136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f867006e470 0x7f8670142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:12.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f866ed9d640 1 --2- 172.21.15.12:0/2565345136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8670153d70 0x7f8670141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:12.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8667fff640 1 -- 172.21.15.12:0/2565345136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8670153d70 msgr2=0x7f8670141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:12.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8667fff640 1 --2- 172.21.15.12:0/2565345136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8670153d70 0x7f8670141830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:12.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8667fff640 1 -- 172.21.15.12:0/2565345136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f867006e470 msgr2=0x7f8670142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:12.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8667fff640 1 --2- 172.21.15.12:0/2565345136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f867006e470 0x7f8670142600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:12.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8667fff640 1 -- 172.21.15.12:0/2565345136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86580094d0 con 0x7f86700c8490 2023-10-30T12:48:12.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f866e59c640 1 --2- 172.21.15.12:0/2565345136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f867006e470 0x7f8670142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:48:12.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8667fff640 1 --2- 172.21.15.12:0/2565345136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86700c8490 0x7f8670142b40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f86580190e0 tx=0x7f865801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:12.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8666ffd640 1 -- 172.21.15.12:0/2565345136 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86580b3800 con 0x7f86700c8490 2023-10-30T12:48:12.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8666ffd640 1 -- 172.21.15.12:0/2565345136 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86580b3960 con 0x7f86700c8490 2023-10-30T12:48:12.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8666ffd640 1 -- 172.21.15.12:0/2565345136 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86580b3d10 con 0x7f86700c8490 2023-10-30T12:48:12.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.358+0000 7f8674f3e640 1 -- 172.21.15.12:0/2565345136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8670141120 con 0x7f86700c8490 2023-10-30T12:48:12.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.362+0000 7f8674f3e640 1 -- 172.21.15.12:0/2565345136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86701f6bb0 con 0x7f86700c8490 2023-10-30T12:48:12.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.362+0000 7f8674f3e640 1 -- 172.21.15.12:0/2565345136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8670146650 con 0x7f86700c8490 2023-10-30T12:48:12.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.362+0000 7f8666ffd640 1 -- 172.21.15.12:0/2565345136 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f86580b2380 con 0x7f86700c8490 2023-10-30T12:48:12.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.362+0000 7f8666ffd640 1 --2- 172.21.15.12:0/2565345136 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f86500664a0 0x7f8650068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:12.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.362+0000 7f8666ffd640 1 -- 172.21.15.12:0/2565345136 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f865811e640 con 0x7f86700c8490 2023-10-30T12:48:12.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.362+0000 7f866e59c640 1 --2- 172.21.15.12:0/2565345136 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f86500664a0 0x7f8650068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:12.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.366+0000 7f866e59c640 1 --2- 172.21.15.12:0/2565345136 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f86500664a0 0x7f8650068960 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f866000b9f0 tx=0x7f8660008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:12.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.370+0000 7f8666ffd640 1 -- 172.21.15.12:0/2565345136 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8670146650 con 0x7f86700c8490 2023-10-30T12:48:12.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.570+0000 7f8674f3e640 1 -- 172.21.15.12:0/2565345136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8670146650 con 0x7f86700c8490 2023-10-30T12:48:12.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.570+0000 7f8666ffd640 1 -- 172.21.15.12:0/2565345136 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+5739 (secure 0 0 0) 0x7f8670146650 con 0x7f86700c8490 2023-10-30T12:48:12.576 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:48:12.576 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:48:10.150263+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:48:12.577 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 68 2023-10-30T12:48:12.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.570+0000 7f8674f3e640 1 -- 172.21.15.12:0/2565345136 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f86500664a0 msgr2=0x7f8650068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:12.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.570+0000 7f8674f3e640 1 --2- 172.21.15.12:0/2565345136 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f86500664a0 0x7f8650068960 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f866000b9f0 tx=0x7f8660008040 comp rx=0 tx=0).stop 2023-10-30T12:48:12.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.570+0000 7f8674f3e640 1 -- 172.21.15.12:0/2565345136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86700c8490 msgr2=0x7f8670142b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:12.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.570+0000 7f8674f3e640 1 --2- 172.21.15.12:0/2565345136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86700c8490 0x7f8670142b40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f86580190e0 tx=0x7f865801ad10 comp rx=0 tx=0).stop 2023-10-30T12:48:12.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.570+0000 7f8674f3e640 1 -- 172.21.15.12:0/2565345136 shutdown_connections 2023-10-30T12:48:12.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.570+0000 7f8674f3e640 1 --2- 172.21.15.12:0/2565345136 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f86500664a0 0x7f8650068960 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:12.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.570+0000 7f8674f3e640 1 --2- 172.21.15.12:0/2565345136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8670153d70 0x7f8670141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:12.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.570+0000 7f8674f3e640 1 --2- 172.21.15.12:0/2565345136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86700c8490 0x7f8670142b40 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:12.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.570+0000 7f8674f3e640 1 --2- 172.21.15.12:0/2565345136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f867006e470 0x7f8670142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:12.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.570+0000 7f8674f3e640 1 -- 172.21.15.12:0/2565345136 >> 172.21.15.12:0/2565345136 conn(0x7f8670071ea0 msgr2=0x7f867013a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:12.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.574+0000 7f8674f3e640 1 -- 172.21.15.12:0/2565345136 shutdown_connections 2023-10-30T12:48:12.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:12.574+0000 7f8674f3e640 1 -- 172.21.15.12:0/2565345136 wait complete. 2023-10-30T12:48:12.598 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=0, replacing gid=5000 2023-10-30T12:48:12.598 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.d 2023-10-30T12:48:13.599 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2023-10-30T12:48:13.600 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-30T12:48:13.600 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-30T12:48:13.603 INFO:tasks.ceph.mds.d:Started 2023-10-30T12:48:13.603 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:48:13.680 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2023-10-30T12:48:13.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f8885348640 1 Processor -- start 2023-10-30T12:48:13.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f8885348640 1 -- start start 2023-10-30T12:48:13.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f8885348640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f88801538f0 0x7f8880159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:13.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f8885348640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f888015cd50 0x7f88800c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:13.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f8885348640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f888015d300 con 0x7f88800cefa0 2023-10-30T12:48:13.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f8885348640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f888015d5e0 con 0x7f888015cd50 2023-10-30T12:48:13.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f8885348640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f888015d8c0 con 0x7f88801538f0 2023-10-30T12:48:13.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f887f7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f888015cd50 0x7f88800c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:13.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f8884b47640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f88800cefa0 0x7f88800cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37264/0 (socket says 172.21.15.12:37264) 2023-10-30T12:48:13.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f8884b47640 1 -- 172.21.15.12:0/3260270872 learned_addr learned my addr 172.21.15.12:0/3260270872 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:48:13.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f887ffff640 1 --2- 172.21.15.12:0/3260270872 >> v2:172.21.15.154:3300/0 conn(0x7f88801538f0 0x7f8880159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:13.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f887f7fe640 1 -- 172.21.15.12:0/3260270872 >> v2:172.21.15.154:3300/0 conn(0x7f88801538f0 msgr2=0x7f8880159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:48:13.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f887f7fe640 1 --2- 172.21.15.12:0/3260270872 >> v2:172.21.15.154:3300/0 conn(0x7f88801538f0 0x7f8880159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:13.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f887f7fe640 1 -- 172.21.15.12:0/3260270872 >> v1:172.21.15.12:6789/0 conn(0x7f88800cefa0 legacy=0x7f88800cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:13.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f887f7fe640 1 -- 172.21.15.12:0/3260270872 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888015dba0 con 0x7f888015cd50 2023-10-30T12:48:13.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.754+0000 7f887effd640 1 -- 172.21.15.12:0/3260270872 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469425530 0 0) 0x7f888015d300 con 0x7f88800cefa0 2023-10-30T12:48:13.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.758+0000 7f887f7fe640 1 --2- 172.21.15.12:0/3260270872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f888015cd50 0x7f88800c9f90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f88800c9cb0 tx=0x7f887401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77896119da90e88 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:13.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.758+0000 7f887effd640 1 -- 172.21.15.12:0/3260270872 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88740a8950 con 0x7f888015cd50 2023-10-30T12:48:13.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.758+0000 7f887effd640 1 -- 172.21.15.12:0/3260270872 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88740a8ab0 con 0x7f888015cd50 2023-10-30T12:48:13.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.758+0000 7f887effd640 1 -- 172.21.15.12:0/3260270872 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88740a8e40 con 0x7f888015cd50 2023-10-30T12:48:13.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.758+0000 7f8885348640 1 -- 172.21.15.12:0/3260270872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f888015cd50 msgr2=0x7f88800c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:13.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.758+0000 7f8885348640 1 --2- 172.21.15.12:0/3260270872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f888015cd50 0x7f88800c9f90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f88800c9cb0 tx=0x7f887401ec50 comp rx=0 tx=0).stop 2023-10-30T12:48:13.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.758+0000 7f8885348640 1 -- 172.21.15.12:0/3260270872 shutdown_connections 2023-10-30T12:48:13.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.758+0000 7f8885348640 1 --2- 172.21.15.12:0/3260270872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f888015cd50 0x7f88800c9f90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:13.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.758+0000 7f8885348640 1 --2- 172.21.15.12:0/3260270872 >> v2:172.21.15.154:3300/0 conn(0x7f88801538f0 0x7f8880159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:13.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.758+0000 7f8885348640 1 -- 172.21.15.12:0/3260270872 >> 172.21.15.12:0/3260270872 conn(0x7f888006a640 msgr2=0x7f8880065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:13.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.758+0000 7f8885348640 1 -- 172.21.15.12:0/3260270872 shutdown_connections 2023-10-30T12:48:13.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.758+0000 7f8885348640 1 -- 172.21.15.12:0/3260270872 wait complete. 2023-10-30T12:48:13.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.758+0000 7f8885348640 1 Processor -- start 2023-10-30T12:48:13.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f8885348640 1 -- start start 2023-10-30T12:48:13.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f8885348640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88800cefa0 0x7f888015b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:13.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f8885348640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88801538f0 0x7f88800b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:13.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f8885348640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f888015cd50 0x7f8880144810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:13.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f8885348640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888015bdd0 con 0x7f88800cefa0 2023-10-30T12:48:13.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f8885348640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88801f2cd0 con 0x7f888015cd50 2023-10-30T12:48:13.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f8885348640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88801f20c0 con 0x7f88801538f0 2023-10-30T12:48:13.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f887ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88800cefa0 0x7f888015b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:13.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f887ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88800cefa0 0x7f888015b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45160/0 (socket says 172.21.15.12:45160) 2023-10-30T12:48:13.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f887ffff640 1 -- 172.21.15.12:0/3208771247 learned_addr learned my addr 172.21.15.12:0/3208771247 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:48:13.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f8884b47640 1 --2- 172.21.15.12:0/3208771247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f888015cd50 0x7f8880144810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:13.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f887f7fe640 1 --2- 172.21.15.12:0/3208771247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88801538f0 0x7f88800b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:13.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f887ffff640 1 -- 172.21.15.12:0/3208771247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88801538f0 msgr2=0x7f88800b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:13.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f887ffff640 1 --2- 172.21.15.12:0/3208771247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88801538f0 0x7f88800b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:13.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f887ffff640 1 -- 172.21.15.12:0/3208771247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f888015cd50 msgr2=0x7f8880144810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:13.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f887ffff640 1 --2- 172.21.15.12:0/3208771247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f888015cd50 0x7f8880144810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:13.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f887ffff640 1 -- 172.21.15.12:0/3208771247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8874009420 con 0x7f88800cefa0 2023-10-30T12:48:13.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f887f7fe640 1 --2- 172.21.15.12:0/3208771247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88801538f0 0x7f88800b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:48:13.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f887ffff640 1 --2- 172.21.15.12:0/3208771247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88800cefa0 0x7f888015b7b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f887000e960 tx=0x7f887000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:13.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f887cff9640 1 -- 172.21.15.12:0/3208771247 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8870009690 con 0x7f88800cefa0 2023-10-30T12:48:13.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f887cff9640 1 -- 172.21.15.12:0/3208771247 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887000ca00 con 0x7f88800cefa0 2023-10-30T12:48:13.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f887cff9640 1 -- 172.21.15.12:0/3208771247 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887000cd70 con 0x7f88800cefa0 2023-10-30T12:48:13.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f8885348640 1 -- 172.21.15.12:0/3208771247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88801f23a0 con 0x7f88800cefa0 2023-10-30T12:48:13.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f8885348640 1 -- 172.21.15.12:0/3208771247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88801f2960 con 0x7f88800cefa0 2023-10-30T12:48:13.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.762+0000 7f8885348640 1 -- 172.21.15.12:0/3208771247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888004e170 con 0x7f88800cefa0 2023-10-30T12:48:13.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.766+0000 7f887cff9640 1 -- 172.21.15.12:0/3208771247 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8870013470 con 0x7f88800cefa0 2023-10-30T12:48:13.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.766+0000 7f887cff9640 1 --2- 172.21.15.12:0/3208771247 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f886c0664d0 0x7f886c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:13.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.766+0000 7f887cff9640 1 -- 172.21.15.12:0/3208771247 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f88700843a0 con 0x7f88800cefa0 2023-10-30T12:48:13.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.766+0000 7f887f7fe640 1 --2- 172.21.15.12:0/3208771247 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f886c0664d0 0x7f886c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:13.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.766+0000 7f887f7fe640 1 --2- 172.21.15.12:0/3208771247 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f886c0664d0 0x7f886c068990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f88740b2000 tx=0x7f8874008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:13.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.770+0000 7f887cff9640 1 -- 172.21.15.12:0/3208771247 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8870051590 con 0x7f88800cefa0 2023-10-30T12:48:13.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.958+0000 7f8885348640 1 -- 172.21.15.12:0/3208771247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88800b2290 con 0x7f88800cefa0 2023-10-30T12:48:13.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.962+0000 7f887cff9640 1 -- 172.21.15.12:0/3208771247 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+5739 (secure 0 0 0) 0x7f8870054bb0 con 0x7f88800cefa0 2023-10-30T12:48:13.969 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:48:13.969 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:48:10.150263+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:48:13.970 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 68 2023-10-30T12:48:13.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.962+0000 7f8885348640 1 -- 172.21.15.12:0/3208771247 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f886c0664d0 msgr2=0x7f886c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:13.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.962+0000 7f8885348640 1 --2- 172.21.15.12:0/3208771247 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f886c0664d0 0x7f886c068990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f88740b2000 tx=0x7f8874008040 comp rx=0 tx=0).stop 2023-10-30T12:48:13.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.962+0000 7f8885348640 1 -- 172.21.15.12:0/3208771247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88800cefa0 msgr2=0x7f888015b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:13.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.962+0000 7f8885348640 1 --2- 172.21.15.12:0/3208771247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88800cefa0 0x7f888015b7b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f887000e960 tx=0x7f887000ee20 comp rx=0 tx=0).stop 2023-10-30T12:48:13.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.962+0000 7f8885348640 1 -- 172.21.15.12:0/3208771247 shutdown_connections 2023-10-30T12:48:13.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.962+0000 7f8885348640 1 --2- 172.21.15.12:0/3208771247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f888015cd50 0x7f8880144810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:13.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.962+0000 7f8885348640 1 --2- 172.21.15.12:0/3208771247 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f886c0664d0 0x7f886c068990 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:13.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.962+0000 7f8885348640 1 --2- 172.21.15.12:0/3208771247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88801538f0 0x7f88800b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:13.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.962+0000 7f8885348640 1 --2- 172.21.15.12:0/3208771247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88800cefa0 0x7f888015b7b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:13.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.962+0000 7f8885348640 1 -- 172.21.15.12:0/3208771247 >> 172.21.15.12:0/3208771247 conn(0x7f888006a640 msgr2=0x7f88800c9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:13.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.962+0000 7f8885348640 1 -- 172.21.15.12:0/3208771247 shutdown_connections 2023-10-30T12:48:13.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:13.962+0000 7f8885348640 1 -- 172.21.15.12:0/3208771247 wait complete. 2023-10-30T12:48:13.988 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2023-10-30T12:48:15.989 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:48:16.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.142+0000 7f818953e640 1 Processor -- start 2023-10-30T12:48:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.142+0000 7f818953e640 1 -- start start 2023-10-30T12:48:16.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.142+0000 7f818953e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8184153d70 0x7f818406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:16.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.142+0000 7f818953e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f818406e470 0x7f81840cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:16.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.142+0000 7f818953e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818406afa0 con 0x7f81840c8490 2023-10-30T12:48:16.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.146+0000 7f818953e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818406d780 con 0x7f818406e470 2023-10-30T12:48:16.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.146+0000 7f818953e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8184154150 con 0x7f8184153d70 2023-10-30T12:48:16.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.146+0000 7f817bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f818406e470 0x7f81840cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:16.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.146+0000 7f8182ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8184153d70 0x7f818406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:16.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.146+0000 7f81837fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f81840c8490 0x7f81840c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:60292/0 (socket says 172.21.15.12:60292) 2023-10-30T12:48:16.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.146+0000 7f81837fe640 1 -- 172.21.15.12:0/3814853085 learned_addr learned my addr 172.21.15.12:0/3814853085 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:48:16.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.146+0000 7f817bfff640 1 -- 172.21.15.12:0/3814853085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8184153d70 msgr2=0x7f818406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:48:16.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.146+0000 7f817bfff640 1 --2- 172.21.15.12:0/3814853085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8184153d70 0x7f818406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:16.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.146+0000 7f817bfff640 1 -- 172.21.15.12:0/3814853085 >> v1:172.21.15.154:6789/0 conn(0x7f81840c8490 legacy=0x7f81840c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:16.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.146+0000 7f817bfff640 1 -- 172.21.15.12:0/3814853085 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818406e030 con 0x7f818406e470 2023-10-30T12:48:16.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f817bfff640 1 --2- 172.21.15.12:0/3814853085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f818406e470 0x7f81840cce00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f816c0097c0 tx=0x7f816c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e634025f28553fd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:16.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f81827fc640 1 -- 172.21.15.12:0/3814853085 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c0ab650 con 0x7f818406e470 2023-10-30T12:48:16.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f81827fc640 1 -- 172.21.15.12:0/3814853085 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c0ab7b0 con 0x7f818406e470 2023-10-30T12:48:16.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f81827fc640 1 -- 172.21.15.12:0/3814853085 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c0abb40 con 0x7f818406e470 2023-10-30T12:48:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 -- 172.21.15.12:0/3814853085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f818406e470 msgr2=0x7f81840cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 --2- 172.21.15.12:0/3814853085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f818406e470 0x7f81840cce00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f816c0097c0 tx=0x7f816c01c450 comp rx=0 tx=0).stop 2023-10-30T12:48:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 -- 172.21.15.12:0/3814853085 shutdown_connections 2023-10-30T12:48:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 --2- 172.21.15.12:0/3814853085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f818406e470 0x7f81840cce00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 --2- 172.21.15.12:0/3814853085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8184153d70 0x7f818406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 -- 172.21.15.12:0/3814853085 >> 172.21.15.12:0/3814853085 conn(0x7f8184071ea0 msgr2=0x7f8184065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 -- 172.21.15.12:0/3814853085 shutdown_connections 2023-10-30T12:48:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 -- 172.21.15.12:0/3814853085 wait complete. 2023-10-30T12:48:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 Processor -- start 2023-10-30T12:48:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 -- start start 2023-10-30T12:48:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818406e470 0x7f8184156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:16.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81840c8490 0x7f818413f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:16.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8184153d70 0x7f8184154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:16.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8184156ba0 con 0x7f818406e470 2023-10-30T12:48:16.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8184140870 con 0x7f8184153d70 2023-10-30T12:48:16.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.150+0000 7f818953e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81840cafa0 con 0x7f81840c8490 2023-10-30T12:48:16.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f8182ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818406e470 0x7f8184156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:16.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f8182ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818406e470 0x7f8184156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45176/0 (socket says 172.21.15.12:45176) 2023-10-30T12:48:16.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f8182ffd640 1 -- 172.21.15.12:0/801653299 learned_addr learned my addr 172.21.15.12:0/801653299 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:48:16.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f817bfff640 1 --2- 172.21.15.12:0/801653299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81840c8490 0x7f818413f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:16.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f81837fe640 1 --2- 172.21.15.12:0/801653299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8184153d70 0x7f8184154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:16.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f8182ffd640 1 -- 172.21.15.12:0/801653299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81840c8490 msgr2=0x7f818413f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:16.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f8182ffd640 1 --2- 172.21.15.12:0/801653299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81840c8490 0x7f818413f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:16.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f8182ffd640 1 -- 172.21.15.12:0/801653299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8184153d70 msgr2=0x7f8184154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:16.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f8182ffd640 1 --2- 172.21.15.12:0/801653299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8184153d70 0x7f8184154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:16.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f8182ffd640 1 -- 172.21.15.12:0/801653299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c009420 con 0x7f818406e470 2023-10-30T12:48:16.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f8182ffd640 1 --2- 172.21.15.12:0/801653299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818406e470 0x7f8184156ff0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f817400e870 tx=0x7f817400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:16.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f817b7fe640 1 -- 172.21.15.12:0/801653299 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174009510 con 0x7f818406e470 2023-10-30T12:48:16.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f817b7fe640 1 -- 172.21.15.12:0/801653299 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817400eec0 con 0x7f818406e470 2023-10-30T12:48:16.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f817b7fe640 1 -- 172.21.15.12:0/801653299 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817400cae0 con 0x7f818406e470 2023-10-30T12:48:16.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f818953e640 1 -- 172.21.15.12:0/801653299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8184151530 con 0x7f818406e470 2023-10-30T12:48:16.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f818953e640 1 -- 172.21.15.12:0/801653299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81840cb240 con 0x7f818406e470 2023-10-30T12:48:16.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f818953e640 1 -- 172.21.15.12:0/801653299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818404e170 con 0x7f818406e470 2023-10-30T12:48:16.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.154+0000 7f817b7fe640 1 -- 172.21.15.12:0/801653299 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f817400cc40 con 0x7f818406e470 2023-10-30T12:48:16.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.158+0000 7f817b7fe640 1 --2- 172.21.15.12:0/801653299 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f81640664a0 0x7f8164068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:48:16.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.158+0000 7f817b7fe640 1 -- 172.21.15.12:0/801653299 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f8174084120 con 0x7f818406e470 2023-10-30T12:48:16.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.158+0000 7f817bfff640 1 --2- 172.21.15.12:0/801653299 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f81640664a0 0x7f8164068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:48:16.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.158+0000 7f817bfff640 1 --2- 172.21.15.12:0/801653299 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f81640664a0 0x7f8164068960 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f816c0af000 tx=0x7f816c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:48:16.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.162+0000 7f817b7fe640 1 -- 172.21.15.12:0/801653299 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f81740513c0 con 0x7f818406e470 2023-10-30T12:48:16.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.350+0000 7f818953e640 1 -- 172.21.15.12:0/801653299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f818406bd00 con 0x7f818406e470 2023-10-30T12:48:16.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.350+0000 7f817b7fe640 1 -- 172.21.15.12:0/801653299 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+6533 (secure 0 0 0) 0x7f8174059e60 con 0x7f818406e470 2023-10-30T12:48:16.357 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:48:16.357 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:48:10.150263+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:48:16.357 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 69 2023-10-30T12:48:16.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.354+0000 7f818953e640 1 -- 172.21.15.12:0/801653299 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f81640664a0 msgr2=0x7f8164068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:16.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.354+0000 7f818953e640 1 --2- 172.21.15.12:0/801653299 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f81640664a0 0x7f8164068960 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f816c0af000 tx=0x7f816c0a7000 comp rx=0 tx=0).stop 2023-10-30T12:48:16.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.354+0000 7f818953e640 1 -- 172.21.15.12:0/801653299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818406e470 msgr2=0x7f8184156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:48:16.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.354+0000 7f818953e640 1 --2- 172.21.15.12:0/801653299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818406e470 0x7f8184156ff0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f817400e870 tx=0x7f817400ec70 comp rx=0 tx=0).stop 2023-10-30T12:48:16.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.354+0000 7f818953e640 1 -- 172.21.15.12:0/801653299 shutdown_connections 2023-10-30T12:48:16.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.354+0000 7f818953e640 1 --2- 172.21.15.12:0/801653299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8184153d70 0x7f8184154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:16.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.354+0000 7f818953e640 1 --2- 172.21.15.12:0/801653299 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f81640664a0 0x7f8164068960 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:16.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.354+0000 7f818953e640 1 --2- 172.21.15.12:0/801653299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81840c8490 0x7f818413f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:16.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.354+0000 7f818953e640 1 --2- 172.21.15.12:0/801653299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f818406e470 0x7f8184156ff0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:48:16.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.354+0000 7f818953e640 1 -- 172.21.15.12:0/801653299 >> 172.21.15.12:0/801653299 conn(0x7f8184071ea0 msgr2=0x7f8184151da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:48:16.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.354+0000 7f818953e640 1 -- 172.21.15.12:0/801653299 shutdown_connections 2023-10-30T12:48:16.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:48:16.354+0000 7f818953e640 1 -- 172.21.15.12:0/801653299 wait complete. 2023-10-30T12:48:16.378 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2023-10-30T12:48:16.379 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 63 secs before thrashing 2023-10-30T12:48:19.382 INFO:tasks.ceph.mds.c.smithi012.stderr:2023-10-30T12:48:19.378+0000 7f262fa5d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-30T12:48:19.383 INFO:tasks.ceph.mds.c.smithi012.stderr:2023-10-30T12:48:19.378+0000 7f262fa5d640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-30T12:48:19.383 INFO:tasks.ceph.mds.c.smithi012.stderr:2023-10-30T12:48:19.378+0000 7f262fa5d640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-30T12:48:35.976 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:48:35.981 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:49:06.039 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:49:06.043 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:49:19.379 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:19.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.533+0000 7f5a52446640 1 Processor -- start 2023-10-30T12:49:19.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.533+0000 7f5a52446640 1 -- start start 2023-10-30T12:49:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.533+0000 7f5a52446640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5a4c1538a0 0x7f5a4c15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.533+0000 7f5a52446640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a4c0cf160 0x7f5a4c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.533+0000 7f5a52446640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4c0cb800 con 0x7f5a4c0ca0e0 2023-10-30T12:49:19.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.533+0000 7f5a52446640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4c15dc30 con 0x7f5a4c1538a0 2023-10-30T12:49:19.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.533+0000 7f5a52446640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c0cb330 con 0x7f5a4c0cf160 2023-10-30T12:49:19.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.533+0000 7f5a4bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5a4c1538a0 0x7f5a4c15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:19.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.533+0000 7f5a509e3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a4c0cf160 0x7f5a4c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:19.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.533+0000 7f5a4b7fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5a4c0ca0e0 0x7f5a4c15a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:57040/0 (socket says 172.21.15.12:57040) 2023-10-30T12:49:19.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.533+0000 7f5a4b7fe640 1 -- 172.21.15.12:0/599097837 learned_addr learned my addr 172.21.15.12:0/599097837 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:49:19.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.533+0000 7f5a4bfff640 1 -- 172.21.15.12:0/599097837 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a4c0cf160 msgr2=0x7f5a4c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:19.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.533+0000 7f5a4bfff640 1 --2- 172.21.15.12:0/599097837 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a4c0cf160 0x7f5a4c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:19.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.533+0000 7f5a4bfff640 1 -- 172.21.15.12:0/599097837 >> v1:172.21.15.154:6789/0 conn(0x7f5a4c0ca0e0 legacy=0x7f5a4c15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:19.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.533+0000 7f5a4bfff640 1 -- 172.21.15.12:0/599097837 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4c0cad70 con 0x7f5a4c1538a0 2023-10-30T12:49:19.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.541+0000 7f5a4bfff640 1 --2- 172.21.15.12:0/599097837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a4c1538a0 0x7f5a4c15d6f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0097c0 tx=0x7f5a3c01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177dd606aee5ccc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:19.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.541+0000 7f5a4affd640 1 -- 172.21.15.12:0/599097837 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c0a8970 con 0x7f5a4c1538a0 2023-10-30T12:49:19.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.541+0000 7f5a4affd640 1 -- 172.21.15.12:0/599097837 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3c0a8ad0 con 0x7f5a4c1538a0 2023-10-30T12:49:19.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.541+0000 7f5a4affd640 1 -- 172.21.15.12:0/599097837 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c0a8e60 con 0x7f5a4c1538a0 2023-10-30T12:49:19.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.541+0000 7f5a52446640 1 -- 172.21.15.12:0/599097837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a4c1538a0 msgr2=0x7f5a4c15d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:19.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.541+0000 7f5a52446640 1 --2- 172.21.15.12:0/599097837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a4c1538a0 0x7f5a4c15d6f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0097c0 tx=0x7f5a3c01ec70 comp rx=0 tx=0).stop 2023-10-30T12:49:19.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.541+0000 7f5a52446640 1 -- 172.21.15.12:0/599097837 shutdown_connections 2023-10-30T12:49:19.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.541+0000 7f5a52446640 1 --2- 172.21.15.12:0/599097837 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a4c0cf160 0x7f5a4c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:19.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.541+0000 7f5a52446640 1 --2- 172.21.15.12:0/599097837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a4c1538a0 0x7f5a4c15d6f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:19.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.541+0000 7f5a52446640 1 -- 172.21.15.12:0/599097837 >> 172.21.15.12:0/599097837 conn(0x7f5a4c06bfe0 msgr2=0x7f5a4c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:19.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.541+0000 7f5a52446640 1 -- 172.21.15.12:0/599097837 shutdown_connections 2023-10-30T12:49:19.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.541+0000 7f5a52446640 1 -- 172.21.15.12:0/599097837 wait complete. 2023-10-30T12:49:19.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a52446640 1 Processor -- start 2023-10-30T12:49:19.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a52446640 1 -- start start 2023-10-30T12:49:19.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a52446640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a4c0ca0e0 0x7f5a4c158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:19.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a52446640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4c0cf160 0x7f5a4c158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:19.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a52446640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a4c1538a0 0x7f5a4c1f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:19.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a52446640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c0c5370 con 0x7f5a4c0cf160 2023-10-30T12:49:19.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a52446640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c1580a0 con 0x7f5a4c1538a0 2023-10-30T12:49:19.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a52446640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c0ceec0 con 0x7f5a4c0ca0e0 2023-10-30T12:49:19.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a4b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4c0cf160 0x7f5a4c158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a4bfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a4c0ca0e0 0x7f5a4c158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a509e3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a4c1538a0 0x7f5a4c1f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a4bfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a4c0ca0e0 0x7f5a4c158530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:53660/0 (socket says 172.21.15.12:53660) 2023-10-30T12:49:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a4bfff640 1 -- 172.21.15.12:0/1233209641 learned_addr learned my addr 172.21.15.12:0/1233209641 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a4b7fe640 1 -- 172.21.15.12:0/1233209641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a4c0ca0e0 msgr2=0x7f5a4c158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a4b7fe640 1 --2- 172.21.15.12:0/1233209641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a4c0ca0e0 0x7f5a4c158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a4b7fe640 1 -- 172.21.15.12:0/1233209641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a4c1538a0 msgr2=0x7f5a4c1f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a4b7fe640 1 --2- 172.21.15.12:0/1233209641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a4c1538a0 0x7f5a4c1f7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a4b7fe640 1 -- 172.21.15.12:0/1233209641 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c009420 con 0x7f5a4c0cf160 2023-10-30T12:49:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.545+0000 7f5a4b7fe640 1 --2- 172.21.15.12:0/1233209641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4c0cf160 0x7f5a4c158a70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000c460 tx=0x7f5a4000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:19.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.549+0000 7f5a48ff9640 1 -- 172.21.15.12:0/1233209641 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40005cf0 con 0x7f5a4c0cf160 2023-10-30T12:49:19.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.549+0000 7f5a48ff9640 1 -- 172.21.15.12:0/1233209641 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a40005e50 con 0x7f5a4c0cf160 2023-10-30T12:49:19.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.549+0000 7f5a52446640 1 -- 172.21.15.12:0/1233209641 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4c0cebe0 con 0x7f5a4c0cf160 2023-10-30T12:49:19.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.549+0000 7f5a48ff9640 1 -- 172.21.15.12:0/1233209641 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40007270 con 0x7f5a4c0cf160 2023-10-30T12:49:19.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.549+0000 7f5a52446640 1 -- 172.21.15.12:0/1233209641 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4c151490 con 0x7f5a4c0cf160 2023-10-30T12:49:19.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.549+0000 7f5a52446640 1 -- 172.21.15.12:0/1233209641 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4c0b2a50 con 0x7f5a4c0cf160 2023-10-30T12:49:19.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.549+0000 7f5a48ff9640 1 -- 172.21.15.12:0/1233209641 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5a400073d0 con 0x7f5a4c0cf160 2023-10-30T12:49:19.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.553+0000 7f5a48ff9640 1 --2- 172.21.15.12:0/1233209641 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5a380664d0 0x7f5a38068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:19.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.553+0000 7f5a48ff9640 1 -- 172.21.15.12:0/1233209641 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f5a40083520 con 0x7f5a4c0cf160 2023-10-30T12:49:19.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.553+0000 7f5a4bfff640 1 --2- 172.21.15.12:0/1233209641 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5a380664d0 0x7f5a38068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.557+0000 7f5a4bfff640 1 --2- 172.21.15.12:0/1233209641 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5a380664d0 0x7f5a38068990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0097c0 tx=0x7f5a3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:19.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.561+0000 7f5a48ff9640 1 -- 172.21.15.12:0/1233209641 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5a40050730 con 0x7f5a4c0cf160 2023-10-30T12:49:19.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.797+0000 7f5a52446640 1 -- 172.21.15.12:0/1233209641 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a4c04e170 con 0x7f5a4c0cf160 2023-10-30T12:49:19.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.801+0000 7f5a48ff9640 1 -- 172.21.15.12:0/1233209641 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6538 (secure 0 0 0) 0x7f5a40053d50 con 0x7f5a4c0cf160 2023-10-30T12:49:19.807 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:19.807 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:04.349963+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:19.807 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2023-10-30T12:49:19.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.801+0000 7f5a52446640 1 -- 172.21.15.12:0/1233209641 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5a380664d0 msgr2=0x7f5a38068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:19.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.801+0000 7f5a52446640 1 --2- 172.21.15.12:0/1233209641 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5a380664d0 0x7f5a38068990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0097c0 tx=0x7f5a3c008040 comp rx=0 tx=0).stop 2023-10-30T12:49:19.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.801+0000 7f5a52446640 1 -- 172.21.15.12:0/1233209641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4c0cf160 msgr2=0x7f5a4c158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:19.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.801+0000 7f5a52446640 1 --2- 172.21.15.12:0/1233209641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4c0cf160 0x7f5a4c158a70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000c460 tx=0x7f5a4000c920 comp rx=0 tx=0).stop 2023-10-30T12:49:19.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.801+0000 7f5a52446640 1 -- 172.21.15.12:0/1233209641 shutdown_connections 2023-10-30T12:49:19.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.801+0000 7f5a52446640 1 --2- 172.21.15.12:0/1233209641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a4c1538a0 0x7f5a4c1f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:19.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.801+0000 7f5a52446640 1 --2- 172.21.15.12:0/1233209641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4c0cf160 0x7f5a4c158a70 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:19.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.801+0000 7f5a52446640 1 --2- 172.21.15.12:0/1233209641 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5a380664d0 0x7f5a38068990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:19.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.801+0000 7f5a52446640 1 --2- 172.21.15.12:0/1233209641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a4c0ca0e0 0x7f5a4c158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:19.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.801+0000 7f5a52446640 1 -- 172.21.15.12:0/1233209641 >> 172.21.15.12:0/1233209641 conn(0x7f5a4c06bfe0 msgr2=0x7f5a4c065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:19.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.801+0000 7f5a52446640 1 -- 172.21.15.12:0/1233209641 shutdown_connections 2023-10-30T12:49:19.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:19.801+0000 7f5a52446640 1 -- 172.21.15.12:0/1233209641 wait complete. 2023-10-30T12:49:19.828 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=1) 2023-10-30T12:49:19.828 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-30T12:49:19.828 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:49:19.846 INFO:tasks.ceph.mds.a:Stopped 2023-10-30T12:49:19.846 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:20.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1ce7d3640 1 Processor -- start 2023-10-30T12:49:20.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1ce7d3640 1 -- start start 2023-10-30T12:49:20.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1ce7d3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb1c815a410 0x7fb1c806a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:20.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1ce7d3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb1c806cb80 0x7fb1c806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:20.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1ce7d3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c806d530 con 0x7fb1c8069e90 2023-10-30T12:49:20.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1ce7d3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c80c8570 con 0x7fb1c815a410 2023-10-30T12:49:20.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1ce7d3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c815a850 con 0x7fb1c806cb80 2023-10-30T12:49:20.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1c77fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb1c815a410 0x7fb1c806a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:20.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1c7fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb1c8069e90 0x7fb1c80c9cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36328/0 (socket says 172.21.15.12:36328) 2023-10-30T12:49:20.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1c7fff640 1 -- 172.21.15.12:0/3088490536 learned_addr learned my addr 172.21.15.12:0/3088490536 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:49:20.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1ccd70640 1 --2- 172.21.15.12:0/3088490536 >> v2:172.21.15.154:3300/0 conn(0x7fb1c806cb80 0x7fb1c806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:20.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1c77fe640 1 -- 172.21.15.12:0/3088490536 >> v2:172.21.15.154:3300/0 conn(0x7fb1c806cb80 msgr2=0x7fb1c806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:20.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1c77fe640 1 --2- 172.21.15.12:0/3088490536 >> v2:172.21.15.154:3300/0 conn(0x7fb1c806cb80 0x7fb1c806cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:20.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1c77fe640 1 -- 172.21.15.12:0/3088490536 >> v1:172.21.15.12:6789/0 conn(0x7fb1c8069e90 legacy=0x7fb1c80c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:20.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1c77fe640 1 -- 172.21.15.12:0/3088490536 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c80c6c10 con 0x7fb1c815a410 2023-10-30T12:49:20.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.029+0000 7fb1c6ffd640 1 -- 172.21.15.12:0/3088490536 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 503805625 0 0) 0x7fb1c806d530 con 0x7fb1c8069e90 2023-10-30T12:49:20.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.033+0000 7fb1c77fe640 1 --2- 172.21.15.12:0/3088490536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1c815a410 0x7fb1c806a270 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80ca5c0 tx=0x7fb1bc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bce1909c0a332 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:20.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.033+0000 7fb1c6ffd640 1 -- 172.21.15.12:0/3088490536 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0ab600 con 0x7fb1c815a410 2023-10-30T12:49:20.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.033+0000 7fb1c6ffd640 1 -- 172.21.15.12:0/3088490536 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc0a7050 con 0x7fb1c815a410 2023-10-30T12:49:20.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.033+0000 7fb1c6ffd640 1 -- 172.21.15.12:0/3088490536 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0ab980 con 0x7fb1c815a410 2023-10-30T12:49:20.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.033+0000 7fb1ce7d3640 1 -- 172.21.15.12:0/3088490536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1c815a410 msgr2=0x7fb1c806a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:20.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.033+0000 7fb1ce7d3640 1 --2- 172.21.15.12:0/3088490536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1c815a410 0x7fb1c806a270 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80ca5c0 tx=0x7fb1bc01c460 comp rx=0 tx=0).stop 2023-10-30T12:49:20.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.033+0000 7fb1ce7d3640 1 -- 172.21.15.12:0/3088490536 shutdown_connections 2023-10-30T12:49:20.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.033+0000 7fb1ce7d3640 1 --2- 172.21.15.12:0/3088490536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1c815a410 0x7fb1c806a270 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:20.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.033+0000 7fb1ce7d3640 1 --2- 172.21.15.12:0/3088490536 >> v2:172.21.15.154:3300/0 conn(0x7fb1c806cb80 0x7fb1c806cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:20.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.033+0000 7fb1ce7d3640 1 -- 172.21.15.12:0/3088490536 >> 172.21.15.12:0/3088490536 conn(0x7fb1c806dfd0 msgr2=0x7fb1c806e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:20.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.033+0000 7fb1ce7d3640 1 -- 172.21.15.12:0/3088490536 shutdown_connections 2023-10-30T12:49:20.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.033+0000 7fb1ce7d3640 1 -- 172.21.15.12:0/3088490536 wait complete. 2023-10-30T12:49:20.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1ce7d3640 1 Processor -- start 2023-10-30T12:49:20.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1ce7d3640 1 -- start start 2023-10-30T12:49:20.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1ce7d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1c8069e90 0x7fb1c81489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:20.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1ce7d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1c806cb80 0x7fb1c81463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:20.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1ce7d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1c815a410 0x7fb1c81f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:20.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1ce7d3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c814b6b0 con 0x7fb1c8069e90 2023-10-30T12:49:20.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1ce7d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c8144af0 con 0x7fb1c806cb80 2023-10-30T12:49:20.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1ce7d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c8148550 con 0x7fb1c815a410 2023-10-30T12:49:20.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1c8069e90 0x7fb1c81489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:20.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1c8069e90 0x7fb1c81489e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51840/0 (socket says 172.21.15.12:51840) 2023-10-30T12:49:20.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1c7fff640 1 -- 172.21.15.12:0/561252616 learned_addr learned my addr 172.21.15.12:0/561252616 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:20.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1c77fe640 1 --2- 172.21.15.12:0/561252616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1c806cb80 0x7fb1c81463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:20.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1ccd70640 1 --2- 172.21.15.12:0/561252616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1c815a410 0x7fb1c81f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1c7fff640 1 -- 172.21.15.12:0/561252616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1c815a410 msgr2=0x7fb1c81f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1c7fff640 1 --2- 172.21.15.12:0/561252616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1c815a410 0x7fb1c81f49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1c7fff640 1 -- 172.21.15.12:0/561252616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1c806cb80 msgr2=0x7fb1c81463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1c7fff640 1 --2- 172.21.15.12:0/561252616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1c806cb80 0x7fb1c81463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1c7fff640 1 -- 172.21.15.12:0/561252616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc009420 con 0x7fb1c8069e90 2023-10-30T12:49:20.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1c77fe640 1 --2- 172.21.15.12:0/561252616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1c806cb80 0x7fb1c81463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:20.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1c7fff640 1 --2- 172.21.15.12:0/561252616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1c8069e90 0x7fb1c81489e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800c520 tx=0x7fb1b800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:20.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1c4ff9640 1 -- 172.21.15.12:0/561252616 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80074e0 con 0x7fb1c8069e90 2023-10-30T12:49:20.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1c4ff9640 1 -- 172.21.15.12:0/561252616 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b8007640 con 0x7fb1c8069e90 2023-10-30T12:49:20.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1c4ff9640 1 -- 172.21.15.12:0/561252616 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80049f0 con 0x7fb1c8069e90 2023-10-30T12:49:20.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1ce7d3640 1 -- 172.21.15.12:0/561252616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c8143b60 con 0x7fb1c8069e90 2023-10-30T12:49:20.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.037+0000 7fb1ce7d3640 1 -- 172.21.15.12:0/561252616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c8145aa0 con 0x7fb1c8069e90 2023-10-30T12:49:20.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.041+0000 7fb1ce7d3640 1 -- 172.21.15.12:0/561252616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c804e170 con 0x7fb1c8069e90 2023-10-30T12:49:20.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.041+0000 7fb1c4ff9640 1 -- 172.21.15.12:0/561252616 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb1b8004b50 con 0x7fb1c8069e90 2023-10-30T12:49:20.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.041+0000 7fb1c4ff9640 1 --2- 172.21.15.12:0/561252616 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb1b4066470 0x7fb1b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.041+0000 7fb1c4ff9640 1 -- 172.21.15.12:0/561252616 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7fb1b80833c0 con 0x7fb1c8069e90 2023-10-30T12:49:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.041+0000 7fb1c77fe640 1 --2- 172.21.15.12:0/561252616 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb1b4066470 0x7fb1b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.041+0000 7fb1c77fe640 1 --2- 172.21.15.12:0/561252616 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb1b4066470 0x7fb1b4068930 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0ad000 tx=0x7fb1bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:20.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.045+0000 7fb1c4ff9640 1 -- 172.21.15.12:0/561252616 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb1b80505b0 con 0x7fb1c8069e90 2023-10-30T12:49:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.241+0000 7fb1ce7d3640 1 -- 172.21.15.12:0/561252616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1c806ef60 con 0x7fb1c8069e90 2023-10-30T12:49:20.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.245+0000 7fb1c4ff9640 1 -- 172.21.15.12:0/561252616 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6538 (secure 0 0 0) 0x7fb1b8018090 con 0x7fb1c8069e90 2023-10-30T12:49:20.251 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:20.251 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:04.349963+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:20.251 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2023-10-30T12:49:20.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.245+0000 7fb1ce7d3640 1 -- 172.21.15.12:0/561252616 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb1b4066470 msgr2=0x7fb1b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:20.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.245+0000 7fb1ce7d3640 1 --2- 172.21.15.12:0/561252616 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb1b4066470 0x7fb1b4068930 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0ad000 tx=0x7fb1bc008040 comp rx=0 tx=0).stop 2023-10-30T12:49:20.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.245+0000 7fb1ce7d3640 1 -- 172.21.15.12:0/561252616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1c8069e90 msgr2=0x7fb1c81489e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:20.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.245+0000 7fb1ce7d3640 1 --2- 172.21.15.12:0/561252616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1c8069e90 0x7fb1c81489e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800c520 tx=0x7fb1b800c9e0 comp rx=0 tx=0).stop 2023-10-30T12:49:20.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.245+0000 7fb1ce7d3640 1 -- 172.21.15.12:0/561252616 shutdown_connections 2023-10-30T12:49:20.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.245+0000 7fb1ce7d3640 1 --2- 172.21.15.12:0/561252616 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb1b4066470 0x7fb1b4068930 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.245+0000 7fb1ce7d3640 1 --2- 172.21.15.12:0/561252616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1c815a410 0x7fb1c81f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.245+0000 7fb1ce7d3640 1 --2- 172.21.15.12:0/561252616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1c806cb80 0x7fb1c81463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.245+0000 7fb1ce7d3640 1 --2- 172.21.15.12:0/561252616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1c8069e90 0x7fb1c81489e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.245+0000 7fb1ce7d3640 1 -- 172.21.15.12:0/561252616 >> 172.21.15.12:0/561252616 conn(0x7fb1c806dfd0 msgr2=0x7fb1c8150180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:20.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.245+0000 7fb1ce7d3640 1 -- 172.21.15.12:0/561252616 shutdown_connections 2023-10-30T12:49:20.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:20.245+0000 7fb1ce7d3640 1 -- 172.21.15.12:0/561252616 wait complete. 2023-10-30T12:49:20.272 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:49:22.273 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:22.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0b03f8640 1 Processor -- start 2023-10-30T12:49:22.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0b03f8640 1 -- start start 2023-10-30T12:49:22.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0b03f8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a80c6b80 0x7ff0a80bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:22.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0b03f8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff0a80c6f60 0x7ff0a80c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:22.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0b03f8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff0a80c8430 0x7ff0a813ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:22.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0b03f8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a80c0a50 con 0x7ff0a80c6f60 2023-10-30T12:49:22.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0b03f8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a806f6e0 con 0x7ff0a80c8430 2023-10-30T12:49:22.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0b03f8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a806f9c0 con 0x7ff0a80c6b80 2023-10-30T12:49:22.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0ad993640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff0a80c6f60 0x7ff0a80c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:22.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0ad993640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff0a80c6f60 0x7ff0a80c0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42434/0 (socket says 172.21.15.12:42434) 2023-10-30T12:49:22.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0ad993640 1 -- 172.21.15.12:0/1349110701 learned_addr learned my addr 172.21.15.12:0/1349110701 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:22.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0ae194640 1 --2- 172.21.15.12:0/1349110701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a80c6b80 0x7ff0a80bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:22.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0ae995640 1 --2- 172.21.15.12:0/1349110701 >> v2:172.21.15.154:3300/0 conn(0x7ff0a80c8430 0x7ff0a813ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:22.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0ad993640 1 -- 172.21.15.12:0/1349110701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a80c6b80 msgr2=0x7ff0a80bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:22.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0ad993640 1 --2- 172.21.15.12:0/1349110701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a80c6b80 0x7ff0a80bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0ad993640 1 -- 172.21.15.12:0/1349110701 >> v2:172.21.15.154:3300/0 conn(0x7ff0a80c8430 msgr2=0x7ff0a813ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:22.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0ad993640 1 --2- 172.21.15.12:0/1349110701 >> v2:172.21.15.154:3300/0 conn(0x7ff0a80c8430 0x7ff0a813ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.425+0000 7ff0ad993640 1 -- 172.21.15.12:0/1349110701 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a806fca0 con 0x7ff0a80c6f60 2023-10-30T12:49:22.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.429+0000 7ff0ad993640 1 --2- 172.21.15.12:0/1349110701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0a80c6f60 0x7ff0a80c0400 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80b0860 tx=0x7ff09801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dad44001de404de server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:22.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.429+0000 7ff0ad192640 1 -- 172.21.15.12:0/1349110701 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0980ac630 con 0x7ff0a80c6f60 2023-10-30T12:49:22.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.429+0000 7ff0ad192640 1 -- 172.21.15.12:0/1349110701 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0980ac790 con 0x7ff0a80c6f60 2023-10-30T12:49:22.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.429+0000 7ff0ad192640 1 -- 172.21.15.12:0/1349110701 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0980acb20 con 0x7ff0a80c6f60 2023-10-30T12:49:22.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.429+0000 7ff0b03f8640 1 -- 172.21.15.12:0/1349110701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0a80c6f60 msgr2=0x7ff0a80c0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:22.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.429+0000 7ff0b03f8640 1 --2- 172.21.15.12:0/1349110701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0a80c6f60 0x7ff0a80c0400 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80b0860 tx=0x7ff09801c500 comp rx=0 tx=0).stop 2023-10-30T12:49:22.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.429+0000 7ff0b03f8640 1 -- 172.21.15.12:0/1349110701 shutdown_connections 2023-10-30T12:49:22.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.429+0000 7ff0b03f8640 1 --2- 172.21.15.12:0/1349110701 >> v2:172.21.15.154:3300/0 conn(0x7ff0a80c8430 0x7ff0a813ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.429+0000 7ff0b03f8640 1 --2- 172.21.15.12:0/1349110701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0a80c6f60 0x7ff0a80c0400 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.429+0000 7ff0b03f8640 1 --2- 172.21.15.12:0/1349110701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a80c6b80 0x7ff0a80bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.429+0000 7ff0b03f8640 1 -- 172.21.15.12:0/1349110701 >> 172.21.15.12:0/1349110701 conn(0x7ff0a80c7910 msgr2=0x7ff0a80c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:22.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0b03f8640 1 -- 172.21.15.12:0/1349110701 shutdown_connections 2023-10-30T12:49:22.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0b03f8640 1 -- 172.21.15.12:0/1349110701 wait complete. 2023-10-30T12:49:22.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0b03f8640 1 Processor -- start 2023-10-30T12:49:22.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0b03f8640 1 -- start start 2023-10-30T12:49:22.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0b03f8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a80c6b80 0x7ff0a815fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:22.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0b03f8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0a80c6f60 0x7ff0a81576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:22.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0b03f8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a80c8430 0x7ff0a81380d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:22.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0b03f8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a81f2960 con 0x7ff0a80c6f60 2023-10-30T12:49:22.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0b03f8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a81f2090 con 0x7ff0a80c6b80 2023-10-30T12:49:22.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0b03f8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a815e170 con 0x7ff0a80c8430 2023-10-30T12:49:22.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0ad993640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0a80c6f60 0x7ff0a81576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:22.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0ad993640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0a80c6f60 0x7ff0a81576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42446/0 (socket says 172.21.15.12:42446) 2023-10-30T12:49:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0ad993640 1 -- 172.21.15.12:0/1152475733 learned_addr learned my addr 172.21.15.12:0/1152475733 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0ae995640 1 --2- 172.21.15.12:0/1152475733 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a80c8430 0x7ff0a81380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0ae194640 1 --2- 172.21.15.12:0/1152475733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a80c6b80 0x7ff0a815fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0ad993640 1 -- 172.21.15.12:0/1152475733 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a80c8430 msgr2=0x7ff0a81380d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0ad993640 1 --2- 172.21.15.12:0/1152475733 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a80c8430 0x7ff0a81380d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0ad993640 1 -- 172.21.15.12:0/1152475733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a80c6b80 msgr2=0x7ff0a815fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0ad993640 1 --2- 172.21.15.12:0/1152475733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a80c6b80 0x7ff0a815fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0ad993640 1 -- 172.21.15.12:0/1152475733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0980094d0 con 0x7ff0a80c6f60 2023-10-30T12:49:22.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.433+0000 7ff0ad993640 1 --2- 172.21.15.12:0/1152475733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0a80c6f60 0x7ff0a81576f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff09801abb0 tx=0x7ff0980192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:22.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.437+0000 7ff09effd640 1 -- 172.21.15.12:0/1152475733 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0980b5640 con 0x7ff0a80c6f60 2023-10-30T12:49:22.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.437+0000 7ff09effd640 1 -- 172.21.15.12:0/1152475733 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0980b57a0 con 0x7ff0a80c6f60 2023-10-30T12:49:22.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.437+0000 7ff09effd640 1 -- 172.21.15.12:0/1152475733 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0980b5b10 con 0x7ff0a80c6f60 2023-10-30T12:49:22.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.437+0000 7ff0b03f8640 1 -- 172.21.15.12:0/1152475733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a815e450 con 0x7ff0a80c6f60 2023-10-30T12:49:22.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.437+0000 7ff0b03f8640 1 -- 172.21.15.12:0/1152475733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a81f22a0 con 0x7ff0a80c6f60 2023-10-30T12:49:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.437+0000 7ff0b03f8640 1 -- 172.21.15.12:0/1152475733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a81f2aa0 con 0x7ff0a80c6f60 2023-10-30T12:49:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.437+0000 7ff09effd640 1 -- 172.21.15.12:0/1152475733 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff0980b3070 con 0x7ff0a80c6f60 2023-10-30T12:49:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.437+0000 7ff09effd640 1 --2- 172.21.15.12:0/1152475733 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff08c0664a0 0x7ff08c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.437+0000 7ff09effd640 1 -- 172.21.15.12:0/1152475733 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7ff09811e420 con 0x7ff0a80c6f60 2023-10-30T12:49:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.437+0000 7ff0ae194640 1 --2- 172.21.15.12:0/1152475733 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff08c0664a0 0x7ff08c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:22.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.441+0000 7ff0ae194640 1 --2- 172.21.15.12:0/1152475733 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff08c0664a0 0x7ff08c068960 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009a70 tx=0x7ff0a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:22.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.441+0000 7ff09effd640 1 -- 172.21.15.12:0/1152475733 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff0a81f2aa0 con 0x7ff0a80c6f60 2023-10-30T12:49:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.633+0000 7ff0b03f8640 1 -- 172.21.15.12:0/1152475733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0a81f2aa0 con 0x7ff0a80c6f60 2023-10-30T12:49:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.633+0000 7ff09effd640 1 -- 172.21.15.12:0/1152475733 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6538 (secure 0 0 0) 0x7ff0a81f2aa0 con 0x7ff0a80c6f60 2023-10-30T12:49:22.642 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:22.642 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:04.349963+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2023-10-30T12:49:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.637+0000 7ff0b03f8640 1 -- 172.21.15.12:0/1152475733 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff08c0664a0 msgr2=0x7ff08c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:22.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.637+0000 7ff0b03f8640 1 --2- 172.21.15.12:0/1152475733 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff08c0664a0 0x7ff08c068960 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009a70 tx=0x7ff0a4008040 comp rx=0 tx=0).stop 2023-10-30T12:49:22.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.637+0000 7ff0b03f8640 1 -- 172.21.15.12:0/1152475733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0a80c6f60 msgr2=0x7ff0a81576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:22.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.637+0000 7ff0b03f8640 1 --2- 172.21.15.12:0/1152475733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0a80c6f60 0x7ff0a81576f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff09801abb0 tx=0x7ff0980192a0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.637+0000 7ff0b03f8640 1 -- 172.21.15.12:0/1152475733 shutdown_connections 2023-10-30T12:49:22.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.637+0000 7ff0b03f8640 1 --2- 172.21.15.12:0/1152475733 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff08c0664a0 0x7ff08c068960 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.637+0000 7ff0b03f8640 1 --2- 172.21.15.12:0/1152475733 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a80c8430 0x7ff0a81380d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.637+0000 7ff0b03f8640 1 --2- 172.21.15.12:0/1152475733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0a80c6f60 0x7ff0a81576f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.637+0000 7ff0b03f8640 1 --2- 172.21.15.12:0/1152475733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a80c6b80 0x7ff0a815fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.637+0000 7ff0b03f8640 1 -- 172.21.15.12:0/1152475733 >> 172.21.15.12:0/1152475733 conn(0x7ff0a80c7910 msgr2=0x7ff0a80c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:22.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.637+0000 7ff0b03f8640 1 -- 172.21.15.12:0/1152475733 shutdown_connections 2023-10-30T12:49:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:22.637+0000 7ff0b03f8640 1 -- 172.21.15.12:0/1152475733 wait complete. 2023-10-30T12:49:22.663 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:49:24.664 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:24.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.809+0000 7f04f14c9640 1 Processor -- start 2023-10-30T12:49:24.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.809+0000 7f04f14c9640 1 -- start start 2023-10-30T12:49:24.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.813+0000 7f04f14c9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f04ec0c6b80 0x7f04ec0bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:24.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.813+0000 7f04f14c9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04ec0c6f60 0x7f04ec0c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.813+0000 7f04f14c9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ec0c0a50 con 0x7f04ec0c8430 2023-10-30T12:49:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.813+0000 7f04f14c9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04ec06f6e0 con 0x7f04ec0c6b80 2023-10-30T12:49:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.813+0000 7f04f14c9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec06f9c0 con 0x7f04ec0c6f60 2023-10-30T12:49:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.813+0000 7f04eaffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f04ec0c6b80 0x7f04ec0bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.813+0000 7f04eaffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f04ec0c6b80 0x7f04ec0bfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42460/0 (socket says 172.21.15.12:42460) 2023-10-30T12:49:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.813+0000 7f04eaffd640 1 -- 172.21.15.12:0/913055725 learned_addr learned my addr 172.21.15.12:0/913055725 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.813+0000 7f04e3fff640 1 --2- 172.21.15.12:0/913055725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04ec0c6f60 0x7f04ec0c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:24.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.813+0000 7f04eaffd640 1 -- 172.21.15.12:0/913055725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04ec0c6f60 msgr2=0x7f04ec0c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:24.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.813+0000 7f04eaffd640 1 --2- 172.21.15.12:0/913055725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04ec0c6f60 0x7f04ec0c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.813+0000 7f04eaffd640 1 -- 172.21.15.12:0/913055725 >> v1:172.21.15.154:6789/0 conn(0x7f04ec0c8430 legacy=0x7f04ec13ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:24.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.813+0000 7f04eaffd640 1 -- 172.21.15.12:0/913055725 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ec06fca0 con 0x7f04ec0c6b80 2023-10-30T12:49:24.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.817+0000 7f04eaffd640 1 --2- 172.21.15.12:0/913055725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04ec0c6b80 0x7f04ec0bfec0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009870 tx=0x7f04dc01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c9d76e89992edd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:24.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.817+0000 7f04ea7fc640 1 -- 172.21.15.12:0/913055725 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc0ac620 con 0x7f04ec0c6b80 2023-10-30T12:49:24.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.817+0000 7f04ea7fc640 1 -- 172.21.15.12:0/913055725 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04dc0ac780 con 0x7f04ec0c6b80 2023-10-30T12:49:24.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.817+0000 7f04ea7fc640 1 -- 172.21.15.12:0/913055725 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc0acb10 con 0x7f04ec0c6b80 2023-10-30T12:49:24.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.817+0000 7f04f14c9640 1 -- 172.21.15.12:0/913055725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04ec0c6b80 msgr2=0x7f04ec0bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:24.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.817+0000 7f04f14c9640 1 --2- 172.21.15.12:0/913055725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04ec0c6b80 0x7f04ec0bfec0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009870 tx=0x7f04dc01c4b0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.817+0000 7f04f14c9640 1 -- 172.21.15.12:0/913055725 shutdown_connections 2023-10-30T12:49:24.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.817+0000 7f04f14c9640 1 --2- 172.21.15.12:0/913055725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04ec0c6f60 0x7f04ec0c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.817+0000 7f04f14c9640 1 --2- 172.21.15.12:0/913055725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04ec0c6b80 0x7f04ec0bfec0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.817+0000 7f04f14c9640 1 -- 172.21.15.12:0/913055725 >> 172.21.15.12:0/913055725 conn(0x7f04ec0c7910 msgr2=0x7f04ec0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:24.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.817+0000 7f04f14c9640 1 -- 172.21.15.12:0/913055725 shutdown_connections 2023-10-30T12:49:24.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.817+0000 7f04f14c9640 1 -- 172.21.15.12:0/913055725 wait complete. 2023-10-30T12:49:24.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.817+0000 7f04f14c9640 1 Processor -- start 2023-10-30T12:49:24.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.817+0000 7f04f14c9640 1 -- start start 2023-10-30T12:49:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04f14c9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04ec0c6b80 0x7f04ec0c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04f14c9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04ec0c6f60 0x7f04ec1556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04f14c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04ec0c8430 0x7f04ec155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04f14c9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec13c370 con 0x7f04ec0c6f60 2023-10-30T12:49:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04f14c9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec1588f0 con 0x7f04ec0c8430 2023-10-30T12:49:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04f14c9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec0ca490 con 0x7f04ec0c6b80 2023-10-30T12:49:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04e3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04ec0c6f60 0x7f04ec1556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04e3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04ec0c6f60 0x7f04ec1556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42474/0 (socket says 172.21.15.12:42474) 2023-10-30T12:49:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04e3fff640 1 -- 172.21.15.12:0/3764268192 learned_addr learned my addr 172.21.15.12:0/3764268192 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04eaffd640 1 --2- 172.21.15.12:0/3764268192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04ec0c6b80 0x7f04ec0c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04eb7fe640 1 --2- 172.21.15.12:0/3764268192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04ec0c8430 0x7f04ec155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04e3fff640 1 -- 172.21.15.12:0/3764268192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04ec0c6b80 msgr2=0x7f04ec0c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04e3fff640 1 --2- 172.21.15.12:0/3764268192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04ec0c6b80 0x7f04ec0c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04e3fff640 1 -- 172.21.15.12:0/3764268192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04ec0c8430 msgr2=0x7f04ec155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04e3fff640 1 --2- 172.21.15.12:0/3764268192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04ec0c8430 0x7f04ec155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04e3fff640 1 -- 172.21.15.12:0/3764268192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04dc0094d0 con 0x7f04ec0c6f60 2023-10-30T12:49:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04e3fff640 1 --2- 172.21.15.12:0/3764268192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04ec0c6f60 0x7f04ec1556a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f04d400d800 tx=0x7f04d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04e37fe640 1 -- 172.21.15.12:0/3764268192 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d400b6c0 con 0x7f04ec0c6f60 2023-10-30T12:49:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04e37fe640 1 -- 172.21.15.12:0/3764268192 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d400b820 con 0x7f04ec0c6f60 2023-10-30T12:49:24.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04e37fe640 1 -- 172.21.15.12:0/3764268192 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d400bb90 con 0x7f04ec0c6f60 2023-10-30T12:49:24.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04f14c9640 1 -- 172.21.15.12:0/3764268192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ec158b20 con 0x7f04ec0c6f60 2023-10-30T12:49:24.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04f14c9640 1 -- 172.21.15.12:0/3764268192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ec0c8c70 con 0x7f04ec0c6f60 2023-10-30T12:49:24.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04f14c9640 1 -- 172.21.15.12:0/3764268192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04ec04e170 con 0x7f04ec0c6f60 2023-10-30T12:49:24.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.821+0000 7f04e37fe640 1 -- 172.21.15.12:0/3764268192 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f04d400bcf0 con 0x7f04ec0c6f60 2023-10-30T12:49:24.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.825+0000 7f04e37fe640 1 --2- 172.21.15.12:0/3764268192 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f04cc0664a0 0x7f04cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:24.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.825+0000 7f04e37fe640 1 -- 172.21.15.12:0/3764268192 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f04d40843a0 con 0x7f04ec0c6f60 2023-10-30T12:49:24.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.825+0000 7f04eaffd640 1 --2- 172.21.15.12:0/3764268192 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f04cc0664a0 0x7f04cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:24.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.825+0000 7f04eaffd640 1 --2- 172.21.15.12:0/3764268192 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f04cc0664a0 0x7f04cc068960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f04dc0ae000 tx=0x7f04dc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:24.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:24.829+0000 7f04e37fe640 1 -- 172.21.15.12:0/3764268192 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f04d40515c0 con 0x7f04ec0c6f60 2023-10-30T12:49:25.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:25.017+0000 7f04f14c9640 1 -- 172.21.15.12:0/3764268192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04ec0c8c70 con 0x7f04ec0c6f60 2023-10-30T12:49:25.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:25.021+0000 7f04e37fe640 1 -- 172.21.15.12:0/3764268192 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6538 (secure 0 0 0) 0x7f04ec0c8c70 con 0x7f04ec0c6f60 2023-10-30T12:49:25.027 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:25.027 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:04.349963+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:25.027 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2023-10-30T12:49:25.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:25.021+0000 7f04f14c9640 1 -- 172.21.15.12:0/3764268192 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f04cc0664a0 msgr2=0x7f04cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:25.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:25.021+0000 7f04f14c9640 1 --2- 172.21.15.12:0/3764268192 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f04cc0664a0 0x7f04cc068960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f04dc0ae000 tx=0x7f04dc0a7000 comp rx=0 tx=0).stop 2023-10-30T12:49:25.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:25.021+0000 7f04f14c9640 1 -- 172.21.15.12:0/3764268192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04ec0c6f60 msgr2=0x7f04ec1556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:25.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:25.021+0000 7f04f14c9640 1 --2- 172.21.15.12:0/3764268192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04ec0c6f60 0x7f04ec1556a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f04d400d800 tx=0x7f04d400dcc0 comp rx=0 tx=0).stop 2023-10-30T12:49:25.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:25.021+0000 7f04f14c9640 1 -- 172.21.15.12:0/3764268192 shutdown_connections 2023-10-30T12:49:25.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:25.021+0000 7f04f14c9640 1 --2- 172.21.15.12:0/3764268192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04ec0c8430 0x7f04ec155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:25.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:25.021+0000 7f04f14c9640 1 --2- 172.21.15.12:0/3764268192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04ec0c6f60 0x7f04ec1556a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:25.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:25.021+0000 7f04f14c9640 1 --2- 172.21.15.12:0/3764268192 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f04cc0664a0 0x7f04cc068960 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:25.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:25.021+0000 7f04f14c9640 1 --2- 172.21.15.12:0/3764268192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04ec0c6b80 0x7f04ec0c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:25.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:25.021+0000 7f04f14c9640 1 -- 172.21.15.12:0/3764268192 >> 172.21.15.12:0/3764268192 conn(0x7f04ec0c7910 msgr2=0x7f04ec0c0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:25.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:25.021+0000 7f04f14c9640 1 -- 172.21.15.12:0/3764268192 shutdown_connections 2023-10-30T12:49:25.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:25.021+0000 7f04f14c9640 1 -- 172.21.15.12:0/3764268192 wait complete. 2023-10-30T12:49:25.048 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:49:27.049 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:27.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f51a1b0f640 1 Processor -- start 2023-10-30T12:49:27.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f51a1b0f640 1 -- start start 2023-10-30T12:49:27.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f51a1b0f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f519c0c6b80 0x7f519c0bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:27.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f51a1b0f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f519c13f4a0 0x7f519c0c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:27.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f51a1b0f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519c0c0a50 con 0x7f519c0c6f60 2023-10-30T12:49:27.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f51a1b0f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c06f6e0 con 0x7f519c0c6b80 2023-10-30T12:49:27.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f51a1b0f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c06f9c0 con 0x7f519c13f4a0 2023-10-30T12:49:27.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f519b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f519c0c6b80 0x7f519c0bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f519b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f519c0c6b80 0x7f519c0bfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42482/0 (socket says 172.21.15.12:42482) 2023-10-30T12:49:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f519b7fe640 1 -- 172.21.15.12:0/3098176064 learned_addr learned my addr 172.21.15.12:0/3098176064 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f519bfff640 1 --2- 172.21.15.12:0/3098176064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f519c13f4a0 0x7f519c0c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f519b7fe640 1 -- 172.21.15.12:0/3098176064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f519c13f4a0 msgr2=0x7f519c0c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f519b7fe640 1 --2- 172.21.15.12:0/3098176064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f519c13f4a0 0x7f519c0c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f519b7fe640 1 -- 172.21.15.12:0/3098176064 >> v1:172.21.15.154:6789/0 conn(0x7f519c0c6f60 legacy=0x7f519c13ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f519b7fe640 1 -- 172.21.15.12:0/3098176064 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c06fca0 con 0x7f519c0c6b80 2023-10-30T12:49:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.197+0000 7f519bfff640 1 --2- 172.21.15.12:0/3098176064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f519c13f4a0 0x7f519c0c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:49:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.201+0000 7f519b7fe640 1 --2- 172.21.15.12:0/3098176064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f519c0c6b80 0x7f519c0bfec0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f518c009870 tx=0x7f518c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafb4b2c841f4388 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.201+0000 7f519a7fc640 1 -- 172.21.15.12:0/3098176064 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0ac630 con 0x7f519c0c6b80 2023-10-30T12:49:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.201+0000 7f519a7fc640 1 -- 172.21.15.12:0/3098176064 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c0ac790 con 0x7f519c0c6b80 2023-10-30T12:49:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.201+0000 7f519a7fc640 1 -- 172.21.15.12:0/3098176064 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0acb20 con 0x7f519c0c6b80 2023-10-30T12:49:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.201+0000 7f51a1b0f640 1 -- 172.21.15.12:0/3098176064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f519c0c6b80 msgr2=0x7f519c0bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.201+0000 7f51a1b0f640 1 --2- 172.21.15.12:0/3098176064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f519c0c6b80 0x7f519c0bfec0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f518c009870 tx=0x7f518c01c500 comp rx=0 tx=0).stop 2023-10-30T12:49:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.201+0000 7f51a1b0f640 1 -- 172.21.15.12:0/3098176064 shutdown_connections 2023-10-30T12:49:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.201+0000 7f51a1b0f640 1 --2- 172.21.15.12:0/3098176064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f519c13f4a0 0x7f519c0c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.201+0000 7f51a1b0f640 1 --2- 172.21.15.12:0/3098176064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f519c0c6b80 0x7f519c0bfec0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.201+0000 7f51a1b0f640 1 -- 172.21.15.12:0/3098176064 >> 172.21.15.12:0/3098176064 conn(0x7f519c0c7910 msgr2=0x7f519c0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.201+0000 7f51a1b0f640 1 -- 172.21.15.12:0/3098176064 shutdown_connections 2023-10-30T12:49:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.201+0000 7f51a1b0f640 1 -- 172.21.15.12:0/3098176064 wait complete. 2023-10-30T12:49:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f51a1b0f640 1 Processor -- start 2023-10-30T12:49:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f51a1b0f640 1 -- start start 2023-10-30T12:49:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f51a1b0f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f519c0c6b80 0x7f519c13bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f51a1b0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f519c0c6f60 0x7f519c13c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f51a1b0f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f519c13f4a0 0x7f519c157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f51a1b0f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0ca080 con 0x7f519c0c6f60 2023-10-30T12:49:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f51a1b0f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c13dc70 con 0x7f519c13f4a0 2023-10-30T12:49:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f51a1b0f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c13d990 con 0x7f519c0c6b80 2023-10-30T12:49:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f519affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f519c0c6f60 0x7f519c13c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f519affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f519c0c6f60 0x7f519c13c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42488/0 (socket says 172.21.15.12:42488) 2023-10-30T12:49:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f519affd640 1 -- 172.21.15.12:0/235175623 learned_addr learned my addr 172.21.15.12:0/235175623 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f519b7fe640 1 --2- 172.21.15.12:0/235175623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f519c0c6b80 0x7f519c13bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f519bfff640 1 --2- 172.21.15.12:0/235175623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f519c13f4a0 0x7f519c157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f519affd640 1 -- 172.21.15.12:0/235175623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f519c0c6b80 msgr2=0x7f519c13bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f519affd640 1 --2- 172.21.15.12:0/235175623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f519c0c6b80 0x7f519c13bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f519affd640 1 -- 172.21.15.12:0/235175623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f519c13f4a0 msgr2=0x7f519c157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f519affd640 1 --2- 172.21.15.12:0/235175623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f519c13f4a0 0x7f519c157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f519affd640 1 -- 172.21.15.12:0/235175623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c0094d0 con 0x7f519c0c6f60 2023-10-30T12:49:27.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f519b7fe640 1 --2- 172.21.15.12:0/235175623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f519c0c6b80 0x7f519c13bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:27.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f519affd640 1 --2- 172.21.15.12:0/235175623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f519c0c6f60 0x7f519c13c4b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f519000c450 tx=0x7f519000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:27.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f5183fff640 1 -- 172.21.15.12:0/235175623 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190012830 con 0x7f519c0c6f60 2023-10-30T12:49:27.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f5183fff640 1 -- 172.21.15.12:0/235175623 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51900129b0 con 0x7f519c0c6f60 2023-10-30T12:49:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f5183fff640 1 -- 172.21.15.12:0/235175623 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190012d60 con 0x7f519c0c6f60 2023-10-30T12:49:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f51a1b0f640 1 -- 172.21.15.12:0/235175623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519c0c9750 con 0x7f519c0c6f60 2023-10-30T12:49:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f51a1b0f640 1 -- 172.21.15.12:0/235175623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c0c9d10 con 0x7f519c0c6f60 2023-10-30T12:49:27.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.205+0000 7f51a1b0f640 1 -- 172.21.15.12:0/235175623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519c0ca1f0 con 0x7f519c0c6f60 2023-10-30T12:49:27.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.209+0000 7f5183fff640 1 -- 172.21.15.12:0/235175623 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f519001c460 con 0x7f519c0c6f60 2023-10-30T12:49:27.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.209+0000 7f5183fff640 1 --2- 172.21.15.12:0/235175623 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51880664a0 0x7f5188068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:27.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.209+0000 7f5183fff640 1 -- 172.21.15.12:0/235175623 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f5190083fc0 con 0x7f519c0c6f60 2023-10-30T12:49:27.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.209+0000 7f519b7fe640 1 --2- 172.21.15.12:0/235175623 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51880664a0 0x7f5188068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:27.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.209+0000 7f519b7fe640 1 --2- 172.21.15.12:0/235175623 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51880664a0 0x7f5188068960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f518c0ae000 tx=0x7f518c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:27.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.213+0000 7f5183fff640 1 -- 172.21.15.12:0/235175623 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5190051120 con 0x7f519c0c6f60 2023-10-30T12:49:27.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.409+0000 7f51a1b0f640 1 -- 172.21.15.12:0/235175623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f519c04e170 con 0x7f519c0c6f60 2023-10-30T12:49:27.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.409+0000 7f5183fff640 1 -- 172.21.15.12:0/235175623 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6538 (secure 0 0 0) 0x7f5190054740 con 0x7f519c0c6f60 2023-10-30T12:49:27.415 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:27.415 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:04.349963+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:27.416 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2023-10-30T12:49:27.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.409+0000 7f51a1b0f640 1 -- 172.21.15.12:0/235175623 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51880664a0 msgr2=0x7f5188068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:27.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.409+0000 7f51a1b0f640 1 --2- 172.21.15.12:0/235175623 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51880664a0 0x7f5188068960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f518c0ae000 tx=0x7f518c008040 comp rx=0 tx=0).stop 2023-10-30T12:49:27.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.409+0000 7f51a1b0f640 1 -- 172.21.15.12:0/235175623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f519c0c6f60 msgr2=0x7f519c13c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:27.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.409+0000 7f51a1b0f640 1 --2- 172.21.15.12:0/235175623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f519c0c6f60 0x7f519c13c4b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f519000c450 tx=0x7f519000c910 comp rx=0 tx=0).stop 2023-10-30T12:49:27.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.409+0000 7f51a1b0f640 1 -- 172.21.15.12:0/235175623 shutdown_connections 2023-10-30T12:49:27.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.409+0000 7f51a1b0f640 1 --2- 172.21.15.12:0/235175623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f519c13f4a0 0x7f519c157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.409+0000 7f51a1b0f640 1 --2- 172.21.15.12:0/235175623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f519c0c6f60 0x7f519c13c4b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.409+0000 7f51a1b0f640 1 --2- 172.21.15.12:0/235175623 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51880664a0 0x7f5188068960 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.409+0000 7f51a1b0f640 1 --2- 172.21.15.12:0/235175623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f519c0c6b80 0x7f519c13bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.409+0000 7f51a1b0f640 1 -- 172.21.15.12:0/235175623 >> 172.21.15.12:0/235175623 conn(0x7f519c0c7910 msgr2=0x7f519c0c1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:27.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.409+0000 7f51a1b0f640 1 -- 172.21.15.12:0/235175623 shutdown_connections 2023-10-30T12:49:27.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:27.409+0000 7f51a1b0f640 1 -- 172.21.15.12:0/235175623 wait complete. 2023-10-30T12:49:27.436 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:49:29.438 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:29.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc501194640 1 Processor -- start 2023-10-30T12:49:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc501194640 1 -- start start 2023-10-30T12:49:29.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc501194640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc4fc1538a0 0x7fc4fc15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc501194640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4fc0ca0e0 0x7fc4fc15a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc501194640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4fc0cb800 con 0x7fc4fc0cf160 2023-10-30T12:49:29.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc501194640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4fc15dc30 con 0x7fc4fc1538a0 2023-10-30T12:49:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc501194640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc0cb330 con 0x7fc4fc0ca0e0 2023-10-30T12:49:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc4fb59e640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc4fc0cf160 0x7fc4fc0cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37460/0 (socket says 172.21.15.12:37460) 2023-10-30T12:49:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc4fb59e640 1 -- 172.21.15.12:0/3537775190 learned_addr learned my addr 172.21.15.12:0/3537775190 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:49:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc4fad9d640 1 --2- 172.21.15.12:0/3537775190 >> v2:172.21.15.154:3300/0 conn(0x7fc4fc1538a0 0x7fc4fc15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc4f3fff640 1 --2- 172.21.15.12:0/3537775190 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4fc0ca0e0 0x7fc4fc15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc4fa59c640 1 -- 172.21.15.12:0/3537775190 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908268577 0 0) 0x7fc4fc0cb800 con 0x7fc4fc0cf160 2023-10-30T12:49:29.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc4fa59c640 1 -- 172.21.15.12:0/3537775190 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4e8003530 con 0x7fc4fc0cf160 2023-10-30T12:49:29.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc4fa59c640 1 -- 172.21.15.12:0/3537775190 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2038748431 0 0) 0x7fc4e8003530 con 0x7fc4fc0cf160 2023-10-30T12:49:29.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc4fa59c640 1 -- 172.21.15.12:0/3537775190 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4fc0ca0e0 msgr2=0x7fc4fc15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:29.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc4fa59c640 1 --2- 172.21.15.12:0/3537775190 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4fc0ca0e0 0x7fc4fc15a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc4fa59c640 1 -- 172.21.15.12:0/3537775190 >> v2:172.21.15.154:3300/0 conn(0x7fc4fc1538a0 msgr2=0x7fc4fc15d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:29.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc4fa59c640 1 --2- 172.21.15.12:0/3537775190 >> v2:172.21.15.154:3300/0 conn(0x7fc4fc1538a0 0x7fc4fc15d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc4fa59c640 1 -- 172.21.15.12:0/3537775190 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc0cad70 con 0x7fc4fc0cf160 2023-10-30T12:49:29.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc4fad9d640 1 --2- 172.21.15.12:0/3537775190 >> v2:172.21.15.154:3300/0 conn(0x7fc4fc1538a0 0x7fc4fc15d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:49:29.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.585+0000 7fc4fa59c640 1 -- 172.21.15.12:0/3537775190 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fc4f4001a00 con 0x7fc4fc0cf160 2023-10-30T12:49:29.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4fa59c640 1 -- 172.21.15.12:0/3537775190 >> v1:172.21.15.12:6789/0 conn(0x7fc4fc0cf160 legacy=0x7fc4fc0cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:29.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4fa59c640 1 --2- 172.21.15.12:0/3537775190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4e8003980 0x7fc4e8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4fa59c640 1 --2- 172.21.15.12:0/3537775190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4e80047d0 0x7fc4e8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4fa59c640 1 --2- 172.21.15.12:0/3537775190 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4fc0ca0e0 0x7fc4e8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4fa59c640 1 -- 172.21.15.12:0/3537775190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc15dc30 con 0x7fc4e80047d0 2023-10-30T12:49:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4fa59c640 1 -- 172.21.15.12:0/3537775190 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc0cb330 con 0x7fc4e8003980 2023-10-30T12:49:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4fa59c640 1 -- 172.21.15.12:0/3537775190 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e800b9c0 con 0x7fc4fc0ca0e0 2023-10-30T12:49:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4f3fff640 1 --2- 172.21.15.12:0/3537775190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4e80047d0 0x7fc4e8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4fad9d640 1 --2- 172.21.15.12:0/3537775190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4e8003980 0x7fc4e8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4fb59e640 1 --2- 172.21.15.12:0/3537775190 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4fc0ca0e0 0x7fc4e8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4f3fff640 1 -- 172.21.15.12:0/3537775190 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4fc0ca0e0 msgr2=0x7fc4e8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4f3fff640 1 --2- 172.21.15.12:0/3537775190 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4fc0ca0e0 0x7fc4e8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4f3fff640 1 -- 172.21.15.12:0/3537775190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4e8003980 msgr2=0x7fc4e8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:29.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4f3fff640 1 --2- 172.21.15.12:0/3537775190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4e8003980 0x7fc4e8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4f3fff640 1 -- 172.21.15.12:0/3537775190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e8003530 con 0x7fc4e80047d0 2023-10-30T12:49:29.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4f3fff640 1 --2- 172.21.15.12:0/3537775190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4e80047d0 0x7fc4e8004c50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009ae0 tx=0x7fc4e401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788437cafec6c58b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:29.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4fa59c640 1 -- 172.21.15.12:0/3537775190 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40a5560 con 0x7fc4e80047d0 2023-10-30T12:49:29.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4fa59c640 1 -- 172.21.15.12:0/3537775190 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e40a56c0 con 0x7fc4e80047d0 2023-10-30T12:49:29.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.589+0000 7fc4fa59c640 1 -- 172.21.15.12:0/3537775190 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40a5a70 con 0x7fc4e80047d0 2023-10-30T12:49:29.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc501194640 1 -- 172.21.15.12:0/3537775190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4e80047d0 msgr2=0x7fc4e8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:29.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc501194640 1 --2- 172.21.15.12:0/3537775190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4e80047d0 0x7fc4e8004c50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009ae0 tx=0x7fc4e401ca50 comp rx=0 tx=0).stop 2023-10-30T12:49:29.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc4fb59e640 1 -- 172.21.15.12:0/3537775190 reap_dead start 2023-10-30T12:49:29.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc501194640 1 -- 172.21.15.12:0/3537775190 shutdown_connections 2023-10-30T12:49:29.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc501194640 1 -- 172.21.15.12:0/3537775190 >> 172.21.15.12:0/3537775190 conn(0x7fc4fc06bfe0 msgr2=0x7fc4fc06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:29.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc501194640 1 -- 172.21.15.12:0/3537775190 shutdown_connections 2023-10-30T12:49:29.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc501194640 1 -- 172.21.15.12:0/3537775190 wait complete. 2023-10-30T12:49:29.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc501194640 1 Processor -- start 2023-10-30T12:49:29.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc501194640 1 -- start start 2023-10-30T12:49:29.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc501194640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4e80047d0 0x7fc4fc150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc501194640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4fc0b1960 0x7fc4fc15d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc501194640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4fc143f70 0x7fc4fc145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc501194640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc15d3f0 con 0x7fc4fc0b1960 2023-10-30T12:49:29.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc501194640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc15cee0 con 0x7fc4fc143f70 2023-10-30T12:49:29.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc501194640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc1454f0 con 0x7fc4e80047d0 2023-10-30T12:49:29.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc4f3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4fc0b1960 0x7fc4fc15d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc4f3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4fc0b1960 0x7fc4fc15d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42512/0 (socket says 172.21.15.12:42512) 2023-10-30T12:49:29.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc4f3fff640 1 -- 172.21.15.12:0/2633418452 learned_addr learned my addr 172.21.15.12:0/2633418452 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:29.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.593+0000 7fc4fad9d640 1 --2- 172.21.15.12:0/2633418452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4e80047d0 0x7fc4fc150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc4fb59e640 1 --2- 172.21.15.12:0/2633418452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4fc143f70 0x7fc4fc145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc4f3fff640 1 -- 172.21.15.12:0/2633418452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4e80047d0 msgr2=0x7fc4fc150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:29.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc4f3fff640 1 --2- 172.21.15.12:0/2633418452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4e80047d0 0x7fc4fc150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc4f3fff640 1 -- 172.21.15.12:0/2633418452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4fc143f70 msgr2=0x7fc4fc145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:29.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc4f3fff640 1 --2- 172.21.15.12:0/2633418452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4fc143f70 0x7fc4fc145980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc4f3fff640 1 -- 172.21.15.12:0/2633418452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc1451c0 con 0x7fc4fc0b1960 2023-10-30T12:49:29.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc4f3fff640 1 --2- 172.21.15.12:0/2633418452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4fc0b1960 0x7fc4fc15d680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc4e401c880 tx=0x7fc4e8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:29.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc4f37fe640 1 -- 172.21.15.12:0/2633418452 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40bc070 con 0x7fc4fc0b1960 2023-10-30T12:49:29.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc4f37fe640 1 -- 172.21.15.12:0/2633418452 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e400f0c0 con 0x7fc4fc0b1960 2023-10-30T12:49:29.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc4f37fe640 1 -- 172.21.15.12:0/2633418452 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4004220 con 0x7fc4fc0b1960 2023-10-30T12:49:29.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc501194640 1 -- 172.21.15.12:0/2633418452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4fc15d110 con 0x7fc4fc0b1960 2023-10-30T12:49:29.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc501194640 1 -- 172.21.15.12:0/2633418452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4fc1efd20 con 0x7fc4fc0b1960 2023-10-30T12:49:29.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc501194640 1 -- 172.21.15.12:0/2633418452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4fc145660 con 0x7fc4fc0b1960 2023-10-30T12:49:29.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc4f37fe640 1 -- 172.21.15.12:0/2633418452 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc4e40b4040 con 0x7fc4fc0b1960 2023-10-30T12:49:29.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc4f37fe640 1 --2- 172.21.15.12:0/2633418452 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4dc0664a0 0x7fc4dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc4f37fe640 1 -- 172.21.15.12:0/2633418452 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7fc4e411df30 con 0x7fc4fc0b1960 2023-10-30T12:49:29.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.597+0000 7fc4fad9d640 1 --2- 172.21.15.12:0/2633418452 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4dc0664a0 0x7fc4dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.601+0000 7fc4fad9d640 1 --2- 172.21.15.12:0/2633418452 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4dc0664a0 0x7fc4dc068960 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec009d30 tx=0x7fc4ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:29.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.601+0000 7fc4f37fe640 1 -- 172.21.15.12:0/2633418452 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc4e40eb1b0 con 0x7fc4fc0b1960 2023-10-30T12:49:29.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.789+0000 7fc501194640 1 -- 172.21.15.12:0/2633418452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4fc04e170 con 0x7fc4fc0b1960 2023-10-30T12:49:29.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.789+0000 7fc4f37fe640 1 -- 172.21.15.12:0/2633418452 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6538 (secure 0 0 0) 0x7fc4e40ee7d0 con 0x7fc4fc0b1960 2023-10-30T12:49:29.798 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:29.799 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:04.349963+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:29.800 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2023-10-30T12:49:29.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.793+0000 7fc501194640 1 -- 172.21.15.12:0/2633418452 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4dc0664a0 msgr2=0x7fc4dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:29.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.793+0000 7fc501194640 1 --2- 172.21.15.12:0/2633418452 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4dc0664a0 0x7fc4dc068960 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec009d30 tx=0x7fc4ec008040 comp rx=0 tx=0).stop 2023-10-30T12:49:29.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.793+0000 7fc501194640 1 -- 172.21.15.12:0/2633418452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4fc0b1960 msgr2=0x7fc4fc15d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:29.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.793+0000 7fc501194640 1 --2- 172.21.15.12:0/2633418452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4fc0b1960 0x7fc4fc15d680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc4e401c880 tx=0x7fc4e8005800 comp rx=0 tx=0).stop 2023-10-30T12:49:29.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.793+0000 7fc501194640 1 -- 172.21.15.12:0/2633418452 shutdown_connections 2023-10-30T12:49:29.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.793+0000 7fc501194640 1 --2- 172.21.15.12:0/2633418452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4fc143f70 0x7fc4fc145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.793+0000 7fc501194640 1 --2- 172.21.15.12:0/2633418452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4fc0b1960 0x7fc4fc15d680 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.793+0000 7fc501194640 1 --2- 172.21.15.12:0/2633418452 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc4dc0664a0 0x7fc4dc068960 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.793+0000 7fc501194640 1 --2- 172.21.15.12:0/2633418452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4e80047d0 0x7fc4fc150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.793+0000 7fc501194640 1 -- 172.21.15.12:0/2633418452 >> 172.21.15.12:0/2633418452 conn(0x7fc4fc06bfe0 msgr2=0x7fc4fc065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:29.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.793+0000 7fc501194640 1 -- 172.21.15.12:0/2633418452 shutdown_connections 2023-10-30T12:49:29.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:29.793+0000 7fc501194640 1 -- 172.21.15.12:0/2633418452 wait complete. 2023-10-30T12:49:29.819 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:49:31.821 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:31.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.965+0000 7f97e7ed6640 1 Processor -- start 2023-10-30T12:49:31.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.965+0000 7f97e7ed6640 1 -- start start 2023-10-30T12:49:31.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e7ed6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97e0070740 0x7f97e00c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:31.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e7ed6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e00c5810 con 0x7f97e01580e0 2023-10-30T12:49:31.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e7ed6640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e015eb90 con 0x7f97e015e700 2023-10-30T12:49:31.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e7ed6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e006acb0 con 0x7f97e0070740 2023-10-30T12:49:31.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e5c72640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f97e01580e0 0x7f97e0069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39692/0 (socket says 172.21.15.12:39692) 2023-10-30T12:49:31.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e5c72640 1 -- 172.21.15.12:0/193954080 learned_addr learned my addr 172.21.15.12:0/193954080 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:49:31.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e5471640 1 --2- 172.21.15.12:0/193954080 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97e0070740 0x7f97e00c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:31.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660949981 0 0) 0x7f97e00c5810 con 0x7f97e01580e0 2023-10-30T12:49:31.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97cc003530 con 0x7f97e01580e0 2023-10-30T12:49:31.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163513100 0 0) 0x7f97e015eb90 con 0x7f97e015e700 2023-10-30T12:49:31.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97e00c5810 con 0x7f97e015e700 2023-10-30T12:49:31.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 271854957 0 0) 0x7f97cc003530 con 0x7f97e01580e0 2023-10-30T12:49:31.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97e0070740 msgr2=0x7f97e00c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:31.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 --2- 172.21.15.12:0/193954080 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97e0070740 0x7f97e00c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:31.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 >> v1:172.21.15.154:6789/0 conn(0x7f97e015e700 legacy=0x7f97e006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:31.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e00c6bb0 con 0x7f97e01580e0 2023-10-30T12:49:31.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f97dc001680 con 0x7f97e01580e0 2023-10-30T12:49:31.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 >> v1:172.21.15.12:6789/0 conn(0x7f97e01580e0 legacy=0x7f97e0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:31.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 --2- 172.21.15.12:0/193954080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97cc003b40 0x7f97cc003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:31.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 --2- 172.21.15.12:0/193954080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97cc0047d0 0x7f97cc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:31.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 --2- 172.21.15.12:0/193954080 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97e0070740 0x7f97cc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:31.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc00ba40 con 0x7f97cc003b40 2023-10-30T12:49:31.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc00bb80 con 0x7f97cc0047d0 2023-10-30T12:49:31.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc00bd40 con 0x7f97e0070740 2023-10-30T12:49:31.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e5c72640 1 --2- 172.21.15.12:0/193954080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97cc003b40 0x7f97cc003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:31.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e5471640 1 --2- 172.21.15.12:0/193954080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97cc0047d0 0x7f97cc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:31.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e6473640 1 --2- 172.21.15.12:0/193954080 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97e0070740 0x7f97cc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:31.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e5c72640 1 -- 172.21.15.12:0/193954080 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97e0070740 msgr2=0x7f97cc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:31.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e5c72640 1 --2- 172.21.15.12:0/193954080 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97e0070740 0x7f97cc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:31.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e5c72640 1 -- 172.21.15.12:0/193954080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97cc0047d0 msgr2=0x7f97cc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:31.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e5c72640 1 --2- 172.21.15.12:0/193954080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97cc0047d0 0x7f97cc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:31.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.969+0000 7f97e5c72640 1 -- 172.21.15.12:0/193954080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc003530 con 0x7f97cc003b40 2023-10-30T12:49:31.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.973+0000 7f97e5c72640 1 --2- 172.21.15.12:0/193954080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97cc003b40 0x7f97cc003f40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f97dc001ec0 tx=0x7f97dc01cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f885d2ee51b652d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.973+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0058e0 con 0x7f97cc003b40 2023-10-30T12:49:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.973+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97dc005a60 con 0x7f97cc003b40 2023-10-30T12:49:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.973+0000 7f97e4c70640 1 -- 172.21.15.12:0/193954080 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc005e10 con 0x7f97cc003b40 2023-10-30T12:49:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.973+0000 7f97e7ed6640 1 -- 172.21.15.12:0/193954080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97cc003b40 msgr2=0x7f97cc003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:31.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.973+0000 7f97e7ed6640 1 --2- 172.21.15.12:0/193954080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97cc003b40 0x7f97cc003f40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f97dc001ec0 tx=0x7f97dc01cb80 comp rx=0 tx=0).stop 2023-10-30T12:49:31.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.973+0000 7f97e6473640 1 -- 172.21.15.12:0/193954080 reap_dead start 2023-10-30T12:49:31.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.973+0000 7f97e7ed6640 1 -- 172.21.15.12:0/193954080 shutdown_connections 2023-10-30T12:49:31.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.973+0000 7f97e7ed6640 1 -- 172.21.15.12:0/193954080 >> 172.21.15.12:0/193954080 conn(0x7f97e00c7910 msgr2=0x7f97e00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:31.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.973+0000 7f97e7ed6640 1 -- 172.21.15.12:0/193954080 shutdown_connections 2023-10-30T12:49:31.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.973+0000 7f97e7ed6640 1 -- 172.21.15.12:0/193954080 wait complete. 2023-10-30T12:49:31.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e7ed6640 1 Processor -- start 2023-10-30T12:49:31.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e7ed6640 1 -- start start 2023-10-30T12:49:31.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e7ed6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97cc003b40 0x7f97e014e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:31.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e7ed6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97e00c1b50 0x7f97e00c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:31.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e7ed6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97e015d390 0x7f97e015d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:31.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e7ed6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e00c3780 con 0x7f97e00c1b50 2023-10-30T12:49:31.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e7ed6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e00c3110 con 0x7f97cc003b40 2023-10-30T12:49:31.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e7ed6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e015c660 con 0x7f97e015d390 2023-10-30T12:49:31.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e5471640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97e00c1b50 0x7f97e00c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:31.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e5471640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97e00c1b50 0x7f97e00c2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34388/0 (socket says 172.21.15.12:34388) 2023-10-30T12:49:31.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e5471640 1 -- 172.21.15.12:0/3836238115 learned_addr learned my addr 172.21.15.12:0/3836238115 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:31.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e5c72640 1 --2- 172.21.15.12:0/3836238115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97cc003b40 0x7f97e014e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:31.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e6473640 1 --2- 172.21.15.12:0/3836238115 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97e015d390 0x7f97e015d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:31.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e5471640 1 -- 172.21.15.12:0/3836238115 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97e015d390 msgr2=0x7f97e015d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:31.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e5471640 1 --2- 172.21.15.12:0/3836238115 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97e015d390 0x7f97e015d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:31.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e5471640 1 -- 172.21.15.12:0/3836238115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97cc003b40 msgr2=0x7f97e014e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:31.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e5471640 1 --2- 172.21.15.12:0/3836238115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97cc003b40 0x7f97e014e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:31.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e5471640 1 -- 172.21.15.12:0/3836238115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e015cf00 con 0x7f97e00c1b50 2023-10-30T12:49:31.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e5471640 1 --2- 172.21.15.12:0/3836238115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97e00c1b50 0x7f97e00c2bd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f97d00119c0 tx=0x7f97d0011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:31.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97d67fc640 1 -- 172.21.15.12:0/3836238115 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d000fa90 con 0x7f97e00c1b50 2023-10-30T12:49:31.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97d67fc640 1 -- 172.21.15.12:0/3836238115 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d000fbf0 con 0x7f97e00c1b50 2023-10-30T12:49:31.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97d67fc640 1 -- 172.21.15.12:0/3836238115 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0014440 con 0x7f97e00c1b50 2023-10-30T12:49:31.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e7ed6640 1 -- 172.21.15.12:0/3836238115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e015dd50 con 0x7f97e00c1b50 2023-10-30T12:49:31.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e7ed6640 1 -- 172.21.15.12:0/3836238115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e015bcb0 con 0x7f97e00c1b50 2023-10-30T12:49:31.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.977+0000 7f97e7ed6640 1 -- 172.21.15.12:0/3836238115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e004e170 con 0x7f97e00c1b50 2023-10-30T12:49:31.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.981+0000 7f97d67fc640 1 -- 172.21.15.12:0/3836238115 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f97d00145a0 con 0x7f97e00c1b50 2023-10-30T12:49:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.981+0000 7f97d67fc640 1 --2- 172.21.15.12:0/3836238115 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f97c40664f0 0x7f97c40689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.981+0000 7f97d67fc640 1 -- 172.21.15.12:0/3836238115 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f97d00848a0 con 0x7f97e00c1b50 2023-10-30T12:49:31.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.981+0000 7f97e5c72640 1 --2- 172.21.15.12:0/3836238115 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f97c40664f0 0x7f97c40689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:31.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.981+0000 7f97e5c72640 1 --2- 172.21.15.12:0/3836238115 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f97c40664f0 0x7f97c40689b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f97dc001ec0 tx=0x7f97dc001a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:31.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:31.985+0000 7f97d67fc640 1 -- 172.21.15.12:0/3836238115 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f97d0051b10 con 0x7f97e00c1b50 2023-10-30T12:49:32.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:32.173+0000 7f97e7ed6640 1 -- 172.21.15.12:0/3836238115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97e006aed0 con 0x7f97e00c1b50 2023-10-30T12:49:32.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:32.173+0000 7f97d67fc640 1 -- 172.21.15.12:0/3836238115 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6538 (secure 0 0 0) 0x7f97d0055130 con 0x7f97e00c1b50 2023-10-30T12:49:32.181 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:32.181 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:04.349963+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2023-10-30T12:49:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:32.177+0000 7f97e7ed6640 1 -- 172.21.15.12:0/3836238115 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f97c40664f0 msgr2=0x7f97c40689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:32.177+0000 7f97e7ed6640 1 --2- 172.21.15.12:0/3836238115 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f97c40664f0 0x7f97c40689b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f97dc001ec0 tx=0x7f97dc001a00 comp rx=0 tx=0).stop 2023-10-30T12:49:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:32.177+0000 7f97e7ed6640 1 -- 172.21.15.12:0/3836238115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97e00c1b50 msgr2=0x7f97e00c2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:32.177+0000 7f97e7ed6640 1 --2- 172.21.15.12:0/3836238115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97e00c1b50 0x7f97e00c2bd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f97d00119c0 tx=0x7f97d0011e80 comp rx=0 tx=0).stop 2023-10-30T12:49:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:32.177+0000 7f97e7ed6640 1 -- 172.21.15.12:0/3836238115 shutdown_connections 2023-10-30T12:49:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:32.177+0000 7f97e7ed6640 1 --2- 172.21.15.12:0/3836238115 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f97c40664f0 0x7f97c40689b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:32.177+0000 7f97e7ed6640 1 --2- 172.21.15.12:0/3836238115 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97e015d390 0x7f97e015d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:32.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:32.177+0000 7f97e7ed6640 1 --2- 172.21.15.12:0/3836238115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97e00c1b50 0x7f97e00c2bd0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:32.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:32.177+0000 7f97e7ed6640 1 --2- 172.21.15.12:0/3836238115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97cc003b40 0x7f97e014e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:32.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:32.177+0000 7f97e7ed6640 1 -- 172.21.15.12:0/3836238115 >> 172.21.15.12:0/3836238115 conn(0x7f97e00c7910 msgr2=0x7f97e0168d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:32.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:32.177+0000 7f97e7ed6640 1 -- 172.21.15.12:0/3836238115 shutdown_connections 2023-10-30T12:49:32.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:32.177+0000 7f97e7ed6640 1 -- 172.21.15.12:0/3836238115 wait complete. 2023-10-30T12:49:32.203 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:49:34.204 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:34.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac38017640 1 Processor -- start 2023-10-30T12:49:34.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac38017640 1 -- start start 2023-10-30T12:49:34.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac38017640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac301538a0 0x7fac3015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:34.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac38017640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac300ca0e0 0x7fac3015a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:34.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac38017640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac300cb800 con 0x7fac300cf160 2023-10-30T12:49:34.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac38017640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac3015dc30 con 0x7fac301538a0 2023-10-30T12:49:34.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac38017640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac300cb330 con 0x7fac300ca0e0 2023-10-30T12:49:34.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac35db3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac301538a0 0x7fac3015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:34.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac35db3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac301538a0 0x7fac3015d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34402/0 (socket says 172.21.15.12:34402) 2023-10-30T12:49:34.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac35db3640 1 -- 172.21.15.12:0/3968334179 learned_addr learned my addr 172.21.15.12:0/3968334179 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:34.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac365b4640 1 --1- 172.21.15.12:0/3968334179 >> v1:172.21.15.12:6789/0 conn(0x7fac300cf160 0x7fac300cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39694/0 (socket says 172.21.15.12:39694) 2023-10-30T12:49:34.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac355b2640 1 --2- 172.21.15.12:0/3968334179 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac300ca0e0 0x7fac3015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:34.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac35db3640 1 -- 172.21.15.12:0/3968334179 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac300ca0e0 msgr2=0x7fac3015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:34.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac35db3640 1 --2- 172.21.15.12:0/3968334179 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac300ca0e0 0x7fac3015a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac35db3640 1 -- 172.21.15.12:0/3968334179 >> v1:172.21.15.12:6789/0 conn(0x7fac300cf160 legacy=0x7fac300cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:34.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac34db1640 1 -- 172.21.15.12:0/3968334179 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67051370 0 0) 0x7fac300cb800 con 0x7fac300cf160 2023-10-30T12:49:34.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.353+0000 7fac35db3640 1 -- 172.21.15.12:0/3968334179 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac300cad70 con 0x7fac301538a0 2023-10-30T12:49:34.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.357+0000 7fac35db3640 1 --2- 172.21.15.12:0/3968334179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac301538a0 0x7fac3015d6f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009870 tx=0x7fac2c01ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2484fc701b9ef122 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:34.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.357+0000 7fac34db1640 1 -- 172.21.15.12:0/3968334179 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0a8a30 con 0x7fac301538a0 2023-10-30T12:49:34.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.357+0000 7fac34db1640 1 -- 172.21.15.12:0/3968334179 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2c0a8b90 con 0x7fac301538a0 2023-10-30T12:49:34.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.357+0000 7fac34db1640 1 -- 172.21.15.12:0/3968334179 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0b0440 con 0x7fac301538a0 2023-10-30T12:49:34.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.357+0000 7fac38017640 1 -- 172.21.15.12:0/3968334179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac301538a0 msgr2=0x7fac3015d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:34.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.357+0000 7fac38017640 1 --2- 172.21.15.12:0/3968334179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac301538a0 0x7fac3015d6f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009870 tx=0x7fac2c01ed30 comp rx=0 tx=0).stop 2023-10-30T12:49:34.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.357+0000 7fac38017640 1 -- 172.21.15.12:0/3968334179 shutdown_connections 2023-10-30T12:49:34.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.357+0000 7fac38017640 1 --2- 172.21.15.12:0/3968334179 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac300ca0e0 0x7fac3015a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.357+0000 7fac38017640 1 --2- 172.21.15.12:0/3968334179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac301538a0 0x7fac3015d6f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.357+0000 7fac38017640 1 -- 172.21.15.12:0/3968334179 >> 172.21.15.12:0/3968334179 conn(0x7fac3006bfe0 msgr2=0x7fac3006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:34.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.357+0000 7fac38017640 1 -- 172.21.15.12:0/3968334179 shutdown_connections 2023-10-30T12:49:34.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.357+0000 7fac38017640 1 -- 172.21.15.12:0/3968334179 wait complete. 2023-10-30T12:49:34.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac38017640 1 Processor -- start 2023-10-30T12:49:34.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac38017640 1 -- start start 2023-10-30T12:49:34.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac38017640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac300ca0e0 0x7fac30140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:34.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac38017640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac300cf160 0x7fac30146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:34.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac38017640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac301538a0 0x7fac30147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:34.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac38017640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac300ce8c0 con 0x7fac300cf160 2023-10-30T12:49:34.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac38017640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3014eed0 con 0x7fac300ca0e0 2023-10-30T12:49:34.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac38017640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac301f3290 con 0x7fac301538a0 2023-10-30T12:49:34.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac355b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac300cf160 0x7fac30146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:34.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac355b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac300cf160 0x7fac30146bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34404/0 (socket says 172.21.15.12:34404) 2023-10-30T12:49:34.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac355b2640 1 -- 172.21.15.12:0/1265295472 learned_addr learned my addr 172.21.15.12:0/1265295472 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:34.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac35db3640 1 --2- 172.21.15.12:0/1265295472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac300ca0e0 0x7fac30140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:34.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac365b4640 1 --2- 172.21.15.12:0/1265295472 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac301538a0 0x7fac30147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:34.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac355b2640 1 -- 172.21.15.12:0/1265295472 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac301538a0 msgr2=0x7fac30147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:34.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac355b2640 1 --2- 172.21.15.12:0/1265295472 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac301538a0 0x7fac30147110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac355b2640 1 -- 172.21.15.12:0/1265295472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac300ca0e0 msgr2=0x7fac30140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:34.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac355b2640 1 --2- 172.21.15.12:0/1265295472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac300ca0e0 0x7fac30140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac355b2640 1 -- 172.21.15.12:0/1265295472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2c0094d0 con 0x7fac300cf160 2023-10-30T12:49:34.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac35db3640 1 --2- 172.21.15.12:0/1265295472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac300ca0e0 0x7fac30140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:34.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac355b2640 1 --2- 172.21.15.12:0/1265295472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac300cf160 0x7fac30146bd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fac2000e9b0 tx=0x7fac2000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:34.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac267fc640 1 -- 172.21.15.12:0/1265295472 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac200096b0 con 0x7fac300cf160 2023-10-30T12:49:34.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac267fc640 1 -- 172.21.15.12:0/1265295472 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2000c9a0 con 0x7fac300cf160 2023-10-30T12:49:34.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac267fc640 1 -- 172.21.15.12:0/1265295472 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2000cd30 con 0x7fac300cf160 2023-10-30T12:49:34.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac38017640 1 -- 172.21.15.12:0/1265295472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac301f2430 con 0x7fac300cf160 2023-10-30T12:49:34.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac38017640 1 -- 172.21.15.12:0/1265295472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac301f29f0 con 0x7fac300cf160 2023-10-30T12:49:34.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.361+0000 7fac38017640 1 -- 172.21.15.12:0/1265295472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac300b2a50 con 0x7fac300cf160 2023-10-30T12:49:34.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.365+0000 7fac267fc640 1 -- 172.21.15.12:0/1265295472 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fac20013470 con 0x7fac300cf160 2023-10-30T12:49:34.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.365+0000 7fac267fc640 1 --2- 172.21.15.12:0/1265295472 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac140664a0 0x7fac14068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:34.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.365+0000 7fac267fc640 1 -- 172.21.15.12:0/1265295472 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6341+0+0 (secure 0 0 0) 0x7fac20084800 con 0x7fac300cf160 2023-10-30T12:49:34.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.365+0000 7fac35db3640 1 --2- 172.21.15.12:0/1265295472 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac140664a0 0x7fac14068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:34.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.365+0000 7fac35db3640 1 --2- 172.21.15.12:0/1265295472 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac140664a0 0x7fac14068960 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0b2000 tx=0x7fac2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:34.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.369+0000 7fac267fc640 1 -- 172.21.15.12:0/1265295472 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fac200518d0 con 0x7fac300cf160 2023-10-30T12:49:34.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.561+0000 7fac38017640 1 -- 172.21.15.12:0/1265295472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac3004e170 con 0x7fac300cf160 2023-10-30T12:49:34.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.561+0000 7fac267fc640 1 -- 172.21.15.12:0/1265295472 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6538 (secure 0 0 0) 0x7fac20054ef0 con 0x7fac300cf160 2023-10-30T12:49:34.568 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:34.568 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:04.349963+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5124,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5124":{"gid":5124,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":39,"addr":"172.21.15.12:6835/559649789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":559649789},{"type":"v1","addr":"172.21.15.12:6835","nonce":559649789}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:34.568 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2023-10-30T12:49:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.561+0000 7fac38017640 1 -- 172.21.15.12:0/1265295472 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac140664a0 msgr2=0x7fac14068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.561+0000 7fac38017640 1 --2- 172.21.15.12:0/1265295472 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac140664a0 0x7fac14068960 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0b2000 tx=0x7fac2c008040 comp rx=0 tx=0).stop 2023-10-30T12:49:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.561+0000 7fac38017640 1 -- 172.21.15.12:0/1265295472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac300cf160 msgr2=0x7fac30146bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.561+0000 7fac38017640 1 --2- 172.21.15.12:0/1265295472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac300cf160 0x7fac30146bd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fac2000e9b0 tx=0x7fac2000ee70 comp rx=0 tx=0).stop 2023-10-30T12:49:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.561+0000 7fac38017640 1 -- 172.21.15.12:0/1265295472 shutdown_connections 2023-10-30T12:49:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.561+0000 7fac38017640 1 --2- 172.21.15.12:0/1265295472 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fac140664a0 0x7fac14068960 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.561+0000 7fac38017640 1 --2- 172.21.15.12:0/1265295472 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac301538a0 0x7fac30147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.561+0000 7fac38017640 1 --2- 172.21.15.12:0/1265295472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac300cf160 0x7fac30146bd0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.561+0000 7fac38017640 1 --2- 172.21.15.12:0/1265295472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac300ca0e0 0x7fac30140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.561+0000 7fac38017640 1 -- 172.21.15.12:0/1265295472 >> 172.21.15.12:0/1265295472 conn(0x7fac3006bfe0 msgr2=0x7fac301527a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.561+0000 7fac38017640 1 -- 172.21.15.12:0/1265295472 shutdown_connections 2023-10-30T12:49:34.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:34.561+0000 7fac38017640 1 -- 172.21.15.12:0/1265295472 wait complete. 2023-10-30T12:49:34.589 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:49:36.119 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:49:36.123 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:49:36.590 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:36.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.737+0000 7f03cb809640 1 Processor -- start 2023-10-30T12:49:36.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.737+0000 7f03cb809640 1 -- start start 2023-10-30T12:49:36.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.741+0000 7f03cb809640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f03c41580e0 0x7f03c4070740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:36.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.741+0000 7f03cb809640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c40c5810 con 0x7f03c40c6540 2023-10-30T12:49:36.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.741+0000 7f03cb809640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c40c6990 con 0x7f03c415e700 2023-10-30T12:49:36.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.741+0000 7f03cb809640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c406acb0 con 0x7f03c41580e0 2023-10-30T12:49:36.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.741+0000 7f03c95a5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f03c41580e0 0x7f03c4070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:36.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.741+0000 7f03c95a5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f03c41580e0 0x7f03c4070740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34420/0 (socket says 172.21.15.12:34420) 2023-10-30T12:49:36.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.741+0000 7f03c95a5640 1 -- 172.21.15.12:0/112667584 learned_addr learned my addr 172.21.15.12:0/112667584 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:36.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.741+0000 7f03c95a5640 1 -- 172.21.15.12:0/112667584 >> v1:172.21.15.154:6789/0 conn(0x7f03c415e700 legacy=0x7f03c406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:36.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.741+0000 7f03c95a5640 1 -- 172.21.15.12:0/112667584 >> v1:172.21.15.12:6789/0 conn(0x7f03c40c6540 legacy=0x7f03c4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:36.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.741+0000 7f03c95a5640 1 -- 172.21.15.12:0/112667584 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c415eb90 con 0x7f03c41580e0 2023-10-30T12:49:36.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.745+0000 7f03c95a5640 1 --2- 172.21.15.12:0/112667584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03c41580e0 0x7f03c4070740 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009870 tx=0x7f03c001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f8ccf692d1354c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.745+0000 7f03bbfff640 1 -- 172.21.15.12:0/112667584 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c00a9400 con 0x7f03c41580e0 2023-10-30T12:49:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.745+0000 7f03bbfff640 1 -- 172.21.15.12:0/112667584 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c00a9560 con 0x7f03c41580e0 2023-10-30T12:49:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.745+0000 7f03bbfff640 1 -- 172.21.15.12:0/112667584 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c00a98f0 con 0x7f03c41580e0 2023-10-30T12:49:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.745+0000 7f03cb809640 1 -- 172.21.15.12:0/112667584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03c41580e0 msgr2=0x7f03c4070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.745+0000 7f03cb809640 1 --2- 172.21.15.12:0/112667584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03c41580e0 0x7f03c4070740 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009870 tx=0x7f03c001c540 comp rx=0 tx=0).stop 2023-10-30T12:49:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.745+0000 7f03cb809640 1 -- 172.21.15.12:0/112667584 shutdown_connections 2023-10-30T12:49:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.745+0000 7f03cb809640 1 --2- 172.21.15.12:0/112667584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03c41580e0 0x7f03c4070740 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.745+0000 7f03cb809640 1 -- 172.21.15.12:0/112667584 >> 172.21.15.12:0/112667584 conn(0x7f03c40c7910 msgr2=0x7f03c40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.745+0000 7f03cb809640 1 -- 172.21.15.12:0/112667584 shutdown_connections 2023-10-30T12:49:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.745+0000 7f03cb809640 1 -- 172.21.15.12:0/112667584 wait complete. 2023-10-30T12:49:36.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.745+0000 7f03cb809640 1 Processor -- start 2023-10-30T12:49:36.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.745+0000 7f03cb809640 1 -- start start 2023-10-30T12:49:36.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03cb809640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03c40c6540 0x7f03c414fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:36.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03cb809640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03c41580e0 0x7f03c40c2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:36.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03cb809640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03c415e700 0x7f03c414ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:36.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03cb809640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c41fa030 con 0x7f03c41580e0 2023-10-30T12:49:36.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03cb809640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c40c3840 con 0x7f03c40c6540 2023-10-30T12:49:36.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03cb809640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c41500a0 con 0x7f03c415e700 2023-10-30T12:49:36.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03c8da4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03c41580e0 0x7f03c40c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:36.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03c8da4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03c41580e0 0x7f03c40c2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34434/0 (socket says 172.21.15.12:34434) 2023-10-30T12:49:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03c8da4640 1 -- 172.21.15.12:0/3287328159 learned_addr learned my addr 172.21.15.12:0/3287328159 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03c95a5640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03c40c6540 0x7f03c414fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03c9da6640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03c415e700 0x7f03c414ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03c8da4640 1 -- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03c415e700 msgr2=0x7f03c414ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03c8da4640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03c415e700 0x7f03c414ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03c8da4640 1 -- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03c40c6540 msgr2=0x7f03c414fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:36.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03c8da4640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03c40c6540 0x7f03c414fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:36.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03c8da4640 1 -- 172.21.15.12:0/3287328159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c00094d0 con 0x7f03c41580e0 2023-10-30T12:49:36.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03c95a5640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03c40c6540 0x7f03c414fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03c9da6640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03c415e700 0x7f03c414ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03c8da4640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03c41580e0 0x7f03c40c2310 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f03b400c530 tx=0x7f03b400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03b9ffb640 1 -- 172.21.15.12:0/3287328159 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4005220 con 0x7f03c41580e0 2023-10-30T12:49:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03b9ffb640 1 -- 172.21.15.12:0/3287328159 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b4005380 con 0x7f03c41580e0 2023-10-30T12:49:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03b9ffb640 1 -- 172.21.15.12:0/3287328159 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4007270 con 0x7f03c41580e0 2023-10-30T12:49:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03cb809640 1 -- 172.21.15.12:0/3287328159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c41f9d50 con 0x7f03c41580e0 2023-10-30T12:49:36.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03cb809640 1 -- 172.21.15.12:0/3287328159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c40c31f0 con 0x7f03c41580e0 2023-10-30T12:49:36.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03cb809640 1 -- 172.21.15.12:0/3287328159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c4070820 con 0x7f03c41580e0 2023-10-30T12:49:36.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.749+0000 7f03b9ffb640 1 -- 172.21.15.12:0/3287328159 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f03b40073d0 con 0x7f03c41580e0 2023-10-30T12:49:36.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.753+0000 7f03b9ffb640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f03a80664a0 0x7f03a8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:36.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.753+0000 7f03b9ffb640 1 -- 172.21.15.12:0/3287328159 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f03b4083af0 con 0x7f03c41580e0 2023-10-30T12:49:36.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.753+0000 7f03c95a5640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f03a80664a0 0x7f03a8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:36.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.753+0000 7f03c95a5640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f03a80664a0 0x7f03a8068960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f03c00ae000 tx=0x7f03c00b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:36.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.757+0000 7f03b9ffb640 1 -- 172.21.15.12:0/3287328159 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f03b4050c00 con 0x7f03c41580e0 2023-10-30T12:49:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.945+0000 7f03cb809640 1 -- 172.21.15.12:0/3287328159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03c404e170 con 0x7f03c41580e0 2023-10-30T12:49:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.949+0000 7f03b9ffb640 1 -- 172.21.15.12:0/3287328159 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5746 (secure 0 0 0) 0x7f03b4054220 con 0x7f03c41580e0 2023-10-30T12:49:36.955 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:36.955 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:35.359799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 73 2023-10-30T12:49:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.949+0000 7f03cb809640 1 -- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f03a80664a0 msgr2=0x7f03a8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.949+0000 7f03cb809640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f03a80664a0 0x7f03a8068960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f03c00ae000 tx=0x7f03c00b3040 comp rx=0 tx=0).stop 2023-10-30T12:49:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.949+0000 7f03cb809640 1 -- 172.21.15.12:0/3287328159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03c41580e0 msgr2=0x7f03c40c2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.949+0000 7f03cb809640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03c41580e0 0x7f03c40c2310 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f03b400c530 tx=0x7f03b400c9f0 comp rx=0 tx=0).stop 2023-10-30T12:49:36.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.949+0000 7f03cb809640 1 -- 172.21.15.12:0/3287328159 shutdown_connections 2023-10-30T12:49:36.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.949+0000 7f03cb809640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f03a80664a0 0x7f03a8068960 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:36.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.949+0000 7f03cb809640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03c415e700 0x7f03c414ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:36.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.949+0000 7f03cb809640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03c41580e0 0x7f03c40c2310 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:36.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.949+0000 7f03cb809640 1 --2- 172.21.15.12:0/3287328159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03c40c6540 0x7f03c414fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:36.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.949+0000 7f03cb809640 1 -- 172.21.15.12:0/3287328159 >> 172.21.15.12:0/3287328159 conn(0x7f03c40c7910 msgr2=0x7f03c40ca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:36.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.949+0000 7f03cb809640 1 -- 172.21.15.12:0/3287328159 shutdown_connections 2023-10-30T12:49:36.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:36.949+0000 7f03cb809640 1 -- 172.21.15.12:0/3287328159 wait complete. 2023-10-30T12:49:36.976 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2023-10-30T12:49:36.976 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:49:36.976 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:37.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.113+0000 7f91ec032640 1 Processor -- start 2023-10-30T12:49:37.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.113+0000 7f91ec032640 1 -- start start 2023-10-30T12:49:37.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.113+0000 7f91ec032640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f91e4151800 0x7f91e406e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:37.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.113+0000 7f91ec032640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f91e40c1ee0 0x7f91e40c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:37.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.113+0000 7f91ec032640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e40c9230 0x7f91e40cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:37.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.113+0000 7f91ec032640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e406dc70 con 0x7f91e40c1ee0 2023-10-30T12:49:37.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.113+0000 7f91ec032640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e406e090 con 0x7f91e4151800 2023-10-30T12:49:37.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.113+0000 7f91ec032640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e40c8a50 con 0x7f91e40c9230 2023-10-30T12:49:37.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.117+0000 7f91e95cd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f91e40c1ee0 0x7f91e40c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:37.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.117+0000 7f91e95cd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f91e40c1ee0 0x7f91e40c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34440/0 (socket says 172.21.15.12:34440) 2023-10-30T12:49:37.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.117+0000 7f91e95cd640 1 -- 172.21.15.12:0/809738289 learned_addr learned my addr 172.21.15.12:0/809738289 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:37.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.117+0000 7f91ea5cf640 1 --2- 172.21.15.12:0/809738289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e40c9230 0x7f91e40cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:37.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.117+0000 7f91e9dce640 1 --2- 172.21.15.12:0/809738289 >> v2:172.21.15.154:3300/0 conn(0x7f91e4151800 0x7f91e406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:37.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.117+0000 7f91e95cd640 1 -- 172.21.15.12:0/809738289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e40c9230 msgr2=0x7f91e40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:37.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.117+0000 7f91e95cd640 1 --2- 172.21.15.12:0/809738289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e40c9230 0x7f91e40cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.117+0000 7f91e95cd640 1 -- 172.21.15.12:0/809738289 >> v2:172.21.15.154:3300/0 conn(0x7f91e4151800 msgr2=0x7f91e406e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:37.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.117+0000 7f91e95cd640 1 --2- 172.21.15.12:0/809738289 >> v2:172.21.15.154:3300/0 conn(0x7f91e4151800 0x7f91e406e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.117+0000 7f91e95cd640 1 -- 172.21.15.12:0/809738289 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e40c3b10 con 0x7f91e40c1ee0 2023-10-30T12:49:37.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.117+0000 7f91e9dce640 1 --2- 172.21.15.12:0/809738289 >> v2:172.21.15.154:3300/0 conn(0x7f91e4151800 0x7f91e406e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:37.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.117+0000 7f91e95cd640 1 --2- 172.21.15.12:0/809738289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e40c1ee0 0x7f91e40c8cf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f91e4070110 tx=0x7f91d401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37c2b156bf33a03 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:37.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.117+0000 7f91e8dcc640 1 -- 172.21.15.12:0/809738289 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d40aa5f0 con 0x7f91e40c1ee0 2023-10-30T12:49:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.117+0000 7f91e8dcc640 1 -- 172.21.15.12:0/809738289 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d40aa750 con 0x7f91e40c1ee0 2023-10-30T12:49:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.117+0000 7f91e8dcc640 1 -- 172.21.15.12:0/809738289 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d40aaae0 con 0x7f91e40c1ee0 2023-10-30T12:49:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 -- 172.21.15.12:0/809738289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e40c1ee0 msgr2=0x7f91e40c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:37.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 --2- 172.21.15.12:0/809738289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e40c1ee0 0x7f91e40c8cf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f91e4070110 tx=0x7f91d401c4d0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 -- 172.21.15.12:0/809738289 shutdown_connections 2023-10-30T12:49:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 --2- 172.21.15.12:0/809738289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e40c9230 0x7f91e40cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 --2- 172.21.15.12:0/809738289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e40c1ee0 0x7f91e40c8cf0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 --2- 172.21.15.12:0/809738289 >> v2:172.21.15.154:3300/0 conn(0x7f91e4151800 0x7f91e406e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 -- 172.21.15.12:0/809738289 >> 172.21.15.12:0/809738289 conn(0x7f91e4071ea0 msgr2=0x7f91e4065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:37.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 -- 172.21.15.12:0/809738289 shutdown_connections 2023-10-30T12:49:37.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 -- 172.21.15.12:0/809738289 wait complete. 2023-10-30T12:49:37.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 Processor -- start 2023-10-30T12:49:37.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 -- start start 2023-10-30T12:49:37.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e40c1ee0 0x7f91e413ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:37.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e40c9230 0x7f91e41f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:37.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e4151800 0x7f91e40c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:37.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e4148160 con 0x7f91e40c9230 2023-10-30T12:49:37.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e40c5470 con 0x7f91e4151800 2023-10-30T12:49:37.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ec032640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e41f6ba0 con 0x7f91e40c1ee0 2023-10-30T12:49:37.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91e95cd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e40c9230 0x7f91e41f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:37.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91e95cd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e40c9230 0x7f91e41f3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34456/0 (socket says 172.21.15.12:34456) 2023-10-30T12:49:37.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91e95cd640 1 -- 172.21.15.12:0/1998514304 learned_addr learned my addr 172.21.15.12:0/1998514304 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:37.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91ea5cf640 1 --2- 172.21.15.12:0/1998514304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e4151800 0x7f91e40c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:37.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91e9dce640 1 --2- 172.21.15.12:0/1998514304 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e40c1ee0 0x7f91e413ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:37.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91e95cd640 1 -- 172.21.15.12:0/1998514304 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e40c1ee0 msgr2=0x7f91e413ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:37.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91e95cd640 1 --2- 172.21.15.12:0/1998514304 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e40c1ee0 0x7f91e413ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91e95cd640 1 -- 172.21.15.12:0/1998514304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e4151800 msgr2=0x7f91e40c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:37.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91e95cd640 1 --2- 172.21.15.12:0/1998514304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e4151800 0x7f91e40c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.121+0000 7f91e95cd640 1 -- 172.21.15.12:0/1998514304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d40094d0 con 0x7f91e40c9230 2023-10-30T12:49:37.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.125+0000 7f91e95cd640 1 --2- 172.21.15.12:0/1998514304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e40c9230 0x7f91e41f3b80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f91d401ab80 tx=0x7f91d401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:37.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.125+0000 7f91da7fc640 1 -- 172.21.15.12:0/1998514304 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d40b3800 con 0x7f91e40c9230 2023-10-30T12:49:37.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.125+0000 7f91da7fc640 1 -- 172.21.15.12:0/1998514304 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d40b3980 con 0x7f91e40c9230 2023-10-30T12:49:37.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.125+0000 7f91da7fc640 1 -- 172.21.15.12:0/1998514304 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d40b3d30 con 0x7f91e40c9230 2023-10-30T12:49:37.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.125+0000 7f91ec032640 1 -- 172.21.15.12:0/1998514304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e41f5e20 con 0x7f91e40c9230 2023-10-30T12:49:37.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.125+0000 7f91ec032640 1 -- 172.21.15.12:0/1998514304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e41f60c0 con 0x7f91e40c9230 2023-10-30T12:49:37.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.125+0000 7f91ec032640 1 -- 172.21.15.12:0/1998514304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e404e170 con 0x7f91e40c9230 2023-10-30T12:49:37.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.125+0000 7f91da7fc640 1 -- 172.21.15.12:0/1998514304 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f91d40b2380 con 0x7f91e40c9230 2023-10-30T12:49:37.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.125+0000 7f91da7fc640 1 --2- 172.21.15.12:0/1998514304 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f91c80664d0 0x7f91c8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:37.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.125+0000 7f91da7fc640 1 -- 172.21.15.12:0/1998514304 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f91d40a7070 con 0x7f91e40c9230 2023-10-30T12:49:37.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.125+0000 7f91e9dce640 1 --2- 172.21.15.12:0/1998514304 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f91c80664d0 0x7f91c8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:37.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.129+0000 7f91e9dce640 1 --2- 172.21.15.12:0/1998514304 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f91c80664d0 0x7f91c8068990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f91e000ba10 tx=0x7f91e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:37.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.129+0000 7f91da7fc640 1 -- 172.21.15.12:0/1998514304 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f91d40eb4c0 con 0x7f91e40c9230 2023-10-30T12:49:37.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.321+0000 7f91ec032640 1 -- 172.21.15.12:0/1998514304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91e4161da0 con 0x7f91e40c9230 2023-10-30T12:49:37.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.321+0000 7f91da7fc640 1 -- 172.21.15.12:0/1998514304 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5746 (secure 0 0 0) 0x7f91d40eeae0 con 0x7f91e40c9230 2023-10-30T12:49:37.330 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:37.330 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:35.359799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:37.330 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 73 2023-10-30T12:49:37.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.325+0000 7f91ec032640 1 -- 172.21.15.12:0/1998514304 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f91c80664d0 msgr2=0x7f91c8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:37.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.325+0000 7f91ec032640 1 --2- 172.21.15.12:0/1998514304 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f91c80664d0 0x7f91c8068990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f91e000ba10 tx=0x7f91e0008040 comp rx=0 tx=0).stop 2023-10-30T12:49:37.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.325+0000 7f91ec032640 1 -- 172.21.15.12:0/1998514304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e40c9230 msgr2=0x7f91e41f3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:37.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.325+0000 7f91ec032640 1 --2- 172.21.15.12:0/1998514304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e40c9230 0x7f91e41f3b80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f91d401ab80 tx=0x7f91d401ad10 comp rx=0 tx=0).stop 2023-10-30T12:49:37.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.325+0000 7f91ec032640 1 -- 172.21.15.12:0/1998514304 shutdown_connections 2023-10-30T12:49:37.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.325+0000 7f91ec032640 1 --2- 172.21.15.12:0/1998514304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e4151800 0x7f91e40c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.325+0000 7f91ec032640 1 --2- 172.21.15.12:0/1998514304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e40c9230 0x7f91e41f3b80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.325+0000 7f91ec032640 1 --2- 172.21.15.12:0/1998514304 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f91c80664d0 0x7f91c8068990 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.325+0000 7f91ec032640 1 --2- 172.21.15.12:0/1998514304 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91e40c1ee0 0x7f91e413ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.325+0000 7f91ec032640 1 -- 172.21.15.12:0/1998514304 >> 172.21.15.12:0/1998514304 conn(0x7f91e4071ea0 msgr2=0x7f91e4150a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:37.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.325+0000 7f91ec032640 1 -- 172.21.15.12:0/1998514304 shutdown_connections 2023-10-30T12:49:37.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:37.325+0000 7f91ec032640 1 -- 172.21.15.12:0/1998514304 wait complete. 2023-10-30T12:49:37.351 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 73, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 73, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5427": { "addr": "172.21.15.12:6836/2219276371", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 2219276371, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 2219276371, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 5427, "incarnation": 57, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 31 }, "gid_5571": { "addr": "172.21.15.12:6837/2752088514", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2752088514, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 2752088514, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5571, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 19 }, "gid_5758": { "addr": "172.21.15.154:6839/2204547612", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 2204547612, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 2204547612, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5758, "incarnation": 73, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 52, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:49:35.359799+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5571, "mds_1": 5758, "mds_2": 5427 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:49:39.353 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:39.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.501+0000 7f544fda1640 1 Processor -- start 2023-10-30T12:49:39.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.501+0000 7f544fda1640 1 -- start start 2023-10-30T12:49:39.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.501+0000 7f544fda1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806f7a0 0x7f544806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:39.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.501+0000 7f544fda1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f54480c6bd0 0x7f5448069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:39.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.501+0000 7f544fda1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f544806a3d0 con 0x7f5448071110 2023-10-30T12:49:39.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.501+0000 7f544fda1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54480700c0 con 0x7f54480c6bd0 2023-10-30T12:49:39.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.501+0000 7f544fda1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5448066cf0 con 0x7f544806f7a0 2023-10-30T12:49:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.501+0000 7f544d33c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f54480c6bd0 0x7f5448069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.501+0000 7f544db3d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806f7a0 0x7f544806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.501+0000 7f544e33e640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5448071110 0x7f5448158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:60218/0 (socket says 172.21.15.12:60218) 2023-10-30T12:49:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.501+0000 7f544e33e640 1 -- 172.21.15.12:0/88264149 learned_addr learned my addr 172.21.15.12:0/88264149 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:49:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.505+0000 7f544db3d640 1 -- 172.21.15.12:0/88264149 >> v2:172.21.15.154:3300/0 conn(0x7f54480c6bd0 msgr2=0x7f5448069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.505+0000 7f544db3d640 1 --2- 172.21.15.12:0/88264149 >> v2:172.21.15.154:3300/0 conn(0x7f54480c6bd0 0x7f5448069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.505+0000 7f544db3d640 1 -- 172.21.15.12:0/88264149 >> v1:172.21.15.154:6789/0 conn(0x7f5448071110 legacy=0x7f5448158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.505+0000 7f544db3d640 1 -- 172.21.15.12:0/88264149 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54480c70d0 con 0x7f544806f7a0 2023-10-30T12:49:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.505+0000 7f544db3d640 1 --2- 172.21.15.12:0/88264149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806f7a0 0x7f544806fb80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f54440097c0 tx=0x7f544401ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce4fa6146505e38a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.505+0000 7f544cb3b640 1 -- 172.21.15.12:0/88264149 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54440a8970 con 0x7f544806f7a0 2023-10-30T12:49:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.505+0000 7f544cb3b640 1 -- 172.21.15.12:0/88264149 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54440a8ad0 con 0x7f544806f7a0 2023-10-30T12:49:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.505+0000 7f544cb3b640 1 -- 172.21.15.12:0/88264149 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54440a8e60 con 0x7f544806f7a0 2023-10-30T12:49:39.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 -- 172.21.15.12:0/88264149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806f7a0 msgr2=0x7f544806fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:39.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 --2- 172.21.15.12:0/88264149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806f7a0 0x7f544806fb80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f54440097c0 tx=0x7f544401ec70 comp rx=0 tx=0).stop 2023-10-30T12:49:39.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 -- 172.21.15.12:0/88264149 shutdown_connections 2023-10-30T12:49:39.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 --2- 172.21.15.12:0/88264149 >> v2:172.21.15.154:3300/0 conn(0x7f54480c6bd0 0x7f5448069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 --2- 172.21.15.12:0/88264149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f544806f7a0 0x7f544806fb80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 -- 172.21.15.12:0/88264149 >> 172.21.15.12:0/88264149 conn(0x7f54480c7910 msgr2=0x7f54480c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 -- 172.21.15.12:0/88264149 shutdown_connections 2023-10-30T12:49:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 -- 172.21.15.12:0/88264149 wait complete. 2023-10-30T12:49:39.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 Processor -- start 2023-10-30T12:49:39.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 -- start start 2023-10-30T12:49:39.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f544806f7a0 0x7f54480ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:39.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5448071110 0x7f54480aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:39.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54480c6bd0 0x7f54480c0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:39.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54481f3530 con 0x7f54480c6bd0 2023-10-30T12:49:39.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54480afc70 con 0x7f544806f7a0 2023-10-30T12:49:39.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544fda1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544815ac10 con 0x7f5448071110 2023-10-30T12:49:39.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544e33e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54480c6bd0 0x7f54480c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:39.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544e33e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54480c6bd0 0x7f54480c0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34472/0 (socket says 172.21.15.12:34472) 2023-10-30T12:49:39.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544e33e640 1 -- 172.21.15.12:0/2646139611 learned_addr learned my addr 172.21.15.12:0/2646139611 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:39.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544db3d640 1 --2- 172.21.15.12:0/2646139611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f544806f7a0 0x7f54480ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:39.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.509+0000 7f544d33c640 1 --2- 172.21.15.12:0/2646139611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5448071110 0x7f54480aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:39.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.513+0000 7f544e33e640 1 -- 172.21.15.12:0/2646139611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5448071110 msgr2=0x7f54480aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:39.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.513+0000 7f544e33e640 1 --2- 172.21.15.12:0/2646139611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5448071110 0x7f54480aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.513+0000 7f544e33e640 1 -- 172.21.15.12:0/2646139611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f544806f7a0 msgr2=0x7f54480ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:39.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.513+0000 7f544e33e640 1 --2- 172.21.15.12:0/2646139611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f544806f7a0 0x7f54480ae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.513+0000 7f544e33e640 1 -- 172.21.15.12:0/2646139611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5444009420 con 0x7f54480c6bd0 2023-10-30T12:49:39.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.513+0000 7f544e33e640 1 --2- 172.21.15.12:0/2646139611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54480c6bd0 0x7f54480c0580 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f544000cab0 tx=0x7f544000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:39.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.513+0000 7f543e7fc640 1 -- 172.21.15.12:0/2646139611 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544000ac30 con 0x7f54480c6bd0 2023-10-30T12:49:39.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.513+0000 7f543e7fc640 1 -- 172.21.15.12:0/2646139611 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544000ad90 con 0x7f54480c6bd0 2023-10-30T12:49:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.513+0000 7f544fda1640 1 -- 172.21.15.12:0/2646139611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54480c00f0 con 0x7f54480c6bd0 2023-10-30T12:49:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.513+0000 7f543e7fc640 1 -- 172.21.15.12:0/2646139611 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440012600 con 0x7f54480c6bd0 2023-10-30T12:49:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.513+0000 7f544fda1640 1 -- 172.21.15.12:0/2646139611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54480af6b0 con 0x7f54480c6bd0 2023-10-30T12:49:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.513+0000 7f544fda1640 1 -- 172.21.15.12:0/2646139611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54480c0280 con 0x7f54480c6bd0 2023-10-30T12:49:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.513+0000 7f543e7fc640 1 -- 172.21.15.12:0/2646139611 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5440012760 con 0x7f54480c6bd0 2023-10-30T12:49:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.513+0000 7f543e7fc640 1 --2- 172.21.15.12:0/2646139611 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f542c0664d0 0x7f542c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.513+0000 7f543e7fc640 1 -- 172.21.15.12:0/2646139611 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f5440083fb0 con 0x7f54480c6bd0 2023-10-30T12:49:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.517+0000 7f544db3d640 1 --2- 172.21.15.12:0/2646139611 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f542c0664d0 0x7f542c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.517+0000 7f544db3d640 1 --2- 172.21.15.12:0/2646139611 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f542c0664d0 0x7f542c068990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54440b2000 tx=0x7f5444008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:39.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.517+0000 7f543e7fc640 1 -- 172.21.15.12:0/2646139611 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5440051170 con 0x7f54480c6bd0 2023-10-30T12:49:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.713+0000 7f544fda1640 1 -- 172.21.15.12:0/2646139611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f544804e170 con 0x7f54480c6bd0 2023-10-30T12:49:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.713+0000 7f543e7fc640 1 -- 172.21.15.12:0/2646139611 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5746 (secure 0 0 0) 0x7f5440054790 con 0x7f54480c6bd0 2023-10-30T12:49:39.720 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:39.720 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:35.359799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 73 2023-10-30T12:49:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.713+0000 7f544fda1640 1 -- 172.21.15.12:0/2646139611 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f542c0664d0 msgr2=0x7f542c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.713+0000 7f544fda1640 1 --2- 172.21.15.12:0/2646139611 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f542c0664d0 0x7f542c068990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54440b2000 tx=0x7f5444008040 comp rx=0 tx=0).stop 2023-10-30T12:49:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.713+0000 7f544fda1640 1 -- 172.21.15.12:0/2646139611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54480c6bd0 msgr2=0x7f54480c0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.713+0000 7f544fda1640 1 --2- 172.21.15.12:0/2646139611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54480c6bd0 0x7f54480c0580 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f544000cab0 tx=0x7f544000cf70 comp rx=0 tx=0).stop 2023-10-30T12:49:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.713+0000 7f544fda1640 1 -- 172.21.15.12:0/2646139611 shutdown_connections 2023-10-30T12:49:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.713+0000 7f544fda1640 1 --2- 172.21.15.12:0/2646139611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54480c6bd0 0x7f54480c0580 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.713+0000 7f544fda1640 1 --2- 172.21.15.12:0/2646139611 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f542c0664d0 0x7f542c068990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.713+0000 7f544fda1640 1 --2- 172.21.15.12:0/2646139611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5448071110 0x7f54480aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.713+0000 7f544fda1640 1 --2- 172.21.15.12:0/2646139611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f544806f7a0 0x7f54480ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.713+0000 7f544fda1640 1 -- 172.21.15.12:0/2646139611 >> 172.21.15.12:0/2646139611 conn(0x7f54480c7910 msgr2=0x7f54480c4580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.713+0000 7f544fda1640 1 -- 172.21.15.12:0/2646139611 shutdown_connections 2023-10-30T12:49:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:39.713+0000 7f544fda1640 1 -- 172.21.15.12:0/2646139611 wait complete. 2023-10-30T12:49:39.741 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:49:41.742 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:41.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.889+0000 7ff1f7009640 1 Processor -- start 2023-10-30T12:49:41.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.889+0000 7ff1f7009640 1 -- start start 2023-10-30T12:49:41.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.893+0000 7ff1f7009640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff1f00c6bd0 0x7ff1f006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:41.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.893+0000 7ff1f7009640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1f0071110 0x7ff1f0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:41.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.893+0000 7ff1f7009640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f0066cf0 con 0x7ff1f006f7a0 2023-10-30T12:49:41.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.893+0000 7ff1f7009640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f00700c0 con 0x7ff1f00c6bd0 2023-10-30T12:49:41.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.893+0000 7ff1f7009640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00c70d0 con 0x7ff1f0071110 2023-10-30T12:49:41.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.893+0000 7ff1effff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff1f00c6bd0 0x7ff1f006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:41.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.893+0000 7ff1f4da5640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff1f006f7a0 0x7ff1f0069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:54382/0 (socket says 172.21.15.12:54382) 2023-10-30T12:49:41.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.893+0000 7ff1f4da5640 1 -- 172.21.15.12:0/4203545070 learned_addr learned my addr 172.21.15.12:0/4203545070 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:49:41.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.893+0000 7ff1f55a6640 1 --2- 172.21.15.12:0/4203545070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1f0071110 0x7ff1f0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:41.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.893+0000 7ff1ef7fe640 1 -- 172.21.15.12:0/4203545070 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745284338 0 0) 0x7ff1f0066cf0 con 0x7ff1f006f7a0 2023-10-30T12:49:41.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.893+0000 7ff1ef7fe640 1 -- 172.21.15.12:0/4203545070 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1e0003530 con 0x7ff1f006f7a0 2023-10-30T12:49:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.893+0000 7ff1effff640 1 -- 172.21.15.12:0/4203545070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1f0071110 msgr2=0x7ff1f0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.893+0000 7ff1effff640 1 --2- 172.21.15.12:0/4203545070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1f0071110 0x7ff1f0158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.893+0000 7ff1effff640 1 -- 172.21.15.12:0/4203545070 >> v1:172.21.15.154:6789/0 conn(0x7ff1f006f7a0 legacy=0x7ff1f0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:41.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.893+0000 7ff1effff640 1 -- 172.21.15.12:0/4203545070 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f00c7390 con 0x7ff1f00c6bd0 2023-10-30T12:49:41.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.897+0000 7ff1effff640 1 --2- 172.21.15.12:0/4203545070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1f00c6bd0 0x7ff1f006fb80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0097c0 tx=0x7ff1dc01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f29d4e8aa46025 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:41.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.897+0000 7ff1ef7fe640 1 -- 172.21.15.12:0/4203545070 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0a8970 con 0x7ff1f00c6bd0 2023-10-30T12:49:41.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.897+0000 7ff1ef7fe640 1 -- 172.21.15.12:0/4203545070 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc0a8ad0 con 0x7ff1f00c6bd0 2023-10-30T12:49:41.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.897+0000 7ff1ef7fe640 1 -- 172.21.15.12:0/4203545070 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0a8e40 con 0x7ff1f00c6bd0 2023-10-30T12:49:41.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.897+0000 7ff1f7009640 1 -- 172.21.15.12:0/4203545070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1f00c6bd0 msgr2=0x7ff1f006fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:41.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.897+0000 7ff1f7009640 1 --2- 172.21.15.12:0/4203545070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1f00c6bd0 0x7ff1f006fb80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0097c0 tx=0x7ff1dc01ec70 comp rx=0 tx=0).stop 2023-10-30T12:49:41.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.897+0000 7ff1f7009640 1 -- 172.21.15.12:0/4203545070 shutdown_connections 2023-10-30T12:49:41.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.897+0000 7ff1f7009640 1 --2- 172.21.15.12:0/4203545070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1f0071110 0x7ff1f0158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:41.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.897+0000 7ff1f7009640 1 --2- 172.21.15.12:0/4203545070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1f00c6bd0 0x7ff1f006fb80 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:41.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.897+0000 7ff1f7009640 1 -- 172.21.15.12:0/4203545070 >> 172.21.15.12:0/4203545070 conn(0x7ff1f00c7910 msgr2=0x7ff1f00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:41.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.897+0000 7ff1f7009640 1 -- 172.21.15.12:0/4203545070 shutdown_connections 2023-10-30T12:49:41.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.897+0000 7ff1f7009640 1 -- 172.21.15.12:0/4203545070 wait complete. 2023-10-30T12:49:41.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.897+0000 7ff1f7009640 1 Processor -- start 2023-10-30T12:49:41.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.897+0000 7ff1f7009640 1 -- start start 2023-10-30T12:49:41.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f7009640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1f006f7a0 0x7ff1f014fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:41.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f7009640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1f0071110 0x7ff1f00bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:41.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f7009640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1f00c6bd0 0x7ff1f00c0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:41.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f7009640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00af980 con 0x7ff1f006f7a0 2023-10-30T12:49:41.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f7009640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f01ef9f0 con 0x7ff1f0071110 2023-10-30T12:49:41.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f7009640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00c0e50 con 0x7ff1f00c6bd0 2023-10-30T12:49:41.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f4da5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1f006f7a0 0x7ff1f014fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:41.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f4da5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1f006f7a0 0x7ff1f014fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39546/0 (socket says 172.21.15.12:39546) 2023-10-30T12:49:41.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f4da5640 1 -- 172.21.15.12:0/4104763885 learned_addr learned my addr 172.21.15.12:0/4104763885 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:41.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1effff640 1 --2- 172.21.15.12:0/4104763885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1f0071110 0x7ff1f00bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:41.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f55a6640 1 --2- 172.21.15.12:0/4104763885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1f00c6bd0 0x7ff1f00c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:41.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f4da5640 1 -- 172.21.15.12:0/4104763885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1f00c6bd0 msgr2=0x7ff1f00c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:41.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f4da5640 1 --2- 172.21.15.12:0/4104763885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1f00c6bd0 0x7ff1f00c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:41.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f4da5640 1 -- 172.21.15.12:0/4104763885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1f0071110 msgr2=0x7ff1f00bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:41.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f4da5640 1 --2- 172.21.15.12:0/4104763885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1f0071110 0x7ff1f00bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:41.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f4da5640 1 -- 172.21.15.12:0/4104763885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc009420 con 0x7ff1f006f7a0 2023-10-30T12:49:41.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f4da5640 1 --2- 172.21.15.12:0/4104763885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1f006f7a0 0x7ff1f014fc20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400c3f0 tx=0x7ff1e400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:41.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1ed7fa640 1 -- 172.21.15.12:0/4104763885 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4005cf0 con 0x7ff1f006f7a0 2023-10-30T12:49:41.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1ed7fa640 1 -- 172.21.15.12:0/4104763885 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e4005e50 con 0x7ff1f006f7a0 2023-10-30T12:49:41.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1ed7fa640 1 -- 172.21.15.12:0/4104763885 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40072f0 con 0x7ff1f006f7a0 2023-10-30T12:49:41.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f7009640 1 -- 172.21.15.12:0/4104763885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f01eee70 con 0x7ff1f006f7a0 2023-10-30T12:49:41.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f7009640 1 -- 172.21.15.12:0/4104763885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f01ef430 con 0x7ff1f006f7a0 2023-10-30T12:49:41.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1f7009640 1 -- 172.21.15.12:0/4104763885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f004e170 con 0x7ff1f006f7a0 2023-10-30T12:49:41.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.901+0000 7ff1ed7fa640 1 -- 172.21.15.12:0/4104763885 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff1e4007450 con 0x7ff1f006f7a0 2023-10-30T12:49:41.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.905+0000 7ff1ed7fa640 1 --2- 172.21.15.12:0/4104763885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff1d40664d0 0x7ff1d4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:41.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.905+0000 7ff1ed7fa640 1 -- 172.21.15.12:0/4104763885 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7ff1e4083700 con 0x7ff1f006f7a0 2023-10-30T12:49:41.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.905+0000 7ff1effff640 1 --2- 172.21.15.12:0/4104763885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff1d40664d0 0x7ff1d4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:41.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.905+0000 7ff1effff640 1 --2- 172.21.15.12:0/4104763885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff1d40664d0 0x7ff1d4068990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0b2000 tx=0x7ff1dc0aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:41.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:41.909+0000 7ff1ed7fa640 1 -- 172.21.15.12:0/4104763885 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff1e40508c0 con 0x7ff1f006f7a0 2023-10-30T12:49:42.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:42.097+0000 7ff1f7009640 1 -- 172.21.15.12:0/4104763885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1f00c0c20 con 0x7ff1f006f7a0 2023-10-30T12:49:42.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:42.097+0000 7ff1ed7fa640 1 -- 172.21.15.12:0/4104763885 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5746 (secure 0 0 0) 0x7ff1e4053ee0 con 0x7ff1f006f7a0 2023-10-30T12:49:42.104 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:42.104 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:35.359799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:42.104 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 73 2023-10-30T12:49:42.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:42.097+0000 7ff1f7009640 1 -- 172.21.15.12:0/4104763885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff1d40664d0 msgr2=0x7ff1d4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:42.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:42.097+0000 7ff1f7009640 1 --2- 172.21.15.12:0/4104763885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff1d40664d0 0x7ff1d4068990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0b2000 tx=0x7ff1dc0aa070 comp rx=0 tx=0).stop 2023-10-30T12:49:42.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:42.097+0000 7ff1f7009640 1 -- 172.21.15.12:0/4104763885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1f006f7a0 msgr2=0x7ff1f014fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:42.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:42.097+0000 7ff1f7009640 1 --2- 172.21.15.12:0/4104763885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1f006f7a0 0x7ff1f014fc20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400c3f0 tx=0x7ff1e400c8b0 comp rx=0 tx=0).stop 2023-10-30T12:49:42.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:42.097+0000 7ff1f7009640 1 -- 172.21.15.12:0/4104763885 shutdown_connections 2023-10-30T12:49:42.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:42.097+0000 7ff1f7009640 1 --2- 172.21.15.12:0/4104763885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff1d40664d0 0x7ff1d4068990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:42.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:42.097+0000 7ff1f7009640 1 --2- 172.21.15.12:0/4104763885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1f00c6bd0 0x7ff1f00c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:42.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:42.097+0000 7ff1f7009640 1 --2- 172.21.15.12:0/4104763885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1f0071110 0x7ff1f00bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:42.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:42.097+0000 7ff1f7009640 1 --2- 172.21.15.12:0/4104763885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1f006f7a0 0x7ff1f014fc20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:42.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:42.097+0000 7ff1f7009640 1 -- 172.21.15.12:0/4104763885 >> 172.21.15.12:0/4104763885 conn(0x7ff1f00c7910 msgr2=0x7ff1f00c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:42.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:42.097+0000 7ff1f7009640 1 -- 172.21.15.12:0/4104763885 shutdown_connections 2023-10-30T12:49:42.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:42.097+0000 7ff1f7009640 1 -- 172.21.15.12:0/4104763885 wait complete. 2023-10-30T12:49:42.125 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:49:44.127 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:44.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.269+0000 7f90ad06b640 1 Processor -- start 2023-10-30T12:49:44.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.269+0000 7f90ad06b640 1 -- start start 2023-10-30T12:49:44.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.273+0000 7f90ad06b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90a81580e0 0x7f90a8070740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:44.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.273+0000 7f90ad06b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90a815e700 0x7f90a806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:44.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.273+0000 7f90ad06b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a80c5810 con 0x7f90a80c6540 2023-10-30T12:49:44.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.273+0000 7f90ad06b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a815eb90 con 0x7f90a815e700 2023-10-30T12:49:44.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.273+0000 7f90ad06b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a80c6960 con 0x7f90a81580e0 2023-10-30T12:49:44.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.273+0000 7f90a759e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90a815e700 0x7f90a806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:44.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.273+0000 7f90a759e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90a815e700 0x7f90a806a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39548/0 (socket says 172.21.15.12:39548) 2023-10-30T12:49:44.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.273+0000 7f90a759e640 1 -- 172.21.15.12:0/1786903669 learned_addr learned my addr 172.21.15.12:0/1786903669 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:44.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.273+0000 7f909ffff640 1 --1- 172.21.15.12:0/1786903669 >> v1:172.21.15.12:6789/0 conn(0x7f90a80c6540 0x7f90a8069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40838/0 (socket says 172.21.15.12:40838) 2023-10-30T12:49:44.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.273+0000 7f90a6d9d640 1 --2- 172.21.15.12:0/1786903669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90a81580e0 0x7f90a8070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:44.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.273+0000 7f90a759e640 1 -- 172.21.15.12:0/1786903669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90a81580e0 msgr2=0x7f90a8070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:44.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.273+0000 7f90a759e640 1 --2- 172.21.15.12:0/1786903669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90a81580e0 0x7f90a8070740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.273+0000 7f90a759e640 1 -- 172.21.15.12:0/1786903669 >> v1:172.21.15.12:6789/0 conn(0x7f90a80c6540 legacy=0x7f90a8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:44.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.273+0000 7f90a759e640 1 -- 172.21.15.12:0/1786903669 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a80c6bb0 con 0x7f90a815e700 2023-10-30T12:49:44.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.277+0000 7f90a759e640 1 --2- 172.21.15.12:0/1786903669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a815e700 0x7f90a806a5a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009e50 tx=0x7f90a001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75a826c90eaa640 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:44.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.277+0000 7f90a659c640 1 -- 172.21.15.12:0/1786903669 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a00a58a0 con 0x7f90a815e700 2023-10-30T12:49:44.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.277+0000 7f90a659c640 1 -- 172.21.15.12:0/1786903669 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a00a5a00 con 0x7f90a815e700 2023-10-30T12:49:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.277+0000 7f90a659c640 1 -- 172.21.15.12:0/1786903669 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a00a5d90 con 0x7f90a815e700 2023-10-30T12:49:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.277+0000 7f90ad06b640 1 -- 172.21.15.12:0/1786903669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a815e700 msgr2=0x7f90a806a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.277+0000 7f90ad06b640 1 --2- 172.21.15.12:0/1786903669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a815e700 0x7f90a806a5a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009e50 tx=0x7f90a001c9e0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.277+0000 7f90ad06b640 1 -- 172.21.15.12:0/1786903669 shutdown_connections 2023-10-30T12:49:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.277+0000 7f90ad06b640 1 --2- 172.21.15.12:0/1786903669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a815e700 0x7f90a806a5a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.277+0000 7f90ad06b640 1 --2- 172.21.15.12:0/1786903669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90a81580e0 0x7f90a8070740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.277+0000 7f90ad06b640 1 -- 172.21.15.12:0/1786903669 >> 172.21.15.12:0/1786903669 conn(0x7f90a80c7910 msgr2=0x7f90a80c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.277+0000 7f90ad06b640 1 -- 172.21.15.12:0/1786903669 shutdown_connections 2023-10-30T12:49:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.277+0000 7f90ad06b640 1 -- 172.21.15.12:0/1786903669 wait complete. 2023-10-30T12:49:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.277+0000 7f90ad06b640 1 Processor -- start 2023-10-30T12:49:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.277+0000 7f90ad06b640 1 -- start start 2023-10-30T12:49:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f90ad06b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90a80c6540 0x7f90a80c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f90ad06b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a81580e0 0x7f90a80718f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f90ad06b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a815e700 0x7f90a81f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f90ad06b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a81fa310 con 0x7f90a81580e0 2023-10-30T12:49:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f90ad06b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a81f9740 con 0x7f90a815e700 2023-10-30T12:49:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f90ad06b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a80722c0 con 0x7f90a80c6540 2023-10-30T12:49:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f909ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a81580e0 0x7f90a80718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f909ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a81580e0 0x7f90a80718f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39550/0 (socket says 172.21.15.12:39550) 2023-10-30T12:49:44.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f909ffff640 1 -- 172.21.15.12:0/2222691465 learned_addr learned my addr 172.21.15.12:0/2222691465 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:44.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f90a6d9d640 1 --2- 172.21.15.12:0/2222691465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90a80c6540 0x7f90a80c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:44.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f90a759e640 1 --2- 172.21.15.12:0/2222691465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a815e700 0x7f90a81f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:44.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f909ffff640 1 -- 172.21.15.12:0/2222691465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90a80c6540 msgr2=0x7f90a80c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:44.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f909ffff640 1 --2- 172.21.15.12:0/2222691465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90a80c6540 0x7f90a80c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f909ffff640 1 -- 172.21.15.12:0/2222691465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a815e700 msgr2=0x7f90a81f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f909ffff640 1 --2- 172.21.15.12:0/2222691465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a815e700 0x7f90a81f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f909ffff640 1 -- 172.21.15.12:0/2222691465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a0009b30 con 0x7f90a81580e0 2023-10-30T12:49:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f90a759e640 1 --2- 172.21.15.12:0/2222691465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a815e700 0x7f90a81f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f909ffff640 1 --2- 172.21.15.12:0/2222691465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a81580e0 0x7f90a80718f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f909000c420 tx=0x7f909000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:44.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f909f7fe640 1 -- 172.21.15.12:0/2222691465 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90900074e0 con 0x7f90a81580e0 2023-10-30T12:49:44.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f909f7fe640 1 -- 172.21.15.12:0/2222691465 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9090007640 con 0x7f90a81580e0 2023-10-30T12:49:44.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f909f7fe640 1 -- 172.21.15.12:0/2222691465 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090004a30 con 0x7f90a81580e0 2023-10-30T12:49:44.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f90ad06b640 1 -- 172.21.15.12:0/2222691465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a81f99e0 con 0x7f90a81580e0 2023-10-30T12:49:44.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f90ad06b640 1 -- 172.21.15.12:0/2222691465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a81f9fa0 con 0x7f90a81580e0 2023-10-30T12:49:44.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f90ad06b640 1 -- 172.21.15.12:0/2222691465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a81fa4c0 con 0x7f90a81580e0 2023-10-30T12:49:44.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.281+0000 7f909f7fe640 1 -- 172.21.15.12:0/2222691465 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9090004b90 con 0x7f90a81580e0 2023-10-30T12:49:44.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.285+0000 7f909f7fe640 1 --2- 172.21.15.12:0/2222691465 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f90880664d0 0x7f9088068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:44.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.285+0000 7f909f7fe640 1 -- 172.21.15.12:0/2222691465 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f9090083ba0 con 0x7f90a81580e0 2023-10-30T12:49:44.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.285+0000 7f90a6d9d640 1 --2- 172.21.15.12:0/2222691465 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f90880664d0 0x7f9088068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:44.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.285+0000 7f90a6d9d640 1 --2- 172.21.15.12:0/2222691465 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f90880664d0 0x7f9088068990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9098009af0 tx=0x7f9098008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:44.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.285+0000 7f909f7fe640 1 -- 172.21.15.12:0/2222691465 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f90a81fa4c0 con 0x7f90a81580e0 2023-10-30T12:49:44.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.477+0000 7f90ad06b640 1 -- 172.21.15.12:0/2222691465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90a81fa130 con 0x7f90a81580e0 2023-10-30T12:49:44.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.477+0000 7f909f7fe640 1 -- 172.21.15.12:0/2222691465 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5748 (secure 0 0 0) 0x7f9090050d40 con 0x7f90a81580e0 2023-10-30T12:49:44.484 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:44.484 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:42.294321+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:resolve","state_seq":24,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:44.485 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 74 2023-10-30T12:49:44.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.477+0000 7f90ad06b640 1 -- 172.21.15.12:0/2222691465 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f90880664d0 msgr2=0x7f9088068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:44.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.477+0000 7f90ad06b640 1 --2- 172.21.15.12:0/2222691465 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f90880664d0 0x7f9088068990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9098009af0 tx=0x7f9098008040 comp rx=0 tx=0).stop 2023-10-30T12:49:44.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.477+0000 7f90ad06b640 1 -- 172.21.15.12:0/2222691465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a81580e0 msgr2=0x7f90a80718f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:44.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.477+0000 7f90ad06b640 1 --2- 172.21.15.12:0/2222691465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a81580e0 0x7f90a80718f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f909000c420 tx=0x7f909000c8e0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.477+0000 7f90ad06b640 1 -- 172.21.15.12:0/2222691465 shutdown_connections 2023-10-30T12:49:44.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.477+0000 7f90ad06b640 1 --2- 172.21.15.12:0/2222691465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a815e700 0x7f90a81f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.477+0000 7f90ad06b640 1 --2- 172.21.15.12:0/2222691465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a81580e0 0x7f90a80718f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.477+0000 7f90ad06b640 1 --2- 172.21.15.12:0/2222691465 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f90880664d0 0x7f9088068990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.477+0000 7f90ad06b640 1 --2- 172.21.15.12:0/2222691465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90a80c6540 0x7f90a80c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.477+0000 7f90ad06b640 1 -- 172.21.15.12:0/2222691465 >> 172.21.15.12:0/2222691465 conn(0x7f90a80c7910 msgr2=0x7f90a80bfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:44.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.477+0000 7f90ad06b640 1 -- 172.21.15.12:0/2222691465 shutdown_connections 2023-10-30T12:49:44.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:44.477+0000 7f90ad06b640 1 -- 172.21.15.12:0/2222691465 wait complete. 2023-10-30T12:49:44.505 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:49:46.506 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:46.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.653+0000 7fb00e594640 1 Processor -- start 2023-10-30T12:49:46.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.653+0000 7fb00e594640 1 -- start start 2023-10-30T12:49:46.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.653+0000 7fb00e594640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb00806f7a0 0x7fb00806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:46.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.653+0000 7fb00e594640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb008071110 0x7fb008158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:46.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.653+0000 7fb00e594640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb008066cf0 con 0x7fb0080c6bd0 2023-10-30T12:49:46.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.653+0000 7fb00e594640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0080700c0 con 0x7fb008071110 2023-10-30T12:49:46.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.653+0000 7fb00e594640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080c70d0 con 0x7fb00806f7a0 2023-10-30T12:49:46.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.657+0000 7fb007fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb00806f7a0 0x7fb00806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:46.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.657+0000 7fb00cb31640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb008071110 0x7fb008158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:46.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.657+0000 7fb00cb31640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb008071110 0x7fb008158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:35878/0 (socket says 172.21.15.12:35878) 2023-10-30T12:49:46.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.657+0000 7fb00cb31640 1 -- 172.21.15.12:0/244265573 learned_addr learned my addr 172.21.15.12:0/244265573 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:46.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.657+0000 7fb0077fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb0080c6bd0 0x7fb008069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:54388/0 (socket says 172.21.15.12:54388) 2023-10-30T12:49:46.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.657+0000 7fb00cb31640 1 -- 172.21.15.12:0/244265573 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb00806f7a0 msgr2=0x7fb00806fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:46.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.657+0000 7fb00cb31640 1 --2- 172.21.15.12:0/244265573 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb00806f7a0 0x7fb00806fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.657+0000 7fb00cb31640 1 -- 172.21.15.12:0/244265573 >> v1:172.21.15.154:6789/0 conn(0x7fb0080c6bd0 legacy=0x7fb008069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:46.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.657+0000 7fb00cb31640 1 -- 172.21.15.12:0/244265573 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0080c7390 con 0x7fb008071110 2023-10-30T12:49:46.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.657+0000 7fb00cb31640 1 --2- 172.21.15.12:0/244265573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb008071110 0x7fb008158110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb000009e50 tx=0x7fb00001f1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2c1f380cbaedf60 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:46.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.657+0000 7fb006ffd640 1 -- 172.21.15.12:0/244265573 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0000af400 con 0x7fb008071110 2023-10-30T12:49:46.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb006ffd640 1 -- 172.21.15.12:0/244265573 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0000ab050 con 0x7fb008071110 2023-10-30T12:49:46.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb006ffd640 1 -- 172.21.15.12:0/244265573 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0000af780 con 0x7fb008071110 2023-10-30T12:49:46.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 -- 172.21.15.12:0/244265573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb008071110 msgr2=0x7fb008158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:46.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 --2- 172.21.15.12:0/244265573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb008071110 0x7fb008158110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb000009e50 tx=0x7fb00001f1f0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 -- 172.21.15.12:0/244265573 shutdown_connections 2023-10-30T12:49:46.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 --2- 172.21.15.12:0/244265573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb008071110 0x7fb008158110 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 --2- 172.21.15.12:0/244265573 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb00806f7a0 0x7fb00806fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 -- 172.21.15.12:0/244265573 >> 172.21.15.12:0/244265573 conn(0x7fb0080c7910 msgr2=0x7fb0080c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:46.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 -- 172.21.15.12:0/244265573 shutdown_connections 2023-10-30T12:49:46.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 -- 172.21.15.12:0/244265573 wait complete. 2023-10-30T12:49:46.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 Processor -- start 2023-10-30T12:49:46.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 -- start start 2023-10-30T12:49:46.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb00806f7a0 0x7fb008155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:46.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb008071110 0x7fb0080c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:46.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0080c6bd0 0x7fb00815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:46.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb008155940 con 0x7fb00806f7a0 2023-10-30T12:49:46.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080c9780 con 0x7fb0080c6bd0 2023-10-30T12:49:46.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb00e594640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080c94a0 con 0x7fb008071110 2023-10-30T12:49:46.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb007fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb00806f7a0 0x7fb008155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:46.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb007fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb00806f7a0 0x7fb008155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39552/0 (socket says 172.21.15.12:39552) 2023-10-30T12:49:46.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.661+0000 7fb007fff640 1 -- 172.21.15.12:0/2501711551 learned_addr learned my addr 172.21.15.12:0/2501711551 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:46.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb00cb31640 1 --2- 172.21.15.12:0/2501711551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0080c6bd0 0x7fb00815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:46.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb0077fe640 1 --2- 172.21.15.12:0/2501711551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb008071110 0x7fb0080c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:46.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb007fff640 1 -- 172.21.15.12:0/2501711551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb008071110 msgr2=0x7fb0080c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:46.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb007fff640 1 --2- 172.21.15.12:0/2501711551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb008071110 0x7fb0080c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb007fff640 1 -- 172.21.15.12:0/2501711551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0080c6bd0 msgr2=0x7fb00815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb007fff640 1 --2- 172.21.15.12:0/2501711551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0080c6bd0 0x7fb00815b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb007fff640 1 -- 172.21.15.12:0/2501711551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb000009b30 con 0x7fb00806f7a0 2023-10-30T12:49:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb007fff640 1 --2- 172.21.15.12:0/2501711551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb00806f7a0 0x7fb008155400 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7faff800e870 tx=0x7faff800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb004ff9640 1 -- 172.21.15.12:0/2501711551 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8009510 con 0x7fb00806f7a0 2023-10-30T12:49:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb00e594640 1 -- 172.21.15.12:0/2501711551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb008158940 con 0x7fb00806f7a0 2023-10-30T12:49:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb004ff9640 1 -- 172.21.15.12:0/2501711551 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff800eec0 con 0x7fb00806f7a0 2023-10-30T12:49:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb004ff9640 1 -- 172.21.15.12:0/2501711551 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff800cb40 con 0x7fb00806f7a0 2023-10-30T12:49:46.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb00e594640 1 -- 172.21.15.12:0/2501711551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb008158f00 con 0x7fb00806f7a0 2023-10-30T12:49:46.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb00e594640 1 -- 172.21.15.12:0/2501711551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00804e170 con 0x7fb00806f7a0 2023-10-30T12:49:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb004ff9640 1 -- 172.21.15.12:0/2501711551 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faff800cca0 con 0x7fb00806f7a0 2023-10-30T12:49:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb004ff9640 1 --2- 172.21.15.12:0/2501711551 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faff40664d0 0x7faff4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.665+0000 7fb004ff9640 1 -- 172.21.15.12:0/2501711551 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7faff8084420 con 0x7fb00806f7a0 2023-10-30T12:49:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.669+0000 7fb0077fe640 1 --2- 172.21.15.12:0/2501711551 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faff40664d0 0x7faff4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.669+0000 7fb0077fe640 1 --2- 172.21.15.12:0/2501711551 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faff40664d0 0x7faff4068990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb0080c4fc0 tx=0x7faffc007cd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.673+0000 7fb004ff9640 1 -- 172.21.15.12:0/2501711551 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faff8051420 con 0x7fb00806f7a0 2023-10-30T12:49:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.861+0000 7fb00e594640 1 -- 172.21.15.12:0/2501711551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb008066510 con 0x7fb00806f7a0 2023-10-30T12:49:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.861+0000 7fb004ff9640 1 -- 172.21.15.12:0/2501711551 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5748 (secure 0 0 0) 0x7faff8054a40 con 0x7fb00806f7a0 2023-10-30T12:49:46.870 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:46.870 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:42.294321+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:resolve","state_seq":24,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:46.870 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 74 2023-10-30T12:49:46.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.865+0000 7fb00e594640 1 -- 172.21.15.12:0/2501711551 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faff40664d0 msgr2=0x7faff4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:46.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.865+0000 7fb00e594640 1 --2- 172.21.15.12:0/2501711551 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faff40664d0 0x7faff4068990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb0080c4fc0 tx=0x7faffc007cd0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.865+0000 7fb00e594640 1 -- 172.21.15.12:0/2501711551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb00806f7a0 msgr2=0x7fb008155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:46.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.865+0000 7fb00e594640 1 --2- 172.21.15.12:0/2501711551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb00806f7a0 0x7fb008155400 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7faff800e870 tx=0x7faff800ec70 comp rx=0 tx=0).stop 2023-10-30T12:49:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.865+0000 7fb00e594640 1 -- 172.21.15.12:0/2501711551 shutdown_connections 2023-10-30T12:49:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.865+0000 7fb00e594640 1 --2- 172.21.15.12:0/2501711551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0080c6bd0 0x7fb00815b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.865+0000 7fb00e594640 1 --2- 172.21.15.12:0/2501711551 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faff40664d0 0x7faff4068990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.865+0000 7fb00e594640 1 --2- 172.21.15.12:0/2501711551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb008071110 0x7fb0080c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.865+0000 7fb00e594640 1 --2- 172.21.15.12:0/2501711551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb00806f7a0 0x7fb008155400 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.865+0000 7fb00e594640 1 -- 172.21.15.12:0/2501711551 >> 172.21.15.12:0/2501711551 conn(0x7fb0080c7910 msgr2=0x7fb0080ca5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.865+0000 7fb00e594640 1 -- 172.21.15.12:0/2501711551 shutdown_connections 2023-10-30T12:49:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:46.865+0000 7fb00e594640 1 -- 172.21.15.12:0/2501711551 wait complete. 2023-10-30T12:49:46.891 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:49:48.892 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:49.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8b003d3640 1 Processor -- start 2023-10-30T12:49:49.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8b003d3640 1 -- start start 2023-10-30T12:49:49.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8b003d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af815d6f0 0x7f8af815a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:49.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8b003d3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8af80cf160 0x7f8af80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:49.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8b003d3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af80cb800 con 0x7f8af81538a0 2023-10-30T12:49:49.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8b003d3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af80cb330 con 0x7f8af80cf160 2023-10-30T12:49:49.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8b003d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af80cad70 con 0x7f8af815d6f0 2023-10-30T12:49:49.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afe16f640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8af81538a0 0x7f8af80ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40852/0 (socket says 172.21.15.12:40852) 2023-10-30T12:49:49.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afe16f640 1 -- 172.21.15.12:0/42541069 learned_addr learned my addr 172.21.15.12:0/42541069 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:49:49.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afe970640 1 --2- 172.21.15.12:0/42541069 >> v2:172.21.15.154:3300/0 conn(0x7f8af80cf160 0x7f8af80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:49.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd96e640 1 --2- 172.21.15.12:0/42541069 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af815d6f0 0x7f8af815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:49.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 -- 172.21.15.12:0/42541069 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374430218 0 0) 0x7f8af80cb800 con 0x7f8af81538a0 2023-10-30T12:49:49.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 -- 172.21.15.12:0/42541069 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ae4003530 con 0x7f8af81538a0 2023-10-30T12:49:49.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 -- 172.21.15.12:0/42541069 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1436223218 0 0) 0x7f8ae4003530 con 0x7f8af81538a0 2023-10-30T12:49:49.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 -- 172.21.15.12:0/42541069 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af815d6f0 msgr2=0x7f8af815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:49.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 --2- 172.21.15.12:0/42541069 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af815d6f0 0x7f8af815a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 -- 172.21.15.12:0/42541069 >> v2:172.21.15.154:3300/0 conn(0x7f8af80cf160 msgr2=0x7f8af80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:49.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 --2- 172.21.15.12:0/42541069 >> v2:172.21.15.154:3300/0 conn(0x7f8af80cf160 0x7f8af80cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 -- 172.21.15.12:0/42541069 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af80cb050 con 0x7f8af81538a0 2023-10-30T12:49:49.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 -- 172.21.15.12:0/42541069 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f8af4001680 con 0x7f8af81538a0 2023-10-30T12:49:49.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 -- 172.21.15.12:0/42541069 >> v1:172.21.15.12:6789/0 conn(0x7f8af81538a0 legacy=0x7f8af80ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:49.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 --2- 172.21.15.12:0/42541069 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af815d6f0 0x7f8ae4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:49.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 --2- 172.21.15.12:0/42541069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ae4003f00 0x7f8ae40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:49.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 --2- 172.21.15.12:0/42541069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4004d10 0x7f8ae4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:49.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 -- 172.21.15.12:0/42541069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af80cb330 con 0x7f8ae4003f00 2023-10-30T12:49:49.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 -- 172.21.15.12:0/42541069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af80cad70 con 0x7f8ae4004d10 2023-10-30T12:49:49.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd16d640 1 -- 172.21.15.12:0/42541069 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae400b9c0 con 0x7f8af815d6f0 2023-10-30T12:49:49.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.041+0000 7f8afd96e640 1 --2- 172.21.15.12:0/42541069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ae4003f00 0x7f8ae40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:49.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.045+0000 7f8afe16f640 1 --2- 172.21.15.12:0/42541069 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af815d6f0 0x7f8ae4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:49.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.045+0000 7f8afe970640 1 --2- 172.21.15.12:0/42541069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4004d10 0x7f8ae4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:49.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.045+0000 7f8afd96e640 1 -- 172.21.15.12:0/42541069 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af815d6f0 msgr2=0x7f8ae4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:49.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.045+0000 7f8afd96e640 1 --2- 172.21.15.12:0/42541069 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af815d6f0 0x7f8ae4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.045+0000 7f8afd96e640 1 -- 172.21.15.12:0/42541069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4004d10 msgr2=0x7f8ae4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:49.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.045+0000 7f8afd96e640 1 --2- 172.21.15.12:0/42541069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4004d10 0x7f8ae4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.045+0000 7f8afd96e640 1 -- 172.21.15.12:0/42541069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae4003530 con 0x7f8ae4003f00 2023-10-30T12:49:49.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.045+0000 7f8afd96e640 1 --2- 172.21.15.12:0/42541069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ae4003f00 0x7f8ae40047d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009ae0 tx=0x7f8ae801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72174732e42b6b04 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:49.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.045+0000 7f8afd16d640 1 -- 172.21.15.12:0/42541069 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80a5640 con 0x7f8ae4003f00 2023-10-30T12:49:49.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.045+0000 7f8afd16d640 1 -- 172.21.15.12:0/42541069 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae80a57a0 con 0x7f8ae4003f00 2023-10-30T12:49:49.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.045+0000 7f8afd16d640 1 -- 172.21.15.12:0/42541069 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80a5b50 con 0x7f8ae4003f00 2023-10-30T12:49:49.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.045+0000 7f8b003d3640 1 -- 172.21.15.12:0/42541069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ae4003f00 msgr2=0x7f8ae40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:49.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.045+0000 7f8b003d3640 1 --2- 172.21.15.12:0/42541069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ae4003f00 0x7f8ae40047d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009ae0 tx=0x7f8ae801ca50 comp rx=0 tx=0).stop 2023-10-30T12:49:49.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.045+0000 7f8afe970640 1 -- 172.21.15.12:0/42541069 reap_dead start 2023-10-30T12:49:49.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8b003d3640 1 -- 172.21.15.12:0/42541069 shutdown_connections 2023-10-30T12:49:49.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8b003d3640 1 -- 172.21.15.12:0/42541069 >> 172.21.15.12:0/42541069 conn(0x7f8af806bfe0 msgr2=0x7f8af806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:49.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8b003d3640 1 -- 172.21.15.12:0/42541069 shutdown_connections 2023-10-30T12:49:49.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8b003d3640 1 -- 172.21.15.12:0/42541069 wait complete. 2023-10-30T12:49:49.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8b003d3640 1 Processor -- start 2023-10-30T12:49:49.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8b003d3640 1 -- start start 2023-10-30T12:49:49.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8b003d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4003f00 0x7f8af814e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:49.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8b003d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af80b4dd0 0x7f8af8065ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:49.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8b003d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8af814cb40 0x7f8af814cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:49.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8b003d3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af814d500 con 0x7f8af814cb40 2023-10-30T12:49:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8b003d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af80b41b0 con 0x7f8ae4003f00 2023-10-30T12:49:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8b003d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af80b3e80 con 0x7f8af80b4dd0 2023-10-30T12:49:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8afe970640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8af814cb40 0x7f8af814cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8afe970640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8af814cb40 0x7f8af814cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39572/0 (socket says 172.21.15.12:39572) 2023-10-30T12:49:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8afe970640 1 -- 172.21.15.12:0/746933994 learned_addr learned my addr 172.21.15.12:0/746933994 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:49.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8afd96e640 1 --2- 172.21.15.12:0/746933994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af80b4dd0 0x7f8af8065ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:49.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8afe16f640 1 --2- 172.21.15.12:0/746933994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4003f00 0x7f8af814e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:49.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8afe970640 1 -- 172.21.15.12:0/746933994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af80b4dd0 msgr2=0x7f8af8065ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:49.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8afe970640 1 --2- 172.21.15.12:0/746933994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af80b4dd0 0x7f8af8065ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8afe970640 1 -- 172.21.15.12:0/746933994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4003f00 msgr2=0x7f8af814e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:49.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8afe970640 1 --2- 172.21.15.12:0/746933994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4003f00 0x7f8af814e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8afe970640 1 -- 172.21.15.12:0/746933994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af80b3700 con 0x7f8af814cb40 2023-10-30T12:49:49.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.049+0000 7f8afe970640 1 --2- 172.21.15.12:0/746933994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8af814cb40 0x7f8af814cfc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8af80caeb0 tx=0x7f8af00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:49.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.053+0000 7f8aeeffd640 1 -- 172.21.15.12:0/746933994 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af000e070 con 0x7f8af814cb40 2023-10-30T12:49:49.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.053+0000 7f8aeeffd640 1 -- 172.21.15.12:0/746933994 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af0011e10 con 0x7f8af814cb40 2023-10-30T12:49:49.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.053+0000 7f8aeeffd640 1 -- 172.21.15.12:0/746933994 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af00039a0 con 0x7f8af814cb40 2023-10-30T12:49:49.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.053+0000 7f8b003d3640 1 -- 172.21.15.12:0/746933994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af81f31c0 con 0x7f8af814cb40 2023-10-30T12:49:49.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.053+0000 7f8b003d3640 1 -- 172.21.15.12:0/746933994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af814d690 con 0x7f8af814cb40 2023-10-30T12:49:49.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.053+0000 7f8b003d3640 1 -- 172.21.15.12:0/746933994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af80b3fc0 con 0x7f8af814cb40 2023-10-30T12:49:49.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.053+0000 7f8aeeffd640 1 -- 172.21.15.12:0/746933994 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8af0003b00 con 0x7f8af814cb40 2023-10-30T12:49:49.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.053+0000 7f8aeeffd640 1 --2- 172.21.15.12:0/746933994 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8adc0664a0 0x7f8adc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:49.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.053+0000 7f8aeeffd640 1 -- 172.21.15.12:0/746933994 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f8af00845d0 con 0x7f8af814cb40 2023-10-30T12:49:49.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.053+0000 7f8afe16f640 1 --2- 172.21.15.12:0/746933994 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8adc0664a0 0x7f8adc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:49.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.057+0000 7f8afe16f640 1 --2- 172.21.15.12:0/746933994 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8adc0664a0 0x7f8adc068960 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8af4000fc0 tx=0x7f8af4007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:49.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.057+0000 7f8aeeffd640 1 -- 172.21.15.12:0/746933994 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8af0051820 con 0x7f8af814cb40 2023-10-30T12:49:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.249+0000 7f8b003d3640 1 -- 172.21.15.12:0/746933994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8af804e170 con 0x7f8af814cb40 2023-10-30T12:49:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.249+0000 7f8aeeffd640 1 -- 172.21.15.12:0/746933994 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5750 (secure 0 0 0) 0x7f8af0054e40 con 0x7f8af814cb40 2023-10-30T12:49:49.256 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:49.256 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:47.444512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:reconnect","state_seq":26,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:49.256 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 75 2023-10-30T12:49:49.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.249+0000 7f8b003d3640 1 -- 172.21.15.12:0/746933994 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8adc0664a0 msgr2=0x7f8adc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:49.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.249+0000 7f8b003d3640 1 --2- 172.21.15.12:0/746933994 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8adc0664a0 0x7f8adc068960 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8af4000fc0 tx=0x7f8af4007040 comp rx=0 tx=0).stop 2023-10-30T12:49:49.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.249+0000 7f8b003d3640 1 -- 172.21.15.12:0/746933994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8af814cb40 msgr2=0x7f8af814cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:49.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.249+0000 7f8b003d3640 1 --2- 172.21.15.12:0/746933994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8af814cb40 0x7f8af814cfc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8af80caeb0 tx=0x7f8af00115a0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.249+0000 7f8b003d3640 1 -- 172.21.15.12:0/746933994 shutdown_connections 2023-10-30T12:49:49.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.249+0000 7f8b003d3640 1 --2- 172.21.15.12:0/746933994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8af814cb40 0x7f8af814cfc0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.249+0000 7f8b003d3640 1 --2- 172.21.15.12:0/746933994 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8adc0664a0 0x7f8adc068960 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.249+0000 7f8b003d3640 1 --2- 172.21.15.12:0/746933994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af80b4dd0 0x7f8af8065ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.249+0000 7f8b003d3640 1 --2- 172.21.15.12:0/746933994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4003f00 0x7f8af814e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.249+0000 7f8b003d3640 1 -- 172.21.15.12:0/746933994 >> 172.21.15.12:0/746933994 conn(0x7f8af806bfe0 msgr2=0x7f8af815b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:49.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.249+0000 7f8b003d3640 1 -- 172.21.15.12:0/746933994 shutdown_connections 2023-10-30T12:49:49.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:49.249+0000 7f8b003d3640 1 -- 172.21.15.12:0/746933994 wait complete. 2023-10-30T12:49:49.278 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 75, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 75, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5427": { "addr": "172.21.15.12:6836/2219276371", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 2219276371, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 2219276371, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 5427, "incarnation": 57, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 31 }, "gid_5571": { "addr": "172.21.15.12:6837/2752088514", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2752088514, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 2752088514, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5571, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 19 }, "gid_5758": { "addr": "172.21.15.154:6839/2204547612", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 2204547612, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 2204547612, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5758, "incarnation": 73, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:reconnect", "state_seq": 26 } }, "last_failure": 0, "last_failure_osd_epoch": 52, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:49:47.444512+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5571, "mds_1": 5758, "mds_2": 5427 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:49:51.280 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:51.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.429+0000 7fda2ac9c640 1 Processor -- start 2023-10-30T12:49:51.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.429+0000 7fda2ac9c640 1 -- start start 2023-10-30T12:49:51.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.429+0000 7fda2ac9c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fda240cf160 0x7fda240cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:51.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.429+0000 7fda2ac9c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda240cb800 con 0x7fda241538a0 2023-10-30T12:49:51.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.429+0000 7fda2ac9c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda240cb330 con 0x7fda2415d6f0 2023-10-30T12:49:51.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.429+0000 7fda2ac9c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda240cad70 con 0x7fda240cf160 2023-10-30T12:49:51.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.429+0000 7fda29239640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fda240cf160 0x7fda240cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:51.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.429+0000 7fda29239640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fda240cf160 0x7fda240cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44786/0 (socket says 172.21.15.12:44786) 2023-10-30T12:49:51.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.429+0000 7fda29239640 1 -- 172.21.15.12:0/1718738574 learned_addr learned my addr 172.21.15.12:0/1718738574 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:51.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.429+0000 7fda28a38640 1 --1- 172.21.15.12:0/1718738574 >> v1:172.21.15.12:6789/0 conn(0x7fda241538a0 0x7fda240ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49874/0 (socket says 172.21.15.12:49874) 2023-10-30T12:49:51.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.429+0000 7fda29239640 1 -- 172.21.15.12:0/1718738574 >> v1:172.21.15.154:6789/0 conn(0x7fda2415d6f0 legacy=0x7fda2415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:51.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.429+0000 7fda29239640 1 -- 172.21.15.12:0/1718738574 >> v1:172.21.15.12:6789/0 conn(0x7fda241538a0 legacy=0x7fda240ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:51.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.429+0000 7fda29239640 1 -- 172.21.15.12:0/1718738574 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda240cb050 con 0x7fda240cf160 2023-10-30T12:49:51.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.433+0000 7fda29239640 1 --2- 172.21.15.12:0/1718738574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda240cf160 0x7fda240cf590 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009e50 tx=0x7fda1c01f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7917581c72d7d231 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:51.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.433+0000 7fda237fe640 1 -- 172.21.15.12:0/1718738574 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c0af400 con 0x7fda240cf160 2023-10-30T12:49:51.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.433+0000 7fda237fe640 1 -- 172.21.15.12:0/1718738574 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1c0ab050 con 0x7fda240cf160 2023-10-30T12:49:51.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.433+0000 7fda237fe640 1 -- 172.21.15.12:0/1718738574 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c0af780 con 0x7fda240cf160 2023-10-30T12:49:51.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.433+0000 7fda2ac9c640 1 -- 172.21.15.12:0/1718738574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda240cf160 msgr2=0x7fda240cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:51.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.433+0000 7fda2ac9c640 1 --2- 172.21.15.12:0/1718738574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda240cf160 0x7fda240cf590 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009e50 tx=0x7fda1c01f1f0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.433+0000 7fda2ac9c640 1 -- 172.21.15.12:0/1718738574 shutdown_connections 2023-10-30T12:49:51.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.433+0000 7fda2ac9c640 1 --2- 172.21.15.12:0/1718738574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda240cf160 0x7fda240cf590 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.433+0000 7fda2ac9c640 1 -- 172.21.15.12:0/1718738574 >> 172.21.15.12:0/1718738574 conn(0x7fda2406bfe0 msgr2=0x7fda2406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:51.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.433+0000 7fda2ac9c640 1 -- 172.21.15.12:0/1718738574 shutdown_connections 2023-10-30T12:49:51.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.433+0000 7fda2ac9c640 1 -- 172.21.15.12:0/1718738574 wait complete. 2023-10-30T12:49:51.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda2ac9c640 1 Processor -- start 2023-10-30T12:49:51.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda2ac9c640 1 -- start start 2023-10-30T12:49:51.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda2ac9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda240cf160 0x7fda241586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:51.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda2ac9c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda241538a0 0x7fda24152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:51.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda2ac9c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda2415d6f0 0x7fda241ef420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:51.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda2ac9c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda241e2c80 con 0x7fda240cf160 2023-10-30T12:49:51.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda2ac9c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda24157fe0 con 0x7fda241538a0 2023-10-30T12:49:51.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda2ac9c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda241582c0 con 0x7fda2415d6f0 2023-10-30T12:49:51.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda28a38640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda240cf160 0x7fda241586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda28a38640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda240cf160 0x7fda241586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44800/0 (socket says 172.21.15.12:44800) 2023-10-30T12:49:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda28a38640 1 -- 172.21.15.12:0/655861836 learned_addr learned my addr 172.21.15.12:0/655861836 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda23fff640 1 --2- 172.21.15.12:0/655861836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda241538a0 0x7fda24152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda29239640 1 --2- 172.21.15.12:0/655861836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda2415d6f0 0x7fda241ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda28a38640 1 -- 172.21.15.12:0/655861836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda2415d6f0 msgr2=0x7fda241ef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda28a38640 1 --2- 172.21.15.12:0/655861836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda2415d6f0 0x7fda241ef420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda28a38640 1 -- 172.21.15.12:0/655861836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda241538a0 msgr2=0x7fda24152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda28a38640 1 --2- 172.21.15.12:0/655861836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda241538a0 0x7fda24152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda28a38640 1 -- 172.21.15.12:0/655861836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c009b30 con 0x7fda240cf160 2023-10-30T12:49:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda28a38640 1 --2- 172.21.15.12:0/655861836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda240cf160 0x7fda241586a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fda1800c490 tx=0x7fda1800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda217fa640 1 -- 172.21.15.12:0/655861836 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18007480 con 0x7fda240cf160 2023-10-30T12:49:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda217fa640 1 -- 172.21.15.12:0/655861836 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda180075e0 con 0x7fda240cf160 2023-10-30T12:49:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda2ac9c640 1 -- 172.21.15.12:0/655861836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda241f33a0 con 0x7fda240cf160 2023-10-30T12:49:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda217fa640 1 -- 172.21.15.12:0/655861836 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18004a10 con 0x7fda240cf160 2023-10-30T12:49:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.437+0000 7fda2ac9c640 1 -- 172.21.15.12:0/655861836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda24152960 con 0x7fda240cf160 2023-10-30T12:49:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.441+0000 7fda2ac9c640 1 -- 172.21.15.12:0/655861836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2404e170 con 0x7fda240cf160 2023-10-30T12:49:51.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.441+0000 7fda217fa640 1 -- 172.21.15.12:0/655861836 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fda18004b70 con 0x7fda240cf160 2023-10-30T12:49:51.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.441+0000 7fda217fa640 1 --2- 172.21.15.12:0/655861836 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda080664d0 0x7fda08068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:51.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.441+0000 7fda217fa640 1 -- 172.21.15.12:0/655861836 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fda180836a0 con 0x7fda240cf160 2023-10-30T12:49:51.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.441+0000 7fda23fff640 1 --2- 172.21.15.12:0/655861836 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda080664d0 0x7fda08068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:51.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.441+0000 7fda23fff640 1 --2- 172.21.15.12:0/655861836 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda080664d0 0x7fda08068990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fda241ed050 tx=0x7fda10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:51.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.445+0000 7fda217fa640 1 -- 172.21.15.12:0/655861836 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fda18050720 con 0x7fda240cf160 2023-10-30T12:49:51.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.637+0000 7fda2ac9c640 1 -- 172.21.15.12:0/655861836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda240b1dd0 con 0x7fda240cf160 2023-10-30T12:49:51.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.637+0000 7fda217fa640 1 -- 172.21.15.12:0/655861836 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5750 (secure 0 0 0) 0x7fda18053d40 con 0x7fda240cf160 2023-10-30T12:49:51.644 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:51.644 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:47.444512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:reconnect","state_seq":26,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:51.646 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 75 2023-10-30T12:49:51.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.637+0000 7fda2ac9c640 1 -- 172.21.15.12:0/655861836 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda080664d0 msgr2=0x7fda08068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:51.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.637+0000 7fda2ac9c640 1 --2- 172.21.15.12:0/655861836 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda080664d0 0x7fda08068990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fda241ed050 tx=0x7fda10009040 comp rx=0 tx=0).stop 2023-10-30T12:49:51.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.637+0000 7fda2ac9c640 1 -- 172.21.15.12:0/655861836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda240cf160 msgr2=0x7fda241586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:51.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.637+0000 7fda2ac9c640 1 --2- 172.21.15.12:0/655861836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda240cf160 0x7fda241586a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fda1800c490 tx=0x7fda1800c950 comp rx=0 tx=0).stop 2023-10-30T12:49:51.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.637+0000 7fda2ac9c640 1 -- 172.21.15.12:0/655861836 shutdown_connections 2023-10-30T12:49:51.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.637+0000 7fda2ac9c640 1 --2- 172.21.15.12:0/655861836 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fda080664d0 0x7fda08068990 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.637+0000 7fda2ac9c640 1 --2- 172.21.15.12:0/655861836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda2415d6f0 0x7fda241ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.637+0000 7fda2ac9c640 1 --2- 172.21.15.12:0/655861836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda241538a0 0x7fda24152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.637+0000 7fda2ac9c640 1 --2- 172.21.15.12:0/655861836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda240cf160 0x7fda241586a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.637+0000 7fda2ac9c640 1 -- 172.21.15.12:0/655861836 >> 172.21.15.12:0/655861836 conn(0x7fda2406bfe0 msgr2=0x7fda240ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:51.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.637+0000 7fda2ac9c640 1 -- 172.21.15.12:0/655861836 shutdown_connections 2023-10-30T12:49:51.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:51.637+0000 7fda2ac9c640 1 -- 172.21.15.12:0/655861836 wait complete. 2023-10-30T12:49:51.666 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:49:53.666 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:53.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.813+0000 7f34e5468640 1 Processor -- start 2023-10-30T12:49:53.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.813+0000 7f34e5468640 1 -- start start 2023-10-30T12:49:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.813+0000 7f34e5468640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34e01538a0 0x7f34e015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.813+0000 7f34e5468640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34e00ca0e0 0x7f34e015a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.813+0000 7f34e5468640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e00cb800 con 0x7f34e00cf160 2023-10-30T12:49:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.813+0000 7f34e5468640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e015dc30 con 0x7f34e01538a0 2023-10-30T12:49:53.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.813+0000 7f34e5468640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e00cb330 con 0x7f34e00ca0e0 2023-10-30T12:49:53.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.817+0000 7f34deffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34e01538a0 0x7f34e015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:53.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.817+0000 7f34d7fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34e00ca0e0 0x7f34e015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:53.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.817+0000 7f34d7fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34e00ca0e0 0x7f34e015a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:45398/0 (socket says 172.21.15.12:45398) 2023-10-30T12:49:53.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.817+0000 7f34d7fff640 1 -- 172.21.15.12:0/3617302536 learned_addr learned my addr 172.21.15.12:0/3617302536 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:53.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.817+0000 7f34df7fe640 1 --1- 172.21.15.12:0/3617302536 >> v1:172.21.15.154:6789/0 conn(0x7f34e00cf160 0x7f34e00cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:39938/0 (socket says 172.21.15.12:39938) 2023-10-30T12:49:53.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.817+0000 7f34d7fff640 1 -- 172.21.15.12:0/3617302536 >> v2:172.21.15.154:3300/0 conn(0x7f34e01538a0 msgr2=0x7f34e015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:53.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.817+0000 7f34d7fff640 1 --2- 172.21.15.12:0/3617302536 >> v2:172.21.15.154:3300/0 conn(0x7f34e01538a0 0x7f34e015d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:53.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.817+0000 7f34d7fff640 1 -- 172.21.15.12:0/3617302536 >> v1:172.21.15.154:6789/0 conn(0x7f34e00cf160 legacy=0x7f34e00cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:53.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.817+0000 7f34d7fff640 1 -- 172.21.15.12:0/3617302536 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e00cad70 con 0x7f34e00ca0e0 2023-10-30T12:49:53.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.817+0000 7f34deffd640 1 --2- 172.21.15.12:0/3617302536 >> v2:172.21.15.154:3300/0 conn(0x7f34e01538a0 0x7f34e015d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:49:53.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34d7fff640 1 --2- 172.21.15.12:0/3617302536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34e00ca0e0 0x7f34e015a090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009870 tx=0x7f34c801ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3671be85f1605fe1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34de7fc640 1 -- 172.21.15.12:0/3617302536 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c80a8a30 con 0x7f34e00ca0e0 2023-10-30T12:49:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34de7fc640 1 -- 172.21.15.12:0/3617302536 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c80a8b90 con 0x7f34e00ca0e0 2023-10-30T12:49:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34de7fc640 1 -- 172.21.15.12:0/3617302536 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c80b0440 con 0x7f34e00ca0e0 2023-10-30T12:49:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 -- 172.21.15.12:0/3617302536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34e00ca0e0 msgr2=0x7f34e015a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 --2- 172.21.15.12:0/3617302536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34e00ca0e0 0x7f34e015a090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009870 tx=0x7f34c801ed30 comp rx=0 tx=0).stop 2023-10-30T12:49:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 -- 172.21.15.12:0/3617302536 shutdown_connections 2023-10-30T12:49:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 --2- 172.21.15.12:0/3617302536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34e00ca0e0 0x7f34e015a090 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 --2- 172.21.15.12:0/3617302536 >> v2:172.21.15.154:3300/0 conn(0x7f34e01538a0 0x7f34e015d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 -- 172.21.15.12:0/3617302536 >> 172.21.15.12:0/3617302536 conn(0x7f34e006bfe0 msgr2=0x7f34e006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 -- 172.21.15.12:0/3617302536 shutdown_connections 2023-10-30T12:49:53.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 -- 172.21.15.12:0/3617302536 wait complete. 2023-10-30T12:49:53.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 Processor -- start 2023-10-30T12:49:53.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 -- start start 2023-10-30T12:49:53.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e00ca0e0 0x7f34e01f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:53.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34e00cf160 0x7f34e01f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:53.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34e01538a0 0x7f34e01f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e0159710 con 0x7f34e00ca0e0 2023-10-30T12:49:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e00c4c40 con 0x7f34e00cf160 2023-10-30T12:49:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.821+0000 7f34e5468640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e0159280 con 0x7f34e01538a0 2023-10-30T12:49:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34deffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e00ca0e0 0x7f34e01f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34deffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e00ca0e0 0x7f34e01f2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44808/0 (socket says 172.21.15.12:44808) 2023-10-30T12:49:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34deffd640 1 -- 172.21.15.12:0/1412096875 learned_addr learned my addr 172.21.15.12:0/1412096875 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34df7fe640 1 --2- 172.21.15.12:0/1412096875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34e01538a0 0x7f34e01f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34d7fff640 1 --2- 172.21.15.12:0/1412096875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34e00cf160 0x7f34e01f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34deffd640 1 -- 172.21.15.12:0/1412096875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34e01538a0 msgr2=0x7f34e01f33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34deffd640 1 --2- 172.21.15.12:0/1412096875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34e01538a0 0x7f34e01f33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34deffd640 1 -- 172.21.15.12:0/1412096875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34e00cf160 msgr2=0x7f34e01f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34deffd640 1 --2- 172.21.15.12:0/1412096875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34e00cf160 0x7f34e01f2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34deffd640 1 -- 172.21.15.12:0/1412096875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d0009420 con 0x7f34e00ca0e0 2023-10-30T12:49:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34deffd640 1 --2- 172.21.15.12:0/1412096875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e00ca0e0 0x7f34e01f2960 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f34d000e980 tx=0x7f34d000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:53.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34d77fe640 1 -- 172.21.15.12:0/1412096875 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d000ca00 con 0x7f34e00ca0e0 2023-10-30T12:49:53.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34d77fe640 1 -- 172.21.15.12:0/1412096875 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d000cb60 con 0x7f34e00ca0e0 2023-10-30T12:49:53.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34d77fe640 1 -- 172.21.15.12:0/1412096875 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d0013470 con 0x7f34e00ca0e0 2023-10-30T12:49:53.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34e5468640 1 -- 172.21.15.12:0/1412096875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c80094d0 con 0x7f34e00ca0e0 2023-10-30T12:49:53.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34e5468640 1 -- 172.21.15.12:0/1412096875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e015ad30 con 0x7f34e00ca0e0 2023-10-30T12:49:53.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34e5468640 1 -- 172.21.15.12:0/1412096875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e00b2a50 con 0x7f34e00ca0e0 2023-10-30T12:49:53.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.825+0000 7f34d77fe640 1 -- 172.21.15.12:0/1412096875 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f34d00135d0 con 0x7f34e00ca0e0 2023-10-30T12:49:53.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.829+0000 7f34d77fe640 1 --2- 172.21.15.12:0/1412096875 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34c00664a0 0x7f34c0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:53.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.829+0000 7f34d77fe640 1 -- 172.21.15.12:0/1412096875 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f34d0084410 con 0x7f34e00ca0e0 2023-10-30T12:49:53.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.829+0000 7f34d7fff640 1 --2- 172.21.15.12:0/1412096875 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34c00664a0 0x7f34c0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:53.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.829+0000 7f34d7fff640 1 --2- 172.21.15.12:0/1412096875 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34c00664a0 0x7f34c0068960 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f34c80b2000 tx=0x7f34c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:53.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:53.833+0000 7f34d77fe640 1 -- 172.21.15.12:0/1412096875 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f34d0051520 con 0x7f34e00ca0e0 2023-10-30T12:49:54.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:54.021+0000 7f34e5468640 1 -- 172.21.15.12:0/1412096875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34e004e170 con 0x7f34e00ca0e0 2023-10-30T12:49:54.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:54.021+0000 7f34d77fe640 1 -- 172.21.15.12:0/1412096875 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+5747 (secure 0 0 0) 0x7f34d0054b40 con 0x7f34e00ca0e0 2023-10-30T12:49:54.030 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:54.031 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:52.465384+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:54.032 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 76 2023-10-30T12:49:54.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:54.025+0000 7f34e5468640 1 -- 172.21.15.12:0/1412096875 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34c00664a0 msgr2=0x7f34c0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:54.025+0000 7f34e5468640 1 --2- 172.21.15.12:0/1412096875 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34c00664a0 0x7f34c0068960 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f34c80b2000 tx=0x7f34c8008040 comp rx=0 tx=0).stop 2023-10-30T12:49:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:54.025+0000 7f34e5468640 1 -- 172.21.15.12:0/1412096875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e00ca0e0 msgr2=0x7f34e01f2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:54.025+0000 7f34e5468640 1 --2- 172.21.15.12:0/1412096875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e00ca0e0 0x7f34e01f2960 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f34d000e980 tx=0x7f34d000ee40 comp rx=0 tx=0).stop 2023-10-30T12:49:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:54.025+0000 7f34e5468640 1 -- 172.21.15.12:0/1412096875 shutdown_connections 2023-10-30T12:49:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:54.025+0000 7f34e5468640 1 --2- 172.21.15.12:0/1412096875 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34c00664a0 0x7f34c0068960 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:54.025+0000 7f34e5468640 1 --2- 172.21.15.12:0/1412096875 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34e01538a0 0x7f34e01f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:54.025+0000 7f34e5468640 1 --2- 172.21.15.12:0/1412096875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34e00cf160 0x7f34e01f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:54.025+0000 7f34e5468640 1 --2- 172.21.15.12:0/1412096875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e00ca0e0 0x7f34e01f2960 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:54.025+0000 7f34e5468640 1 -- 172.21.15.12:0/1412096875 >> 172.21.15.12:0/1412096875 conn(0x7f34e006bfe0 msgr2=0x7f34e0145e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:54.025+0000 7f34e5468640 1 -- 172.21.15.12:0/1412096875 shutdown_connections 2023-10-30T12:49:54.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:54.025+0000 7f34e5468640 1 -- 172.21.15.12:0/1412096875 wait complete. 2023-10-30T12:49:54.051 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:49:56.052 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:56.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.201+0000 7f27fd8d6640 1 Processor -- start 2023-10-30T12:49:56.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.201+0000 7f27fd8d6640 1 -- start start 2023-10-30T12:49:56.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.201+0000 7f27fd8d6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f27f81538a0 0x7f27f815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:56.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.201+0000 7f27fd8d6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f80cb800 con 0x7f27f80cf160 2023-10-30T12:49:56.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.201+0000 7f27fd8d6640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f815dc30 con 0x7f27f80ca0e0 2023-10-30T12:49:56.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.201+0000 7f27fd8d6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f80cb330 con 0x7f27f81538a0 2023-10-30T12:49:56.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.201+0000 7f27f6ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f27f81538a0 0x7f27f815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:56.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.201+0000 7f27f77fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f27f80cf160 0x7f27f8164070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49878/0 (socket says 172.21.15.12:49878) 2023-10-30T12:49:56.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.201+0000 7f27f6ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f27f81538a0 0x7f27f815d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44814/0 (socket says 172.21.15.12:44814) 2023-10-30T12:49:56.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.201+0000 7f27f77fe640 1 -- 172.21.15.12:0/1358284623 learned_addr learned my addr 172.21.15.12:0/1358284623 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:49:56.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.201+0000 7f27f6ffd640 1 -- 172.21.15.12:0/1358284623 >> v1:172.21.15.154:6789/0 conn(0x7f27f80ca0e0 legacy=0x7f27f815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:56.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.201+0000 7f27f6ffd640 1 -- 172.21.15.12:0/1358284623 >> v1:172.21.15.12:6789/0 conn(0x7f27f80cf160 legacy=0x7f27f8164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:56.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.201+0000 7f27f67fc640 1 -- 172.21.15.12:0/1358284623 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274675071 0 0) 0x7f27f80cb800 con 0x7f27f80cf160 2023-10-30T12:49:56.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.201+0000 7f27f6ffd640 1 -- 172.21.15.12:0/1358284623 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f80cad70 con 0x7f27f81538a0 2023-10-30T12:49:56.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.205+0000 7f27f6ffd640 1 --2- 172.21.15.12:0/1358284623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27f81538a0 0x7f27f815d6f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f27e80097c0 tx=0x7f27e801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d52ba9ccf8625b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:56.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.205+0000 7f27f67fc640 1 -- 172.21.15.12:0/1358284623 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e80a8970 con 0x7f27f81538a0 2023-10-30T12:49:56.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.205+0000 7f27f67fc640 1 -- 172.21.15.12:0/1358284623 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e80a8ad0 con 0x7f27f81538a0 2023-10-30T12:49:56.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.205+0000 7f27f67fc640 1 -- 172.21.15.12:0/1358284623 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e80a8e60 con 0x7f27f81538a0 2023-10-30T12:49:56.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.205+0000 7f27fd8d6640 1 -- 172.21.15.12:0/1358284623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27f81538a0 msgr2=0x7f27f815d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:56.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.205+0000 7f27fd8d6640 1 --2- 172.21.15.12:0/1358284623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27f81538a0 0x7f27f815d6f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f27e80097c0 tx=0x7f27e801ec70 comp rx=0 tx=0).stop 2023-10-30T12:49:56.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.205+0000 7f27fd8d6640 1 -- 172.21.15.12:0/1358284623 shutdown_connections 2023-10-30T12:49:56.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.205+0000 7f27fd8d6640 1 --2- 172.21.15.12:0/1358284623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27f81538a0 0x7f27f815d6f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:56.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.205+0000 7f27fd8d6640 1 -- 172.21.15.12:0/1358284623 >> 172.21.15.12:0/1358284623 conn(0x7f27f806bfe0 msgr2=0x7f27f806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:56.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.205+0000 7f27fd8d6640 1 -- 172.21.15.12:0/1358284623 shutdown_connections 2023-10-30T12:49:56.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.205+0000 7f27fd8d6640 1 -- 172.21.15.12:0/1358284623 wait complete. 2023-10-30T12:49:56.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27fd8d6640 1 Processor -- start 2023-10-30T12:49:56.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27fd8d6640 1 -- start start 2023-10-30T12:49:56.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27fd8d6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27f80ca0e0 0x7f27f80c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:56.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27fd8d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27f80cf160 0x7f27f81507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:56.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27fd8d6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27f81538a0 0x7f27f80b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:56.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27fd8d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f8157830 con 0x7f27f80cf160 2023-10-30T12:49:56.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27fd8d6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f81575c0 con 0x7f27f81538a0 2023-10-30T12:49:56.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27fd8d6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f80b3d90 con 0x7f27f80ca0e0 2023-10-30T12:49:56.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27effff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27f80cf160 0x7f27f81507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:56.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27effff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27f80cf160 0x7f27f81507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44830/0 (socket says 172.21.15.12:44830) 2023-10-30T12:49:56.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27effff640 1 -- 172.21.15.12:0/4091465278 learned_addr learned my addr 172.21.15.12:0/4091465278 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:56.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27f77fe640 1 --2- 172.21.15.12:0/4091465278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27f81538a0 0x7f27f80b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:56.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27f6ffd640 1 --2- 172.21.15.12:0/4091465278 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27f80ca0e0 0x7f27f80c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:56.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27effff640 1 -- 172.21.15.12:0/4091465278 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27f80ca0e0 msgr2=0x7f27f80c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:56.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27effff640 1 --2- 172.21.15.12:0/4091465278 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27f80ca0e0 0x7f27f80c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:56.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27effff640 1 -- 172.21.15.12:0/4091465278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27f81538a0 msgr2=0x7f27f80b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:56.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27effff640 1 --2- 172.21.15.12:0/4091465278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27f81538a0 0x7f27f80b42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:56.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27effff640 1 -- 172.21.15.12:0/4091465278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e8009420 con 0x7f27f80cf160 2023-10-30T12:49:56.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27f77fe640 1 --2- 172.21.15.12:0/4091465278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27f81538a0 0x7f27f80b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:56.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27effff640 1 --2- 172.21.15.12:0/4091465278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27f80cf160 0x7f27f81507d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f27e000c3c0 tx=0x7f27e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:56.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27f4ff9640 1 -- 172.21.15.12:0/4091465278 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00052d0 con 0x7f27f80cf160 2023-10-30T12:49:56.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27f4ff9640 1 -- 172.21.15.12:0/4091465278 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e0005430 con 0x7f27f80cf160 2023-10-30T12:49:56.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27f4ff9640 1 -- 172.21.15.12:0/4091465278 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0003840 con 0x7f27f80cf160 2023-10-30T12:49:56.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27fd8d6640 1 -- 172.21.15.12:0/4091465278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f8159040 con 0x7f27f80cf160 2023-10-30T12:49:56.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.209+0000 7f27fd8d6640 1 -- 172.21.15.12:0/4091465278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f80b4010 con 0x7f27f80cf160 2023-10-30T12:49:56.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.213+0000 7f27fd8d6640 1 -- 172.21.15.12:0/4091465278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f804e170 con 0x7f27f80cf160 2023-10-30T12:49:56.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.213+0000 7f27f4ff9640 1 -- 172.21.15.12:0/4091465278 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f27e00039a0 con 0x7f27f80cf160 2023-10-30T12:49:56.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.213+0000 7f27f4ff9640 1 --2- 172.21.15.12:0/4091465278 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f27d80664f0 0x7f27d80689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:56.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.213+0000 7f27f4ff9640 1 -- 172.21.15.12:0/4091465278 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f27e0083540 con 0x7f27f80cf160 2023-10-30T12:49:56.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.213+0000 7f27f6ffd640 1 --2- 172.21.15.12:0/4091465278 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f27d80664f0 0x7f27d80689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:56.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.217+0000 7f27f6ffd640 1 --2- 172.21.15.12:0/4091465278 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f27d80664f0 0x7f27d80689b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f27e80b2000 tx=0x7f27e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:56.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.217+0000 7f27f4ff9640 1 -- 172.21.15.12:0/4091465278 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f27e00506d0 con 0x7f27f80cf160 2023-10-30T12:49:56.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.409+0000 7f27fd8d6640 1 -- 172.21.15.12:0/4091465278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27f8159b40 con 0x7f27f80cf160 2023-10-30T12:49:56.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.409+0000 7f27f4ff9640 1 -- 172.21.15.12:0/4091465278 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+5747 (secure 0 0 0) 0x7f27e0053cf0 con 0x7f27f80cf160 2023-10-30T12:49:56.418 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:56.419 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:52.465384+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:56.419 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 76 2023-10-30T12:49:56.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.413+0000 7f27fd8d6640 1 -- 172.21.15.12:0/4091465278 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f27d80664f0 msgr2=0x7f27d80689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:56.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.413+0000 7f27fd8d6640 1 --2- 172.21.15.12:0/4091465278 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f27d80664f0 0x7f27d80689b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f27e80b2000 tx=0x7f27e8008040 comp rx=0 tx=0).stop 2023-10-30T12:49:56.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.413+0000 7f27fd8d6640 1 -- 172.21.15.12:0/4091465278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27f80cf160 msgr2=0x7f27f81507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:56.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.413+0000 7f27fd8d6640 1 --2- 172.21.15.12:0/4091465278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27f80cf160 0x7f27f81507d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f27e000c3c0 tx=0x7f27e000c880 comp rx=0 tx=0).stop 2023-10-30T12:49:56.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.413+0000 7f27fd8d6640 1 -- 172.21.15.12:0/4091465278 shutdown_connections 2023-10-30T12:49:56.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.413+0000 7f27fd8d6640 1 --2- 172.21.15.12:0/4091465278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27f81538a0 0x7f27f80b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:56.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.413+0000 7f27fd8d6640 1 --2- 172.21.15.12:0/4091465278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27f80cf160 0x7f27f81507d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:56.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.413+0000 7f27fd8d6640 1 --2- 172.21.15.12:0/4091465278 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f27d80664f0 0x7f27d80689b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:56.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.413+0000 7f27fd8d6640 1 --2- 172.21.15.12:0/4091465278 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27f80ca0e0 0x7f27f80c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:56.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.413+0000 7f27fd8d6640 1 -- 172.21.15.12:0/4091465278 >> 172.21.15.12:0/4091465278 conn(0x7f27f806bfe0 msgr2=0x7f27f8152310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:56.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.413+0000 7f27fd8d6640 1 -- 172.21.15.12:0/4091465278 shutdown_connections 2023-10-30T12:49:56.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:56.413+0000 7f27fd8d6640 1 -- 172.21.15.12:0/4091465278 wait complete. 2023-10-30T12:49:56.439 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:49:58.441 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc94add3640 1 Processor -- start 2023-10-30T12:49:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc94add3640 1 -- start start 2023-10-30T12:49:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc94add3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc9441538a0 0x7fc94415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc94add3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc9440cf160 0x7fc9440cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc94add3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9440cb800 con 0x7fc9440ca0e0 2023-10-30T12:49:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc94add3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc94415dc30 con 0x7fc9440cf160 2023-10-30T12:49:58.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc94add3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9440cb330 con 0x7fc9441538a0 2023-10-30T12:49:58.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc949370640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc9440cf160 0x7fc9440cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:58.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc949370640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc9440cf160 0x7fc9440cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44840/0 (socket says 172.21.15.12:44840) 2023-10-30T12:49:58.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc949370640 1 -- 172.21.15.12:0/2982470802 learned_addr learned my addr 172.21.15.12:0/2982470802 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:58.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc948b6f640 1 --2- 172.21.15.12:0/2982470802 >> v2:172.21.15.154:3300/0 conn(0x7fc9441538a0 0x7fc94415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:58.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc949370640 1 -- 172.21.15.12:0/2982470802 >> v2:172.21.15.154:3300/0 conn(0x7fc9441538a0 msgr2=0x7fc94415d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:58.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc949370640 1 --2- 172.21.15.12:0/2982470802 >> v2:172.21.15.154:3300/0 conn(0x7fc9441538a0 0x7fc94415d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:58.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc949370640 1 -- 172.21.15.12:0/2982470802 >> v1:172.21.15.154:6789/0 conn(0x7fc9440ca0e0 legacy=0x7fc94415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:58.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc949370640 1 -- 172.21.15.12:0/2982470802 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9440cad70 con 0x7fc9440cf160 2023-10-30T12:49:58.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.581+0000 7fc948b6f640 1 --2- 172.21.15.12:0/2982470802 >> v2:172.21.15.154:3300/0 conn(0x7fc9441538a0 0x7fc94415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:58.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.585+0000 7fc949370640 1 --2- 172.21.15.12:0/2982470802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9440cf160 0x7fc9440cf590 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc9440cfd10 tx=0x7fc93c01f180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a2554baa93a4a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:58.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.585+0000 7fc9437fe640 1 -- 172.21.15.12:0/2982470802 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0af400 con 0x7fc9440cf160 2023-10-30T12:49:58.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.585+0000 7fc9437fe640 1 -- 172.21.15.12:0/2982470802 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c0ab050 con 0x7fc9440cf160 2023-10-30T12:49:58.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.585+0000 7fc9437fe640 1 -- 172.21.15.12:0/2982470802 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0af780 con 0x7fc9440cf160 2023-10-30T12:49:58.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.585+0000 7fc94add3640 1 -- 172.21.15.12:0/2982470802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9440cf160 msgr2=0x7fc9440cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:58.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.585+0000 7fc94add3640 1 --2- 172.21.15.12:0/2982470802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9440cf160 0x7fc9440cf590 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc9440cfd10 tx=0x7fc93c01f180 comp rx=0 tx=0).stop 2023-10-30T12:49:58.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.585+0000 7fc94add3640 1 -- 172.21.15.12:0/2982470802 shutdown_connections 2023-10-30T12:49:58.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.585+0000 7fc94add3640 1 --2- 172.21.15.12:0/2982470802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9440cf160 0x7fc9440cf590 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:58.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.585+0000 7fc94add3640 1 --2- 172.21.15.12:0/2982470802 >> v2:172.21.15.154:3300/0 conn(0x7fc9441538a0 0x7fc94415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:58.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.585+0000 7fc94add3640 1 -- 172.21.15.12:0/2982470802 >> 172.21.15.12:0/2982470802 conn(0x7fc94406bfe0 msgr2=0x7fc94406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:58.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.585+0000 7fc94add3640 1 -- 172.21.15.12:0/2982470802 shutdown_connections 2023-10-30T12:49:58.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.585+0000 7fc94add3640 1 -- 172.21.15.12:0/2982470802 wait complete. 2023-10-30T12:49:58.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc94add3640 1 Processor -- start 2023-10-30T12:49:58.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc94add3640 1 -- start start 2023-10-30T12:49:58.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc94add3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9440ca0e0 0x7fc9440ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:58.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc94add3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9440cf160 0x7fc9440b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:58.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc94add3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9441538a0 0x7fc9440b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:58.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc94add3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9441ef730 con 0x7fc9441538a0 2023-10-30T12:49:58.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc94add3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440b3720 con 0x7fc9440ca0e0 2023-10-30T12:49:58.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc94add3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440b1960 con 0x7fc9440cf160 2023-10-30T12:49:58.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc949370640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9441538a0 0x7fc9440b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:58.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc949370640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9441538a0 0x7fc9440b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44848/0 (socket says 172.21.15.12:44848) 2023-10-30T12:49:58.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc949370640 1 -- 172.21.15.12:0/2132992215 learned_addr learned my addr 172.21.15.12:0/2132992215 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:58.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc948b6f640 1 --2- 172.21.15.12:0/2132992215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9440ca0e0 0x7fc9440ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:58.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc943fff640 1 --2- 172.21.15.12:0/2132992215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9440cf160 0x7fc9440b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:58.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc949370640 1 -- 172.21.15.12:0/2132992215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9440cf160 msgr2=0x7fc9440b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:58.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc949370640 1 --2- 172.21.15.12:0/2132992215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9440cf160 0x7fc9440b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:58.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc949370640 1 -- 172.21.15.12:0/2132992215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9440ca0e0 msgr2=0x7fc9440ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:58.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc949370640 1 --2- 172.21.15.12:0/2132992215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9440ca0e0 0x7fc9440ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:58.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc949370640 1 -- 172.21.15.12:0/2132992215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c009b30 con 0x7fc9441538a0 2023-10-30T12:49:58.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc948b6f640 1 --2- 172.21.15.12:0/2132992215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9440ca0e0 0x7fc9440ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:58.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc949370640 1 --2- 172.21.15.12:0/2132992215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9441538a0 0x7fc9440b4080 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0b2040 tx=0x7fc93c01bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc9417fa640 1 -- 172.21.15.12:0/2132992215 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0afd10 con 0x7fc9441538a0 2023-10-30T12:49:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc9417fa640 1 -- 172.21.15.12:0/2132992215 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c0afe70 con 0x7fc9441538a0 2023-10-30T12:49:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc9417fa640 1 -- 172.21.15.12:0/2132992215 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0ba640 con 0x7fc9441538a0 2023-10-30T12:49:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc94add3640 1 -- 172.21.15.12:0/2132992215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9440b1bb0 con 0x7fc9441538a0 2023-10-30T12:49:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc94add3640 1 -- 172.21.15.12:0/2132992215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9440b3340 con 0x7fc9441538a0 2023-10-30T12:49:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.589+0000 7fc94add3640 1 -- 172.21.15.12:0/2132992215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9440b2a50 con 0x7fc9441538a0 2023-10-30T12:49:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.593+0000 7fc9417fa640 1 -- 172.21.15.12:0/2132992215 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc93c0b6040 con 0x7fc9441538a0 2023-10-30T12:49:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.593+0000 7fc9417fa640 1 --2- 172.21.15.12:0/2132992215 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9280664a0 0x7fc928068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.593+0000 7fc9417fa640 1 -- 172.21.15.12:0/2132992215 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fc93c1223d0 con 0x7fc9441538a0 2023-10-30T12:49:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.593+0000 7fc948b6f640 1 --2- 172.21.15.12:0/2132992215 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9280664a0 0x7fc928068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.593+0000 7fc948b6f640 1 --2- 172.21.15.12:0/2132992215 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9280664a0 0x7fc928068960 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc938009970 tx=0x7fc938008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:58.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.597+0000 7fc9417fa640 1 -- 172.21.15.12:0/2132992215 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc93c0ef4f0 con 0x7fc9441538a0 2023-10-30T12:49:58.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.789+0000 7fc94add3640 1 -- 172.21.15.12:0/2132992215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9440b31a0 con 0x7fc9441538a0 2023-10-30T12:49:58.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.789+0000 7fc9417fa640 1 -- 172.21.15.12:0/2132992215 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+5747 (secure 0 0 0) 0x7fc9440b31a0 con 0x7fc9441538a0 2023-10-30T12:49:58.798 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:58.798 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:57.544220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:58.799 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 77 2023-10-30T12:49:58.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.793+0000 7fc94add3640 1 -- 172.21.15.12:0/2132992215 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9280664a0 msgr2=0x7fc928068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:58.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.793+0000 7fc94add3640 1 --2- 172.21.15.12:0/2132992215 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9280664a0 0x7fc928068960 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc938009970 tx=0x7fc938008040 comp rx=0 tx=0).stop 2023-10-30T12:49:58.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.793+0000 7fc94add3640 1 -- 172.21.15.12:0/2132992215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9441538a0 msgr2=0x7fc9440b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:58.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.793+0000 7fc94add3640 1 --2- 172.21.15.12:0/2132992215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9441538a0 0x7fc9440b4080 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0b2040 tx=0x7fc93c01bf20 comp rx=0 tx=0).stop 2023-10-30T12:49:58.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.793+0000 7fc94add3640 1 -- 172.21.15.12:0/2132992215 shutdown_connections 2023-10-30T12:49:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.793+0000 7fc94add3640 1 --2- 172.21.15.12:0/2132992215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9441538a0 0x7fc9440b4080 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.793+0000 7fc94add3640 1 --2- 172.21.15.12:0/2132992215 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9280664a0 0x7fc928068960 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.793+0000 7fc94add3640 1 --2- 172.21.15.12:0/2132992215 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9440cf160 0x7fc9440b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.793+0000 7fc94add3640 1 --2- 172.21.15.12:0/2132992215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9440ca0e0 0x7fc9440ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.793+0000 7fc94add3640 1 -- 172.21.15.12:0/2132992215 >> 172.21.15.12:0/2132992215 conn(0x7fc94406bfe0 msgr2=0x7fc944065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.793+0000 7fc94add3640 1 -- 172.21.15.12:0/2132992215 shutdown_connections 2023-10-30T12:49:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.793+0000 7fc94add3640 1 -- 172.21.15.12:0/2132992215 wait complete. 2023-10-30T12:49:58.819 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d has gained rank=1, replacing gid=5124 2023-10-30T12:49:58.819 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.a 2023-10-30T12:49:58.819 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2023-10-30T12:49:58.820 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-30T12:49:58.820 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-30T12:49:58.823 INFO:tasks.ceph.mds.a:Started 2023-10-30T12:49:58.824 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:58.912 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2023-10-30T12:49:58.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.961+0000 7fa0e6446640 1 Processor -- start 2023-10-30T12:49:58.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.961+0000 7fa0e6446640 1 -- start start 2023-10-30T12:49:58.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.965+0000 7fa0e6446640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa0e01538a0 0x7fa0e015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:58.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.965+0000 7fa0e6446640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa0e00ca0e0 0x7fa0e015a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:58.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.965+0000 7fa0e6446640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e00cb800 con 0x7fa0e00cf160 2023-10-30T12:49:58.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.965+0000 7fa0e6446640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e015dc30 con 0x7fa0e01538a0 2023-10-30T12:49:58.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.965+0000 7fa0e6446640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e00cb330 con 0x7fa0e00ca0e0 2023-10-30T12:49:58.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.965+0000 7fa0dffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa0e01538a0 0x7fa0e015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.965+0000 7fa0dffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa0e01538a0 0x7fa0e015d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44872/0 (socket says 172.21.15.12:44872) 2023-10-30T12:49:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.965+0000 7fa0dffff640 1 -- 172.21.15.12:0/3637643616 learned_addr learned my addr 172.21.15.12:0/3637643616 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.965+0000 7fa0e49e3640 1 --1- 172.21.15.12:0/3637643616 >> v1:172.21.15.12:6789/0 conn(0x7fa0e00cf160 0x7fa0e00cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49892/0 (socket says 172.21.15.12:49892) 2023-10-30T12:49:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.965+0000 7fa0df7fe640 1 --2- 172.21.15.12:0/3637643616 >> v2:172.21.15.154:3300/0 conn(0x7fa0e00ca0e0 0x7fa0e015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.965+0000 7fa0dffff640 1 -- 172.21.15.12:0/3637643616 >> v2:172.21.15.154:3300/0 conn(0x7fa0e00ca0e0 msgr2=0x7fa0e015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.965+0000 7fa0dffff640 1 --2- 172.21.15.12:0/3637643616 >> v2:172.21.15.154:3300/0 conn(0x7fa0e00ca0e0 0x7fa0e015a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:58.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.965+0000 7fa0dffff640 1 -- 172.21.15.12:0/3637643616 >> v1:172.21.15.12:6789/0 conn(0x7fa0e00cf160 legacy=0x7fa0e00cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:58.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.965+0000 7fa0dffff640 1 -- 172.21.15.12:0/3637643616 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e00cad70 con 0x7fa0e01538a0 2023-10-30T12:49:58.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.969+0000 7fa0dffff640 1 --2- 172.21.15.12:0/3637643616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e01538a0 0x7fa0e015d6f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00b30a0 tx=0x7fa0d001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa40c3d37b28894 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:58.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.969+0000 7fa0deffd640 1 -- 172.21.15.12:0/3637643616 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d00ac620 con 0x7fa0e01538a0 2023-10-30T12:49:58.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.969+0000 7fa0deffd640 1 -- 172.21.15.12:0/3637643616 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d00ac780 con 0x7fa0e01538a0 2023-10-30T12:49:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.969+0000 7fa0deffd640 1 -- 172.21.15.12:0/3637643616 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d00acb10 con 0x7fa0e01538a0 2023-10-30T12:49:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.969+0000 7fa0e6446640 1 -- 172.21.15.12:0/3637643616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e01538a0 msgr2=0x7fa0e015d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.969+0000 7fa0e6446640 1 --2- 172.21.15.12:0/3637643616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e01538a0 0x7fa0e015d6f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00b30a0 tx=0x7fa0d001c420 comp rx=0 tx=0).stop 2023-10-30T12:49:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.969+0000 7fa0e6446640 1 -- 172.21.15.12:0/3637643616 shutdown_connections 2023-10-30T12:49:58.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.969+0000 7fa0e6446640 1 --2- 172.21.15.12:0/3637643616 >> v2:172.21.15.154:3300/0 conn(0x7fa0e00ca0e0 0x7fa0e015a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:58.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.969+0000 7fa0e6446640 1 --2- 172.21.15.12:0/3637643616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e01538a0 0x7fa0e015d6f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:58.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.969+0000 7fa0e6446640 1 -- 172.21.15.12:0/3637643616 >> 172.21.15.12:0/3637643616 conn(0x7fa0e006bfe0 msgr2=0x7fa0e006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:58.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.969+0000 7fa0e6446640 1 -- 172.21.15.12:0/3637643616 shutdown_connections 2023-10-30T12:49:58.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.969+0000 7fa0e6446640 1 -- 172.21.15.12:0/3637643616 wait complete. 2023-10-30T12:49:58.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.969+0000 7fa0e6446640 1 Processor -- start 2023-10-30T12:49:58.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0e6446640 1 -- start start 2023-10-30T12:49:58.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0e6446640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0e00ca0e0 0x7fa0e01e4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:58.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0e6446640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e00cf160 0x7fa0e0153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:58.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0e6446640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0e01538a0 0x7fa0e01e3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:58.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0e6446640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e01e41a0 con 0x7fa0e00cf160 2023-10-30T12:49:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0e6446640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e01e34f0 con 0x7fa0e01538a0 2023-10-30T12:49:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0e6446640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e0158040 con 0x7fa0e00ca0e0 2023-10-30T12:49:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0df7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e00cf160 0x7fa0e0153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0df7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e00cf160 0x7fa0e0153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44884/0 (socket says 172.21.15.12:44884) 2023-10-30T12:49:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0df7fe640 1 -- 172.21.15.12:0/3169609499 learned_addr learned my addr 172.21.15.12:0/3169609499 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:49:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0dffff640 1 --2- 172.21.15.12:0/3169609499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0e00ca0e0 0x7fa0e01e4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0e49e3640 1 --2- 172.21.15.12:0/3169609499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0e01538a0 0x7fa0e01e3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0df7fe640 1 -- 172.21.15.12:0/3169609499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0e00ca0e0 msgr2=0x7fa0e01e4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0df7fe640 1 --2- 172.21.15.12:0/3169609499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0e00ca0e0 0x7fa0e01e4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0df7fe640 1 -- 172.21.15.12:0/3169609499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0e01538a0 msgr2=0x7fa0e01e3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:58.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0df7fe640 1 --2- 172.21.15.12:0/3169609499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0e01538a0 0x7fa0e01e3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:58.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0df7fe640 1 -- 172.21.15.12:0/3169609499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d00094d0 con 0x7fa0e00cf160 2023-10-30T12:49:58.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0e49e3640 1 --2- 172.21.15.12:0/3169609499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0e01538a0 0x7fa0e01e3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:58.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0df7fe640 1 --2- 172.21.15.12:0/3169609499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e00cf160 0x7fa0e0153c80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400e960 tx=0x7fa0d400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:58.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0dcff9640 1 -- 172.21.15.12:0/3169609499 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4009690 con 0x7fa0e00cf160 2023-10-30T12:49:58.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0dcff9640 1 -- 172.21.15.12:0/3169609499 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d400ca00 con 0x7fa0e00cf160 2023-10-30T12:49:58.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0dcff9640 1 -- 172.21.15.12:0/3169609499 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d400cd70 con 0x7fa0e00cf160 2023-10-30T12:49:58.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0e6446640 1 -- 172.21.15.12:0/3169609499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e01e2b80 con 0x7fa0e00cf160 2023-10-30T12:49:58.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.973+0000 7fa0e6446640 1 -- 172.21.15.12:0/3169609499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e0156ec0 con 0x7fa0e00cf160 2023-10-30T12:49:58.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.977+0000 7fa0e6446640 1 -- 172.21.15.12:0/3169609499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e00b2a50 con 0x7fa0e00cf160 2023-10-30T12:49:58.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.977+0000 7fa0dcff9640 1 -- 172.21.15.12:0/3169609499 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa0d4013470 con 0x7fa0e00cf160 2023-10-30T12:49:58.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.977+0000 7fa0dcff9640 1 --2- 172.21.15.12:0/3169609499 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa0cc0664a0 0x7fa0cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:58.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.977+0000 7fa0dcff9640 1 -- 172.21.15.12:0/3169609499 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fa0d4084710 con 0x7fa0e00cf160 2023-10-30T12:49:58.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.977+0000 7fa0dffff640 1 --2- 172.21.15.12:0/3169609499 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa0cc0664a0 0x7fa0cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:58.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.977+0000 7fa0dffff640 1 --2- 172.21.15.12:0/3169609499 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa0cc0664a0 0x7fa0cc068960 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa0d00ae000 tx=0x7fa0d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:58.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:58.981+0000 7fa0dcff9640 1 -- 172.21.15.12:0/3169609499 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa0d40518d0 con 0x7fa0e00cf160 2023-10-30T12:49:59.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:59.213+0000 7fa0e6446640 1 -- 172.21.15.12:0/3169609499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0e004e170 con 0x7fa0e00cf160 2023-10-30T12:49:59.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:59.213+0000 7fa0dcff9640 1 -- 172.21.15.12:0/3169609499 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+5747 (secure 0 0 0) 0x7fa0d4054ef0 con 0x7fa0e00cf160 2023-10-30T12:49:59.222 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:49:59.222 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:57.544220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:59.223 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 77 2023-10-30T12:49:59.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:59.217+0000 7fa0e6446640 1 -- 172.21.15.12:0/3169609499 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa0cc0664a0 msgr2=0x7fa0cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:59.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:59.217+0000 7fa0e6446640 1 --2- 172.21.15.12:0/3169609499 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa0cc0664a0 0x7fa0cc068960 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa0d00ae000 tx=0x7fa0d0008040 comp rx=0 tx=0).stop 2023-10-30T12:49:59.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:59.217+0000 7fa0e6446640 1 -- 172.21.15.12:0/3169609499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e00cf160 msgr2=0x7fa0e0153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:59.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:59.217+0000 7fa0e6446640 1 --2- 172.21.15.12:0/3169609499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e00cf160 0x7fa0e0153c80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400e960 tx=0x7fa0d400ee20 comp rx=0 tx=0).stop 2023-10-30T12:49:59.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:59.217+0000 7fa0e6446640 1 -- 172.21.15.12:0/3169609499 shutdown_connections 2023-10-30T12:49:59.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:59.217+0000 7fa0e6446640 1 --2- 172.21.15.12:0/3169609499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0e01538a0 0x7fa0e01e3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:59.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:59.217+0000 7fa0e6446640 1 --2- 172.21.15.12:0/3169609499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e00cf160 0x7fa0e0153c80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:59.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:59.217+0000 7fa0e6446640 1 --2- 172.21.15.12:0/3169609499 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa0cc0664a0 0x7fa0cc068960 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:59.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:59.217+0000 7fa0e6446640 1 --2- 172.21.15.12:0/3169609499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0e00ca0e0 0x7fa0e01e4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:59.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:59.217+0000 7fa0e6446640 1 -- 172.21.15.12:0/3169609499 >> 172.21.15.12:0/3169609499 conn(0x7fa0e006bfe0 msgr2=0x7fa0e0151ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:59.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:59.217+0000 7fa0e6446640 1 -- 172.21.15.12:0/3169609499 shutdown_connections 2023-10-30T12:49:59.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:49:59.217+0000 7fa0e6446640 1 -- 172.21.15.12:0/3169609499 wait complete. 2023-10-30T12:49:59.243 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2023-10-30T12:49:59.388 INFO:tasks.ceph.mds.c.smithi012.stderr:2023-10-30T12:49:59.381+0000 7f262fa5d640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-30T12:50:01.245 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:01.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.389+0000 7ff27e71d640 1 Processor -- start 2023-10-30T12:50:01.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.389+0000 7ff27e71d640 1 -- start start 2023-10-30T12:50:01.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.389+0000 7ff27e71d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2780c6bd0 0x7ff27806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:01.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.389+0000 7ff27e71d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff278071110 0x7ff278158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:01.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.389+0000 7ff27e71d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff278066cf0 con 0x7ff27806f7a0 2023-10-30T12:50:01.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.389+0000 7ff27e71d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2780700c0 con 0x7ff278071110 2023-10-30T12:50:01.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.389+0000 7ff27e71d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780c70d0 con 0x7ff2780c6bd0 2023-10-30T12:50:01.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.393+0000 7ff2777fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2780c6bd0 0x7ff27806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:01.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.393+0000 7ff277fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff27806f7a0 0x7ff278069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:40418/0 (socket says 172.21.15.12:40418) 2023-10-30T12:50:01.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.393+0000 7ff277fff640 1 -- 172.21.15.12:0/3288519675 learned_addr learned my addr 172.21.15.12:0/3288519675 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:50:01.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.393+0000 7ff27ccba640 1 --2- 172.21.15.12:0/3288519675 >> v2:172.21.15.154:3300/0 conn(0x7ff278071110 0x7ff278158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:01.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.393+0000 7ff276ffd640 1 -- 172.21.15.12:0/3288519675 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914841983 0 0) 0x7ff278066cf0 con 0x7ff27806f7a0 2023-10-30T12:50:01.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.393+0000 7ff276ffd640 1 -- 172.21.15.12:0/3288519675 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff260003530 con 0x7ff27806f7a0 2023-10-30T12:50:01.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.393+0000 7ff2777fe640 1 -- 172.21.15.12:0/3288519675 >> v2:172.21.15.154:3300/0 conn(0x7ff278071110 msgr2=0x7ff278158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:01.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.393+0000 7ff2777fe640 1 --2- 172.21.15.12:0/3288519675 >> v2:172.21.15.154:3300/0 conn(0x7ff278071110 0x7ff278158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:01.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.393+0000 7ff2777fe640 1 -- 172.21.15.12:0/3288519675 >> v1:172.21.15.154:6789/0 conn(0x7ff27806f7a0 legacy=0x7ff278069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:01.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.393+0000 7ff2777fe640 1 -- 172.21.15.12:0/3288519675 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2780c7390 con 0x7ff2780c6bd0 2023-10-30T12:50:01.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.393+0000 7ff27ccba640 1 --2- 172.21.15.12:0/3288519675 >> v2:172.21.15.154:3300/0 conn(0x7ff278071110 0x7ff278158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:50:01.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.397+0000 7ff2777fe640 1 --2- 172.21.15.12:0/3288519675 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2780c6bd0 0x7ff27806fb80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0097c0 tx=0x7ff26c01ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1ff8cb678e37826 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:01.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.397+0000 7ff276ffd640 1 -- 172.21.15.12:0/3288519675 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0a8970 con 0x7ff2780c6bd0 2023-10-30T12:50:01.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.397+0000 7ff276ffd640 1 -- 172.21.15.12:0/3288519675 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26c0a8ad0 con 0x7ff2780c6bd0 2023-10-30T12:50:01.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.397+0000 7ff276ffd640 1 -- 172.21.15.12:0/3288519675 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0a8e60 con 0x7ff2780c6bd0 2023-10-30T12:50:01.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.397+0000 7ff27e71d640 1 -- 172.21.15.12:0/3288519675 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2780c6bd0 msgr2=0x7ff27806fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:01.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.397+0000 7ff27e71d640 1 --2- 172.21.15.12:0/3288519675 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2780c6bd0 0x7ff27806fb80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0097c0 tx=0x7ff26c01ec70 comp rx=0 tx=0).stop 2023-10-30T12:50:01.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.397+0000 7ff27e71d640 1 -- 172.21.15.12:0/3288519675 shutdown_connections 2023-10-30T12:50:01.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.397+0000 7ff27e71d640 1 --2- 172.21.15.12:0/3288519675 >> v2:172.21.15.154:3300/0 conn(0x7ff278071110 0x7ff278158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:01.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.397+0000 7ff27e71d640 1 --2- 172.21.15.12:0/3288519675 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2780c6bd0 0x7ff27806fb80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:01.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.397+0000 7ff27e71d640 1 -- 172.21.15.12:0/3288519675 >> 172.21.15.12:0/3288519675 conn(0x7ff2780c7910 msgr2=0x7ff2780c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:01.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.397+0000 7ff27e71d640 1 -- 172.21.15.12:0/3288519675 shutdown_connections 2023-10-30T12:50:01.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.397+0000 7ff27e71d640 1 -- 172.21.15.12:0/3288519675 wait complete. 2023-10-30T12:50:01.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.397+0000 7ff27e71d640 1 Processor -- start 2023-10-30T12:50:01.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.397+0000 7ff27e71d640 1 -- start start 2023-10-30T12:50:01.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff27e71d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff27806f7a0 0x7ff27815a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:01.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff27e71d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff278071110 0x7ff2781edf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:01.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff27e71d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2780c6bd0 0x7ff278146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:01.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff27e71d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27814c660 con 0x7ff278071110 2023-10-30T12:50:01.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff27e71d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27814dfb0 con 0x7ff2780c6bd0 2023-10-30T12:50:01.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff27e71d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27814c1d0 con 0x7ff27806f7a0 2023-10-30T12:50:01.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff2777fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff278071110 0x7ff2781edf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:01.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff2777fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff278071110 0x7ff2781edf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40504/0 (socket says 172.21.15.12:40504) 2023-10-30T12:50:01.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff2777fe640 1 -- 172.21.15.12:0/1182532639 learned_addr learned my addr 172.21.15.12:0/1182532639 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:01.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff277fff640 1 --2- 172.21.15.12:0/1182532639 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff27806f7a0 0x7ff27815a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:01.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff27ccba640 1 --2- 172.21.15.12:0/1182532639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2780c6bd0 0x7ff278146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:01.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff2777fe640 1 -- 172.21.15.12:0/1182532639 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff27806f7a0 msgr2=0x7ff27815a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:01.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff2777fe640 1 --2- 172.21.15.12:0/1182532639 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff27806f7a0 0x7ff27815a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:01.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff2777fe640 1 -- 172.21.15.12:0/1182532639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2780c6bd0 msgr2=0x7ff278146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:01.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff2777fe640 1 --2- 172.21.15.12:0/1182532639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2780c6bd0 0x7ff278146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:01.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff2777fe640 1 -- 172.21.15.12:0/1182532639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff270009a70 con 0x7ff278071110 2023-10-30T12:50:01.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff2777fe640 1 --2- 172.21.15.12:0/1182532639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff278071110 0x7ff2781edf10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff26c01b880 tx=0x7ff26c01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:01.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff274ff9640 1 -- 172.21.15.12:0/1182532639 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0b1d10 con 0x7ff278071110 2023-10-30T12:50:01.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff274ff9640 1 -- 172.21.15.12:0/1182532639 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26c0b1e70 con 0x7ff278071110 2023-10-30T12:50:01.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff274ff9640 1 -- 172.21.15.12:0/1182532639 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0b9630 con 0x7ff278071110 2023-10-30T12:50:01.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff27e71d640 1 -- 172.21.15.12:0/1182532639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26c009420 con 0x7ff278071110 2023-10-30T12:50:01.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff27e71d640 1 -- 172.21.15.12:0/1182532639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2781580c0 con 0x7ff278071110 2023-10-30T12:50:01.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff27e71d640 1 -- 172.21.15.12:0/1182532639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27814c340 con 0x7ff278071110 2023-10-30T12:50:01.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.401+0000 7ff274ff9640 1 -- 172.21.15.12:0/1182532639 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff26c0b7040 con 0x7ff278071110 2023-10-30T12:50:01.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.405+0000 7ff274ff9640 1 --2- 172.21.15.12:0/1182532639 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2640664a0 0x7ff264068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:01.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.405+0000 7ff274ff9640 1 -- 172.21.15.12:0/1182532639 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7ff26c121910 con 0x7ff278071110 2023-10-30T12:50:01.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.405+0000 7ff277fff640 1 --2- 172.21.15.12:0/1182532639 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2640664a0 0x7ff264068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:01.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.405+0000 7ff277fff640 1 --2- 172.21.15.12:0/1182532639 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2640664a0 0x7ff264068960 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff268007fd0 tx=0x7ff268009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:01.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.409+0000 7ff274ff9640 1 -- 172.21.15.12:0/1182532639 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff26c0eeab0 con 0x7ff278071110 2023-10-30T12:50:01.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.673+0000 7ff27e71d640 1 -- 172.21.15.12:0/1182532639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2781582b0 con 0x7ff278071110 2023-10-30T12:50:01.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.673+0000 7ff274ff9640 1 -- 172.21.15.12:0/1182532639 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+6538 (secure 0 0 0) 0x7ff26c0b5020 con 0x7ff278071110 2023-10-30T12:50:01.679 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:50:01.679 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:49:57.544220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:01.680 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 78 2023-10-30T12:50:01.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.673+0000 7ff27e71d640 1 -- 172.21.15.12:0/1182532639 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2640664a0 msgr2=0x7ff264068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:01.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.673+0000 7ff27e71d640 1 --2- 172.21.15.12:0/1182532639 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2640664a0 0x7ff264068960 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff268007fd0 tx=0x7ff268009040 comp rx=0 tx=0).stop 2023-10-30T12:50:01.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.673+0000 7ff27e71d640 1 -- 172.21.15.12:0/1182532639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff278071110 msgr2=0x7ff2781edf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:01.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.673+0000 7ff27e71d640 1 --2- 172.21.15.12:0/1182532639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff278071110 0x7ff2781edf10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff26c01b880 tx=0x7ff26c01ba10 comp rx=0 tx=0).stop 2023-10-30T12:50:01.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.673+0000 7ff27e71d640 1 -- 172.21.15.12:0/1182532639 shutdown_connections 2023-10-30T12:50:01.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.673+0000 7ff27e71d640 1 --2- 172.21.15.12:0/1182532639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2780c6bd0 0x7ff278146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.673+0000 7ff27e71d640 1 --2- 172.21.15.12:0/1182532639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff278071110 0x7ff2781edf10 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.673+0000 7ff27e71d640 1 --2- 172.21.15.12:0/1182532639 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2640664a0 0x7ff264068960 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.673+0000 7ff27e71d640 1 --2- 172.21.15.12:0/1182532639 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff27806f7a0 0x7ff27815a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.673+0000 7ff27e71d640 1 -- 172.21.15.12:0/1182532639 >> 172.21.15.12:0/1182532639 conn(0x7ff2780c7910 msgr2=0x7ff2780c8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.673+0000 7ff27e71d640 1 -- 172.21.15.12:0/1182532639 shutdown_connections 2023-10-30T12:50:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:01.673+0000 7ff27e71d640 1 -- 172.21.15.12:0/1182532639 wait complete. 2023-10-30T12:50:01.701 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2023-10-30T12:50:01.701 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 31 secs before thrashing 2023-10-30T12:50:06.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:50:06.262 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:50:32.702 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:32.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.865+0000 7f0a4a53e640 1 Processor -- start 2023-10-30T12:50:32.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.865+0000 7f0a4a53e640 1 -- start start 2023-10-30T12:50:32.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.869+0000 7f0a4a53e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0a440c6fe0 0x7f0a44071260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:32.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.869+0000 7f0a4a53e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a440c5f60 0x7f0a440cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:32.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.869+0000 7f0a4a53e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a440700b0 con 0x7f0a44153bf0 2023-10-30T12:50:32.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.869+0000 7f0a4a53e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a440c6810 con 0x7f0a440c6fe0 2023-10-30T12:50:32.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.869+0000 7f0a4a53e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a44159900 con 0x7f0a440c5f60 2023-10-30T12:50:32.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.869+0000 7f0a437fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0a440c6fe0 0x7f0a44071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:32.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.869+0000 7f0a437fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0a440c6fe0 0x7f0a44071260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44578/0 (socket says 172.21.15.12:44578) 2023-10-30T12:50:32.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.869+0000 7f0a437fe640 1 -- 172.21.15.12:0/1891620414 learned_addr learned my addr 172.21.15.12:0/1891620414 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:32.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.869+0000 7f0a48adb640 1 --2- 172.21.15.12:0/1891620414 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a440c5f60 0x7f0a440cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:32.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.869+0000 7f0a437fe640 1 -- 172.21.15.12:0/1891620414 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a440c5f60 msgr2=0x7f0a440cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:32.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.869+0000 7f0a437fe640 1 --2- 172.21.15.12:0/1891620414 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a440c5f60 0x7f0a440cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:32.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.869+0000 7f0a437fe640 1 -- 172.21.15.12:0/1891620414 >> v1:172.21.15.154:6789/0 conn(0x7f0a44153bf0 legacy=0x7f0a440c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:32.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.869+0000 7f0a437fe640 1 -- 172.21.15.12:0/1891620414 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a440c6530 con 0x7f0a440c6fe0 2023-10-30T12:50:32.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.869+0000 7f0a48adb640 1 --2- 172.21.15.12:0/1891620414 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a440c5f60 0x7f0a440cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:32.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.873+0000 7f0a437fe640 1 --2- 172.21.15.12:0/1891620414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a440c6fe0 0x7f0a44071260 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009870 tx=0x7f0a3801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5c821327516c36 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:32.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.873+0000 7f0a42ffd640 1 -- 172.21.15.12:0/1891620414 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a380a8a30 con 0x7f0a440c6fe0 2023-10-30T12:50:32.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.873+0000 7f0a42ffd640 1 -- 172.21.15.12:0/1891620414 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a380a8b90 con 0x7f0a440c6fe0 2023-10-30T12:50:32.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.873+0000 7f0a42ffd640 1 -- 172.21.15.12:0/1891620414 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a380b0440 con 0x7f0a440c6fe0 2023-10-30T12:50:32.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.873+0000 7f0a4a53e640 1 -- 172.21.15.12:0/1891620414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a440c6fe0 msgr2=0x7f0a44071260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:32.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.873+0000 7f0a4a53e640 1 --2- 172.21.15.12:0/1891620414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a440c6fe0 0x7f0a44071260 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009870 tx=0x7f0a3801ed30 comp rx=0 tx=0).stop 2023-10-30T12:50:32.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.873+0000 7f0a4a53e640 1 -- 172.21.15.12:0/1891620414 shutdown_connections 2023-10-30T12:50:32.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.873+0000 7f0a4a53e640 1 --2- 172.21.15.12:0/1891620414 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a440c5f60 0x7f0a440cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:32.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.873+0000 7f0a4a53e640 1 --2- 172.21.15.12:0/1891620414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a440c6fe0 0x7f0a44071260 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:32.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.873+0000 7f0a4a53e640 1 -- 172.21.15.12:0/1891620414 >> 172.21.15.12:0/1891620414 conn(0x7f0a4406c6e0 msgr2=0x7f0a44069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:32.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.873+0000 7f0a4a53e640 1 -- 172.21.15.12:0/1891620414 shutdown_connections 2023-10-30T12:50:32.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.873+0000 7f0a4a53e640 1 -- 172.21.15.12:0/1891620414 wait complete. 2023-10-30T12:50:32.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.873+0000 7f0a4a53e640 1 Processor -- start 2023-10-30T12:50:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.873+0000 7f0a4a53e640 1 -- start start 2023-10-30T12:50:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a4a53e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a440c5f60 0x7f0a441e4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a4a53e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a440c6fe0 0x7f0a441e45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a4a53e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a44153bf0 0x7f0a4414c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a4a53e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4414eac0 con 0x7f0a440c5f60 2023-10-30T12:50:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a4a53e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4414d130 con 0x7f0a440c6fe0 2023-10-30T12:50:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a4a53e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4414ed10 con 0x7f0a44153bf0 2023-10-30T12:50:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a440c5f60 0x7f0a441e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a440c5f60 0x7f0a441e4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44580/0 (socket says 172.21.15.12:44580) 2023-10-30T12:50:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a43fff640 1 -- 172.21.15.12:0/3989037233 learned_addr learned my addr 172.21.15.12:0/3989037233 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:32.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a48adb640 1 --2- 172.21.15.12:0/3989037233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a44153bf0 0x7f0a4414c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:32.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a437fe640 1 --2- 172.21.15.12:0/3989037233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a440c6fe0 0x7f0a441e45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:32.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a43fff640 1 -- 172.21.15.12:0/3989037233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a44153bf0 msgr2=0x7f0a4414c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:32.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a43fff640 1 --2- 172.21.15.12:0/3989037233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a44153bf0 0x7f0a4414c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:32.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a43fff640 1 -- 172.21.15.12:0/3989037233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a440c6fe0 msgr2=0x7f0a441e45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:32.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a43fff640 1 --2- 172.21.15.12:0/3989037233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a440c6fe0 0x7f0a441e45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:32.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a43fff640 1 -- 172.21.15.12:0/3989037233 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a380094d0 con 0x7f0a440c5f60 2023-10-30T12:50:32.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a48adb640 1 --2- 172.21.15.12:0/3989037233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a44153bf0 0x7f0a4414c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:32.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a43fff640 1 --2- 172.21.15.12:0/3989037233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a440c5f60 0x7f0a441e4090 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c500 tx=0x7f0a3400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:32.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a40ff9640 1 -- 172.21.15.12:0/3989037233 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34005220 con 0x7f0a440c5f60 2023-10-30T12:50:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a40ff9640 1 -- 172.21.15.12:0/3989037233 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a34005380 con 0x7f0a440c5f60 2023-10-30T12:50:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a40ff9640 1 -- 172.21.15.12:0/3989037233 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34007270 con 0x7f0a440c5f60 2023-10-30T12:50:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a4a53e640 1 -- 172.21.15.12:0/3989037233 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4414eff0 con 0x7f0a440c5f60 2023-10-30T12:50:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a4a53e640 1 -- 172.21.15.12:0/3989037233 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a440c4f60 con 0x7f0a440c5f60 2023-10-30T12:50:32.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a4a53e640 1 -- 172.21.15.12:0/3989037233 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a4404e170 con 0x7f0a440c5f60 2023-10-30T12:50:32.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.877+0000 7f0a40ff9640 1 -- 172.21.15.12:0/3989037233 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0a340073d0 con 0x7f0a440c5f60 2023-10-30T12:50:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.881+0000 7f0a40ff9640 1 --2- 172.21.15.12:0/3989037233 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a300664a0 0x7f0a30068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.881+0000 7f0a40ff9640 1 -- 172.21.15.12:0/3989037233 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f0a34083a90 con 0x7f0a440c5f60 2023-10-30T12:50:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.881+0000 7f0a437fe640 1 --2- 172.21.15.12:0/3989037233 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a300664a0 0x7f0a30068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.881+0000 7f0a437fe640 1 --2- 172.21.15.12:0/3989037233 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a300664a0 0x7f0a30068960 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0a380b2000 tx=0x7f0a38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:32.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:32.885+0000 7f0a40ff9640 1 -- 172.21.15.12:0/3989037233 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0a34050bb0 con 0x7f0a440c5f60 2023-10-30T12:50:33.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.073+0000 7f0a4a53e640 1 -- 172.21.15.12:0/3989037233 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a441e3cb0 con 0x7f0a440c5f60 2023-10-30T12:50:33.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.073+0000 7f0a40ff9640 1 -- 172.21.15.12:0/3989037233 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+6541 (secure 0 0 0) 0x7f0a340541d0 con 0x7f0a440c5f60 2023-10-30T12:50:33.080 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:50:33.080 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:50:13.522016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:33.080 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2023-10-30T12:50:33.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.073+0000 7f0a4a53e640 1 -- 172.21.15.12:0/3989037233 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a300664a0 msgr2=0x7f0a30068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:33.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.073+0000 7f0a4a53e640 1 --2- 172.21.15.12:0/3989037233 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a300664a0 0x7f0a30068960 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0a380b2000 tx=0x7f0a38009040 comp rx=0 tx=0).stop 2023-10-30T12:50:33.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.073+0000 7f0a4a53e640 1 -- 172.21.15.12:0/3989037233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a440c5f60 msgr2=0x7f0a441e4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:33.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.073+0000 7f0a4a53e640 1 --2- 172.21.15.12:0/3989037233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a440c5f60 0x7f0a441e4090 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c500 tx=0x7f0a3400c9c0 comp rx=0 tx=0).stop 2023-10-30T12:50:33.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.073+0000 7f0a4a53e640 1 -- 172.21.15.12:0/3989037233 shutdown_connections 2023-10-30T12:50:33.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.073+0000 7f0a4a53e640 1 --2- 172.21.15.12:0/3989037233 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a300664a0 0x7f0a30068960 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:33.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.073+0000 7f0a4a53e640 1 --2- 172.21.15.12:0/3989037233 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a44153bf0 0x7f0a4414c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:33.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.073+0000 7f0a4a53e640 1 --2- 172.21.15.12:0/3989037233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a440c6fe0 0x7f0a441e45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:33.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.073+0000 7f0a4a53e640 1 --2- 172.21.15.12:0/3989037233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a440c5f60 0x7f0a441e4090 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:33.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.073+0000 7f0a4a53e640 1 -- 172.21.15.12:0/3989037233 >> 172.21.15.12:0/3989037233 conn(0x7f0a4406c6e0 msgr2=0x7f0a44159000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:33.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.073+0000 7f0a4a53e640 1 -- 172.21.15.12:0/3989037233 shutdown_connections 2023-10-30T12:50:33.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.073+0000 7f0a4a53e640 1 -- 172.21.15.12:0/3989037233 wait complete. 2023-10-30T12:50:33.101 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=2) 2023-10-30T12:50:33.102 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-10-30T12:50:33.102 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:50:33.118 INFO:tasks.ceph.mds.e:Stopped 2023-10-30T12:50:33.119 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:33.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb930501640 1 Processor -- start 2023-10-30T12:50:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb930501640 1 -- start start 2023-10-30T12:50:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb930501640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb928151800 0x7fb92806e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb930501640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9280c1ee0 0x7fb9280c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb930501640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb92806e090 con 0x7fb9280c9230 2023-10-30T12:50:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb930501640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb92806dc70 con 0x7fb928151800 2023-10-30T12:50:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb930501640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9280c8a50 con 0x7fb9280c1ee0 2023-10-30T12:50:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb92e29d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb928151800 0x7fb92806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb92da9c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9280c1ee0 0x7fb9280c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb92e29d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb928151800 0x7fb92806e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:49210/0 (socket says 172.21.15.12:49210) 2023-10-30T12:50:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb92e29d640 1 -- 172.21.15.12:0/877109318 learned_addr learned my addr 172.21.15.12:0/877109318 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb92ea9e640 1 --1- 172.21.15.12:0/877109318 >> v1:172.21.15.154:6789/0 conn(0x7fb9280c9230 0x7fb9280cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:52044/0 (socket says 172.21.15.12:52044) 2023-10-30T12:50:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb92e29d640 1 -- 172.21.15.12:0/877109318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9280c1ee0 msgr2=0x7fb9280c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb92e29d640 1 --2- 172.21.15.12:0/877109318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9280c1ee0 0x7fb9280c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb92e29d640 1 -- 172.21.15.12:0/877109318 >> v1:172.21.15.154:6789/0 conn(0x7fb9280c9230 legacy=0x7fb9280cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.305+0000 7fb92e29d640 1 -- 172.21.15.12:0/877109318 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9280c3b10 con 0x7fb928151800 2023-10-30T12:50:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.309+0000 7fb92e29d640 1 --2- 172.21.15.12:0/877109318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb928151800 0x7fb92806e470 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb924009870 tx=0x7fb92401c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7787c82088aa3dbf server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:33.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.309+0000 7fb92d29b640 1 -- 172.21.15.12:0/877109318 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240aa670 con 0x7fb928151800 2023-10-30T12:50:33.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.309+0000 7fb92d29b640 1 -- 172.21.15.12:0/877109318 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9240aa7d0 con 0x7fb928151800 2023-10-30T12:50:33.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.309+0000 7fb92d29b640 1 -- 172.21.15.12:0/877109318 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240aab60 con 0x7fb928151800 2023-10-30T12:50:33.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.309+0000 7fb930501640 1 -- 172.21.15.12:0/877109318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb928151800 msgr2=0x7fb92806e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:33.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.309+0000 7fb930501640 1 --2- 172.21.15.12:0/877109318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb928151800 0x7fb92806e470 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb924009870 tx=0x7fb92401c510 comp rx=0 tx=0).stop 2023-10-30T12:50:33.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.309+0000 7fb930501640 1 -- 172.21.15.12:0/877109318 shutdown_connections 2023-10-30T12:50:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.309+0000 7fb930501640 1 --2- 172.21.15.12:0/877109318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9280c1ee0 0x7fb9280c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.309+0000 7fb930501640 1 --2- 172.21.15.12:0/877109318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb928151800 0x7fb92806e470 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.309+0000 7fb930501640 1 -- 172.21.15.12:0/877109318 >> 172.21.15.12:0/877109318 conn(0x7fb928071ea0 msgr2=0x7fb928065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.309+0000 7fb930501640 1 -- 172.21.15.12:0/877109318 shutdown_connections 2023-10-30T12:50:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.309+0000 7fb930501640 1 -- 172.21.15.12:0/877109318 wait complete. 2023-10-30T12:50:33.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb930501640 1 Processor -- start 2023-10-30T12:50:33.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb930501640 1 -- start start 2023-10-30T12:50:33.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb930501640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9280c1ee0 0x7fb9280c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:33.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb930501640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9280c9230 0x7fb92813b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:33.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb930501640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb928151800 0x7fb92813be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:33.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb930501640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9280c5a10 con 0x7fb928151800 2023-10-30T12:50:33.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb930501640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9280c4b80 con 0x7fb9280c1ee0 2023-10-30T12:50:33.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb930501640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92814b480 con 0x7fb9280c9230 2023-10-30T12:50:33.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb92e29d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9280c1ee0 0x7fb9280c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:33.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb92ea9e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb928151800 0x7fb92813be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:33.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb92e29d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9280c1ee0 0x7fb9280c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:49226/0 (socket says 172.21.15.12:49226) 2023-10-30T12:50:33.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb92e29d640 1 -- 172.21.15.12:0/2863363961 learned_addr learned my addr 172.21.15.12:0/2863363961 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:33.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb92da9c640 1 --2- 172.21.15.12:0/2863363961 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9280c9230 0x7fb92813b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:33.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb92ea9e640 1 -- 172.21.15.12:0/2863363961 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9280c9230 msgr2=0x7fb92813b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:33.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb92ea9e640 1 --2- 172.21.15.12:0/2863363961 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9280c9230 0x7fb92813b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb92ea9e640 1 -- 172.21.15.12:0/2863363961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9280c1ee0 msgr2=0x7fb9280c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb92ea9e640 1 --2- 172.21.15.12:0/2863363961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9280c1ee0 0x7fb9280c5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb92ea9e640 1 -- 172.21.15.12:0/2863363961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9240094d0 con 0x7fb928151800 2023-10-30T12:50:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb92e29d640 1 --2- 172.21.15.12:0/2863363961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9280c1ee0 0x7fb9280c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:50:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb92ea9e640 1 --2- 172.21.15.12:0/2863363961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb928151800 0x7fb92813be60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb9280c2350 tx=0x7fb92000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:33.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb91effd640 1 -- 172.21.15.12:0/2863363961 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920004d70 con 0x7fb928151800 2023-10-30T12:50:33.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb91effd640 1 -- 172.21.15.12:0/2863363961 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92000a640 con 0x7fb928151800 2023-10-30T12:50:33.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb91effd640 1 -- 172.21.15.12:0/2863363961 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92000a9f0 con 0x7fb928151800 2023-10-30T12:50:33.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb930501640 1 -- 172.21.15.12:0/2863363961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9280c4e00 con 0x7fb928151800 2023-10-30T12:50:33.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.313+0000 7fb930501640 1 -- 172.21.15.12:0/2863363961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9280c53c0 con 0x7fb928151800 2023-10-30T12:50:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.317+0000 7fb930501640 1 -- 172.21.15.12:0/2863363961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9280c5b80 con 0x7fb928151800 2023-10-30T12:50:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.317+0000 7fb91effd640 1 -- 172.21.15.12:0/2863363961 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb92000ab50 con 0x7fb928151800 2023-10-30T12:50:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.317+0000 7fb91effd640 1 --2- 172.21.15.12:0/2863363961 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb90c0664a0 0x7fb90c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.317+0000 7fb91effd640 1 -- 172.21.15.12:0/2863363961 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fb920083540 con 0x7fb928151800 2023-10-30T12:50:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.317+0000 7fb92e29d640 1 --2- 172.21.15.12:0/2863363961 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb90c0664a0 0x7fb90c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.317+0000 7fb92e29d640 1 --2- 172.21.15.12:0/2863363961 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb90c0664a0 0x7fb90c068960 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb924009870 tx=0x7fb9240b5040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:33.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.321+0000 7fb91effd640 1 -- 172.21.15.12:0/2863363961 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb920050670 con 0x7fb928151800 2023-10-30T12:50:33.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.517+0000 7fb930501640 1 -- 172.21.15.12:0/2863363961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb92804e170 con 0x7fb928151800 2023-10-30T12:50:33.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.517+0000 7fb91effd640 1 -- 172.21.15.12:0/2863363961 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+6541 (secure 0 0 0) 0x7fb920053c90 con 0x7fb928151800 2023-10-30T12:50:33.523 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:50:33.524 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:50:13.522016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:33.524 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2023-10-30T12:50:33.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.517+0000 7fb930501640 1 -- 172.21.15.12:0/2863363961 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb90c0664a0 msgr2=0x7fb90c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:33.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.517+0000 7fb930501640 1 --2- 172.21.15.12:0/2863363961 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb90c0664a0 0x7fb90c068960 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb924009870 tx=0x7fb9240b5040 comp rx=0 tx=0).stop 2023-10-30T12:50:33.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.517+0000 7fb930501640 1 -- 172.21.15.12:0/2863363961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb928151800 msgr2=0x7fb92813be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:33.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.517+0000 7fb930501640 1 --2- 172.21.15.12:0/2863363961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb928151800 0x7fb92813be60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb9280c2350 tx=0x7fb92000cb90 comp rx=0 tx=0).stop 2023-10-30T12:50:33.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.517+0000 7fb930501640 1 -- 172.21.15.12:0/2863363961 shutdown_connections 2023-10-30T12:50:33.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.517+0000 7fb930501640 1 --2- 172.21.15.12:0/2863363961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb928151800 0x7fb92813be60 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:33.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.517+0000 7fb930501640 1 --2- 172.21.15.12:0/2863363961 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb90c0664a0 0x7fb90c068960 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.517+0000 7fb930501640 1 --2- 172.21.15.12:0/2863363961 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9280c9230 0x7fb92813b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.517+0000 7fb930501640 1 --2- 172.21.15.12:0/2863363961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9280c1ee0 0x7fb9280c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.517+0000 7fb930501640 1 -- 172.21.15.12:0/2863363961 >> 172.21.15.12:0/2863363961 conn(0x7fb928071ea0 msgr2=0x7fb92814f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.517+0000 7fb930501640 1 -- 172.21.15.12:0/2863363961 shutdown_connections 2023-10-30T12:50:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:33.517+0000 7fb930501640 1 -- 172.21.15.12:0/2863363961 wait complete. 2023-10-30T12:50:33.552 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:50:35.552 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:35.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a9446640 1 Processor -- start 2023-10-30T12:50:35.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a9446640 1 -- start start 2023-10-30T12:50:35.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a9446640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f94a406f7a0 0x7f94a406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:35.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a9446640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94a40c6bd0 0x7f94a4069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:35.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a9446640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a406a3d0 con 0x7f94a4071110 2023-10-30T12:50:35.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a9446640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a40700c0 con 0x7f94a406f7a0 2023-10-30T12:50:35.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a9446640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a4066cf0 con 0x7f94a40c6bd0 2023-10-30T12:50:35.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a37fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f94a4071110 0x7f94a4158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34894/0 (socket says 172.21.15.12:34894) 2023-10-30T12:50:35.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a37fe640 1 -- 172.21.15.12:0/4161757355 learned_addr learned my addr 172.21.15.12:0/4161757355 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:50:35.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f949bfff640 1 --2- 172.21.15.12:0/4161757355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94a40c6bd0 0x7f94a4069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:35.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a2ffd640 1 --2- 172.21.15.12:0/4161757355 >> v2:172.21.15.154:3300/0 conn(0x7f94a406f7a0 0x7f94a406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:35.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a27fc640 1 -- 172.21.15.12:0/4161757355 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296113780 0 0) 0x7f94a406a3d0 con 0x7f94a4071110 2023-10-30T12:50:35.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a27fc640 1 -- 172.21.15.12:0/4161757355 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9490003530 con 0x7f94a4071110 2023-10-30T12:50:35.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a2ffd640 1 -- 172.21.15.12:0/4161757355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94a40c6bd0 msgr2=0x7f94a4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:35.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a2ffd640 1 --2- 172.21.15.12:0/4161757355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94a40c6bd0 0x7f94a4069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:35.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a2ffd640 1 -- 172.21.15.12:0/4161757355 >> v1:172.21.15.12:6789/0 conn(0x7f94a4071110 legacy=0x7f94a4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.709+0000 7f94a2ffd640 1 -- 172.21.15.12:0/4161757355 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a40c70d0 con 0x7f94a406f7a0 2023-10-30T12:50:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.713+0000 7f94a2ffd640 1 --2- 172.21.15.12:0/4161757355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94a406f7a0 0x7f94a406fb80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f94940097c0 tx=0x7f949401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7881816b9aab2f6d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.713+0000 7f94a27fc640 1 -- 172.21.15.12:0/4161757355 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94940a8970 con 0x7f94a406f7a0 2023-10-30T12:50:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.713+0000 7f94a27fc640 1 -- 172.21.15.12:0/4161757355 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94940a8ad0 con 0x7f94a406f7a0 2023-10-30T12:50:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.713+0000 7f94a27fc640 1 -- 172.21.15.12:0/4161757355 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94940a8e40 con 0x7f94a406f7a0 2023-10-30T12:50:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.713+0000 7f94a9446640 1 -- 172.21.15.12:0/4161757355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94a406f7a0 msgr2=0x7f94a406fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.713+0000 7f94a9446640 1 --2- 172.21.15.12:0/4161757355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94a406f7a0 0x7f94a406fb80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f94940097c0 tx=0x7f949401ec70 comp rx=0 tx=0).stop 2023-10-30T12:50:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.713+0000 7f94a9446640 1 -- 172.21.15.12:0/4161757355 shutdown_connections 2023-10-30T12:50:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.713+0000 7f94a9446640 1 --2- 172.21.15.12:0/4161757355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94a40c6bd0 0x7f94a4069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.713+0000 7f94a9446640 1 --2- 172.21.15.12:0/4161757355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94a406f7a0 0x7f94a406fb80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.713+0000 7f94a9446640 1 -- 172.21.15.12:0/4161757355 >> 172.21.15.12:0/4161757355 conn(0x7f94a40c7910 msgr2=0x7f94a40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.713+0000 7f94a9446640 1 -- 172.21.15.12:0/4161757355 shutdown_connections 2023-10-30T12:50:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.713+0000 7f94a9446640 1 -- 172.21.15.12:0/4161757355 wait complete. 2023-10-30T12:50:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f94a9446640 1 Processor -- start 2023-10-30T12:50:35.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f94a9446640 1 -- start start 2023-10-30T12:50:35.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f94a9446640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94a406f7a0 0x7f94a40c1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:35.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f94a9446640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94a4071110 0x7f94a414be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:35.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f94a9446640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94a40c6bd0 0x7f94a414c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:35.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f94a9446640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a40adf50 con 0x7f94a4071110 2023-10-30T12:50:35.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f94a9446640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a40ad970 con 0x7f94a406f7a0 2023-10-30T12:50:35.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f94a9446640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a414d920 con 0x7f94a40c6bd0 2023-10-30T12:50:35.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f949bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94a4071110 0x7f94a414be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:35.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f949bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94a4071110 0x7f94a414be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44586/0 (socket says 172.21.15.12:44586) 2023-10-30T12:50:35.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f949bfff640 1 -- 172.21.15.12:0/1692742065 learned_addr learned my addr 172.21.15.12:0/1692742065 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:35.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f94a2ffd640 1 --2- 172.21.15.12:0/1692742065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94a406f7a0 0x7f94a40c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:35.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f94a37fe640 1 --2- 172.21.15.12:0/1692742065 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94a40c6bd0 0x7f94a414c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:35.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f949bfff640 1 -- 172.21.15.12:0/1692742065 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94a40c6bd0 msgr2=0x7f94a414c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:35.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f949bfff640 1 --2- 172.21.15.12:0/1692742065 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94a40c6bd0 0x7f94a414c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:35.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f949bfff640 1 -- 172.21.15.12:0/1692742065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94a406f7a0 msgr2=0x7f94a40c1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:35.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f949bfff640 1 --2- 172.21.15.12:0/1692742065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94a406f7a0 0x7f94a40c1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:35.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f949bfff640 1 -- 172.21.15.12:0/1692742065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9494009420 con 0x7f94a4071110 2023-10-30T12:50:35.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f94a2ffd640 1 --2- 172.21.15.12:0/1692742065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94a406f7a0 0x7f94a40c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:35.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f949bfff640 1 --2- 172.21.15.12:0/1692742065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94a4071110 0x7f94a414be60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f948c00e870 tx=0x7f948c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:35.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f949b7fe640 1 -- 172.21.15.12:0/1692742065 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c009510 con 0x7f94a4071110 2023-10-30T12:50:35.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f949b7fe640 1 -- 172.21.15.12:0/1692742065 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c00eec0 con 0x7f94a4071110 2023-10-30T12:50:35.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f94a9446640 1 -- 172.21.15.12:0/1692742065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a40af720 con 0x7f94a4071110 2023-10-30T12:50:35.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f949b7fe640 1 -- 172.21.15.12:0/1692742065 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c00cb40 con 0x7f94a4071110 2023-10-30T12:50:35.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.717+0000 7f94a9446640 1 -- 172.21.15.12:0/1692742065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a4150ad0 con 0x7f94a4071110 2023-10-30T12:50:35.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.721+0000 7f94a9446640 1 -- 172.21.15.12:0/1692742065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a404e170 con 0x7f94a4071110 2023-10-30T12:50:35.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.721+0000 7f949b7fe640 1 -- 172.21.15.12:0/1692742065 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f948c00ccc0 con 0x7f94a4071110 2023-10-30T12:50:35.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.721+0000 7f949b7fe640 1 --2- 172.21.15.12:0/1692742065 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94840664a0 0x7f9484068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.721+0000 7f949b7fe640 1 -- 172.21.15.12:0/1692742065 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f948c084240 con 0x7f94a4071110 2023-10-30T12:50:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.721+0000 7f94a2ffd640 1 --2- 172.21.15.12:0/1692742065 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94840664a0 0x7f9484068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.721+0000 7f94a2ffd640 1 --2- 172.21.15.12:0/1692742065 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94840664a0 0x7f9484068960 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f94940b2000 tx=0x7f94940aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.725+0000 7f949b7fe640 1 -- 172.21.15.12:0/1692742065 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f948c0513e0 con 0x7f94a4071110 2023-10-30T12:50:35.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.917+0000 7f94a9446640 1 -- 172.21.15.12:0/1692742065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94a40c4200 con 0x7f94a4071110 2023-10-30T12:50:35.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.917+0000 7f949b7fe640 1 -- 172.21.15.12:0/1692742065 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+6541 (secure 0 0 0) 0x7f948c054a00 con 0x7f94a4071110 2023-10-30T12:50:35.925 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:50:35.925 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:50:13.522016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:35.925 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2023-10-30T12:50:35.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.921+0000 7f94a9446640 1 -- 172.21.15.12:0/1692742065 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94840664a0 msgr2=0x7f9484068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:35.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.921+0000 7f94a9446640 1 --2- 172.21.15.12:0/1692742065 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94840664a0 0x7f9484068960 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f94940b2000 tx=0x7f94940aa070 comp rx=0 tx=0).stop 2023-10-30T12:50:35.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.921+0000 7f94a9446640 1 -- 172.21.15.12:0/1692742065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94a4071110 msgr2=0x7f94a414be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:35.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.921+0000 7f94a9446640 1 --2- 172.21.15.12:0/1692742065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94a4071110 0x7f94a414be60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f948c00e870 tx=0x7f948c00ec70 comp rx=0 tx=0).stop 2023-10-30T12:50:35.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.921+0000 7f94a9446640 1 -- 172.21.15.12:0/1692742065 shutdown_connections 2023-10-30T12:50:35.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.921+0000 7f94a9446640 1 --2- 172.21.15.12:0/1692742065 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94840664a0 0x7f9484068960 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:35.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.921+0000 7f94a9446640 1 --2- 172.21.15.12:0/1692742065 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94a40c6bd0 0x7f94a414c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:35.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.921+0000 7f94a9446640 1 --2- 172.21.15.12:0/1692742065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94a4071110 0x7f94a414be60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:35.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.921+0000 7f94a9446640 1 --2- 172.21.15.12:0/1692742065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94a406f7a0 0x7f94a40c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:35.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.921+0000 7f94a9446640 1 -- 172.21.15.12:0/1692742065 >> 172.21.15.12:0/1692742065 conn(0x7f94a40c7910 msgr2=0x7f94a4069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:35.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.921+0000 7f94a9446640 1 -- 172.21.15.12:0/1692742065 shutdown_connections 2023-10-30T12:50:35.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:35.921+0000 7f94a9446640 1 -- 172.21.15.12:0/1692742065 wait complete. 2023-10-30T12:50:35.946 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:50:36.338 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:50:36.342 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:50:37.950 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:38.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.097+0000 7fca0d194640 1 Processor -- start 2023-10-30T12:50:38.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.097+0000 7fca0d194640 1 -- start start 2023-10-30T12:50:38.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0d194640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fca08157010 0x7fca080715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:38.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0d194640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fca080c8460 0x7fca08070650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:38.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0d194640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08159810 0x7fca08071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:38.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0d194640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca080c21b0 con 0x7fca080c8460 2023-10-30T12:50:38.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0d194640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca080c1530 con 0x7fca08157010 2023-10-30T12:50:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0d194640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca080c0f70 con 0x7fca08159810 2023-10-30T12:50:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fca080c8460 0x7fca08070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fca080c8460 0x7fca08070650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44596/0 (socket says 172.21.15.12:44596) 2023-10-30T12:50:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0659c640 1 -- 172.21.15.12:0/2470064049 learned_addr learned my addr 172.21.15.12:0/2470064049 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca06d9d640 1 --2- 172.21.15.12:0/2470064049 >> v2:172.21.15.154:3300/0 conn(0x7fca08157010 0x7fca080715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0759e640 1 --2- 172.21.15.12:0/2470064049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08159810 0x7fca08071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0659c640 1 -- 172.21.15.12:0/2470064049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08159810 msgr2=0x7fca08071bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0659c640 1 --2- 172.21.15.12:0/2470064049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08159810 0x7fca08071bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0659c640 1 -- 172.21.15.12:0/2470064049 >> v2:172.21.15.154:3300/0 conn(0x7fca08157010 msgr2=0x7fca080715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0659c640 1 --2- 172.21.15.12:0/2470064049 >> v2:172.21.15.154:3300/0 conn(0x7fca08157010 0x7fca080715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0659c640 1 -- 172.21.15.12:0/2470064049 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca080c1250 con 0x7fca080c8460 2023-10-30T12:50:38.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca0659c640 1 --2- 172.21.15.12:0/2470064049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca080c8460 0x7fca08070650 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fca0806d550 tx=0x7fc9fc01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff136866aa0cba8e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:38.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.101+0000 7fca05d9b640 1 -- 172.21.15.12:0/2470064049 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc0a89b0 con 0x7fca080c8460 2023-10-30T12:50:38.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca05d9b640 1 -- 172.21.15.12:0/2470064049 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9fc0a8b10 con 0x7fca080c8460 2023-10-30T12:50:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca05d9b640 1 -- 172.21.15.12:0/2470064049 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc0a8ea0 con 0x7fca080c8460 2023-10-30T12:50:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 -- 172.21.15.12:0/2470064049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca080c8460 msgr2=0x7fca08070650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 --2- 172.21.15.12:0/2470064049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca080c8460 0x7fca08070650 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fca0806d550 tx=0x7fc9fc01ecf0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 -- 172.21.15.12:0/2470064049 shutdown_connections 2023-10-30T12:50:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 --2- 172.21.15.12:0/2470064049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08159810 0x7fca08071bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 --2- 172.21.15.12:0/2470064049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca080c8460 0x7fca08070650 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 --2- 172.21.15.12:0/2470064049 >> v2:172.21.15.154:3300/0 conn(0x7fca08157010 0x7fca080715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 -- 172.21.15.12:0/2470064049 >> 172.21.15.12:0/2470064049 conn(0x7fca080cbeb0 msgr2=0x7fca080cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 -- 172.21.15.12:0/2470064049 shutdown_connections 2023-10-30T12:50:38.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 -- 172.21.15.12:0/2470064049 wait complete. 2023-10-30T12:50:38.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 Processor -- start 2023-10-30T12:50:38.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 -- start start 2023-10-30T12:50:38.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca080c8460 0x7fca081e4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:38.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08157010 0x7fca08148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:38.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca08159810 0x7fca081491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:38.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca08069fc0 con 0x7fca080c8460 2023-10-30T12:50:38.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0806b910 con 0x7fca08159810 2023-10-30T12:50:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.105+0000 7fca0d194640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0806cd70 con 0x7fca08157010 2023-10-30T12:50:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fca06d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca080c8460 0x7fca081e4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fca06d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca080c8460 0x7fca081e4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44602/0 (socket says 172.21.15.12:44602) 2023-10-30T12:50:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fca06d9d640 1 -- 172.21.15.12:0/2165820656 learned_addr learned my addr 172.21.15.12:0/2165820656 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fca0659c640 1 --2- 172.21.15.12:0/2165820656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08157010 0x7fca08148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fca0759e640 1 --2- 172.21.15.12:0/2165820656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca08159810 0x7fca081491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fca06d9d640 1 -- 172.21.15.12:0/2165820656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08157010 msgr2=0x7fca08148c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fca06d9d640 1 --2- 172.21.15.12:0/2165820656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08157010 0x7fca08148c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fca06d9d640 1 -- 172.21.15.12:0/2165820656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca08159810 msgr2=0x7fca081491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fca06d9d640 1 --2- 172.21.15.12:0/2165820656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca08159810 0x7fca081491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fca06d9d640 1 -- 172.21.15.12:0/2165820656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc0094d0 con 0x7fca080c8460 2023-10-30T12:50:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fca0759e640 1 --2- 172.21.15.12:0/2165820656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca08159810 0x7fca081491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fca06d9d640 1 --2- 172.21.15.12:0/2165820656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca080c8460 0x7fca081e4e90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800d800 tx=0x7fc9f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fc9ef7fe640 1 -- 172.21.15.12:0/2165820656 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f80096a0 con 0x7fca080c8460 2023-10-30T12:50:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fc9ef7fe640 1 -- 172.21.15.12:0/2165820656 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f8009800 con 0x7fca080c8460 2023-10-30T12:50:38.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fca0d194640 1 -- 172.21.15.12:0/2165820656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0806b670 con 0x7fca080c8460 2023-10-30T12:50:38.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fc9ef7fe640 1 -- 172.21.15.12:0/2165820656 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8009b70 con 0x7fca080c8460 2023-10-30T12:50:38.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fca0d194640 1 -- 172.21.15.12:0/2165820656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca08152e10 con 0x7fca080c8460 2023-10-30T12:50:38.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fca0d194640 1 -- 172.21.15.12:0/2165820656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0804e170 con 0x7fca080c8460 2023-10-30T12:50:38.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.109+0000 7fc9ef7fe640 1 -- 172.21.15.12:0/2165820656 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc9f8009cd0 con 0x7fca080c8460 2023-10-30T12:50:38.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.113+0000 7fc9ef7fe640 1 --2- 172.21.15.12:0/2165820656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9f40664a0 0x7fc9f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:38.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.113+0000 7fc9ef7fe640 1 -- 172.21.15.12:0/2165820656 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fc9f80848a0 con 0x7fca080c8460 2023-10-30T12:50:38.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.113+0000 7fca0659c640 1 --2- 172.21.15.12:0/2165820656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9f40664a0 0x7fc9f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:38.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.113+0000 7fca0659c640 1 --2- 172.21.15.12:0/2165820656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9f40664a0 0x7fc9f4068960 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc0b2000 tx=0x7fc9fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:38.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.117+0000 7fc9ef7fe640 1 -- 172.21.15.12:0/2165820656 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc9f80519b0 con 0x7fca080c8460 2023-10-30T12:50:38.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.305+0000 7fca0d194640 1 -- 172.21.15.12:0/2165820656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca080b0d70 con 0x7fca080c8460 2023-10-30T12:50:38.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.305+0000 7fc9ef7fe640 1 -- 172.21.15.12:0/2165820656 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+6541 (secure 0 0 0) 0x7fc9f8054fd0 con 0x7fca080c8460 2023-10-30T12:50:38.311 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:50:38.311 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:50:13.522016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:38.312 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2023-10-30T12:50:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.305+0000 7fca0d194640 1 -- 172.21.15.12:0/2165820656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9f40664a0 msgr2=0x7fc9f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.305+0000 7fca0d194640 1 --2- 172.21.15.12:0/2165820656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9f40664a0 0x7fc9f4068960 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc0b2000 tx=0x7fc9fc009040 comp rx=0 tx=0).stop 2023-10-30T12:50:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.305+0000 7fca0d194640 1 -- 172.21.15.12:0/2165820656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca080c8460 msgr2=0x7fca081e4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.305+0000 7fca0d194640 1 --2- 172.21.15.12:0/2165820656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca080c8460 0x7fca081e4e90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800d800 tx=0x7fc9f800dcc0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.305+0000 7fca0d194640 1 -- 172.21.15.12:0/2165820656 shutdown_connections 2023-10-30T12:50:38.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.305+0000 7fca0d194640 1 --2- 172.21.15.12:0/2165820656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca08159810 0x7fca081491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.305+0000 7fca0d194640 1 --2- 172.21.15.12:0/2165820656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9f40664a0 0x7fc9f4068960 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.305+0000 7fca0d194640 1 --2- 172.21.15.12:0/2165820656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08157010 0x7fca08148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.305+0000 7fca0d194640 1 --2- 172.21.15.12:0/2165820656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca080c8460 0x7fca081e4e90 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.305+0000 7fca0d194640 1 -- 172.21.15.12:0/2165820656 >> 172.21.15.12:0/2165820656 conn(0x7fca080cbeb0 msgr2=0x7fca080cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:38.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.305+0000 7fca0d194640 1 -- 172.21.15.12:0/2165820656 shutdown_connections 2023-10-30T12:50:38.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:38.305+0000 7fca0d194640 1 -- 172.21.15.12:0/2165820656 wait complete. 2023-10-30T12:50:38.332 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:50:40.334 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:40.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.477+0000 7f5eea948640 1 Processor -- start 2023-10-30T12:50:40.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5eea948640 1 -- start start 2023-10-30T12:50:40.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5eea948640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5ee41538f0 0x7f5ee4159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:40.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5eea948640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee40ca6a0 con 0x7f5ee40cefa0 2023-10-30T12:50:40.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5eea948640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee415d300 con 0x7f5ee415cd50 2023-10-30T12:50:40.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5eea948640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee415d5e0 con 0x7f5ee41538f0 2023-10-30T12:50:40.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5eea147640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5ee40cefa0 0x7f5ee41640e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34908/0 (socket says 172.21.15.12:34908) 2023-10-30T12:50:40.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5eea147640 1 -- 172.21.15.12:0/2032589076 learned_addr learned my addr 172.21.15.12:0/2032589076 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:50:40.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee9946640 1 --2- 172.21.15.12:0/2032589076 >> v2:172.21.15.154:3300/0 conn(0x7f5ee41538f0 0x7f5ee4159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:40.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee8944640 1 -- 172.21.15.12:0/2032589076 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2460806985 0 0) 0x7f5ee40ca6a0 con 0x7f5ee40cefa0 2023-10-30T12:50:40.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee8944640 1 -- 172.21.15.12:0/2032589076 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed0003530 con 0x7f5ee40cefa0 2023-10-30T12:50:40.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee8944640 1 -- 172.21.15.12:0/2032589076 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 826616031 0 0) 0x7f5ed0003530 con 0x7f5ee40cefa0 2023-10-30T12:50:40.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee8944640 1 -- 172.21.15.12:0/2032589076 >> v2:172.21.15.154:3300/0 conn(0x7f5ee41538f0 msgr2=0x7f5ee4159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:40.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee8944640 1 --2- 172.21.15.12:0/2032589076 >> v2:172.21.15.154:3300/0 conn(0x7f5ee41538f0 0x7f5ee4159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee8944640 1 -- 172.21.15.12:0/2032589076 >> v1:172.21.15.154:6789/0 conn(0x7f5ee415cd50 legacy=0x7f5ee40c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:40.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee8944640 1 -- 172.21.15.12:0/2032589076 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee415d8c0 con 0x7f5ee40cefa0 2023-10-30T12:50:40.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee8944640 1 -- 172.21.15.12:0/2032589076 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f5edc001a00 con 0x7f5ee40cefa0 2023-10-30T12:50:40.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee8944640 1 -- 172.21.15.12:0/2032589076 >> v1:172.21.15.12:6789/0 conn(0x7f5ee40cefa0 legacy=0x7f5ee41640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:40.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee8944640 1 --2- 172.21.15.12:0/2032589076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed0003980 0x7f5ed0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:40.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee8944640 1 --2- 172.21.15.12:0/2032589076 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ed00047d0 0x7f5ed0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:40.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee8944640 1 --2- 172.21.15.12:0/2032589076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ed0005190 0x7f5ed000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:40.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee8944640 1 -- 172.21.15.12:0/2032589076 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee415d5e0 con 0x7f5ed0005190 2023-10-30T12:50:40.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee8944640 1 -- 172.21.15.12:0/2032589076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed000c040 con 0x7f5ed0003980 2023-10-30T12:50:40.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee8944640 1 -- 172.21.15.12:0/2032589076 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed000c200 con 0x7f5ed00047d0 2023-10-30T12:50:40.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5eea147640 1 --2- 172.21.15.12:0/2032589076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ed0005190 0x7f5ed000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:40.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee9145640 1 --2- 172.21.15.12:0/2032589076 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ed00047d0 0x7f5ed0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:40.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5ee9946640 1 --2- 172.21.15.12:0/2032589076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed0003980 0x7f5ed0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:40.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5eea147640 1 -- 172.21.15.12:0/2032589076 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ed00047d0 msgr2=0x7f5ed0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:40.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5eea147640 1 --2- 172.21.15.12:0/2032589076 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ed00047d0 0x7f5ed0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5eea147640 1 -- 172.21.15.12:0/2032589076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed0003980 msgr2=0x7f5ed0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:40.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5eea147640 1 --2- 172.21.15.12:0/2032589076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed0003980 0x7f5ed0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.481+0000 7f5eea147640 1 -- 172.21.15.12:0/2032589076 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed0003530 con 0x7f5ed0005190 2023-10-30T12:50:40.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.485+0000 7f5eea147640 1 -- 172.21.15.12:0/2032589076 reap_dead start 2023-10-30T12:50:40.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.485+0000 7f5eea147640 1 --2- 172.21.15.12:0/2032589076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ed0005190 0x7f5ed000b990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00a830 tx=0x7f5edc01d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea32f64346c33abb server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:40.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.485+0000 7f5ee8944640 1 -- 172.21.15.12:0/2032589076 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc0072d0 con 0x7f5ed0005190 2023-10-30T12:50:40.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.485+0000 7f5ee8944640 1 -- 172.21.15.12:0/2032589076 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5edc0a8070 con 0x7f5ed0005190 2023-10-30T12:50:40.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.485+0000 7f5ee8944640 1 -- 172.21.15.12:0/2032589076 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc0076b0 con 0x7f5ed0005190 2023-10-30T12:50:40.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.485+0000 7f5eea948640 1 -- 172.21.15.12:0/2032589076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ed0005190 msgr2=0x7f5ed000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:40.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.485+0000 7f5eea948640 1 --2- 172.21.15.12:0/2032589076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ed0005190 0x7f5ed000b990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00a830 tx=0x7f5edc01d340 comp rx=0 tx=0).stop 2023-10-30T12:50:40.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.485+0000 7f5eea948640 1 -- 172.21.15.12:0/2032589076 shutdown_connections 2023-10-30T12:50:40.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.485+0000 7f5eea948640 1 --2- 172.21.15.12:0/2032589076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ed0005190 0x7f5ed000b990 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.485+0000 7f5eea948640 1 -- 172.21.15.12:0/2032589076 >> 172.21.15.12:0/2032589076 conn(0x7f5ee406a640 msgr2=0x7f5ee4065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:40.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.485+0000 7f5eea948640 1 -- 172.21.15.12:0/2032589076 shutdown_connections 2023-10-30T12:50:40.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.485+0000 7f5eea948640 1 -- 172.21.15.12:0/2032589076 wait complete. 2023-10-30T12:50:40.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5eea948640 1 Processor -- start 2023-10-30T12:50:40.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5eea948640 1 -- start start 2023-10-30T12:50:40.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5eea948640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed0005190 0x7f5ee4152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:40.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5eea948640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ee40b2ea0 0x7f5ee4158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:40.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5eea948640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ee41502f0 0x7f5ee413ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:40.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5eea948640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee41595d0 con 0x7f5ee40b2ea0 2023-10-30T12:50:40.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5eea948640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee415cbb0 con 0x7f5ed0005190 2023-10-30T12:50:40.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5eea948640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee415c610 con 0x7f5ee41502f0 2023-10-30T12:50:40.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5ee9145640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ee40b2ea0 0x7f5ee4158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:40.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5ee9145640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ee40b2ea0 0x7f5ee4158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44620/0 (socket says 172.21.15.12:44620) 2023-10-30T12:50:40.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5ee9145640 1 -- 172.21.15.12:0/225553772 learned_addr learned my addr 172.21.15.12:0/225553772 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:40.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5ee9946640 1 --2- 172.21.15.12:0/225553772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed0005190 0x7f5ee4152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:40.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5eea147640 1 --2- 172.21.15.12:0/225553772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ee41502f0 0x7f5ee413ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:40.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5ee9145640 1 -- 172.21.15.12:0/225553772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ee41502f0 msgr2=0x7f5ee413ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:40.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5ee9145640 1 --2- 172.21.15.12:0/225553772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ee41502f0 0x7f5ee413ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5ee9145640 1 -- 172.21.15.12:0/225553772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed0005190 msgr2=0x7f5ee4152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:40.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5ee9145640 1 --2- 172.21.15.12:0/225553772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed0005190 0x7f5ee4152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5ee9145640 1 -- 172.21.15.12:0/225553772 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee415bdd0 con 0x7f5ee40b2ea0 2023-10-30T12:50:40.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5ee9145640 1 --2- 172.21.15.12:0/225553772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ee40b2ea0 0x7f5ee4158380 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400e4e0 tx=0x7f5ed400e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:40.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5eda7fc640 1 -- 172.21.15.12:0/225553772 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40052e0 con 0x7f5ee40b2ea0 2023-10-30T12:50:40.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5eda7fc640 1 -- 172.21.15.12:0/225553772 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed4005440 con 0x7f5ee40b2ea0 2023-10-30T12:50:40.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5eea948640 1 -- 172.21.15.12:0/225553772 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee415c8d0 con 0x7f5ee40b2ea0 2023-10-30T12:50:40.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5eda7fc640 1 -- 172.21.15.12:0/225553772 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4003840 con 0x7f5ee40b2ea0 2023-10-30T12:50:40.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.489+0000 7f5eea948640 1 -- 172.21.15.12:0/225553772 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee41588c0 con 0x7f5ee40b2ea0 2023-10-30T12:50:40.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.493+0000 7f5eea948640 1 -- 172.21.15.12:0/225553772 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee41581d0 con 0x7f5ee40b2ea0 2023-10-30T12:50:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.493+0000 7f5eda7fc640 1 -- 172.21.15.12:0/225553772 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5ed40039a0 con 0x7f5ee40b2ea0 2023-10-30T12:50:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.493+0000 7f5eda7fc640 1 --2- 172.21.15.12:0/225553772 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ec80664a0 0x7f5ec8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.493+0000 7f5eda7fc640 1 -- 172.21.15.12:0/225553772 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f5ed4083d80 con 0x7f5ee40b2ea0 2023-10-30T12:50:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.493+0000 7f5ee9946640 1 --2- 172.21.15.12:0/225553772 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ec80664a0 0x7f5ec8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:40.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.493+0000 7f5ee9946640 1 --2- 172.21.15.12:0/225553772 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ec80664a0 0x7f5ec8068960 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009a60 tx=0x7f5ee0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:40.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.497+0000 7f5eda7fc640 1 -- 172.21.15.12:0/225553772 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5ed4050f40 con 0x7f5ee40b2ea0 2023-10-30T12:50:40.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.697+0000 7f5eea948640 1 -- 172.21.15.12:0/225553772 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ee415a700 con 0x7f5ee40b2ea0 2023-10-30T12:50:40.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.701+0000 7f5eda7fc640 1 -- 172.21.15.12:0/225553772 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+6541 (secure 0 0 0) 0x7f5ee415a700 con 0x7f5ee40b2ea0 2023-10-30T12:50:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:50:40.707 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:50:13.522016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:40.708 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2023-10-30T12:50:40.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.701+0000 7f5eea948640 1 -- 172.21.15.12:0/225553772 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ec80664a0 msgr2=0x7f5ec8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:40.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.701+0000 7f5eea948640 1 --2- 172.21.15.12:0/225553772 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ec80664a0 0x7f5ec8068960 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009a60 tx=0x7f5ee0008040 comp rx=0 tx=0).stop 2023-10-30T12:50:40.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.701+0000 7f5eea948640 1 -- 172.21.15.12:0/225553772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ee40b2ea0 msgr2=0x7f5ee4158380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:40.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.701+0000 7f5eea948640 1 --2- 172.21.15.12:0/225553772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ee40b2ea0 0x7f5ee4158380 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400e4e0 tx=0x7f5ed400e9a0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.701+0000 7f5eea948640 1 -- 172.21.15.12:0/225553772 shutdown_connections 2023-10-30T12:50:40.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.701+0000 7f5eea948640 1 --2- 172.21.15.12:0/225553772 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ec80664a0 0x7f5ec8068960 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.701+0000 7f5eea948640 1 --2- 172.21.15.12:0/225553772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ee41502f0 0x7f5ee413ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.701+0000 7f5eea948640 1 --2- 172.21.15.12:0/225553772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ee40b2ea0 0x7f5ee4158380 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.701+0000 7f5eea948640 1 --2- 172.21.15.12:0/225553772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed0005190 0x7f5ee4152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.701+0000 7f5eea948640 1 -- 172.21.15.12:0/225553772 >> 172.21.15.12:0/225553772 conn(0x7f5ee406a640 msgr2=0x7f5ee40caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:40.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.701+0000 7f5eea948640 1 -- 172.21.15.12:0/225553772 shutdown_connections 2023-10-30T12:50:40.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:40.701+0000 7f5eea948640 1 -- 172.21.15.12:0/225553772 wait complete. 2023-10-30T12:50:40.727 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:50:42.728 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:42.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.865+0000 7f347d246640 1 Processor -- start 2023-10-30T12:50:42.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.865+0000 7f347d246640 1 -- start start 2023-10-30T12:50:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.869+0000 7f347d246640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f34780c6b80 0x7f34780bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.869+0000 7f347d246640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34780c6f60 0x7f34780c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.869+0000 7f347d246640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34780c0a50 con 0x7f34780c8430 2023-10-30T12:50:42.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.869+0000 7f347d246640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347806f6e0 con 0x7f34780c6b80 2023-10-30T12:50:42.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.869+0000 7f347d246640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347806f9c0 con 0x7f34780c6f60 2023-10-30T12:50:42.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.869+0000 7f3476d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f34780c6b80 0x7f34780bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:42.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.869+0000 7f3476d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f34780c6b80 0x7f34780bfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57944/0 (socket says 172.21.15.12:57944) 2023-10-30T12:50:42.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.869+0000 7f3476d9d640 1 -- 172.21.15.12:0/2635782890 learned_addr learned my addr 172.21.15.12:0/2635782890 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:42.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.869+0000 7f347659c640 1 --2- 172.21.15.12:0/2635782890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34780c6f60 0x7f34780c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:42.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.869+0000 7f3476d9d640 1 -- 172.21.15.12:0/2635782890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34780c6f60 msgr2=0x7f34780c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:42.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.869+0000 7f3476d9d640 1 --2- 172.21.15.12:0/2635782890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34780c6f60 0x7f34780c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.869+0000 7f3476d9d640 1 -- 172.21.15.12:0/2635782890 >> v1:172.21.15.154:6789/0 conn(0x7f34780c8430 legacy=0x7f347813ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:42.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.869+0000 7f3476d9d640 1 -- 172.21.15.12:0/2635782890 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347806fca0 con 0x7f34780c6b80 2023-10-30T12:50:42.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.873+0000 7f3476d9d640 1 --2- 172.21.15.12:0/2635782890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34780c6b80 0x7f34780bfec0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3468009870 tx=0x7f346801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dffece0a24ce99 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:42.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.873+0000 7f3475d9b640 1 -- 172.21.15.12:0/2635782890 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34680ac620 con 0x7f34780c6b80 2023-10-30T12:50:42.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.873+0000 7f3475d9b640 1 -- 172.21.15.12:0/2635782890 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34680ac780 con 0x7f34780c6b80 2023-10-30T12:50:42.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.873+0000 7f3475d9b640 1 -- 172.21.15.12:0/2635782890 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34680acb10 con 0x7f34780c6b80 2023-10-30T12:50:42.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.873+0000 7f347d246640 1 -- 172.21.15.12:0/2635782890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34780c6b80 msgr2=0x7f34780bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:42.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.873+0000 7f347d246640 1 --2- 172.21.15.12:0/2635782890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34780c6b80 0x7f34780bfec0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3468009870 tx=0x7f346801c4b0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.873+0000 7f347d246640 1 -- 172.21.15.12:0/2635782890 shutdown_connections 2023-10-30T12:50:42.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.873+0000 7f347d246640 1 --2- 172.21.15.12:0/2635782890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34780c6f60 0x7f34780c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.873+0000 7f347d246640 1 --2- 172.21.15.12:0/2635782890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34780c6b80 0x7f34780bfec0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.873+0000 7f347d246640 1 -- 172.21.15.12:0/2635782890 >> 172.21.15.12:0/2635782890 conn(0x7f34780c7910 msgr2=0x7f34780c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:42.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.873+0000 7f347d246640 1 -- 172.21.15.12:0/2635782890 shutdown_connections 2023-10-30T12:50:42.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.873+0000 7f347d246640 1 -- 172.21.15.12:0/2635782890 wait complete. 2023-10-30T12:50:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.873+0000 7f347d246640 1 Processor -- start 2023-10-30T12:50:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.873+0000 7f347d246640 1 -- start start 2023-10-30T12:50:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347d246640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34780c6b80 0x7f34780c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347d246640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34780c6f60 0x7f34781556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347d246640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34780c8430 0x7f3478155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347d246640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347813c370 con 0x7f34780c8430 2023-10-30T12:50:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347d246640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34781588f0 con 0x7f34780c6b80 2023-10-30T12:50:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347d246640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34780ca490 con 0x7f34780c6f60 2023-10-30T12:50:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34780c8430 0x7f3478155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34780c8430 0x7f3478155be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57956/0 (socket says 172.21.15.12:57956) 2023-10-30T12:50:42.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347759e640 1 -- 172.21.15.12:0/550037815 learned_addr learned my addr 172.21.15.12:0/550037815 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:42.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f3476d9d640 1 --2- 172.21.15.12:0/550037815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34780c6b80 0x7f34780c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:42.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347659c640 1 --2- 172.21.15.12:0/550037815 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34780c6f60 0x7f34781556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:42.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347759e640 1 -- 172.21.15.12:0/550037815 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34780c6f60 msgr2=0x7f34781556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:42.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347759e640 1 --2- 172.21.15.12:0/550037815 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34780c6f60 0x7f34781556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347759e640 1 -- 172.21.15.12:0/550037815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34780c6b80 msgr2=0x7f34780c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:42.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347759e640 1 --2- 172.21.15.12:0/550037815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34780c6b80 0x7f34780c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347759e640 1 -- 172.21.15.12:0/550037815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34680094d0 con 0x7f34780c8430 2023-10-30T12:50:42.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f3476d9d640 1 --2- 172.21.15.12:0/550037815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34780c6b80 0x7f34780c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:42.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347759e640 1 --2- 172.21.15.12:0/550037815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34780c8430 0x7f3478155be0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f347000c730 tx=0x7f347000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:42.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f345f7fe640 1 -- 172.21.15.12:0/550037815 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470003b80 con 0x7f34780c8430 2023-10-30T12:50:42.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f345f7fe640 1 -- 172.21.15.12:0/550037815 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3470003ce0 con 0x7f34780c8430 2023-10-30T12:50:42.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347d246640 1 -- 172.21.15.12:0/550037815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3478158b20 con 0x7f34780c8430 2023-10-30T12:50:42.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f345f7fe640 1 -- 172.21.15.12:0/550037815 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347000aa30 con 0x7f34780c8430 2023-10-30T12:50:42.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347d246640 1 -- 172.21.15.12:0/550037815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34780c9190 con 0x7f34780c8430 2023-10-30T12:50:42.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f347d246640 1 -- 172.21.15.12:0/550037815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347804e170 con 0x7f34780c8430 2023-10-30T12:50:42.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f345f7fe640 1 -- 172.21.15.12:0/550037815 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f347000ab90 con 0x7f34780c8430 2023-10-30T12:50:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.877+0000 7f345f7fe640 1 --2- 172.21.15.12:0/550037815 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34640664a0 0x7f3464068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.881+0000 7f345f7fe640 1 -- 172.21.15.12:0/550037815 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f3470083ff0 con 0x7f34780c8430 2023-10-30T12:50:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.881+0000 7f3476d9d640 1 --2- 172.21.15.12:0/550037815 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34640664a0 0x7f3464068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.881+0000 7f3476d9d640 1 --2- 172.21.15.12:0/550037815 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34640664a0 0x7f3464068960 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f34680ae000 tx=0x7f34680a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:42.881+0000 7f345f7fe640 1 -- 172.21.15.12:0/550037815 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f34700510d0 con 0x7f34780c8430 2023-10-30T12:50:43.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:43.109+0000 7f347d246640 1 -- 172.21.15.12:0/550037815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f347813f030 con 0x7f34780c8430 2023-10-30T12:50:43.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:43.109+0000 7f345f7fe640 1 -- 172.21.15.12:0/550037815 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+6541 (secure 0 0 0) 0x7f34700546f0 con 0x7f34780c8430 2023-10-30T12:50:43.117 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:50:43.117 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:50:13.522016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:43.118 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2023-10-30T12:50:43.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:43.113+0000 7f347d246640 1 -- 172.21.15.12:0/550037815 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34640664a0 msgr2=0x7f3464068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:43.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:43.113+0000 7f347d246640 1 --2- 172.21.15.12:0/550037815 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34640664a0 0x7f3464068960 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f34680ae000 tx=0x7f34680a7000 comp rx=0 tx=0).stop 2023-10-30T12:50:43.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:43.113+0000 7f347d246640 1 -- 172.21.15.12:0/550037815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34780c8430 msgr2=0x7f3478155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:43.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:43.113+0000 7f347d246640 1 --2- 172.21.15.12:0/550037815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34780c8430 0x7f3478155be0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f347000c730 tx=0x7f347000cbf0 comp rx=0 tx=0).stop 2023-10-30T12:50:43.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:43.113+0000 7f347d246640 1 -- 172.21.15.12:0/550037815 shutdown_connections 2023-10-30T12:50:43.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:43.113+0000 7f347d246640 1 --2- 172.21.15.12:0/550037815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34780c8430 0x7f3478155be0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:43.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:43.113+0000 7f347d246640 1 --2- 172.21.15.12:0/550037815 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f34640664a0 0x7f3464068960 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:43.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:43.113+0000 7f347d246640 1 --2- 172.21.15.12:0/550037815 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34780c6f60 0x7f34781556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:43.113+0000 7f347d246640 1 --2- 172.21.15.12:0/550037815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34780c6b80 0x7f34780c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:43.113+0000 7f347d246640 1 -- 172.21.15.12:0/550037815 >> 172.21.15.12:0/550037815 conn(0x7f34780c7910 msgr2=0x7f34780c0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:43.113+0000 7f347d246640 1 -- 172.21.15.12:0/550037815 shutdown_connections 2023-10-30T12:50:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:43.113+0000 7f347d246640 1 -- 172.21.15.12:0/550037815 wait complete. 2023-10-30T12:50:43.143 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:50:45.144 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:45.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe40673e640 1 Processor -- start 2023-10-30T12:50:45.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe40673e640 1 -- start start 2023-10-30T12:50:45.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe40673e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4001538a0 0x7fe40015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:45.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe40673e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4000cb800 con 0x7fe4000cf160 2023-10-30T12:50:45.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe40673e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40015dc30 con 0x7fe4000ca0e0 2023-10-30T12:50:45.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe40673e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4000cb330 con 0x7fe4001538a0 2023-10-30T12:50:45.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe404cdb640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe4000cf160 0x7fe400164070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38910/0 (socket says 172.21.15.12:38910) 2023-10-30T12:50:45.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe404cdb640 1 -- 172.21.15.12:0/3697643960 learned_addr learned my addr 172.21.15.12:0/3697643960 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:50:45.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3fffff640 1 --2- 172.21.15.12:0/3697643960 >> v2:172.21.15.154:3300/0 conn(0x7fe4001538a0 0x7fe40015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:45.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3feffd640 1 -- 172.21.15.12:0/3697643960 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676847256 0 0) 0x7fe4000cb800 con 0x7fe4000cf160 2023-10-30T12:50:45.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3feffd640 1 -- 172.21.15.12:0/3697643960 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3e8003530 con 0x7fe4000cf160 2023-10-30T12:50:45.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3feffd640 1 -- 172.21.15.12:0/3697643960 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3337737068 0 0) 0x7fe3e8003530 con 0x7fe4000cf160 2023-10-30T12:50:45.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3feffd640 1 -- 172.21.15.12:0/3697643960 >> v2:172.21.15.154:3300/0 conn(0x7fe4001538a0 msgr2=0x7fe40015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:45.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3feffd640 1 --2- 172.21.15.12:0/3697643960 >> v2:172.21.15.154:3300/0 conn(0x7fe4001538a0 0x7fe40015d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3feffd640 1 -- 172.21.15.12:0/3697643960 >> v1:172.21.15.154:6789/0 conn(0x7fe4000ca0e0 legacy=0x7fe40015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:45.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3feffd640 1 -- 172.21.15.12:0/3697643960 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4000cad70 con 0x7fe4000cf160 2023-10-30T12:50:45.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3feffd640 1 -- 172.21.15.12:0/3697643960 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fe3f8001a00 con 0x7fe4000cf160 2023-10-30T12:50:45.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3feffd640 1 -- 172.21.15.12:0/3697643960 >> v1:172.21.15.12:6789/0 conn(0x7fe4000cf160 legacy=0x7fe400164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:45.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3feffd640 1 --2- 172.21.15.12:0/3697643960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e8003980 0x7fe3e8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:45.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3feffd640 1 --2- 172.21.15.12:0/3697643960 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e80047d0 0x7fe3e8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:45.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3feffd640 1 --2- 172.21.15.12:0/3697643960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e8005190 0x7fe3e800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:45.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3feffd640 1 -- 172.21.15.12:0/3697643960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000cb330 con 0x7fe3e8003980 2023-10-30T12:50:45.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3feffd640 1 -- 172.21.15.12:0/3697643960 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e800c040 con 0x7fe3e8005190 2023-10-30T12:50:45.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3feffd640 1 -- 172.21.15.12:0/3697643960 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e800c200 con 0x7fe3e80047d0 2023-10-30T12:50:45.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3fffff640 1 --2- 172.21.15.12:0/3697643960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e8003980 0x7fe3e8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:45.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe404cdb640 1 --2- 172.21.15.12:0/3697643960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e8005190 0x7fe3e800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:45.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3ff7fe640 1 --2- 172.21.15.12:0/3697643960 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e80047d0 0x7fe3e8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:45.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3fffff640 1 -- 172.21.15.12:0/3697643960 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e80047d0 msgr2=0x7fe3e8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:45.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3fffff640 1 --2- 172.21.15.12:0/3697643960 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e80047d0 0x7fe3e8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3fffff640 1 -- 172.21.15.12:0/3697643960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e8005190 msgr2=0x7fe3e800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:45.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3fffff640 1 --2- 172.21.15.12:0/3697643960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3e8005190 0x7fe3e800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3fffff640 1 -- 172.21.15.12:0/3697643960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e8003530 con 0x7fe3e8003980 2023-10-30T12:50:45.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe404cdb640 1 -- 172.21.15.12:0/3697643960 reap_dead start 2023-10-30T12:50:45.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.285+0000 7fe3ff7fe640 1 --2- 172.21.15.12:0/3697643960 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3e80047d0 0x7fe3e8004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:50:45.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.289+0000 7fe3fffff640 1 --2- 172.21.15.12:0/3697643960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e8003980 0x7fe3e8003da0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009ae0 tx=0x7fe3f001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8634d2e3533173d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:45.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.289+0000 7fe3feffd640 1 -- 172.21.15.12:0/3697643960 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00a5530 con 0x7fe3e8003980 2023-10-30T12:50:45.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.289+0000 7fe3feffd640 1 -- 172.21.15.12:0/3697643960 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f00a5690 con 0x7fe3e8003980 2023-10-30T12:50:45.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.289+0000 7fe3feffd640 1 -- 172.21.15.12:0/3697643960 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00a5a40 con 0x7fe3e8003980 2023-10-30T12:50:45.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.289+0000 7fe40673e640 1 -- 172.21.15.12:0/3697643960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e8003980 msgr2=0x7fe3e8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:45.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.289+0000 7fe40673e640 1 --2- 172.21.15.12:0/3697643960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e8003980 0x7fe3e8003da0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009ae0 tx=0x7fe3f001ca50 comp rx=0 tx=0).stop 2023-10-30T12:50:45.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.289+0000 7fe40673e640 1 -- 172.21.15.12:0/3697643960 shutdown_connections 2023-10-30T12:50:45.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.289+0000 7fe40673e640 1 --2- 172.21.15.12:0/3697643960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e8003980 0x7fe3e8003da0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.289+0000 7fe40673e640 1 -- 172.21.15.12:0/3697643960 >> 172.21.15.12:0/3697643960 conn(0x7fe40006bfe0 msgr2=0x7fe40006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:45.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.289+0000 7fe40673e640 1 -- 172.21.15.12:0/3697643960 shutdown_connections 2023-10-30T12:50:45.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.289+0000 7fe40673e640 1 -- 172.21.15.12:0/3697643960 wait complete. 2023-10-30T12:50:45.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe40673e640 1 Processor -- start 2023-10-30T12:50:45.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe40673e640 1 -- start start 2023-10-30T12:50:45.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe40673e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e8003980 0x7fe4001f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:45.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe40673e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe400150c20 0x7fe4001e5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:45.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe40673e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4001f28b0 0x7fe4001e6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:45.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe40673e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001e53a0 con 0x7fe3e8003980 2023-10-30T12:50:45.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe40673e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001f0340 con 0x7fe4001f28b0 2023-10-30T12:50:45.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe40673e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001e4500 con 0x7fe400150c20 2023-10-30T12:50:45.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe3fffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e8003980 0x7fe4001f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:45.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe3fffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e8003980 0x7fe4001f3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57966/0 (socket says 172.21.15.12:57966) 2023-10-30T12:50:45.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe3fffff640 1 -- 172.21.15.12:0/2533695792 learned_addr learned my addr 172.21.15.12:0/2533695792 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:45.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe3ff7fe640 1 --2- 172.21.15.12:0/2533695792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe400150c20 0x7fe4001e5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:45.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe404cdb640 1 --2- 172.21.15.12:0/2533695792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4001f28b0 0x7fe4001e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:45.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe3fffff640 1 -- 172.21.15.12:0/2533695792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe400150c20 msgr2=0x7fe4001e5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:45.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe3fffff640 1 --2- 172.21.15.12:0/2533695792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe400150c20 0x7fe4001e5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe3fffff640 1 -- 172.21.15.12:0/2533695792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4001f28b0 msgr2=0x7fe4001e6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:45.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe3fffff640 1 --2- 172.21.15.12:0/2533695792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4001f28b0 0x7fe4001e6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe3fffff640 1 -- 172.21.15.12:0/2533695792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4001f3790 con 0x7fe3e8003980 2023-10-30T12:50:45.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe3fffff640 1 --2- 172.21.15.12:0/2533695792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e8003980 0x7fe4001f3250 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00b0040 tx=0x7fe3f00197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:45.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe3fcff9640 1 -- 172.21.15.12:0/2533695792 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00ba070 con 0x7fe3e8003980 2023-10-30T12:50:45.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe3fcff9640 1 -- 172.21.15.12:0/2533695792 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f0004020 con 0x7fe3e8003980 2023-10-30T12:50:45.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe3fcff9640 1 -- 172.21.15.12:0/2533695792 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00043d0 con 0x7fe3e8003980 2023-10-30T12:50:45.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe40673e640 1 -- 172.21.15.12:0/2533695792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4001590d0 con 0x7fe3e8003980 2023-10-30T12:50:45.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.293+0000 7fe40673e640 1 -- 172.21.15.12:0/2533695792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4001ef7c0 con 0x7fe3e8003980 2023-10-30T12:50:45.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.297+0000 7fe40673e640 1 -- 172.21.15.12:0/2533695792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40004e170 con 0x7fe3e8003980 2023-10-30T12:50:45.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.297+0000 7fe3fcff9640 1 -- 172.21.15.12:0/2533695792 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe3f00b4070 con 0x7fe3e8003980 2023-10-30T12:50:45.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.297+0000 7fe3fcff9640 1 --2- 172.21.15.12:0/2533695792 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe3ec0664a0 0x7fe3ec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:45.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.297+0000 7fe3fcff9640 1 -- 172.21.15.12:0/2533695792 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fe3f011ddf0 con 0x7fe3e8003980 2023-10-30T12:50:45.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.297+0000 7fe3ff7fe640 1 --2- 172.21.15.12:0/2533695792 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe3ec0664a0 0x7fe3ec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:45.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.297+0000 7fe3ff7fe640 1 --2- 172.21.15.12:0/2533695792 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe3ec0664a0 0x7fe3ec068960 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe400159570 tx=0x7fe3f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:45.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.301+0000 7fe3fcff9640 1 -- 172.21.15.12:0/2533695792 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe3f00eaf90 con 0x7fe3e8003980 2023-10-30T12:50:45.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.505+0000 7fe40673e640 1 -- 172.21.15.12:0/2533695792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe400157a40 con 0x7fe3e8003980 2023-10-30T12:50:45.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.505+0000 7fe3fcff9640 1 -- 172.21.15.12:0/2533695792 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+6541 (secure 0 0 0) 0x7fe3f00b2020 con 0x7fe3e8003980 2023-10-30T12:50:45.511 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:50:45.511 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:50:13.522016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:45.512 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2023-10-30T12:50:45.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.505+0000 7fe40673e640 1 -- 172.21.15.12:0/2533695792 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe3ec0664a0 msgr2=0x7fe3ec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:45.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.505+0000 7fe40673e640 1 --2- 172.21.15.12:0/2533695792 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe3ec0664a0 0x7fe3ec068960 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe400159570 tx=0x7fe3f4009040 comp rx=0 tx=0).stop 2023-10-30T12:50:45.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.505+0000 7fe40673e640 1 -- 172.21.15.12:0/2533695792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e8003980 msgr2=0x7fe4001f3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:45.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.505+0000 7fe40673e640 1 --2- 172.21.15.12:0/2533695792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e8003980 0x7fe4001f3250 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00b0040 tx=0x7fe3f00197f0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.505+0000 7fe40673e640 1 -- 172.21.15.12:0/2533695792 shutdown_connections 2023-10-30T12:50:45.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.505+0000 7fe40673e640 1 --2- 172.21.15.12:0/2533695792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4001f28b0 0x7fe4001e6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.505+0000 7fe40673e640 1 --2- 172.21.15.12:0/2533695792 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe3ec0664a0 0x7fe3ec068960 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.505+0000 7fe40673e640 1 --2- 172.21.15.12:0/2533695792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe400150c20 0x7fe4001e5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.505+0000 7fe40673e640 1 --2- 172.21.15.12:0/2533695792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e8003980 0x7fe4001f3250 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.505+0000 7fe40673e640 1 -- 172.21.15.12:0/2533695792 >> 172.21.15.12:0/2533695792 conn(0x7fe40006bfe0 msgr2=0x7fe400151920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:45.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.505+0000 7fe40673e640 1 -- 172.21.15.12:0/2533695792 shutdown_connections 2023-10-30T12:50:45.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:45.505+0000 7fe40673e640 1 -- 172.21.15.12:0/2533695792 wait complete. 2023-10-30T12:50:45.533 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:50:47.535 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:47.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d0c4c9640 1 Processor -- start 2023-10-30T12:50:47.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d0c4c9640 1 -- start start 2023-10-30T12:50:47.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d0c4c9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5d0415d6f0 0x7f5d0415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:47.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d0c4c9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d040cb800 con 0x7f5d040cf160 2023-10-30T12:50:47.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d0c4c9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d040cb330 con 0x7f5d041538a0 2023-10-30T12:50:47.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d0c4c9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d040cad70 con 0x7f5d0415d6f0 2023-10-30T12:50:47.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d0aa66640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5d040cf160 0x7f5d04164070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38922/0 (socket says 172.21.15.12:38922) 2023-10-30T12:50:47.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d0aa66640 1 -- 172.21.15.12:0/2107044128 learned_addr learned my addr 172.21.15.12:0/2107044128 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:50:47.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09a64640 1 --2- 172.21.15.12:0/2107044128 >> v2:172.21.15.154:3300/0 conn(0x7f5d0415d6f0 0x7f5d0415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:47.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09263640 1 -- 172.21.15.12:0/2107044128 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3675813983 0 0) 0x7f5d040cb800 con 0x7f5d040cf160 2023-10-30T12:50:47.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09263640 1 -- 172.21.15.12:0/2107044128 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cf0003530 con 0x7f5d040cf160 2023-10-30T12:50:47.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09263640 1 -- 172.21.15.12:0/2107044128 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2592931943 0 0) 0x7f5cf0003530 con 0x7f5d040cf160 2023-10-30T12:50:47.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09263640 1 -- 172.21.15.12:0/2107044128 >> v2:172.21.15.154:3300/0 conn(0x7f5d0415d6f0 msgr2=0x7f5d0415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:47.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09263640 1 --2- 172.21.15.12:0/2107044128 >> v2:172.21.15.154:3300/0 conn(0x7f5d0415d6f0 0x7f5d0415a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09263640 1 -- 172.21.15.12:0/2107044128 >> v1:172.21.15.154:6789/0 conn(0x7f5d041538a0 legacy=0x7f5d040ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:47.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09263640 1 -- 172.21.15.12:0/2107044128 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d040cb050 con 0x7f5d040cf160 2023-10-30T12:50:47.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09263640 1 -- 172.21.15.12:0/2107044128 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f5cfc001a00 con 0x7f5d040cf160 2023-10-30T12:50:47.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09263640 1 -- 172.21.15.12:0/2107044128 >> v1:172.21.15.12:6789/0 conn(0x7f5d040cf160 legacy=0x7f5d04164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:47.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09263640 1 --2- 172.21.15.12:0/2107044128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cf0003980 0x7f5cf0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:47.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09263640 1 --2- 172.21.15.12:0/2107044128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cf00047d0 0x7f5cf0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:47.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09263640 1 --2- 172.21.15.12:0/2107044128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cf0005190 0x7f5cf000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:47.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09263640 1 -- 172.21.15.12:0/2107044128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d040cad70 con 0x7f5cf00047d0 2023-10-30T12:50:47.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09263640 1 -- 172.21.15.12:0/2107044128 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf000c040 con 0x7f5cf0005190 2023-10-30T12:50:47.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09263640 1 -- 172.21.15.12:0/2107044128 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf000c200 con 0x7f5cf0003980 2023-10-30T12:50:47.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d09a64640 1 --2- 172.21.15.12:0/2107044128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cf00047d0 0x7f5cf0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:47.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d0a265640 1 --2- 172.21.15.12:0/2107044128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cf0003980 0x7f5cf0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:47.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.705+0000 7f5d0aa66640 1 --2- 172.21.15.12:0/2107044128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cf0005190 0x7f5cf000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:47.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.709+0000 7f5d09a64640 1 -- 172.21.15.12:0/2107044128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cf0003980 msgr2=0x7f5cf0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:47.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.709+0000 7f5d09a64640 1 --2- 172.21.15.12:0/2107044128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cf0003980 0x7f5cf0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.709+0000 7f5d09a64640 1 -- 172.21.15.12:0/2107044128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cf0005190 msgr2=0x7f5cf000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:47.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.709+0000 7f5d09a64640 1 --2- 172.21.15.12:0/2107044128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cf0005190 0x7f5cf000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.709+0000 7f5d09a64640 1 -- 172.21.15.12:0/2107044128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf0003530 con 0x7f5cf00047d0 2023-10-30T12:50:47.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.709+0000 7f5d0aa66640 1 -- 172.21.15.12:0/2107044128 reap_dead start 2023-10-30T12:50:47.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.709+0000 7f5d09a64640 1 --2- 172.21.15.12:0/2107044128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cf00047d0 0x7f5cf0004c50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009ae0 tx=0x7f5cf401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16994903f8ac7ffd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:47.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.709+0000 7f5d09263640 1 -- 172.21.15.12:0/2107044128 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40a5640 con 0x7f5cf00047d0 2023-10-30T12:50:47.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.709+0000 7f5d09263640 1 -- 172.21.15.12:0/2107044128 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf40a57a0 con 0x7f5cf00047d0 2023-10-30T12:50:47.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.709+0000 7f5d09263640 1 -- 172.21.15.12:0/2107044128 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40a5b50 con 0x7f5cf00047d0 2023-10-30T12:50:47.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.709+0000 7f5d0c4c9640 1 -- 172.21.15.12:0/2107044128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cf00047d0 msgr2=0x7f5cf0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:47.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.709+0000 7f5d0c4c9640 1 --2- 172.21.15.12:0/2107044128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cf00047d0 0x7f5cf0004c50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009ae0 tx=0x7f5cf401ca50 comp rx=0 tx=0).stop 2023-10-30T12:50:47.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.709+0000 7f5d0c4c9640 1 -- 172.21.15.12:0/2107044128 shutdown_connections 2023-10-30T12:50:47.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.709+0000 7f5d0c4c9640 1 --2- 172.21.15.12:0/2107044128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cf00047d0 0x7f5cf0004c50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.709+0000 7f5d0c4c9640 1 -- 172.21.15.12:0/2107044128 >> 172.21.15.12:0/2107044128 conn(0x7f5d0406bfe0 msgr2=0x7f5d0406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:47.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0c4c9640 1 -- 172.21.15.12:0/2107044128 shutdown_connections 2023-10-30T12:50:47.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0c4c9640 1 -- 172.21.15.12:0/2107044128 wait complete. 2023-10-30T12:50:47.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0c4c9640 1 Processor -- start 2023-10-30T12:50:47.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0c4c9640 1 -- start start 2023-10-30T12:50:47.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0c4c9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cf00047d0 0x7f5d04154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0c4c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d041e2aa0 0x7f5d040b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0c4c9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d041e3f30 0x7f5d041e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0c4c9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d041542c0 con 0x7f5d041e3f30 2023-10-30T12:50:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0c4c9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d041e48f0 con 0x7f5d041e2aa0 2023-10-30T12:50:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0c4c9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04159660 con 0x7f5cf00047d0 2023-10-30T12:50:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0aa66640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d041e3f30 0x7f5d041e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d09a64640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d041e2aa0 0x7f5d040b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0a265640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cf00047d0 0x7f5d04154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d09a64640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d041e2aa0 0x7f5d040b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:50724/0 (socket says 172.21.15.12:50724) 2023-10-30T12:50:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d09a64640 1 -- 172.21.15.12:0/4090815248 learned_addr learned my addr 172.21.15.12:0/4090815248 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0aa66640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d041e3f30 0x7f5d041e43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57986/0 (socket says 172.21.15.12:57986) 2023-10-30T12:50:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0aa66640 1 -- 172.21.15.12:0/4090815248 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cf00047d0 msgr2=0x7f5d04154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0aa66640 1 --2- 172.21.15.12:0/4090815248 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cf00047d0 0x7f5d04154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0aa66640 1 -- 172.21.15.12:0/4090815248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d041e2aa0 msgr2=0x7f5d040b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:47.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0aa66640 1 --2- 172.21.15.12:0/4090815248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d041e2aa0 0x7f5d040b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0aa66640 1 -- 172.21.15.12:0/4090815248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d04153fe0 con 0x7f5d041e3f30 2023-10-30T12:50:47.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5d0aa66640 1 --2- 172.21.15.12:0/4090815248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d041e3f30 0x7f5d041e43b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5d040cba80 tx=0x7f5cfc007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:47.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5cfaffd640 1 -- 172.21.15.12:0/4090815248 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc00e980 con 0x7f5d041e3f30 2023-10-30T12:50:47.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5cfaffd640 1 -- 172.21.15.12:0/4090815248 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cfc00eb20 con 0x7f5d041e3f30 2023-10-30T12:50:47.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.713+0000 7f5cfaffd640 1 -- 172.21.15.12:0/4090815248 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc00eed0 con 0x7f5d041e3f30 2023-10-30T12:50:47.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.717+0000 7f5d0c4c9640 1 -- 172.21.15.12:0/4090815248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d041e2fd0 con 0x7f5d041e3f30 2023-10-30T12:50:47.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.717+0000 7f5d0c4c9640 1 -- 172.21.15.12:0/4090815248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d041e3a10 con 0x7f5d041e3f30 2023-10-30T12:50:47.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.717+0000 7f5d0c4c9640 1 -- 172.21.15.12:0/4090815248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d04159820 con 0x7f5d041e3f30 2023-10-30T12:50:47.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.717+0000 7f5cfaffd640 1 -- 172.21.15.12:0/4090815248 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5cfc01d460 con 0x7f5d041e3f30 2023-10-30T12:50:47.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.717+0000 7f5cfaffd640 1 --2- 172.21.15.12:0/4090815248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ce80664a0 0x7f5ce8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:47.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.717+0000 7f5cfaffd640 1 -- 172.21.15.12:0/4090815248 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f5cfc084350 con 0x7f5d041e3f30 2023-10-30T12:50:47.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.717+0000 7f5d0a265640 1 --2- 172.21.15.12:0/4090815248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ce80664a0 0x7f5ce8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:47.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.717+0000 7f5d0a265640 1 --2- 172.21.15.12:0/4090815248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ce80664a0 0x7f5ce8068960 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5d00007fd0 tx=0x7f5d00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:47.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.721+0000 7f5cfaffd640 1 -- 172.21.15.12:0/4090815248 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5cfc0515a0 con 0x7f5d041e3f30 2023-10-30T12:50:47.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.913+0000 7f5d0c4c9640 1 -- 172.21.15.12:0/4090815248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d0404e170 con 0x7f5d041e3f30 2023-10-30T12:50:47.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.913+0000 7f5cfaffd640 1 -- 172.21.15.12:0/4090815248 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+6541 (secure 0 0 0) 0x7f5cfc054bc0 con 0x7f5d041e3f30 2023-10-30T12:50:47.921 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:50:47.921 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:50:13.522016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:47.922 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2023-10-30T12:50:47.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.917+0000 7f5d0c4c9640 1 -- 172.21.15.12:0/4090815248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ce80664a0 msgr2=0x7f5ce8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:47.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.917+0000 7f5d0c4c9640 1 --2- 172.21.15.12:0/4090815248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ce80664a0 0x7f5ce8068960 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5d00007fd0 tx=0x7f5d00009040 comp rx=0 tx=0).stop 2023-10-30T12:50:47.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.917+0000 7f5d0c4c9640 1 -- 172.21.15.12:0/4090815248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d041e3f30 msgr2=0x7f5d041e43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:47.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.917+0000 7f5d0c4c9640 1 --2- 172.21.15.12:0/4090815248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d041e3f30 0x7f5d041e43b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5d040cba80 tx=0x7f5cfc007af0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.917+0000 7f5d0c4c9640 1 -- 172.21.15.12:0/4090815248 shutdown_connections 2023-10-30T12:50:47.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.917+0000 7f5d0c4c9640 1 --2- 172.21.15.12:0/4090815248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d041e3f30 0x7f5d041e43b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.917+0000 7f5d0c4c9640 1 --2- 172.21.15.12:0/4090815248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d041e2aa0 0x7f5d040b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.917+0000 7f5d0c4c9640 1 --2- 172.21.15.12:0/4090815248 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ce80664a0 0x7f5ce8068960 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.917+0000 7f5d0c4c9640 1 --2- 172.21.15.12:0/4090815248 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cf00047d0 0x7f5d04154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.917+0000 7f5d0c4c9640 1 -- 172.21.15.12:0/4090815248 >> 172.21.15.12:0/4090815248 conn(0x7f5d0406bfe0 msgr2=0x7f5d04066310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:47.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.917+0000 7f5d0c4c9640 1 -- 172.21.15.12:0/4090815248 shutdown_connections 2023-10-30T12:50:47.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:47.917+0000 7f5d0c4c9640 1 -- 172.21.15.12:0/4090815248 wait complete. 2023-10-30T12:50:47.942 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:50:49.943 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:50.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.121+0000 7fa46253e640 1 Processor -- start 2023-10-30T12:50:50.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.121+0000 7fa46253e640 1 -- start start 2023-10-30T12:50:50.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.125+0000 7fa46253e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa45c1538a0 0x7fa45c15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:50.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.125+0000 7fa46253e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa45c0ca0e0 0x7fa45c15a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:50.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.125+0000 7fa46253e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa45c0cb800 con 0x7fa45c0cf160 2023-10-30T12:50:50.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.125+0000 7fa46253e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa45c15dc30 con 0x7fa45c0ca0e0 2023-10-30T12:50:50.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.125+0000 7fa46253e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa45c0cb330 con 0x7fa45c1538a0 2023-10-30T12:50:50.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.125+0000 7fa45b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa45c0ca0e0 0x7fa45c15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:50.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.125+0000 7fa45b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa45c0ca0e0 0x7fa45c15a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57990/0 (socket says 172.21.15.12:57990) 2023-10-30T12:50:50.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.125+0000 7fa45b7fe640 1 -- 172.21.15.12:0/3870748783 learned_addr learned my addr 172.21.15.12:0/3870748783 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:50.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.125+0000 7fa45bfff640 1 --2- 172.21.15.12:0/3870748783 >> v2:172.21.15.154:3300/0 conn(0x7fa45c1538a0 0x7fa45c15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:50.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.125+0000 7fa45b7fe640 1 -- 172.21.15.12:0/3870748783 >> v2:172.21.15.154:3300/0 conn(0x7fa45c1538a0 msgr2=0x7fa45c15d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:50.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.125+0000 7fa45b7fe640 1 --2- 172.21.15.12:0/3870748783 >> v2:172.21.15.154:3300/0 conn(0x7fa45c1538a0 0x7fa45c15d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:50.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.125+0000 7fa45b7fe640 1 -- 172.21.15.12:0/3870748783 >> v1:172.21.15.154:6789/0 conn(0x7fa45c0cf160 legacy=0x7fa45c0cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:50.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.125+0000 7fa45b7fe640 1 -- 172.21.15.12:0/3870748783 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45c0cad70 con 0x7fa45c0ca0e0 2023-10-30T12:50:50.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.129+0000 7fa45b7fe640 1 --2- 172.21.15.12:0/3870748783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa45c0ca0e0 0x7fa45c15a090 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0cfd10 tx=0x7fa45001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d25396af942fc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:50.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.129+0000 7fa45affd640 1 -- 172.21.15.12:0/3870748783 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4500a89f0 con 0x7fa45c0ca0e0 2023-10-30T12:50:50.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.129+0000 7fa45affd640 1 -- 172.21.15.12:0/3870748783 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4500a8b50 con 0x7fa45c0ca0e0 2023-10-30T12:50:50.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.129+0000 7fa45affd640 1 -- 172.21.15.12:0/3870748783 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4500b0440 con 0x7fa45c0ca0e0 2023-10-30T12:50:50.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.129+0000 7fa46253e640 1 -- 172.21.15.12:0/3870748783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa45c0ca0e0 msgr2=0x7fa45c15a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:50.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.129+0000 7fa46253e640 1 --2- 172.21.15.12:0/3870748783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa45c0ca0e0 0x7fa45c15a090 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0cfd10 tx=0x7fa45001ecf0 comp rx=0 tx=0).stop 2023-10-30T12:50:50.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.133+0000 7fa46253e640 1 -- 172.21.15.12:0/3870748783 shutdown_connections 2023-10-30T12:50:50.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.133+0000 7fa46253e640 1 --2- 172.21.15.12:0/3870748783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa45c0ca0e0 0x7fa45c15a090 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:50.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.133+0000 7fa46253e640 1 --2- 172.21.15.12:0/3870748783 >> v2:172.21.15.154:3300/0 conn(0x7fa45c1538a0 0x7fa45c15d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:50.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.133+0000 7fa46253e640 1 -- 172.21.15.12:0/3870748783 >> 172.21.15.12:0/3870748783 conn(0x7fa45c06bfe0 msgr2=0x7fa45c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:50.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.133+0000 7fa46253e640 1 -- 172.21.15.12:0/3870748783 shutdown_connections 2023-10-30T12:50:50.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.133+0000 7fa46253e640 1 -- 172.21.15.12:0/3870748783 wait complete. 2023-10-30T12:50:50.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.133+0000 7fa46253e640 1 Processor -- start 2023-10-30T12:50:50.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.133+0000 7fa46253e640 1 -- start start 2023-10-30T12:50:50.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.133+0000 7fa46253e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa45c0ca0e0 0x7fa45c140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:50.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.133+0000 7fa46253e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa45c0cf160 0x7fa45c0ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:50.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa46253e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa45c1538a0 0x7fa45c0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:50.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa46253e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c1e4380 con 0x7fa45c0cf160 2023-10-30T12:50:50.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa46253e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c158620 con 0x7fa45c1538a0 2023-10-30T12:50:50.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa46253e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c159690 con 0x7fa45c0ca0e0 2023-10-30T12:50:50.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa45b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa45c0cf160 0x7fa45c0ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:50.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa45b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa45c0cf160 0x7fa45c0ccfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57998/0 (socket says 172.21.15.12:57998) 2023-10-30T12:50:50.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa45b7fe640 1 -- 172.21.15.12:0/4253451984 learned_addr learned my addr 172.21.15.12:0/4253451984 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:50.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa460adb640 1 --2- 172.21.15.12:0/4253451984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa45c1538a0 0x7fa45c0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:50.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa45bfff640 1 --2- 172.21.15.12:0/4253451984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa45c0ca0e0 0x7fa45c140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:50.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa45b7fe640 1 -- 172.21.15.12:0/4253451984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa45c0ca0e0 msgr2=0x7fa45c140ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:50.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa45b7fe640 1 --2- 172.21.15.12:0/4253451984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa45c0ca0e0 0x7fa45c140ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:50.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa45b7fe640 1 -- 172.21.15.12:0/4253451984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa45c1538a0 msgr2=0x7fa45c0cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:50.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa45b7fe640 1 --2- 172.21.15.12:0/4253451984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa45c1538a0 0x7fa45c0cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:50.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa45b7fe640 1 -- 172.21.15.12:0/4253451984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4500094d0 con 0x7fa45c0cf160 2023-10-30T12:50:50.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa45bfff640 1 --2- 172.21.15.12:0/4253451984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa45c0ca0e0 0x7fa45c140ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:50.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa45b7fe640 1 --2- 172.21.15.12:0/4253451984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa45c0cf160 0x7fa45c0ccfb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa45001b900 tx=0x7fa45001ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa458ff9640 1 -- 172.21.15.12:0/4253451984 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4500ba600 con 0x7fa45c0cf160 2023-10-30T12:50:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa458ff9640 1 -- 172.21.15.12:0/4253451984 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4500ba760 con 0x7fa45c0cf160 2023-10-30T12:50:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa46253e640 1 -- 172.21.15.12:0/4253451984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45c1e51e0 con 0x7fa45c0cf160 2023-10-30T12:50:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa458ff9640 1 -- 172.21.15.12:0/4253451984 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4500bab10 con 0x7fa45c0cf160 2023-10-30T12:50:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa46253e640 1 -- 172.21.15.12:0/4253451984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45c1e5610 con 0x7fa45c0cf160 2023-10-30T12:50:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa46253e640 1 -- 172.21.15.12:0/4253451984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45c0b2a50 con 0x7fa45c0cf160 2023-10-30T12:50:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.137+0000 7fa458ff9640 1 -- 172.21.15.12:0/4253451984 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa4500bac70 con 0x7fa45c0cf160 2023-10-30T12:50:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.141+0000 7fa458ff9640 1 --2- 172.21.15.12:0/4253451984 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4480664d0 0x7fa448068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.141+0000 7fa458ff9640 1 -- 172.21.15.12:0/4253451984 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fa450122560 con 0x7fa45c0cf160 2023-10-30T12:50:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.141+0000 7fa45bfff640 1 --2- 172.21.15.12:0/4253451984 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4480664d0 0x7fa448068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:50.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.141+0000 7fa45bfff640 1 --2- 172.21.15.12:0/4253451984 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4480664d0 0x7fa448068990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0cf070 tx=0x7fa44c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:50.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.145+0000 7fa458ff9640 1 -- 172.21.15.12:0/4253451984 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa4500ef7a0 con 0x7fa45c0cf160 2023-10-30T12:50:50.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.341+0000 7fa46253e640 1 -- 172.21.15.12:0/4253451984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa45c158790 con 0x7fa45c0cf160 2023-10-30T12:50:50.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.341+0000 7fa458ff9640 1 -- 172.21.15.12:0/4253451984 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+6541 (secure 0 0 0) 0x7fa4500f2dc0 con 0x7fa45c0cf160 2023-10-30T12:50:50.350 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:50:50.350 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:50:13.522016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5427":{"gid":5427,"name":"e","rank":2,"incarnation":57,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6836/2219276371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":2219276371},{"type":"v1","addr":"172.21.15.12:6836","nonce":2219276371}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:50.350 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2023-10-30T12:50:50.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.345+0000 7fa46253e640 1 -- 172.21.15.12:0/4253451984 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4480664d0 msgr2=0x7fa448068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:50.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.345+0000 7fa46253e640 1 --2- 172.21.15.12:0/4253451984 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4480664d0 0x7fa448068990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0cf070 tx=0x7fa44c008040 comp rx=0 tx=0).stop 2023-10-30T12:50:50.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.345+0000 7fa46253e640 1 -- 172.21.15.12:0/4253451984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa45c0cf160 msgr2=0x7fa45c0ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:50.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.345+0000 7fa46253e640 1 --2- 172.21.15.12:0/4253451984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa45c0cf160 0x7fa45c0ccfb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa45001b900 tx=0x7fa45001ba90 comp rx=0 tx=0).stop 2023-10-30T12:50:50.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.345+0000 7fa46253e640 1 -- 172.21.15.12:0/4253451984 shutdown_connections 2023-10-30T12:50:50.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.345+0000 7fa46253e640 1 --2- 172.21.15.12:0/4253451984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa45c1538a0 0x7fa45c0cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:50.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.345+0000 7fa46253e640 1 --2- 172.21.15.12:0/4253451984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa45c0cf160 0x7fa45c0ccfb0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:50.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.345+0000 7fa46253e640 1 --2- 172.21.15.12:0/4253451984 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa4480664d0 0x7fa448068990 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:50.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.345+0000 7fa46253e640 1 --2- 172.21.15.12:0/4253451984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa45c0ca0e0 0x7fa45c140ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:50.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.345+0000 7fa46253e640 1 -- 172.21.15.12:0/4253451984 >> 172.21.15.12:0/4253451984 conn(0x7fa45c06bfe0 msgr2=0x7fa45c0c5210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:50.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.345+0000 7fa46253e640 1 -- 172.21.15.12:0/4253451984 shutdown_connections 2023-10-30T12:50:50.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:50.345+0000 7fa46253e640 1 -- 172.21.15.12:0/4253451984 wait complete. 2023-10-30T12:50:50.371 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:50:52.372 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:52.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f80455d3640 1 Processor -- start 2023-10-30T12:50:52.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f80455d3640 1 -- start start 2023-10-30T12:50:52.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f80455d3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f80400c90d0 0x7f80400cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f80455d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80400cf4d0 0x7f804015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f80455d3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804015ce30 con 0x7f80400c8cf0 2023-10-30T12:50:52.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f80455d3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8040159880 con 0x7f80400c90d0 2023-10-30T12:50:52.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f80455d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8040159ae0 con 0x7f80400cf4d0 2023-10-30T12:50:52.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f803effd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f80400c8cf0 0x7f80400c9e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:32844/0 (socket says 172.21.15.12:32844) 2023-10-30T12:50:52.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f803effd640 1 -- 172.21.15.12:0/3691573302 learned_addr learned my addr 172.21.15.12:0/3691573302 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:50:52.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f803e7fc640 1 --2- 172.21.15.12:0/3691573302 >> v2:172.21.15.12:3300/0 conn(0x7f80400c90d0 0x7f80400cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f803f7fe640 1 --2- 172.21.15.12:0/3691573302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80400cf4d0 0x7f804015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f803dffb640 1 -- 172.21.15.12:0/3691573302 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542690848 0 0) 0x7f804015ce30 con 0x7f80400c8cf0 2023-10-30T12:50:52.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f803dffb640 1 -- 172.21.15.12:0/3691573302 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8028003530 con 0x7f80400c8cf0 2023-10-30T12:50:52.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f803e7fc640 1 -- 172.21.15.12:0/3691573302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80400cf4d0 msgr2=0x7f804015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:52.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f803e7fc640 1 --2- 172.21.15.12:0/3691573302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80400cf4d0 0x7f804015d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f803e7fc640 1 -- 172.21.15.12:0/3691573302 >> v1:172.21.15.12:6789/0 conn(0x7f80400c8cf0 legacy=0x7f80400c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f803e7fc640 1 -- 172.21.15.12:0/3691573302 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8040159dc0 con 0x7f80400c90d0 2023-10-30T12:50:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.525+0000 7f803f7fe640 1 --2- 172.21.15.12:0/3691573302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80400cf4d0 0x7f804015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.529+0000 7f803e7fc640 1 --2- 172.21.15.12:0/3691573302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80400c90d0 0x7f80400cef90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f80340097c0 tx=0x7f803401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78b8543d388c90f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.529+0000 7f803dffb640 1 -- 172.21.15.12:0/3691573302 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80340ad660 con 0x7f80400c90d0 2023-10-30T12:50:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.529+0000 7f803dffb640 1 -- 172.21.15.12:0/3691573302 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80340ad7c0 con 0x7f80400c90d0 2023-10-30T12:50:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.529+0000 7f803dffb640 1 -- 172.21.15.12:0/3691573302 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80340adb50 con 0x7f80400c90d0 2023-10-30T12:50:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.529+0000 7f80455d3640 1 -- 172.21.15.12:0/3691573302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80400c90d0 msgr2=0x7f80400cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.529+0000 7f80455d3640 1 --2- 172.21.15.12:0/3691573302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80400c90d0 0x7f80400cef90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f80340097c0 tx=0x7f803401c450 comp rx=0 tx=0).stop 2023-10-30T12:50:52.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.529+0000 7f80455d3640 1 -- 172.21.15.12:0/3691573302 shutdown_connections 2023-10-30T12:50:52.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.529+0000 7f80455d3640 1 --2- 172.21.15.12:0/3691573302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80400cf4d0 0x7f804015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.529+0000 7f80455d3640 1 --2- 172.21.15.12:0/3691573302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80400c90d0 0x7f80400cef90 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.529+0000 7f80455d3640 1 -- 172.21.15.12:0/3691573302 >> 172.21.15.12:0/3691573302 conn(0x7f804006be40 msgr2=0x7f804006c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.529+0000 7f80455d3640 1 -- 172.21.15.12:0/3691573302 shutdown_connections 2023-10-30T12:50:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.529+0000 7f80455d3640 1 -- 172.21.15.12:0/3691573302 wait complete. 2023-10-30T12:50:52.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f80455d3640 1 Processor -- start 2023-10-30T12:50:52.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f80455d3640 1 -- start start 2023-10-30T12:50:52.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f80455d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80400c8cf0 0x7f80401f1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f80455d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80400c90d0 0x7f8040155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f80455d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80400cf4d0 0x7f804015c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f80455d3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80401562c0 con 0x7f80400cf4d0 2023-10-30T12:50:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f80455d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80401590a0 con 0x7f80400c8cf0 2023-10-30T12:50:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f80455d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80400c58e0 con 0x7f80400c90d0 2023-10-30T12:50:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f803f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80400cf4d0 0x7f804015c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f803f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80400cf4d0 0x7f804015c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36434/0 (socket says 172.21.15.12:36434) 2023-10-30T12:50:52.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f803f7fe640 1 -- 172.21.15.12:0/423116209 learned_addr learned my addr 172.21.15.12:0/423116209 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:52.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f803effd640 1 --2- 172.21.15.12:0/423116209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80400c8cf0 0x7f80401f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f803e7fc640 1 --2- 172.21.15.12:0/423116209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80400c90d0 0x7f8040155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f803f7fe640 1 -- 172.21.15.12:0/423116209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80400c90d0 msgr2=0x7f8040155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f803f7fe640 1 --2- 172.21.15.12:0/423116209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80400c90d0 0x7f8040155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f803f7fe640 1 -- 172.21.15.12:0/423116209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80400c8cf0 msgr2=0x7f80401f1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f803f7fe640 1 --2- 172.21.15.12:0/423116209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80400c8cf0 0x7f80401f1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f803f7fe640 1 -- 172.21.15.12:0/423116209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8034009420 con 0x7f80400cf4d0 2023-10-30T12:50:52.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f803effd640 1 --2- 172.21.15.12:0/423116209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80400c8cf0 0x7f80401f1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:52.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f803f7fe640 1 --2- 172.21.15.12:0/423116209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80400cf4d0 0x7f804015c460 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8038007ef0 tx=0x7f803800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f80277fe640 1 -- 172.21.15.12:0/423116209 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8038003ba0 con 0x7f80400cf4d0 2023-10-30T12:50:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f80277fe640 1 -- 172.21.15.12:0/423116209 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803800f040 con 0x7f80400cf4d0 2023-10-30T12:50:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f80455d3640 1 -- 172.21.15.12:0/423116209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80400c4a80 con 0x7f80400cf4d0 2023-10-30T12:50:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f80277fe640 1 -- 172.21.15.12:0/423116209 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803800de80 con 0x7f80400cf4d0 2023-10-30T12:50:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f80455d3640 1 -- 172.21.15.12:0/423116209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80400c5040 con 0x7f80400cf4d0 2023-10-30T12:50:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.533+0000 7f80455d3640 1 -- 172.21.15.12:0/423116209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80401541f0 con 0x7f80400cf4d0 2023-10-30T12:50:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.537+0000 7f80277fe640 1 -- 172.21.15.12:0/423116209 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8038004030 con 0x7f80400cf4d0 2023-10-30T12:50:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.537+0000 7f80277fe640 1 --2- 172.21.15.12:0/423116209 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f802c0664a0 0x7f802c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.537+0000 7f80277fe640 1 -- 172.21.15.12:0/423116209 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f80380844e0 con 0x7f80400cf4d0 2023-10-30T12:50:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.537+0000 7f803effd640 1 --2- 172.21.15.12:0/423116209 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f802c0664a0 0x7f802c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.537+0000 7f803effd640 1 --2- 172.21.15.12:0/423116209 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f802c0664a0 0x7f802c068960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f80300011f0 tx=0x7f8030009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.541+0000 7f80277fe640 1 -- 172.21.15.12:0/423116209 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f80401541f0 con 0x7f80400cf4d0 2023-10-30T12:50:52.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.761+0000 7f80455d3640 1 -- 172.21.15.12:0/423116209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80401541f0 con 0x7f80400cf4d0 2023-10-30T12:50:52.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.761+0000 7f80277fe640 1 -- 172.21.15.12:0/423116209 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5746 (secure 0 0 0) 0x7f80401541f0 con 0x7f80400cf4d0 2023-10-30T12:50:52.770 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:50:52.770 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:50:50.379976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:52.771 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 81 2023-10-30T12:50:52.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.765+0000 7f80455d3640 1 -- 172.21.15.12:0/423116209 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f802c0664a0 msgr2=0x7f802c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.765+0000 7f80455d3640 1 --2- 172.21.15.12:0/423116209 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f802c0664a0 0x7f802c068960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f80300011f0 tx=0x7f8030009040 comp rx=0 tx=0).stop 2023-10-30T12:50:52.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.765+0000 7f80455d3640 1 -- 172.21.15.12:0/423116209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80400cf4d0 msgr2=0x7f804015c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.765+0000 7f80455d3640 1 --2- 172.21.15.12:0/423116209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80400cf4d0 0x7f804015c460 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8038007ef0 tx=0x7f803800d4a0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.765+0000 7f80455d3640 1 -- 172.21.15.12:0/423116209 shutdown_connections 2023-10-30T12:50:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.765+0000 7f80455d3640 1 --2- 172.21.15.12:0/423116209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80400cf4d0 0x7f804015c460 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.765+0000 7f80455d3640 1 --2- 172.21.15.12:0/423116209 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f802c0664a0 0x7f802c068960 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.765+0000 7f80455d3640 1 --2- 172.21.15.12:0/423116209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80400c90d0 0x7f8040155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.765+0000 7f80455d3640 1 --2- 172.21.15.12:0/423116209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80400c8cf0 0x7f80401f1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.765+0000 7f80455d3640 1 -- 172.21.15.12:0/423116209 >> 172.21.15.12:0/423116209 conn(0x7f804006be40 msgr2=0x7f80400b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:52.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.765+0000 7f80455d3640 1 -- 172.21.15.12:0/423116209 shutdown_connections 2023-10-30T12:50:52.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.765+0000 7f80455d3640 1 -- 172.21.15.12:0/423116209 wait complete. 2023-10-30T12:50:52.791 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2023-10-30T12:50:52.791 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:50:52.791 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:52.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.929+0000 7f6fc4032640 1 Processor -- start 2023-10-30T12:50:52.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.929+0000 7f6fc4032640 1 -- start start 2023-10-30T12:50:52.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.929+0000 7f6fc4032640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6fbc159e60 0x7f6fbc06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.929+0000 7f6fc4032640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6fbc06a720 0x7f6fbc06ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.929+0000 7f6fc4032640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc06fa80 con 0x7f6fbc0bfec0 2023-10-30T12:50:52.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.929+0000 7f6fc4032640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc071290 con 0x7f6fbc06a720 2023-10-30T12:50:52.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.929+0000 7f6fc4032640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc071570 con 0x7f6fbc159e60 2023-10-30T12:50:52.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.929+0000 7f6fc25cf640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6fbc06a720 0x7f6fbc06ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.929+0000 7f6fc15cd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f6fbc0bfec0 0x7f6fbc0c1a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:32860/0 (socket says 172.21.15.12:32860) 2023-10-30T12:50:52.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.929+0000 7f6fc15cd640 1 -- 172.21.15.12:0/3937210281 learned_addr learned my addr 172.21.15.12:0/3937210281 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:50:52.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.929+0000 7f6fc1dce640 1 --2- 172.21.15.12:0/3937210281 >> v2:172.21.15.154:3300/0 conn(0x7f6fbc159e60 0x7f6fbc06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.929+0000 7f6fc25cf640 1 -- 172.21.15.12:0/3937210281 >> v2:172.21.15.154:3300/0 conn(0x7f6fbc159e60 msgr2=0x7f6fbc06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:52.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.929+0000 7f6fc25cf640 1 --2- 172.21.15.12:0/3937210281 >> v2:172.21.15.154:3300/0 conn(0x7f6fbc159e60 0x7f6fbc06f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.929+0000 7f6fc25cf640 1 -- 172.21.15.12:0/3937210281 >> v1:172.21.15.12:6789/0 conn(0x7f6fbc0bfec0 legacy=0x7f6fbc0c1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.929+0000 7f6fc25cf640 1 -- 172.21.15.12:0/3937210281 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc071850 con 0x7f6fbc06a720 2023-10-30T12:50:52.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.933+0000 7f6fc25cf640 1 --2- 172.21.15.12:0/3937210281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc06a720 0x7f6fbc06ab90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0c8950 tx=0x7f6fb401f100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad04898b55e661b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:52.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.933+0000 7f6fc0dcc640 1 -- 172.21.15.12:0/3937210281 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40af600 con 0x7f6fbc06a720 2023-10-30T12:50:52.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.933+0000 7f6fc0dcc640 1 -- 172.21.15.12:0/3937210281 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb40ab050 con 0x7f6fbc06a720 2023-10-30T12:50:52.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.933+0000 7f6fc0dcc640 1 -- 172.21.15.12:0/3937210281 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40af980 con 0x7f6fbc06a720 2023-10-30T12:50:52.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.933+0000 7f6fc4032640 1 -- 172.21.15.12:0/3937210281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc06a720 msgr2=0x7f6fbc06ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.933+0000 7f6fc4032640 1 --2- 172.21.15.12:0/3937210281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc06a720 0x7f6fbc06ab90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0c8950 tx=0x7f6fb401f100 comp rx=0 tx=0).stop 2023-10-30T12:50:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.933+0000 7f6fc4032640 1 -- 172.21.15.12:0/3937210281 shutdown_connections 2023-10-30T12:50:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.933+0000 7f6fc4032640 1 --2- 172.21.15.12:0/3937210281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc06a720 0x7f6fbc06ab90 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.933+0000 7f6fc4032640 1 --2- 172.21.15.12:0/3937210281 >> v2:172.21.15.154:3300/0 conn(0x7f6fbc159e60 0x7f6fbc06f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.933+0000 7f6fc4032640 1 -- 172.21.15.12:0/3937210281 >> 172.21.15.12:0/3937210281 conn(0x7f6fbc0c7910 msgr2=0x7f6fbc0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:52.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.933+0000 7f6fc4032640 1 -- 172.21.15.12:0/3937210281 shutdown_connections 2023-10-30T12:50:52.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.933+0000 7f6fc4032640 1 -- 172.21.15.12:0/3937210281 wait complete. 2023-10-30T12:50:52.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc4032640 1 Processor -- start 2023-10-30T12:50:52.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc4032640 1 -- start start 2023-10-30T12:50:52.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc4032640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fbc06a720 0x7f6fbc155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc4032640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc0bfec0 0x7f6fbc0c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc4032640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fbc159e60 0x7f6fbc1eb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc4032640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0c2f00 con 0x7f6fbc0bfec0 2023-10-30T12:50:52.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc4032640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc156310 con 0x7f6fbc159e60 2023-10-30T12:50:52.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc4032640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc156b70 con 0x7f6fbc06a720 2023-10-30T12:50:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc15cd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc0bfec0 0x7f6fbc0c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc15cd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc0bfec0 0x7f6fbc0c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36444/0 (socket says 172.21.15.12:36444) 2023-10-30T12:50:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc15cd640 1 -- 172.21.15.12:0/2840169685 learned_addr learned my addr 172.21.15.12:0/2840169685 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc25cf640 1 --2- 172.21.15.12:0/2840169685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fbc159e60 0x7f6fbc1eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc1dce640 1 --2- 172.21.15.12:0/2840169685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fbc06a720 0x7f6fbc155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc15cd640 1 -- 172.21.15.12:0/2840169685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fbc06a720 msgr2=0x7f6fbc155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc15cd640 1 --2- 172.21.15.12:0/2840169685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fbc06a720 0x7f6fbc155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc15cd640 1 -- 172.21.15.12:0/2840169685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fbc159e60 msgr2=0x7f6fbc1eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc15cd640 1 --2- 172.21.15.12:0/2840169685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fbc159e60 0x7f6fbc1eb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc15cd640 1 -- 172.21.15.12:0/2840169685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb4009a70 con 0x7f6fbc0bfec0 2023-10-30T12:50:52.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc15cd640 1 --2- 172.21.15.12:0/2840169685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc0bfec0 0x7f6fbc0c9e40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00c500 tx=0x7f6fac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:52.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fb27fc640 1 -- 172.21.15.12:0/2840169685 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac005cf0 con 0x7f6fbc0bfec0 2023-10-30T12:50:52.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fb27fc640 1 -- 172.21.15.12:0/2840169685 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac005e50 con 0x7f6fbc0bfec0 2023-10-30T12:50:52.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fb27fc640 1 -- 172.21.15.12:0/2840169685 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0072f0 con 0x7f6fbc0bfec0 2023-10-30T12:50:52.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc4032640 1 -- 172.21.15.12:0/2840169685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc1565b0 con 0x7f6fbc0bfec0 2023-10-30T12:50:52.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.937+0000 7f6fc4032640 1 -- 172.21.15.12:0/2840169685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc1eaf60 con 0x7f6fbc0bfec0 2023-10-30T12:50:52.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.941+0000 7f6fc4032640 1 -- 172.21.15.12:0/2840169685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fbc04e170 con 0x7f6fbc0bfec0 2023-10-30T12:50:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.941+0000 7f6fb27fc640 1 -- 172.21.15.12:0/2840169685 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6fac007450 con 0x7f6fbc0bfec0 2023-10-30T12:50:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.941+0000 7f6fb27fc640 1 --2- 172.21.15.12:0/2840169685 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6fa00664a0 0x7f6fa0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.941+0000 7f6fb27fc640 1 -- 172.21.15.12:0/2840169685 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f6fac083660 con 0x7f6fbc0bfec0 2023-10-30T12:50:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.941+0000 7f6fc1dce640 1 --2- 172.21.15.12:0/2840169685 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6fa00664a0 0x7f6fa0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.945+0000 7f6fc1dce640 1 --2- 172.21.15.12:0/2840169685 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6fa00664a0 0x7f6fa0068960 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800b930 tx=0x7f6fb8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:52.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:52.945+0000 7f6fb27fc640 1 -- 172.21.15.12:0/2840169685 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6fac050850 con 0x7f6fbc0bfec0 2023-10-30T12:50:53.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:53.133+0000 7f6fc4032640 1 -- 172.21.15.12:0/2840169685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fbc071b00 con 0x7f6fbc0bfec0 2023-10-30T12:50:53.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:53.137+0000 7f6fb27fc640 1 -- 172.21.15.12:0/2840169685 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5746 (secure 0 0 0) 0x7f6fac053e70 con 0x7f6fbc0bfec0 2023-10-30T12:50:53.143 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:50:53.143 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:50:50.379976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:53.144 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 81 2023-10-30T12:50:53.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:53.137+0000 7f6fc4032640 1 -- 172.21.15.12:0/2840169685 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6fa00664a0 msgr2=0x7f6fa0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:53.137+0000 7f6fc4032640 1 --2- 172.21.15.12:0/2840169685 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6fa00664a0 0x7f6fa0068960 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800b930 tx=0x7f6fb8008040 comp rx=0 tx=0).stop 2023-10-30T12:50:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:53.137+0000 7f6fc4032640 1 -- 172.21.15.12:0/2840169685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc0bfec0 msgr2=0x7f6fbc0c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:53.137+0000 7f6fc4032640 1 --2- 172.21.15.12:0/2840169685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc0bfec0 0x7f6fbc0c9e40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00c500 tx=0x7f6fac00c9c0 comp rx=0 tx=0).stop 2023-10-30T12:50:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:53.137+0000 7f6fc4032640 1 -- 172.21.15.12:0/2840169685 shutdown_connections 2023-10-30T12:50:53.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:53.137+0000 7f6fc4032640 1 --2- 172.21.15.12:0/2840169685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fbc159e60 0x7f6fbc1eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:53.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:53.137+0000 7f6fc4032640 1 --2- 172.21.15.12:0/2840169685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc0bfec0 0x7f6fbc0c9e40 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:53.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:53.137+0000 7f6fc4032640 1 --2- 172.21.15.12:0/2840169685 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6fa00664a0 0x7f6fa0068960 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:53.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:53.137+0000 7f6fc4032640 1 --2- 172.21.15.12:0/2840169685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fbc06a720 0x7f6fbc155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:53.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:53.137+0000 7f6fc4032640 1 -- 172.21.15.12:0/2840169685 >> 172.21.15.12:0/2840169685 conn(0x7f6fbc0c7910 msgr2=0x7f6fbc14f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:53.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:53.137+0000 7f6fc4032640 1 -- 172.21.15.12:0/2840169685 shutdown_connections 2023-10-30T12:50:53.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:53.137+0000 7f6fc4032640 1 -- 172.21.15.12:0/2840169685 wait complete. 2023-10-30T12:50:53.164 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 81, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 81, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5571": { "addr": "172.21.15.12:6837/2752088514", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2752088514, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 2752088514, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5571, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 19 }, "gid_5758": { "addr": "172.21.15.154:6839/2204547612", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 2204547612, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 2204547612, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5758, "incarnation": 73, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_5877": { "addr": "172.21.15.12:6835/2306258157", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 2306258157, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 2306258157, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5877, "incarnation": 81, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:50:50.379976+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5571, "mds_1": 5758, "mds_2": 5877 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:50:55.166 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:55.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.301+0000 7f0b1aa9f640 1 Processor -- start 2023-10-30T12:50:55.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.301+0000 7f0b1aa9f640 1 -- start start 2023-10-30T12:50:55.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b1aa9f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0b14071b90 0x7f0b1415ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:55.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b1aa9f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b140c6290 0x7f0b14161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:55.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b1aa9f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b1415b060 con 0x7f0b140717b0 2023-10-30T12:50:55.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b1aa9f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b14157530 con 0x7f0b14071b90 2023-10-30T12:50:55.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b1aa9f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140c4ab0 con 0x7f0b140c6290 2023-10-30T12:50:55.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b19a9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0b140717b0 0x7f0b141513c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:32862/0 (socket says 172.21.15.12:32862) 2023-10-30T12:50:55.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b19a9d640 1 -- 172.21.15.12:0/1258833364 learned_addr learned my addr 172.21.15.12:0/1258833364 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:50:55.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b1929c640 1 --2- 172.21.15.12:0/1258833364 >> v2:172.21.15.12:3300/0 conn(0x7f0b14071b90 0x7f0b1415ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:55.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b1a29e640 1 --2- 172.21.15.12:0/1258833364 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b140c6290 0x7f0b14161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:55.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b18a9b640 1 -- 172.21.15.12:0/1258833364 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102182881 0 0) 0x7f0b1415b060 con 0x7f0b140717b0 2023-10-30T12:50:55.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b18a9b640 1 -- 172.21.15.12:0/1258833364 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b00003530 con 0x7f0b140717b0 2023-10-30T12:50:55.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b1929c640 1 -- 172.21.15.12:0/1258833364 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b140c6290 msgr2=0x7f0b14161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:55.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b1929c640 1 --2- 172.21.15.12:0/1258833364 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b140c6290 0x7f0b14161eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:55.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b1929c640 1 -- 172.21.15.12:0/1258833364 >> v1:172.21.15.12:6789/0 conn(0x7f0b140717b0 legacy=0x7f0b141513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b1929c640 1 -- 172.21.15.12:0/1258833364 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b140c4d90 con 0x7f0b14071b90 2023-10-30T12:50:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.305+0000 7f0b1a29e640 1 --2- 172.21.15.12:0/1258833364 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b140c6290 0x7f0b14161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:55.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1929c640 1 --2- 172.21.15.12:0/1258833364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b14071b90 0x7f0b1415ab20 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0b040097c0 tx=0x7f0b0401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff6c90b863ab7b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:55.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b18a9b640 1 -- 172.21.15.12:0/1258833364 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b040a8970 con 0x7f0b14071b90 2023-10-30T12:50:55.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b18a9b640 1 -- 172.21.15.12:0/1258833364 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b040a8ad0 con 0x7f0b14071b90 2023-10-30T12:50:55.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b18a9b640 1 -- 172.21.15.12:0/1258833364 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b040a8e60 con 0x7f0b14071b90 2023-10-30T12:50:55.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 -- 172.21.15.12:0/1258833364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b14071b90 msgr2=0x7f0b1415ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:55.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 --2- 172.21.15.12:0/1258833364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b14071b90 0x7f0b1415ab20 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0b040097c0 tx=0x7f0b0401ec70 comp rx=0 tx=0).stop 2023-10-30T12:50:55.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 -- 172.21.15.12:0/1258833364 shutdown_connections 2023-10-30T12:50:55.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 --2- 172.21.15.12:0/1258833364 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b140c6290 0x7f0b14161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:55.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 --2- 172.21.15.12:0/1258833364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b14071b90 0x7f0b1415ab20 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:55.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 -- 172.21.15.12:0/1258833364 >> 172.21.15.12:0/1258833364 conn(0x7f0b140c75d0 msgr2=0x7f0b140c79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:55.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 -- 172.21.15.12:0/1258833364 shutdown_connections 2023-10-30T12:50:55.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 -- 172.21.15.12:0/1258833364 wait complete. 2023-10-30T12:50:55.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 Processor -- start 2023-10-30T12:50:55.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 -- start start 2023-10-30T12:50:55.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b140717b0 0x7f0b1414bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:55.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b14071b90 0x7f0b1414a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:55.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b140c6290 0x7f0b1414f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:55.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1414a9d0 con 0x7f0b140717b0 2023-10-30T12:50:55.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b141f0550 con 0x7f0b140c6290 2023-10-30T12:50:55.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.309+0000 7f0b1aa9f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b141efc40 con 0x7f0b14071b90 2023-10-30T12:50:55.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b19a9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b140717b0 0x7f0b1414bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:55.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b19a9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b140717b0 0x7f0b1414bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36470/0 (socket says 172.21.15.12:36470) 2023-10-30T12:50:55.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b19a9d640 1 -- 172.21.15.12:0/360305672 learned_addr learned my addr 172.21.15.12:0/360305672 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:55.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b1929c640 1 --2- 172.21.15.12:0/360305672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b14071b90 0x7f0b1414a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:55.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b1a29e640 1 --2- 172.21.15.12:0/360305672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b140c6290 0x7f0b1414f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:55.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b19a9d640 1 -- 172.21.15.12:0/360305672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b14071b90 msgr2=0x7f0b1414a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:55.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b19a9d640 1 --2- 172.21.15.12:0/360305672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b14071b90 0x7f0b1414a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:55.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b19a9d640 1 -- 172.21.15.12:0/360305672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b140c6290 msgr2=0x7f0b1414f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:55.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b19a9d640 1 --2- 172.21.15.12:0/360305672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b140c6290 0x7f0b1414f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:55.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b19a9d640 1 -- 172.21.15.12:0/360305672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b04009420 con 0x7f0b140717b0 2023-10-30T12:50:55.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b19a9d640 1 --2- 172.21.15.12:0/360305672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b140717b0 0x7f0b1414bdf0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c450 tx=0x7f0b1000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:55.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b0a7fc640 1 -- 172.21.15.12:0/360305672 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10007480 con 0x7f0b140717b0 2023-10-30T12:50:55.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b0a7fc640 1 -- 172.21.15.12:0/360305672 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b100075e0 con 0x7f0b140717b0 2023-10-30T12:50:55.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b0a7fc640 1 -- 172.21.15.12:0/360305672 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100059d0 con 0x7f0b140717b0 2023-10-30T12:50:55.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b1aa9f640 1 -- 172.21.15.12:0/360305672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1414abf0 con 0x7f0b140717b0 2023-10-30T12:50:55.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b1aa9f640 1 -- 172.21.15.12:0/360305672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b141eff00 con 0x7f0b140717b0 2023-10-30T12:50:55.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b1aa9f640 1 -- 172.21.15.12:0/360305672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1404e170 con 0x7f0b140717b0 2023-10-30T12:50:55.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.313+0000 7f0b0a7fc640 1 -- 172.21.15.12:0/360305672 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0b10005b30 con 0x7f0b140717b0 2023-10-30T12:50:55.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.317+0000 7f0b0a7fc640 1 --2- 172.21.15.12:0/360305672 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0af80664a0 0x7f0af8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:55.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.317+0000 7f0b0a7fc640 1 -- 172.21.15.12:0/360305672 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f0b10083790 con 0x7f0b140717b0 2023-10-30T12:50:55.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.317+0000 7f0b1929c640 1 --2- 172.21.15.12:0/360305672 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0af80664a0 0x7f0af8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:55.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.317+0000 7f0b1929c640 1 --2- 172.21.15.12:0/360305672 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0af80664a0 0x7f0af8068960 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0b040b2000 tx=0x7f0b04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:55.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.321+0000 7f0b0a7fc640 1 -- 172.21.15.12:0/360305672 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0b100542f0 con 0x7f0b140717b0 2023-10-30T12:50:55.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.525+0000 7f0b1aa9f640 1 -- 172.21.15.12:0/360305672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b140c3410 con 0x7f0b140717b0 2023-10-30T12:50:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.525+0000 7f0b0a7fc640 1 -- 172.21.15.12:0/360305672 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5746 (secure 0 0 0) 0x7f0b10017020 con 0x7f0b140717b0 2023-10-30T12:50:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:50:55.534 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:50:50.379976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 81 2023-10-30T12:50:55.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.529+0000 7f0b1aa9f640 1 -- 172.21.15.12:0/360305672 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0af80664a0 msgr2=0x7f0af8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.529+0000 7f0b1aa9f640 1 --2- 172.21.15.12:0/360305672 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0af80664a0 0x7f0af8068960 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0b040b2000 tx=0x7f0b04008040 comp rx=0 tx=0).stop 2023-10-30T12:50:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.529+0000 7f0b1aa9f640 1 -- 172.21.15.12:0/360305672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b140717b0 msgr2=0x7f0b1414bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.529+0000 7f0b1aa9f640 1 --2- 172.21.15.12:0/360305672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b140717b0 0x7f0b1414bdf0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c450 tx=0x7f0b1000c910 comp rx=0 tx=0).stop 2023-10-30T12:50:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.529+0000 7f0b1aa9f640 1 -- 172.21.15.12:0/360305672 shutdown_connections 2023-10-30T12:50:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.529+0000 7f0b1aa9f640 1 --2- 172.21.15.12:0/360305672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b140c6290 0x7f0b1414f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.529+0000 7f0b1aa9f640 1 --2- 172.21.15.12:0/360305672 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0af80664a0 0x7f0af8068960 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.529+0000 7f0b1aa9f640 1 --2- 172.21.15.12:0/360305672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b14071b90 0x7f0b1414a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.529+0000 7f0b1aa9f640 1 --2- 172.21.15.12:0/360305672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b140717b0 0x7f0b1414bdf0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.529+0000 7f0b1aa9f640 1 -- 172.21.15.12:0/360305672 >> 172.21.15.12:0/360305672 conn(0x7f0b140c75d0 msgr2=0x7f0b1406ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.529+0000 7f0b1aa9f640 1 -- 172.21.15.12:0/360305672 shutdown_connections 2023-10-30T12:50:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:55.529+0000 7f0b1aa9f640 1 -- 172.21.15.12:0/360305672 wait complete. 2023-10-30T12:50:55.566 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:50:57.567 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.749+0000 7fc0bffc4640 1 Processor -- start 2023-10-30T12:50:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.749+0000 7fc0bffc4640 1 -- start start 2023-10-30T12:50:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0bffc4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0b81538a0 0x7fc0b815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0bffc4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b80cf160 0x7fc0b80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0bffc4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b80cb800 con 0x7fc0b80ca0e0 2023-10-30T12:50:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0bffc4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b815dc30 con 0x7fc0b81538a0 2023-10-30T12:50:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0bffc4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b80cb330 con 0x7fc0b80cf160 2023-10-30T12:50:57.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0be561640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b80cf160 0x7fc0b80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:57.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0bdd60640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0b81538a0 0x7fc0b815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:57.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0be561640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b80cf160 0x7fc0b80cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:56834/0 (socket says 172.21.15.12:56834) 2023-10-30T12:50:57.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0bdd60640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0b81538a0 0x7fc0b815d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:38182/0 (socket says 172.21.15.12:38182) 2023-10-30T12:50:57.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0be561640 1 -- 172.21.15.12:0/634962647 learned_addr learned my addr 172.21.15.12:0/634962647 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0bd55f640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc0b80ca0e0 0x7fc0b815a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:54194/0 (socket says 172.21.15.12:54194) 2023-10-30T12:50:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0be561640 1 -- 172.21.15.12:0/634962647 >> v2:172.21.15.154:3300/0 conn(0x7fc0b81538a0 msgr2=0x7fc0b815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0be561640 1 --2- 172.21.15.12:0/634962647 >> v2:172.21.15.154:3300/0 conn(0x7fc0b81538a0 0x7fc0b815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0be561640 1 -- 172.21.15.12:0/634962647 >> v1:172.21.15.154:6789/0 conn(0x7fc0b80ca0e0 legacy=0x7fc0b815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0be561640 1 -- 172.21.15.12:0/634962647 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b80cad70 con 0x7fc0b80cf160 2023-10-30T12:50:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.753+0000 7fc0bdd60640 1 --2- 172.21.15.12:0/634962647 >> v2:172.21.15.154:3300/0 conn(0x7fc0b81538a0 0x7fc0b815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:50:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.757+0000 7fc0be561640 1 --2- 172.21.15.12:0/634962647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b80cf160 0x7fc0b80cf590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0009e50 tx=0x7fc0b001f1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcec2764f46ee12c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.757+0000 7fc0bcd5e640 1 -- 172.21.15.12:0/634962647 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b00af400 con 0x7fc0b80cf160 2023-10-30T12:50:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.757+0000 7fc0bcd5e640 1 -- 172.21.15.12:0/634962647 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b00ab050 con 0x7fc0b80cf160 2023-10-30T12:50:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.757+0000 7fc0bcd5e640 1 -- 172.21.15.12:0/634962647 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b00af780 con 0x7fc0b80cf160 2023-10-30T12:50:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.757+0000 7fc0bffc4640 1 -- 172.21.15.12:0/634962647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b80cf160 msgr2=0x7fc0b80cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.757+0000 7fc0bffc4640 1 --2- 172.21.15.12:0/634962647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b80cf160 0x7fc0b80cf590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0009e50 tx=0x7fc0b001f1f0 comp rx=0 tx=0).stop 2023-10-30T12:50:57.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.757+0000 7fc0bffc4640 1 -- 172.21.15.12:0/634962647 shutdown_connections 2023-10-30T12:50:57.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.757+0000 7fc0bffc4640 1 --2- 172.21.15.12:0/634962647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b80cf160 0x7fc0b80cf590 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:57.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.757+0000 7fc0bffc4640 1 --2- 172.21.15.12:0/634962647 >> v2:172.21.15.154:3300/0 conn(0x7fc0b81538a0 0x7fc0b815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:57.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.757+0000 7fc0bffc4640 1 -- 172.21.15.12:0/634962647 >> 172.21.15.12:0/634962647 conn(0x7fc0b806bfe0 msgr2=0x7fc0b806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:57.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.757+0000 7fc0bffc4640 1 -- 172.21.15.12:0/634962647 shutdown_connections 2023-10-30T12:50:57.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.757+0000 7fc0bffc4640 1 -- 172.21.15.12:0/634962647 wait complete. 2023-10-30T12:50:57.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.757+0000 7fc0bffc4640 1 Processor -- start 2023-10-30T12:50:57.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bffc4640 1 -- start start 2023-10-30T12:50:57.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bffc4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0b80ca0e0 0x7fc0b81e6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:57.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bffc4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0b80cf160 0x7fc0b80ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:57.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bffc4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b81538a0 0x7fc0b81ef110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:57.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bffc4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b81e6c70 con 0x7fc0b80ca0e0 2023-10-30T12:50:57.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bffc4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b81f35c0 con 0x7fc0b80cf160 2023-10-30T12:50:57.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bffc4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b8159460 con 0x7fc0b81538a0 2023-10-30T12:50:57.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bdd60640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0b80ca0e0 0x7fc0b81e6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:57.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bdd60640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0b80ca0e0 0x7fc0b81e6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36482/0 (socket says 172.21.15.12:36482) 2023-10-30T12:50:57.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bdd60640 1 -- 172.21.15.12:0/2055218108 learned_addr learned my addr 172.21.15.12:0/2055218108 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:50:57.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bd55f640 1 --2- 172.21.15.12:0/2055218108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0b80cf160 0x7fc0b80ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:57.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0be561640 1 --2- 172.21.15.12:0/2055218108 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b81538a0 0x7fc0b81ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:57.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bdd60640 1 -- 172.21.15.12:0/2055218108 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b81538a0 msgr2=0x7fc0b81ef110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:57.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bdd60640 1 --2- 172.21.15.12:0/2055218108 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b81538a0 0x7fc0b81ef110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:57.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bdd60640 1 -- 172.21.15.12:0/2055218108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0b80cf160 msgr2=0x7fc0b80ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:57.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bdd60640 1 --2- 172.21.15.12:0/2055218108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0b80cf160 0x7fc0b80ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:57.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bdd60640 1 -- 172.21.15.12:0/2055218108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b4009420 con 0x7fc0b80ca0e0 2023-10-30T12:50:57.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bdd60640 1 --2- 172.21.15.12:0/2055218108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0b80ca0e0 0x7fc0b81e6730 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400e9b0 tx=0x7fc0b400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:57.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0ae7fc640 1 -- 172.21.15.12:0/2055218108 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b400ca00 con 0x7fc0b80ca0e0 2023-10-30T12:50:57.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0ae7fc640 1 -- 172.21.15.12:0/2055218108 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b400cb60 con 0x7fc0b80ca0e0 2023-10-30T12:50:57.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0ae7fc640 1 -- 172.21.15.12:0/2055218108 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4013470 con 0x7fc0b80ca0e0 2023-10-30T12:50:57.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bffc4640 1 -- 172.21.15.12:0/2055218108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b0009b30 con 0x7fc0b80ca0e0 2023-10-30T12:50:57.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bffc4640 1 -- 172.21.15.12:0/2055218108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b81f2ae0 con 0x7fc0b80ca0e0 2023-10-30T12:50:57.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.761+0000 7fc0bffc4640 1 -- 172.21.15.12:0/2055218108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b80b2a50 con 0x7fc0b80ca0e0 2023-10-30T12:50:57.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.765+0000 7fc0ae7fc640 1 -- 172.21.15.12:0/2055218108 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc0b40135d0 con 0x7fc0b80ca0e0 2023-10-30T12:50:57.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.765+0000 7fc0ae7fc640 1 --2- 172.21.15.12:0/2055218108 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc09c0664a0 0x7fc09c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:57.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.765+0000 7fc0ae7fc640 1 -- 172.21.15.12:0/2055218108 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7fc0b4084550 con 0x7fc0b80ca0e0 2023-10-30T12:50:57.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.765+0000 7fc0bd55f640 1 --2- 172.21.15.12:0/2055218108 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc09c0664a0 0x7fc09c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:57.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.765+0000 7fc0bd55f640 1 --2- 172.21.15.12:0/2055218108 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc09c0664a0 0x7fc09c068960 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8158780 tx=0x7fc0a8007cd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:57.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.769+0000 7fc0ae7fc640 1 -- 172.21.15.12:0/2055218108 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc0b4051570 con 0x7fc0b80ca0e0 2023-10-30T12:50:57.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.957+0000 7fc0bffc4640 1 -- 172.21.15.12:0/2055218108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0b804e170 con 0x7fc0b80ca0e0 2023-10-30T12:50:57.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.957+0000 7fc0ae7fc640 1 -- 172.21.15.12:0/2055218108 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+5748 (secure 0 0 0) 0x7fc0b4054b90 con 0x7fc0b80ca0e0 2023-10-30T12:50:57.966 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:50:57.966 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:50:56.971225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:resolve","state_seq":16,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 82 2023-10-30T12:50:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.961+0000 7fc0bffc4640 1 -- 172.21.15.12:0/2055218108 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc09c0664a0 msgr2=0x7fc09c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.961+0000 7fc0bffc4640 1 --2- 172.21.15.12:0/2055218108 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc09c0664a0 0x7fc09c068960 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8158780 tx=0x7fc0a8007cd0 comp rx=0 tx=0).stop 2023-10-30T12:50:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.961+0000 7fc0bffc4640 1 -- 172.21.15.12:0/2055218108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0b80ca0e0 msgr2=0x7fc0b81e6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.961+0000 7fc0bffc4640 1 --2- 172.21.15.12:0/2055218108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0b80ca0e0 0x7fc0b81e6730 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400e9b0 tx=0x7fc0b400ee70 comp rx=0 tx=0).stop 2023-10-30T12:50:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.961+0000 7fc0bffc4640 1 -- 172.21.15.12:0/2055218108 shutdown_connections 2023-10-30T12:50:57.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.961+0000 7fc0bffc4640 1 --2- 172.21.15.12:0/2055218108 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc09c0664a0 0x7fc09c068960 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:57.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.961+0000 7fc0bffc4640 1 --2- 172.21.15.12:0/2055218108 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0b81538a0 0x7fc0b81ef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:57.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.961+0000 7fc0bffc4640 1 --2- 172.21.15.12:0/2055218108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0b80cf160 0x7fc0b80ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:57.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.961+0000 7fc0bffc4640 1 --2- 172.21.15.12:0/2055218108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0b80ca0e0 0x7fc0b81e6730 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:57.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.961+0000 7fc0bffc4640 1 -- 172.21.15.12:0/2055218108 >> 172.21.15.12:0/2055218108 conn(0x7fc0b806bfe0 msgr2=0x7fc0b8066420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:57.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.961+0000 7fc0bffc4640 1 -- 172.21.15.12:0/2055218108 shutdown_connections 2023-10-30T12:50:57.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:50:57.961+0000 7fc0bffc4640 1 -- 172.21.15.12:0/2055218108 wait complete. 2023-10-30T12:50:57.986 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:50:59.988 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:00.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.129+0000 7f7088dc5640 1 Processor -- start 2023-10-30T12:51:00.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.129+0000 7f7088dc5640 1 -- start start 2023-10-30T12:51:00.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.129+0000 7f7088dc5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f708406e470 0x7f708406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:00.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.129+0000 7f7088dc5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f70840c8490 0x7f70840c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:00.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.129+0000 7f7088dc5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708406afa0 con 0x7f7084153d70 2023-10-30T12:51:00.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.129+0000 7f7088dc5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708406d780 con 0x7f70840c8490 2023-10-30T12:51:00.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.129+0000 7f7088dc5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7084154150 con 0x7f708406e470 2023-10-30T12:51:00.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.133+0000 7f7081d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f708406e470 0x7f708406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:00.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.133+0000 7f7082d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f70840c8490 0x7f70840c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:00.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.133+0000 7f7081d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f708406e470 0x7f708406ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:56850/0 (socket says 172.21.15.12:56850) 2023-10-30T12:51:00.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.133+0000 7f7081d9b640 1 -- 172.21.15.12:0/2323504141 learned_addr learned my addr 172.21.15.12:0/2323504141 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:51:00.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.133+0000 7f7081d9b640 1 -- 172.21.15.12:0/2323504141 >> v2:172.21.15.154:3300/0 conn(0x7f70840c8490 msgr2=0x7f70840c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:00.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.133+0000 7f7081d9b640 1 --2- 172.21.15.12:0/2323504141 >> v2:172.21.15.154:3300/0 conn(0x7f70840c8490 0x7f70840c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:00.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.133+0000 7f7081d9b640 1 -- 172.21.15.12:0/2323504141 >> v1:172.21.15.154:6789/0 conn(0x7f7084153d70 legacy=0x7f70840cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:00.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.133+0000 7f7081d9b640 1 -- 172.21.15.12:0/2323504141 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708406e030 con 0x7f708406e470 2023-10-30T12:51:00.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.133+0000 7f7082d9d640 1 --2- 172.21.15.12:0/2323504141 >> v2:172.21.15.154:3300/0 conn(0x7f70840c8490 0x7f70840c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:51:00.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.137+0000 7f7081d9b640 1 --2- 172.21.15.12:0/2323504141 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f708406e470 0x7f708406ca80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7078009870 tx=0x7f707801c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97d62a637b9b928d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:00.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.137+0000 7f708159a640 1 -- 172.21.15.12:0/2323504141 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70780aa670 con 0x7f708406e470 2023-10-30T12:51:00.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.137+0000 7f708159a640 1 -- 172.21.15.12:0/2323504141 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70780aa7d0 con 0x7f708406e470 2023-10-30T12:51:00.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.137+0000 7f708159a640 1 -- 172.21.15.12:0/2323504141 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70780aab60 con 0x7f708406e470 2023-10-30T12:51:00.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.137+0000 7f7088dc5640 1 -- 172.21.15.12:0/2323504141 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f708406e470 msgr2=0x7f708406ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:00.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.137+0000 7f7088dc5640 1 --2- 172.21.15.12:0/2323504141 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f708406e470 0x7f708406ca80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7078009870 tx=0x7f707801c510 comp rx=0 tx=0).stop 2023-10-30T12:51:00.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.137+0000 7f7088dc5640 1 -- 172.21.15.12:0/2323504141 shutdown_connections 2023-10-30T12:51:00.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.137+0000 7f7088dc5640 1 --2- 172.21.15.12:0/2323504141 >> v2:172.21.15.154:3300/0 conn(0x7f70840c8490 0x7f70840c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:00.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.137+0000 7f7088dc5640 1 --2- 172.21.15.12:0/2323504141 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f708406e470 0x7f708406ca80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:00.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.137+0000 7f7088dc5640 1 -- 172.21.15.12:0/2323504141 >> 172.21.15.12:0/2323504141 conn(0x7f7084071ea0 msgr2=0x7f7084065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:00.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.137+0000 7f7088dc5640 1 -- 172.21.15.12:0/2323504141 shutdown_connections 2023-10-30T12:51:00.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.137+0000 7f7088dc5640 1 -- 172.21.15.12:0/2323504141 wait complete. 2023-10-30T12:51:00.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.137+0000 7f7088dc5640 1 Processor -- start 2023-10-30T12:51:00.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.137+0000 7f7088dc5640 1 -- start start 2023-10-30T12:51:00.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7088dc5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f708406e470 0x7f7084149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:00.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7088dc5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70840c8490 0x7f708414cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:00.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7088dc5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7084153d70 0x7f708414d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:00.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7088dc5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7084154330 con 0x7f7084153d70 2023-10-30T12:51:00.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7088dc5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7084155190 con 0x7f708406e470 2023-10-30T12:51:00.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7088dc5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7084154580 con 0x7f70840c8490 2023-10-30T12:51:00.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7082d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7084153d70 0x7f708414d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:00.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7082d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7084153d70 0x7f708414d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36486/0 (socket says 172.21.15.12:36486) 2023-10-30T12:51:00.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7082d9d640 1 -- 172.21.15.12:0/2844979351 learned_addr learned my addr 172.21.15.12:0/2844979351 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:51:00.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f708259c640 1 --2- 172.21.15.12:0/2844979351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f708406e470 0x7f7084149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:00.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7081d9b640 1 --2- 172.21.15.12:0/2844979351 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70840c8490 0x7f708414cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:00.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7082d9d640 1 -- 172.21.15.12:0/2844979351 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70840c8490 msgr2=0x7f708414cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:00.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7082d9d640 1 --2- 172.21.15.12:0/2844979351 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70840c8490 0x7f708414cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7082d9d640 1 -- 172.21.15.12:0/2844979351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f708406e470 msgr2=0x7f7084149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7082d9d640 1 --2- 172.21.15.12:0/2844979351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f708406e470 0x7f7084149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7082d9d640 1 -- 172.21.15.12:0/2844979351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c009a70 con 0x7f7084153d70 2023-10-30T12:51:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f708259c640 1 --2- 172.21.15.12:0/2844979351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f708406e470 0x7f7084149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:51:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7082d9d640 1 --2- 172.21.15.12:0/2844979351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7084153d70 0x7f708414d1d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f707c009f00 tx=0x7f707c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:00.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7072ffd640 1 -- 172.21.15.12:0/2844979351 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c009d70 con 0x7f7084153d70 2023-10-30T12:51:00.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7072ffd640 1 -- 172.21.15.12:0/2844979351 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c00f040 con 0x7f7084153d70 2023-10-30T12:51:00.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7072ffd640 1 -- 172.21.15.12:0/2844979351 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c003bc0 con 0x7f7084153d70 2023-10-30T12:51:00.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7088dc5640 1 -- 172.21.15.12:0/2844979351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70780094d0 con 0x7f7084153d70 2023-10-30T12:51:00.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7088dc5640 1 -- 172.21.15.12:0/2844979351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7084154b40 con 0x7f7084153d70 2023-10-30T12:51:00.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7088dc5640 1 -- 172.21.15.12:0/2844979351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708404e170 con 0x7f7084153d70 2023-10-30T12:51:00.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.141+0000 7f7072ffd640 1 -- 172.21.15.12:0/2844979351 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f707c014480 con 0x7f7084153d70 2023-10-30T12:51:00.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.145+0000 7f7072ffd640 1 --2- 172.21.15.12:0/2844979351 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f70640664a0 0x7f7064068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:00.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.145+0000 7f7072ffd640 1 -- 172.21.15.12:0/2844979351 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f707c084df0 con 0x7f7084153d70 2023-10-30T12:51:00.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.145+0000 7f708259c640 1 --2- 172.21.15.12:0/2844979351 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f70640664a0 0x7f7064068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:00.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.145+0000 7f708259c640 1 --2- 172.21.15.12:0/2844979351 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f70640664a0 0x7f7064068960 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7074000f70 tx=0x7f7074009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:00.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.149+0000 7f7072ffd640 1 -- 172.21.15.12:0/2844979351 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f707c051f30 con 0x7f7084153d70 2023-10-30T12:51:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.349+0000 7f7088dc5640 1 -- 172.21.15.12:0/2844979351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70841552d0 con 0x7f7084153d70 2023-10-30T12:51:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.349+0000 7f7072ffd640 1 -- 172.21.15.12:0/2844979351 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+5748 (secure 0 0 0) 0x7f707c055550 con 0x7f7084153d70 2023-10-30T12:51:00.356 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:51:00.356 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:50:56.971225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:resolve","state_seq":16,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 82 2023-10-30T12:51:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.349+0000 7f7088dc5640 1 -- 172.21.15.12:0/2844979351 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f70640664a0 msgr2=0x7f7064068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.349+0000 7f7088dc5640 1 --2- 172.21.15.12:0/2844979351 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f70640664a0 0x7f7064068960 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7074000f70 tx=0x7f7074009040 comp rx=0 tx=0).stop 2023-10-30T12:51:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.349+0000 7f7088dc5640 1 -- 172.21.15.12:0/2844979351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7084153d70 msgr2=0x7f708414d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.349+0000 7f7088dc5640 1 --2- 172.21.15.12:0/2844979351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7084153d70 0x7f708414d1d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f707c009f00 tx=0x7f707c00c490 comp rx=0 tx=0).stop 2023-10-30T12:51:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.349+0000 7f7088dc5640 1 -- 172.21.15.12:0/2844979351 shutdown_connections 2023-10-30T12:51:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.349+0000 7f7088dc5640 1 --2- 172.21.15.12:0/2844979351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7084153d70 0x7f708414d1d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.349+0000 7f7088dc5640 1 --2- 172.21.15.12:0/2844979351 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f70640664a0 0x7f7064068960 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.349+0000 7f7088dc5640 1 --2- 172.21.15.12:0/2844979351 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70840c8490 0x7f708414cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.349+0000 7f7088dc5640 1 --2- 172.21.15.12:0/2844979351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f708406e470 0x7f7084149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.349+0000 7f7088dc5640 1 -- 172.21.15.12:0/2844979351 >> 172.21.15.12:0/2844979351 conn(0x7f7084071ea0 msgr2=0x7f708406f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.353+0000 7f7088dc5640 1 -- 172.21.15.12:0/2844979351 shutdown_connections 2023-10-30T12:51:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:00.353+0000 7f7088dc5640 1 -- 172.21.15.12:0/2844979351 wait complete. 2023-10-30T12:51:00.385 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:51:02.386 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:02.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.541+0000 7f88f1b1d640 1 Processor -- start 2023-10-30T12:51:02.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.541+0000 7f88f1b1d640 1 -- start start 2023-10-30T12:51:02.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.541+0000 7f88f1b1d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f88ec0c7990 0x7f88ec0c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:02.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.541+0000 7f88f1b1d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ec0c8da0 con 0x7f88ec157080 2023-10-30T12:51:02.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.541+0000 7f88f1b1d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ec06cb60 con 0x7f88ec06fdb0 2023-10-30T12:51:02.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.541+0000 7f88f1b1d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88ec157460 con 0x7f88ec0c7990 2023-10-30T12:51:02.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.541+0000 7f88eb7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f88ec157080 0x7f88ec071b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58142/0 (socket says 172.21.15.12:58142) 2023-10-30T12:51:02.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.541+0000 7f88eb7fe640 1 -- 172.21.15.12:0/3669607375 learned_addr learned my addr 172.21.15.12:0/3669607375 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:51:02.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.541+0000 7f88eaffd640 1 --2- 172.21.15.12:0/3669607375 >> v2:172.21.15.12:3300/0 conn(0x7f88ec0c7990 0x7f88ec0c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:02.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.541+0000 7f88ea7fc640 1 -- 172.21.15.12:0/3669607375 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572763849 0 0) 0x7f88ec0c8da0 con 0x7f88ec157080 2023-10-30T12:51:02.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.541+0000 7f88ea7fc640 1 -- 172.21.15.12:0/3669607375 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d4003530 con 0x7f88ec157080 2023-10-30T12:51:02.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.541+0000 7f88eaffd640 1 -- 172.21.15.12:0/3669607375 >> v1:172.21.15.154:6789/0 conn(0x7f88ec06fdb0 legacy=0x7f88ec164220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:02.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.541+0000 7f88eaffd640 1 -- 172.21.15.12:0/3669607375 >> v1:172.21.15.12:6789/0 conn(0x7f88ec157080 legacy=0x7f88ec071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:02.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.541+0000 7f88eaffd640 1 -- 172.21.15.12:0/3669607375 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec06ce20 con 0x7f88ec0c7990 2023-10-30T12:51:02.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.545+0000 7f88eaffd640 1 --2- 172.21.15.12:0/3669607375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88ec0c7990 0x7f88ec0c7ef0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f88e00097c0 tx=0x7f88e001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881f23eec865c59c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:02.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.545+0000 7f88ea7fc640 1 -- 172.21.15.12:0/3669607375 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e00ab600 con 0x7f88ec0c7990 2023-10-30T12:51:02.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.545+0000 7f88ea7fc640 1 -- 172.21.15.12:0/3669607375 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e00a7050 con 0x7f88ec0c7990 2023-10-30T12:51:02.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.545+0000 7f88ea7fc640 1 -- 172.21.15.12:0/3669607375 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e00ab980 con 0x7f88ec0c7990 2023-10-30T12:51:02.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.545+0000 7f88f1b1d640 1 -- 172.21.15.12:0/3669607375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88ec0c7990 msgr2=0x7f88ec0c7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:02.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.545+0000 7f88f1b1d640 1 --2- 172.21.15.12:0/3669607375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88ec0c7990 0x7f88ec0c7ef0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f88e00097c0 tx=0x7f88e001c360 comp rx=0 tx=0).stop 2023-10-30T12:51:02.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.545+0000 7f88f1b1d640 1 -- 172.21.15.12:0/3669607375 shutdown_connections 2023-10-30T12:51:02.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.545+0000 7f88f1b1d640 1 --2- 172.21.15.12:0/3669607375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88ec0c7990 0x7f88ec0c7ef0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.545+0000 7f88f1b1d640 1 -- 172.21.15.12:0/3669607375 >> 172.21.15.12:0/3669607375 conn(0x7f88ec069e90 msgr2=0x7f88ec06a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:02.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.545+0000 7f88f1b1d640 1 -- 172.21.15.12:0/3669607375 shutdown_connections 2023-10-30T12:51:02.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.545+0000 7f88f1b1d640 1 -- 172.21.15.12:0/3669607375 wait complete. 2023-10-30T12:51:02.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88f1b1d640 1 Processor -- start 2023-10-30T12:51:02.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88f1b1d640 1 -- start start 2023-10-30T12:51:02.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88f1b1d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88ec06fdb0 0x7f88ec1561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:02.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88f1b1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88ec0c7990 0x7f88ec142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:02.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88f1b1d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88ec157080 0x7f88ec142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:02.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88f1b1d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec1f74b0 con 0x7f88ec0c7990 2023-10-30T12:51:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88f1b1d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec155d10 con 0x7f88ec06fdb0 2023-10-30T12:51:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88f1b1d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec1f6f50 con 0x7f88ec157080 2023-10-30T12:51:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88eaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88ec0c7990 0x7f88ec142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88eaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88ec0c7990 0x7f88ec142770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39686/0 (socket says 172.21.15.12:39686) 2023-10-30T12:51:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88eaffd640 1 -- 172.21.15.12:0/2078271067 learned_addr learned my addr 172.21.15.12:0/2078271067 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:51:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88eb7fe640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88ec06fdb0 0x7f88ec1561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88ebfff640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88ec157080 0x7f88ec142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88eaffd640 1 -- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88ec157080 msgr2=0x7f88ec142cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:02.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88eaffd640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88ec157080 0x7f88ec142cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88eaffd640 1 -- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88ec06fdb0 msgr2=0x7f88ec1561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:02.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88eaffd640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88ec06fdb0 0x7f88ec1561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88eaffd640 1 -- 172.21.15.12:0/2078271067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e0009420 con 0x7f88ec0c7990 2023-10-30T12:51:02.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88eb7fe640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88ec06fdb0 0x7f88ec1561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:51:02.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88ebfff640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88ec157080 0x7f88ec142cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:51:02.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88eaffd640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88ec0c7990 0x7f88ec142770 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f88e0018f70 tx=0x7f88e0019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:02.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88d3fff640 1 -- 172.21.15.12:0/2078271067 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e00b4430 con 0x7f88ec0c7990 2023-10-30T12:51:02.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88d3fff640 1 -- 172.21.15.12:0/2078271067 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e00b4590 con 0x7f88ec0c7990 2023-10-30T12:51:02.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88d3fff640 1 -- 172.21.15.12:0/2078271067 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e00b4940 con 0x7f88ec0c7990 2023-10-30T12:51:02.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88f1b1d640 1 -- 172.21.15.12:0/2078271067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ec155140 con 0x7f88ec0c7990 2023-10-30T12:51:02.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88f1b1d640 1 -- 172.21.15.12:0/2078271067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ec1553e0 con 0x7f88ec0c7990 2023-10-30T12:51:02.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.549+0000 7f88f1b1d640 1 -- 172.21.15.12:0/2078271067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ec04e170 con 0x7f88ec0c7990 2023-10-30T12:51:02.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.553+0000 7f88d3fff640 1 -- 172.21.15.12:0/2078271067 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f88e00b2040 con 0x7f88ec0c7990 2023-10-30T12:51:02.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.553+0000 7f88d3fff640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f88d80664d0 0x7f88d8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:02.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.553+0000 7f88d3fff640 1 -- 172.21.15.12:0/2078271067 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f88e011e010 con 0x7f88ec0c7990 2023-10-30T12:51:02.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.553+0000 7f88eb7fe640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f88d80664d0 0x7f88d8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:02.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.553+0000 7f88eb7fe640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f88d80664d0 0x7f88d8068990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0011f0 tx=0x7f88dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:02.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.557+0000 7f88d3fff640 1 -- 172.21.15.12:0/2078271067 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f88e00eb0c0 con 0x7f88ec0c7990 2023-10-30T12:51:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.753+0000 7f88f1b1d640 1 -- 172.21.15.12:0/2078271067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88ec0b16f0 con 0x7f88ec0c7990 2023-10-30T12:51:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.753+0000 7f88d3fff640 1 -- 172.21.15.12:0/2078271067 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+5748 (secure 0 0 0) 0x7f88e00ee6e0 con 0x7f88ec0c7990 2023-10-30T12:51:02.761 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:51:02.761 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:50:56.971225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:resolve","state_seq":16,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 82 2023-10-30T12:51:02.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.757+0000 7f88f1b1d640 1 -- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f88d80664d0 msgr2=0x7f88d8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:02.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.757+0000 7f88f1b1d640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f88d80664d0 0x7f88d8068990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0011f0 tx=0x7f88dc009040 comp rx=0 tx=0).stop 2023-10-30T12:51:02.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.757+0000 7f88f1b1d640 1 -- 172.21.15.12:0/2078271067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88ec0c7990 msgr2=0x7f88ec142770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:02.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.757+0000 7f88f1b1d640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88ec0c7990 0x7f88ec142770 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f88e0018f70 tx=0x7f88e0019100 comp rx=0 tx=0).stop 2023-10-30T12:51:02.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.757+0000 7f88f1b1d640 1 -- 172.21.15.12:0/2078271067 shutdown_connections 2023-10-30T12:51:02.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.757+0000 7f88f1b1d640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f88d80664d0 0x7f88d8068990 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.757+0000 7f88f1b1d640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88ec157080 0x7f88ec142cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.757+0000 7f88f1b1d640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88ec0c7990 0x7f88ec142770 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.757+0000 7f88f1b1d640 1 --2- 172.21.15.12:0/2078271067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88ec06fdb0 0x7f88ec1561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.757+0000 7f88f1b1d640 1 -- 172.21.15.12:0/2078271067 >> 172.21.15.12:0/2078271067 conn(0x7f88ec069e90 msgr2=0x7f88ec0c27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:02.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.757+0000 7f88f1b1d640 1 -- 172.21.15.12:0/2078271067 shutdown_connections 2023-10-30T12:51:02.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:02.757+0000 7f88f1b1d640 1 -- 172.21.15.12:0/2078271067 wait complete. 2023-10-30T12:51:02.782 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:51:04.783 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:04.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.921+0000 7f82c849e640 1 Processor -- start 2023-10-30T12:51:04.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.921+0000 7f82c849e640 1 -- start start 2023-10-30T12:51:04.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.921+0000 7f82c849e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82c00c6bd0 0x7f82c006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:04.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.921+0000 7f82c849e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c0071110 0x7f82c0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:04.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.921+0000 7f82c849e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c0066cf0 con 0x7f82c006f7a0 2023-10-30T12:51:04.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.921+0000 7f82c849e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c00700c0 con 0x7f82c00c6bd0 2023-10-30T12:51:04.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.921+0000 7f82c849e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c00c70d0 con 0x7f82c0071110 2023-10-30T12:51:04.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.921+0000 7f82c623a640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f82c006f7a0 0x7f82c0069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58150/0 (socket says 172.21.15.12:58150) 2023-10-30T12:51:04.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.921+0000 7f82c623a640 1 -- 172.21.15.12:0/1019203916 learned_addr learned my addr 172.21.15.12:0/1019203916 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:51:04.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.921+0000 7f82c6a3b640 1 --2- 172.21.15.12:0/1019203916 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c0071110 0x7f82c0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:04.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.921+0000 7f82c5a39640 1 --2- 172.21.15.12:0/1019203916 >> v2:172.21.15.154:3300/0 conn(0x7f82c00c6bd0 0x7f82c006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:04.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.921+0000 7f82c5238640 1 -- 172.21.15.12:0/1019203916 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573446126 0 0) 0x7f82c0066cf0 con 0x7f82c006f7a0 2023-10-30T12:51:04.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.921+0000 7f82c5238640 1 -- 172.21.15.12:0/1019203916 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82ac003530 con 0x7f82c006f7a0 2023-10-30T12:51:04.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c5238640 1 -- 172.21.15.12:0/1019203916 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3652092652 0 0) 0x7f82ac003530 con 0x7f82c006f7a0 2023-10-30T12:51:04.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c5238640 1 -- 172.21.15.12:0/1019203916 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c0071110 msgr2=0x7f82c0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:04.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c5238640 1 --2- 172.21.15.12:0/1019203916 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c0071110 0x7f82c0158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:04.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c5238640 1 -- 172.21.15.12:0/1019203916 >> v2:172.21.15.154:3300/0 conn(0x7f82c00c6bd0 msgr2=0x7f82c006fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:04.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c5238640 1 --2- 172.21.15.12:0/1019203916 >> v2:172.21.15.154:3300/0 conn(0x7f82c00c6bd0 0x7f82c006fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:04.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c5238640 1 -- 172.21.15.12:0/1019203916 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c00c7390 con 0x7f82c006f7a0 2023-10-30T12:51:04.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c5238640 1 -- 172.21.15.12:0/1019203916 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f82bc001680 con 0x7f82c006f7a0 2023-10-30T12:51:04.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c5238640 1 -- 172.21.15.12:0/1019203916 >> v1:172.21.15.12:6789/0 conn(0x7f82c006f7a0 legacy=0x7f82c0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:04.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c5238640 1 --2- 172.21.15.12:0/1019203916 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c0071110 0x7f82ac003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:04.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c5238640 1 --2- 172.21.15.12:0/1019203916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82ac003f00 0x7f82ac0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:04.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c5238640 1 --2- 172.21.15.12:0/1019203916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82ac004d10 0x7f82ac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:04.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c5238640 1 -- 172.21.15.12:0/1019203916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c00700c0 con 0x7f82ac004d10 2023-10-30T12:51:04.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c5238640 1 -- 172.21.15.12:0/1019203916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c00c70d0 con 0x7f82ac003f00 2023-10-30T12:51:04.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c5238640 1 -- 172.21.15.12:0/1019203916 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac00b9c0 con 0x7f82c0071110 2023-10-30T12:51:04.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c6a3b640 1 --2- 172.21.15.12:0/1019203916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82ac004d10 0x7f82ac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:04.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c5a39640 1 --2- 172.21.15.12:0/1019203916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82ac003f00 0x7f82ac0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:04.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c623a640 1 --2- 172.21.15.12:0/1019203916 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c0071110 0x7f82ac003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:04.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c6a3b640 1 -- 172.21.15.12:0/1019203916 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c0071110 msgr2=0x7f82ac003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:04.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c6a3b640 1 --2- 172.21.15.12:0/1019203916 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82c0071110 0x7f82ac003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:04.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c6a3b640 1 -- 172.21.15.12:0/1019203916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82ac003f00 msgr2=0x7f82ac0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:04.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c6a3b640 1 --2- 172.21.15.12:0/1019203916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82ac003f00 0x7f82ac0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:04.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c6a3b640 1 -- 172.21.15.12:0/1019203916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac003530 con 0x7f82ac004d10 2023-10-30T12:51:04.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.925+0000 7f82c6a3b640 1 --2- 172.21.15.12:0/1019203916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82ac004d10 0x7f82ac005190 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009fc0 tx=0x7f82b801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5dd63c45c1032a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:04.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c5238640 1 -- 172.21.15.12:0/1019203916 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80a5c70 con 0x7f82ac004d10 2023-10-30T12:51:04.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c5238640 1 -- 172.21.15.12:0/1019203916 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b800f040 con 0x7f82ac004d10 2023-10-30T12:51:04.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c5238640 1 -- 172.21.15.12:0/1019203916 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80ad440 con 0x7f82ac004d10 2023-10-30T12:51:04.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c849e640 1 -- 172.21.15.12:0/1019203916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82ac004d10 msgr2=0x7f82ac005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:04.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c849e640 1 --2- 172.21.15.12:0/1019203916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82ac004d10 0x7f82ac005190 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009fc0 tx=0x7f82b801cc60 comp rx=0 tx=0).stop 2023-10-30T12:51:04.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c6a3b640 1 -- 172.21.15.12:0/1019203916 reap_dead start 2023-10-30T12:51:04.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c849e640 1 -- 172.21.15.12:0/1019203916 shutdown_connections 2023-10-30T12:51:04.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c849e640 1 -- 172.21.15.12:0/1019203916 >> 172.21.15.12:0/1019203916 conn(0x7f82c00c7910 msgr2=0x7f82c00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:04.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c849e640 1 -- 172.21.15.12:0/1019203916 shutdown_connections 2023-10-30T12:51:04.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c849e640 1 -- 172.21.15.12:0/1019203916 wait complete. 2023-10-30T12:51:04.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c849e640 1 Processor -- start 2023-10-30T12:51:04.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c849e640 1 -- start start 2023-10-30T12:51:04.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c849e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac004d10 0x7f82c01472a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:04.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c849e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c00c4040 0x7f82c014d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:04.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c849e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c0148bb0 0x7f82c014e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:04.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c849e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0146e80 con 0x7f82c0148bb0 2023-10-30T12:51:04.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c849e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c014ead0 con 0x7f82c00c4040 2023-10-30T12:51:04.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c849e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0146570 con 0x7f82ac004d10 2023-10-30T12:51:04.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c6a3b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c0148bb0 0x7f82c014e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:04.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c6a3b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c0148bb0 0x7f82c014e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39716/0 (socket says 172.21.15.12:39716) 2023-10-30T12:51:04.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c6a3b640 1 -- 172.21.15.12:0/2142200302 learned_addr learned my addr 172.21.15.12:0/2142200302 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:51:04.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.929+0000 7f82c5a39640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c00c4040 0x7f82c014d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:04.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82c623a640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac004d10 0x7f82c01472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:04.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82c6a3b640 1 -- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac004d10 msgr2=0x7f82c01472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:04.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82c6a3b640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac004d10 0x7f82c01472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:04.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82c6a3b640 1 -- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c00c4040 msgr2=0x7f82c014d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:04.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82c6a3b640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c00c4040 0x7f82c014d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:04.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82c6a3b640 1 -- 172.21.15.12:0/2142200302 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c0149180 con 0x7f82c0148bb0 2023-10-30T12:51:04.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82c5a39640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c00c4040 0x7f82c014d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:51:04.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82c623a640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac004d10 0x7f82c01472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:51:04.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82c6a3b640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c0148bb0 0x7f82c014e590 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f82b80b0040 tx=0x7f82b8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:04.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82b6ffd640 1 -- 172.21.15.12:0/2142200302 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80bc070 con 0x7f82c0148bb0 2023-10-30T12:51:04.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82b6ffd640 1 -- 172.21.15.12:0/2142200302 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b80039a0 con 0x7f82c0148bb0 2023-10-30T12:51:04.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82c849e640 1 -- 172.21.15.12:0/2142200302 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c0149460 con 0x7f82c0148bb0 2023-10-30T12:51:04.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82b6ffd640 1 -- 172.21.15.12:0/2142200302 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8003b30 con 0x7f82c0148bb0 2023-10-30T12:51:04.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82c849e640 1 -- 172.21.15.12:0/2142200302 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c0146aa0 con 0x7f82c0148bb0 2023-10-30T12:51:04.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82c849e640 1 -- 172.21.15.12:0/2142200302 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c0146c60 con 0x7f82c0148bb0 2023-10-30T12:51:04.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82b6ffd640 1 -- 172.21.15.12:0/2142200302 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f82b80b4040 con 0x7f82c0148bb0 2023-10-30T12:51:04.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82b6ffd640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82a4066520 0x7f82a40689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:04.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.933+0000 7f82b6ffd640 1 -- 172.21.15.12:0/2142200302 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f82b811e460 con 0x7f82c0148bb0 2023-10-30T12:51:04.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.937+0000 7f82c623a640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82a4066520 0x7f82a40689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:04.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.937+0000 7f82c623a640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82a4066520 0x7f82a40689e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f82bc000c10 tx=0x7f82bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:04.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:04.937+0000 7f82b6ffd640 1 -- 172.21.15.12:0/2142200302 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f82b80eb640 con 0x7f82c0148bb0 2023-10-30T12:51:05.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:05.129+0000 7f82c849e640 1 -- 172.21.15.12:0/2142200302 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82c0146fc0 con 0x7f82c0148bb0 2023-10-30T12:51:05.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:05.129+0000 7f82b6ffd640 1 -- 172.21.15.12:0/2142200302 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5750 (secure 0 0 0) 0x7f82c0146fc0 con 0x7f82c0148bb0 2023-10-30T12:51:05.135 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:51:05.135 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:51:02.531225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:05.136 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 83 2023-10-30T12:51:05.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:05.129+0000 7f82c849e640 1 -- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82a4066520 msgr2=0x7f82a40689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:05.129+0000 7f82c849e640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82a4066520 0x7f82a40689e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f82bc000c10 tx=0x7f82bc009040 comp rx=0 tx=0).stop 2023-10-30T12:51:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:05.129+0000 7f82c849e640 1 -- 172.21.15.12:0/2142200302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c0148bb0 msgr2=0x7f82c014e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:05.129+0000 7f82c849e640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c0148bb0 0x7f82c014e590 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f82b80b0040 tx=0x7f82b8009750 comp rx=0 tx=0).stop 2023-10-30T12:51:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:05.129+0000 7f82c849e640 1 -- 172.21.15.12:0/2142200302 shutdown_connections 2023-10-30T12:51:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:05.129+0000 7f82c849e640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c0148bb0 0x7f82c014e590 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:05.129+0000 7f82c849e640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c00c4040 0x7f82c014d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:05.129+0000 7f82c849e640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82a4066520 0x7f82a40689e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:05.129+0000 7f82c849e640 1 --2- 172.21.15.12:0/2142200302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f82ac004d10 0x7f82c01472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:05.129+0000 7f82c849e640 1 -- 172.21.15.12:0/2142200302 >> 172.21.15.12:0/2142200302 conn(0x7f82c00c7910 msgr2=0x7f82c00c2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:05.129+0000 7f82c849e640 1 -- 172.21.15.12:0/2142200302 shutdown_connections 2023-10-30T12:51:05.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:05.129+0000 7f82c849e640 1 -- 172.21.15.12:0/2142200302 wait complete. 2023-10-30T12:51:05.159 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 83, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 83, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5571": { "addr": "172.21.15.12:6837/2752088514", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2752088514, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 2752088514, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5571, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 19 }, "gid_5758": { "addr": "172.21.15.154:6839/2204547612", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 2204547612, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 2204547612, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5758, "incarnation": 73, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_5877": { "addr": "172.21.15.12:6835/2306258157", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 2306258157, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 2306258157, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5877, "incarnation": 81, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:reconnect", "state_seq": 18 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:51:02.531225+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5571, "mds_1": 5758, "mds_2": 5877 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:51:06.389 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:51:06.393 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:51:07.161 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:07.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.309+0000 7f202553e640 1 Processor -- start 2023-10-30T12:51:07.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.309+0000 7f202553e640 1 -- start start 2023-10-30T12:51:07.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.309+0000 7f202553e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f20201538a0 0x7f202015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:07.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.309+0000 7f202553e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20200cb800 con 0x7f20200cf160 2023-10-30T12:51:07.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.309+0000 7f202553e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202015dc30 con 0x7f20200ca0e0 2023-10-30T12:51:07.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.309+0000 7f202553e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20200cb330 con 0x7f20201538a0 2023-10-30T12:51:07.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.309+0000 7f201effd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f20201538a0 0x7f202015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:07.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.309+0000 7f201f7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f20200cf160 0x7f2020164070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58160/0 (socket says 172.21.15.12:58160) 2023-10-30T12:51:07.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.309+0000 7f201f7fe640 1 -- 172.21.15.12:0/192760409 learned_addr learned my addr 172.21.15.12:0/192760409 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:51:07.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.309+0000 7f201effd640 1 -- 172.21.15.12:0/192760409 >> v1:172.21.15.154:6789/0 conn(0x7f20200ca0e0 legacy=0x7f202015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:07.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.309+0000 7f201effd640 1 -- 172.21.15.12:0/192760409 >> v1:172.21.15.12:6789/0 conn(0x7f20200cf160 legacy=0x7f2020164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:07.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.309+0000 7f201effd640 1 -- 172.21.15.12:0/192760409 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20200cad70 con 0x7f20201538a0 2023-10-30T12:51:07.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.313+0000 7f201effd640 1 --2- 172.21.15.12:0/192760409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20201538a0 0x7f202015d6f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f20100097c0 tx=0x7f201001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549bc116a8d5b163 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:07.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.313+0000 7f201e7fc640 1 -- 172.21.15.12:0/192760409 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20100a8970 con 0x7f20201538a0 2023-10-30T12:51:07.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.313+0000 7f201e7fc640 1 -- 172.21.15.12:0/192760409 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20100a8ad0 con 0x7f20201538a0 2023-10-30T12:51:07.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.313+0000 7f201e7fc640 1 -- 172.21.15.12:0/192760409 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20100a8e60 con 0x7f20201538a0 2023-10-30T12:51:07.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.313+0000 7f202553e640 1 -- 172.21.15.12:0/192760409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20201538a0 msgr2=0x7f202015d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:07.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.313+0000 7f202553e640 1 --2- 172.21.15.12:0/192760409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20201538a0 0x7f202015d6f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f20100097c0 tx=0x7f201001ec70 comp rx=0 tx=0).stop 2023-10-30T12:51:07.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.313+0000 7f202553e640 1 -- 172.21.15.12:0/192760409 shutdown_connections 2023-10-30T12:51:07.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.313+0000 7f202553e640 1 --2- 172.21.15.12:0/192760409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20201538a0 0x7f202015d6f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:07.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.313+0000 7f202553e640 1 -- 172.21.15.12:0/192760409 >> 172.21.15.12:0/192760409 conn(0x7f202006bfe0 msgr2=0x7f202006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:07.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.313+0000 7f202553e640 1 -- 172.21.15.12:0/192760409 shutdown_connections 2023-10-30T12:51:07.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.313+0000 7f202553e640 1 -- 172.21.15.12:0/192760409 wait complete. 2023-10-30T12:51:07.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f202553e640 1 Processor -- start 2023-10-30T12:51:07.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f202553e640 1 -- start start 2023-10-30T12:51:07.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f202553e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20200ca0e0 0x7f20201f00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:07.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f202553e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20200cf160 0x7f2020152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:07.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f202553e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20201538a0 0x7f2020140b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:07.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f202553e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2020145000 con 0x7f20201538a0 2023-10-30T12:51:07.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f202553e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20200cedd0 con 0x7f20200cf160 2023-10-30T12:51:07.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f202553e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20201f0600 con 0x7f20200ca0e0 2023-10-30T12:51:07.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f201f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20201538a0 0x7f2020140b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:07.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f201f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20201538a0 0x7f2020140b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39742/0 (socket says 172.21.15.12:39742) 2023-10-30T12:51:07.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f201f7fe640 1 -- 172.21.15.12:0/3039246539 learned_addr learned my addr 172.21.15.12:0/3039246539 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:51:07.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f201effd640 1 --2- 172.21.15.12:0/3039246539 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20200ca0e0 0x7f20201f00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:07.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f2017fff640 1 --2- 172.21.15.12:0/3039246539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20200cf160 0x7f2020152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:07.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f201f7fe640 1 -- 172.21.15.12:0/3039246539 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20200ca0e0 msgr2=0x7f20201f00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:07.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f201f7fe640 1 --2- 172.21.15.12:0/3039246539 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20200ca0e0 0x7f20201f00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:07.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f201f7fe640 1 -- 172.21.15.12:0/3039246539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20200cf160 msgr2=0x7f2020152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:07.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f201f7fe640 1 --2- 172.21.15.12:0/3039246539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20200cf160 0x7f2020152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:07.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f201f7fe640 1 -- 172.21.15.12:0/3039246539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2010009420 con 0x7f20201538a0 2023-10-30T12:51:07.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f201f7fe640 1 --2- 172.21.15.12:0/3039246539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20201538a0 0x7f2020140b40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f201800cad0 tx=0x7f201800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:07.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f20177fe640 1 -- 172.21.15.12:0/3039246539 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201800ac30 con 0x7f20201538a0 2023-10-30T12:51:07.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f20177fe640 1 -- 172.21.15.12:0/3039246539 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201800ad90 con 0x7f20201538a0 2023-10-30T12:51:07.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f20177fe640 1 -- 172.21.15.12:0/3039246539 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018012600 con 0x7f20201538a0 2023-10-30T12:51:07.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f202553e640 1 -- 172.21.15.12:0/3039246539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20200ce570 con 0x7f20201538a0 2023-10-30T12:51:07.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.317+0000 7f202553e640 1 -- 172.21.15.12:0/3039246539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20200ceaf0 con 0x7f20201538a0 2023-10-30T12:51:07.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.321+0000 7f202553e640 1 -- 172.21.15.12:0/3039246539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20201451b0 con 0x7f20201538a0 2023-10-30T12:51:07.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.321+0000 7f20177fe640 1 -- 172.21.15.12:0/3039246539 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2018007070 con 0x7f20201538a0 2023-10-30T12:51:07.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.321+0000 7f20177fe640 1 --2- 172.21.15.12:0/3039246539 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20000664a0 0x7f2000068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:07.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.321+0000 7f20177fe640 1 -- 172.21.15.12:0/3039246539 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f2018083ff0 con 0x7f20201538a0 2023-10-30T12:51:07.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.321+0000 7f201effd640 1 --2- 172.21.15.12:0/3039246539 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20000664a0 0x7f2000068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:07.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.321+0000 7f201effd640 1 --2- 172.21.15.12:0/3039246539 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20000664a0 0x7f2000068960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f20100b2000 tx=0x7f2010008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:07.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.325+0000 7f20177fe640 1 -- 172.21.15.12:0/3039246539 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f20180b5090 con 0x7f20201538a0 2023-10-30T12:51:07.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.513+0000 7f202553e640 1 -- 172.21.15.12:0/3039246539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2020159b40 con 0x7f20201538a0 2023-10-30T12:51:07.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.517+0000 7f20177fe640 1 -- 172.21.15.12:0/3039246539 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5750 (secure 0 0 0) 0x7f2018017090 con 0x7f20201538a0 2023-10-30T12:51:07.523 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:51:07.523 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:51:02.531225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:07.524 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 83 2023-10-30T12:51:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.517+0000 7f202553e640 1 -- 172.21.15.12:0/3039246539 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20000664a0 msgr2=0x7f2000068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.517+0000 7f202553e640 1 --2- 172.21.15.12:0/3039246539 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20000664a0 0x7f2000068960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f20100b2000 tx=0x7f2010008040 comp rx=0 tx=0).stop 2023-10-30T12:51:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.517+0000 7f202553e640 1 -- 172.21.15.12:0/3039246539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20201538a0 msgr2=0x7f2020140b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.517+0000 7f202553e640 1 --2- 172.21.15.12:0/3039246539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20201538a0 0x7f2020140b40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f201800cad0 tx=0x7f201800cf90 comp rx=0 tx=0).stop 2023-10-30T12:51:07.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.517+0000 7f202553e640 1 -- 172.21.15.12:0/3039246539 shutdown_connections 2023-10-30T12:51:07.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.517+0000 7f202553e640 1 --2- 172.21.15.12:0/3039246539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20201538a0 0x7f2020140b40 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:07.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.517+0000 7f202553e640 1 --2- 172.21.15.12:0/3039246539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20200cf160 0x7f2020152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:07.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.517+0000 7f202553e640 1 --2- 172.21.15.12:0/3039246539 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20000664a0 0x7f2000068960 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:07.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.517+0000 7f202553e640 1 --2- 172.21.15.12:0/3039246539 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20200ca0e0 0x7f20201f00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:07.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.517+0000 7f202553e640 1 -- 172.21.15.12:0/3039246539 >> 172.21.15.12:0/3039246539 conn(0x7f202006bfe0 msgr2=0x7f2020142e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:07.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.517+0000 7f202553e640 1 -- 172.21.15.12:0/3039246539 shutdown_connections 2023-10-30T12:51:07.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:07.517+0000 7f202553e640 1 -- 172.21.15.12:0/3039246539 wait complete. 2023-10-30T12:51:07.543 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:51:09.544 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:09.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.693+0000 7f2759c9c640 1 Processor -- start 2023-10-30T12:51:09.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.693+0000 7f2759c9c640 1 -- start start 2023-10-30T12:51:09.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.693+0000 7f2759c9c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f27540bfec0 0x7f27540c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:09.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.693+0000 7f2759c9c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f275406a720 0x7f275406abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:09.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.693+0000 7f2759c9c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2754071290 con 0x7f2754159e60 2023-10-30T12:51:09.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.693+0000 7f2759c9c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2754071570 con 0x7f27540bfec0 2023-10-30T12:51:09.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.693+0000 7f2759c9c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2754071850 con 0x7f275406a720 2023-10-30T12:51:09.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.693+0000 7f27537fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2754159e60 0x7f275406f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58174/0 (socket says 172.21.15.12:58174) 2023-10-30T12:51:09.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.693+0000 7f27537fe640 1 -- 172.21.15.12:0/401644249 learned_addr learned my addr 172.21.15.12:0/401644249 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:51:09.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.693+0000 7f2752ffd640 1 --2- 172.21.15.12:0/401644249 >> v2:172.21.15.12:3300/0 conn(0x7f27540bfec0 0x7f27540c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:09.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.693+0000 7f2753fff640 1 --2- 172.21.15.12:0/401644249 >> v2:172.21.15.154:3300/0 conn(0x7f275406a720 0x7f275406abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:09.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.697+0000 7f27527fc640 1 -- 172.21.15.12:0/401644249 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866289415 0 0) 0x7f2754071290 con 0x7f2754159e60 2023-10-30T12:51:09.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.697+0000 7f27527fc640 1 -- 172.21.15.12:0/401644249 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f273c003530 con 0x7f2754159e60 2023-10-30T12:51:09.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.697+0000 7f2752ffd640 1 -- 172.21.15.12:0/401644249 >> v2:172.21.15.154:3300/0 conn(0x7f275406a720 msgr2=0x7f275406abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:09.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.697+0000 7f2752ffd640 1 --2- 172.21.15.12:0/401644249 >> v2:172.21.15.154:3300/0 conn(0x7f275406a720 0x7f275406abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.697+0000 7f2752ffd640 1 -- 172.21.15.12:0/401644249 >> v1:172.21.15.12:6789/0 conn(0x7f2754159e60 legacy=0x7f275406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:09.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.697+0000 7f2752ffd640 1 -- 172.21.15.12:0/401644249 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2754071b30 con 0x7f27540bfec0 2023-10-30T12:51:09.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.697+0000 7f2752ffd640 1 --2- 172.21.15.12:0/401644249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27540bfec0 0x7f27540c1a40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2754071e10 tx=0x7f274801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9242ea62854a898b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:09.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.697+0000 7f27527fc640 1 -- 172.21.15.12:0/401644249 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27480a8950 con 0x7f27540bfec0 2023-10-30T12:51:09.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.697+0000 7f27527fc640 1 -- 172.21.15.12:0/401644249 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27480a8ab0 con 0x7f27540bfec0 2023-10-30T12:51:09.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.697+0000 7f27527fc640 1 -- 172.21.15.12:0/401644249 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27480a8e40 con 0x7f27540bfec0 2023-10-30T12:51:09.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.697+0000 7f2759c9c640 1 -- 172.21.15.12:0/401644249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27540bfec0 msgr2=0x7f27540c1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:09.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.697+0000 7f2759c9c640 1 --2- 172.21.15.12:0/401644249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27540bfec0 0x7f27540c1a40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2754071e10 tx=0x7f274801ec50 comp rx=0 tx=0).stop 2023-10-30T12:51:09.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2759c9c640 1 -- 172.21.15.12:0/401644249 shutdown_connections 2023-10-30T12:51:09.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2759c9c640 1 --2- 172.21.15.12:0/401644249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27540bfec0 0x7f27540c1a40 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2759c9c640 1 --2- 172.21.15.12:0/401644249 >> v2:172.21.15.154:3300/0 conn(0x7f275406a720 0x7f275406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2759c9c640 1 -- 172.21.15.12:0/401644249 >> 172.21.15.12:0/401644249 conn(0x7f27540c7910 msgr2=0x7f27540c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:09.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2759c9c640 1 -- 172.21.15.12:0/401644249 shutdown_connections 2023-10-30T12:51:09.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2759c9c640 1 -- 172.21.15.12:0/401644249 wait complete. 2023-10-30T12:51:09.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2759c9c640 1 Processor -- start 2023-10-30T12:51:09.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2759c9c640 1 -- start start 2023-10-30T12:51:09.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2759c9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f275406a720 0x7f2754158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:09.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2759c9c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27540bfec0 0x7f275414fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:09.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2759c9c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2754159e60 0x7f27541eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:09.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2759c9c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27541eea90 con 0x7f275406a720 2023-10-30T12:51:09.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2759c9c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2754145ec0 con 0x7f27540bfec0 2023-10-30T12:51:09.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2759c9c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2754159350 con 0x7f2754159e60 2023-10-30T12:51:09.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f27537fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f275406a720 0x7f2754158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:09.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f27537fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f275406a720 0x7f2754158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39756/0 (socket says 172.21.15.12:39756) 2023-10-30T12:51:09.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f27537fe640 1 -- 172.21.15.12:0/1325800429 learned_addr learned my addr 172.21.15.12:0/1325800429 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:51:09.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2752ffd640 1 --2- 172.21.15.12:0/1325800429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27540bfec0 0x7f275414fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:09.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f2753fff640 1 --2- 172.21.15.12:0/1325800429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2754159e60 0x7f27541eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:09.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f27537fe640 1 -- 172.21.15.12:0/1325800429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2754159e60 msgr2=0x7f27541eeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:09.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f27537fe640 1 --2- 172.21.15.12:0/1325800429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2754159e60 0x7f27541eeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f27537fe640 1 -- 172.21.15.12:0/1325800429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27540bfec0 msgr2=0x7f275414fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:09.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f27537fe640 1 --2- 172.21.15.12:0/1325800429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27540bfec0 0x7f275414fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f27537fe640 1 -- 172.21.15.12:0/1325800429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2748009420 con 0x7f275406a720 2023-10-30T12:51:09.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f27537fe640 1 --2- 172.21.15.12:0/1325800429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f275406a720 0x7f2754158d10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f274400c420 tx=0x7f274400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:09.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.701+0000 7f273bfff640 1 -- 172.21.15.12:0/1325800429 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744005cf0 con 0x7f275406a720 2023-10-30T12:51:09.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.705+0000 7f273bfff640 1 -- 172.21.15.12:0/1325800429 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2744005e50 con 0x7f275406a720 2023-10-30T12:51:09.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.705+0000 7f2759c9c640 1 -- 172.21.15.12:0/1325800429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2754146110 con 0x7f275406a720 2023-10-30T12:51:09.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.705+0000 7f273bfff640 1 -- 172.21.15.12:0/1325800429 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27440072d0 con 0x7f275406a720 2023-10-30T12:51:09.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.705+0000 7f2759c9c640 1 -- 172.21.15.12:0/1325800429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27541466d0 con 0x7f275406a720 2023-10-30T12:51:09.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.705+0000 7f2759c9c640 1 -- 172.21.15.12:0/1325800429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275404e170 con 0x7f275406a720 2023-10-30T12:51:09.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.705+0000 7f273bfff640 1 -- 172.21.15.12:0/1325800429 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2744007430 con 0x7f275406a720 2023-10-30T12:51:09.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.705+0000 7f273bfff640 1 --2- 172.21.15.12:0/1325800429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f27400664d0 0x7f2740068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:09.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.705+0000 7f273bfff640 1 -- 172.21.15.12:0/1325800429 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f27440836f0 con 0x7f275406a720 2023-10-30T12:51:09.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.705+0000 7f2752ffd640 1 --2- 172.21.15.12:0/1325800429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f27400664d0 0x7f2740068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:09.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.709+0000 7f2752ffd640 1 --2- 172.21.15.12:0/1325800429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f27400664d0 0x7f2740068990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f27480b2000 tx=0x7f2748008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:09.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.709+0000 7f273bfff640 1 -- 172.21.15.12:0/1325800429 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2744050850 con 0x7f275406a720 2023-10-30T12:51:09.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.901+0000 7f2759c9c640 1 -- 172.21.15.12:0/1325800429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27540c8fd0 con 0x7f275406a720 2023-10-30T12:51:09.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.901+0000 7f273bfff640 1 -- 172.21.15.12:0/1325800429 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+5747 (secure 0 0 0) 0x7f2744053e70 con 0x7f275406a720 2023-10-30T12:51:09.908 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:51:09.909 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:51:08.536108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:09.909 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 84 2023-10-30T12:51:09.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.901+0000 7f2759c9c640 1 -- 172.21.15.12:0/1325800429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f27400664d0 msgr2=0x7f2740068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:09.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.901+0000 7f2759c9c640 1 --2- 172.21.15.12:0/1325800429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f27400664d0 0x7f2740068990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f27480b2000 tx=0x7f2748008040 comp rx=0 tx=0).stop 2023-10-30T12:51:09.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.901+0000 7f2759c9c640 1 -- 172.21.15.12:0/1325800429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f275406a720 msgr2=0x7f2754158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:09.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.901+0000 7f2759c9c640 1 --2- 172.21.15.12:0/1325800429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f275406a720 0x7f2754158d10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f274400c420 tx=0x7f274400c8e0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.905+0000 7f2759c9c640 1 -- 172.21.15.12:0/1325800429 shutdown_connections 2023-10-30T12:51:09.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.905+0000 7f2759c9c640 1 --2- 172.21.15.12:0/1325800429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f27400664d0 0x7f2740068990 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.905+0000 7f2759c9c640 1 --2- 172.21.15.12:0/1325800429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2754159e60 0x7f27541eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.905+0000 7f2759c9c640 1 --2- 172.21.15.12:0/1325800429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27540bfec0 0x7f275414fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.905+0000 7f2759c9c640 1 --2- 172.21.15.12:0/1325800429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f275406a720 0x7f2754158d10 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.905+0000 7f2759c9c640 1 -- 172.21.15.12:0/1325800429 >> 172.21.15.12:0/1325800429 conn(0x7f27540c7910 msgr2=0x7f27540ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:09.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.905+0000 7f2759c9c640 1 -- 172.21.15.12:0/1325800429 shutdown_connections 2023-10-30T12:51:09.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:09.905+0000 7f2759c9c640 1 -- 172.21.15.12:0/1325800429 wait complete. 2023-10-30T12:51:09.929 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:51:11.931 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.077+0000 7f831fe46640 1 Processor -- start 2023-10-30T12:51:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.077+0000 7f831fe46640 1 -- start start 2023-10-30T12:51:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.081+0000 7f831fe46640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8318150920 0x7f831806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.081+0000 7f831fe46640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f83180c2570 0x7f83180cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.081+0000 7f831fe46640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83180c68b0 con 0x7f83180c29d0 2023-10-30T12:51:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.081+0000 7f831fe46640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8318150d00 con 0x7f83180c2570 2023-10-30T12:51:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.081+0000 7f831fe46640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831806d960 con 0x7f8318150920 2023-10-30T12:51:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.081+0000 7f831d3e1640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f83180c2570 0x7f83180cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.081+0000 7f831d3e1640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f83180c2570 0x7f83180cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35152/0 (socket says 172.21.15.12:35152) 2023-10-30T12:51:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.081+0000 7f831d3e1640 1 -- 172.21.15.12:0/2180701779 learned_addr learned my addr 172.21.15.12:0/2180701779 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:51:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.081+0000 7f831dbe2640 1 --2- 172.21.15.12:0/2180701779 >> v2:172.21.15.154:3300/0 conn(0x7f8318150920 0x7f831806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:12.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.081+0000 7f831d3e1640 1 -- 172.21.15.12:0/2180701779 >> v2:172.21.15.154:3300/0 conn(0x7f8318150920 msgr2=0x7f831806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:12.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.081+0000 7f831d3e1640 1 --2- 172.21.15.12:0/2180701779 >> v2:172.21.15.154:3300/0 conn(0x7f8318150920 0x7f831806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:12.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.081+0000 7f831d3e1640 1 -- 172.21.15.12:0/2180701779 >> v1:172.21.15.154:6789/0 conn(0x7f83180c29d0 legacy=0x7f8318156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:12.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.081+0000 7f831d3e1640 1 -- 172.21.15.12:0/2180701779 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83180c2e90 con 0x7f83180c2570 2023-10-30T12:51:12.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.085+0000 7f831d3e1640 1 --2- 172.21.15.12:0/2180701779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83180c2570 0x7f83180cb0d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f831806feb0 tx=0x7f830801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6013c9d6e602c64c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:12.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.085+0000 7f831cbe0640 1 -- 172.21.15.12:0/2180701779 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83080ab5f0 con 0x7f83180c2570 2023-10-30T12:51:12.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.085+0000 7f831cbe0640 1 -- 172.21.15.12:0/2180701779 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83080ab750 con 0x7f83180c2570 2023-10-30T12:51:12.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.085+0000 7f831cbe0640 1 -- 172.21.15.12:0/2180701779 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83080abae0 con 0x7f83180c2570 2023-10-30T12:51:12.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.085+0000 7f831fe46640 1 -- 172.21.15.12:0/2180701779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83180c2570 msgr2=0x7f83180cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:12.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.085+0000 7f831fe46640 1 --2- 172.21.15.12:0/2180701779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83180c2570 0x7f83180cb0d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f831806feb0 tx=0x7f830801c500 comp rx=0 tx=0).stop 2023-10-30T12:51:12.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.085+0000 7f831fe46640 1 -- 172.21.15.12:0/2180701779 shutdown_connections 2023-10-30T12:51:12.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.085+0000 7f831fe46640 1 --2- 172.21.15.12:0/2180701779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83180c2570 0x7f83180cb0d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:12.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.085+0000 7f831fe46640 1 --2- 172.21.15.12:0/2180701779 >> v2:172.21.15.154:3300/0 conn(0x7f8318150920 0x7f831806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:12.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.085+0000 7f831fe46640 1 -- 172.21.15.12:0/2180701779 >> 172.21.15.12:0/2180701779 conn(0x7f831806dfd0 msgr2=0x7f831806e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:12.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.085+0000 7f831fe46640 1 -- 172.21.15.12:0/2180701779 shutdown_connections 2023-10-30T12:51:12.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.085+0000 7f831fe46640 1 -- 172.21.15.12:0/2180701779 wait complete. 2023-10-30T12:51:12.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.085+0000 7f831fe46640 1 Processor -- start 2023-10-30T12:51:12.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.085+0000 7f831fe46640 1 -- start start 2023-10-30T12:51:12.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831fe46640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83180c2570 0x7f831814e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:12.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831fe46640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83180c29d0 0x7f831814ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:12.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831fe46640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8318150920 0x7f8318158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:12.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831fe46640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831814e280 con 0x7f83180c2570 2023-10-30T12:51:12.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831fe46640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8318154d30 con 0x7f83180c29d0 2023-10-30T12:51:12.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831fe46640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8318155890 con 0x7f8318150920 2023-10-30T12:51:12.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831dbe2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83180c2570 0x7f831814e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:12.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831dbe2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83180c2570 0x7f831814e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35160/0 (socket says 172.21.15.12:35160) 2023-10-30T12:51:12.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831dbe2640 1 -- 172.21.15.12:0/1619423078 learned_addr learned my addr 172.21.15.12:0/1619423078 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:51:12.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831d3e1640 1 --2- 172.21.15.12:0/1619423078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83180c29d0 0x7f831814ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:12.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831e3e3640 1 --2- 172.21.15.12:0/1619423078 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8318150920 0x7f8318158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:12.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831dbe2640 1 -- 172.21.15.12:0/1619423078 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8318150920 msgr2=0x7f8318158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:12.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831dbe2640 1 --2- 172.21.15.12:0/1619423078 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8318150920 0x7f8318158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:12.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831dbe2640 1 -- 172.21.15.12:0/1619423078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83180c29d0 msgr2=0x7f831814ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:12.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831dbe2640 1 --2- 172.21.15.12:0/1619423078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83180c29d0 0x7f831814ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:12.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831dbe2640 1 -- 172.21.15.12:0/1619423078 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83080094d0 con 0x7f83180c2570 2023-10-30T12:51:12.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831dbe2640 1 --2- 172.21.15.12:0/1619423078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83180c2570 0x7f831814e710 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f831400e870 tx=0x7f831400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:12.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f830e7fc640 1 -- 172.21.15.12:0/1619423078 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8314009510 con 0x7f83180c2570 2023-10-30T12:51:12.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f830e7fc640 1 -- 172.21.15.12:0/1619423078 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831400eec0 con 0x7f83180c2570 2023-10-30T12:51:12.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f830e7fc640 1 -- 172.21.15.12:0/1619423078 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831400cb40 con 0x7f83180c2570 2023-10-30T12:51:12.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831fe46640 1 -- 172.21.15.12:0/1619423078 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8318154ff0 con 0x7f83180c2570 2023-10-30T12:51:12.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831fe46640 1 -- 172.21.15.12:0/1619423078 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83181555b0 con 0x7f83180c2570 2023-10-30T12:51:12.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f831fe46640 1 -- 172.21.15.12:0/1619423078 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f831804e170 con 0x7f83180c2570 2023-10-30T12:51:12.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.089+0000 7f830e7fc640 1 -- 172.21.15.12:0/1619423078 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f831400cca0 con 0x7f83180c2570 2023-10-30T12:51:12.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.093+0000 7f830e7fc640 1 --2- 172.21.15.12:0/1619423078 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82fc0664a0 0x7f82fc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:12.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.093+0000 7f830e7fc640 1 -- 172.21.15.12:0/1619423078 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f83140841a0 con 0x7f83180c2570 2023-10-30T12:51:12.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.093+0000 7f831d3e1640 1 --2- 172.21.15.12:0/1619423078 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82fc0664a0 0x7f82fc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:12.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.093+0000 7f831d3e1640 1 --2- 172.21.15.12:0/1619423078 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82fc0664a0 0x7f82fc068960 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f83080af000 tx=0x7f83080b6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:12.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.097+0000 7f830e7fc640 1 -- 172.21.15.12:0/1619423078 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8314051430 con 0x7f83180c2570 2023-10-30T12:51:12.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.285+0000 7f831fe46640 1 -- 172.21.15.12:0/1619423078 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f831814dce0 con 0x7f83180c2570 2023-10-30T12:51:12.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.289+0000 7f830e7fc640 1 -- 172.21.15.12:0/1619423078 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+5747 (secure 0 0 0) 0x7f8314054a50 con 0x7f83180c2570 2023-10-30T12:51:12.294 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:51:12.295 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:51:08.536108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:12.295 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 84 2023-10-30T12:51:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.289+0000 7f831fe46640 1 -- 172.21.15.12:0/1619423078 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82fc0664a0 msgr2=0x7f82fc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.289+0000 7f831fe46640 1 --2- 172.21.15.12:0/1619423078 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82fc0664a0 0x7f82fc068960 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f83080af000 tx=0x7f83080b6040 comp rx=0 tx=0).stop 2023-10-30T12:51:12.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.289+0000 7f831fe46640 1 -- 172.21.15.12:0/1619423078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83180c2570 msgr2=0x7f831814e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:12.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.289+0000 7f831fe46640 1 --2- 172.21.15.12:0/1619423078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83180c2570 0x7f831814e710 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f831400e870 tx=0x7f831400ec70 comp rx=0 tx=0).stop 2023-10-30T12:51:12.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.289+0000 7f831fe46640 1 -- 172.21.15.12:0/1619423078 shutdown_connections 2023-10-30T12:51:12.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.289+0000 7f831fe46640 1 --2- 172.21.15.12:0/1619423078 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f82fc0664a0 0x7f82fc068960 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:12.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.289+0000 7f831fe46640 1 --2- 172.21.15.12:0/1619423078 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8318150920 0x7f8318158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:12.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.289+0000 7f831fe46640 1 --2- 172.21.15.12:0/1619423078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83180c29d0 0x7f831814ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:12.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.289+0000 7f831fe46640 1 --2- 172.21.15.12:0/1619423078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83180c2570 0x7f831814e710 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:12.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.289+0000 7f831fe46640 1 -- 172.21.15.12:0/1619423078 >> 172.21.15.12:0/1619423078 conn(0x7f831806dfd0 msgr2=0x7f83180714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:12.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.289+0000 7f831fe46640 1 -- 172.21.15.12:0/1619423078 shutdown_connections 2023-10-30T12:51:12.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:12.289+0000 7f831fe46640 1 -- 172.21.15.12:0/1619423078 wait complete. 2023-10-30T12:51:12.315 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:51:14.316 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:14.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.452+0000 7fd8a550f640 1 Processor -- start 2023-10-30T12:51:14.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.452+0000 7fd8a550f640 1 -- start start 2023-10-30T12:51:14.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd8a550f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8a00c6bd0 0x7fd8a006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:14.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd8a550f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a0066cf0 con 0x7fd8a006f7a0 2023-10-30T12:51:14.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd8a550f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a00c70d0 con 0x7fd8a0071110 2023-10-30T12:51:14.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd8a550f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a00700c0 con 0x7fd8a00c6bd0 2023-10-30T12:51:14.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89effd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd8a006f7a0 0x7fd8a0069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59512/0 (socket says 172.21.15.12:59512) 2023-10-30T12:51:14.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89effd640 1 -- 172.21.15.12:0/796890662 learned_addr learned my addr 172.21.15.12:0/796890662 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:51:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd897fff640 1 --2- 172.21.15.12:0/796890662 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8a00c6bd0 0x7fd8a006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89e7fc640 1 -- 172.21.15.12:0/796890662 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136143450 0 0) 0x7fd8a0066cf0 con 0x7fd8a006f7a0 2023-10-30T12:51:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89e7fc640 1 -- 172.21.15.12:0/796890662 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd88c003530 con 0x7fd8a006f7a0 2023-10-30T12:51:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89e7fc640 1 -- 172.21.15.12:0/796890662 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2847741237 0 0) 0x7fd88c003530 con 0x7fd8a006f7a0 2023-10-30T12:51:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89e7fc640 1 -- 172.21.15.12:0/796890662 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8a00c6bd0 msgr2=0x7fd8a006fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89e7fc640 1 --2- 172.21.15.12:0/796890662 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8a00c6bd0 0x7fd8a006fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89e7fc640 1 -- 172.21.15.12:0/796890662 >> v1:172.21.15.154:6789/0 conn(0x7fd8a0071110 legacy=0x7fd8a0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89e7fc640 1 -- 172.21.15.12:0/796890662 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a00c7390 con 0x7fd8a006f7a0 2023-10-30T12:51:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89e7fc640 1 -- 172.21.15.12:0/796890662 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fd890001680 con 0x7fd8a006f7a0 2023-10-30T12:51:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89e7fc640 1 -- 172.21.15.12:0/796890662 >> v1:172.21.15.12:6789/0 conn(0x7fd8a006f7a0 legacy=0x7fd8a0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89e7fc640 1 --2- 172.21.15.12:0/796890662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd88c0039b0 0x7fd88c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89e7fc640 1 --2- 172.21.15.12:0/796890662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd88c0047d0 0x7fd88c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89e7fc640 1 --2- 172.21.15.12:0/796890662 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8a00c6bd0 0x7fd88c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89e7fc640 1 -- 172.21.15.12:0/796890662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a00700c0 con 0x7fd88c0047d0 2023-10-30T12:51:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89e7fc640 1 -- 172.21.15.12:0/796890662 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c00bad0 con 0x7fd88c0039b0 2023-10-30T12:51:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89e7fc640 1 -- 172.21.15.12:0/796890662 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c00bc90 con 0x7fd8a00c6bd0 2023-10-30T12:51:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd897fff640 1 --2- 172.21.15.12:0/796890662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd88c0047d0 0x7fd88c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89effd640 1 --2- 172.21.15.12:0/796890662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd88c0039b0 0x7fd88c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd89f7fe640 1 --2- 172.21.15.12:0/796890662 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8a00c6bd0 0x7fd88c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd897fff640 1 -- 172.21.15.12:0/796890662 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8a00c6bd0 msgr2=0x7fd88c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd897fff640 1 --2- 172.21.15.12:0/796890662 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd8a00c6bd0 0x7fd88c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd897fff640 1 -- 172.21.15.12:0/796890662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd88c0039b0 msgr2=0x7fd88c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd897fff640 1 --2- 172.21.15.12:0/796890662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd88c0039b0 0x7fd88c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.456+0000 7fd897fff640 1 -- 172.21.15.12:0/796890662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c003530 con 0x7fd88c0047d0 2023-10-30T12:51:14.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.460+0000 7fd897fff640 1 --2- 172.21.15.12:0/796890662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd88c0047d0 0x7fd88c004c50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd888009ab0 tx=0x7fd88801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4641f4b80f80843 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:14.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.460+0000 7fd89e7fc640 1 -- 172.21.15.12:0/796890662 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8880a5600 con 0x7fd88c0047d0 2023-10-30T12:51:14.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.460+0000 7fd89e7fc640 1 -- 172.21.15.12:0/796890662 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8880a5760 con 0x7fd88c0047d0 2023-10-30T12:51:14.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.460+0000 7fd89e7fc640 1 -- 172.21.15.12:0/796890662 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8880a5b10 con 0x7fd88c0047d0 2023-10-30T12:51:14.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.460+0000 7fd8a550f640 1 -- 172.21.15.12:0/796890662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd88c0047d0 msgr2=0x7fd88c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.460+0000 7fd8a550f640 1 --2- 172.21.15.12:0/796890662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd88c0047d0 0x7fd88c004c50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd888009ab0 tx=0x7fd88801ca10 comp rx=0 tx=0).stop 2023-10-30T12:51:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.460+0000 7fd89f7fe640 1 -- 172.21.15.12:0/796890662 reap_dead start 2023-10-30T12:51:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.460+0000 7fd8a550f640 1 -- 172.21.15.12:0/796890662 shutdown_connections 2023-10-30T12:51:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.460+0000 7fd8a550f640 1 -- 172.21.15.12:0/796890662 >> 172.21.15.12:0/796890662 conn(0x7fd8a00c7910 msgr2=0x7fd8a00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.460+0000 7fd8a550f640 1 -- 172.21.15.12:0/796890662 shutdown_connections 2023-10-30T12:51:14.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.460+0000 7fd8a550f640 1 -- 172.21.15.12:0/796890662 wait complete. 2023-10-30T12:51:14.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.460+0000 7fd8a550f640 1 Processor -- start 2023-10-30T12:51:14.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd8a550f640 1 -- start start 2023-10-30T12:51:14.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd8a550f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd88c0047d0 0x7fd8a0156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:14.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd8a550f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8a0158950 0x7fd8a0155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:14.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd8a550f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a01ef2e0 0x7fd8a01f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:14.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd8a550f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a01565d0 con 0x7fd8a0158950 2023-10-30T12:51:14.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd8a550f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a0155a90 con 0x7fd8a01ef2e0 2023-10-30T12:51:14.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd8a550f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a01f2ac0 con 0x7fd88c0047d0 2023-10-30T12:51:14.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd897fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8a0158950 0x7fd8a0155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:14.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd897fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8a0158950 0x7fd8a0155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35170/0 (socket says 172.21.15.12:35170) 2023-10-30T12:51:14.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd897fff640 1 -- 172.21.15.12:0/1795830656 learned_addr learned my addr 172.21.15.12:0/1795830656 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:51:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd89effd640 1 --2- 172.21.15.12:0/1795830656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd88c0047d0 0x7fd8a0156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd89f7fe640 1 --2- 172.21.15.12:0/1795830656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a01ef2e0 0x7fd8a01f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd897fff640 1 -- 172.21.15.12:0/1795830656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd88c0047d0 msgr2=0x7fd8a0156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd897fff640 1 --2- 172.21.15.12:0/1795830656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd88c0047d0 0x7fd8a0156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd897fff640 1 -- 172.21.15.12:0/1795830656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a01ef2e0 msgr2=0x7fd8a01f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd897fff640 1 --2- 172.21.15.12:0/1795830656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a01ef2e0 0x7fd8a01f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd897fff640 1 -- 172.21.15.12:0/1795830656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a0157360 con 0x7fd8a0158950 2023-10-30T12:51:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd897fff640 1 --2- 172.21.15.12:0/1795830656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8a0158950 0x7fd8a0155550 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd88801c840 tx=0x7fd88c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd8977fe640 1 -- 172.21.15.12:0/1795830656 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888004230 con 0x7fd8a0158950 2023-10-30T12:51:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd8977fe640 1 -- 172.21.15.12:0/1795830656 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd888004390 con 0x7fd8a0158950 2023-10-30T12:51:14.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd8a550f640 1 -- 172.21.15.12:0/1795830656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a0156da0 con 0x7fd8a0158950 2023-10-30T12:51:14.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd8977fe640 1 -- 172.21.15.12:0/1795830656 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8880ad850 con 0x7fd8a0158950 2023-10-30T12:51:14.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd8a550f640 1 -- 172.21.15.12:0/1795830656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a01ee800 con 0x7fd8a0158950 2023-10-30T12:51:14.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.464+0000 7fd8a550f640 1 -- 172.21.15.12:0/1795830656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a01ee660 con 0x7fd8a0158950 2023-10-30T12:51:14.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.468+0000 7fd8977fe640 1 -- 172.21.15.12:0/1795830656 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd8880b4040 con 0x7fd8a0158950 2023-10-30T12:51:14.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.468+0000 7fd8977fe640 1 --2- 172.21.15.12:0/1795830656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd8800664a0 0x7fd880068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:14.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.468+0000 7fd8977fe640 1 -- 172.21.15.12:0/1795830656 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7fd88811e0f0 con 0x7fd8a0158950 2023-10-30T12:51:14.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.468+0000 7fd89effd640 1 --2- 172.21.15.12:0/1795830656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd8800664a0 0x7fd880068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:14.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.468+0000 7fd89effd640 1 --2- 172.21.15.12:0/1795830656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd8800664a0 0x7fd880068960 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd8900011f0 tx=0x7fd890001180 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:14.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.472+0000 7fd8977fe640 1 -- 172.21.15.12:0/1795830656 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd8a01ee660 con 0x7fd8a0158950 2023-10-30T12:51:14.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.668+0000 7fd8a550f640 1 -- 172.21.15.12:0/1795830656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8a01ee660 con 0x7fd8a0158950 2023-10-30T12:51:14.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.668+0000 7fd8977fe640 1 -- 172.21.15.12:0/1795830656 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+5747 (secure 0 0 0) 0x7fd8a01ee660 con 0x7fd8a0158950 2023-10-30T12:51:14.678 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:51:14.678 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:51:13.607295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:14.678 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 85 2023-10-30T12:51:14.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.672+0000 7fd8a550f640 1 -- 172.21.15.12:0/1795830656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd8800664a0 msgr2=0x7fd880068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:14.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.672+0000 7fd8a550f640 1 --2- 172.21.15.12:0/1795830656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd8800664a0 0x7fd880068960 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd8900011f0 tx=0x7fd890001180 comp rx=0 tx=0).stop 2023-10-30T12:51:14.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.672+0000 7fd8a550f640 1 -- 172.21.15.12:0/1795830656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8a0158950 msgr2=0x7fd8a0155550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:14.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.672+0000 7fd8a550f640 1 --2- 172.21.15.12:0/1795830656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8a0158950 0x7fd8a0155550 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd88801c840 tx=0x7fd88c005800 comp rx=0 tx=0).stop 2023-10-30T12:51:14.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.672+0000 7fd8a550f640 1 -- 172.21.15.12:0/1795830656 shutdown_connections 2023-10-30T12:51:14.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.672+0000 7fd8a550f640 1 --2- 172.21.15.12:0/1795830656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a01ef2e0 0x7fd8a01f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:14.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.672+0000 7fd8a550f640 1 --2- 172.21.15.12:0/1795830656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8a0158950 0x7fd8a0155550 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:14.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.672+0000 7fd8a550f640 1 --2- 172.21.15.12:0/1795830656 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd8800664a0 0x7fd880068960 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:14.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.672+0000 7fd8a550f640 1 --2- 172.21.15.12:0/1795830656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd88c0047d0 0x7fd8a0156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:14.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.672+0000 7fd8a550f640 1 -- 172.21.15.12:0/1795830656 >> 172.21.15.12:0/1795830656 conn(0x7fd8a00c7910 msgr2=0x7fd8a00c0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:14.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.672+0000 7fd8a550f640 1 -- 172.21.15.12:0/1795830656 shutdown_connections 2023-10-30T12:51:14.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:14.672+0000 7fd8a550f640 1 -- 172.21.15.12:0/1795830656 wait complete. 2023-10-30T12:51:14.699 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=2, replacing gid=5427 2023-10-30T12:51:14.699 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 6 secs before reviving mds.e 2023-10-30T12:51:20.700 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2023-10-30T12:51:20.700 INFO:tasks.ceph.mds.e:Restarting daemon 2023-10-30T12:51:20.701 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-10-30T12:51:20.704 INFO:tasks.ceph.mds.e:Started 2023-10-30T12:51:20.705 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:20.797 INFO:tasks.ceph.mds.e.smithi012.stdout:starting mds.e at 2023-10-30T12:51:20.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3713dab640 1 Processor -- start 2023-10-30T12:51:20.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3713dab640 1 -- start start 2023-10-30T12:51:20.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3713dab640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c1538a0 0x7f370c15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:20.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3713dab640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f370c0ca0e0 0x7f370c15a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:20.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3713dab640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c0cb800 con 0x7f370c0cf160 2023-10-30T12:51:20.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3713dab640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c15dc30 con 0x7f370c0ca0e0 2023-10-30T12:51:20.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3713dab640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0cb330 con 0x7f370c1538a0 2023-10-30T12:51:20.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3711b47640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c1538a0 0x7f370c15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:20.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3711346640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f370c0ca0e0 0x7f370c15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:20.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3711b47640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c1538a0 0x7f370c15d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:33446/0 (socket says 172.21.15.12:33446) 2023-10-30T12:51:20.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3711b47640 1 -- 172.21.15.12:0/3636022572 learned_addr learned my addr 172.21.15.12:0/3636022572 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:51:20.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3711346640 1 -- 172.21.15.12:0/3636022572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c1538a0 msgr2=0x7f370c15d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:20.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3711346640 1 --2- 172.21.15.12:0/3636022572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c1538a0 0x7f370c15d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:20.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3711346640 1 -- 172.21.15.12:0/3636022572 >> v1:172.21.15.154:6789/0 conn(0x7f370c0cf160 legacy=0x7f370c0cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:20.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3711346640 1 -- 172.21.15.12:0/3636022572 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c0cad70 con 0x7f370c0ca0e0 2023-10-30T12:51:20.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.848+0000 7f3711b47640 1 --2- 172.21.15.12:0/3636022572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c1538a0 0x7f370c15d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:51:20.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.852+0000 7f3711346640 1 --2- 172.21.15.12:0/3636022572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f370c0ca0e0 0x7f370c15a090 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0097c0 tx=0x7f36fc01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92dbdd7175e1797e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:20.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.852+0000 7f3710b45640 1 -- 172.21.15.12:0/3636022572 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0a8970 con 0x7f370c0ca0e0 2023-10-30T12:51:20.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.852+0000 7f3710b45640 1 -- 172.21.15.12:0/3636022572 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc0a8ad0 con 0x7f370c0ca0e0 2023-10-30T12:51:20.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.852+0000 7f3710b45640 1 -- 172.21.15.12:0/3636022572 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0a8e60 con 0x7f370c0ca0e0 2023-10-30T12:51:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.852+0000 7f3713dab640 1 -- 172.21.15.12:0/3636022572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f370c0ca0e0 msgr2=0x7f370c15a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.852+0000 7f3713dab640 1 --2- 172.21.15.12:0/3636022572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f370c0ca0e0 0x7f370c15a090 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0097c0 tx=0x7f36fc01ec70 comp rx=0 tx=0).stop 2023-10-30T12:51:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.852+0000 7f3713dab640 1 -- 172.21.15.12:0/3636022572 shutdown_connections 2023-10-30T12:51:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.852+0000 7f3713dab640 1 --2- 172.21.15.12:0/3636022572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f370c0ca0e0 0x7f370c15a090 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:20.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.852+0000 7f3713dab640 1 --2- 172.21.15.12:0/3636022572 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c1538a0 0x7f370c15d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:20.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.852+0000 7f3713dab640 1 -- 172.21.15.12:0/3636022572 >> 172.21.15.12:0/3636022572 conn(0x7f370c06bfe0 msgr2=0x7f370c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:20.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.852+0000 7f3713dab640 1 -- 172.21.15.12:0/3636022572 shutdown_connections 2023-10-30T12:51:20.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.852+0000 7f3713dab640 1 -- 172.21.15.12:0/3636022572 wait complete. 2023-10-30T12:51:20.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3713dab640 1 Processor -- start 2023-10-30T12:51:20.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3713dab640 1 -- start start 2023-10-30T12:51:20.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3713dab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f370c0ca0e0 0x7f370c1e2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:20.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3713dab640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f370c0cf160 0x7f370c065d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3713dab640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c1538a0 0x7f370c1e33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3713dab640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c1e4cf0 con 0x7f370c0cf160 2023-10-30T12:51:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3713dab640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c1e30d0 con 0x7f370c0ca0e0 2023-10-30T12:51:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3713dab640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c147790 con 0x7f370c1538a0 2023-10-30T12:51:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3711346640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f370c0cf160 0x7f370c065d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3711b47640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f370c0ca0e0 0x7f370c1e2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3711346640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f370c0cf160 0x7f370c065d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44700/0 (socket says 172.21.15.12:44700) 2023-10-30T12:51:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3711346640 1 -- 172.21.15.12:0/3275036658 learned_addr learned my addr 172.21.15.12:0/3275036658 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:51:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3712348640 1 --2- 172.21.15.12:0/3275036658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c1538a0 0x7f370c1e33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3711346640 1 -- 172.21.15.12:0/3275036658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c1538a0 msgr2=0x7f370c1e33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3711346640 1 --2- 172.21.15.12:0/3275036658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c1538a0 0x7f370c1e33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3711346640 1 -- 172.21.15.12:0/3275036658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f370c0ca0e0 msgr2=0x7f370c1e2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3711346640 1 --2- 172.21.15.12:0/3275036658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f370c0ca0e0 0x7f370c1e2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3711346640 1 -- 172.21.15.12:0/3275036658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37080094d0 con 0x7f370c0cf160 2023-10-30T12:51:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.856+0000 7f3711346640 1 --2- 172.21.15.12:0/3275036658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f370c0cf160 0x7f370c065d50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f36fc01d320 tx=0x7f36fc01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.860+0000 7f37027fc640 1 -- 172.21.15.12:0/3275036658 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0bf070 con 0x7f370c0cf160 2023-10-30T12:51:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.860+0000 7f37027fc640 1 -- 172.21.15.12:0/3275036658 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc006e20 con 0x7f370c0cf160 2023-10-30T12:51:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.860+0000 7f37027fc640 1 -- 172.21.15.12:0/3275036658 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0b1e20 con 0x7f370c0cf160 2023-10-30T12:51:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.860+0000 7f3713dab640 1 -- 172.21.15.12:0/3275036658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36fc009420 con 0x7f370c0cf160 2023-10-30T12:51:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.860+0000 7f3713dab640 1 -- 172.21.15.12:0/3275036658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c146b80 con 0x7f370c0cf160 2023-10-30T12:51:20.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.860+0000 7f3713dab640 1 -- 172.21.15.12:0/3275036658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c0b2a50 con 0x7f370c0cf160 2023-10-30T12:51:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.860+0000 7f37027fc640 1 -- 172.21.15.12:0/3275036658 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f36fc0b7040 con 0x7f370c0cf160 2023-10-30T12:51:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.860+0000 7f37027fc640 1 --2- 172.21.15.12:0/3275036658 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f36f00664a0 0x7f36f0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.860+0000 7f37027fc640 1 -- 172.21.15.12:0/3275036658 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f36fc121380 con 0x7f370c0cf160 2023-10-30T12:51:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.864+0000 7f3711b47640 1 --2- 172.21.15.12:0/3275036658 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f36f00664a0 0x7f36f0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.864+0000 7f3711b47640 1 --2- 172.21.15.12:0/3275036658 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f36f00664a0 0x7f36f0068960 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3708006cb0 tx=0x7f3708009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:20.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:20.868+0000 7f37027fc640 1 -- 172.21.15.12:0/3275036658 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f36fc0ee4f0 con 0x7f370c0cf160 2023-10-30T12:51:21.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:21.060+0000 7f3713dab640 1 -- 172.21.15.12:0/3275036658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f370c04e170 con 0x7f370c0cf160 2023-10-30T12:51:21.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:21.060+0000 7f37027fc640 1 -- 172.21.15.12:0/3275036658 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+5747 (secure 0 0 0) 0x7f36fc0f1b10 con 0x7f370c0cf160 2023-10-30T12:51:21.067 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:51:21.068 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:51:13.607295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:21.068 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 85 2023-10-30T12:51:21.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:21.060+0000 7f3713dab640 1 -- 172.21.15.12:0/3275036658 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f36f00664a0 msgr2=0x7f36f0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:21.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:21.060+0000 7f3713dab640 1 --2- 172.21.15.12:0/3275036658 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f36f00664a0 0x7f36f0068960 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3708006cb0 tx=0x7f3708009240 comp rx=0 tx=0).stop 2023-10-30T12:51:21.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:21.060+0000 7f3713dab640 1 -- 172.21.15.12:0/3275036658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f370c0cf160 msgr2=0x7f370c065d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:21.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:21.060+0000 7f3713dab640 1 --2- 172.21.15.12:0/3275036658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f370c0cf160 0x7f370c065d50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f36fc01d320 tx=0x7f36fc01ba10 comp rx=0 tx=0).stop 2023-10-30T12:51:21.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:21.060+0000 7f3713dab640 1 -- 172.21.15.12:0/3275036658 shutdown_connections 2023-10-30T12:51:21.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:21.060+0000 7f3713dab640 1 --2- 172.21.15.12:0/3275036658 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f36f00664a0 0x7f36f0068960 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:21.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:21.060+0000 7f3713dab640 1 --2- 172.21.15.12:0/3275036658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c1538a0 0x7f370c1e33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:21.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:21.060+0000 7f3713dab640 1 --2- 172.21.15.12:0/3275036658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f370c0cf160 0x7f370c065d50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:21.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:21.060+0000 7f3713dab640 1 --2- 172.21.15.12:0/3275036658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f370c0ca0e0 0x7f370c1e2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:21.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:21.060+0000 7f3713dab640 1 -- 172.21.15.12:0/3275036658 >> 172.21.15.12:0/3275036658 conn(0x7f370c06bfe0 msgr2=0x7f370c0cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:21.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:21.060+0000 7f3713dab640 1 -- 172.21.15.12:0/3275036658 shutdown_connections 2023-10-30T12:51:21.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:21.060+0000 7f3713dab640 1 -- 172.21.15.12:0/3275036658 wait complete. 2023-10-30T12:51:21.088 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2023-10-30T12:51:23.089 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:23.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.228+0000 7feb98217640 1 Processor -- start 2023-10-30T12:51:23.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.228+0000 7feb98217640 1 -- start start 2023-10-30T12:51:23.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.228+0000 7feb98217640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feb900c1ee0 0x7feb9006e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:23.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.228+0000 7feb98217640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7feb900c8260 0x7feb900cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:23.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.228+0000 7feb98217640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb9006e090 con 0x7feb90151800 2023-10-30T12:51:23.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.228+0000 7feb98217640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb9006dc70 con 0x7feb900c1ee0 2023-10-30T12:51:23.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.228+0000 7feb98217640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb900c8a50 con 0x7feb900c8260 2023-10-30T12:51:23.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.228+0000 7feb957b2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feb900c1ee0 0x7feb9006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:23.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.228+0000 7feb957b2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feb900c1ee0 0x7feb9006e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44708/0 (socket says 172.21.15.12:44708) 2023-10-30T12:51:23.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.228+0000 7feb957b2640 1 -- 172.21.15.12:0/3365596252 learned_addr learned my addr 172.21.15.12:0/3365596252 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:51:23.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.228+0000 7feb967b4640 1 --2- 172.21.15.12:0/3365596252 >> v2:172.21.15.154:3300/0 conn(0x7feb900c8260 0x7feb900cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.232+0000 7feb957b2640 1 -- 172.21.15.12:0/3365596252 >> v2:172.21.15.154:3300/0 conn(0x7feb900c8260 msgr2=0x7feb900cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.232+0000 7feb957b2640 1 --2- 172.21.15.12:0/3365596252 >> v2:172.21.15.154:3300/0 conn(0x7feb900c8260 0x7feb900cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.232+0000 7feb957b2640 1 -- 172.21.15.12:0/3365596252 >> v1:172.21.15.154:6789/0 conn(0x7feb90151800 legacy=0x7feb900c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.232+0000 7feb957b2640 1 -- 172.21.15.12:0/3365596252 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb900c3b10 con 0x7feb900c1ee0 2023-10-30T12:51:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.232+0000 7feb967b4640 1 --2- 172.21.15.12:0/3365596252 >> v2:172.21.15.154:3300/0 conn(0x7feb900c8260 0x7feb900cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:51:23.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.232+0000 7feb957b2640 1 --2- 172.21.15.12:0/3365596252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb900c1ee0 0x7feb9006e470 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7feb900c9900 tx=0x7feb8001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6974c8083e1f9eba server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.232+0000 7feb94fb1640 1 -- 172.21.15.12:0/3365596252 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb800ac640 con 0x7feb900c1ee0 2023-10-30T12:51:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.232+0000 7feb94fb1640 1 -- 172.21.15.12:0/3365596252 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb800ac7a0 con 0x7feb900c1ee0 2023-10-30T12:51:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.232+0000 7feb94fb1640 1 -- 172.21.15.12:0/3365596252 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb800acb30 con 0x7feb900c1ee0 2023-10-30T12:51:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.232+0000 7feb98217640 1 -- 172.21.15.12:0/3365596252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb900c1ee0 msgr2=0x7feb9006e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.232+0000 7feb98217640 1 --2- 172.21.15.12:0/3365596252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb900c1ee0 0x7feb9006e470 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7feb900c9900 tx=0x7feb8001c4d0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.232+0000 7feb98217640 1 -- 172.21.15.12:0/3365596252 shutdown_connections 2023-10-30T12:51:23.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.232+0000 7feb98217640 1 --2- 172.21.15.12:0/3365596252 >> v2:172.21.15.154:3300/0 conn(0x7feb900c8260 0x7feb900cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.232+0000 7feb98217640 1 --2- 172.21.15.12:0/3365596252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb900c1ee0 0x7feb9006e470 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.232+0000 7feb98217640 1 -- 172.21.15.12:0/3365596252 >> 172.21.15.12:0/3365596252 conn(0x7feb90071ea0 msgr2=0x7feb90065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:23.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb98217640 1 -- 172.21.15.12:0/3365596252 shutdown_connections 2023-10-30T12:51:23.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb98217640 1 -- 172.21.15.12:0/3365596252 wait complete. 2023-10-30T12:51:23.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb98217640 1 Processor -- start 2023-10-30T12:51:23.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb98217640 1 -- start start 2023-10-30T12:51:23.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb98217640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb900c1ee0 0x7feb90146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:23.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb98217640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb900c8260 0x7feb9014c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:23.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb98217640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb90151800 0x7feb901f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:23.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb98217640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb90156560 con 0x7feb90151800 2023-10-30T12:51:23.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb98217640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb901569f0 con 0x7feb900c1ee0 2023-10-30T12:51:23.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb98217640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb90157140 con 0x7feb900c8260 2023-10-30T12:51:23.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb967b4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb90151800 0x7feb901f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:23.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb967b4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb90151800 0x7feb901f3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44714/0 (socket says 172.21.15.12:44714) 2023-10-30T12:51:23.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb967b4640 1 -- 172.21.15.12:0/78976748 learned_addr learned my addr 172.21.15.12:0/78976748 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:51:23.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb957b2640 1 --2- 172.21.15.12:0/78976748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb900c8260 0x7feb9014c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:23.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb95fb3640 1 --2- 172.21.15.12:0/78976748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb900c1ee0 0x7feb90146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:23.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb967b4640 1 -- 172.21.15.12:0/78976748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb900c8260 msgr2=0x7feb9014c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:23.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb967b4640 1 --2- 172.21.15.12:0/78976748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb900c8260 0x7feb9014c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb967b4640 1 -- 172.21.15.12:0/78976748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb900c1ee0 msgr2=0x7feb90146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:23.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb967b4640 1 --2- 172.21.15.12:0/78976748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb900c1ee0 0x7feb90146b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb967b4640 1 -- 172.21.15.12:0/78976748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb800094d0 con 0x7feb90151800 2023-10-30T12:51:23.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb95fb3640 1 --2- 172.21.15.12:0/78976748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb900c1ee0 0x7feb90146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:51:23.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb967b4640 1 --2- 172.21.15.12:0/78976748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb90151800 0x7feb901f3910 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7feb88007df0 tx=0x7feb8800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:23.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb867fc640 1 -- 172.21.15.12:0/78976748 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb880039a0 con 0x7feb90151800 2023-10-30T12:51:23.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb867fc640 1 -- 172.21.15.12:0/78976748 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8800f040 con 0x7feb90151800 2023-10-30T12:51:23.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb98217640 1 -- 172.21.15.12:0/78976748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb90157400 con 0x7feb90151800 2023-10-30T12:51:23.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.236+0000 7feb867fc640 1 -- 172.21.15.12:0/78976748 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88003d40 con 0x7feb90151800 2023-10-30T12:51:23.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.240+0000 7feb98217640 1 -- 172.21.15.12:0/78976748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb90155f10 con 0x7feb90151800 2023-10-30T12:51:23.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.240+0000 7feb98217640 1 -- 172.21.15.12:0/78976748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb90156710 con 0x7feb90151800 2023-10-30T12:51:23.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.240+0000 7feb867fc640 1 -- 172.21.15.12:0/78976748 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feb88004030 con 0x7feb90151800 2023-10-30T12:51:23.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.240+0000 7feb867fc640 1 --2- 172.21.15.12:0/78976748 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb740664a0 0x7feb74068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:23.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.240+0000 7feb867fc640 1 -- 172.21.15.12:0/78976748 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7feb88084480 con 0x7feb90151800 2023-10-30T12:51:23.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.240+0000 7feb95fb3640 1 --2- 172.21.15.12:0/78976748 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb740664a0 0x7feb74068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:23.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.244+0000 7feb95fb3640 1 --2- 172.21.15.12:0/78976748 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb740664a0 0x7feb74068960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feb8c000f70 tx=0x7feb8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:23.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.244+0000 7feb867fc640 1 -- 172.21.15.12:0/78976748 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feb90156710 con 0x7feb90151800 2023-10-30T12:51:23.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.436+0000 7feb98217640 1 -- 172.21.15.12:0/78976748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb90156710 con 0x7feb90151800 2023-10-30T12:51:23.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.440+0000 7feb867fc640 1 -- 172.21.15.12:0/78976748 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+6538 (secure 0 0 0) 0x7feb90156710 con 0x7feb90151800 2023-10-30T12:51:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:51:23.447 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6021,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:51:13.607295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:23.447 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 86 2023-10-30T12:51:23.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.440+0000 7feb98217640 1 -- 172.21.15.12:0/78976748 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb740664a0 msgr2=0x7feb74068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:23.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.440+0000 7feb98217640 1 --2- 172.21.15.12:0/78976748 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb740664a0 0x7feb74068960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feb8c000f70 tx=0x7feb8c009040 comp rx=0 tx=0).stop 2023-10-30T12:51:23.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.440+0000 7feb98217640 1 -- 172.21.15.12:0/78976748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb90151800 msgr2=0x7feb901f3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:23.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.440+0000 7feb98217640 1 --2- 172.21.15.12:0/78976748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb90151800 0x7feb901f3910 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7feb88007df0 tx=0x7feb8800d4a0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.440+0000 7feb98217640 1 -- 172.21.15.12:0/78976748 shutdown_connections 2023-10-30T12:51:23.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.440+0000 7feb98217640 1 --2- 172.21.15.12:0/78976748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb90151800 0x7feb901f3910 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.440+0000 7feb98217640 1 --2- 172.21.15.12:0/78976748 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb740664a0 0x7feb74068960 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.440+0000 7feb98217640 1 --2- 172.21.15.12:0/78976748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb900c8260 0x7feb9014c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.440+0000 7feb98217640 1 --2- 172.21.15.12:0/78976748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb900c1ee0 0x7feb90146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.440+0000 7feb98217640 1 -- 172.21.15.12:0/78976748 >> 172.21.15.12:0/78976748 conn(0x7feb90071ea0 msgr2=0x7feb90144df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:23.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.440+0000 7feb98217640 1 -- 172.21.15.12:0/78976748 shutdown_connections 2023-10-30T12:51:23.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:51:23.440+0000 7feb98217640 1 -- 172.21.15.12:0/78976748 wait complete. 2023-10-30T12:51:23.467 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2023-10-30T12:51:23.468 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 100 secs before thrashing 2023-10-30T12:51:36.724 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:51:36.728 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:51:47.201 INFO:tasks.workunit.client.0.smithi012.stderr:+ timeout 3600 git clone https://git.ceph.com/ceph.git 2023-10-30T12:51:47.205 INFO:tasks.workunit.client.0.smithi012.stderr:Cloning into 'ceph'... 2023-10-30T12:52:06.781 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:52:06.790 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:52:36.907 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:52:36.913 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:53:03.468 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:03.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.604+0000 7f681b53e640 1 Processor -- start 2023-10-30T12:53:03.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.604+0000 7f681b53e640 1 -- start start 2023-10-30T12:53:03.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f681b53e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f68140c6bd0 0x7f681406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:03.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f681b53e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6814071110 0x7f6814158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:03.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f681b53e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6814066cf0 con 0x7f681406f7a0 2023-10-30T12:53:03.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f681b53e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68140700c0 con 0x7f68140c6bd0 2023-10-30T12:53:03.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f681b53e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68140c70d0 con 0x7f6814071110 2023-10-30T12:53:03.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f6818ad9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f68140c6bd0 0x7f681406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:03.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f6819adb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6814071110 0x7f6814158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:03.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f6819adb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6814071110 0x7f6814158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:60700/0 (socket says 172.21.15.12:60700) 2023-10-30T12:53:03.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f6819adb640 1 -- 172.21.15.12:0/446751604 learned_addr learned my addr 172.21.15.12:0/446751604 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:03.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f680bfff640 1 -- 172.21.15.12:0/446751604 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362385658 0 0) 0x7f6814066cf0 con 0x7f681406f7a0 2023-10-30T12:53:03.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f680bfff640 1 -- 172.21.15.12:0/446751604 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6800003530 con 0x7f681406f7a0 2023-10-30T12:53:03.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f6818ad9640 1 -- 172.21.15.12:0/446751604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6814071110 msgr2=0x7f6814158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:03.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f6818ad9640 1 --2- 172.21.15.12:0/446751604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6814071110 0x7f6814158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:03.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f6818ad9640 1 -- 172.21.15.12:0/446751604 >> v1:172.21.15.154:6789/0 conn(0x7f681406f7a0 legacy=0x7f6814069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:03.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f6818ad9640 1 -- 172.21.15.12:0/446751604 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68140c7390 con 0x7f68140c6bd0 2023-10-30T12:53:03.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.608+0000 7f6819adb640 1 --2- 172.21.15.12:0/446751604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6814071110 0x7f6814158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:53:03.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.612+0000 7f6818ad9640 1 --2- 172.21.15.12:0/446751604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68140c6bd0 0x7f681406fb80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f68040097c0 tx=0x7f680401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90ff948a32e5ee2a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:03.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.612+0000 7f680bfff640 1 -- 172.21.15.12:0/446751604 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68040a8970 con 0x7f68140c6bd0 2023-10-30T12:53:03.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.612+0000 7f680bfff640 1 -- 172.21.15.12:0/446751604 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68040a8ad0 con 0x7f68140c6bd0 2023-10-30T12:53:03.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.612+0000 7f680bfff640 1 -- 172.21.15.12:0/446751604 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68040a8e60 con 0x7f68140c6bd0 2023-10-30T12:53:03.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.612+0000 7f681b53e640 1 -- 172.21.15.12:0/446751604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68140c6bd0 msgr2=0x7f681406fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:03.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.612+0000 7f681b53e640 1 --2- 172.21.15.12:0/446751604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68140c6bd0 0x7f681406fb80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f68040097c0 tx=0x7f680401ec70 comp rx=0 tx=0).stop 2023-10-30T12:53:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.612+0000 7f681b53e640 1 -- 172.21.15.12:0/446751604 shutdown_connections 2023-10-30T12:53:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.612+0000 7f681b53e640 1 --2- 172.21.15.12:0/446751604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6814071110 0x7f6814158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.612+0000 7f681b53e640 1 --2- 172.21.15.12:0/446751604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68140c6bd0 0x7f681406fb80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.612+0000 7f681b53e640 1 -- 172.21.15.12:0/446751604 >> 172.21.15.12:0/446751604 conn(0x7f68140c7910 msgr2=0x7f68140c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.612+0000 7f681b53e640 1 -- 172.21.15.12:0/446751604 shutdown_connections 2023-10-30T12:53:03.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.612+0000 7f681b53e640 1 -- 172.21.15.12:0/446751604 wait complete. 2023-10-30T12:53:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f681b53e640 1 Processor -- start 2023-10-30T12:53:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f681b53e640 1 -- start start 2023-10-30T12:53:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f681b53e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f681406f7a0 0x7f68141f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f681b53e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6814071110 0x7f6814155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f681b53e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68140c6bd0 0x7f6814156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f681b53e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68140c1370 con 0x7f6814071110 2023-10-30T12:53:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f681b53e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6814155a70 con 0x7f68140c6bd0 2023-10-30T12:53:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f681b53e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68141f3480 con 0x7f681406f7a0 2023-10-30T12:53:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f6818ad9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6814071110 0x7f6814155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:03.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f6818ad9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6814071110 0x7f6814155f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56338/0 (socket says 172.21.15.12:56338) 2023-10-30T12:53:03.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f6818ad9640 1 -- 172.21.15.12:0/2465173717 learned_addr learned my addr 172.21.15.12:0/2465173717 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:03.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f6819adb640 1 --2- 172.21.15.12:0/2465173717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68140c6bd0 0x7f6814156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:03.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f68192da640 1 --2- 172.21.15.12:0/2465173717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f681406f7a0 0x7f68141f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:03.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f6818ad9640 1 -- 172.21.15.12:0/2465173717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f681406f7a0 msgr2=0x7f68141f2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:03.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f6818ad9640 1 --2- 172.21.15.12:0/2465173717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f681406f7a0 0x7f68141f2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:03.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f6818ad9640 1 -- 172.21.15.12:0/2465173717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68140c6bd0 msgr2=0x7f6814156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:03.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f6818ad9640 1 --2- 172.21.15.12:0/2465173717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68140c6bd0 0x7f6814156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:03.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f6818ad9640 1 -- 172.21.15.12:0/2465173717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c009b30 con 0x7f6814071110 2023-10-30T12:53:03.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f6818ad9640 1 --2- 172.21.15.12:0/2465173717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6814071110 0x7f6814155f00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f680401c8a0 tx=0x7f680401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:03.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f6809ffb640 1 -- 172.21.15.12:0/2465173717 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68040b0d40 con 0x7f6814071110 2023-10-30T12:53:03.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f6809ffb640 1 -- 172.21.15.12:0/2465173717 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68040bb420 con 0x7f6814071110 2023-10-30T12:53:03.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f6809ffb640 1 -- 172.21.15.12:0/2465173717 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68040bb7d0 con 0x7f6814071110 2023-10-30T12:53:03.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f681b53e640 1 -- 172.21.15.12:0/2465173717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6804009420 con 0x7f6814071110 2023-10-30T12:53:03.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.616+0000 7f681b53e640 1 -- 172.21.15.12:0/2465173717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68140c18a0 con 0x7f6814071110 2023-10-30T12:53:03.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.620+0000 7f681b53e640 1 -- 172.21.15.12:0/2465173717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681404e170 con 0x7f6814071110 2023-10-30T12:53:03.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.620+0000 7f6809ffb640 1 -- 172.21.15.12:0/2465173717 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f68040b7040 con 0x7f6814071110 2023-10-30T12:53:03.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.620+0000 7f6809ffb640 1 --2- 172.21.15.12:0/2465173717 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f67f80664a0 0x7f67f8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:03.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.620+0000 7f6809ffb640 1 -- 172.21.15.12:0/2465173717 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f6804121fc0 con 0x7f6814071110 2023-10-30T12:53:03.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.620+0000 7f68192da640 1 --2- 172.21.15.12:0/2465173717 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f67f80664a0 0x7f67f8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:03.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.620+0000 7f68192da640 1 --2- 172.21.15.12:0/2465173717 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f67f80664a0 0x7f67f8068960 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f68100011f0 tx=0x7f6810009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:03.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.624+0000 7f6809ffb640 1 -- 172.21.15.12:0/2465173717 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f68040ef1b0 con 0x7f6814071110 2023-10-30T12:53:03.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.816+0000 7f681b53e640 1 -- 172.21.15.12:0/2465173717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6814155bb0 con 0x7f6814071110 2023-10-30T12:53:03.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.816+0000 7f6809ffb640 1 -- 172.21.15.12:0/2465173717 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6537 (secure 0 0 0) 0x7f68040f27d0 con 0x7f6814071110 2023-10-30T12:53:03.823 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:03.823 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6021,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:52:24.610147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:03.824 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 88 2023-10-30T12:53:03.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.816+0000 7f681b53e640 1 -- 172.21.15.12:0/2465173717 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f67f80664a0 msgr2=0x7f67f8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:03.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.816+0000 7f681b53e640 1 --2- 172.21.15.12:0/2465173717 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f67f80664a0 0x7f67f8068960 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f68100011f0 tx=0x7f6810009040 comp rx=0 tx=0).stop 2023-10-30T12:53:03.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.816+0000 7f681b53e640 1 -- 172.21.15.12:0/2465173717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6814071110 msgr2=0x7f6814155f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:03.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.816+0000 7f681b53e640 1 --2- 172.21.15.12:0/2465173717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6814071110 0x7f6814155f00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f680401c8a0 tx=0x7f680401ba10 comp rx=0 tx=0).stop 2023-10-30T12:53:03.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.816+0000 7f681b53e640 1 -- 172.21.15.12:0/2465173717 shutdown_connections 2023-10-30T12:53:03.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.816+0000 7f681b53e640 1 --2- 172.21.15.12:0/2465173717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68140c6bd0 0x7f6814156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:03.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.816+0000 7f681b53e640 1 --2- 172.21.15.12:0/2465173717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6814071110 0x7f6814155f00 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:03.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.816+0000 7f681b53e640 1 --2- 172.21.15.12:0/2465173717 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f67f80664a0 0x7f67f8068960 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:03.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.816+0000 7f681b53e640 1 --2- 172.21.15.12:0/2465173717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f681406f7a0 0x7f68141f2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:03.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.816+0000 7f681b53e640 1 -- 172.21.15.12:0/2465173717 >> 172.21.15.12:0/2465173717 conn(0x7f68140c7910 msgr2=0x7f68140c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:03.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.816+0000 7f681b53e640 1 -- 172.21.15.12:0/2465173717 shutdown_connections 2023-10-30T12:53:03.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:03.816+0000 7f681b53e640 1 -- 172.21.15.12:0/2465173717 wait complete. 2023-10-30T12:53:03.844 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=0) 2023-10-30T12:53:03.845 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-30T12:53:03.845 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:53:03.862 INFO:tasks.ceph.mds.c:Stopped 2023-10-30T12:53:03.862 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:04.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.044+0000 7f0abce79640 1 Processor -- start 2023-10-30T12:53:04.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.044+0000 7f0abce79640 1 -- start start 2023-10-30T12:53:04.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.044+0000 7f0abce79640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0ab815a410 0x7f0ab806a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:04.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.044+0000 7f0abce79640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0ab806cb80 0x7f0ab806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:04.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.044+0000 7f0abce79640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab806d530 con 0x7f0ab8069e90 2023-10-30T12:53:04.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.044+0000 7f0abce79640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab80c8570 con 0x7f0ab806cb80 2023-10-30T12:53:04.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.044+0000 7f0abce79640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab815a850 con 0x7f0ab815a410 2023-10-30T12:53:04.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.044+0000 7f0ab6d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0ab806cb80 0x7f0ab806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:04.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.048+0000 7f0ab6d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0ab806cb80 0x7f0ab806cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56348/0 (socket says 172.21.15.12:56348) 2023-10-30T12:53:04.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.048+0000 7f0ab6d9d640 1 -- 172.21.15.12:0/2098138020 learned_addr learned my addr 172.21.15.12:0/2098138020 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:04.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.048+0000 7f0ab5d9b640 1 --2- 172.21.15.12:0/2098138020 >> v2:172.21.15.154:3300/0 conn(0x7f0ab815a410 0x7f0ab806a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:04.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.048+0000 7f0ab6d9d640 1 -- 172.21.15.12:0/2098138020 >> v2:172.21.15.154:3300/0 conn(0x7f0ab815a410 msgr2=0x7f0ab806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:04.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.048+0000 7f0ab6d9d640 1 --2- 172.21.15.12:0/2098138020 >> v2:172.21.15.154:3300/0 conn(0x7f0ab815a410 0x7f0ab806a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:04.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.048+0000 7f0ab6d9d640 1 -- 172.21.15.12:0/2098138020 >> v1:172.21.15.154:6789/0 conn(0x7f0ab8069e90 legacy=0x7f0ab80c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:04.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.048+0000 7f0ab6d9d640 1 -- 172.21.15.12:0/2098138020 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80c6c10 con 0x7f0ab806cb80 2023-10-30T12:53:04.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.048+0000 7f0ab6d9d640 1 --2- 172.21.15.12:0/2098138020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ab806cb80 0x7f0ab806cff0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0ab806feb0 tx=0x7f0ab001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b86f52164755ad9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:04.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.048+0000 7f0ab559a640 1 -- 172.21.15.12:0/2098138020 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00a5690 con 0x7f0ab806cb80 2023-10-30T12:53:04.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.048+0000 7f0ab559a640 1 -- 172.21.15.12:0/2098138020 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab00a57f0 con 0x7f0ab806cb80 2023-10-30T12:53:04.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.048+0000 7f0ab559a640 1 -- 172.21.15.12:0/2098138020 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00a5b80 con 0x7f0ab806cb80 2023-10-30T12:53:04.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 -- 172.21.15.12:0/2098138020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ab806cb80 msgr2=0x7f0ab806cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:04.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 --2- 172.21.15.12:0/2098138020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ab806cb80 0x7f0ab806cff0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0ab806feb0 tx=0x7f0ab001c9d0 comp rx=0 tx=0).stop 2023-10-30T12:53:04.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 -- 172.21.15.12:0/2098138020 shutdown_connections 2023-10-30T12:53:04.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 --2- 172.21.15.12:0/2098138020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ab806cb80 0x7f0ab806cff0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:04.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 --2- 172.21.15.12:0/2098138020 >> v2:172.21.15.154:3300/0 conn(0x7f0ab815a410 0x7f0ab806a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:04.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 -- 172.21.15.12:0/2098138020 >> 172.21.15.12:0/2098138020 conn(0x7f0ab806dfd0 msgr2=0x7f0ab806e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:04.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 -- 172.21.15.12:0/2098138020 shutdown_connections 2023-10-30T12:53:04.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 -- 172.21.15.12:0/2098138020 wait complete. 2023-10-30T12:53:04.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 Processor -- start 2023-10-30T12:53:04.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 -- start start 2023-10-30T12:53:04.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8069e90 0x7f0ab8147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:04.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ab806cb80 0x7f0ab806b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:04.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab815a410 0x7f0ab806bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:04.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab815d840 con 0x7f0ab806cb80 2023-10-30T12:53:04.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab814e270 con 0x7f0ab815a410 2023-10-30T12:53:04.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0abce79640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab814e550 con 0x7f0ab8069e90 2023-10-30T12:53:04.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0ab5d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ab806cb80 0x7f0ab806b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:04.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0ab5d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ab806cb80 0x7f0ab806b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56360/0 (socket says 172.21.15.12:56360) 2023-10-30T12:53:04.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0ab5d9b640 1 -- 172.21.15.12:0/2315238925 learned_addr learned my addr 172.21.15.12:0/2315238925 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:04.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0ab6d9d640 1 --2- 172.21.15.12:0/2315238925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab815a410 0x7f0ab806bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:04.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0ab659c640 1 --2- 172.21.15.12:0/2315238925 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8069e90 0x7f0ab8147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:04.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0ab5d9b640 1 -- 172.21.15.12:0/2315238925 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8069e90 msgr2=0x7f0ab8147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:04.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0ab5d9b640 1 --2- 172.21.15.12:0/2315238925 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8069e90 0x7f0ab8147ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:04.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0ab5d9b640 1 -- 172.21.15.12:0/2315238925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab815a410 msgr2=0x7f0ab806bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:04.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0ab5d9b640 1 --2- 172.21.15.12:0/2315238925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab815a410 0x7f0ab806bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:04.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.052+0000 7f0ab5d9b640 1 -- 172.21.15.12:0/2315238925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab0009b30 con 0x7f0ab806cb80 2023-10-30T12:53:04.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.056+0000 7f0ab5d9b640 1 --2- 172.21.15.12:0/2315238925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ab806cb80 0x7f0ab806b580 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00d5f0 tx=0x7f0aac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:04.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.056+0000 7f0aa6ffd640 1 -- 172.21.15.12:0/2315238925 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0125f0 con 0x7f0ab806cb80 2023-10-30T12:53:04.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.056+0000 7f0aa6ffd640 1 -- 172.21.15.12:0/2315238925 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac012750 con 0x7f0ab806cb80 2023-10-30T12:53:04.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.056+0000 7f0aa6ffd640 1 -- 172.21.15.12:0/2315238925 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac012ae0 con 0x7f0ab806cb80 2023-10-30T12:53:04.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.056+0000 7f0abce79640 1 -- 172.21.15.12:0/2315238925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab814da90 con 0x7f0ab806cb80 2023-10-30T12:53:04.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.056+0000 7f0abce79640 1 -- 172.21.15.12:0/2315238925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab814dd70 con 0x7f0ab806cb80 2023-10-30T12:53:04.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.056+0000 7f0abce79640 1 -- 172.21.15.12:0/2315238925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab804e170 con 0x7f0ab806cb80 2023-10-30T12:53:04.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.056+0000 7f0aa6ffd640 1 -- 172.21.15.12:0/2315238925 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0aac012c40 con 0x7f0ab806cb80 2023-10-30T12:53:04.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.056+0000 7f0aa6ffd640 1 --2- 172.21.15.12:0/2315238925 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a980664a0 0x7f0a98068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:04.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.056+0000 7f0aa6ffd640 1 -- 172.21.15.12:0/2315238925 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f0aac084720 con 0x7f0ab806cb80 2023-10-30T12:53:04.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.056+0000 7f0ab659c640 1 --2- 172.21.15.12:0/2315238925 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a980664a0 0x7f0a98068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:04.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.060+0000 7f0ab659c640 1 --2- 172.21.15.12:0/2315238925 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a980664a0 0x7f0a98068960 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8000f70 tx=0x7f0aa8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:04.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.060+0000 7f0aa6ffd640 1 -- 172.21.15.12:0/2315238925 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0aac051740 con 0x7f0ab806cb80 2023-10-30T12:53:04.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.252+0000 7f0abce79640 1 -- 172.21.15.12:0/2315238925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ab806ef60 con 0x7f0ab806cb80 2023-10-30T12:53:04.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.252+0000 7f0aa6ffd640 1 -- 172.21.15.12:0/2315238925 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6537 (secure 0 0 0) 0x7f0aac007380 con 0x7f0ab806cb80 2023-10-30T12:53:04.261 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:04.262 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6021,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:52:24.610147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:04.263 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 88 2023-10-30T12:53:04.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.256+0000 7f0abce79640 1 -- 172.21.15.12:0/2315238925 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a980664a0 msgr2=0x7f0a98068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:04.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.256+0000 7f0abce79640 1 --2- 172.21.15.12:0/2315238925 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a980664a0 0x7f0a98068960 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8000f70 tx=0x7f0aa8009040 comp rx=0 tx=0).stop 2023-10-30T12:53:04.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.256+0000 7f0abce79640 1 -- 172.21.15.12:0/2315238925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ab806cb80 msgr2=0x7f0ab806b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:04.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.256+0000 7f0abce79640 1 --2- 172.21.15.12:0/2315238925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ab806cb80 0x7f0ab806b580 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00d5f0 tx=0x7f0aac00dab0 comp rx=0 tx=0).stop 2023-10-30T12:53:04.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.256+0000 7f0abce79640 1 -- 172.21.15.12:0/2315238925 shutdown_connections 2023-10-30T12:53:04.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.256+0000 7f0abce79640 1 --2- 172.21.15.12:0/2315238925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab815a410 0x7f0ab806bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:04.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.256+0000 7f0abce79640 1 --2- 172.21.15.12:0/2315238925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ab806cb80 0x7f0ab806b580 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:04.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.256+0000 7f0abce79640 1 --2- 172.21.15.12:0/2315238925 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a980664a0 0x7f0a98068960 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:04.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.256+0000 7f0abce79640 1 --2- 172.21.15.12:0/2315238925 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8069e90 0x7f0ab8147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:04.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.256+0000 7f0abce79640 1 -- 172.21.15.12:0/2315238925 >> 172.21.15.12:0/2315238925 conn(0x7f0ab806dfd0 msgr2=0x7f0ab80c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:04.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.256+0000 7f0abce79640 1 -- 172.21.15.12:0/2315238925 shutdown_connections 2023-10-30T12:53:04.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:04.256+0000 7f0abce79640 1 -- 172.21.15.12:0/2315238925 wait complete. 2023-10-30T12:53:04.282 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:53:06.283 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:06.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.428+0000 7f4dd8a17640 1 Processor -- start 2023-10-30T12:53:06.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.428+0000 7f4dd8a17640 1 -- start start 2023-10-30T12:53:06.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.432+0000 7f4dd8a17640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4dd415d6f0 0x7f4dd415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:06.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.432+0000 7f4dd8a17640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd40cb800 con 0x7f4dd41538a0 2023-10-30T12:53:06.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.432+0000 7f4dd8a17640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd40cb330 con 0x7f4dd40cf160 2023-10-30T12:53:06.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.432+0000 7f4dd8a17640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd40cad70 con 0x7f4dd415d6f0 2023-10-30T12:53:06.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.432+0000 7f4dcbfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4dd415d6f0 0x7f4dd415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:06.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.432+0000 7f4dcbfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4dd415d6f0 0x7f4dd415a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56372/0 (socket says 172.21.15.12:56372) 2023-10-30T12:53:06.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.432+0000 7f4dcbfff640 1 -- 172.21.15.12:0/1871258306 learned_addr learned my addr 172.21.15.12:0/1871258306 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:06.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.432+0000 7f4dcbfff640 1 -- 172.21.15.12:0/1871258306 >> v1:172.21.15.154:6789/0 conn(0x7f4dd40cf160 legacy=0x7f4dd4164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:06.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.432+0000 7f4dcbfff640 1 -- 172.21.15.12:0/1871258306 >> v1:172.21.15.12:6789/0 conn(0x7f4dd41538a0 legacy=0x7f4dd40ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:06.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.432+0000 7f4dcbfff640 1 -- 172.21.15.12:0/1871258306 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd40cb050 con 0x7f4dd415d6f0 2023-10-30T12:53:06.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.436+0000 7f4dcbfff640 1 --2- 172.21.15.12:0/1871258306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd415d6f0 0x7f4dd415a090 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009870 tx=0x7f4dbc01ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73d5308793ab6d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:06.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.436+0000 7f4dd1d9b640 1 -- 172.21.15.12:0/1871258306 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc0a8a30 con 0x7f4dd415d6f0 2023-10-30T12:53:06.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.436+0000 7f4dd1d9b640 1 -- 172.21.15.12:0/1871258306 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dbc0a8b90 con 0x7f4dd415d6f0 2023-10-30T12:53:06.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.436+0000 7f4dd1d9b640 1 -- 172.21.15.12:0/1871258306 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc0b0440 con 0x7f4dd415d6f0 2023-10-30T12:53:06.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.436+0000 7f4dd8a17640 1 -- 172.21.15.12:0/1871258306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd415d6f0 msgr2=0x7f4dd415a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:06.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.436+0000 7f4dd8a17640 1 --2- 172.21.15.12:0/1871258306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd415d6f0 0x7f4dd415a090 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009870 tx=0x7f4dbc01ed30 comp rx=0 tx=0).stop 2023-10-30T12:53:06.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.436+0000 7f4dd8a17640 1 -- 172.21.15.12:0/1871258306 shutdown_connections 2023-10-30T12:53:06.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.436+0000 7f4dd8a17640 1 --2- 172.21.15.12:0/1871258306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd415d6f0 0x7f4dd415a090 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:06.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.436+0000 7f4dd8a17640 1 -- 172.21.15.12:0/1871258306 >> 172.21.15.12:0/1871258306 conn(0x7f4dd406bfe0 msgr2=0x7f4dd406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:06.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.436+0000 7f4dd8a17640 1 -- 172.21.15.12:0/1871258306 shutdown_connections 2023-10-30T12:53:06.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.436+0000 7f4dd8a17640 1 -- 172.21.15.12:0/1871258306 wait complete. 2023-10-30T12:53:06.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.436+0000 7f4dd8a17640 1 Processor -- start 2023-10-30T12:53:06.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.436+0000 7f4dd8a17640 1 -- start start 2023-10-30T12:53:06.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd8a17640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40cf160 0x7f4dd4148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:06.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd8a17640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd41538a0 0x7f4dd41492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:06.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd8a17640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd415d6f0 0x7f4dd41503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:06.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd8a17640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd41ef170 con 0x7f4dd415d6f0 2023-10-30T12:53:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd8a17640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd41ef360 con 0x7f4dd41538a0 2023-10-30T12:53:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd8a17640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd41ef620 con 0x7f4dd40cf160 2023-10-30T12:53:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd415d6f0 0x7f4dd41503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd415d6f0 0x7f4dd41503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56374/0 (socket says 172.21.15.12:56374) 2023-10-30T12:53:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd2d9d640 1 -- 172.21.15.12:0/388022466 learned_addr learned my addr 172.21.15.12:0/388022466 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dcbfff640 1 --2- 172.21.15.12:0/388022466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd41538a0 0x7f4dd41492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd259c640 1 --2- 172.21.15.12:0/388022466 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40cf160 0x7f4dd4148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd2d9d640 1 -- 172.21.15.12:0/388022466 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40cf160 msgr2=0x7f4dd4148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd2d9d640 1 --2- 172.21.15.12:0/388022466 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40cf160 0x7f4dd4148d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:06.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd2d9d640 1 -- 172.21.15.12:0/388022466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd41538a0 msgr2=0x7f4dd41492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:06.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd2d9d640 1 --2- 172.21.15.12:0/388022466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd41538a0 0x7f4dd41492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:06.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd2d9d640 1 -- 172.21.15.12:0/388022466 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dbc0094d0 con 0x7f4dd415d6f0 2023-10-30T12:53:06.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dcbfff640 1 --2- 172.21.15.12:0/388022466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd41538a0 0x7f4dd41492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:06.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd2d9d640 1 --2- 172.21.15.12:0/388022466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd415d6f0 0x7f4dd41503d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00c730 tx=0x7f4dcc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:06.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dcaffd640 1 -- 172.21.15.12:0/388022466 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc003b80 con 0x7f4dd415d6f0 2023-10-30T12:53:06.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dcaffd640 1 -- 172.21.15.12:0/388022466 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc003ce0 con 0x7f4dd415d6f0 2023-10-30T12:53:06.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd8a17640 1 -- 172.21.15.12:0/388022466 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd41ef900 con 0x7f4dd415d6f0 2023-10-30T12:53:06.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dcaffd640 1 -- 172.21.15.12:0/388022466 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc005310 con 0x7f4dd415d6f0 2023-10-30T12:53:06.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd8a17640 1 -- 172.21.15.12:0/388022466 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd41efec0 con 0x7f4dd415d6f0 2023-10-30T12:53:06.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dd8a17640 1 -- 172.21.15.12:0/388022466 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd41f00b0 con 0x7f4dd415d6f0 2023-10-30T12:53:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.440+0000 7f4dcaffd640 1 -- 172.21.15.12:0/388022466 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4dcc005680 con 0x7f4dd415d6f0 2023-10-30T12:53:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.444+0000 7f4dcaffd640 1 --2- 172.21.15.12:0/388022466 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4db40664a0 0x7f4db4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.444+0000 7f4dcaffd640 1 -- 172.21.15.12:0/388022466 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f4dcc084160 con 0x7f4dd415d6f0 2023-10-30T12:53:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.444+0000 7f4dd259c640 1 --2- 172.21.15.12:0/388022466 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4db40664a0 0x7f4db4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:06.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.444+0000 7f4dd259c640 1 --2- 172.21.15.12:0/388022466 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4db40664a0 0x7f4db4068960 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4000f70 tx=0x7f4dc4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:06.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.448+0000 7f4dcaffd640 1 -- 172.21.15.12:0/388022466 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4dcc051220 con 0x7f4dd415d6f0 2023-10-30T12:53:06.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.636+0000 7f4dd8a17640 1 -- 172.21.15.12:0/388022466 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dd404e170 con 0x7f4dd415d6f0 2023-10-30T12:53:06.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.640+0000 7f4dcaffd640 1 -- 172.21.15.12:0/388022466 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6537 (secure 0 0 0) 0x7f4dcc054840 con 0x7f4dd415d6f0 2023-10-30T12:53:06.647 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:06.647 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6021,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:52:24.610147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:06.647 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 88 2023-10-30T12:53:06.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.640+0000 7f4dd8a17640 1 -- 172.21.15.12:0/388022466 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4db40664a0 msgr2=0x7f4db4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:06.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.640+0000 7f4dd8a17640 1 --2- 172.21.15.12:0/388022466 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4db40664a0 0x7f4db4068960 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4000f70 tx=0x7f4dc4009040 comp rx=0 tx=0).stop 2023-10-30T12:53:06.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.640+0000 7f4dd8a17640 1 -- 172.21.15.12:0/388022466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd415d6f0 msgr2=0x7f4dd41503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:06.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.640+0000 7f4dd8a17640 1 --2- 172.21.15.12:0/388022466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd415d6f0 0x7f4dd41503d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00c730 tx=0x7f4dcc00cbf0 comp rx=0 tx=0).stop 2023-10-30T12:53:06.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.640+0000 7f4dd8a17640 1 -- 172.21.15.12:0/388022466 shutdown_connections 2023-10-30T12:53:06.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.640+0000 7f4dd8a17640 1 --2- 172.21.15.12:0/388022466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd415d6f0 0x7f4dd41503d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:06.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.640+0000 7f4dd8a17640 1 --2- 172.21.15.12:0/388022466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4dd41538a0 0x7f4dd41492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:06.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.640+0000 7f4dd8a17640 1 --2- 172.21.15.12:0/388022466 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4db40664a0 0x7f4db4068960 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:06.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.640+0000 7f4dd8a17640 1 --2- 172.21.15.12:0/388022466 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4dd40cf160 0x7f4dd4148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:06.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.640+0000 7f4dd8a17640 1 -- 172.21.15.12:0/388022466 >> 172.21.15.12:0/388022466 conn(0x7f4dd406bfe0 msgr2=0x7f4dd40ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:06.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.640+0000 7f4dd8a17640 1 -- 172.21.15.12:0/388022466 shutdown_connections 2023-10-30T12:53:06.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:06.640+0000 7f4dd8a17640 1 -- 172.21.15.12:0/388022466 wait complete. 2023-10-30T12:53:06.668 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:53:07.023 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:53:07.027 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:53:08.668 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:08.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.816+0000 7f1bac479640 1 Processor -- start 2023-10-30T12:53:08.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.816+0000 7f1bac479640 1 -- start start 2023-10-30T12:53:08.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.816+0000 7f1bac479640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1ba40c6b80 0x7f1ba40bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:08.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.816+0000 7f1bac479640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ba40c6f60 0x7f1ba40c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:08.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.820+0000 7f1bac479640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba40c0a50 con 0x7f1ba40c8430 2023-10-30T12:53:08.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.820+0000 7f1bac479640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba406f6e0 con 0x7f1ba40c6b80 2023-10-30T12:53:08.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.820+0000 7f1bac479640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba406f9c0 con 0x7f1ba40c6f60 2023-10-30T12:53:08.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.820+0000 7f1baaa16640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f1ba40c8430 0x7f1ba413ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:53024/0 (socket says 172.21.15.12:53024) 2023-10-30T12:53:08.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.820+0000 7f1baaa16640 1 -- 172.21.15.12:0/4252985732 learned_addr learned my addr 172.21.15.12:0/4252985732 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:53:08.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.820+0000 7f1baa215640 1 --2- 172.21.15.12:0/4252985732 >> v2:172.21.15.154:3300/0 conn(0x7f1ba40c6b80 0x7f1ba40bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:08.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.820+0000 7f1ba9a14640 1 --2- 172.21.15.12:0/4252985732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ba40c6f60 0x7f1ba40c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:08.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.820+0000 7f1ba9213640 1 -- 172.21.15.12:0/4252985732 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727157172 0 0) 0x7f1ba40c0a50 con 0x7f1ba40c8430 2023-10-30T12:53:08.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.820+0000 7f1ba9213640 1 -- 172.21.15.12:0/4252985732 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b90003530 con 0x7f1ba40c8430 2023-10-30T12:53:08.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.820+0000 7f1ba9213640 1 -- 172.21.15.12:0/4252985732 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 840554218 0 0) 0x7f1b90003530 con 0x7f1ba40c8430 2023-10-30T12:53:08.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.820+0000 7f1baa215640 1 -- 172.21.15.12:0/4252985732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ba40c6f60 msgr2=0x7f1ba40c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:08.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.820+0000 7f1baa215640 1 --2- 172.21.15.12:0/4252985732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ba40c6f60 0x7f1ba40c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:08.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.820+0000 7f1baa215640 1 -- 172.21.15.12:0/4252985732 >> v1:172.21.15.12:6789/0 conn(0x7f1ba40c8430 legacy=0x7f1ba413ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:08.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.820+0000 7f1baa215640 1 -- 172.21.15.12:0/4252985732 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba406fca0 con 0x7f1ba40c6b80 2023-10-30T12:53:08.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.820+0000 7f1ba9a14640 1 --2- 172.21.15.12:0/4252985732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ba40c6f60 0x7f1ba40c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:53:08.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1baa215640 1 --2- 172.21.15.12:0/4252985732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ba40c6b80 0x7f1ba40bfec0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00097c0 tx=0x7f1ba001e520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82723623a6ba221c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:08.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1ba9213640 1 -- 172.21.15.12:0/4252985732 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c001a00 con 0x7f1ba40c6b80 2023-10-30T12:53:08.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1ba9213640 1 -- 172.21.15.12:0/4252985732 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba00a9050 con 0x7f1ba40c6b80 2023-10-30T12:53:08.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1ba9213640 1 -- 172.21.15.12:0/4252985732 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba00ad620 con 0x7f1ba40c6b80 2023-10-30T12:53:08.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1bac479640 1 -- 172.21.15.12:0/4252985732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ba40c6b80 msgr2=0x7f1ba40bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:08.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1bac479640 1 --2- 172.21.15.12:0/4252985732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ba40c6b80 0x7f1ba40bfec0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00097c0 tx=0x7f1ba001e520 comp rx=0 tx=0).stop 2023-10-30T12:53:08.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1bac479640 1 -- 172.21.15.12:0/4252985732 shutdown_connections 2023-10-30T12:53:08.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1bac479640 1 --2- 172.21.15.12:0/4252985732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ba40c6f60 0x7f1ba40c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:08.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1bac479640 1 --2- 172.21.15.12:0/4252985732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ba40c6b80 0x7f1ba40bfec0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:08.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1bac479640 1 -- 172.21.15.12:0/4252985732 >> 172.21.15.12:0/4252985732 conn(0x7f1ba40c7910 msgr2=0x7f1ba40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:08.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1bac479640 1 -- 172.21.15.12:0/4252985732 shutdown_connections 2023-10-30T12:53:08.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1bac479640 1 -- 172.21.15.12:0/4252985732 wait complete. 2023-10-30T12:53:08.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1bac479640 1 Processor -- start 2023-10-30T12:53:08.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1bac479640 1 -- start start 2023-10-30T12:53:08.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1bac479640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ba40c6b80 0x7f1ba40c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:08.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.824+0000 7f1bac479640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ba40c6f60 0x7f1ba40c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:08.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1bac479640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ba40c8430 0x7f1ba413ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:08.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1bac479640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba4153a10 con 0x7f1ba40c6f60 2023-10-30T12:53:08.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1bac479640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba4134010 con 0x7f1ba40c8430 2023-10-30T12:53:08.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1bac479640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba4134540 con 0x7f1ba40c6b80 2023-10-30T12:53:08.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1ba9a14640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ba40c6f60 0x7f1ba40c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:08.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1ba9a14640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ba40c6f60 0x7f1ba40c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56378/0 (socket says 172.21.15.12:56378) 2023-10-30T12:53:08.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1ba9a14640 1 -- 172.21.15.12:0/728956634 learned_addr learned my addr 172.21.15.12:0/728956634 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:08.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1baa215640 1 --2- 172.21.15.12:0/728956634 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ba40c6b80 0x7f1ba40c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:08.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1baaa16640 1 --2- 172.21.15.12:0/728956634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ba40c8430 0x7f1ba413ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:08.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1ba9a14640 1 -- 172.21.15.12:0/728956634 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ba40c6b80 msgr2=0x7f1ba40c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:08.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1ba9a14640 1 --2- 172.21.15.12:0/728956634 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ba40c6b80 0x7f1ba40c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:08.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1ba9a14640 1 -- 172.21.15.12:0/728956634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ba40c8430 msgr2=0x7f1ba413ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:08.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1ba9a14640 1 --2- 172.21.15.12:0/728956634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ba40c8430 0x7f1ba413ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:08.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1ba9a14640 1 -- 172.21.15.12:0/728956634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b94009420 con 0x7f1ba40c6f60 2023-10-30T12:53:08.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1baa215640 1 --2- 172.21.15.12:0/728956634 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ba40c6b80 0x7f1ba40c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:08.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1ba9a14640 1 --2- 172.21.15.12:0/728956634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ba40c6f60 0x7f1ba40c94c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400ebe0 tx=0x7f1b9400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:08.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1b9affd640 1 -- 172.21.15.12:0/728956634 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9400eee0 con 0x7f1ba40c6f60 2023-10-30T12:53:08.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1b9affd640 1 -- 172.21.15.12:0/728956634 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9400cc30 con 0x7f1ba40c6f60 2023-10-30T12:53:08.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1bac479640 1 -- 172.21.15.12:0/728956634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba0009420 con 0x7f1ba40c6f60 2023-10-30T12:53:08.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1b9affd640 1 -- 172.21.15.12:0/728956634 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94013470 con 0x7f1ba40c6f60 2023-10-30T12:53:08.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1bac479640 1 -- 172.21.15.12:0/728956634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba41f1c30 con 0x7f1ba40c6f60 2023-10-30T12:53:08.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1bac479640 1 -- 172.21.15.12:0/728956634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba404e170 con 0x7f1ba40c6f60 2023-10-30T12:53:08.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.828+0000 7f1b9affd640 1 -- 172.21.15.12:0/728956634 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1b940135d0 con 0x7f1ba40c6f60 2023-10-30T12:53:08.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.832+0000 7f1b9affd640 1 --2- 172.21.15.12:0/728956634 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1b880664a0 0x7f1b88068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:08.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.832+0000 7f1b9affd640 1 -- 172.21.15.12:0/728956634 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f1b940848a0 con 0x7f1ba40c6f60 2023-10-30T12:53:08.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.832+0000 7f1baa215640 1 --2- 172.21.15.12:0/728956634 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1b880664a0 0x7f1b88068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:08.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.832+0000 7f1baa215640 1 --2- 172.21.15.12:0/728956634 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1b880664a0 0x7f1b88068960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00af000 tx=0x7f1ba0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:08.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:08.836+0000 7f1b9affd640 1 -- 172.21.15.12:0/728956634 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1b940519e0 con 0x7f1ba40c6f60 2023-10-30T12:53:09.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:09.024+0000 7f1bac479640 1 -- 172.21.15.12:0/728956634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ba4135220 con 0x7f1ba40c6f60 2023-10-30T12:53:09.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:09.024+0000 7f1b9affd640 1 -- 172.21.15.12:0/728956634 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6537 (secure 0 0 0) 0x7f1b94055000 con 0x7f1ba40c6f60 2023-10-30T12:53:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:09.032 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6021,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:52:24.610147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:09.032 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 88 2023-10-30T12:53:09.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:09.024+0000 7f1bac479640 1 -- 172.21.15.12:0/728956634 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1b880664a0 msgr2=0x7f1b88068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:09.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:09.024+0000 7f1bac479640 1 --2- 172.21.15.12:0/728956634 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1b880664a0 0x7f1b88068960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00af000 tx=0x7f1ba0008040 comp rx=0 tx=0).stop 2023-10-30T12:53:09.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:09.024+0000 7f1bac479640 1 -- 172.21.15.12:0/728956634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ba40c6f60 msgr2=0x7f1ba40c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:09.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:09.024+0000 7f1bac479640 1 --2- 172.21.15.12:0/728956634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ba40c6f60 0x7f1ba40c94c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400ebe0 tx=0x7f1b9400c520 comp rx=0 tx=0).stop 2023-10-30T12:53:09.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:09.024+0000 7f1bac479640 1 -- 172.21.15.12:0/728956634 shutdown_connections 2023-10-30T12:53:09.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:09.024+0000 7f1bac479640 1 --2- 172.21.15.12:0/728956634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ba40c8430 0x7f1ba413ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:09.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:09.024+0000 7f1bac479640 1 --2- 172.21.15.12:0/728956634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ba40c6f60 0x7f1ba40c94c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:09.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:09.024+0000 7f1bac479640 1 --2- 172.21.15.12:0/728956634 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1b880664a0 0x7f1b88068960 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:09.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:09.024+0000 7f1bac479640 1 --2- 172.21.15.12:0/728956634 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ba40c6b80 0x7f1ba40c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:09.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:09.024+0000 7f1bac479640 1 -- 172.21.15.12:0/728956634 >> 172.21.15.12:0/728956634 conn(0x7f1ba40c7910 msgr2=0x7f1ba4134770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:09.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:09.024+0000 7f1bac479640 1 -- 172.21.15.12:0/728956634 shutdown_connections 2023-10-30T12:53:09.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:09.024+0000 7f1bac479640 1 -- 172.21.15.12:0/728956634 wait complete. 2023-10-30T12:53:09.052 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:53:11.053 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:11.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.200+0000 7fa20c7ab640 1 Processor -- start 2023-10-30T12:53:11.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.200+0000 7fa20c7ab640 1 -- start start 2023-10-30T12:53:11.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.200+0000 7fa20c7ab640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2040c05d0 0x7fa20413ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:11.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.200+0000 7fa20c7ab640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2040c0a50 con 0x7fa2040c6b80 2023-10-30T12:53:11.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.200+0000 7fa20c7ab640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20406f6e0 con 0x7fa2040c6f60 2023-10-30T12:53:11.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.200+0000 7fa20c7ab640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20406f9c0 con 0x7fa2040c05d0 2023-10-30T12:53:11.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.200+0000 7fa20a547640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa2040c6b80 0x7fa2040bfec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37384/0 (socket says 172.21.15.12:37384) 2023-10-30T12:53:11.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.200+0000 7fa20a547640 1 -- 172.21.15.12:0/1373379228 learned_addr learned my addr 172.21.15.12:0/1373379228 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:53:11.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209d46640 1 --2- 172.21.15.12:0/1373379228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2040c05d0 0x7fa20413ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126775149 0 0) 0x7fa2040c0a50 con 0x7fa2040c6b80 2023-10-30T12:53:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1f0005740 con 0x7fa2040c6b80 2023-10-30T12:53:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727782568 0 0) 0x7fa20406f6e0 con 0x7fa2040c6f60 2023-10-30T12:53:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2040c0a50 con 0x7fa2040c6f60 2023-10-30T12:53:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4280969632 0 0) 0x7fa1f0005740 con 0x7fa2040c6b80 2023-10-30T12:53:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2040c05d0 msgr2=0x7fa20413ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 --2- 172.21.15.12:0/1373379228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2040c05d0 0x7fa20413ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 >> v1:172.21.15.154:6789/0 conn(0x7fa2040c6f60 legacy=0x7fa2041566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20406fca0 con 0x7fa2040c6b80 2023-10-30T12:53:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fa200001680 con 0x7fa2040c6b80 2023-10-30T12:53:11.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 >> v1:172.21.15.12:6789/0 conn(0x7fa2040c6b80 legacy=0x7fa2040bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:11.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 --2- 172.21.15.12:0/1373379228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1f0005d50 0x7fa1f0006150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:11.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 --2- 172.21.15.12:0/1373379228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2040c05d0 0x7fa1f00069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:11.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 --2- 172.21.15.12:0/1373379228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f0006f20 0x7fa1f00073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:11.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20406f9c0 con 0x7fa1f0005d50 2023-10-30T12:53:11.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f000dc50 con 0x7fa1f0006f20 2023-10-30T12:53:11.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f000de10 con 0x7fa2040c05d0 2023-10-30T12:53:11.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa20a547640 1 --2- 172.21.15.12:0/1373379228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1f0005d50 0x7fa1f0006150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:11.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa209d46640 1 --2- 172.21.15.12:0/1373379228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2040c05d0 0x7fa1f00069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:11.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa20ad48640 1 --2- 172.21.15.12:0/1373379228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f0006f20 0x7fa1f00073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:11.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa20a547640 1 -- 172.21.15.12:0/1373379228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2040c05d0 msgr2=0x7fa1f00069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:11.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa20a547640 1 --2- 172.21.15.12:0/1373379228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2040c05d0 0x7fa1f00069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:11.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa20a547640 1 -- 172.21.15.12:0/1373379228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f0006f20 msgr2=0x7fa1f00073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:11.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa20a547640 1 --2- 172.21.15.12:0/1373379228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f0006f20 0x7fa1f00073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:11.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa20a547640 1 -- 172.21.15.12:0/1373379228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f0005740 con 0x7fa1f0005d50 2023-10-30T12:53:11.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.204+0000 7fa20ad48640 1 --2- 172.21.15.12:0/1373379228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f0006f20 0x7fa1f00073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:53:11.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.208+0000 7fa20a547640 1 --2- 172.21.15.12:0/1373379228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1f0005d50 0x7fa1f0006150 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa20000c160 tx=0x7fa20001cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4101d22eb946d12 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:11.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.208+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2000058e0 con 0x7fa1f0005d50 2023-10-30T12:53:11.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.208+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa200005a60 con 0x7fa1f0005d50 2023-10-30T12:53:11.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.208+0000 7fa209545640 1 -- 172.21.15.12:0/1373379228 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200005e10 con 0x7fa1f0005d50 2023-10-30T12:53:11.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.208+0000 7fa20c7ab640 1 -- 172.21.15.12:0/1373379228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1f0005d50 msgr2=0x7fa1f0006150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:11.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.208+0000 7fa20c7ab640 1 --2- 172.21.15.12:0/1373379228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1f0005d50 0x7fa1f0006150 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa20000c160 tx=0x7fa20001cbe0 comp rx=0 tx=0).stop 2023-10-30T12:53:11.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.208+0000 7fa20ad48640 1 -- 172.21.15.12:0/1373379228 reap_dead start 2023-10-30T12:53:11.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.208+0000 7fa20c7ab640 1 -- 172.21.15.12:0/1373379228 shutdown_connections 2023-10-30T12:53:11.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.208+0000 7fa20c7ab640 1 -- 172.21.15.12:0/1373379228 >> 172.21.15.12:0/1373379228 conn(0x7fa2040c7910 msgr2=0x7fa2040c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:11.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.208+0000 7fa20c7ab640 1 -- 172.21.15.12:0/1373379228 shutdown_connections 2023-10-30T12:53:11.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.208+0000 7fa20c7ab640 1 -- 172.21.15.12:0/1373379228 wait complete. 2023-10-30T12:53:11.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.208+0000 7fa20c7ab640 1 Processor -- start 2023-10-30T12:53:11.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.208+0000 7fa20c7ab640 1 -- start start 2023-10-30T12:53:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.208+0000 7fa20c7ab640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1f0005d50 0x7fa2041544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.208+0000 7fa20c7ab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa204156460 0x7fa2041f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20c7ab640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2041e5be0 0x7fa2040c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20c7ab640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2041e63c0 con 0x7fa1f0005d50 2023-10-30T12:53:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20c7ab640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2041e4ef0 con 0x7fa204156460 2023-10-30T12:53:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20c7ab640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2041f32c0 con 0x7fa2041e5be0 2023-10-30T12:53:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20a547640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1f0005d50 0x7fa2041544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20a547640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1f0005d50 0x7fa2041544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57426/0 (socket says 172.21.15.12:57426) 2023-10-30T12:53:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20a547640 1 -- 172.21.15.12:0/2820291731 learned_addr learned my addr 172.21.15.12:0/2820291731 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa209d46640 1 --2- 172.21.15.12:0/2820291731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa204156460 0x7fa2041f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:11.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20ad48640 1 --2- 172.21.15.12:0/2820291731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2041e5be0 0x7fa2040c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:11.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20a547640 1 -- 172.21.15.12:0/2820291731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2041e5be0 msgr2=0x7fa2040c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:11.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20a547640 1 --2- 172.21.15.12:0/2820291731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2041e5be0 0x7fa2040c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:11.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20a547640 1 -- 172.21.15.12:0/2820291731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa204156460 msgr2=0x7fa2041f2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:11.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20a547640 1 --2- 172.21.15.12:0/2820291731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa204156460 0x7fa2041f2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:11.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20a547640 1 -- 172.21.15.12:0/2820291731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2041e5750 con 0x7fa1f0005d50 2023-10-30T12:53:11.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20a547640 1 --2- 172.21.15.12:0/2820291731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1f0005d50 0x7fa2041544a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa2000ab040 tx=0x7fa1f0007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:11.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa1faffd640 1 -- 172.21.15.12:0/2820291731 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200001be0 con 0x7fa1f0005d50 2023-10-30T12:53:11.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa1faffd640 1 -- 172.21.15.12:0/2820291731 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa200005df0 con 0x7fa1f0005d50 2023-10-30T12:53:11.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa1faffd640 1 -- 172.21.15.12:0/2820291731 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200001680 con 0x7fa1f0005d50 2023-10-30T12:53:11.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20c7ab640 1 -- 172.21.15.12:0/2820291731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2041e4120 con 0x7fa1f0005d50 2023-10-30T12:53:11.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20c7ab640 1 -- 172.21.15.12:0/2820291731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2041e5100 con 0x7fa1f0005d50 2023-10-30T12:53:11.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa20c7ab640 1 -- 172.21.15.12:0/2820291731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20404e170 con 0x7fa1f0005d50 2023-10-30T12:53:11.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.212+0000 7fa1faffd640 1 -- 172.21.15.12:0/2820291731 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa2000a57d0 con 0x7fa1f0005d50 2023-10-30T12:53:11.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.216+0000 7fa1faffd640 1 --2- 172.21.15.12:0/2820291731 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa1e80664d0 0x7fa1e8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:11.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.216+0000 7fa1faffd640 1 -- 172.21.15.12:0/2820291731 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7fa20011d9e0 con 0x7fa1f0005d50 2023-10-30T12:53:11.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.216+0000 7fa209d46640 1 --2- 172.21.15.12:0/2820291731 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa1e80664d0 0x7fa1e8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:11.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.216+0000 7fa209d46640 1 --2- 172.21.15.12:0/2820291731 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa1e80664d0 0x7fa1e8068990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa2041e3e50 tx=0x7fa1f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:11.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.220+0000 7fa1faffd640 1 -- 172.21.15.12:0/2820291731 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa2000eaa00 con 0x7fa1f0005d50 2023-10-30T12:53:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.408+0000 7fa20c7ab640 1 -- 172.21.15.12:0/2820291731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa204071cc0 con 0x7fa1f0005d50 2023-10-30T12:53:11.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.408+0000 7fa1faffd640 1 -- 172.21.15.12:0/2820291731 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6537 (secure 0 0 0) 0x7fa2000ee020 con 0x7fa1f0005d50 2023-10-30T12:53:11.417 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:11.418 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6021,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:52:24.610147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:11.420 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 88 2023-10-30T12:53:11.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.408+0000 7fa20c7ab640 1 -- 172.21.15.12:0/2820291731 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa1e80664d0 msgr2=0x7fa1e8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:11.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.408+0000 7fa20c7ab640 1 --2- 172.21.15.12:0/2820291731 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa1e80664d0 0x7fa1e8068990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa2041e3e50 tx=0x7fa1f4008040 comp rx=0 tx=0).stop 2023-10-30T12:53:11.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.408+0000 7fa20c7ab640 1 -- 172.21.15.12:0/2820291731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1f0005d50 msgr2=0x7fa2041544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:11.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.408+0000 7fa20c7ab640 1 --2- 172.21.15.12:0/2820291731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1f0005d50 0x7fa2041544a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa2000ab040 tx=0x7fa1f0007a10 comp rx=0 tx=0).stop 2023-10-30T12:53:11.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.408+0000 7fa20c7ab640 1 -- 172.21.15.12:0/2820291731 shutdown_connections 2023-10-30T12:53:11.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.408+0000 7fa20c7ab640 1 --2- 172.21.15.12:0/2820291731 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa1e80664d0 0x7fa1e8068990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:11.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.408+0000 7fa20c7ab640 1 --2- 172.21.15.12:0/2820291731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2041e5be0 0x7fa2040c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:11.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.408+0000 7fa20c7ab640 1 --2- 172.21.15.12:0/2820291731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa204156460 0x7fa2041f2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:11.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.408+0000 7fa20c7ab640 1 --2- 172.21.15.12:0/2820291731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1f0005d50 0x7fa2041544a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:11.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.408+0000 7fa20c7ab640 1 -- 172.21.15.12:0/2820291731 >> 172.21.15.12:0/2820291731 conn(0x7fa2040c7910 msgr2=0x7fa2040c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:11.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.408+0000 7fa20c7ab640 1 -- 172.21.15.12:0/2820291731 shutdown_connections 2023-10-30T12:53:11.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:11.408+0000 7fa20c7ab640 1 -- 172.21.15.12:0/2820291731 wait complete. 2023-10-30T12:53:11.438 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:53:13.439 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:13.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.588+0000 7f7dac11d640 1 Processor -- start 2023-10-30T12:53:13.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.588+0000 7f7dac11d640 1 -- start start 2023-10-30T12:53:13.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.588+0000 7f7dac11d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7da4156f60 0x7f7da4071b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:13.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.588+0000 7f7dac11d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da4157340 con 0x7f7da406fd50 2023-10-30T12:53:13.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.588+0000 7f7dac11d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da406ca30 con 0x7f7da415d520 2023-10-30T12:53:13.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.588+0000 7f7dac11d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da406bea0 con 0x7f7da4156f60 2023-10-30T12:53:13.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.588+0000 7f7da96b8640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f7da406fd50 0x7f7da415ce10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37396/0 (socket says 172.21.15.12:37396) 2023-10-30T12:53:13.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.588+0000 7f7da96b8640 1 -- 172.21.15.12:0/3637584587 learned_addr learned my addr 172.21.15.12:0/3637584587 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:53:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da9eb9640 1 --2- 172.21.15.12:0/3637584587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7da4156f60 0x7f7da4071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933455298 0 0) 0x7f7da4157340 con 0x7f7da406fd50 2023-10-30T12:53:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d90003530 con 0x7f7da406fd50 2023-10-30T12:53:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339391952 0 0) 0x7f7da406ca30 con 0x7f7da415d520 2023-10-30T12:53:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7da4157340 con 0x7f7da415d520 2023-10-30T12:53:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3320855174 0 0) 0x7f7d90003530 con 0x7f7da406fd50 2023-10-30T12:53:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7da4156f60 msgr2=0x7f7da4071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 --2- 172.21.15.12:0/3637584587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7da4156f60 0x7f7da4071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 >> v1:172.21.15.154:6789/0 conn(0x7f7da415d520 legacy=0x7f7da4164160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da406cf80 con 0x7f7da406fd50 2023-10-30T12:53:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f7d94001680 con 0x7f7da406fd50 2023-10-30T12:53:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 >> v1:172.21.15.12:6789/0 conn(0x7f7da406fd50 legacy=0x7f7da415ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 --2- 172.21.15.12:0/3637584587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d90003b40 0x7f7d90003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 --2- 172.21.15.12:0/3637584587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d900047d0 0x7f7d90004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 --2- 172.21.15.12:0/3637584587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7da4156f60 0x7f7d90005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da406bea0 con 0x7f7d90003b40 2023-10-30T12:53:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9000ba40 con 0x7f7d900047d0 2023-10-30T12:53:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9000bbd0 con 0x7f7da4156f60 2023-10-30T12:53:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da9eb9640 1 --2- 172.21.15.12:0/3637584587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d90003b40 0x7f7d90003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7daa6ba640 1 --2- 172.21.15.12:0/3637584587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7da4156f60 0x7f7d90005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da96b8640 1 --2- 172.21.15.12:0/3637584587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d900047d0 0x7f7d90004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da9eb9640 1 -- 172.21.15.12:0/3637584587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7da4156f60 msgr2=0x7f7d90005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da9eb9640 1 --2- 172.21.15.12:0/3637584587 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7da4156f60 0x7f7d90005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da9eb9640 1 -- 172.21.15.12:0/3637584587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d900047d0 msgr2=0x7f7d90004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da9eb9640 1 --2- 172.21.15.12:0/3637584587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d900047d0 0x7f7d90004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.592+0000 7f7da9eb9640 1 -- 172.21.15.12:0/3637584587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d90003530 con 0x7f7d90003b40 2023-10-30T12:53:13.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.596+0000 7f7da9eb9640 1 --2- 172.21.15.12:0/3637584587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d90003b40 0x7f7d90003f40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009ae0 tx=0x7f7da001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62246dd682be306b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:13.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.596+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da00a64c0 con 0x7f7d90003b40 2023-10-30T12:53:13.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.596+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da00a6620 con 0x7f7d90003b40 2023-10-30T12:53:13.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.596+0000 7f7da8eb7640 1 -- 172.21.15.12:0/3637584587 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da00a69d0 con 0x7f7d90003b40 2023-10-30T12:53:13.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.596+0000 7f7dac11d640 1 -- 172.21.15.12:0/3637584587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d90003b40 msgr2=0x7f7d90003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:13.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.596+0000 7f7dac11d640 1 --2- 172.21.15.12:0/3637584587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d90003b40 0x7f7d90003f40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009ae0 tx=0x7f7da001c9f0 comp rx=0 tx=0).stop 2023-10-30T12:53:13.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.596+0000 7f7daa6ba640 1 -- 172.21.15.12:0/3637584587 reap_dead start 2023-10-30T12:53:13.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.596+0000 7f7dac11d640 1 -- 172.21.15.12:0/3637584587 shutdown_connections 2023-10-30T12:53:13.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.596+0000 7f7dac11d640 1 -- 172.21.15.12:0/3637584587 >> 172.21.15.12:0/3637584587 conn(0x7f7da4069e90 msgr2=0x7f7da406a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:13.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.596+0000 7f7dac11d640 1 -- 172.21.15.12:0/3637584587 shutdown_connections 2023-10-30T12:53:13.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.596+0000 7f7dac11d640 1 -- 172.21.15.12:0/3637584587 wait complete. 2023-10-30T12:53:13.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.596+0000 7f7dac11d640 1 Processor -- start 2023-10-30T12:53:13.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.596+0000 7f7dac11d640 1 -- start start 2023-10-30T12:53:13.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7dac11d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d90003b40 0x7f7da40c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:13.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7dac11d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7da40c55b0 0x7f7da40cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:13.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7dac11d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7da40c4310 0x7f7da40ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:13.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7dac11d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da40cc3e0 con 0x7f7d90003b40 2023-10-30T12:53:13.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7dac11d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da40c6d30 con 0x7f7da40c4310 2023-10-30T12:53:13.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7dac11d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da4154080 con 0x7f7da40c55b0 2023-10-30T12:53:13.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7da9eb9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d90003b40 0x7f7da40c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:13.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7da9eb9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d90003b40 0x7f7da40c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57448/0 (socket says 172.21.15.12:57448) 2023-10-30T12:53:13.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7da9eb9640 1 -- 172.21.15.12:0/1154792253 learned_addr learned my addr 172.21.15.12:0/1154792253 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:13.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7daa6ba640 1 --2- 172.21.15.12:0/1154792253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7da40c4310 0x7f7da40ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:13.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7da96b8640 1 --2- 172.21.15.12:0/1154792253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7da40c55b0 0x7f7da40cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:13.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7da9eb9640 1 -- 172.21.15.12:0/1154792253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7da40c55b0 msgr2=0x7f7da40cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:13.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7da9eb9640 1 --2- 172.21.15.12:0/1154792253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7da40c55b0 0x7f7da40cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:13.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7da9eb9640 1 -- 172.21.15.12:0/1154792253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7da40c4310 msgr2=0x7f7da40ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:13.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7da9eb9640 1 --2- 172.21.15.12:0/1154792253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7da40c4310 0x7f7da40ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:13.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7da9eb9640 1 -- 172.21.15.12:0/1154792253 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da41f5f00 con 0x7f7d90003b40 2023-10-30T12:53:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7da9eb9640 1 --2- 172.21.15.12:0/1154792253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d90003b40 0x7f7da40c5ec0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7da001c820 tx=0x7f7d90005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7d9a7fc640 1 -- 172.21.15.12:0/1154792253 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da00a6990 con 0x7f7d90003b40 2023-10-30T12:53:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7d9a7fc640 1 -- 172.21.15.12:0/1154792253 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da00a6b10 con 0x7f7d90003b40 2023-10-30T12:53:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7d9a7fc640 1 -- 172.21.15.12:0/1154792253 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da000f0c0 con 0x7f7d90003b40 2023-10-30T12:53:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7dac11d640 1 -- 172.21.15.12:0/1154792253 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da41f61e0 con 0x7f7d90003b40 2023-10-30T12:53:13.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7dac11d640 1 -- 172.21.15.12:0/1154792253 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da40c7260 con 0x7f7d90003b40 2023-10-30T12:53:13.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7dac11d640 1 -- 172.21.15.12:0/1154792253 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da40c7450 con 0x7f7d90003b40 2023-10-30T12:53:13.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.600+0000 7f7d9a7fc640 1 -- 172.21.15.12:0/1154792253 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7da0006040 con 0x7f7d90003b40 2023-10-30T12:53:13.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.604+0000 7f7d9a7fc640 1 --2- 172.21.15.12:0/1154792253 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7d880664a0 0x7f7d88068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:13.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.604+0000 7f7d9a7fc640 1 -- 172.21.15.12:0/1154792253 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f7da00aa070 con 0x7f7d90003b40 2023-10-30T12:53:13.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.604+0000 7f7da96b8640 1 --2- 172.21.15.12:0/1154792253 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7d880664a0 0x7f7d88068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:13.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.604+0000 7f7da96b8640 1 --2- 172.21.15.12:0/1154792253 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7d880664a0 0x7f7d88068960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7da41f5e30 tx=0x7f7d94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:13.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.608+0000 7f7d9a7fc640 1 -- 172.21.15.12:0/1154792253 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7da00ec230 con 0x7f7d90003b40 2023-10-30T12:53:13.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.804+0000 7f7dac11d640 1 -- 172.21.15.12:0/1154792253 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7da404e170 con 0x7f7d90003b40 2023-10-30T12:53:13.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.804+0000 7f7d9a7fc640 1 -- 172.21.15.12:0/1154792253 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6537 (secure 0 0 0) 0x7f7da00ef850 con 0x7f7d90003b40 2023-10-30T12:53:13.813 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:13.813 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6021,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:52:24.610147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:13.813 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 88 2023-10-30T12:53:13.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.808+0000 7f7dac11d640 1 -- 172.21.15.12:0/1154792253 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7d880664a0 msgr2=0x7f7d88068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:13.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.808+0000 7f7dac11d640 1 --2- 172.21.15.12:0/1154792253 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7d880664a0 0x7f7d88068960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7da41f5e30 tx=0x7f7d94009040 comp rx=0 tx=0).stop 2023-10-30T12:53:13.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.808+0000 7f7dac11d640 1 -- 172.21.15.12:0/1154792253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d90003b40 msgr2=0x7f7da40c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:13.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.808+0000 7f7dac11d640 1 --2- 172.21.15.12:0/1154792253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d90003b40 0x7f7da40c5ec0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7da001c820 tx=0x7f7d90005800 comp rx=0 tx=0).stop 2023-10-30T12:53:13.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.808+0000 7f7dac11d640 1 -- 172.21.15.12:0/1154792253 shutdown_connections 2023-10-30T12:53:13.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.808+0000 7f7dac11d640 1 --2- 172.21.15.12:0/1154792253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7da40c4310 0x7f7da40ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:13.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.808+0000 7f7dac11d640 1 --2- 172.21.15.12:0/1154792253 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7d880664a0 0x7f7d88068960 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:13.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.808+0000 7f7dac11d640 1 --2- 172.21.15.12:0/1154792253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7da40c55b0 0x7f7da40cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:13.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.808+0000 7f7dac11d640 1 --2- 172.21.15.12:0/1154792253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d90003b40 0x7f7da40c5ec0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:13.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.808+0000 7f7dac11d640 1 -- 172.21.15.12:0/1154792253 >> 172.21.15.12:0/1154792253 conn(0x7f7da4069e90 msgr2=0x7f7da40c7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:13.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.808+0000 7f7dac11d640 1 -- 172.21.15.12:0/1154792253 shutdown_connections 2023-10-30T12:53:13.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:13.808+0000 7f7dac11d640 1 -- 172.21.15.12:0/1154792253 wait complete. 2023-10-30T12:53:13.834 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:53:15.835 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:15.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8edd165640 1 Processor -- start 2023-10-30T12:53:15.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8edd165640 1 -- start start 2023-10-30T12:53:15.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8edd165640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8ed813f9c0 0x7f8ed80c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:15.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8edd165640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed81566e0 con 0x7f8ed815a290 2023-10-30T12:53:15.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8edd165640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed81569c0 con 0x7f8ed80c5f70 2023-10-30T12:53:15.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8edd165640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed8157360 con 0x7f8ed813f9c0 2023-10-30T12:53:15.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8ed6d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8ed813f9c0 0x7f8ed80c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:15.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8ed6d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8ed813f9c0 0x7f8ed80c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57464/0 (socket says 172.21.15.12:57464) 2023-10-30T12:53:15.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8ed6d9d640 1 -- 172.21.15.12:0/2632187781 learned_addr learned my addr 172.21.15.12:0/2632187781 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:15.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8ed759e640 1 --1- 172.21.15.12:0/2632187781 >> v1:172.21.15.12:6789/0 conn(0x7f8ed815a290 0x7f8ed806b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37412/0 (socket says 172.21.15.12:37412) 2023-10-30T12:53:15.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8ed6d9d640 1 -- 172.21.15.12:0/2632187781 >> v1:172.21.15.154:6789/0 conn(0x7f8ed80c5f70 legacy=0x7f8ed8159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:15.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8ed6d9d640 1 -- 172.21.15.12:0/2632187781 >> v1:172.21.15.12:6789/0 conn(0x7f8ed815a290 legacy=0x7f8ed806b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:15.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8ed6d9d640 1 -- 172.21.15.12:0/2632187781 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed8156ca0 con 0x7f8ed813f9c0 2023-10-30T12:53:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8ed659c640 1 -- 172.21.15.12:0/2632187781 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053988618 0 0) 0x7f8ed81566e0 con 0x7f8ed815a290 2023-10-30T12:53:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8ed6d9d640 1 --2- 172.21.15.12:0/2632187781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ed813f9c0 0x7f8ed80c5a30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009870 tx=0x7f8ec801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d794f1cd8aa306 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8ed659c640 1 -- 172.21.15.12:0/2632187781 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80ab600 con 0x7f8ed813f9c0 2023-10-30T12:53:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8ed659c640 1 -- 172.21.15.12:0/2632187781 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec80a7050 con 0x7f8ed813f9c0 2023-10-30T12:53:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8ed659c640 1 -- 172.21.15.12:0/2632187781 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80ab980 con 0x7f8ed813f9c0 2023-10-30T12:53:15.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8edd165640 1 -- 172.21.15.12:0/2632187781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ed813f9c0 msgr2=0x7f8ed80c5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:15.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8edd165640 1 --2- 172.21.15.12:0/2632187781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ed813f9c0 0x7f8ed80c5a30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009870 tx=0x7f8ec801c540 comp rx=0 tx=0).stop 2023-10-30T12:53:15.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8edd165640 1 -- 172.21.15.12:0/2632187781 shutdown_connections 2023-10-30T12:53:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8edd165640 1 --2- 172.21.15.12:0/2632187781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ed813f9c0 0x7f8ed80c5a30 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8edd165640 1 -- 172.21.15.12:0/2632187781 >> 172.21.15.12:0/2632187781 conn(0x7f8ed80cace0 msgr2=0x7f8ed80cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8edd165640 1 -- 172.21.15.12:0/2632187781 shutdown_connections 2023-10-30T12:53:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.988+0000 7f8edd165640 1 -- 172.21.15.12:0/2632187781 wait complete. 2023-10-30T12:53:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8edd165640 1 Processor -- start 2023-10-30T12:53:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8edd165640 1 -- start start 2023-10-30T12:53:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8edd165640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed80c5f70 0x7f8ed815f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8edd165640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ed813f9c0 0x7f8ed815fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8edd165640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ed815a290 0x7f8ed8155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8edd165640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed8154830 con 0x7f8ed815a290 2023-10-30T12:53:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8edd165640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed813b700 con 0x7f8ed80c5f70 2023-10-30T12:53:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8edd165640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed80c7720 con 0x7f8ed813f9c0 2023-10-30T12:53:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8ed759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ed815a290 0x7f8ed8155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8ed759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ed815a290 0x7f8ed8155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57478/0 (socket says 172.21.15.12:57478) 2023-10-30T12:53:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8ed759e640 1 -- 172.21.15.12:0/157879978 learned_addr learned my addr 172.21.15.12:0/157879978 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8ed6d9d640 1 --2- 172.21.15.12:0/157879978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed80c5f70 0x7f8ed815f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8ecffff640 1 --2- 172.21.15.12:0/157879978 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ed813f9c0 0x7f8ed815fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8ed759e640 1 -- 172.21.15.12:0/157879978 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ed813f9c0 msgr2=0x7f8ed815fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8ed759e640 1 --2- 172.21.15.12:0/157879978 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ed813f9c0 0x7f8ed815fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8ed759e640 1 -- 172.21.15.12:0/157879978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed80c5f70 msgr2=0x7f8ed815f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8ed759e640 1 --2- 172.21.15.12:0/157879978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed80c5f70 0x7f8ed815f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8ed759e640 1 -- 172.21.15.12:0/157879978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec80094d0 con 0x7f8ed815a290 2023-10-30T12:53:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8ed759e640 1 --2- 172.21.15.12:0/157879978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ed815a290 0x7f8ed8155050 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000c7c0 tx=0x7f8ed000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:16.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8ecf7fe640 1 -- 172.21.15.12:0/157879978 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed000ce70 con 0x7f8ed815a290 2023-10-30T12:53:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8ecf7fe640 1 -- 172.21.15.12:0/157879978 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed000a5e0 con 0x7f8ed815a290 2023-10-30T12:53:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8edd165640 1 -- 172.21.15.12:0/157879978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed80c7ca0 con 0x7f8ed815a290 2023-10-30T12:53:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8ecf7fe640 1 -- 172.21.15.12:0/157879978 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed000a9b0 con 0x7f8ed815a290 2023-10-30T12:53:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8edd165640 1 -- 172.21.15.12:0/157879978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed815f080 con 0x7f8ed815a290 2023-10-30T12:53:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.992+0000 7f8edd165640 1 -- 172.21.15.12:0/157879978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed804e170 con 0x7f8ed815a290 2023-10-30T12:53:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.996+0000 7f8ecf7fe640 1 -- 172.21.15.12:0/157879978 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8ed000ab10 con 0x7f8ed815a290 2023-10-30T12:53:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.996+0000 7f8ecf7fe640 1 --2- 172.21.15.12:0/157879978 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8eb80664d0 0x7f8eb8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.996+0000 7f8ecf7fe640 1 -- 172.21.15.12:0/157879978 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f8ed0083990 con 0x7f8ed815a290 2023-10-30T12:53:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.996+0000 7f8ed6d9d640 1 --2- 172.21.15.12:0/157879978 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8eb80664d0 0x7f8eb8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:15.996+0000 7f8ed6d9d640 1 --2- 172.21.15.12:0/157879978 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8eb80664d0 0x7f8eb8068990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80cba00 tx=0x7f8ec8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:16.004+0000 7f8ecf7fe640 1 -- 172.21.15.12:0/157879978 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8ed0050a60 con 0x7f8ed815a290 2023-10-30T12:53:16.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:16.196+0000 7f8edd165640 1 -- 172.21.15.12:0/157879978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ed80c5810 con 0x7f8ed815a290 2023-10-30T12:53:16.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:16.196+0000 7f8ecf7fe640 1 -- 172.21.15.12:0/157879978 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6537 (secure 0 0 0) 0x7f8ed0018020 con 0x7f8ed815a290 2023-10-30T12:53:16.203 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:16.203 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6021,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:52:24.610147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:16.203 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 88 2023-10-30T12:53:16.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:16.196+0000 7f8edd165640 1 -- 172.21.15.12:0/157879978 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8eb80664d0 msgr2=0x7f8eb8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:16.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:16.196+0000 7f8edd165640 1 --2- 172.21.15.12:0/157879978 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8eb80664d0 0x7f8eb8068990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80cba00 tx=0x7f8ec8008040 comp rx=0 tx=0).stop 2023-10-30T12:53:16.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:16.196+0000 7f8edd165640 1 -- 172.21.15.12:0/157879978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ed815a290 msgr2=0x7f8ed8155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:16.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:16.196+0000 7f8edd165640 1 --2- 172.21.15.12:0/157879978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ed815a290 0x7f8ed8155050 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000c7c0 tx=0x7f8ed000cc80 comp rx=0 tx=0).stop 2023-10-30T12:53:16.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:16.196+0000 7f8edd165640 1 -- 172.21.15.12:0/157879978 shutdown_connections 2023-10-30T12:53:16.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:16.196+0000 7f8edd165640 1 --2- 172.21.15.12:0/157879978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ed815a290 0x7f8ed8155050 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:16.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:16.196+0000 7f8edd165640 1 --2- 172.21.15.12:0/157879978 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f8eb80664d0 0x7f8eb8068990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:16.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:16.196+0000 7f8edd165640 1 --2- 172.21.15.12:0/157879978 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ed813f9c0 0x7f8ed815fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:16.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:16.196+0000 7f8edd165640 1 --2- 172.21.15.12:0/157879978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed80c5f70 0x7f8ed815f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:16.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:16.196+0000 7f8edd165640 1 -- 172.21.15.12:0/157879978 >> 172.21.15.12:0/157879978 conn(0x7f8ed80cace0 msgr2=0x7f8ed81614f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:16.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:16.196+0000 7f8edd165640 1 -- 172.21.15.12:0/157879978 shutdown_connections 2023-10-30T12:53:16.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:16.196+0000 7f8edd165640 1 -- 172.21.15.12:0/157879978 wait complete. 2023-10-30T12:53:16.224 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:53:18.226 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:18.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.376+0000 7fb2cc194640 1 Processor -- start 2023-10-30T12:53:18.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.376+0000 7fb2cc194640 1 -- start start 2023-10-30T12:53:18.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.376+0000 7fb2cc194640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb2c415d6f0 0x7fb2c415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:18.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.376+0000 7fb2cc194640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c40cb800 con 0x7fb2c40cf160 2023-10-30T12:53:18.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.376+0000 7fb2cc194640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c40cb330 con 0x7fb2c41538a0 2023-10-30T12:53:18.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.376+0000 7fb2cc194640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c40cad70 con 0x7fb2c415d6f0 2023-10-30T12:53:18.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.376+0000 7fb2c972f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb2c415d6f0 0x7fb2c415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:18.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.376+0000 7fb2c972f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb2c415d6f0 0x7fb2c415a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57486/0 (socket says 172.21.15.12:57486) 2023-10-30T12:53:18.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.376+0000 7fb2c972f640 1 -- 172.21.15.12:0/14853342 learned_addr learned my addr 172.21.15.12:0/14853342 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:18.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.376+0000 7fb2c972f640 1 -- 172.21.15.12:0/14853342 >> v1:172.21.15.154:6789/0 conn(0x7fb2c41538a0 legacy=0x7fb2c40ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:18.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.376+0000 7fb2c972f640 1 -- 172.21.15.12:0/14853342 >> v1:172.21.15.12:6789/0 conn(0x7fb2c40cf160 legacy=0x7fb2c4164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:18.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.376+0000 7fb2c972f640 1 -- 172.21.15.12:0/14853342 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c40cb050 con 0x7fb2c415d6f0 2023-10-30T12:53:18.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.380+0000 7fb2c972f640 1 --2- 172.21.15.12:0/14853342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2c415d6f0 0x7fb2c415a090 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009870 tx=0x7fb2b401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bc57d19762e00f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:18.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.380+0000 7fb2c8f2e640 1 -- 172.21.15.12:0/14853342 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40a8a30 con 0x7fb2c415d6f0 2023-10-30T12:53:18.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.380+0000 7fb2c8f2e640 1 -- 172.21.15.12:0/14853342 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b40a8b90 con 0x7fb2c415d6f0 2023-10-30T12:53:18.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.380+0000 7fb2c8f2e640 1 -- 172.21.15.12:0/14853342 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40b0440 con 0x7fb2c415d6f0 2023-10-30T12:53:18.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.380+0000 7fb2cc194640 1 -- 172.21.15.12:0/14853342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2c415d6f0 msgr2=0x7fb2c415a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:18.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.380+0000 7fb2cc194640 1 --2- 172.21.15.12:0/14853342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2c415d6f0 0x7fb2c415a090 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009870 tx=0x7fb2b401ed30 comp rx=0 tx=0).stop 2023-10-30T12:53:18.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.380+0000 7fb2cc194640 1 -- 172.21.15.12:0/14853342 shutdown_connections 2023-10-30T12:53:18.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.380+0000 7fb2cc194640 1 --2- 172.21.15.12:0/14853342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2c415d6f0 0x7fb2c415a090 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:18.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.380+0000 7fb2cc194640 1 -- 172.21.15.12:0/14853342 >> 172.21.15.12:0/14853342 conn(0x7fb2c406bfe0 msgr2=0x7fb2c406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:18.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.380+0000 7fb2cc194640 1 -- 172.21.15.12:0/14853342 shutdown_connections 2023-10-30T12:53:18.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.380+0000 7fb2cc194640 1 -- 172.21.15.12:0/14853342 wait complete. 2023-10-30T12:53:18.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2cc194640 1 Processor -- start 2023-10-30T12:53:18.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2cc194640 1 -- start start 2023-10-30T12:53:18.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2cc194640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2c40cf160 0x7fb2c41e3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:18.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2cc194640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2c41538a0 0x7fb2c41592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:18.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2cc194640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2c415d6f0 0x7fb2c40c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:18.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2cc194640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c40c8ce0 con 0x7fb2c415d6f0 2023-10-30T12:53:18.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2cc194640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c40c8310 con 0x7fb2c41538a0 2023-10-30T12:53:18.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2cc194640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c40c7060 con 0x7fb2c40cf160 2023-10-30T12:53:18.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2ca731640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2c415d6f0 0x7fb2c40c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:18.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2ca731640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2c415d6f0 0x7fb2c40c87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57488/0 (socket says 172.21.15.12:57488) 2023-10-30T12:53:18.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2ca731640 1 -- 172.21.15.12:0/1003165040 learned_addr learned my addr 172.21.15.12:0/1003165040 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:18.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2c972f640 1 --2- 172.21.15.12:0/1003165040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2c41538a0 0x7fb2c41592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:18.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2c9f30640 1 --2- 172.21.15.12:0/1003165040 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2c40cf160 0x7fb2c41e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:18.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2ca731640 1 -- 172.21.15.12:0/1003165040 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2c40cf160 msgr2=0x7fb2c41e3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:18.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2ca731640 1 --2- 172.21.15.12:0/1003165040 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2c40cf160 0x7fb2c41e3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:18.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2ca731640 1 -- 172.21.15.12:0/1003165040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2c41538a0 msgr2=0x7fb2c41592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:18.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2ca731640 1 --2- 172.21.15.12:0/1003165040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2c41538a0 0x7fb2c41592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:18.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2ca731640 1 -- 172.21.15.12:0/1003165040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b40094d0 con 0x7fb2c415d6f0 2023-10-30T12:53:18.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2ca731640 1 --2- 172.21.15.12:0/1003165040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2c415d6f0 0x7fb2c40c87a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00c730 tx=0x7fb2bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:18.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2ba7fc640 1 -- 172.21.15.12:0/1003165040 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc003b80 con 0x7fb2c415d6f0 2023-10-30T12:53:18.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2ba7fc640 1 -- 172.21.15.12:0/1003165040 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc003ce0 con 0x7fb2c415d6f0 2023-10-30T12:53:18.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2ba7fc640 1 -- 172.21.15.12:0/1003165040 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc005310 con 0x7fb2c415d6f0 2023-10-30T12:53:18.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2cc194640 1 -- 172.21.15.12:0/1003165040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c40c7d00 con 0x7fb2c415d6f0 2023-10-30T12:53:18.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.384+0000 7fb2cc194640 1 -- 172.21.15.12:0/1003165040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c40c6aa0 con 0x7fb2c415d6f0 2023-10-30T12:53:18.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.388+0000 7fb2cc194640 1 -- 172.21.15.12:0/1003165040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c404e170 con 0x7fb2c415d6f0 2023-10-30T12:53:18.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.388+0000 7fb2ba7fc640 1 -- 172.21.15.12:0/1003165040 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb2bc005680 con 0x7fb2c415d6f0 2023-10-30T12:53:18.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.388+0000 7fb2ba7fc640 1 --2- 172.21.15.12:0/1003165040 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb2a80664a0 0x7fb2a8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:18.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.388+0000 7fb2ba7fc640 1 -- 172.21.15.12:0/1003165040 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6513+0+0 (secure 0 0 0) 0x7fb2bc0840d0 con 0x7fb2c415d6f0 2023-10-30T12:53:18.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.388+0000 7fb2c9f30640 1 --2- 172.21.15.12:0/1003165040 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb2a80664a0 0x7fb2a8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:18.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.388+0000 7fb2c9f30640 1 --2- 172.21.15.12:0/1003165040 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb2a80664a0 0x7fb2a8068960 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0000f70 tx=0x7fb2c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:18.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.392+0000 7fb2ba7fc640 1 -- 172.21.15.12:0/1003165040 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb2bc051220 con 0x7fb2c415d6f0 2023-10-30T12:53:18.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.584+0000 7fb2cc194640 1 -- 172.21.15.12:0/1003165040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2c40c8450 con 0x7fb2c415d6f0 2023-10-30T12:53:18.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.588+0000 7fb2ba7fc640 1 -- 172.21.15.12:0/1003165040 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6537 (secure 0 0 0) 0x7fb2bc054840 con 0x7fb2c415d6f0 2023-10-30T12:53:18.591 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:18.591 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6021,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:52:24.610147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5571,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":19,"addr":"172.21.15.12:6837/2752088514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2752088514},{"type":"v1","addr":"172.21.15.12:6837","nonce":2752088514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:18.591 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 88 2023-10-30T12:53:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.588+0000 7fb2cc194640 1 -- 172.21.15.12:0/1003165040 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb2a80664a0 msgr2=0x7fb2a8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.588+0000 7fb2cc194640 1 --2- 172.21.15.12:0/1003165040 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb2a80664a0 0x7fb2a8068960 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0000f70 tx=0x7fb2c0009040 comp rx=0 tx=0).stop 2023-10-30T12:53:18.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.588+0000 7fb2cc194640 1 -- 172.21.15.12:0/1003165040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2c415d6f0 msgr2=0x7fb2c40c87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:18.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.588+0000 7fb2cc194640 1 --2- 172.21.15.12:0/1003165040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2c415d6f0 0x7fb2c40c87a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00c730 tx=0x7fb2bc00cbf0 comp rx=0 tx=0).stop 2023-10-30T12:53:18.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.588+0000 7fb2cc194640 1 -- 172.21.15.12:0/1003165040 shutdown_connections 2023-10-30T12:53:18.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.588+0000 7fb2cc194640 1 --2- 172.21.15.12:0/1003165040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2c415d6f0 0x7fb2c40c87a0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:18.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.588+0000 7fb2cc194640 1 --2- 172.21.15.12:0/1003165040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2c41538a0 0x7fb2c41592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:18.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.588+0000 7fb2cc194640 1 --2- 172.21.15.12:0/1003165040 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb2a80664a0 0x7fb2a8068960 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:18.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.588+0000 7fb2cc194640 1 --2- 172.21.15.12:0/1003165040 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2c40cf160 0x7fb2c41e3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:18.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.588+0000 7fb2cc194640 1 -- 172.21.15.12:0/1003165040 >> 172.21.15.12:0/1003165040 conn(0x7fb2c406bfe0 msgr2=0x7fb2c40ce1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:18.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.588+0000 7fb2cc194640 1 -- 172.21.15.12:0/1003165040 shutdown_connections 2023-10-30T12:53:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:18.588+0000 7fb2cc194640 1 -- 172.21.15.12:0/1003165040 wait complete. 2023-10-30T12:53:18.611 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:53:20.612 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:20.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.760+0000 7fb45abc5640 1 Processor -- start 2023-10-30T12:53:20.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.760+0000 7fb45abc5640 1 -- start start 2023-10-30T12:53:20.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.764+0000 7fb45abc5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb45406e470 0x7fb45406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:20.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.764+0000 7fb45abc5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb4540c8490 0x7fb4540c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:20.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.764+0000 7fb45abc5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45406afa0 con 0x7fb454153d70 2023-10-30T12:53:20.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.764+0000 7fb45abc5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb45406d780 con 0x7fb45406e470 2023-10-30T12:53:20.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.764+0000 7fb45abc5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb454154150 con 0x7fb4540c8490 2023-10-30T12:53:20.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.764+0000 7fb458961640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb454153d70 0x7fb4540cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37442/0 (socket says 172.21.15.12:37442) 2023-10-30T12:53:20.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.764+0000 7fb458961640 1 -- 172.21.15.12:0/718403037 learned_addr learned my addr 172.21.15.12:0/718403037 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:53:20.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.764+0000 7fb453fff640 1 --2- 172.21.15.12:0/718403037 >> v2:172.21.15.12:3300/0 conn(0x7fb45406e470 0x7fb45406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:20.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.764+0000 7fb459162640 1 --2- 172.21.15.12:0/718403037 >> v2:172.21.15.154:3300/0 conn(0x7fb4540c8490 0x7fb4540c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:20.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.764+0000 7fb453fff640 1 -- 172.21.15.12:0/718403037 >> v2:172.21.15.154:3300/0 conn(0x7fb4540c8490 msgr2=0x7fb4540c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:20.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.764+0000 7fb453fff640 1 --2- 172.21.15.12:0/718403037 >> v2:172.21.15.154:3300/0 conn(0x7fb4540c8490 0x7fb4540c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:20.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.764+0000 7fb453fff640 1 -- 172.21.15.12:0/718403037 >> v1:172.21.15.12:6789/0 conn(0x7fb454153d70 legacy=0x7fb4540cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:20.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.764+0000 7fb453fff640 1 -- 172.21.15.12:0/718403037 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45406e030 con 0x7fb45406e470 2023-10-30T12:53:20.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb453fff640 1 --2- 172.21.15.12:0/718403037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45406e470 0x7fb45406ca80 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb4540cddf0 tx=0x7fb44001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01561c7773a6e0a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:20.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb4537fe640 1 -- 172.21.15.12:0/718403037 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400a7070 con 0x7fb45406e470 2023-10-30T12:53:20.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb4537fe640 1 -- 172.21.15.12:0/718403037 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4400ab430 con 0x7fb45406e470 2023-10-30T12:53:20.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb4537fe640 1 -- 172.21.15.12:0/718403037 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400ab7c0 con 0x7fb45406e470 2023-10-30T12:53:20.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 -- 172.21.15.12:0/718403037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45406e470 msgr2=0x7fb45406ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:20.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 --2- 172.21.15.12:0/718403037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45406e470 0x7fb45406ca80 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb4540cddf0 tx=0x7fb44001c430 comp rx=0 tx=0).stop 2023-10-30T12:53:20.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 -- 172.21.15.12:0/718403037 shutdown_connections 2023-10-30T12:53:20.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 --2- 172.21.15.12:0/718403037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45406e470 0x7fb45406ca80 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:20.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 --2- 172.21.15.12:0/718403037 >> v2:172.21.15.154:3300/0 conn(0x7fb4540c8490 0x7fb4540c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:20.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 -- 172.21.15.12:0/718403037 >> 172.21.15.12:0/718403037 conn(0x7fb454071ea0 msgr2=0x7fb454065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:20.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 -- 172.21.15.12:0/718403037 shutdown_connections 2023-10-30T12:53:20.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 -- 172.21.15.12:0/718403037 wait complete. 2023-10-30T12:53:20.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 Processor -- start 2023-10-30T12:53:20.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 -- start start 2023-10-30T12:53:20.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45406e470 0x7fb4541430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:20.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4540c8490 0x7fb4541f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:20.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb454153d70 0x7fb454150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:20.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb454143600 con 0x7fb45406e470 2023-10-30T12:53:20.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45414f710 con 0x7fb454153d70 2023-10-30T12:53:20.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb45abc5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4541f6710 con 0x7fb4540c8490 2023-10-30T12:53:20.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.768+0000 7fb458961640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45406e470 0x7fb4541430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:20.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb458961640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45406e470 0x7fb4541430c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57506/0 (socket says 172.21.15.12:57506) 2023-10-30T12:53:20.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb458961640 1 -- 172.21.15.12:0/1416911576 learned_addr learned my addr 172.21.15.12:0/1416911576 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:20.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb459162640 1 --2- 172.21.15.12:0/1416911576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb454153d70 0x7fb454150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:20.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb453fff640 1 --2- 172.21.15.12:0/1416911576 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4540c8490 0x7fb4541f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:20.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb458961640 1 -- 172.21.15.12:0/1416911576 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4540c8490 msgr2=0x7fb4541f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:20.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb458961640 1 --2- 172.21.15.12:0/1416911576 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4540c8490 0x7fb4541f3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:20.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb458961640 1 -- 172.21.15.12:0/1416911576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb454153d70 msgr2=0x7fb454150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:20.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb458961640 1 --2- 172.21.15.12:0/1416911576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb454153d70 0x7fb454150a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:20.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb458961640 1 -- 172.21.15.12:0/1416911576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb440009420 con 0x7fb45406e470 2023-10-30T12:53:20.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb459162640 1 --2- 172.21.15.12:0/1416911576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb454153d70 0x7fb454150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:20.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb458961640 1 --2- 172.21.15.12:0/1416911576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45406e470 0x7fb4541430c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb44800c530 tx=0x7fb44800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:20.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb4517fa640 1 -- 172.21.15.12:0/1416911576 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448005cf0 con 0x7fb45406e470 2023-10-30T12:53:20.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb4517fa640 1 -- 172.21.15.12:0/1416911576 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb448005e50 con 0x7fb45406e470 2023-10-30T12:53:20.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb4517fa640 1 -- 172.21.15.12:0/1416911576 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448007270 con 0x7fb45406e470 2023-10-30T12:53:20.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb45abc5640 1 -- 172.21.15.12:0/1416911576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4541f6c40 con 0x7fb45406e470 2023-10-30T12:53:20.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb45abc5640 1 -- 172.21.15.12:0/1416911576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb454148260 con 0x7fb45406e470 2023-10-30T12:53:20.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb45abc5640 1 -- 172.21.15.12:0/1416911576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb454149ba0 con 0x7fb45406e470 2023-10-30T12:53:20.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb4517fa640 1 -- 172.21.15.12:0/1416911576 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb4480073d0 con 0x7fb45406e470 2023-10-30T12:53:20.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb4517fa640 1 --2- 172.21.15.12:0/1416911576 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb4380664a0 0x7fb438068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:20.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.772+0000 7fb4517fa640 1 -- 172.21.15.12:0/1416911576 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fb448083670 con 0x7fb45406e470 2023-10-30T12:53:20.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.776+0000 7fb453fff640 1 --2- 172.21.15.12:0/1416911576 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb4380664a0 0x7fb438068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:20.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.776+0000 7fb453fff640 1 --2- 172.21.15.12:0/1416911576 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb4380664a0 0x7fb438068960 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb4400af000 tx=0x7fb4400a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:20.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.776+0000 7fb4517fa640 1 -- 172.21.15.12:0/1416911576 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb448050810 con 0x7fb45406e470 2023-10-30T12:53:20.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.968+0000 7fb45abc5640 1 -- 172.21.15.12:0/1416911576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb45404e170 con 0x7fb45406e470 2023-10-30T12:53:20.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.968+0000 7fb4517fa640 1 -- 172.21.15.12:0/1416911576 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5742 (secure 0 0 0) 0x7fb448053e30 con 0x7fb45406e470 2023-10-30T12:53:20.972 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:20.972 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:20.432438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:20.972 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 89 2023-10-30T12:53:20.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.968+0000 7fb45abc5640 1 -- 172.21.15.12:0/1416911576 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb4380664a0 msgr2=0x7fb438068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:20.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.968+0000 7fb45abc5640 1 --2- 172.21.15.12:0/1416911576 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb4380664a0 0x7fb438068960 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb4400af000 tx=0x7fb4400a7000 comp rx=0 tx=0).stop 2023-10-30T12:53:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.968+0000 7fb45abc5640 1 -- 172.21.15.12:0/1416911576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45406e470 msgr2=0x7fb4541430c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.968+0000 7fb45abc5640 1 --2- 172.21.15.12:0/1416911576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45406e470 0x7fb4541430c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb44800c530 tx=0x7fb44800c9f0 comp rx=0 tx=0).stop 2023-10-30T12:53:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.968+0000 7fb45abc5640 1 -- 172.21.15.12:0/1416911576 shutdown_connections 2023-10-30T12:53:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.968+0000 7fb45abc5640 1 --2- 172.21.15.12:0/1416911576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb454153d70 0x7fb454150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.968+0000 7fb45abc5640 1 --2- 172.21.15.12:0/1416911576 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb4380664a0 0x7fb438068960 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:20.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.968+0000 7fb45abc5640 1 --2- 172.21.15.12:0/1416911576 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb4540c8490 0x7fb4541f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:20.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.968+0000 7fb45abc5640 1 --2- 172.21.15.12:0/1416911576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb45406e470 0x7fb4541430c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:20.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.968+0000 7fb45abc5640 1 -- 172.21.15.12:0/1416911576 >> 172.21.15.12:0/1416911576 conn(0x7fb454071ea0 msgr2=0x7fb454152bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:20.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.968+0000 7fb45abc5640 1 -- 172.21.15.12:0/1416911576 shutdown_connections 2023-10-30T12:53:20.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:20.968+0000 7fb45abc5640 1 -- 172.21.15.12:0/1416911576 wait complete. 2023-10-30T12:53:20.993 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2023-10-30T12:53:20.993 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:53:20.993 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:21.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.136+0000 7f2cfd49c640 1 Processor -- start 2023-10-30T12:53:21.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.136+0000 7f2cfd49c640 1 -- start start 2023-10-30T12:53:21.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.136+0000 7f2cfd49c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2cf806f7a0 0x7f2cf806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:21.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.136+0000 7f2cfd49c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2cf80c6bd0 0x7f2cf8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:21.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.136+0000 7f2cfd49c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf806a3d0 con 0x7f2cf8071110 2023-10-30T12:53:21.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.136+0000 7f2cfd49c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf80700c0 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.136+0000 7f2cfd49c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf8066cf0 con 0x7f2cf806f7a0 2023-10-30T12:53:21.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.136+0000 7f2cf67fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2cf80c6bd0 0x7f2cf8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:21.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.136+0000 7f2cf67fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2cf80c6bd0 0x7f2cf8069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39552/0 (socket says 172.21.15.12:39552) 2023-10-30T12:53:21.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.136+0000 7f2cf67fc640 1 -- 172.21.15.12:0/3517856486 learned_addr learned my addr 172.21.15.12:0/3517856486 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:21.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.136+0000 7f2cf6ffd640 1 --2- 172.21.15.12:0/3517856486 >> v2:172.21.15.154:3300/0 conn(0x7f2cf806f7a0 0x7f2cf806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:21.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.136+0000 7f2cf67fc640 1 -- 172.21.15.12:0/3517856486 >> v2:172.21.15.154:3300/0 conn(0x7f2cf806f7a0 msgr2=0x7f2cf806fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:21.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.136+0000 7f2cf67fc640 1 --2- 172.21.15.12:0/3517856486 >> v2:172.21.15.154:3300/0 conn(0x7f2cf806f7a0 0x7f2cf806fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.136+0000 7f2cf67fc640 1 -- 172.21.15.12:0/3517856486 >> v1:172.21.15.154:6789/0 conn(0x7f2cf8071110 legacy=0x7f2cf8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:21.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.136+0000 7f2cf67fc640 1 -- 172.21.15.12:0/3517856486 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf80c70d0 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.140+0000 7f2cf67fc640 1 --2- 172.21.15.12:0/3517856486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf80c6bd0 0x7f2cf8069e90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2cf806c830 tx=0x7f2cec01ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94e83343dd99c8a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:21.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.140+0000 7f2cf5ffb640 1 -- 172.21.15.12:0/3517856486 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec0a89b0 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.140+0000 7f2cf5ffb640 1 -- 172.21.15.12:0/3517856486 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cec0a8b10 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.140+0000 7f2cf5ffb640 1 -- 172.21.15.12:0/3517856486 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec0a8ea0 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.140+0000 7f2cfd49c640 1 -- 172.21.15.12:0/3517856486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf80c6bd0 msgr2=0x7f2cf8069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:21.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.140+0000 7f2cfd49c640 1 --2- 172.21.15.12:0/3517856486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf80c6bd0 0x7f2cf8069e90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2cf806c830 tx=0x7f2cec01ecb0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.140+0000 7f2cfd49c640 1 -- 172.21.15.12:0/3517856486 shutdown_connections 2023-10-30T12:53:21.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.140+0000 7f2cfd49c640 1 --2- 172.21.15.12:0/3517856486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf80c6bd0 0x7f2cf8069e90 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.140+0000 7f2cfd49c640 1 --2- 172.21.15.12:0/3517856486 >> v2:172.21.15.154:3300/0 conn(0x7f2cf806f7a0 0x7f2cf806fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.140+0000 7f2cfd49c640 1 -- 172.21.15.12:0/3517856486 >> 172.21.15.12:0/3517856486 conn(0x7f2cf80c7910 msgr2=0x7f2cf80c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:21.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.140+0000 7f2cfd49c640 1 -- 172.21.15.12:0/3517856486 shutdown_connections 2023-10-30T12:53:21.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.140+0000 7f2cfd49c640 1 -- 172.21.15.12:0/3517856486 wait complete. 2023-10-30T12:53:21.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.140+0000 7f2cfd49c640 1 Processor -- start 2023-10-30T12:53:21.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cfd49c640 1 -- start start 2023-10-30T12:53:21.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cfd49c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cf806f7a0 0x7f2cf81471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:21.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cfd49c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2cf8071110 0x7f2cf80c1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:21.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cfd49c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf80c6bd0 0x7f2cf80c19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:21.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cfd49c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf80c03f0 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cfd49c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf80bfec0 con 0x7f2cf806f7a0 2023-10-30T12:53:21.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cfd49c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf80c0f80 con 0x7f2cf8071110 2023-10-30T12:53:21.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cf77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf80c6bd0 0x7f2cf80c19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:21.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cf77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf80c6bd0 0x7f2cf80c19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39556/0 (socket says 172.21.15.12:39556) 2023-10-30T12:53:21.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cf77fe640 1 -- 172.21.15.12:0/4260315878 learned_addr learned my addr 172.21.15.12:0/4260315878 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:21.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cf6ffd640 1 --2- 172.21.15.12:0/4260315878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cf806f7a0 0x7f2cf81471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:21.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cf67fc640 1 --2- 172.21.15.12:0/4260315878 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2cf8071110 0x7f2cf80c1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:21.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cf77fe640 1 -- 172.21.15.12:0/4260315878 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2cf8071110 msgr2=0x7f2cf80c1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:21.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cf77fe640 1 --2- 172.21.15.12:0/4260315878 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2cf8071110 0x7f2cf80c1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cf77fe640 1 -- 172.21.15.12:0/4260315878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cf806f7a0 msgr2=0x7f2cf81471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:21.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cf77fe640 1 --2- 172.21.15.12:0/4260315878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cf806f7a0 0x7f2cf81471c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cf77fe640 1 -- 172.21.15.12:0/4260315878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec0094d0 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cf6ffd640 1 --2- 172.21.15.12:0/4260315878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cf806f7a0 0x7f2cf81471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:21.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cf77fe640 1 --2- 172.21.15.12:0/4260315878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf80c6bd0 0x7f2cf80c19a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000b970 tx=0x7f2cf000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:21.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cdf7fe640 1 -- 172.21.15.12:0/4260315878 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf000cb40 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cdf7fe640 1 -- 172.21.15.12:0/4260315878 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf000cca0 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cdf7fe640 1 -- 172.21.15.12:0/4260315878 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0014680 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cfd49c640 1 -- 172.21.15.12:0/4260315878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf80c0110 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cfd49c640 1 -- 172.21.15.12:0/4260315878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf806a090 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.144+0000 7f2cfd49c640 1 -- 172.21.15.12:0/4260315878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf81ef9f0 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.148+0000 7f2cdf7fe640 1 -- 172.21.15.12:0/4260315878 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2cf001d3b0 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.148+0000 7f2cdf7fe640 1 --2- 172.21.15.12:0/4260315878 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2ce40664d0 0x7f2ce4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:21.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.148+0000 7f2cdf7fe640 1 -- 172.21.15.12:0/4260315878 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f2cf0083fe0 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.148+0000 7f2cf6ffd640 1 --2- 172.21.15.12:0/4260315878 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2ce40664d0 0x7f2ce4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:21.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.148+0000 7f2cf6ffd640 1 --2- 172.21.15.12:0/4260315878 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2ce40664d0 0x7f2ce4068990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800b9f0 tx=0x7f2ce8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:21.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.152+0000 7f2cdf7fe640 1 -- 172.21.15.12:0/4260315878 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2cf81ef9f0 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.344+0000 7f2cfd49c640 1 -- 172.21.15.12:0/4260315878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cf81ef9f0 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.344+0000 7f2cdf7fe640 1 -- 172.21.15.12:0/4260315878 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5742 (secure 0 0 0) 0x7f2cf81ef9f0 con 0x7f2cf80c6bd0 2023-10-30T12:53:21.350 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:21.350 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:20.432438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:21.350 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 89 2023-10-30T12:53:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.348+0000 7f2cfd49c640 1 -- 172.21.15.12:0/4260315878 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2ce40664d0 msgr2=0x7f2ce4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.348+0000 7f2cfd49c640 1 --2- 172.21.15.12:0/4260315878 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2ce40664d0 0x7f2ce4068990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800b9f0 tx=0x7f2ce8008040 comp rx=0 tx=0).stop 2023-10-30T12:53:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.348+0000 7f2cfd49c640 1 -- 172.21.15.12:0/4260315878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf80c6bd0 msgr2=0x7f2cf80c19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.348+0000 7f2cfd49c640 1 --2- 172.21.15.12:0/4260315878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf80c6bd0 0x7f2cf80c19a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000b970 tx=0x7f2cf000be30 comp rx=0 tx=0).stop 2023-10-30T12:53:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.348+0000 7f2cfd49c640 1 -- 172.21.15.12:0/4260315878 shutdown_connections 2023-10-30T12:53:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.348+0000 7f2cfd49c640 1 --2- 172.21.15.12:0/4260315878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf80c6bd0 0x7f2cf80c19a0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.348+0000 7f2cfd49c640 1 --2- 172.21.15.12:0/4260315878 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2ce40664d0 0x7f2ce4068990 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.348+0000 7f2cfd49c640 1 --2- 172.21.15.12:0/4260315878 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2cf8071110 0x7f2cf80c1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.348+0000 7f2cfd49c640 1 --2- 172.21.15.12:0/4260315878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cf806f7a0 0x7f2cf81471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.348+0000 7f2cfd49c640 1 -- 172.21.15.12:0/4260315878 >> 172.21.15.12:0/4260315878 conn(0x7f2cf80c7910 msgr2=0x7f2cf80c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.348+0000 7f2cfd49c640 1 -- 172.21.15.12:0/4260315878 shutdown_connections 2023-10-30T12:53:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:21.348+0000 7f2cfd49c640 1 -- 172.21.15.12:0/4260315878 wait complete. 2023-10-30T12:53:21.372 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 89, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 89, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5758": { "addr": "172.21.15.154:6839/2204547612", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 2204547612, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 2204547612, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 5758, "incarnation": 73, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_5877": { "addr": "172.21.15.12:6835/2306258157", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 2306258157, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 2306258157, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 5877, "incarnation": 81, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_6021": { "addr": "172.21.15.12:6836/3877967042", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 3877967042, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 3877967042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6021, "incarnation": 89, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 54, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:53:20.432438+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6021, "mds_1": 5758, "mds_2": 5877 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:53:23.373 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:23.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.524+0000 7fc2c4e25640 1 Processor -- start 2023-10-30T12:53:23.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.524+0000 7fc2c4e25640 1 -- start start 2023-10-30T12:53:23.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.524+0000 7fc2c4e25640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc2c006e470 0x7fc2c006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:23.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.524+0000 7fc2c4e25640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2c00c8490 0x7fc2c00c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:23.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.524+0000 7fc2c4e25640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c006afa0 con 0x7fc2c0153d70 2023-10-30T12:53:23.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.524+0000 7fc2c4e25640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c006d780 con 0x7fc2c006e470 2023-10-30T12:53:23.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.524+0000 7fc2c4e25640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c0154150 con 0x7fc2c00c8490 2023-10-30T12:53:23.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.524+0000 7fc2bdd9b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc2c006e470 0x7fc2c006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:23.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.524+0000 7fc2bdd9b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc2c006e470 0x7fc2c006ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39566/0 (socket says 172.21.15.12:39566) 2023-10-30T12:53:23.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.524+0000 7fc2bdd9b640 1 -- 172.21.15.12:0/1966848964 learned_addr learned my addr 172.21.15.12:0/1966848964 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:23.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.524+0000 7fc2bed9d640 1 --2- 172.21.15.12:0/1966848964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2c00c8490 0x7fc2c00c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:23.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.524+0000 7fc2bdd9b640 1 -- 172.21.15.12:0/1966848964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2c00c8490 msgr2=0x7fc2c00c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:23.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.524+0000 7fc2bdd9b640 1 --2- 172.21.15.12:0/1966848964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2c00c8490 0x7fc2c00c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:23.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.524+0000 7fc2bdd9b640 1 -- 172.21.15.12:0/1966848964 >> v1:172.21.15.154:6789/0 conn(0x7fc2c0153d70 legacy=0x7fc2c00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:23.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.524+0000 7fc2bdd9b640 1 -- 172.21.15.12:0/1966848964 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c006e030 con 0x7fc2c006e470 2023-10-30T12:53:23.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.528+0000 7fc2bdd9b640 1 --2- 172.21.15.12:0/1966848964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c006e470 0x7fc2c006ca80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009870 tx=0x7fc2b401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79538459cee014b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:23.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.528+0000 7fc2bd59a640 1 -- 172.21.15.12:0/1966848964 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b40aa670 con 0x7fc2c006e470 2023-10-30T12:53:23.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.528+0000 7fc2bd59a640 1 -- 172.21.15.12:0/1966848964 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b40aa7d0 con 0x7fc2c006e470 2023-10-30T12:53:23.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.528+0000 7fc2bd59a640 1 -- 172.21.15.12:0/1966848964 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b40aab60 con 0x7fc2c006e470 2023-10-30T12:53:23.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.528+0000 7fc2c4e25640 1 -- 172.21.15.12:0/1966848964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c006e470 msgr2=0x7fc2c006ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:23.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.528+0000 7fc2c4e25640 1 --2- 172.21.15.12:0/1966848964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c006e470 0x7fc2c006ca80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009870 tx=0x7fc2b401c510 comp rx=0 tx=0).stop 2023-10-30T12:53:23.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.528+0000 7fc2c4e25640 1 -- 172.21.15.12:0/1966848964 shutdown_connections 2023-10-30T12:53:23.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.528+0000 7fc2c4e25640 1 --2- 172.21.15.12:0/1966848964 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2c00c8490 0x7fc2c00c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:23.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.528+0000 7fc2c4e25640 1 --2- 172.21.15.12:0/1966848964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c006e470 0x7fc2c006ca80 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:23.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.528+0000 7fc2c4e25640 1 -- 172.21.15.12:0/1966848964 >> 172.21.15.12:0/1966848964 conn(0x7fc2c0071ea0 msgr2=0x7fc2c0065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:23.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.528+0000 7fc2c4e25640 1 -- 172.21.15.12:0/1966848964 shutdown_connections 2023-10-30T12:53:23.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.528+0000 7fc2c4e25640 1 -- 172.21.15.12:0/1966848964 wait complete. 2023-10-30T12:53:23.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2c4e25640 1 Processor -- start 2023-10-30T12:53:23.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2c4e25640 1 -- start start 2023-10-30T12:53:23.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2c4e25640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2c006e470 0x7fc2c01f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:23.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2c4e25640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c00c8490 0x7fc2c0152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:23.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2c4e25640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2c0153d70 0x7fc2c013f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:23.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2c4e25640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c0150930 con 0x7fc2c00c8490 2023-10-30T12:53:23.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2c4e25640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c01f60f0 con 0x7fc2c0153d70 2023-10-30T12:53:23.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2c4e25640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c014fe90 con 0x7fc2c006e470 2023-10-30T12:53:23.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2bdd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c00c8490 0x7fc2c0152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:23.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2bdd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c00c8490 0x7fc2c0152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39572/0 (socket says 172.21.15.12:39572) 2023-10-30T12:53:23.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2bdd9b640 1 -- 172.21.15.12:0/3427671857 learned_addr learned my addr 172.21.15.12:0/3427671857 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:23.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2bed9d640 1 --2- 172.21.15.12:0/3427671857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2c0153d70 0x7fc2c013f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:23.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2be59c640 1 --2- 172.21.15.12:0/3427671857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2c006e470 0x7fc2c01f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:23.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2bdd9b640 1 -- 172.21.15.12:0/3427671857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2c006e470 msgr2=0x7fc2c01f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:23.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2bdd9b640 1 --2- 172.21.15.12:0/3427671857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2c006e470 0x7fc2c01f5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:23.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2bdd9b640 1 -- 172.21.15.12:0/3427671857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2c0153d70 msgr2=0x7fc2c013f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:23.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2bdd9b640 1 --2- 172.21.15.12:0/3427671857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2c0153d70 0x7fc2c013f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:23.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2bdd9b640 1 -- 172.21.15.12:0/3427671857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b40094d0 con 0x7fc2c00c8490 2023-10-30T12:53:23.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2be59c640 1 --2- 172.21.15.12:0/3427671857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2c006e470 0x7fc2c01f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:23.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2bdd9b640 1 --2- 172.21.15.12:0/3427671857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c00c8490 0x7fc2c0152850 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc2b40ac040 tx=0x7fc2b401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:23.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2aeffd640 1 -- 172.21.15.12:0/3427671857 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b40b3800 con 0x7fc2c00c8490 2023-10-30T12:53:23.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2aeffd640 1 -- 172.21.15.12:0/3427671857 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b40b3960 con 0x7fc2c00c8490 2023-10-30T12:53:23.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2c4e25640 1 -- 172.21.15.12:0/3427671857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c014f600 con 0x7fc2c00c8490 2023-10-30T12:53:23.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2aeffd640 1 -- 172.21.15.12:0/3427671857 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b40b3d10 con 0x7fc2c00c8490 2023-10-30T12:53:23.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.532+0000 7fc2c4e25640 1 -- 172.21.15.12:0/3427671857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c014ab40 con 0x7fc2c00c8490 2023-10-30T12:53:23.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.536+0000 7fc2c4e25640 1 -- 172.21.15.12:0/3427671857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c0150a70 con 0x7fc2c00c8490 2023-10-30T12:53:23.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.536+0000 7fc2aeffd640 1 -- 172.21.15.12:0/3427671857 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc2b40b2380 con 0x7fc2c00c8490 2023-10-30T12:53:23.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.536+0000 7fc2aeffd640 1 --2- 172.21.15.12:0/3427671857 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc2a00664a0 0x7fc2a0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:23.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.536+0000 7fc2aeffd640 1 -- 172.21.15.12:0/3427671857 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fc2b411e390 con 0x7fc2c00c8490 2023-10-30T12:53:23.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.536+0000 7fc2be59c640 1 --2- 172.21.15.12:0/3427671857 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc2a00664a0 0x7fc2a0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:23.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.536+0000 7fc2be59c640 1 --2- 172.21.15.12:0/3427671857 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc2a00664a0 0x7fc2a0068960 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0000f70 tx=0x7fc2b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:23.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.540+0000 7fc2aeffd640 1 -- 172.21.15.12:0/3427671857 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc2c0150a70 con 0x7fc2c00c8490 2023-10-30T12:53:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.732+0000 7fc2c4e25640 1 -- 172.21.15.12:0/3427671857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2c0150a70 con 0x7fc2c00c8490 2023-10-30T12:53:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.732+0000 7fc2aeffd640 1 -- 172.21.15.12:0/3427671857 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5742 (secure 0 0 0) 0x7fc2c0150a70 con 0x7fc2c00c8490 2023-10-30T12:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:23.737 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:20.432438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:23.737 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 89 2023-10-30T12:53:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.736+0000 7fc2c4e25640 1 -- 172.21.15.12:0/3427671857 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc2a00664a0 msgr2=0x7fc2a0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.736+0000 7fc2c4e25640 1 --2- 172.21.15.12:0/3427671857 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc2a00664a0 0x7fc2a0068960 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0000f70 tx=0x7fc2b0009040 comp rx=0 tx=0).stop 2023-10-30T12:53:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.736+0000 7fc2c4e25640 1 -- 172.21.15.12:0/3427671857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c00c8490 msgr2=0x7fc2c0152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.736+0000 7fc2c4e25640 1 --2- 172.21.15.12:0/3427671857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c00c8490 0x7fc2c0152850 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc2b40ac040 tx=0x7fc2b401ad50 comp rx=0 tx=0).stop 2023-10-30T12:53:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.736+0000 7fc2c4e25640 1 -- 172.21.15.12:0/3427671857 shutdown_connections 2023-10-30T12:53:23.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.736+0000 7fc2c4e25640 1 --2- 172.21.15.12:0/3427671857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2c0153d70 0x7fc2c013f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:23.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.736+0000 7fc2c4e25640 1 --2- 172.21.15.12:0/3427671857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c00c8490 0x7fc2c0152850 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:23.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.736+0000 7fc2c4e25640 1 --2- 172.21.15.12:0/3427671857 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc2a00664a0 0x7fc2a0068960 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:23.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.736+0000 7fc2c4e25640 1 --2- 172.21.15.12:0/3427671857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2c006e470 0x7fc2c01f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:23.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.736+0000 7fc2c4e25640 1 -- 172.21.15.12:0/3427671857 >> 172.21.15.12:0/3427671857 conn(0x7fc2c0071ea0 msgr2=0x7fc2c014e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:23.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.736+0000 7fc2c4e25640 1 -- 172.21.15.12:0/3427671857 shutdown_connections 2023-10-30T12:53:23.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:23.736+0000 7fc2c4e25640 1 -- 172.21.15.12:0/3427671857 wait complete. 2023-10-30T12:53:23.757 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:25.758 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:25.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.900+0000 7f78e759e640 1 Processor -- start 2023-10-30T12:53:25.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.900+0000 7f78e759e640 1 -- start start 2023-10-30T12:53:25.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.904+0000 7f78e759e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f78e80c3640 0x7f78e815a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:25.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.904+0000 7f78e759e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78e80c6c20 0x7f78e8156590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:25.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.904+0000 7f78e759e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e80c3b30 con 0x7f78e80c6840 2023-10-30T12:53:25.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.904+0000 7f78e759e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e80c2aa0 con 0x7f78e80c3640 2023-10-30T12:53:25.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.904+0000 7f78e759e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e80c3df0 con 0x7f78e80c6c20 2023-10-30T12:53:25.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.904+0000 7f78dffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f78e80c3640 0x7f78e815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:25.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.904+0000 7f78dffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f78e80c3640 0x7f78e815a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39578/0 (socket says 172.21.15.12:39578) 2023-10-30T12:53:25.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.904+0000 7f78dffff640 1 -- 172.21.15.12:0/3864880246 learned_addr learned my addr 172.21.15.12:0/3864880246 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:25.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.904+0000 7f78e6d9d640 1 --2- 172.21.15.12:0/3864880246 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78e80c6c20 0x7f78e8156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:25.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.904+0000 7f78dffff640 1 -- 172.21.15.12:0/3864880246 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78e80c6c20 msgr2=0x7f78e8156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:25.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.904+0000 7f78dffff640 1 --2- 172.21.15.12:0/3864880246 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78e80c6c20 0x7f78e8156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:25.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.904+0000 7f78dffff640 1 -- 172.21.15.12:0/3864880246 >> v1:172.21.15.12:6789/0 conn(0x7f78e80c6840 legacy=0x7f78e8153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:25.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.904+0000 7f78dffff640 1 -- 172.21.15.12:0/3864880246 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e8159ba0 con 0x7f78e80c3640 2023-10-30T12:53:25.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.904+0000 7f78e6d9d640 1 --2- 172.21.15.12:0/3864880246 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78e80c6c20 0x7f78e8156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:53:25.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.904+0000 7f78dffff640 1 --2- 172.21.15.12:0/3864880246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e80c3640 0x7f78e815a680 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009870 tx=0x7f78d001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b922c90050ec26bd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:25.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e5d9b640 1 -- 172.21.15.12:0/3864880246 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00ab650 con 0x7f78e80c3640 2023-10-30T12:53:25.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e5d9b640 1 -- 172.21.15.12:0/3864880246 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d00ab7b0 con 0x7f78e80c3640 2023-10-30T12:53:25.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e5d9b640 1 -- 172.21.15.12:0/3864880246 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00abb40 con 0x7f78e80c3640 2023-10-30T12:53:25.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 -- 172.21.15.12:0/3864880246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e80c3640 msgr2=0x7f78e815a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:25.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 --2- 172.21.15.12:0/3864880246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e80c3640 0x7f78e815a680 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009870 tx=0x7f78d001c540 comp rx=0 tx=0).stop 2023-10-30T12:53:25.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 -- 172.21.15.12:0/3864880246 shutdown_connections 2023-10-30T12:53:25.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 --2- 172.21.15.12:0/3864880246 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78e80c6c20 0x7f78e8156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:25.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 --2- 172.21.15.12:0/3864880246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e80c3640 0x7f78e815a680 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:25.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 -- 172.21.15.12:0/3864880246 >> 172.21.15.12:0/3864880246 conn(0x7f78e80c75d0 msgr2=0x7f78e80c79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:25.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 -- 172.21.15.12:0/3864880246 shutdown_connections 2023-10-30T12:53:25.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 -- 172.21.15.12:0/3864880246 wait complete. 2023-10-30T12:53:25.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 Processor -- start 2023-10-30T12:53:25.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 -- start start 2023-10-30T12:53:25.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e80c3640 0x7f78e814f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:25.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78e80c6840 0x7f78e806f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:25.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78e80c6c20 0x7f78e806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:25.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8157c50 con 0x7f78e80c3640 2023-10-30T12:53:25.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8158f00 con 0x7f78e80c6840 2023-10-30T12:53:25.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e759e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e814ec90 con 0x7f78e80c6c20 2023-10-30T12:53:25.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e80c3640 0x7f78e814f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:25.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.908+0000 7f78e659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e80c3640 0x7f78e814f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39592/0 (socket says 172.21.15.12:39592) 2023-10-30T12:53:25.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78e659c640 1 -- 172.21.15.12:0/2555439941 learned_addr learned my addr 172.21.15.12:0/2555439941 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:25.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78e6d9d640 1 --2- 172.21.15.12:0/2555439941 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78e80c6c20 0x7f78e806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:25.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78dffff640 1 --2- 172.21.15.12:0/2555439941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78e80c6840 0x7f78e806f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:25.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78e659c640 1 -- 172.21.15.12:0/2555439941 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78e80c6c20 msgr2=0x7f78e806fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:25.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78e659c640 1 --2- 172.21.15.12:0/2555439941 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78e80c6c20 0x7f78e806fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:25.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78e659c640 1 -- 172.21.15.12:0/2555439941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78e80c6840 msgr2=0x7f78e806f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:25.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78e659c640 1 --2- 172.21.15.12:0/2555439941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78e80c6840 0x7f78e806f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:25.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78e659c640 1 -- 172.21.15.12:0/2555439941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d00094d0 con 0x7f78e80c3640 2023-10-30T12:53:25.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78dffff640 1 --2- 172.21.15.12:0/2555439941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78e80c6840 0x7f78e806f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:25.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78e659c640 1 --2- 172.21.15.12:0/2555439941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e80c3640 0x7f78e814f450 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f78d800c450 tx=0x7f78d800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:25.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78deffd640 1 -- 172.21.15.12:0/2555439941 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8005480 con 0x7f78e80c3640 2023-10-30T12:53:25.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78deffd640 1 -- 172.21.15.12:0/2555439941 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d8003600 con 0x7f78e80c3640 2023-10-30T12:53:25.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78deffd640 1 -- 172.21.15.12:0/2555439941 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d80039b0 con 0x7f78e80c3640 2023-10-30T12:53:25.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78e759e640 1 -- 172.21.15.12:0/2555439941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e8159380 con 0x7f78e80c3640 2023-10-30T12:53:25.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78e759e640 1 -- 172.21.15.12:0/2555439941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e814ef30 con 0x7f78e80c3640 2023-10-30T12:53:25.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78e759e640 1 -- 172.21.15.12:0/2555439941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e814f120 con 0x7f78e80c3640 2023-10-30T12:53:25.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78deffd640 1 -- 172.21.15.12:0/2555439941 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f78d8003b10 con 0x7f78e80c3640 2023-10-30T12:53:25.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.912+0000 7f78deffd640 1 --2- 172.21.15.12:0/2555439941 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f78c80664a0 0x7f78c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:25.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.916+0000 7f78deffd640 1 -- 172.21.15.12:0/2555439941 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f78d8083860 con 0x7f78e80c3640 2023-10-30T12:53:25.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.916+0000 7f78dffff640 1 --2- 172.21.15.12:0/2555439941 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f78c80664a0 0x7f78c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:25.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.916+0000 7f78dffff640 1 --2- 172.21.15.12:0/2555439941 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f78c80664a0 0x7f78c8068960 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f78d00af000 tx=0x7f78d00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:25.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:25.920+0000 7f78deffd640 1 -- 172.21.15.12:0/2555439941 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f78d8050830 con 0x7f78e80c3640 2023-10-30T12:53:26.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:26.108+0000 7f78e759e640 1 -- 172.21.15.12:0/2555439941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78e804e170 con 0x7f78e80c3640 2023-10-30T12:53:26.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:26.108+0000 7f78deffd640 1 -- 172.21.15.12:0/2555439941 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5742 (secure 0 0 0) 0x7f78d8017020 con 0x7f78e80c3640 2023-10-30T12:53:26.113 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:26.113 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:20.432438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:26.113 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 89 2023-10-30T12:53:26.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:26.112+0000 7f78e759e640 1 -- 172.21.15.12:0/2555439941 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f78c80664a0 msgr2=0x7f78c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:26.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:26.112+0000 7f78e759e640 1 --2- 172.21.15.12:0/2555439941 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f78c80664a0 0x7f78c8068960 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f78d00af000 tx=0x7f78d00a7000 comp rx=0 tx=0).stop 2023-10-30T12:53:26.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:26.112+0000 7f78e759e640 1 -- 172.21.15.12:0/2555439941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e80c3640 msgr2=0x7f78e814f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:26.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:26.112+0000 7f78e759e640 1 --2- 172.21.15.12:0/2555439941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e80c3640 0x7f78e814f450 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f78d800c450 tx=0x7f78d800c910 comp rx=0 tx=0).stop 2023-10-30T12:53:26.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:26.112+0000 7f78e759e640 1 -- 172.21.15.12:0/2555439941 shutdown_connections 2023-10-30T12:53:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:26.112+0000 7f78e759e640 1 --2- 172.21.15.12:0/2555439941 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f78c80664a0 0x7f78c8068960 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:26.112+0000 7f78e759e640 1 --2- 172.21.15.12:0/2555439941 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78e80c6c20 0x7f78e806fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:26.112+0000 7f78e759e640 1 --2- 172.21.15.12:0/2555439941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78e80c6840 0x7f78e806f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:26.112+0000 7f78e759e640 1 --2- 172.21.15.12:0/2555439941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e80c3640 0x7f78e814f450 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:26.112+0000 7f78e759e640 1 -- 172.21.15.12:0/2555439941 >> 172.21.15.12:0/2555439941 conn(0x7f78e80c75d0 msgr2=0x7f78e81505e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:26.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:26.112+0000 7f78e759e640 1 -- 172.21.15.12:0/2555439941 shutdown_connections 2023-10-30T12:53:26.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:26.112+0000 7f78e759e640 1 -- 172.21.15.12:0/2555439941 wait complete. 2023-10-30T12:53:26.134 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:28.135 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:28.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.288+0000 7f96e81a1640 1 Processor -- start 2023-10-30T12:53:28.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.288+0000 7f96e81a1640 1 -- start start 2023-10-30T12:53:28.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.292+0000 7f96e81a1640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f96e00cf160 0x7f96e00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:28.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.292+0000 7f96e81a1640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e00cb800 con 0x7f96e015d6f0 2023-10-30T12:53:28.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.292+0000 7f96e81a1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e00cb330 con 0x7f96e01538a0 2023-10-30T12:53:28.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.292+0000 7f96e81a1640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e00cad70 con 0x7f96e00cf160 2023-10-30T12:53:28.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.292+0000 7f96e573c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f96e015d6f0 0x7f96e015a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45608/0 (socket says 172.21.15.12:45608) 2023-10-30T12:53:28.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.292+0000 7f96e573c640 1 -- 172.21.15.12:0/4272228357 learned_addr learned my addr 172.21.15.12:0/4272228357 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:53:28.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.292+0000 7f96e673e640 1 --2- 172.21.15.12:0/4272228357 >> v2:172.21.15.12:3300/0 conn(0x7f96e00cf160 0x7f96e00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:28.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.292+0000 7f96e673e640 1 -- 172.21.15.12:0/4272228357 >> v1:172.21.15.154:6789/0 conn(0x7f96e01538a0 legacy=0x7f96e00ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:28.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.292+0000 7f96e673e640 1 -- 172.21.15.12:0/4272228357 >> v1:172.21.15.12:6789/0 conn(0x7f96e015d6f0 legacy=0x7f96e015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:28.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.292+0000 7f96e673e640 1 -- 172.21.15.12:0/4272228357 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e00cb050 con 0x7f96e00cf160 2023-10-30T12:53:28.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.292+0000 7f96e673e640 1 --2- 172.21.15.12:0/4272228357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96e00cf160 0x7f96e00cf590 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009d90 tx=0x7f96d801f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dca4fd2435d34b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:28.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.292+0000 7f96e4f3b640 1 -- 172.21.15.12:0/4272228357 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d80af600 con 0x7f96e00cf160 2023-10-30T12:53:28.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e4f3b640 1 -- 172.21.15.12:0/4272228357 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d80ab050 con 0x7f96e00cf160 2023-10-30T12:53:28.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e4f3b640 1 -- 172.21.15.12:0/4272228357 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d80af980 con 0x7f96e00cf160 2023-10-30T12:53:28.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e81a1640 1 -- 172.21.15.12:0/4272228357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96e00cf160 msgr2=0x7f96e00cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:28.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e81a1640 1 --2- 172.21.15.12:0/4272228357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96e00cf160 0x7f96e00cf590 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009d90 tx=0x7f96d801f130 comp rx=0 tx=0).stop 2023-10-30T12:53:28.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e81a1640 1 -- 172.21.15.12:0/4272228357 shutdown_connections 2023-10-30T12:53:28.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e81a1640 1 --2- 172.21.15.12:0/4272228357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96e00cf160 0x7f96e00cf590 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:28.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e81a1640 1 -- 172.21.15.12:0/4272228357 >> 172.21.15.12:0/4272228357 conn(0x7f96e006bfe0 msgr2=0x7f96e006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:28.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e81a1640 1 -- 172.21.15.12:0/4272228357 shutdown_connections 2023-10-30T12:53:28.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e81a1640 1 -- 172.21.15.12:0/4272228357 wait complete. 2023-10-30T12:53:28.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e81a1640 1 Processor -- start 2023-10-30T12:53:28.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e81a1640 1 -- start start 2023-10-30T12:53:28.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e81a1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96e00cf160 0x7f96e0154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:28.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e81a1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e01538a0 0x7f96e00b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:28.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e81a1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96e015d6f0 0x7f96e01e2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:28.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e81a1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e01559b0 con 0x7f96e015d6f0 2023-10-30T12:53:28.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e81a1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e0159410 con 0x7f96e01538a0 2023-10-30T12:53:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e81a1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e01e3de0 con 0x7f96e00cf160 2023-10-30T12:53:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e673e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96e015d6f0 0x7f96e01e2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e673e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96e015d6f0 0x7f96e01e2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39610/0 (socket says 172.21.15.12:39610) 2023-10-30T12:53:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.296+0000 7f96e673e640 1 -- 172.21.15.12:0/1964789637 learned_addr learned my addr 172.21.15.12:0/1964789637 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96e573c640 1 --2- 172.21.15.12:0/1964789637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e01538a0 0x7f96e00b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:28.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96e5f3d640 1 --2- 172.21.15.12:0/1964789637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96e00cf160 0x7f96e0154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:28.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96e673e640 1 -- 172.21.15.12:0/1964789637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96e00cf160 msgr2=0x7f96e0154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:28.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96e673e640 1 --2- 172.21.15.12:0/1964789637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96e00cf160 0x7f96e0154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:28.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96e673e640 1 -- 172.21.15.12:0/1964789637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e01538a0 msgr2=0x7f96e00b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:28.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96e673e640 1 --2- 172.21.15.12:0/1964789637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e01538a0 0x7f96e00b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:28.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96e673e640 1 -- 172.21.15.12:0/1964789637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d8009a70 con 0x7f96e015d6f0 2023-10-30T12:53:28.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96e5f3d640 1 --2- 172.21.15.12:0/1964789637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96e00cf160 0x7f96e0154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:28.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96e673e640 1 --2- 172.21.15.12:0/1964789637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96e015d6f0 0x7f96e01e2a70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009d90 tx=0x7f96d801bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:28.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96d67fc640 1 -- 172.21.15.12:0/1964789637 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d80b9410 con 0x7f96e015d6f0 2023-10-30T12:53:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96d67fc640 1 -- 172.21.15.12:0/1964789637 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d80b9570 con 0x7f96e015d6f0 2023-10-30T12:53:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96d67fc640 1 -- 172.21.15.12:0/1964789637 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d80b9920 con 0x7f96e015d6f0 2023-10-30T12:53:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96e81a1640 1 -- 172.21.15.12:0/1964789637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e0156ce0 con 0x7f96e015d6f0 2023-10-30T12:53:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96e81a1640 1 -- 172.21.15.12:0/1964789637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e0155300 con 0x7f96e015d6f0 2023-10-30T12:53:28.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96e81a1640 1 -- 172.21.15.12:0/1964789637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e004e170 con 0x7f96e015d6f0 2023-10-30T12:53:28.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96d67fc640 1 -- 172.21.15.12:0/1964789637 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f96d80b6050 con 0x7f96e015d6f0 2023-10-30T12:53:28.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.300+0000 7f96d67fc640 1 --2- 172.21.15.12:0/1964789637 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f96c40664a0 0x7f96c4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:28.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.304+0000 7f96d67fc640 1 -- 172.21.15.12:0/1964789637 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f96d8122780 con 0x7f96e015d6f0 2023-10-30T12:53:28.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.304+0000 7f96e5f3d640 1 --2- 172.21.15.12:0/1964789637 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f96c40664a0 0x7f96c4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:28.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.304+0000 7f96e5f3d640 1 --2- 172.21.15.12:0/1964789637 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f96c40664a0 0x7f96c4068960 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f96dc000f70 tx=0x7f96dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:28.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.304+0000 7f96d67fc640 1 -- 172.21.15.12:0/1964789637 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f96d80ef8a0 con 0x7f96e015d6f0 2023-10-30T12:53:28.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.496+0000 7f96e81a1640 1 -- 172.21.15.12:0/1964789637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96e00b1dd0 con 0x7f96e015d6f0 2023-10-30T12:53:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.496+0000 7f96d67fc640 1 -- 172.21.15.12:0/1964789637 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5742 (secure 0 0 0) 0x7f96d80f2ec0 con 0x7f96e015d6f0 2023-10-30T12:53:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:28.503 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:20.432438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:28.505 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 89 2023-10-30T12:53:28.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.500+0000 7f96e81a1640 1 -- 172.21.15.12:0/1964789637 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f96c40664a0 msgr2=0x7f96c4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:28.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.500+0000 7f96e81a1640 1 --2- 172.21.15.12:0/1964789637 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f96c40664a0 0x7f96c4068960 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f96dc000f70 tx=0x7f96dc009040 comp rx=0 tx=0).stop 2023-10-30T12:53:28.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.500+0000 7f96e81a1640 1 -- 172.21.15.12:0/1964789637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96e015d6f0 msgr2=0x7f96e01e2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:28.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.500+0000 7f96e81a1640 1 --2- 172.21.15.12:0/1964789637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96e015d6f0 0x7f96e01e2a70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009d90 tx=0x7f96d801bed0 comp rx=0 tx=0).stop 2023-10-30T12:53:28.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.500+0000 7f96e81a1640 1 -- 172.21.15.12:0/1964789637 shutdown_connections 2023-10-30T12:53:28.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.500+0000 7f96e81a1640 1 --2- 172.21.15.12:0/1964789637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96e015d6f0 0x7f96e01e2a70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:28.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.500+0000 7f96e81a1640 1 --2- 172.21.15.12:0/1964789637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e01538a0 0x7f96e00b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:28.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.500+0000 7f96e81a1640 1 --2- 172.21.15.12:0/1964789637 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f96c40664a0 0x7f96c4068960 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:28.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.500+0000 7f96e81a1640 1 --2- 172.21.15.12:0/1964789637 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96e00cf160 0x7f96e0154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:28.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.500+0000 7f96e81a1640 1 -- 172.21.15.12:0/1964789637 >> 172.21.15.12:0/1964789637 conn(0x7f96e006bfe0 msgr2=0x7f96e00735c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:28.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.500+0000 7f96e81a1640 1 -- 172.21.15.12:0/1964789637 shutdown_connections 2023-10-30T12:53:28.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:28.500+0000 7f96e81a1640 1 -- 172.21.15.12:0/1964789637 wait complete. 2023-10-30T12:53:28.524 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:30.525 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:30.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e09868640 1 Processor -- start 2023-10-30T12:53:30.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e09868640 1 -- start start 2023-10-30T12:53:30.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e09868640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0e040cc0b0 0x7f0e0406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:30.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e09868640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e040b1ea0 0x7f0e040cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:30.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e09868640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0406ade0 con 0x7f0e04153c30 2023-10-30T12:53:30.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e09868640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e041596e0 con 0x7f0e040cc0b0 2023-10-30T12:53:30.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e09868640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e041598f0 con 0x7f0e040b1ea0 2023-10-30T12:53:30.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e02ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0e04153c30 0x7f0e040cb9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45618/0 (socket says 172.21.15.12:45618) 2023-10-30T12:53:30.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e02ffd640 1 -- 172.21.15.12:0/325433472 learned_addr learned my addr 172.21.15.12:0/325433472 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:53:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e037fe640 1 --2- 172.21.15.12:0/325433472 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e040b1ea0 0x7f0e040cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0dfbfff640 1 --2- 172.21.15.12:0/325433472 >> v2:172.21.15.154:3300/0 conn(0x7f0e040cc0b0 0x7f0e0406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e027fc640 1 -- 172.21.15.12:0/325433472 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409717740 0 0) 0x7f0df4001120 con 0x7f0e04153c30 2023-10-30T12:53:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e027fc640 1 -- 172.21.15.12:0/325433472 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0df0003530 con 0x7f0e04153c30 2023-10-30T12:53:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e027fc640 1 -- 172.21.15.12:0/325433472 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 22736897 0 0) 0x7f0df0003530 con 0x7f0e04153c30 2023-10-30T12:53:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e027fc640 1 -- 172.21.15.12:0/325433472 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e040b1ea0 msgr2=0x7f0e040cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e027fc640 1 --2- 172.21.15.12:0/325433472 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e040b1ea0 0x7f0e040cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e027fc640 1 -- 172.21.15.12:0/325433472 >> v2:172.21.15.154:3300/0 conn(0x7f0e040cc0b0 msgr2=0x7f0e0406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e027fc640 1 --2- 172.21.15.12:0/325433472 >> v2:172.21.15.154:3300/0 conn(0x7f0e040cc0b0 0x7f0e0406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e027fc640 1 -- 172.21.15.12:0/325433472 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e04159b90 con 0x7f0e04153c30 2023-10-30T12:53:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e027fc640 1 -- 172.21.15.12:0/325433472 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f0df4001810 con 0x7f0e04153c30 2023-10-30T12:53:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e027fc640 1 -- 172.21.15.12:0/325433472 >> v1:172.21.15.12:6789/0 conn(0x7f0e04153c30 legacy=0x7f0e040cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.676+0000 7f0e027fc640 1 --2- 172.21.15.12:0/325433472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0df0003980 0x7f0df0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0e027fc640 1 --2- 172.21.15.12:0/325433472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0df00047d0 0x7f0df0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0e027fc640 1 --2- 172.21.15.12:0/325433472 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e040b1ea0 0x7f0df0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:30.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0e027fc640 1 -- 172.21.15.12:0/325433472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e041596e0 con 0x7f0df00047d0 2023-10-30T12:53:30.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0e027fc640 1 -- 172.21.15.12:0/325433472 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e041598f0 con 0x7f0df0003980 2023-10-30T12:53:30.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0e027fc640 1 -- 172.21.15.12:0/325433472 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df000b990 con 0x7f0e040b1ea0 2023-10-30T12:53:30.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0dfbfff640 1 --2- 172.21.15.12:0/325433472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0df00047d0 0x7f0df0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:30.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0e02ffd640 1 --2- 172.21.15.12:0/325433472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0df0003980 0x7f0df0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:30.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0e037fe640 1 --2- 172.21.15.12:0/325433472 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e040b1ea0 0x7f0df0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:30.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0dfbfff640 1 -- 172.21.15.12:0/325433472 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e040b1ea0 msgr2=0x7f0df0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:30.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0dfbfff640 1 --2- 172.21.15.12:0/325433472 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e040b1ea0 0x7f0df0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:30.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0dfbfff640 1 -- 172.21.15.12:0/325433472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0df0003980 msgr2=0x7f0df0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:30.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0dfbfff640 1 --2- 172.21.15.12:0/325433472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0df0003980 0x7f0df0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:30.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0dfbfff640 1 -- 172.21.15.12:0/325433472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df0003530 con 0x7f0df00047d0 2023-10-30T12:53:30.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0dfbfff640 1 --2- 172.21.15.12:0/325433472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0df00047d0 0x7f0df0004c50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009ac0 tx=0x7f0dec01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888bdb2109919225 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:30.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0e027fc640 1 -- 172.21.15.12:0/325433472 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0059a0 con 0x7f0df00047d0 2023-10-30T12:53:30.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0e027fc640 1 -- 172.21.15.12:0/325433472 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dec005b00 con 0x7f0df00047d0 2023-10-30T12:53:30.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.680+0000 7f0e027fc640 1 -- 172.21.15.12:0/325433472 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec006030 con 0x7f0df00047d0 2023-10-30T12:53:30.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e09868640 1 -- 172.21.15.12:0/325433472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0df00047d0 msgr2=0x7f0df0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:30.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e09868640 1 --2- 172.21.15.12:0/325433472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0df00047d0 0x7f0df0004c50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009ac0 tx=0x7f0dec01ca70 comp rx=0 tx=0).stop 2023-10-30T12:53:30.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e037fe640 1 -- 172.21.15.12:0/325433472 reap_dead start 2023-10-30T12:53:30.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e09868640 1 -- 172.21.15.12:0/325433472 shutdown_connections 2023-10-30T12:53:30.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e09868640 1 -- 172.21.15.12:0/325433472 >> 172.21.15.12:0/325433472 conn(0x7f0e0406bf40 msgr2=0x7f0e0406c350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:30.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e09868640 1 -- 172.21.15.12:0/325433472 shutdown_connections 2023-10-30T12:53:30.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e09868640 1 -- 172.21.15.12:0/325433472 wait complete. 2023-10-30T12:53:30.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e09868640 1 Processor -- start 2023-10-30T12:53:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e09868640 1 -- start start 2023-10-30T12:53:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e09868640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0df00047d0 0x7f0e040cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e09868640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e0415d680 0x7f0e0414ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e09868640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e0414f270 0x7f0e0414fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e09868640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e04144440 con 0x7f0e0415d680 2023-10-30T12:53:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e09868640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e040ccdf0 con 0x7f0df00047d0 2023-10-30T12:53:30.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e09868640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e040cd090 con 0x7f0e0414f270 2023-10-30T12:53:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0dfbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e0415d680 0x7f0e0414ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0dfbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e0415d680 0x7f0e0414ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39624/0 (socket says 172.21.15.12:39624) 2023-10-30T12:53:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0dfbfff640 1 -- 172.21.15.12:0/1997109094 learned_addr learned my addr 172.21.15.12:0/1997109094 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e037fe640 1 --2- 172.21.15.12:0/1997109094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e0414f270 0x7f0e0414fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0e02ffd640 1 --2- 172.21.15.12:0/1997109094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0df00047d0 0x7f0e040cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0dfbfff640 1 -- 172.21.15.12:0/1997109094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e0414f270 msgr2=0x7f0e0414fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0dfbfff640 1 --2- 172.21.15.12:0/1997109094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e0414f270 0x7f0e0414fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0dfbfff640 1 -- 172.21.15.12:0/1997109094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0df00047d0 msgr2=0x7f0e040cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0dfbfff640 1 --2- 172.21.15.12:0/1997109094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0df00047d0 0x7f0e040cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.684+0000 7f0dfbfff640 1 -- 172.21.15.12:0/1997109094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0414dff0 con 0x7f0e0415d680 2023-10-30T12:53:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.688+0000 7f0dfbfff640 1 --2- 172.21.15.12:0/1997109094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e0415d680 0x7f0e0414ed30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0dec01c8a0 tx=0x7f0df0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.688+0000 7f0e00ff9640 1 -- 172.21.15.12:0/1997109094 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0059a0 con 0x7f0e0415d680 2023-10-30T12:53:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.688+0000 7f0e00ff9640 1 -- 172.21.15.12:0/1997109094 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dec004020 con 0x7f0e0415d680 2023-10-30T12:53:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.688+0000 7f0e09868640 1 -- 172.21.15.12:0/1997109094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e040cde20 con 0x7f0e0415d680 2023-10-30T12:53:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.688+0000 7f0e00ff9640 1 -- 172.21.15.12:0/1997109094 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0043d0 con 0x7f0e0415d680 2023-10-30T12:53:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.688+0000 7f0e09868640 1 -- 172.21.15.12:0/1997109094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e040ce3e0 con 0x7f0e0415d680 2023-10-30T12:53:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.688+0000 7f0e09868640 1 -- 172.21.15.12:0/1997109094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0404e170 con 0x7f0e0415d680 2023-10-30T12:53:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.688+0000 7f0e00ff9640 1 -- 172.21.15.12:0/1997109094 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0dec003620 con 0x7f0e0415d680 2023-10-30T12:53:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.688+0000 7f0e00ff9640 1 --2- 172.21.15.12:0/1997109094 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0de40664a0 0x7f0de4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.688+0000 7f0e00ff9640 1 -- 172.21.15.12:0/1997109094 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f0dec11f130 con 0x7f0e0415d680 2023-10-30T12:53:30.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.688+0000 7f0e02ffd640 1 --2- 172.21.15.12:0/1997109094 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0de40664a0 0x7f0de4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:30.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.692+0000 7f0e02ffd640 1 --2- 172.21.15.12:0/1997109094 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0de40664a0 0x7f0de4068960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0df40012c0 tx=0x7f0df4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:30.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.692+0000 7f0e00ff9640 1 -- 172.21.15.12:0/1997109094 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0dec0ec2c0 con 0x7f0e0415d680 2023-10-30T12:53:30.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.888+0000 7f0e09868640 1 -- 172.21.15.12:0/1997109094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e04144420 con 0x7f0e0415d680 2023-10-30T12:53:30.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.888+0000 7f0e00ff9640 1 -- 172.21.15.12:0/1997109094 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5744 (secure 0 0 0) 0x7f0dec0ef8e0 con 0x7f0e0415d680 2023-10-30T12:53:30.893 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:30.893 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:29.807534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:resolve","state_seq":34,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:30.895 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 90 2023-10-30T12:53:30.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.888+0000 7f0e09868640 1 -- 172.21.15.12:0/1997109094 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0de40664a0 msgr2=0x7f0de4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.888+0000 7f0e09868640 1 --2- 172.21.15.12:0/1997109094 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0de40664a0 0x7f0de4068960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0df40012c0 tx=0x7f0df4009040 comp rx=0 tx=0).stop 2023-10-30T12:53:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.888+0000 7f0e09868640 1 -- 172.21.15.12:0/1997109094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e0415d680 msgr2=0x7f0e0414ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.888+0000 7f0e09868640 1 --2- 172.21.15.12:0/1997109094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e0415d680 0x7f0e0414ed30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0dec01c8a0 tx=0x7f0df0005800 comp rx=0 tx=0).stop 2023-10-30T12:53:30.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.888+0000 7f0e09868640 1 -- 172.21.15.12:0/1997109094 shutdown_connections 2023-10-30T12:53:30.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.888+0000 7f0e09868640 1 --2- 172.21.15.12:0/1997109094 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0de40664a0 0x7f0de4068960 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:30.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.888+0000 7f0e09868640 1 --2- 172.21.15.12:0/1997109094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e0414f270 0x7f0e0414fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:30.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.888+0000 7f0e09868640 1 --2- 172.21.15.12:0/1997109094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e0415d680 0x7f0e0414ed30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:30.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.888+0000 7f0e09868640 1 --2- 172.21.15.12:0/1997109094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0df00047d0 0x7f0e040cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:30.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.888+0000 7f0e09868640 1 -- 172.21.15.12:0/1997109094 >> 172.21.15.12:0/1997109094 conn(0x7f0e0406bf40 msgr2=0x7f0e04158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:30.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.888+0000 7f0e09868640 1 -- 172.21.15.12:0/1997109094 shutdown_connections 2023-10-30T12:53:30.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:30.888+0000 7f0e09868640 1 -- 172.21.15.12:0/1997109094 wait complete. 2023-10-30T12:53:30.914 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:32.915 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:33.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.064+0000 7f2fb750f640 1 Processor -- start 2023-10-30T12:53:33.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.068+0000 7f2fb750f640 1 -- start start 2023-10-30T12:53:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.068+0000 7f2fb750f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2fb015d6f0 0x7f2fb015a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.068+0000 7f2fb750f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2fb00cf160 0x7f2fb00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.068+0000 7f2fb750f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb00cb800 con 0x7f2fb01538a0 2023-10-30T12:53:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.068+0000 7f2fb750f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb00cb330 con 0x7f2fb015d6f0 2023-10-30T12:53:33.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.068+0000 7f2fb750f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb00cad70 con 0x7f2fb00cf160 2023-10-30T12:53:33.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.068+0000 7f2fb4aaa640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2fb015d6f0 0x7f2fb015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:33.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.068+0000 7f2fb4aaa640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2fb015d6f0 0x7f2fb015a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45178/0 (socket says 172.21.15.12:45178) 2023-10-30T12:53:33.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.068+0000 7f2fb4aaa640 1 -- 172.21.15.12:0/4189499733 learned_addr learned my addr 172.21.15.12:0/4189499733 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:33.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.068+0000 7f2fb5aac640 1 --2- 172.21.15.12:0/4189499733 >> v2:172.21.15.154:3300/0 conn(0x7f2fb00cf160 0x7f2fb00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:33.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.068+0000 7f2fb4aaa640 1 -- 172.21.15.12:0/4189499733 >> v2:172.21.15.154:3300/0 conn(0x7f2fb00cf160 msgr2=0x7f2fb00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:33.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.068+0000 7f2fb4aaa640 1 --2- 172.21.15.12:0/4189499733 >> v2:172.21.15.154:3300/0 conn(0x7f2fb00cf160 0x7f2fb00cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:33.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.068+0000 7f2fb4aaa640 1 -- 172.21.15.12:0/4189499733 >> v1:172.21.15.154:6789/0 conn(0x7f2fb01538a0 legacy=0x7f2fb00ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:33.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.068+0000 7f2fb4aaa640 1 -- 172.21.15.12:0/4189499733 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb00cb050 con 0x7f2fb015d6f0 2023-10-30T12:53:33.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.072+0000 7f2fb4aaa640 1 --2- 172.21.15.12:0/4189499733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb015d6f0 0x7f2fb015a090 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00b5330 tx=0x7f2fa001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5d68b9df99cf4a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:33.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.072+0000 7f2fa7fff640 1 -- 172.21.15.12:0/4189499733 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa00a89f0 con 0x7f2fb015d6f0 2023-10-30T12:53:33.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.072+0000 7f2fa7fff640 1 -- 172.21.15.12:0/4189499733 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa00a8b50 con 0x7f2fb015d6f0 2023-10-30T12:53:33.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.072+0000 7f2fa7fff640 1 -- 172.21.15.12:0/4189499733 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa00b0440 con 0x7f2fb015d6f0 2023-10-30T12:53:33.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.072+0000 7f2fb750f640 1 -- 172.21.15.12:0/4189499733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb015d6f0 msgr2=0x7f2fb015a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:33.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.072+0000 7f2fb750f640 1 --2- 172.21.15.12:0/4189499733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb015d6f0 0x7f2fb015a090 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00b5330 tx=0x7f2fa001ecf0 comp rx=0 tx=0).stop 2023-10-30T12:53:33.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.072+0000 7f2fb750f640 1 -- 172.21.15.12:0/4189499733 shutdown_connections 2023-10-30T12:53:33.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.072+0000 7f2fb750f640 1 --2- 172.21.15.12:0/4189499733 >> v2:172.21.15.154:3300/0 conn(0x7f2fb00cf160 0x7f2fb00cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:33.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.072+0000 7f2fb750f640 1 --2- 172.21.15.12:0/4189499733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb015d6f0 0x7f2fb015a090 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:33.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.072+0000 7f2fb750f640 1 -- 172.21.15.12:0/4189499733 >> 172.21.15.12:0/4189499733 conn(0x7f2fb006bfe0 msgr2=0x7f2fb006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:33.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.072+0000 7f2fb750f640 1 -- 172.21.15.12:0/4189499733 shutdown_connections 2023-10-30T12:53:33.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.072+0000 7f2fb750f640 1 -- 172.21.15.12:0/4189499733 wait complete. 2023-10-30T12:53:33.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.072+0000 7f2fb750f640 1 Processor -- start 2023-10-30T12:53:33.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.072+0000 7f2fb750f640 1 -- start start 2023-10-30T12:53:33.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb750f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb00cf160 0x7f2fb0157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:33.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb750f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb01538a0 0x7f2fb01ef0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:33.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb750f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb015d6f0 0x7f2fb015bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:33.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb750f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00cef50 con 0x7f2fb00cf160 2023-10-30T12:53:33.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb750f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00ce9b0 con 0x7f2fb015d6f0 2023-10-30T12:53:33.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb750f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb0159130 con 0x7f2fb01538a0 2023-10-30T12:53:33.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb52ab640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb00cf160 0x7f2fb0157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:33.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb52ab640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb00cf160 0x7f2fb0157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45180/0 (socket says 172.21.15.12:45180) 2023-10-30T12:53:33.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb52ab640 1 -- 172.21.15.12:0/3365212951 learned_addr learned my addr 172.21.15.12:0/3365212951 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:33.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb5aac640 1 --2- 172.21.15.12:0/3365212951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb015d6f0 0x7f2fb015bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:33.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb4aaa640 1 --2- 172.21.15.12:0/3365212951 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb01538a0 0x7f2fb01ef0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:33.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb52ab640 1 -- 172.21.15.12:0/3365212951 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb01538a0 msgr2=0x7f2fb01ef0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:33.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb52ab640 1 --2- 172.21.15.12:0/3365212951 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb01538a0 0x7f2fb01ef0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:33.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb52ab640 1 -- 172.21.15.12:0/3365212951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb015d6f0 msgr2=0x7f2fb015bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:33.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb52ab640 1 --2- 172.21.15.12:0/3365212951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb015d6f0 0x7f2fb015bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:33.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb52ab640 1 -- 172.21.15.12:0/3365212951 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa00094d0 con 0x7f2fb00cf160 2023-10-30T12:53:33.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb5aac640 1 --2- 172.21.15.12:0/3365212951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb015d6f0 0x7f2fb015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:33.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb52ab640 1 --2- 172.21.15.12:0/3365212951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb00cf160 0x7f2fb0157f70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00c500 tx=0x7f2fac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:33.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fa5ffb640 1 -- 172.21.15.12:0/3365212951 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac005220 con 0x7f2fb00cf160 2023-10-30T12:53:33.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fa5ffb640 1 -- 172.21.15.12:0/3365212951 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac005380 con 0x7f2fb00cf160 2023-10-30T12:53:33.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fa5ffb640 1 -- 172.21.15.12:0/3365212951 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac007270 con 0x7f2fb00cf160 2023-10-30T12:53:33.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb750f640 1 -- 172.21.15.12:0/3365212951 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb00cec70 con 0x7f2fb00cf160 2023-10-30T12:53:33.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb750f640 1 -- 172.21.15.12:0/3365212951 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb01f2a60 con 0x7f2fb00cf160 2023-10-30T12:53:33.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.076+0000 7f2fb750f640 1 -- 172.21.15.12:0/3365212951 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb00b2a50 con 0x7f2fb00cf160 2023-10-30T12:53:33.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.080+0000 7f2fa5ffb640 1 -- 172.21.15.12:0/3365212951 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2fac0073d0 con 0x7f2fb00cf160 2023-10-30T12:53:33.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.080+0000 7f2fa5ffb640 1 --2- 172.21.15.12:0/3365212951 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f94066520 0x7f2f940689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:33.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.080+0000 7f2fa5ffb640 1 -- 172.21.15.12:0/3365212951 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f2fac083ab0 con 0x7f2fb00cf160 2023-10-30T12:53:33.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.080+0000 7f2fb4aaa640 1 --2- 172.21.15.12:0/3365212951 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f94066520 0x7f2f940689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:33.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.080+0000 7f2fb4aaa640 1 --2- 172.21.15.12:0/3365212951 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f94066520 0x7f2f940689e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00b2000 tx=0x7f2fa00aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:33.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.084+0000 7f2fa5ffb640 1 -- 172.21.15.12:0/3365212951 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2fac050ba0 con 0x7f2fb00cf160 2023-10-30T12:53:33.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.272+0000 7f2fb750f640 1 -- 172.21.15.12:0/3365212951 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fb004e170 con 0x7f2fb00cf160 2023-10-30T12:53:33.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.272+0000 7f2fa5ffb640 1 -- 172.21.15.12:0/3365212951 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5744 (secure 0 0 0) 0x7f2fac0541c0 con 0x7f2fb00cf160 2023-10-30T12:53:33.276 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:33.276 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:29.807534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:resolve","state_seq":34,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:33.277 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 90 2023-10-30T12:53:33.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.272+0000 7f2fb750f640 1 -- 172.21.15.12:0/3365212951 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f94066520 msgr2=0x7f2f940689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:33.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.272+0000 7f2fb750f640 1 --2- 172.21.15.12:0/3365212951 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f94066520 0x7f2f940689e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00b2000 tx=0x7f2fa00aa070 comp rx=0 tx=0).stop 2023-10-30T12:53:33.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.272+0000 7f2fb750f640 1 -- 172.21.15.12:0/3365212951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb00cf160 msgr2=0x7f2fb0157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:33.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.272+0000 7f2fb750f640 1 --2- 172.21.15.12:0/3365212951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb00cf160 0x7f2fb0157f70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00c500 tx=0x7f2fac00c9c0 comp rx=0 tx=0).stop 2023-10-30T12:53:33.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.272+0000 7f2fb750f640 1 -- 172.21.15.12:0/3365212951 shutdown_connections 2023-10-30T12:53:33.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.272+0000 7f2fb750f640 1 --2- 172.21.15.12:0/3365212951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb015d6f0 0x7f2fb015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:33.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.272+0000 7f2fb750f640 1 --2- 172.21.15.12:0/3365212951 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f94066520 0x7f2f940689e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:33.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.272+0000 7f2fb750f640 1 --2- 172.21.15.12:0/3365212951 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb01538a0 0x7f2fb01ef0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:33.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.272+0000 7f2fb750f640 1 --2- 172.21.15.12:0/3365212951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb00cf160 0x7f2fb0157f70 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:33.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.272+0000 7f2fb750f640 1 -- 172.21.15.12:0/3365212951 >> 172.21.15.12:0/3365212951 conn(0x7f2fb006bfe0 msgr2=0x7f2fb00c4900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:33.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.272+0000 7f2fb750f640 1 -- 172.21.15.12:0/3365212951 shutdown_connections 2023-10-30T12:53:33.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:33.276+0000 7f2fb750f640 1 -- 172.21.15.12:0/3365212951 wait complete. 2023-10-30T12:53:33.297 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 90, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 90, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5758": { "addr": "172.21.15.154:6839/2204547612", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 2204547612, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 2204547612, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 5758, "incarnation": 73, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_5877": { "addr": "172.21.15.12:6835/2306258157", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 2306258157, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 2306258157, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 5877, "incarnation": 81, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_6021": { "addr": "172.21.15.12:6836/3877967042", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 3877967042, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 3877967042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6021, "incarnation": 89, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:resolve", "state_seq": 34 } }, "last_failure": 0, "last_failure_osd_epoch": 54, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:53:29.807534+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6021, "mds_1": 5758, "mds_2": 5877 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:53:35.299 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:35.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.448+0000 7f4b46267640 1 Processor -- start 2023-10-30T12:53:35.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.448+0000 7f4b46267640 1 -- start start 2023-10-30T12:53:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b46267640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4b401538f0 0x7f4b40159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b46267640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b400ca6a0 con 0x7f4b4015cd50 2023-10-30T12:53:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b46267640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4015d300 con 0x7f4b400cefa0 2023-10-30T12:53:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b46267640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4015d5e0 con 0x7f4b401538f0 2023-10-30T12:53:35.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b44a64640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4b4015cd50 0x7f4b400c9f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58556/0 (socket says 172.21.15.12:58556) 2023-10-30T12:53:35.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b44a64640 1 -- 172.21.15.12:0/1779848312 learned_addr learned my addr 172.21.15.12:0/1779848312 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:53:35.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b45265640 1 --2- 172.21.15.12:0/1779848312 >> v2:172.21.15.154:3300/0 conn(0x7f4b401538f0 0x7f4b40159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:35.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b37fff640 1 -- 172.21.15.12:0/1779848312 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987079742 0 0) 0x7f4b400ca6a0 con 0x7f4b4015cd50 2023-10-30T12:53:35.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b37fff640 1 -- 172.21.15.12:0/1779848312 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b2c003530 con 0x7f4b4015cd50 2023-10-30T12:53:35.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b37fff640 1 -- 172.21.15.12:0/1779848312 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1833569651 0 0) 0x7f4b2c003530 con 0x7f4b4015cd50 2023-10-30T12:53:35.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b37fff640 1 -- 172.21.15.12:0/1779848312 >> v2:172.21.15.154:3300/0 conn(0x7f4b401538f0 msgr2=0x7f4b40159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:35.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b37fff640 1 --2- 172.21.15.12:0/1779848312 >> v2:172.21.15.154:3300/0 conn(0x7f4b401538f0 0x7f4b40159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:35.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b37fff640 1 -- 172.21.15.12:0/1779848312 >> v1:172.21.15.154:6789/0 conn(0x7f4b400cefa0 legacy=0x7f4b401640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:35.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b37fff640 1 -- 172.21.15.12:0/1779848312 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4015d8c0 con 0x7f4b4015cd50 2023-10-30T12:53:35.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b37fff640 1 -- 172.21.15.12:0/1779848312 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f4b30001680 con 0x7f4b4015cd50 2023-10-30T12:53:35.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b37fff640 1 -- 172.21.15.12:0/1779848312 >> v1:172.21.15.12:6789/0 conn(0x7f4b4015cd50 legacy=0x7f4b400c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:35.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b37fff640 1 --2- 172.21.15.12:0/1779848312 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b2c003980 0x7f4b2c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:35.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b37fff640 1 --2- 172.21.15.12:0/1779848312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b2c0047d0 0x7f4b2c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:35.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b37fff640 1 --2- 172.21.15.12:0/1779848312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b2c005190 0x7f4b2c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:35.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b37fff640 1 -- 172.21.15.12:0/1779848312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4015d5e0 con 0x7f4b2c0047d0 2023-10-30T12:53:35.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b37fff640 1 -- 172.21.15.12:0/1779848312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c00c040 con 0x7f4b2c005190 2023-10-30T12:53:35.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b37fff640 1 -- 172.21.15.12:0/1779848312 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c00c200 con 0x7f4b2c003980 2023-10-30T12:53:35.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b44a64640 1 --2- 172.21.15.12:0/1779848312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b2c0047d0 0x7f4b2c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:35.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b45265640 1 --2- 172.21.15.12:0/1779848312 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b2c003980 0x7f4b2c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:35.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b45a66640 1 --2- 172.21.15.12:0/1779848312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b2c005190 0x7f4b2c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:35.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b44a64640 1 -- 172.21.15.12:0/1779848312 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b2c003980 msgr2=0x7f4b2c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:35.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b44a64640 1 --2- 172.21.15.12:0/1779848312 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b2c003980 0x7f4b2c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:35.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b44a64640 1 -- 172.21.15.12:0/1779848312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b2c005190 msgr2=0x7f4b2c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:35.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b44a64640 1 --2- 172.21.15.12:0/1779848312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b2c005190 0x7f4b2c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:35.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b44a64640 1 -- 172.21.15.12:0/1779848312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c003530 con 0x7f4b2c0047d0 2023-10-30T12:53:35.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.452+0000 7f4b45a66640 1 -- 172.21.15.12:0/1779848312 reap_dead start 2023-10-30T12:53:35.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.456+0000 7f4b44a64640 1 --2- 172.21.15.12:0/1779848312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b2c0047d0 0x7f4b2c004c50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000c210 tx=0x7f4b3001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7952a13b5a65d264 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:35.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.456+0000 7f4b37fff640 1 -- 172.21.15.12:0/1779848312 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b300a5b40 con 0x7f4b2c0047d0 2023-10-30T12:53:35.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.456+0000 7f4b37fff640 1 -- 172.21.15.12:0/1779848312 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b300a5ca0 con 0x7f4b2c0047d0 2023-10-30T12:53:35.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.456+0000 7f4b37fff640 1 -- 172.21.15.12:0/1779848312 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30005b20 con 0x7f4b2c0047d0 2023-10-30T12:53:35.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.456+0000 7f4b46267640 1 -- 172.21.15.12:0/1779848312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b2c0047d0 msgr2=0x7f4b2c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:35.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.456+0000 7f4b46267640 1 --2- 172.21.15.12:0/1779848312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b2c0047d0 0x7f4b2c004c50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000c210 tx=0x7f4b3001cc40 comp rx=0 tx=0).stop 2023-10-30T12:53:35.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.456+0000 7f4b46267640 1 -- 172.21.15.12:0/1779848312 shutdown_connections 2023-10-30T12:53:35.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.456+0000 7f4b46267640 1 --2- 172.21.15.12:0/1779848312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b2c0047d0 0x7f4b2c004c50 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:35.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.456+0000 7f4b46267640 1 -- 172.21.15.12:0/1779848312 >> 172.21.15.12:0/1779848312 conn(0x7f4b4006a640 msgr2=0x7f4b40065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:35.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.456+0000 7f4b46267640 1 -- 172.21.15.12:0/1779848312 shutdown_connections 2023-10-30T12:53:35.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.456+0000 7f4b46267640 1 -- 172.21.15.12:0/1779848312 wait complete. 2023-10-30T12:53:35.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b46267640 1 Processor -- start 2023-10-30T12:53:35.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b46267640 1 -- start start 2023-10-30T12:53:35.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b46267640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b2c0047d0 0x7f4b400b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:35.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b46267640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b4013c820 0x7f4b401480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:35.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b46267640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b401411d0 0x7f4b4013bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:35.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b46267640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400b4320 con 0x7f4b4013c820 2023-10-30T12:53:35.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b46267640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400b4ee0 con 0x7f4b401411d0 2023-10-30T12:53:35.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b46267640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40142300 con 0x7f4b2c0047d0 2023-10-30T12:53:35.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b44a64640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b4013c820 0x7f4b401480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:35.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b44a64640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b4013c820 0x7f4b401480e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45204/0 (socket says 172.21.15.12:45204) 2023-10-30T12:53:35.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b44a64640 1 -- 172.21.15.12:0/3583230050 learned_addr learned my addr 172.21.15.12:0/3583230050 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:35.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b45265640 1 --2- 172.21.15.12:0/3583230050 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b2c0047d0 0x7f4b400b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:35.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b45a66640 1 --2- 172.21.15.12:0/3583230050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b401411d0 0x7f4b4013bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:35.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b44a64640 1 -- 172.21.15.12:0/3583230050 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b2c0047d0 msgr2=0x7f4b400b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:35.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b44a64640 1 --2- 172.21.15.12:0/3583230050 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b2c0047d0 0x7f4b400b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:35.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b44a64640 1 -- 172.21.15.12:0/3583230050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b401411d0 msgr2=0x7f4b4013bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:35.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b44a64640 1 --2- 172.21.15.12:0/3583230050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b401411d0 0x7f4b4013bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:35.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b44a64640 1 -- 172.21.15.12:0/3583230050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b40140d40 con 0x7f4b4013c820 2023-10-30T12:53:35.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b45a66640 1 --2- 172.21.15.12:0/3583230050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b401411d0 0x7f4b4013bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:35.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b44a64640 1 --2- 172.21.15.12:0/3583230050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b4013c820 0x7f4b401480e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000c210 tx=0x7f4b2c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:35.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b35ffb640 1 -- 172.21.15.12:0/3583230050 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b300b3070 con 0x7f4b4013c820 2023-10-30T12:53:35.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b35ffb640 1 -- 172.21.15.12:0/3583230050 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b30000ee0 con 0x7f4b4013c820 2023-10-30T12:53:35.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b46267640 1 -- 172.21.15.12:0/3583230050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b40142830 con 0x7f4b4013c820 2023-10-30T12:53:35.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b35ffb640 1 -- 172.21.15.12:0/3583230050 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30001640 con 0x7f4b4013c820 2023-10-30T12:53:35.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.460+0000 7f4b46267640 1 -- 172.21.15.12:0/3583230050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b40142550 con 0x7f4b4013c820 2023-10-30T12:53:35.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.464+0000 7f4b46267640 1 -- 172.21.15.12:0/3583230050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b40140ed0 con 0x7f4b4013c820 2023-10-30T12:53:35.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.464+0000 7f4b35ffb640 1 -- 172.21.15.12:0/3583230050 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4b30007140 con 0x7f4b4013c820 2023-10-30T12:53:35.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.464+0000 7f4b35ffb640 1 --2- 172.21.15.12:0/3583230050 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4b240664a0 0x7f4b24068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:35.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.464+0000 7f4b35ffb640 1 -- 172.21.15.12:0/3583230050 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f4b3011ddd0 con 0x7f4b4013c820 2023-10-30T12:53:35.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.464+0000 7f4b45265640 1 --2- 172.21.15.12:0/3583230050 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4b240664a0 0x7f4b24068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:35.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.464+0000 7f4b45265640 1 --2- 172.21.15.12:0/3583230050 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4b240664a0 0x7f4b24068960 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4b40074490 tx=0x7f4b3c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:35.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.468+0000 7f4b35ffb640 1 -- 172.21.15.12:0/3583230050 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4b300eae30 con 0x7f4b4013c820 2023-10-30T12:53:35.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.660+0000 7f4b46267640 1 -- 172.21.15.12:0/3583230050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b4004e170 con 0x7f4b4013c820 2023-10-30T12:53:35.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.660+0000 7f4b35ffb640 1 -- 172.21.15.12:0/3583230050 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5744 (secure 0 0 0) 0x7f4b300ee450 con 0x7f4b4013c820 2023-10-30T12:53:35.665 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:35.665 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:29.807534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:resolve","state_seq":34,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:35.665 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 90 2023-10-30T12:53:35.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.664+0000 7f4b46267640 1 -- 172.21.15.12:0/3583230050 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4b240664a0 msgr2=0x7f4b24068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.664+0000 7f4b46267640 1 --2- 172.21.15.12:0/3583230050 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4b240664a0 0x7f4b24068960 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4b40074490 tx=0x7f4b3c009190 comp rx=0 tx=0).stop 2023-10-30T12:53:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.664+0000 7f4b46267640 1 -- 172.21.15.12:0/3583230050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b4013c820 msgr2=0x7f4b401480e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.664+0000 7f4b46267640 1 --2- 172.21.15.12:0/3583230050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b4013c820 0x7f4b401480e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000c210 tx=0x7f4b2c005800 comp rx=0 tx=0).stop 2023-10-30T12:53:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.664+0000 7f4b46267640 1 -- 172.21.15.12:0/3583230050 shutdown_connections 2023-10-30T12:53:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.664+0000 7f4b46267640 1 --2- 172.21.15.12:0/3583230050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b401411d0 0x7f4b4013bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.664+0000 7f4b46267640 1 --2- 172.21.15.12:0/3583230050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b4013c820 0x7f4b401480e0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.664+0000 7f4b46267640 1 --2- 172.21.15.12:0/3583230050 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4b240664a0 0x7f4b24068960 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.664+0000 7f4b46267640 1 --2- 172.21.15.12:0/3583230050 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b2c0047d0 0x7f4b400b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.664+0000 7f4b46267640 1 -- 172.21.15.12:0/3583230050 >> 172.21.15.12:0/3583230050 conn(0x7f4b4006a640 msgr2=0x7f4b400c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.664+0000 7f4b46267640 1 -- 172.21.15.12:0/3583230050 shutdown_connections 2023-10-30T12:53:35.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:35.664+0000 7f4b46267640 1 -- 172.21.15.12:0/3583230050 wait complete. 2023-10-30T12:53:35.686 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:37.360 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:53:37.365 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:53:37.687 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.832+0000 7fdd210ea640 1 Processor -- start 2023-10-30T12:53:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.832+0000 7fdd210ea640 1 -- start start 2023-10-30T12:53:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.836+0000 7fdd210ea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd1c15a410 0x7fdd1c06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.836+0000 7fdd210ea640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdd1c06cb80 0x7fdd1c06d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.836+0000 7fdd210ea640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd1c06d550 con 0x7fdd1c069e90 2023-10-30T12:53:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.836+0000 7fdd210ea640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1c0c8570 con 0x7fdd1c06cb80 2023-10-30T12:53:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.836+0000 7fdd210ea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c15a850 con 0x7fdd1c15a410 2023-10-30T12:53:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.836+0000 7fdd1ad9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdd1c069e90 0x7fdd1c0c9cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58562/0 (socket says 172.21.15.12:58562) 2023-10-30T12:53:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.836+0000 7fdd1ad9d640 1 -- 172.21.15.12:0/2633335142 learned_addr learned my addr 172.21.15.12:0/2633335142 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:53:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.836+0000 7fdd1b59e640 1 --2- 172.21.15.12:0/2633335142 >> v2:172.21.15.12:3300/0 conn(0x7fdd1c06cb80 0x7fdd1c06d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.836+0000 7fdd1a59c640 1 --2- 172.21.15.12:0/2633335142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd1c15a410 0x7fdd1c06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.836+0000 7fdd1b59e640 1 -- 172.21.15.12:0/2633335142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd1c15a410 msgr2=0x7fdd1c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.836+0000 7fdd1b59e640 1 --2- 172.21.15.12:0/2633335142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd1c15a410 0x7fdd1c06a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.836+0000 7fdd1b59e640 1 -- 172.21.15.12:0/2633335142 >> v1:172.21.15.12:6789/0 conn(0x7fdd1c069e90 legacy=0x7fdd1c0c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.836+0000 7fdd1b59e640 1 -- 172.21.15.12:0/2633335142 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c0c6c10 con 0x7fdd1c06cb80 2023-10-30T12:53:37.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.840+0000 7fdd1b59e640 1 --2- 172.21.15.12:0/2633335142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1c06cb80 0x7fdd1c06d010 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009d90 tx=0x7fdd1401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3853b6801e96fe13 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:37.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.840+0000 7fdd19d9b640 1 -- 172.21.15.12:0/2633335142 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd140a5810 con 0x7fdd1c06cb80 2023-10-30T12:53:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.840+0000 7fdd19d9b640 1 -- 172.21.15.12:0/2633335142 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd140a5970 con 0x7fdd1c06cb80 2023-10-30T12:53:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.840+0000 7fdd19d9b640 1 -- 172.21.15.12:0/2633335142 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd140a5d00 con 0x7fdd1c06cb80 2023-10-30T12:53:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.840+0000 7fdd210ea640 1 -- 172.21.15.12:0/2633335142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1c06cb80 msgr2=0x7fdd1c06d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.840+0000 7fdd210ea640 1 --2- 172.21.15.12:0/2633335142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1c06cb80 0x7fdd1c06d010 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009d90 tx=0x7fdd1401c950 comp rx=0 tx=0).stop 2023-10-30T12:53:37.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.840+0000 7fdd210ea640 1 -- 172.21.15.12:0/2633335142 shutdown_connections 2023-10-30T12:53:37.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.840+0000 7fdd210ea640 1 --2- 172.21.15.12:0/2633335142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1c06cb80 0x7fdd1c06d010 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:37.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.840+0000 7fdd210ea640 1 --2- 172.21.15.12:0/2633335142 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd1c15a410 0x7fdd1c06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:37.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.840+0000 7fdd210ea640 1 -- 172.21.15.12:0/2633335142 >> 172.21.15.12:0/2633335142 conn(0x7fdd1c06dfd0 msgr2=0x7fdd1c06e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:37.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.840+0000 7fdd210ea640 1 -- 172.21.15.12:0/2633335142 shutdown_connections 2023-10-30T12:53:37.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.840+0000 7fdd210ea640 1 -- 172.21.15.12:0/2633335142 wait complete. 2023-10-30T12:53:37.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.840+0000 7fdd210ea640 1 Processor -- start 2023-10-30T12:53:37.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.840+0000 7fdd210ea640 1 -- start start 2023-10-30T12:53:37.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd210ea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd1c069e90 0x7fdd1c1f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:37.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd210ea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1c06cb80 0x7fdd1c071120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:37.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd210ea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd1c15a410 0x7fdd1c071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:37.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd210ea640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c072010 con 0x7fdd1c06cb80 2023-10-30T12:53:37.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd210ea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c1f5dd0 con 0x7fdd1c069e90 2023-10-30T12:53:37.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd210ea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c1f1e30 con 0x7fdd1c15a410 2023-10-30T12:53:37.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd1a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1c06cb80 0x7fdd1c071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:37.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd1a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1c06cb80 0x7fdd1c071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45214/0 (socket says 172.21.15.12:45214) 2023-10-30T12:53:37.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd1a59c640 1 -- 172.21.15.12:0/1900054114 learned_addr learned my addr 172.21.15.12:0/1900054114 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:37.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd1ad9d640 1 --2- 172.21.15.12:0/1900054114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd1c069e90 0x7fdd1c1f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:37.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd1b59e640 1 --2- 172.21.15.12:0/1900054114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd1c15a410 0x7fdd1c071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:37.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd1a59c640 1 -- 172.21.15.12:0/1900054114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd1c15a410 msgr2=0x7fdd1c071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:37.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd1a59c640 1 --2- 172.21.15.12:0/1900054114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd1c15a410 0x7fdd1c071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:37.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd1a59c640 1 -- 172.21.15.12:0/1900054114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd1c069e90 msgr2=0x7fdd1c1f22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:37.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd1a59c640 1 --2- 172.21.15.12:0/1900054114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd1c069e90 0x7fdd1c1f22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:37.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd1a59c640 1 -- 172.21.15.12:0/1900054114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd14009a70 con 0x7fdd1c06cb80 2023-10-30T12:53:37.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd1a59c640 1 --2- 172.21.15.12:0/1900054114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1c06cb80 0x7fdd1c071120 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000d620 tx=0x7fdd1000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:37.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd037fe640 1 -- 172.21.15.12:0/1900054114 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000b620 con 0x7fdd1c06cb80 2023-10-30T12:53:37.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd037fe640 1 -- 172.21.15.12:0/1900054114 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1000b780 con 0x7fdd1c06cb80 2023-10-30T12:53:37.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd037fe640 1 -- 172.21.15.12:0/1900054114 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000bb30 con 0x7fdd1c06cb80 2023-10-30T12:53:37.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd210ea640 1 -- 172.21.15.12:0/1900054114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c150a60 con 0x7fdd1c06cb80 2023-10-30T12:53:37.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd210ea640 1 -- 172.21.15.12:0/1900054114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c072260 con 0x7fdd1c06cb80 2023-10-30T12:53:37.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd210ea640 1 -- 172.21.15.12:0/1900054114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1c04e170 con 0x7fdd1c06cb80 2023-10-30T12:53:37.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.844+0000 7fdd037fe640 1 -- 172.21.15.12:0/1900054114 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdd1000bc90 con 0x7fdd1c06cb80 2023-10-30T12:53:37.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.848+0000 7fdd037fe640 1 --2- 172.21.15.12:0/1900054114 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdd080664d0 0x7fdd08068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:37.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.848+0000 7fdd037fe640 1 -- 172.21.15.12:0/1900054114 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fdd100845f0 con 0x7fdd1c06cb80 2023-10-30T12:53:37.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.848+0000 7fdd1ad9d640 1 --2- 172.21.15.12:0/1900054114 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdd080664d0 0x7fdd08068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:37.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.848+0000 7fdd1ad9d640 1 --2- 172.21.15.12:0/1900054114 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdd080664d0 0x7fdd08068990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c005890 tx=0x7fdd0c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:37.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:37.852+0000 7fdd037fe640 1 -- 172.21.15.12:0/1900054114 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdd10051710 con 0x7fdd1c06cb80 2023-10-30T12:53:38.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:38.040+0000 7fdd210ea640 1 -- 172.21.15.12:0/1900054114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd1c0c6350 con 0x7fdd1c06cb80 2023-10-30T12:53:38.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:38.040+0000 7fdd037fe640 1 -- 172.21.15.12:0/1900054114 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+5746 (secure 0 0 0) 0x7fdd10007380 con 0x7fdd1c06cb80 2023-10-30T12:53:38.044 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:38.045 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:35.630349+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:reconnect","state_seq":36,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:38.045 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 91 2023-10-30T12:53:38.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:38.044+0000 7fdd210ea640 1 -- 172.21.15.12:0/1900054114 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdd080664d0 msgr2=0x7fdd08068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:38.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:38.044+0000 7fdd210ea640 1 --2- 172.21.15.12:0/1900054114 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdd080664d0 0x7fdd08068990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c005890 tx=0x7fdd0c009040 comp rx=0 tx=0).stop 2023-10-30T12:53:38.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:38.044+0000 7fdd210ea640 1 -- 172.21.15.12:0/1900054114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1c06cb80 msgr2=0x7fdd1c071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:38.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:38.044+0000 7fdd210ea640 1 --2- 172.21.15.12:0/1900054114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1c06cb80 0x7fdd1c071120 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000d620 tx=0x7fdd1000dae0 comp rx=0 tx=0).stop 2023-10-30T12:53:38.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:38.044+0000 7fdd210ea640 1 -- 172.21.15.12:0/1900054114 shutdown_connections 2023-10-30T12:53:38.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:38.044+0000 7fdd210ea640 1 --2- 172.21.15.12:0/1900054114 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdd080664d0 0x7fdd08068990 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:38.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:38.044+0000 7fdd210ea640 1 --2- 172.21.15.12:0/1900054114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd1c15a410 0x7fdd1c071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:38.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:38.044+0000 7fdd210ea640 1 --2- 172.21.15.12:0/1900054114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1c06cb80 0x7fdd1c071120 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:38.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:38.044+0000 7fdd210ea640 1 --2- 172.21.15.12:0/1900054114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd1c069e90 0x7fdd1c1f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:38.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:38.044+0000 7fdd210ea640 1 -- 172.21.15.12:0/1900054114 >> 172.21.15.12:0/1900054114 conn(0x7fdd1c06dfd0 msgr2=0x7fdd1c158fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:38.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:38.044+0000 7fdd210ea640 1 -- 172.21.15.12:0/1900054114 shutdown_connections 2023-10-30T12:53:38.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:38.044+0000 7fdd210ea640 1 -- 172.21.15.12:0/1900054114 wait complete. 2023-10-30T12:53:38.065 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:40.066 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:40.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.216+0000 7f1675cea640 1 Processor -- start 2023-10-30T12:53:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.216+0000 7f1675cea640 1 -- start start 2023-10-30T12:53:40.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f1675cea640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f16700c6b80 0x7f16700bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:40.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f1675cea640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f16700c6f60 0x7f16700c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:40.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f1675cea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16700c8430 0x7f167013ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:40.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f1675cea640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16700c0a50 con 0x7f16700c6f60 2023-10-30T12:53:40.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f1675cea640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f167006f6e0 con 0x7f16700c6b80 2023-10-30T12:53:40.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f1675cea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167006f9c0 con 0x7f16700c8430 2023-10-30T12:53:40.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f166effd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f16700c6f60 0x7f16700c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:40.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f166effd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f16700c6f60 0x7f16700c0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45224/0 (socket says 172.21.15.12:45224) 2023-10-30T12:53:40.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f166effd640 1 -- 172.21.15.12:0/2059682284 learned_addr learned my addr 172.21.15.12:0/2059682284 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:40.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f166f7fe640 1 --2- 172.21.15.12:0/2059682284 >> v2:172.21.15.154:3300/0 conn(0x7f16700c6b80 0x7f16700bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:40.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f166ffff640 1 --2- 172.21.15.12:0/2059682284 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16700c8430 0x7f167013ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:40.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f166effd640 1 -- 172.21.15.12:0/2059682284 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16700c8430 msgr2=0x7f167013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:40.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f166effd640 1 --2- 172.21.15.12:0/2059682284 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16700c8430 0x7f167013ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:40.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f166effd640 1 -- 172.21.15.12:0/2059682284 >> v2:172.21.15.154:3300/0 conn(0x7f16700c6b80 msgr2=0x7f16700bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:40.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f166effd640 1 --2- 172.21.15.12:0/2059682284 >> v2:172.21.15.154:3300/0 conn(0x7f16700c6b80 0x7f16700bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:40.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.220+0000 7f166effd640 1 -- 172.21.15.12:0/2059682284 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167006fca0 con 0x7f16700c6f60 2023-10-30T12:53:40.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.224+0000 7f166effd640 1 --2- 172.21.15.12:0/2059682284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16700c6f60 0x7f16700c0400 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f167006f1b0 tx=0x7f166401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9557018cb39bd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:40.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.224+0000 7f166e7fc640 1 -- 172.21.15.12:0/2059682284 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16640ac630 con 0x7f16700c6f60 2023-10-30T12:53:40.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.224+0000 7f166e7fc640 1 -- 172.21.15.12:0/2059682284 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16640ac790 con 0x7f16700c6f60 2023-10-30T12:53:40.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.224+0000 7f166e7fc640 1 -- 172.21.15.12:0/2059682284 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16640acb20 con 0x7f16700c6f60 2023-10-30T12:53:40.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.224+0000 7f1675cea640 1 -- 172.21.15.12:0/2059682284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16700c6f60 msgr2=0x7f16700c0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:40.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.224+0000 7f1675cea640 1 --2- 172.21.15.12:0/2059682284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16700c6f60 0x7f16700c0400 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f167006f1b0 tx=0x7f166401c500 comp rx=0 tx=0).stop 2023-10-30T12:53:40.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.224+0000 7f1675cea640 1 -- 172.21.15.12:0/2059682284 shutdown_connections 2023-10-30T12:53:40.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.224+0000 7f1675cea640 1 --2- 172.21.15.12:0/2059682284 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16700c8430 0x7f167013ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:40.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.224+0000 7f1675cea640 1 --2- 172.21.15.12:0/2059682284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16700c6f60 0x7f16700c0400 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:40.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.224+0000 7f1675cea640 1 --2- 172.21.15.12:0/2059682284 >> v2:172.21.15.154:3300/0 conn(0x7f16700c6b80 0x7f16700bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:40.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.224+0000 7f1675cea640 1 -- 172.21.15.12:0/2059682284 >> 172.21.15.12:0/2059682284 conn(0x7f16700c7910 msgr2=0x7f16700c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:40.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.224+0000 7f1675cea640 1 -- 172.21.15.12:0/2059682284 shutdown_connections 2023-10-30T12:53:40.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.224+0000 7f1675cea640 1 -- 172.21.15.12:0/2059682284 wait complete. 2023-10-30T12:53:40.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.224+0000 7f1675cea640 1 Processor -- start 2023-10-30T12:53:40.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.224+0000 7f1675cea640 1 -- start start 2023-10-30T12:53:40.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f1675cea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16700c6b80 0x7f1670139ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:40.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f1675cea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16700c6f60 0x7f1670138620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:40.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f1675cea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16700c8430 0x7f1670138b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:40.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f1675cea640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1670155f00 con 0x7f16700c6f60 2023-10-30T12:53:40.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f1675cea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16701f59f0 con 0x7f16700c8430 2023-10-30T12:53:40.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f1675cea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16701f5cd0 con 0x7f16700c6b80 2023-10-30T12:53:40.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f166effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16700c6f60 0x7f1670138620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:40.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f166effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16700c6f60 0x7f1670138620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45228/0 (socket says 172.21.15.12:45228) 2023-10-30T12:53:40.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f166effd640 1 -- 172.21.15.12:0/3193613343 learned_addr learned my addr 172.21.15.12:0/3193613343 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:40.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f166ffff640 1 --2- 172.21.15.12:0/3193613343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16700c8430 0x7f1670138b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:40.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f166f7fe640 1 --2- 172.21.15.12:0/3193613343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16700c6b80 0x7f1670139ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:40.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f166effd640 1 -- 172.21.15.12:0/3193613343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16700c6b80 msgr2=0x7f1670139ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:40.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f166effd640 1 --2- 172.21.15.12:0/3193613343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16700c6b80 0x7f1670139ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:40.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f166effd640 1 -- 172.21.15.12:0/3193613343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16700c8430 msgr2=0x7f1670138b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:40.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f166effd640 1 --2- 172.21.15.12:0/3193613343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16700c8430 0x7f1670138b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:40.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f166effd640 1 -- 172.21.15.12:0/3193613343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16640094d0 con 0x7f16700c6f60 2023-10-30T12:53:40.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f166effd640 1 --2- 172.21.15.12:0/3193613343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16700c6f60 0x7f1670138620 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f166401abb0 tx=0x7f16640192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:40.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f1657fff640 1 -- 172.21.15.12:0/3193613343 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16640b5640 con 0x7f16700c6f60 2023-10-30T12:53:40.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f1657fff640 1 -- 172.21.15.12:0/3193613343 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16640b57a0 con 0x7f16700c6f60 2023-10-30T12:53:40.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f1675cea640 1 -- 172.21.15.12:0/3193613343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16701f5fb0 con 0x7f16700c6f60 2023-10-30T12:53:40.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f1657fff640 1 -- 172.21.15.12:0/3193613343 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16640b5b10 con 0x7f16700c6f60 2023-10-30T12:53:40.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f1675cea640 1 -- 172.21.15.12:0/3193613343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16701f6570 con 0x7f16700c6f60 2023-10-30T12:53:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.228+0000 7f1675cea640 1 -- 172.21.15.12:0/3193613343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16701f6760 con 0x7f16700c6f60 2023-10-30T12:53:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.232+0000 7f1657fff640 1 -- 172.21.15.12:0/3193613343 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f16640b3070 con 0x7f16700c6f60 2023-10-30T12:53:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.232+0000 7f1657fff640 1 --2- 172.21.15.12:0/3193613343 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f165c0664a0 0x7f165c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.232+0000 7f1657fff640 1 -- 172.21.15.12:0/3193613343 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f166411e310 con 0x7f16700c6f60 2023-10-30T12:53:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.232+0000 7f166f7fe640 1 --2- 172.21.15.12:0/3193613343 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f165c0664a0 0x7f165c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.232+0000 7f166f7fe640 1 --2- 172.21.15.12:0/3193613343 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f165c0664a0 0x7f165c068960 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1660009ae0 tx=0x7f1660009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.236+0000 7f1657fff640 1 -- 172.21.15.12:0/3193613343 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f16640eb420 con 0x7f16700c6f60 2023-10-30T12:53:40.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.424+0000 7f1675cea640 1 -- 172.21.15.12:0/3193613343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f167004e170 con 0x7f16700c6f60 2023-10-30T12:53:40.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.424+0000 7f1657fff640 1 -- 172.21.15.12:0/3193613343 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+5745 (secure 0 0 0) 0x7f16640eea40 con 0x7f16700c6f60 2023-10-30T12:53:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:40.429 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":92,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:38.529414+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:reconnect","state_seq":36,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:40.429 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 92 2023-10-30T12:53:40.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.428+0000 7f1675cea640 1 -- 172.21.15.12:0/3193613343 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f165c0664a0 msgr2=0x7f165c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:40.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.428+0000 7f1675cea640 1 --2- 172.21.15.12:0/3193613343 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f165c0664a0 0x7f165c068960 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1660009ae0 tx=0x7f1660009190 comp rx=0 tx=0).stop 2023-10-30T12:53:40.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.428+0000 7f1675cea640 1 -- 172.21.15.12:0/3193613343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16700c6f60 msgr2=0x7f1670138620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:40.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.428+0000 7f1675cea640 1 --2- 172.21.15.12:0/3193613343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16700c6f60 0x7f1670138620 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f166401abb0 tx=0x7f16640192a0 comp rx=0 tx=0).stop 2023-10-30T12:53:40.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.428+0000 7f1675cea640 1 -- 172.21.15.12:0/3193613343 shutdown_connections 2023-10-30T12:53:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.428+0000 7f1675cea640 1 --2- 172.21.15.12:0/3193613343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16700c8430 0x7f1670138b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.428+0000 7f1675cea640 1 --2- 172.21.15.12:0/3193613343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16700c6f60 0x7f1670138620 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.428+0000 7f1675cea640 1 --2- 172.21.15.12:0/3193613343 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f165c0664a0 0x7f165c068960 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.428+0000 7f1675cea640 1 --2- 172.21.15.12:0/3193613343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16700c6b80 0x7f1670139ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.428+0000 7f1675cea640 1 -- 172.21.15.12:0/3193613343 >> 172.21.15.12:0/3193613343 conn(0x7f16700c7910 msgr2=0x7f16700c18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:40.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.428+0000 7f1675cea640 1 -- 172.21.15.12:0/3193613343 shutdown_connections 2023-10-30T12:53:40.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:40.428+0000 7f1675cea640 1 -- 172.21.15.12:0/3193613343 wait complete. 2023-10-30T12:53:40.450 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:42.452 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:42.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd72b63640 1 Processor -- start 2023-10-30T12:53:42.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd72b63640 1 -- start start 2023-10-30T12:53:42.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd72b63640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd6c1538f0 0x7efd6c159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:42.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd72b63640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd6c0ca6a0 con 0x7efd6c0cefa0 2023-10-30T12:53:42.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd72b63640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd6c15d300 con 0x7efd6c15cd50 2023-10-30T12:53:42.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd72b63640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c15d5e0 con 0x7efd6c1538f0 2023-10-30T12:53:42.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd72362640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7efd6c0cefa0 0x7efd6c1640e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:57292/0 (socket says 172.21.15.12:57292) 2023-10-30T12:53:42.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd72362640 1 -- 172.21.15.12:0/1281784773 learned_addr learned my addr 172.21.15.12:0/1281784773 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:53:42.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd71b61640 1 --2- 172.21.15.12:0/1281784773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd6c1538f0 0x7efd6c159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:42.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd70b5f640 1 -- 172.21.15.12:0/1281784773 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347238549 0 0) 0x7efd6c0ca6a0 con 0x7efd6c0cefa0 2023-10-30T12:53:42.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd70b5f640 1 -- 172.21.15.12:0/1281784773 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd58003530 con 0x7efd6c0cefa0 2023-10-30T12:53:42.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd70b5f640 1 -- 172.21.15.12:0/1281784773 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2197574124 0 0) 0x7efd58003530 con 0x7efd6c0cefa0 2023-10-30T12:53:42.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd70b5f640 1 -- 172.21.15.12:0/1281784773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd6c1538f0 msgr2=0x7efd6c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:42.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd70b5f640 1 --2- 172.21.15.12:0/1281784773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd6c1538f0 0x7efd6c159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:42.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd70b5f640 1 -- 172.21.15.12:0/1281784773 >> v1:172.21.15.154:6789/0 conn(0x7efd6c15cd50 legacy=0x7efd6c0c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:42.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd70b5f640 1 -- 172.21.15.12:0/1281784773 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c15d8c0 con 0x7efd6c0cefa0 2023-10-30T12:53:42.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd70b5f640 1 -- 172.21.15.12:0/1281784773 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7efd64001a00 con 0x7efd6c0cefa0 2023-10-30T12:53:42.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd70b5f640 1 -- 172.21.15.12:0/1281784773 >> v1:172.21.15.12:6789/0 conn(0x7efd6c0cefa0 legacy=0x7efd6c1640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:42.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd70b5f640 1 --2- 172.21.15.12:0/1281784773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd580039b0 0x7efd58003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:42.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd70b5f640 1 --2- 172.21.15.12:0/1281784773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd6c1538f0 0x7efd580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:42.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd70b5f640 1 --2- 172.21.15.12:0/1281784773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd58004d10 0x7efd58005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:42.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd70b5f640 1 -- 172.21.15.12:0/1281784773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c15d5e0 con 0x7efd580039b0 2023-10-30T12:53:42.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd70b5f640 1 -- 172.21.15.12:0/1281784773 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5800b990 con 0x7efd58004d10 2023-10-30T12:53:42.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd70b5f640 1 -- 172.21.15.12:0/1281784773 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5800bb50 con 0x7efd6c1538f0 2023-10-30T12:53:42.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd71b61640 1 --2- 172.21.15.12:0/1281784773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd580039b0 0x7efd58003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:42.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.604+0000 7efd71360640 1 --2- 172.21.15.12:0/1281784773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd6c1538f0 0x7efd580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:42.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.608+0000 7efd72362640 1 --2- 172.21.15.12:0/1281784773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd58004d10 0x7efd58005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:42.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.608+0000 7efd71b61640 1 -- 172.21.15.12:0/1281784773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd6c1538f0 msgr2=0x7efd580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:42.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.608+0000 7efd71b61640 1 --2- 172.21.15.12:0/1281784773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd6c1538f0 0x7efd580047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:42.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.608+0000 7efd71b61640 1 -- 172.21.15.12:0/1281784773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd58004d10 msgr2=0x7efd58005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:42.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.608+0000 7efd71b61640 1 --2- 172.21.15.12:0/1281784773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd58004d10 0x7efd58005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:42.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.608+0000 7efd71b61640 1 -- 172.21.15.12:0/1281784773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd58003530 con 0x7efd580039b0 2023-10-30T12:53:42.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.608+0000 7efd71b61640 1 --2- 172.21.15.12:0/1281784773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd580039b0 0x7efd58003d90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efd68009ae0 tx=0x7efd6801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35123034a5ed8669 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:42.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.608+0000 7efd70b5f640 1 -- 172.21.15.12:0/1281784773 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd680a5580 con 0x7efd580039b0 2023-10-30T12:53:42.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.608+0000 7efd70b5f640 1 -- 172.21.15.12:0/1281784773 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd680a56e0 con 0x7efd580039b0 2023-10-30T12:53:42.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.608+0000 7efd70b5f640 1 -- 172.21.15.12:0/1281784773 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd680a5a90 con 0x7efd580039b0 2023-10-30T12:53:42.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.608+0000 7efd72b63640 1 -- 172.21.15.12:0/1281784773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd580039b0 msgr2=0x7efd58003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:42.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.608+0000 7efd72b63640 1 --2- 172.21.15.12:0/1281784773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd580039b0 0x7efd58003d90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efd68009ae0 tx=0x7efd6801c990 comp rx=0 tx=0).stop 2023-10-30T12:53:42.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.608+0000 7efd72362640 1 -- 172.21.15.12:0/1281784773 reap_dead start 2023-10-30T12:53:42.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.608+0000 7efd72b63640 1 -- 172.21.15.12:0/1281784773 shutdown_connections 2023-10-30T12:53:42.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.608+0000 7efd72b63640 1 -- 172.21.15.12:0/1281784773 >> 172.21.15.12:0/1281784773 conn(0x7efd6c06a640 msgr2=0x7efd6c065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:42.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd72b63640 1 -- 172.21.15.12:0/1281784773 shutdown_connections 2023-10-30T12:53:42.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd72b63640 1 -- 172.21.15.12:0/1281784773 wait complete. 2023-10-30T12:53:42.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd72b63640 1 Processor -- start 2023-10-30T12:53:42.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd72b63640 1 -- start start 2023-10-30T12:53:42.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd72b63640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd580039b0 0x7efd6c15c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:42.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd72b63640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd6c0b2ea0 0x7efd6c0b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:42.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd72b63640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd6c1eed60 0x7efd6c1ef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:42.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd72b63640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c152170 con 0x7efd580039b0 2023-10-30T12:53:42.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd72b63640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c152c20 con 0x7efd6c1eed60 2023-10-30T12:53:42.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd72b63640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c1ef720 con 0x7efd6c0b2ea0 2023-10-30T12:53:42.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd71b61640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd580039b0 0x7efd6c15c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:42.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd71b61640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd580039b0 0x7efd6c15c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52016/0 (socket says 172.21.15.12:52016) 2023-10-30T12:53:42.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd71b61640 1 -- 172.21.15.12:0/3456001090 learned_addr learned my addr 172.21.15.12:0/3456001090 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:42.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd71360640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd6c0b2ea0 0x7efd6c0b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:42.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd72362640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd6c1eed60 0x7efd6c1ef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:42.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd71b61640 1 -- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd6c0b2ea0 msgr2=0x7efd6c0b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:42.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd71b61640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd6c0b2ea0 0x7efd6c0b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:42.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd71b61640 1 -- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd6c1eed60 msgr2=0x7efd6c1ef1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:42.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd71b61640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd6c1eed60 0x7efd6c1ef1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:42.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd71b61640 1 -- 172.21.15.12:0/3456001090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c15c8a0 con 0x7efd580039b0 2023-10-30T12:53:42.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd71360640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd6c0b2ea0 0x7efd6c0b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:42.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd72362640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd6c1eed60 0x7efd6c1ef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:42.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd71b61640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd580039b0 0x7efd6c15c060 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7efd68009ae0 tx=0x7efd58005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:42.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.612+0000 7efd627fc640 1 -- 172.21.15.12:0/3456001090 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd680bc070 con 0x7efd580039b0 2023-10-30T12:53:42.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.616+0000 7efd627fc640 1 -- 172.21.15.12:0/3456001090 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd68004020 con 0x7efd580039b0 2023-10-30T12:53:42.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.616+0000 7efd627fc640 1 -- 172.21.15.12:0/3456001090 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd680a5580 con 0x7efd580039b0 2023-10-30T12:53:42.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.616+0000 7efd72b63640 1 -- 172.21.15.12:0/3456001090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6c152380 con 0x7efd580039b0 2023-10-30T12:53:42.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.616+0000 7efd72b63640 1 -- 172.21.15.12:0/3456001090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6c152940 con 0x7efd580039b0 2023-10-30T12:53:42.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.616+0000 7efd72b63640 1 -- 172.21.15.12:0/3456001090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6c04e170 con 0x7efd580039b0 2023-10-30T12:53:42.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.616+0000 7efd627fc640 1 -- 172.21.15.12:0/3456001090 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efd680b4040 con 0x7efd580039b0 2023-10-30T12:53:42.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.616+0000 7efd627fc640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efd500664a0 0x7efd50068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:42.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.616+0000 7efd627fc640 1 -- 172.21.15.12:0/3456001090 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7efd6811e170 con 0x7efd580039b0 2023-10-30T12:53:42.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.616+0000 7efd71360640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efd500664a0 0x7efd50068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:42.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.620+0000 7efd71360640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efd500664a0 0x7efd50068960 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0b1790 tx=0x7efd5c00a9d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:42.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.620+0000 7efd627fc640 1 -- 172.21.15.12:0/3456001090 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efd680eb260 con 0x7efd580039b0 2023-10-30T12:53:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.816+0000 7efd72b63640 1 -- 172.21.15.12:0/3456001090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd6c1ee980 con 0x7efd580039b0 2023-10-30T12:53:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.816+0000 7efd627fc640 1 -- 172.21.15.12:0/3456001090 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+5742 (secure 0 0 0) 0x7efd680ee880 con 0x7efd580039b0 2023-10-30T12:53:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:42.820 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:41.646061+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:rejoin","state_seq":38,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 93 2023-10-30T12:53:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.816+0000 7efd72b63640 1 -- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efd500664a0 msgr2=0x7efd50068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.816+0000 7efd72b63640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efd500664a0 0x7efd50068960 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0b1790 tx=0x7efd5c00a9d0 comp rx=0 tx=0).stop 2023-10-30T12:53:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.816+0000 7efd72b63640 1 -- 172.21.15.12:0/3456001090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd580039b0 msgr2=0x7efd6c15c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.816+0000 7efd72b63640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd580039b0 0x7efd6c15c060 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7efd68009ae0 tx=0x7efd58005800 comp rx=0 tx=0).stop 2023-10-30T12:53:42.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.816+0000 7efd72b63640 1 -- 172.21.15.12:0/3456001090 shutdown_connections 2023-10-30T12:53:42.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.816+0000 7efd72b63640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd6c1eed60 0x7efd6c1ef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:42.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.816+0000 7efd72b63640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7efd500664a0 0x7efd50068960 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:42.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.816+0000 7efd72b63640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd6c0b2ea0 0x7efd6c0b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:42.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.816+0000 7efd72b63640 1 --2- 172.21.15.12:0/3456001090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd580039b0 0x7efd6c15c060 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:42.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.816+0000 7efd72b63640 1 -- 172.21.15.12:0/3456001090 >> 172.21.15.12:0/3456001090 conn(0x7efd6c06a640 msgr2=0x7efd6c1490e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:42.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.816+0000 7efd72b63640 1 -- 172.21.15.12:0/3456001090 shutdown_connections 2023-10-30T12:53:42.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:42.816+0000 7efd72b63640 1 -- 172.21.15.12:0/3456001090 wait complete. 2023-10-30T12:53:42.841 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:44.843 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:44.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.992+0000 7f1b939ab640 1 Processor -- start 2023-10-30T12:53:44.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.992+0000 7f1b939ab640 1 -- start start 2023-10-30T12:53:44.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b939ab640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b8c153bf0 0x7f1b8c071260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:44.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b939ab640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c0c6fe0 0x7f1b8c0c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:44.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b939ab640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b8c0700b0 con 0x7f1b8c0c79e0 2023-10-30T12:53:44.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b939ab640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b8c0c6810 con 0x7f1b8c153bf0 2023-10-30T12:53:44.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b939ab640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c159900 con 0x7f1b8c0c6fe0 2023-10-30T12:53:45.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b91f48640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f1b8c0c79e0 0x7f1b8c0cb570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:57298/0 (socket says 172.21.15.12:57298) 2023-10-30T12:53:45.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b91f48640 1 -- 172.21.15.12:0/2723435255 learned_addr learned my addr 172.21.15.12:0/2723435255 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:53:45.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b91747640 1 --2- 172.21.15.12:0/2723435255 >> v2:172.21.15.154:3300/0 conn(0x7f1b8c153bf0 0x7f1b8c071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:45.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b90f46640 1 --2- 172.21.15.12:0/2723435255 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c0c6fe0 0x7f1b8c0c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:45.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 -- 172.21.15.12:0/2723435255 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922879807 0 0) 0x7f1b8c0700b0 con 0x7f1b8c0c79e0 2023-10-30T12:53:45.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 -- 172.21.15.12:0/2723435255 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b78003530 con 0x7f1b8c0c79e0 2023-10-30T12:53:45.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 -- 172.21.15.12:0/2723435255 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1881970033 0 0) 0x7f1b78003530 con 0x7f1b8c0c79e0 2023-10-30T12:53:45.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 -- 172.21.15.12:0/2723435255 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c0c6fe0 msgr2=0x7f1b8c0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:45.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 --2- 172.21.15.12:0/2723435255 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c0c6fe0 0x7f1b8c0c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:45.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 -- 172.21.15.12:0/2723435255 >> v2:172.21.15.154:3300/0 conn(0x7f1b8c153bf0 msgr2=0x7f1b8c071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:45.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 --2- 172.21.15.12:0/2723435255 >> v2:172.21.15.154:3300/0 conn(0x7f1b8c153bf0 0x7f1b8c071260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:45.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 -- 172.21.15.12:0/2723435255 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c0c6530 con 0x7f1b8c0c79e0 2023-10-30T12:53:45.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 -- 172.21.15.12:0/2723435255 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f1b84001a00 con 0x7f1b8c0c79e0 2023-10-30T12:53:45.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 -- 172.21.15.12:0/2723435255 >> v1:172.21.15.12:6789/0 conn(0x7f1b8c0c79e0 legacy=0x7f1b8c0cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:45.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 --2- 172.21.15.12:0/2723435255 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c0c6fe0 0x7f1b78003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:45.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 --2- 172.21.15.12:0/2723435255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b78003f00 0x7f1b780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:45.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 --2- 172.21.15.12:0/2723435255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b78004d10 0x7f1b78005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:45.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 -- 172.21.15.12:0/2723435255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c0c6810 con 0x7f1b78004d10 2023-10-30T12:53:45.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 -- 172.21.15.12:0/2723435255 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c159900 con 0x7f1b78003f00 2023-10-30T12:53:45.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b83fff640 1 -- 172.21.15.12:0/2723435255 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7800b9c0 con 0x7f1b8c0c6fe0 2023-10-30T12:53:45.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b91f48640 1 --2- 172.21.15.12:0/2723435255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b78004d10 0x7f1b78005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b91747640 1 --2- 172.21.15.12:0/2723435255 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c0c6fe0 0x7f1b78003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b90f46640 1 --2- 172.21.15.12:0/2723435255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b78003f00 0x7f1b780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b91f48640 1 -- 172.21.15.12:0/2723435255 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c0c6fe0 msgr2=0x7f1b78003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b91f48640 1 --2- 172.21.15.12:0/2723435255 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c0c6fe0 0x7f1b78003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b91f48640 1 -- 172.21.15.12:0/2723435255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b78003f00 msgr2=0x7f1b780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b91f48640 1 --2- 172.21.15.12:0/2723435255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b78003f00 0x7f1b780047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b91f48640 1 -- 172.21.15.12:0/2723435255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b78003530 con 0x7f1b78004d10 2023-10-30T12:53:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b90f46640 1 --2- 172.21.15.12:0/2723435255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b78003f00 0x7f1b780047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:44.996+0000 7f1b91747640 1 --2- 172.21.15.12:0/2723435255 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c0c6fe0 0x7f1b78003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:45.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.000+0000 7f1b91f48640 1 --2- 172.21.15.12:0/2723435255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b78004d10 0x7f1b78005190 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400a860 tx=0x7f1b8401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a21377b0d006432 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:45.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.000+0000 7f1b83fff640 1 -- 172.21.15.12:0/2723435255 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b840ad630 con 0x7f1b78004d10 2023-10-30T12:53:45.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.000+0000 7f1b83fff640 1 -- 172.21.15.12:0/2723435255 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b840a8070 con 0x7f1b78004d10 2023-10-30T12:53:45.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.000+0000 7f1b83fff640 1 -- 172.21.15.12:0/2723435255 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b840ada30 con 0x7f1b78004d10 2023-10-30T12:53:45.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.000+0000 7f1b939ab640 1 -- 172.21.15.12:0/2723435255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b78004d10 msgr2=0x7f1b78005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:45.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.000+0000 7f1b939ab640 1 --2- 172.21.15.12:0/2723435255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b78004d10 0x7f1b78005190 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400a860 tx=0x7f1b8401d360 comp rx=0 tx=0).stop 2023-10-30T12:53:45.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.000+0000 7f1b91f48640 1 -- 172.21.15.12:0/2723435255 reap_dead start 2023-10-30T12:53:45.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.000+0000 7f1b939ab640 1 -- 172.21.15.12:0/2723435255 shutdown_connections 2023-10-30T12:53:45.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.000+0000 7f1b939ab640 1 -- 172.21.15.12:0/2723435255 >> 172.21.15.12:0/2723435255 conn(0x7f1b8c06c6e0 msgr2=0x7f1b8c069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:45.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.000+0000 7f1b939ab640 1 -- 172.21.15.12:0/2723435255 shutdown_connections 2023-10-30T12:53:45.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.000+0000 7f1b939ab640 1 -- 172.21.15.12:0/2723435255 wait complete. 2023-10-30T12:53:45.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.000+0000 7f1b939ab640 1 Processor -- start 2023-10-30T12:53:45.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.000+0000 7f1b939ab640 1 -- start start 2023-10-30T12:53:45.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b939ab640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b78004d10 0x7f1b8c1ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:45.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b939ab640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c1efee0 0x7f1b8c1f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:45.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b939ab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b8c1f2240 0x7f1b8c149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:45.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b939ab640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c14ce00 con 0x7f1b78004d10 2023-10-30T12:53:45.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b939ab640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c155ff0 con 0x7f1b8c1f2240 2023-10-30T12:53:45.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b939ab640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c14c850 con 0x7f1b8c1efee0 2023-10-30T12:53:45.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b91747640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b78004d10 0x7f1b8c1ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:45.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b91747640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b78004d10 0x7f1b8c1ef9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52038/0 (socket says 172.21.15.12:52038) 2023-10-30T12:53:45.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b91747640 1 -- 172.21.15.12:0/3909229487 learned_addr learned my addr 172.21.15.12:0/3909229487 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:45.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b91f48640 1 --2- 172.21.15.12:0/3909229487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b8c1f2240 0x7f1b8c149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:45.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b90f46640 1 --2- 172.21.15.12:0/3909229487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c1efee0 0x7f1b8c1f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:45.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b91747640 1 -- 172.21.15.12:0/3909229487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c1efee0 msgr2=0x7f1b8c1f1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:45.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b91747640 1 --2- 172.21.15.12:0/3909229487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c1efee0 0x7f1b8c1f1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:45.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b91747640 1 -- 172.21.15.12:0/3909229487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b8c1f2240 msgr2=0x7f1b8c149c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:45.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b91747640 1 --2- 172.21.15.12:0/3909229487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b8c1f2240 0x7f1b8c149c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:45.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b91747640 1 -- 172.21.15.12:0/3909229487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c14cb20 con 0x7f1b78004d10 2023-10-30T12:53:45.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b91747640 1 --2- 172.21.15.12:0/3909229487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b78004d10 0x7f1b8c1ef9a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009fd0 tx=0x7f1b88011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:45.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b81ffb640 1 -- 172.21.15.12:0/3909229487 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88004220 con 0x7f1b78004d10 2023-10-30T12:53:45.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b81ffb640 1 -- 172.21.15.12:0/3909229487 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b88004380 con 0x7f1b78004d10 2023-10-30T12:53:45.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b81ffb640 1 -- 172.21.15.12:0/3909229487 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8800f9e0 con 0x7f1b78004d10 2023-10-30T12:53:45.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b939ab640 1 -- 172.21.15.12:0/3909229487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8c1556c0 con 0x7f1b78004d10 2023-10-30T12:53:45.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b939ab640 1 -- 172.21.15.12:0/3909229487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8c155c80 con 0x7f1b78004d10 2023-10-30T12:53:45.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.004+0000 7f1b939ab640 1 -- 172.21.15.12:0/3909229487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b8c04e170 con 0x7f1b78004d10 2023-10-30T12:53:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.008+0000 7f1b81ffb640 1 -- 172.21.15.12:0/3909229487 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1b8800fb40 con 0x7f1b78004d10 2023-10-30T12:53:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.008+0000 7f1b81ffb640 1 --2- 172.21.15.12:0/3909229487 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1b700664d0 0x7f1b70068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.008+0000 7f1b81ffb640 1 -- 172.21.15.12:0/3909229487 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f1b88084850 con 0x7f1b78004d10 2023-10-30T12:53:45.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.008+0000 7f1b90f46640 1 --2- 172.21.15.12:0/3909229487 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1b700664d0 0x7f1b70068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:45.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.008+0000 7f1b90f46640 1 --2- 172.21.15.12:0/3909229487 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1b700664d0 0x7f1b70068990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c1f2b60 tx=0x7f1b7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:45.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.012+0000 7f1b81ffb640 1 -- 172.21.15.12:0/3909229487 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1b880518b0 con 0x7f1b78004d10 2023-10-30T12:53:45.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.204+0000 7f1b939ab640 1 -- 172.21.15.12:0/3909229487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b8c06fe90 con 0x7f1b78004d10 2023-10-30T12:53:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.204+0000 7f1b81ffb640 1 -- 172.21.15.12:0/3909229487 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+5742 (secure 0 0 0) 0x7f1b88054ed0 con 0x7f1b78004d10 2023-10-30T12:53:45.209 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:45.210 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:41.646061+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:rejoin","state_seq":38,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:45.210 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 93 2023-10-30T12:53:45.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.208+0000 7f1b939ab640 1 -- 172.21.15.12:0/3909229487 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1b700664d0 msgr2=0x7f1b70068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:45.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.208+0000 7f1b939ab640 1 --2- 172.21.15.12:0/3909229487 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1b700664d0 0x7f1b70068990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c1f2b60 tx=0x7f1b7c008040 comp rx=0 tx=0).stop 2023-10-30T12:53:45.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.208+0000 7f1b939ab640 1 -- 172.21.15.12:0/3909229487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b78004d10 msgr2=0x7f1b8c1ef9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:45.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.208+0000 7f1b939ab640 1 --2- 172.21.15.12:0/3909229487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b78004d10 0x7f1b8c1ef9a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009fd0 tx=0x7f1b88011b30 comp rx=0 tx=0).stop 2023-10-30T12:53:45.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.208+0000 7f1b939ab640 1 -- 172.21.15.12:0/3909229487 shutdown_connections 2023-10-30T12:53:45.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.208+0000 7f1b939ab640 1 --2- 172.21.15.12:0/3909229487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b8c1f2240 0x7f1b8c149c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:45.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.208+0000 7f1b939ab640 1 --2- 172.21.15.12:0/3909229487 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1b700664d0 0x7f1b70068990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:45.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.208+0000 7f1b939ab640 1 --2- 172.21.15.12:0/3909229487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c1efee0 0x7f1b8c1f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:45.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.208+0000 7f1b939ab640 1 --2- 172.21.15.12:0/3909229487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b78004d10 0x7f1b8c1ef9a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:45.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.208+0000 7f1b939ab640 1 -- 172.21.15.12:0/3909229487 >> 172.21.15.12:0/3909229487 conn(0x7f1b8c06c6e0 msgr2=0x7f1b8c151620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.208+0000 7f1b939ab640 1 -- 172.21.15.12:0/3909229487 shutdown_connections 2023-10-30T12:53:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:45.208+0000 7f1b939ab640 1 -- 172.21.15.12:0/3909229487 wait complete. 2023-10-30T12:53:45.231 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 93, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 93, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5758": { "addr": "172.21.15.154:6839/2204547612", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 2204547612, "type": "v2" }, { "addr": "172.21.15.154:6839", "nonce": 2204547612, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5758, "incarnation": 73, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_5877": { "addr": "172.21.15.12:6835/2306258157", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 2306258157, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 2306258157, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 5877, "incarnation": 81, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_6021": { "addr": "172.21.15.12:6836/3877967042", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 3877967042, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 3877967042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6021, "incarnation": 89, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:rejoin", "state_seq": 38 } }, "last_failure": 0, "last_failure_osd_epoch": 54, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:53:41.646061+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6021, "mds_1": 5758, "mds_2": 5877 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:53:47.233 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:47.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.384+0000 7f513e86b640 1 Processor -- start 2023-10-30T12:53:47.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.384+0000 7f513e86b640 1 -- start start 2023-10-30T12:53:47.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.388+0000 7f513e86b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f51380c6b80 0x7f51380bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:47.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.388+0000 7f513e86b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f51380c6f60 0x7f51380c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:47.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.388+0000 7f513e86b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51380c0a50 con 0x7f51380c8430 2023-10-30T12:53:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.388+0000 7f513e86b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f513806f6e0 con 0x7f51380c6f60 2023-10-30T12:53:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.388+0000 7f513e86b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f513806f9c0 con 0x7f51380c6b80 2023-10-30T12:53:47.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.388+0000 7f51377fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f51380c6f60 0x7f51380c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:47.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.388+0000 7f51377fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f51380c6f60 0x7f51380c0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52042/0 (socket says 172.21.15.12:52042) 2023-10-30T12:53:47.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.388+0000 7f51377fe640 1 -- 172.21.15.12:0/3799227301 learned_addr learned my addr 172.21.15.12:0/3799227301 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:47.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.388+0000 7f5137fff640 1 --2- 172.21.15.12:0/3799227301 >> v2:172.21.15.154:3300/0 conn(0x7f51380c6b80 0x7f51380bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.388+0000 7f51377fe640 1 -- 172.21.15.12:0/3799227301 >> v2:172.21.15.154:3300/0 conn(0x7f51380c6b80 msgr2=0x7f51380bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.388+0000 7f51377fe640 1 --2- 172.21.15.12:0/3799227301 >> v2:172.21.15.154:3300/0 conn(0x7f51380c6b80 0x7f51380bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.388+0000 7f51377fe640 1 -- 172.21.15.12:0/3799227301 >> v1:172.21.15.154:6789/0 conn(0x7f51380c8430 legacy=0x7f513813ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.388+0000 7f51377fe640 1 -- 172.21.15.12:0/3799227301 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513806fca0 con 0x7f51380c6f60 2023-10-30T12:53:47.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.392+0000 7f51377fe640 1 --2- 172.21.15.12:0/3799227301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51380c6f60 0x7f51380c0400 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f513806f1b0 tx=0x7f512c01ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9730225756f605 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:47.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.392+0000 7f5136ffd640 1 -- 172.21.15.12:0/3799227301 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0a7bb0 con 0x7f51380c6f60 2023-10-30T12:53:47.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.392+0000 7f5136ffd640 1 -- 172.21.15.12:0/3799227301 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c0a7d10 con 0x7f51380c6f60 2023-10-30T12:53:47.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.392+0000 7f5136ffd640 1 -- 172.21.15.12:0/3799227301 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0af660 con 0x7f51380c6f60 2023-10-30T12:53:47.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.392+0000 7f513e86b640 1 -- 172.21.15.12:0/3799227301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51380c6f60 msgr2=0x7f51380c0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.392+0000 7f513e86b640 1 --2- 172.21.15.12:0/3799227301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51380c6f60 0x7f51380c0400 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f513806f1b0 tx=0x7f512c01ebd0 comp rx=0 tx=0).stop 2023-10-30T12:53:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.392+0000 7f513e86b640 1 -- 172.21.15.12:0/3799227301 shutdown_connections 2023-10-30T12:53:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.392+0000 7f513e86b640 1 --2- 172.21.15.12:0/3799227301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51380c6f60 0x7f51380c0400 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.392+0000 7f513e86b640 1 --2- 172.21.15.12:0/3799227301 >> v2:172.21.15.154:3300/0 conn(0x7f51380c6b80 0x7f51380bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:47.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.392+0000 7f513e86b640 1 -- 172.21.15.12:0/3799227301 >> 172.21.15.12:0/3799227301 conn(0x7f51380c7910 msgr2=0x7f51380c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:47.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.392+0000 7f513e86b640 1 -- 172.21.15.12:0/3799227301 shutdown_connections 2023-10-30T12:53:47.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.392+0000 7f513e86b640 1 -- 172.21.15.12:0/3799227301 wait complete. 2023-10-30T12:53:47.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.392+0000 7f513e86b640 1 Processor -- start 2023-10-30T12:53:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.392+0000 7f513e86b640 1 -- start start 2023-10-30T12:53:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f513e86b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51380c6b80 0x7f51380c3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f513e86b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51380c6f60 0x7f51380c3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f513e86b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51380c8430 0x7f5138138500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f513e86b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5138154da0 con 0x7f51380c6f60 2023-10-30T12:53:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f513e86b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51380c2d30 con 0x7f51380c6b80 2023-10-30T12:53:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f513e86b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5138154fd0 con 0x7f51380c8430 2023-10-30T12:53:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f51377fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51380c6f60 0x7f51380c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f51377fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51380c6f60 0x7f51380c3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52054/0 (socket says 172.21.15.12:52054) 2023-10-30T12:53:47.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f51377fe640 1 -- 172.21.15.12:0/1759710181 learned_addr learned my addr 172.21.15.12:0/1759710181 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:47.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f5137fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51380c6b80 0x7f51380c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:47.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f513ce08640 1 --2- 172.21.15.12:0/1759710181 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51380c8430 0x7f5138138500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:47.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f51377fe640 1 -- 172.21.15.12:0/1759710181 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51380c8430 msgr2=0x7f5138138500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:47.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f51377fe640 1 --2- 172.21.15.12:0/1759710181 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51380c8430 0x7f5138138500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f51377fe640 1 -- 172.21.15.12:0/1759710181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51380c6b80 msgr2=0x7f51380c3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f51377fe640 1 --2- 172.21.15.12:0/1759710181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51380c6b80 0x7f51380c3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f51377fe640 1 -- 172.21.15.12:0/1759710181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c00b770 con 0x7f51380c6f60 2023-10-30T12:53:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f51377fe640 1 --2- 172.21.15.12:0/1759710181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51380c6f60 0x7f51380c3640 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f512c01b7e0 tx=0x7f512c01b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:47.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f5134ff9640 1 -- 172.21.15.12:0/1759710181 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0ba660 con 0x7f51380c6f60 2023-10-30T12:53:47.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f5134ff9640 1 -- 172.21.15.12:0/1759710181 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c0ba7c0 con 0x7f51380c6f60 2023-10-30T12:53:47.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f5134ff9640 1 -- 172.21.15.12:0/1759710181 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0bab70 con 0x7f51380c6f60 2023-10-30T12:53:47.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f513e86b640 1 -- 172.21.15.12:0/1759710181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51381552b0 con 0x7f51380c6f60 2023-10-30T12:53:47.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f513e86b640 1 -- 172.21.15.12:0/1759710181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5138155870 con 0x7f51380c6f60 2023-10-30T12:53:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f513e86b640 1 -- 172.21.15.12:0/1759710181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513804e170 con 0x7f51380c6f60 2023-10-30T12:53:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.396+0000 7f5134ff9640 1 -- 172.21.15.12:0/1759710181 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f512c0c2360 con 0x7f51380c6f60 2023-10-30T12:53:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.400+0000 7f5134ff9640 1 --2- 172.21.15.12:0/1759710181 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51240664a0 0x7f5124068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.400+0000 7f5134ff9640 1 -- 172.21.15.12:0/1759710181 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f512c11f0d0 con 0x7f51380c6f60 2023-10-30T12:53:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.400+0000 7f5137fff640 1 --2- 172.21.15.12:0/1759710181 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51240664a0 0x7f5124068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.400+0000 7f5137fff640 1 --2- 172.21.15.12:0/1759710181 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51240664a0 0x7f5124068960 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5128009b60 tx=0x7f5128009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.404+0000 7f5134ff9640 1 -- 172.21.15.12:0/1759710181 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f512c0ec1f0 con 0x7f51380c6f60 2023-10-30T12:53:47.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.592+0000 7f513e86b640 1 -- 172.21.15.12:0/1759710181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51380c1010 con 0x7f51380c6f60 2023-10-30T12:53:47.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.596+0000 7f5134ff9640 1 -- 172.21.15.12:0/1759710181 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+5742 (secure 0 0 0) 0x7f512c0bf090 con 0x7f51380c6f60 2023-10-30T12:53:47.598 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:47.599 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:41.646061+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:rejoin","state_seq":38,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:47.599 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 93 2023-10-30T12:53:47.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.596+0000 7f513e86b640 1 -- 172.21.15.12:0/1759710181 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51240664a0 msgr2=0x7f5124068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:47.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.596+0000 7f513e86b640 1 --2- 172.21.15.12:0/1759710181 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51240664a0 0x7f5124068960 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5128009b60 tx=0x7f5128009190 comp rx=0 tx=0).stop 2023-10-30T12:53:47.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.596+0000 7f513e86b640 1 -- 172.21.15.12:0/1759710181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51380c6f60 msgr2=0x7f51380c3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:47.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.596+0000 7f513e86b640 1 --2- 172.21.15.12:0/1759710181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51380c6f60 0x7f51380c3640 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f512c01b7e0 tx=0x7f512c01b970 comp rx=0 tx=0).stop 2023-10-30T12:53:47.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.596+0000 7f513e86b640 1 -- 172.21.15.12:0/1759710181 shutdown_connections 2023-10-30T12:53:47.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.596+0000 7f513e86b640 1 --2- 172.21.15.12:0/1759710181 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f51240664a0 0x7f5124068960 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:47.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.596+0000 7f513e86b640 1 --2- 172.21.15.12:0/1759710181 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51380c8430 0x7f5138138500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:47.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.596+0000 7f513e86b640 1 --2- 172.21.15.12:0/1759710181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51380c6f60 0x7f51380c3640 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:47.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.596+0000 7f513e86b640 1 --2- 172.21.15.12:0/1759710181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51380c6b80 0x7f51380c3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:47.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.596+0000 7f513e86b640 1 -- 172.21.15.12:0/1759710181 >> 172.21.15.12:0/1759710181 conn(0x7f51380c7910 msgr2=0x7f5138069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:47.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.596+0000 7f513e86b640 1 -- 172.21.15.12:0/1759710181 shutdown_connections 2023-10-30T12:53:47.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:47.596+0000 7f513e86b640 1 -- 172.21.15.12:0/1759710181 wait complete. 2023-10-30T12:53:47.621 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:49.622 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:49.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd9eb0f640 1 Processor -- start 2023-10-30T12:53:49.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd9eb0f640 1 -- start start 2023-10-30T12:53:49.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd9eb0f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd981538a0 0x7fcd9815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:49.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd9eb0f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd980ca0e0 0x7fcd9815a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:49.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd9eb0f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd980cb800 con 0x7fcd980cf160 2023-10-30T12:53:49.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd9eb0f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9815dc30 con 0x7fcd981538a0 2023-10-30T12:53:49.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd9eb0f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd980cb330 con 0x7fcd980ca0e0 2023-10-30T12:53:49.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd9d0ac640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fcd980cf160 0x7fcd980cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:57310/0 (socket says 172.21.15.12:57310) 2023-10-30T12:53:49.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd9d0ac640 1 -- 172.21.15.12:0/2794160729 learned_addr learned my addr 172.21.15.12:0/2794160729 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:53:49.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd9c8ab640 1 --2- 172.21.15.12:0/2794160729 >> v2:172.21.15.154:3300/0 conn(0x7fcd981538a0 0x7fcd9815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:49.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd97fff640 1 --2- 172.21.15.12:0/2794160729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd980ca0e0 0x7fcd9815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:49.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 -- 172.21.15.12:0/2794160729 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632777942 0 0) 0x7fcd980cb800 con 0x7fcd980cf160 2023-10-30T12:53:49.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 -- 172.21.15.12:0/2794160729 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd88003530 con 0x7fcd980cf160 2023-10-30T12:53:49.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 -- 172.21.15.12:0/2794160729 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2721360215 0 0) 0x7fcd88003530 con 0x7fcd980cf160 2023-10-30T12:53:49.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 -- 172.21.15.12:0/2794160729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd980ca0e0 msgr2=0x7fcd9815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:49.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 --2- 172.21.15.12:0/2794160729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd980ca0e0 0x7fcd9815a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:49.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 -- 172.21.15.12:0/2794160729 >> v2:172.21.15.154:3300/0 conn(0x7fcd981538a0 msgr2=0x7fcd9815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:49.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 --2- 172.21.15.12:0/2794160729 >> v2:172.21.15.154:3300/0 conn(0x7fcd981538a0 0x7fcd9815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:49.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 -- 172.21.15.12:0/2794160729 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd980cad70 con 0x7fcd980cf160 2023-10-30T12:53:49.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 -- 172.21.15.12:0/2794160729 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fcd90001a00 con 0x7fcd980cf160 2023-10-30T12:53:49.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 -- 172.21.15.12:0/2794160729 >> v1:172.21.15.12:6789/0 conn(0x7fcd980cf160 legacy=0x7fcd980cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:49.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 --2- 172.21.15.12:0/2794160729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd88003980 0x7fcd88003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:49.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 --2- 172.21.15.12:0/2794160729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd880047d0 0x7fcd88004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:49.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 --2- 172.21.15.12:0/2794160729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd980ca0e0 0x7fcd88005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:49.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 -- 172.21.15.12:0/2794160729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9815dc30 con 0x7fcd88003980 2023-10-30T12:53:49.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 -- 172.21.15.12:0/2794160729 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd980cb330 con 0x7fcd880047d0 2023-10-30T12:53:49.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd977fe640 1 -- 172.21.15.12:0/2794160729 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8800b9c0 con 0x7fcd980ca0e0 2023-10-30T12:53:49.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.776+0000 7fcd9c8ab640 1 --2- 172.21.15.12:0/2794160729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd88003980 0x7fcd88003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:49.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.780+0000 7fcd97fff640 1 --2- 172.21.15.12:0/2794160729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd880047d0 0x7fcd88004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:49.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.780+0000 7fcd9d0ac640 1 --2- 172.21.15.12:0/2794160729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd980ca0e0 0x7fcd88005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:49.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.780+0000 7fcd9c8ab640 1 -- 172.21.15.12:0/2794160729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd980ca0e0 msgr2=0x7fcd88005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:49.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.780+0000 7fcd9c8ab640 1 --2- 172.21.15.12:0/2794160729 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd980ca0e0 0x7fcd88005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.780+0000 7fcd9c8ab640 1 -- 172.21.15.12:0/2794160729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd880047d0 msgr2=0x7fcd88004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.780+0000 7fcd9c8ab640 1 --2- 172.21.15.12:0/2794160729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd880047d0 0x7fcd88004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.780+0000 7fcd9c8ab640 1 -- 172.21.15.12:0/2794160729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd88003530 con 0x7fcd88003980 2023-10-30T12:53:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.780+0000 7fcd9c8ab640 1 --2- 172.21.15.12:0/2794160729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd88003980 0x7fcd88003da0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009ae0 tx=0x7fcd8c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82cadcc41d9e6e5c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.780+0000 7fcd977fe640 1 -- 172.21.15.12:0/2794160729 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c0a64a0 con 0x7fcd88003980 2023-10-30T12:53:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.780+0000 7fcd977fe640 1 -- 172.21.15.12:0/2794160729 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c0a6600 con 0x7fcd88003980 2023-10-30T12:53:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.780+0000 7fcd977fe640 1 -- 172.21.15.12:0/2794160729 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c0a69b0 con 0x7fcd88003980 2023-10-30T12:53:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.780+0000 7fcd9eb0f640 1 -- 172.21.15.12:0/2794160729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd88003980 msgr2=0x7fcd88003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.780+0000 7fcd9eb0f640 1 --2- 172.21.15.12:0/2794160729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd88003980 0x7fcd88003da0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009ae0 tx=0x7fcd8c01ca90 comp rx=0 tx=0).stop 2023-10-30T12:53:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.780+0000 7fcd9d0ac640 1 -- 172.21.15.12:0/2794160729 reap_dead start 2023-10-30T12:53:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9eb0f640 1 -- 172.21.15.12:0/2794160729 shutdown_connections 2023-10-30T12:53:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9eb0f640 1 -- 172.21.15.12:0/2794160729 >> 172.21.15.12:0/2794160729 conn(0x7fcd9806bfe0 msgr2=0x7fcd9806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9eb0f640 1 -- 172.21.15.12:0/2794160729 shutdown_connections 2023-10-30T12:53:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9eb0f640 1 -- 172.21.15.12:0/2794160729 wait complete. 2023-10-30T12:53:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9eb0f640 1 Processor -- start 2023-10-30T12:53:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9eb0f640 1 -- start start 2023-10-30T12:53:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9eb0f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd88003980 0x7fcd980ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9eb0f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd981f0000 0x7fcd98157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9eb0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd980cd550 0x7fcd980b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:49.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9eb0f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd980b4f00 con 0x7fcd980cd550 2023-10-30T12:53:49.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9eb0f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd981efb70 con 0x7fcd88003980 2023-10-30T12:53:49.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9eb0f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd980b3650 con 0x7fcd981f0000 2023-10-30T12:53:49.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9d0ac640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd980cd550 0x7fcd980b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:49.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9d0ac640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd980cd550 0x7fcd980b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52066/0 (socket says 172.21.15.12:52066) 2023-10-30T12:53:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9d0ac640 1 -- 172.21.15.12:0/740239130 learned_addr learned my addr 172.21.15.12:0/740239130 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd97fff640 1 --2- 172.21.15.12:0/740239130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd981f0000 0x7fcd98157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9c8ab640 1 --2- 172.21.15.12:0/740239130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd88003980 0x7fcd980ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9d0ac640 1 -- 172.21.15.12:0/740239130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd981f0000 msgr2=0x7fcd98157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9d0ac640 1 --2- 172.21.15.12:0/740239130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd981f0000 0x7fcd98157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:49.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9d0ac640 1 -- 172.21.15.12:0/740239130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd88003980 msgr2=0x7fcd980ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:49.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9d0ac640 1 --2- 172.21.15.12:0/740239130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd88003980 0x7fcd980ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:49.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9d0ac640 1 -- 172.21.15.12:0/740239130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd980b4980 con 0x7fcd980cd550 2023-10-30T12:53:49.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.784+0000 7fcd9d0ac640 1 --2- 172.21.15.12:0/740239130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd980cd550 0x7fcd980b3110 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcd980ccd10 tx=0x7fcd90011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:49.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.788+0000 7fcd957fa640 1 -- 172.21.15.12:0/740239130 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd900137a0 con 0x7fcd980cd550 2023-10-30T12:53:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.788+0000 7fcd957fa640 1 -- 172.21.15.12:0/740239130 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd90013940 con 0x7fcd980cd550 2023-10-30T12:53:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.788+0000 7fcd9eb0f640 1 -- 172.21.15.12:0/740239130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd980b4c20 con 0x7fcd980cd550 2023-10-30T12:53:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.788+0000 7fcd957fa640 1 -- 172.21.15.12:0/740239130 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90013cf0 con 0x7fcd980cd550 2023-10-30T12:53:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.788+0000 7fcd9eb0f640 1 -- 172.21.15.12:0/740239130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd981ef800 con 0x7fcd980cd550 2023-10-30T12:53:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.788+0000 7fcd9eb0f640 1 -- 172.21.15.12:0/740239130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd981ef380 con 0x7fcd980cd550 2023-10-30T12:53:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.788+0000 7fcd957fa640 1 -- 172.21.15.12:0/740239130 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcd9001f460 con 0x7fcd980cd550 2023-10-30T12:53:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.788+0000 7fcd957fa640 1 --2- 172.21.15.12:0/740239130 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcd7c066470 0x7fcd7c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.788+0000 7fcd957fa640 1 -- 172.21.15.12:0/740239130 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fcd900871f0 con 0x7fcd980cd550 2023-10-30T12:53:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.788+0000 7fcd9c8ab640 1 --2- 172.21.15.12:0/740239130 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcd7c066470 0x7fcd7c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.792+0000 7fcd9c8ab640 1 --2- 172.21.15.12:0/740239130 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcd7c066470 0x7fcd7c068930 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0ad000 tx=0x7fcd8c0b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.792+0000 7fcd957fa640 1 -- 172.21.15.12:0/740239130 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcd981ef380 con 0x7fcd980cd550 2023-10-30T12:53:49.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.984+0000 7fcd9eb0f640 1 -- 172.21.15.12:0/740239130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd981ef380 con 0x7fcd980cd550 2023-10-30T12:53:49.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.984+0000 7fcd957fa640 1 -- 172.21.15.12:0/740239130 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+5748 (secure 0 0 0) 0x7fcd981ef380 con 0x7fcd980cd550 2023-10-30T12:53:49.989 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:49.989 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:47.666860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:clientreplay","state_seq":40,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:49.990 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 94 2023-10-30T12:53:49.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.988+0000 7fcd9eb0f640 1 -- 172.21.15.12:0/740239130 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcd7c066470 msgr2=0x7fcd7c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:49.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.988+0000 7fcd9eb0f640 1 --2- 172.21.15.12:0/740239130 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcd7c066470 0x7fcd7c068930 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0ad000 tx=0x7fcd8c0b2040 comp rx=0 tx=0).stop 2023-10-30T12:53:49.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.988+0000 7fcd9eb0f640 1 -- 172.21.15.12:0/740239130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd980cd550 msgr2=0x7fcd980b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:49.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.988+0000 7fcd9eb0f640 1 --2- 172.21.15.12:0/740239130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd980cd550 0x7fcd980b3110 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcd980ccd10 tx=0x7fcd90011a10 comp rx=0 tx=0).stop 2023-10-30T12:53:49.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.988+0000 7fcd9eb0f640 1 -- 172.21.15.12:0/740239130 shutdown_connections 2023-10-30T12:53:49.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.988+0000 7fcd9eb0f640 1 --2- 172.21.15.12:0/740239130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd980cd550 0x7fcd980b3110 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:49.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.988+0000 7fcd9eb0f640 1 --2- 172.21.15.12:0/740239130 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcd7c066470 0x7fcd7c068930 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:49.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.988+0000 7fcd9eb0f640 1 --2- 172.21.15.12:0/740239130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd981f0000 0x7fcd98157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:49.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.988+0000 7fcd9eb0f640 1 --2- 172.21.15.12:0/740239130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd88003980 0x7fcd980ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:49.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.988+0000 7fcd9eb0f640 1 -- 172.21.15.12:0/740239130 >> 172.21.15.12:0/740239130 conn(0x7fcd9806bfe0 msgr2=0x7fcd980c4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:49.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.988+0000 7fcd9eb0f640 1 -- 172.21.15.12:0/740239130 shutdown_connections 2023-10-30T12:53:49.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:49.988+0000 7fcd9eb0f640 1 -- 172.21.15.12:0/740239130 wait complete. 2023-10-30T12:53:50.011 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:52.012 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8e0a17640 1 Processor -- start 2023-10-30T12:53:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8e0a17640 1 -- start start 2023-10-30T12:53:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8e0a17640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb8dc15cd50 0x7fb8dc0c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8e0a17640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8dc0cefa0 0x7fb8dc0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8e0a17640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8dc15a6c0 con 0x7fb8dc1538f0 2023-10-30T12:53:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8e0a17640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8dc15d300 con 0x7fb8dc15cd50 2023-10-30T12:53:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8e0a17640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc15d5e0 con 0x7fb8dc0cefa0 2023-10-30T12:53:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8db7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb8dc1538f0 0x7fb8dc159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44028/0 (socket says 172.21.15.12:44028) 2023-10-30T12:53:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8db7fe640 1 -- 172.21.15.12:0/2336074330 learned_addr learned my addr 172.21.15.12:0/2336074330 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:53:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8daffd640 1 --2- 172.21.15.12:0/2336074330 >> v2:172.21.15.12:3300/0 conn(0x7fb8dc15cd50 0x7fb8dc0c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8dbfff640 1 --2- 172.21.15.12:0/2336074330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8dc0cefa0 0x7fb8dc0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:52.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8daffd640 1 -- 172.21.15.12:0/2336074330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8dc0cefa0 msgr2=0x7fb8dc0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:52.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8daffd640 1 --2- 172.21.15.12:0/2336074330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8dc0cefa0 0x7fb8dc0cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:52.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8daffd640 1 -- 172.21.15.12:0/2336074330 >> v1:172.21.15.12:6789/0 conn(0x7fb8dc1538f0 legacy=0x7fb8dc159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:52.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8daffd640 1 -- 172.21.15.12:0/2336074330 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc15d8c0 con 0x7fb8dc15cd50 2023-10-30T12:53:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.164+0000 7fb8dbfff640 1 --2- 172.21.15.12:0/2336074330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8dc0cefa0 0x7fb8dc0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.168+0000 7fb8daffd640 1 --2- 172.21.15.12:0/2336074330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8dc15cd50 0x7fb8dc0c9f90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00097c0 tx=0x7fb8d001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9801d708e1cd5ab server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:52.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.168+0000 7fb8da7fc640 1 -- 172.21.15.12:0/2336074330 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00a8970 con 0x7fb8dc15cd50 2023-10-30T12:53:52.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.168+0000 7fb8da7fc640 1 -- 172.21.15.12:0/2336074330 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d00a8ad0 con 0x7fb8dc15cd50 2023-10-30T12:53:52.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.168+0000 7fb8da7fc640 1 -- 172.21.15.12:0/2336074330 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00a8e60 con 0x7fb8dc15cd50 2023-10-30T12:53:52.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.168+0000 7fb8e0a17640 1 -- 172.21.15.12:0/2336074330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8dc15cd50 msgr2=0x7fb8dc0c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:52.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.168+0000 7fb8e0a17640 1 --2- 172.21.15.12:0/2336074330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8dc15cd50 0x7fb8dc0c9f90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00097c0 tx=0x7fb8d001ec70 comp rx=0 tx=0).stop 2023-10-30T12:53:52.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.168+0000 7fb8e0a17640 1 -- 172.21.15.12:0/2336074330 shutdown_connections 2023-10-30T12:53:52.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.168+0000 7fb8e0a17640 1 --2- 172.21.15.12:0/2336074330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8dc0cefa0 0x7fb8dc0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:52.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.168+0000 7fb8e0a17640 1 --2- 172.21.15.12:0/2336074330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8dc15cd50 0x7fb8dc0c9f90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:52.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.168+0000 7fb8e0a17640 1 -- 172.21.15.12:0/2336074330 >> 172.21.15.12:0/2336074330 conn(0x7fb8dc06a640 msgr2=0x7fb8dc065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:52.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.168+0000 7fb8e0a17640 1 -- 172.21.15.12:0/2336074330 shutdown_connections 2023-10-30T12:53:52.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.168+0000 7fb8e0a17640 1 -- 172.21.15.12:0/2336074330 wait complete. 2023-10-30T12:53:52.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8e0a17640 1 Processor -- start 2023-10-30T12:53:52.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8e0a17640 1 -- start start 2023-10-30T12:53:52.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8e0a17640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8dc0cefa0 0x7fb8dc0b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:52.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8e0a17640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8dc1538f0 0x7fb8dc147de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:52.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8e0a17640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8dc15cd50 0x7fb8dc148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:52.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8e0a17640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc151820 con 0x7fb8dc0cefa0 2023-10-30T12:53:52.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8e0a17640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc144000 con 0x7fb8dc15cd50 2023-10-30T12:53:52.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8e0a17640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc0b2f50 con 0x7fb8dc1538f0 2023-10-30T12:53:52.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8db7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8dc0cefa0 0x7fb8dc0b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:52.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8db7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8dc0cefa0 0x7fb8dc0b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46540/0 (socket says 172.21.15.12:46540) 2023-10-30T12:53:52.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8db7fe640 1 -- 172.21.15.12:0/2652449735 learned_addr learned my addr 172.21.15.12:0/2652449735 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:52.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8dbfff640 1 --2- 172.21.15.12:0/2652449735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8dc15cd50 0x7fb8dc148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:52.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8daffd640 1 --2- 172.21.15.12:0/2652449735 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8dc1538f0 0x7fb8dc147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:52.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8db7fe640 1 -- 172.21.15.12:0/2652449735 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8dc1538f0 msgr2=0x7fb8dc147de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:52.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8db7fe640 1 --2- 172.21.15.12:0/2652449735 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8dc1538f0 0x7fb8dc147de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:52.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8db7fe640 1 -- 172.21.15.12:0/2652449735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8dc15cd50 msgr2=0x7fb8dc148320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:52.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8db7fe640 1 --2- 172.21.15.12:0/2652449735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8dc15cd50 0x7fb8dc148320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:52.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8db7fe640 1 -- 172.21.15.12:0/2652449735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d0009420 con 0x7fb8dc0cefa0 2023-10-30T12:53:52.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8db7fe640 1 --2- 172.21.15.12:0/2652449735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8dc0cefa0 0x7fb8dc0b45f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00c500 tx=0x7fb8cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:52.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8c3fff640 1 -- 172.21.15.12:0/2652449735 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc005cf0 con 0x7fb8dc0cefa0 2023-10-30T12:53:52.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8c3fff640 1 -- 172.21.15.12:0/2652449735 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8cc005e50 con 0x7fb8dc0cefa0 2023-10-30T12:53:52.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8e0a17640 1 -- 172.21.15.12:0/2652449735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8dc14d740 con 0x7fb8dc0cefa0 2023-10-30T12:53:52.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8c3fff640 1 -- 172.21.15.12:0/2652449735 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc0072f0 con 0x7fb8dc0cefa0 2023-10-30T12:53:52.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.172+0000 7fb8e0a17640 1 -- 172.21.15.12:0/2652449735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8dc0b31a0 con 0x7fb8dc0cefa0 2023-10-30T12:53:52.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.176+0000 7fb8e0a17640 1 -- 172.21.15.12:0/2652449735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8dc04e170 con 0x7fb8dc0cefa0 2023-10-30T12:53:52.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.176+0000 7fb8c3fff640 1 -- 172.21.15.12:0/2652449735 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb8cc007450 con 0x7fb8dc0cefa0 2023-10-30T12:53:52.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.176+0000 7fb8c3fff640 1 --2- 172.21.15.12:0/2652449735 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb8c80664a0 0x7fb8c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:52.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.176+0000 7fb8c3fff640 1 -- 172.21.15.12:0/2652449735 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fb8cc083760 con 0x7fb8dc0cefa0 2023-10-30T12:53:52.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.176+0000 7fb8daffd640 1 --2- 172.21.15.12:0/2652449735 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb8c80664a0 0x7fb8c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:52.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.176+0000 7fb8daffd640 1 --2- 172.21.15.12:0/2652449735 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb8c80664a0 0x7fb8c8068960 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00b2000 tx=0x7fb8d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:52.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.180+0000 7fb8c3fff640 1 -- 172.21.15.12:0/2652449735 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb8cc050900 con 0x7fb8dc0cefa0 2023-10-30T12:53:52.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.380+0000 7fb8e0a17640 1 -- 172.21.15.12:0/2652449735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8dc0b2290 con 0x7fb8dc0cefa0 2023-10-30T12:53:52.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.380+0000 7fb8c3fff640 1 -- 172.21.15.12:0/2652449735 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+5748 (secure 0 0 0) 0x7fb8cc053f20 con 0x7fb8dc0cefa0 2023-10-30T12:53:52.383 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:52.383 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:47.666860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:clientreplay","state_seq":40,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:52.383 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 94 2023-10-30T12:53:52.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.380+0000 7fb8e0a17640 1 -- 172.21.15.12:0/2652449735 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb8c80664a0 msgr2=0x7fb8c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:52.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.380+0000 7fb8e0a17640 1 --2- 172.21.15.12:0/2652449735 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb8c80664a0 0x7fb8c8068960 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00b2000 tx=0x7fb8d0008040 comp rx=0 tx=0).stop 2023-10-30T12:53:52.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.380+0000 7fb8e0a17640 1 -- 172.21.15.12:0/2652449735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8dc0cefa0 msgr2=0x7fb8dc0b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:52.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.380+0000 7fb8e0a17640 1 --2- 172.21.15.12:0/2652449735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8dc0cefa0 0x7fb8dc0b45f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00c500 tx=0x7fb8cc00c9c0 comp rx=0 tx=0).stop 2023-10-30T12:53:52.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.380+0000 7fb8e0a17640 1 -- 172.21.15.12:0/2652449735 shutdown_connections 2023-10-30T12:53:52.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.380+0000 7fb8e0a17640 1 --2- 172.21.15.12:0/2652449735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8dc15cd50 0x7fb8dc148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:52.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.380+0000 7fb8e0a17640 1 --2- 172.21.15.12:0/2652449735 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb8c80664a0 0x7fb8c8068960 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:52.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.380+0000 7fb8e0a17640 1 --2- 172.21.15.12:0/2652449735 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8dc1538f0 0x7fb8dc147de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:52.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.380+0000 7fb8e0a17640 1 --2- 172.21.15.12:0/2652449735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8dc0cefa0 0x7fb8dc0b45f0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:52.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.380+0000 7fb8e0a17640 1 -- 172.21.15.12:0/2652449735 >> 172.21.15.12:0/2652449735 conn(0x7fb8dc06a640 msgr2=0x7fb8dc0c3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:52.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.380+0000 7fb8e0a17640 1 -- 172.21.15.12:0/2652449735 shutdown_connections 2023-10-30T12:53:52.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:52.380+0000 7fb8e0a17640 1 -- 172.21.15.12:0/2652449735 wait complete. 2023-10-30T12:53:52.404 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:54.405 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:54.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.547+0000 7f5cc9765640 1 Processor -- start 2023-10-30T12:53:54.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.547+0000 7f5cc9765640 1 -- start start 2023-10-30T12:53:54.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.547+0000 7f5cc9765640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cc41538a0 0x7f5cc415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.547+0000 7f5cc9765640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5cc40cf160 0x7f5cc40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.547+0000 7f5cc9765640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc40cb800 con 0x7f5cc40ca0e0 2023-10-30T12:53:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.547+0000 7f5cc9765640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc415dc30 con 0x7f5cc40cf160 2023-10-30T12:53:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.547+0000 7f5cc9765640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc40cb330 con 0x7f5cc41538a0 2023-10-30T12:53:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.547+0000 7f5cbbfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5cc40ca0e0 0x7f5cc415a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44044/0 (socket says 172.21.15.12:44044) 2023-10-30T12:53:54.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.547+0000 7f5cbbfff640 1 -- 172.21.15.12:0/3573789757 learned_addr learned my addr 172.21.15.12:0/3573789757 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:53:54.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc2ffd640 1 --2- 172.21.15.12:0/3573789757 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cc41538a0 0x7f5cc415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:54.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc37fe640 1 --2- 172.21.15.12:0/3573789757 >> v2:172.21.15.154:3300/0 conn(0x7f5cc40cf160 0x7f5cc40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:54.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 -- 172.21.15.12:0/3573789757 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142191993 0 0) 0x7f5cc40cb800 con 0x7f5cc40ca0e0 2023-10-30T12:53:54.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 -- 172.21.15.12:0/3573789757 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cb0003530 con 0x7f5cc40ca0e0 2023-10-30T12:53:54.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 -- 172.21.15.12:0/3573789757 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 433366588 0 0) 0x7f5cb0003530 con 0x7f5cc40ca0e0 2023-10-30T12:53:54.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 -- 172.21.15.12:0/3573789757 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cc41538a0 msgr2=0x7f5cc415d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:54.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 --2- 172.21.15.12:0/3573789757 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cc41538a0 0x7f5cc415d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:54.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 -- 172.21.15.12:0/3573789757 >> v2:172.21.15.154:3300/0 conn(0x7f5cc40cf160 msgr2=0x7f5cc40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:54.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 --2- 172.21.15.12:0/3573789757 >> v2:172.21.15.154:3300/0 conn(0x7f5cc40cf160 0x7f5cc40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:54.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 -- 172.21.15.12:0/3573789757 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc40cad70 con 0x7f5cc40ca0e0 2023-10-30T12:53:54.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 -- 172.21.15.12:0/3573789757 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f5cac001680 con 0x7f5cc40ca0e0 2023-10-30T12:53:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 -- 172.21.15.12:0/3573789757 >> v1:172.21.15.12:6789/0 conn(0x7f5cc40ca0e0 legacy=0x7f5cc415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 --2- 172.21.15.12:0/3573789757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cb0003980 0x7f5cb0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 --2- 172.21.15.12:0/3573789757 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cc41538a0 0x7f5cb00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 --2- 172.21.15.12:0/3573789757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cb0004d10 0x7f5cb0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 -- 172.21.15.12:0/3573789757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc415dc30 con 0x7f5cb0003980 2023-10-30T12:53:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 -- 172.21.15.12:0/3573789757 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc40cb330 con 0x7f5cb0004d10 2023-10-30T12:53:54.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc27fc640 1 -- 172.21.15.12:0/3573789757 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb000b990 con 0x7f5cc41538a0 2023-10-30T12:53:54.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc2ffd640 1 --2- 172.21.15.12:0/3573789757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cb0003980 0x7f5cb0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:54.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cbbfff640 1 --2- 172.21.15.12:0/3573789757 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cc41538a0 0x7f5cb00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:54.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc37fe640 1 --2- 172.21.15.12:0/3573789757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cb0004d10 0x7f5cb0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:54.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc2ffd640 1 -- 172.21.15.12:0/3573789757 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cc41538a0 msgr2=0x7f5cb00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:54.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc2ffd640 1 --2- 172.21.15.12:0/3573789757 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cc41538a0 0x7f5cb00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:54.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc2ffd640 1 -- 172.21.15.12:0/3573789757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cb0004d10 msgr2=0x7f5cb0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:54.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc2ffd640 1 --2- 172.21.15.12:0/3573789757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cb0004d10 0x7f5cb0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:54.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.551+0000 7f5cc2ffd640 1 -- 172.21.15.12:0/3573789757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb0003530 con 0x7f5cb0003980 2023-10-30T12:53:54.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.555+0000 7f5cc2ffd640 1 --2- 172.21.15.12:0/3573789757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cb0003980 0x7f5cb0003da0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009ae0 tx=0x7f5cb401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cdc621bbe0075d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:54.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.555+0000 7f5cc27fc640 1 -- 172.21.15.12:0/3573789757 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40a5530 con 0x7f5cb0003980 2023-10-30T12:53:54.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.555+0000 7f5cc27fc640 1 -- 172.21.15.12:0/3573789757 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb40a5690 con 0x7f5cb0003980 2023-10-30T12:53:54.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.555+0000 7f5cc27fc640 1 -- 172.21.15.12:0/3573789757 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40a5a40 con 0x7f5cb0003980 2023-10-30T12:53:54.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.555+0000 7f5cc9765640 1 -- 172.21.15.12:0/3573789757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cb0003980 msgr2=0x7f5cb0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:54.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.555+0000 7f5cc9765640 1 --2- 172.21.15.12:0/3573789757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cb0003980 0x7f5cb0003da0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009ae0 tx=0x7f5cb401ca50 comp rx=0 tx=0).stop 2023-10-30T12:53:54.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.555+0000 7f5cc37fe640 1 -- 172.21.15.12:0/3573789757 reap_dead start 2023-10-30T12:53:54.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.555+0000 7f5cc9765640 1 -- 172.21.15.12:0/3573789757 shutdown_connections 2023-10-30T12:53:54.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.555+0000 7f5cc9765640 1 -- 172.21.15.12:0/3573789757 >> 172.21.15.12:0/3573789757 conn(0x7f5cc406bfe0 msgr2=0x7f5cc406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:54.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.555+0000 7f5cc9765640 1 -- 172.21.15.12:0/3573789757 shutdown_connections 2023-10-30T12:53:54.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.555+0000 7f5cc9765640 1 -- 172.21.15.12:0/3573789757 wait complete. 2023-10-30T12:53:54.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.555+0000 7f5cc9765640 1 Processor -- start 2023-10-30T12:53:54.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.555+0000 7f5cc9765640 1 -- start start 2023-10-30T12:53:54.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc9765640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cb0003980 0x7f5cc4147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:54.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc9765640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cc4156ec0 0x7f5cc41573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:54.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc9765640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cc40cebf0 0x7f5cc415cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:54.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc9765640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc41f01d0 con 0x7f5cb0003980 2023-10-30T12:53:54.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc9765640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc40b3250 con 0x7f5cc40cebf0 2023-10-30T12:53:54.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc9765640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc40cd9d0 con 0x7f5cc4156ec0 2023-10-30T12:53:54.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cb0003980 0x7f5cc4147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:54.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cb0003980 0x7f5cc4147dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46558/0 (socket says 172.21.15.12:46558) 2023-10-30T12:53:54.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc2ffd640 1 -- 172.21.15.12:0/1134899022 learned_addr learned my addr 172.21.15.12:0/1134899022 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:54.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cbbfff640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cc4156ec0 0x7f5cc41573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:54.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc37fe640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cc40cebf0 0x7f5cc415cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:54.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc2ffd640 1 -- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cc4156ec0 msgr2=0x7f5cc41573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:54.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc2ffd640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cc4156ec0 0x7f5cc41573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:54.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc2ffd640 1 -- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cc40cebf0 msgr2=0x7f5cc415cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:54.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc2ffd640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cc40cebf0 0x7f5cc415cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:54.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc2ffd640 1 -- 172.21.15.12:0/1134899022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc41f06c0 con 0x7f5cb0003980 2023-10-30T12:53:54.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cbbfff640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cc4156ec0 0x7f5cc41573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:53:54.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc37fe640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cc40cebf0 0x7f5cc415cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:53:54.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc2ffd640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cb0003980 0x7f5cc4147dc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5cb401c880 tx=0x7f5cb0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:54.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cbb7fe640 1 -- 172.21.15.12:0/1134899022 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40bc070 con 0x7f5cb0003980 2023-10-30T12:53:54.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cbb7fe640 1 -- 172.21.15.12:0/1134899022 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb400f0c0 con 0x7f5cb0003980 2023-10-30T12:53:54.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cbb7fe640 1 -- 172.21.15.12:0/1134899022 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4004220 con 0x7f5cb0003980 2023-10-30T12:53:54.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc9765640 1 -- 172.21.15.12:0/1134899022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc40cd0e0 con 0x7f5cb0003980 2023-10-30T12:53:54.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc9765640 1 -- 172.21.15.12:0/1134899022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc40cd380 con 0x7f5cb0003980 2023-10-30T12:53:54.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cc9765640 1 -- 172.21.15.12:0/1134899022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc404e170 con 0x7f5cb0003980 2023-10-30T12:53:54.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.559+0000 7f5cbb7fe640 1 -- 172.21.15.12:0/1134899022 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5cb40b4040 con 0x7f5cb0003980 2023-10-30T12:53:54.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.563+0000 7f5cbb7fe640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ca40664d0 0x7f5ca4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:54.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.563+0000 7f5cbb7fe640 1 -- 172.21.15.12:0/1134899022 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f5cb411df20 con 0x7f5cb0003980 2023-10-30T12:53:54.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.563+0000 7f5cbbfff640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ca40664d0 0x7f5ca4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:54.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.563+0000 7f5cbbfff640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ca40664d0 0x7f5ca4068990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5cac001830 tx=0x7f5cac001c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:54.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.567+0000 7f5cbb7fe640 1 -- 172.21.15.12:0/1134899022 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5cb40eb010 con 0x7f5cb0003980 2023-10-30T12:53:54.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.763+0000 7f5cc9765640 1 -- 172.21.15.12:0/1134899022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cc40d0100 con 0x7f5cb0003980 2023-10-30T12:53:54.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.763+0000 7f5cbb7fe640 1 -- 172.21.15.12:0/1134899022 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+5742 (secure 0 0 0) 0x7f5cb4004380 con 0x7f5cb0003980 2023-10-30T12:53:54.769 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:54.769 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":95,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:54.102275+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:54.770 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 95 2023-10-30T12:53:54.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.767+0000 7f5cc9765640 1 -- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ca40664d0 msgr2=0x7f5ca4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:54.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.767+0000 7f5cc9765640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ca40664d0 0x7f5ca4068990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5cac001830 tx=0x7f5cac001c90 comp rx=0 tx=0).stop 2023-10-30T12:53:54.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.767+0000 7f5cc9765640 1 -- 172.21.15.12:0/1134899022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cb0003980 msgr2=0x7f5cc4147dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:54.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.767+0000 7f5cc9765640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cb0003980 0x7f5cc4147dc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5cb401c880 tx=0x7f5cb0005800 comp rx=0 tx=0).stop 2023-10-30T12:53:54.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.767+0000 7f5cc9765640 1 -- 172.21.15.12:0/1134899022 shutdown_connections 2023-10-30T12:53:54.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.767+0000 7f5cc9765640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cc40cebf0 0x7f5cc415cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:54.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.767+0000 7f5cc9765640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5ca40664d0 0x7f5ca4068990 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:54.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.767+0000 7f5cc9765640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cc4156ec0 0x7f5cc41573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:54.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.767+0000 7f5cc9765640 1 --2- 172.21.15.12:0/1134899022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5cb0003980 0x7f5cc4147dc0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:54.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.767+0000 7f5cc9765640 1 -- 172.21.15.12:0/1134899022 >> 172.21.15.12:0/1134899022 conn(0x7f5cc406bfe0 msgr2=0x7f5cc4152660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:54.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.767+0000 7f5cc9765640 1 -- 172.21.15.12:0/1134899022 shutdown_connections 2023-10-30T12:53:54.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:54.767+0000 7f5cc9765640 1 -- 172.21.15.12:0/1134899022 wait complete. 2023-10-30T12:53:54.789 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=0, replacing gid=5571 2023-10-30T12:53:54.790 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.c 2023-10-30T12:53:55.791 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2023-10-30T12:53:55.792 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-30T12:53:55.792 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-30T12:53:55.796 INFO:tasks.ceph.mds.c:Started 2023-10-30T12:53:55.797 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:55.895 INFO:tasks.ceph.mds.c.smithi012.stdout:starting mds.c at 2023-10-30T12:53:55.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f7994640 1 Processor -- start 2023-10-30T12:53:55.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f7994640 1 -- start start 2023-10-30T12:53:55.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f7994640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71f01538a0 0x7f71f015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:55.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f7994640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f71f00cf160 0x7f71f00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:55.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f7994640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f00cb800 con 0x7f71f00ca0e0 2023-10-30T12:53:55.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f7994640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f015dc30 con 0x7f71f00cf160 2023-10-30T12:53:55.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f7994640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00cb330 con 0x7f71f01538a0 2023-10-30T12:53:55.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f5730640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71f01538a0 0x7f71f015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:55.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f5f31640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f71f00cf160 0x7f71f00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:55.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f5730640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71f01538a0 0x7f71f015d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.12:53118/0 (socket says 172.21.15.12:53118) 2023-10-30T12:53:55.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f4f2f640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f71f00ca0e0 0x7f71f015a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:35922/0 (socket says 172.21.15.12:35922) 2023-10-30T12:53:55.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f5f31640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f71f00cf160 0x7f71f00cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:57828/0 (socket says 172.21.15.12:57828) 2023-10-30T12:53:55.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f4f2f640 1 -- 172.21.15.12:0/2736553970 learned_addr learned my addr 172.21.15.12:0/2736553970 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:53:55.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f5f31640 1 -- 172.21.15.12:0/2736553970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71f01538a0 msgr2=0x7f71f015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:55.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f5f31640 1 --2- 172.21.15.12:0/2736553970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71f01538a0 0x7f71f015d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:55.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f5f31640 1 -- 172.21.15.12:0/2736553970 >> v1:172.21.15.154:6789/0 conn(0x7f71f00ca0e0 legacy=0x7f71f015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:55.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f5f31640 1 -- 172.21.15.12:0/2736553970 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f00cad70 con 0x7f71f00cf160 2023-10-30T12:53:55.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.951+0000 7f71f5730640 1 --2- 172.21.15.12:0/2736553970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71f01538a0 0x7f71f015d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:53:55.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.955+0000 7f71f5f31640 1 --2- 172.21.15.12:0/2736553970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f00cf160 0x7f71f00cf590 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009d90 tx=0x7f71e801f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1982d35f5231b17 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:55.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.955+0000 7f71e7fff640 1 -- 172.21.15.12:0/2736553970 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e80af600 con 0x7f71f00cf160 2023-10-30T12:53:55.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.955+0000 7f71e7fff640 1 -- 172.21.15.12:0/2736553970 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e80ab050 con 0x7f71f00cf160 2023-10-30T12:53:55.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.955+0000 7f71e7fff640 1 -- 172.21.15.12:0/2736553970 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e80af980 con 0x7f71f00cf160 2023-10-30T12:53:55.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.955+0000 7f71f7994640 1 -- 172.21.15.12:0/2736553970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f00cf160 msgr2=0x7f71f00cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:55.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.955+0000 7f71f7994640 1 --2- 172.21.15.12:0/2736553970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f00cf160 0x7f71f00cf590 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009d90 tx=0x7f71e801f130 comp rx=0 tx=0).stop 2023-10-30T12:53:55.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.955+0000 7f71f7994640 1 -- 172.21.15.12:0/2736553970 shutdown_connections 2023-10-30T12:53:55.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.955+0000 7f71f7994640 1 --2- 172.21.15.12:0/2736553970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f00cf160 0x7f71f00cf590 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:55.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.955+0000 7f71f7994640 1 --2- 172.21.15.12:0/2736553970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71f01538a0 0x7f71f015d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:55.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.955+0000 7f71f7994640 1 -- 172.21.15.12:0/2736553970 >> 172.21.15.12:0/2736553970 conn(0x7f71f006bfe0 msgr2=0x7f71f006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:55.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.955+0000 7f71f7994640 1 -- 172.21.15.12:0/2736553970 shutdown_connections 2023-10-30T12:53:55.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.955+0000 7f71f7994640 1 -- 172.21.15.12:0/2736553970 wait complete. 2023-10-30T12:53:55.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f7994640 1 Processor -- start 2023-10-30T12:53:55.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f7994640 1 -- start start 2023-10-30T12:53:55.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f7994640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f00ca0e0 0x7f71f015a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:55.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f7994640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71f00cf160 0x7f71f015b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:55.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f7994640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71f01538a0 0x7f71f015b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:55.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f7994640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f01e3960 con 0x7f71f00cf160 2023-10-30T12:53:55.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f7994640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f01e4750 con 0x7f71f00ca0e0 2023-10-30T12:53:55.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f7994640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f01e3ba0 con 0x7f71f01538a0 2023-10-30T12:53:55.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f4f2f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71f00cf160 0x7f71f015b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:55.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f4f2f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71f00cf160 0x7f71f015b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46568/0 (socket says 172.21.15.12:46568) 2023-10-30T12:53:55.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f4f2f640 1 -- 172.21.15.12:0/1347487542 learned_addr learned my addr 172.21.15.12:0/1347487542 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:55.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f5730640 1 --2- 172.21.15.12:0/1347487542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f00ca0e0 0x7f71f015a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:55.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f5f31640 1 --2- 172.21.15.12:0/1347487542 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71f01538a0 0x7f71f015b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:55.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f4f2f640 1 -- 172.21.15.12:0/1347487542 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71f01538a0 msgr2=0x7f71f015b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:55.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f4f2f640 1 --2- 172.21.15.12:0/1347487542 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71f01538a0 0x7f71f015b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f4f2f640 1 -- 172.21.15.12:0/1347487542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f00ca0e0 msgr2=0x7f71f015a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f4f2f640 1 --2- 172.21.15.12:0/1347487542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f00ca0e0 0x7f71f015a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f4f2f640 1 -- 172.21.15.12:0/1347487542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ec009420 con 0x7f71f00cf160 2023-10-30T12:53:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f4f2f640 1 --2- 172.21.15.12:0/1347487542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71f00cf160 0x7f71f015b300 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c500 tx=0x7f71e000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:55.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71e5ffb640 1 -- 172.21.15.12:0/1347487542 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0005cf0 con 0x7f71f00cf160 2023-10-30T12:53:55.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71e5ffb640 1 -- 172.21.15.12:0/1347487542 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e0005e50 con 0x7f71f00cf160 2023-10-30T12:53:55.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71e5ffb640 1 -- 172.21.15.12:0/1347487542 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e00072f0 con 0x7f71f00cf160 2023-10-30T12:53:55.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f7994640 1 -- 172.21.15.12:0/1347487542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e8009a70 con 0x7f71f00cf160 2023-10-30T12:53:55.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.959+0000 7f71f7994640 1 -- 172.21.15.12:0/1347487542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f01e3430 con 0x7f71f00cf160 2023-10-30T12:53:55.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.963+0000 7f71f7994640 1 -- 172.21.15.12:0/1347487542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f00b2a50 con 0x7f71f00cf160 2023-10-30T12:53:55.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.963+0000 7f71e5ffb640 1 -- 172.21.15.12:0/1347487542 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f71e0007450 con 0x7f71f00cf160 2023-10-30T12:53:55.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.963+0000 7f71e5ffb640 1 --2- 172.21.15.12:0/1347487542 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f71d40664a0 0x7f71d4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:55.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.963+0000 7f71e5ffb640 1 -- 172.21.15.12:0/1347487542 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f71e00836d0 con 0x7f71f00cf160 2023-10-30T12:53:55.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.963+0000 7f71f5730640 1 --2- 172.21.15.12:0/1347487542 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f71d40664a0 0x7f71d4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:55.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.963+0000 7f71f5730640 1 --2- 172.21.15.12:0/1347487542 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f71d40664a0 0x7f71d4068960 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009be0 tx=0x7f71ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:55.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:55.967+0000 7f71e5ffb640 1 -- 172.21.15.12:0/1347487542 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f71e0050870 con 0x7f71f00cf160 2023-10-30T12:53:56.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:56.155+0000 7f71f7994640 1 -- 172.21.15.12:0/1347487542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71f004e170 con 0x7f71f00cf160 2023-10-30T12:53:56.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:56.159+0000 7f71e5ffb640 1 -- 172.21.15.12:0/1347487542 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+5742 (secure 0 0 0) 0x7f71e0053e90 con 0x7f71f00cf160 2023-10-30T12:53:56.162 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:56.163 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":95,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:54.102275+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:56.163 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 95 2023-10-30T12:53:56.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:56.159+0000 7f71f7994640 1 -- 172.21.15.12:0/1347487542 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f71d40664a0 msgr2=0x7f71d4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:56.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:56.159+0000 7f71f7994640 1 --2- 172.21.15.12:0/1347487542 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f71d40664a0 0x7f71d4068960 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009be0 tx=0x7f71ec008040 comp rx=0 tx=0).stop 2023-10-30T12:53:56.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:56.159+0000 7f71f7994640 1 -- 172.21.15.12:0/1347487542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71f00cf160 msgr2=0x7f71f015b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:56.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:56.159+0000 7f71f7994640 1 --2- 172.21.15.12:0/1347487542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71f00cf160 0x7f71f015b300 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c500 tx=0x7f71e000c9c0 comp rx=0 tx=0).stop 2023-10-30T12:53:56.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:56.159+0000 7f71f7994640 1 -- 172.21.15.12:0/1347487542 shutdown_connections 2023-10-30T12:53:56.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:56.159+0000 7f71f7994640 1 --2- 172.21.15.12:0/1347487542 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f71d40664a0 0x7f71d4068960 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:56.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:56.159+0000 7f71f7994640 1 --2- 172.21.15.12:0/1347487542 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71f01538a0 0x7f71f015b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:56.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:56.159+0000 7f71f7994640 1 --2- 172.21.15.12:0/1347487542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71f00cf160 0x7f71f015b300 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:56.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:56.159+0000 7f71f7994640 1 --2- 172.21.15.12:0/1347487542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f00ca0e0 0x7f71f015a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:56.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:56.159+0000 7f71f7994640 1 -- 172.21.15.12:0/1347487542 >> 172.21.15.12:0/1347487542 conn(0x7f71f006bfe0 msgr2=0x7f71f00c4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:56.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:56.159+0000 7f71f7994640 1 -- 172.21.15.12:0/1347487542 shutdown_connections 2023-10-30T12:53:56.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:56.159+0000 7f71f7994640 1 -- 172.21.15.12:0/1347487542 wait complete. 2023-10-30T12:53:56.183 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2023-10-30T12:53:58.184 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:58.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.335+0000 7fd6f8f1d640 1 Processor -- start 2023-10-30T12:53:58.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.335+0000 7fd6f8f1d640 1 -- start start 2023-10-30T12:53:58.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.335+0000 7fd6f8f1d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6f40bfec0 0x7fd6f40c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:58.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.335+0000 7fd6f8f1d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f4071290 con 0x7fd6f4159e60 2023-10-30T12:53:58.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.335+0000 7fd6f8f1d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f4071570 con 0x7fd6f406a720 2023-10-30T12:53:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.335+0000 7fd6f8f1d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f4071850 con 0x7fd6f40bfec0 2023-10-30T12:53:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.335+0000 7fd6f259c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd6f4159e60 0x7fd6f406f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44076/0 (socket says 172.21.15.12:44076) 2023-10-30T12:53:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.335+0000 7fd6f259c640 1 -- 172.21.15.12:0/1158174071 learned_addr learned my addr 172.21.15.12:0/1158174071 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:53:58.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.335+0000 7fd6ebfff640 1 --2- 172.21.15.12:0/1158174071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6f40bfec0 0x7fd6f40c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:58.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f1d9b640 1 -- 172.21.15.12:0/1158174071 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561479979 0 0) 0x7fd6f4071290 con 0x7fd6f4159e60 2023-10-30T12:53:58.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f1d9b640 1 -- 172.21.15.12:0/1158174071 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6e0003530 con 0x7fd6f4159e60 2023-10-30T12:53:58.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f1d9b640 1 -- 172.21.15.12:0/1158174071 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 737155418 0 0) 0x7fd6e0003530 con 0x7fd6f4159e60 2023-10-30T12:53:58.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f1d9b640 1 -- 172.21.15.12:0/1158174071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6f40bfec0 msgr2=0x7fd6f40c1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:58.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f1d9b640 1 --2- 172.21.15.12:0/1158174071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6f40bfec0 0x7fd6f40c1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:58.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f1d9b640 1 -- 172.21.15.12:0/1158174071 >> v1:172.21.15.154:6789/0 conn(0x7fd6f406a720 legacy=0x7fd6f4160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:58.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f1d9b640 1 -- 172.21.15.12:0/1158174071 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f4071b30 con 0x7fd6f4159e60 2023-10-30T12:53:58.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f1d9b640 1 -- 172.21.15.12:0/1158174071 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fd6e4001680 con 0x7fd6f4159e60 2023-10-30T12:53:58.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f1d9b640 1 -- 172.21.15.12:0/1158174071 >> v1:172.21.15.12:6789/0 conn(0x7fd6f4159e60 legacy=0x7fd6f406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:58.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f1d9b640 1 --2- 172.21.15.12:0/1158174071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6f40bfec0 0x7fd6e00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:58.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f1d9b640 1 --2- 172.21.15.12:0/1158174071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e0003f30 0x7fd6e00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:58.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f1d9b640 1 --2- 172.21.15.12:0/1158174071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0004d10 0x7fd6e0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:58.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f1d9b640 1 -- 172.21.15.12:0/1158174071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f4071850 con 0x7fd6e0003f30 2023-10-30T12:53:58.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f1d9b640 1 -- 172.21.15.12:0/1158174071 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e000dd10 con 0x7fd6e0004d10 2023-10-30T12:53:58.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f1d9b640 1 -- 172.21.15.12:0/1158174071 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e000ded0 con 0x7fd6f40bfec0 2023-10-30T12:53:58.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6ebfff640 1 --2- 172.21.15.12:0/1158174071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e0003f30 0x7fd6e00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:58.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f2d9d640 1 --2- 172.21.15.12:0/1158174071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0004d10 0x7fd6e0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:58.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6f259c640 1 --2- 172.21.15.12:0/1158174071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6f40bfec0 0x7fd6e00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:58.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6ebfff640 1 -- 172.21.15.12:0/1158174071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6f40bfec0 msgr2=0x7fd6e00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:58.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6ebfff640 1 --2- 172.21.15.12:0/1158174071 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6f40bfec0 0x7fd6e00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:58.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6ebfff640 1 -- 172.21.15.12:0/1158174071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0004d10 msgr2=0x7fd6e0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:58.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6ebfff640 1 --2- 172.21.15.12:0/1158174071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0004d10 0x7fd6e0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:58.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.339+0000 7fd6ebfff640 1 -- 172.21.15.12:0/1158174071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e0003530 con 0x7fd6e0003f30 2023-10-30T12:53:58.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.343+0000 7fd6ebfff640 1 --2- 172.21.15.12:0/1158174071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e0003f30 0x7fd6e00047d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009ae0 tx=0x7fd6dc01cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218fefa65b3e611 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:58.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.343+0000 7fd6f1d9b640 1 -- 172.21.15.12:0/1158174071 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0a56e0 con 0x7fd6e0003f30 2023-10-30T12:53:58.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.343+0000 7fd6f1d9b640 1 -- 172.21.15.12:0/1158174071 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc0a5840 con 0x7fd6e0003f30 2023-10-30T12:53:58.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.343+0000 7fd6f1d9b640 1 -- 172.21.15.12:0/1158174071 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0a5bf0 con 0x7fd6e0003f30 2023-10-30T12:53:58.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.343+0000 7fd6f8f1d640 1 -- 172.21.15.12:0/1158174071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e0003f30 msgr2=0x7fd6e00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:58.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.343+0000 7fd6f8f1d640 1 --2- 172.21.15.12:0/1158174071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e0003f30 0x7fd6e00047d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009ae0 tx=0x7fd6dc01cae0 comp rx=0 tx=0).stop 2023-10-30T12:53:58.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.343+0000 7fd6f2d9d640 1 -- 172.21.15.12:0/1158174071 reap_dead start 2023-10-30T12:53:58.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.343+0000 7fd6f8f1d640 1 -- 172.21.15.12:0/1158174071 shutdown_connections 2023-10-30T12:53:58.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.343+0000 7fd6f8f1d640 1 -- 172.21.15.12:0/1158174071 >> 172.21.15.12:0/1158174071 conn(0x7fd6f40c7910 msgr2=0x7fd6f40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:58.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.343+0000 7fd6f8f1d640 1 -- 172.21.15.12:0/1158174071 shutdown_connections 2023-10-30T12:53:58.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.343+0000 7fd6f8f1d640 1 -- 172.21.15.12:0/1158174071 wait complete. 2023-10-30T12:53:58.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.343+0000 7fd6f8f1d640 1 Processor -- start 2023-10-30T12:53:58.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.343+0000 7fd6f8f1d640 1 -- start start 2023-10-30T12:53:58.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6f8f1d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0003f30 0x7fd6f41ea830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:58.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6f8f1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6f414ae50 0x7fd6f40c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:58.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6f8f1d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6f4156be0 0x7fd6f40c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:58.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6f8f1d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f414b320 con 0x7fd6f414ae50 2023-10-30T12:53:58.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6f8f1d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f414b600 con 0x7fd6e0003f30 2023-10-30T12:53:58.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6f8f1d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f414b8e0 con 0x7fd6f4156be0 2023-10-30T12:53:58.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6ebfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6f414ae50 0x7fd6f40c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:58.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6ebfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6f414ae50 0x7fd6f40c41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46590/0 (socket says 172.21.15.12:46590) 2023-10-30T12:53:58.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6ebfff640 1 -- 172.21.15.12:0/975323510 learned_addr learned my addr 172.21.15.12:0/975323510 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:53:58.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6f259c640 1 --2- 172.21.15.12:0/975323510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0003f30 0x7fd6f41ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:58.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6f2d9d640 1 --2- 172.21.15.12:0/975323510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6f4156be0 0x7fd6f40c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:58.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6ebfff640 1 -- 172.21.15.12:0/975323510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6f4156be0 msgr2=0x7fd6f40c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:58.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6ebfff640 1 --2- 172.21.15.12:0/975323510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6f4156be0 0x7fd6f40c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:58.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6ebfff640 1 -- 172.21.15.12:0/975323510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0003f30 msgr2=0x7fd6f41ea830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:58.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6ebfff640 1 --2- 172.21.15.12:0/975323510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0003f30 0x7fd6f41ea830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:58.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6ebfff640 1 -- 172.21.15.12:0/975323510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f414bbc0 con 0x7fd6f414ae50 2023-10-30T12:53:58.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6f259c640 1 --2- 172.21.15.12:0/975323510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0003f30 0x7fd6f41ea830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:58.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6ebfff640 1 --2- 172.21.15.12:0/975323510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6f414ae50 0x7fd6f40c41b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009ae0 tx=0x7fd6e0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:58.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6eaffd640 1 -- 172.21.15.12:0/975323510 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0be070 con 0x7fd6f414ae50 2023-10-30T12:53:58.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6eaffd640 1 -- 172.21.15.12:0/975323510 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc004020 con 0x7fd6f414ae50 2023-10-30T12:53:58.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6eaffd640 1 -- 172.21.15.12:0/975323510 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc004390 con 0x7fd6f414ae50 2023-10-30T12:53:58.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6f8f1d640 1 -- 172.21.15.12:0/975323510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f414c670 con 0x7fd6f414ae50 2023-10-30T12:53:58.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6f8f1d640 1 -- 172.21.15.12:0/975323510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f40ca290 con 0x7fd6f414ae50 2023-10-30T12:53:58.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6f8f1d640 1 -- 172.21.15.12:0/975323510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f406b0c0 con 0x7fd6f414ae50 2023-10-30T12:53:58.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.347+0000 7fd6eaffd640 1 -- 172.21.15.12:0/975323510 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd6dc0b4040 con 0x7fd6f414ae50 2023-10-30T12:53:58.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.351+0000 7fd6eaffd640 1 --2- 172.21.15.12:0/975323510 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd6d40664d0 0x7fd6d4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:58.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.351+0000 7fd6eaffd640 1 -- 172.21.15.12:0/975323510 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fd6dc11dff0 con 0x7fd6f414ae50 2023-10-30T12:53:58.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.351+0000 7fd6f259c640 1 --2- 172.21.15.12:0/975323510 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd6d40664d0 0x7fd6d4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:58.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.351+0000 7fd6f259c640 1 --2- 172.21.15.12:0/975323510 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd6d40664d0 0x7fd6d4068990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4007d50 tx=0x7fd6e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:58.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.355+0000 7fd6eaffd640 1 -- 172.21.15.12:0/975323510 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd6dc0eb2b0 con 0x7fd6f414ae50 2023-10-30T12:53:58.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.543+0000 7fd6f8f1d640 1 -- 172.21.15.12:0/975323510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6f414c4d0 con 0x7fd6f414ae50 2023-10-30T12:53:58.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.543+0000 7fd6eaffd640 1 -- 172.21.15.12:0/975323510 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+6533 (secure 0 0 0) 0x7fd6f414c4d0 con 0x7fd6f414ae50 2023-10-30T12:53:58.547 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:53:58.548 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:53:54.102275+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:58.548 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 96 2023-10-30T12:53:58.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.543+0000 7fd6f8f1d640 1 -- 172.21.15.12:0/975323510 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd6d40664d0 msgr2=0x7fd6d4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:58.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.543+0000 7fd6f8f1d640 1 --2- 172.21.15.12:0/975323510 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd6d40664d0 0x7fd6d4068990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4007d50 tx=0x7fd6e4009040 comp rx=0 tx=0).stop 2023-10-30T12:53:58.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.543+0000 7fd6f8f1d640 1 -- 172.21.15.12:0/975323510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6f414ae50 msgr2=0x7fd6f40c41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:58.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.543+0000 7fd6f8f1d640 1 --2- 172.21.15.12:0/975323510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6f414ae50 0x7fd6f40c41b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009ae0 tx=0x7fd6e0005800 comp rx=0 tx=0).stop 2023-10-30T12:53:58.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.543+0000 7fd6f8f1d640 1 -- 172.21.15.12:0/975323510 shutdown_connections 2023-10-30T12:53:58.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.543+0000 7fd6f8f1d640 1 --2- 172.21.15.12:0/975323510 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd6d40664d0 0x7fd6d4068990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:58.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.543+0000 7fd6f8f1d640 1 --2- 172.21.15.12:0/975323510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6f4156be0 0x7fd6f40c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:58.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.543+0000 7fd6f8f1d640 1 --2- 172.21.15.12:0/975323510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6f414ae50 0x7fd6f40c41b0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:58.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.543+0000 7fd6f8f1d640 1 --2- 172.21.15.12:0/975323510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e0003f30 0x7fd6f41ea830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:58.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.543+0000 7fd6f8f1d640 1 -- 172.21.15.12:0/975323510 >> 172.21.15.12:0/975323510 conn(0x7fd6f40c7910 msgr2=0x7fd6f40c7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:58.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.543+0000 7fd6f8f1d640 1 -- 172.21.15.12:0/975323510 shutdown_connections 2023-10-30T12:53:58.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:53:58.543+0000 7fd6f8f1d640 1 -- 172.21.15.12:0/975323510 wait complete. 2023-10-30T12:53:58.568 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2023-10-30T12:53:58.569 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 9 secs before thrashing 2023-10-30T12:54:02.409 INFO:tasks.ceph.mds.e.smithi012.stderr:2023-10-30T12:54:02.403+0000 7f0f23052640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-30T12:54:02.409 INFO:tasks.ceph.mds.e.smithi012.stderr:2023-10-30T12:54:02.403+0000 7f0f23052640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-30T12:54:02.410 INFO:tasks.ceph.mds.e.smithi012.stderr:2023-10-30T12:54:02.403+0000 7f0f23052640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-30T12:54:07.414 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:54:07.418 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:54:07.570 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:07.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.719+0000 7f31d7f65640 1 Processor -- start 2023-10-30T12:54:07.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.719+0000 7f31d7f65640 1 -- start start 2023-10-30T12:54:07.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.719+0000 7f31d7f65640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f31d0150560 0x7f31d006afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:07.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.719+0000 7f31d7f65640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d0150940 con 0x7f31d00c68b0 2023-10-30T12:54:07.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d7f65640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d01573f0 con 0x7f31d006d000 2023-10-30T12:54:07.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d7f65640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d0156590 con 0x7f31d0150560 2023-10-30T12:54:07.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d5500640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f31d00c68b0 0x7f31d006c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40190/0 (socket says 172.21.15.12:40190) 2023-10-30T12:54:07.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d5500640 1 -- 172.21.15.12:0/2507670865 learned_addr learned my addr 172.21.15.12:0/2507670865 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:54:07.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d5d01640 1 --2- 172.21.15.12:0/2507670865 >> v2:172.21.15.154:3300/0 conn(0x7f31d0150560 0x7f31d006afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:07.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d4cff640 1 -- 172.21.15.12:0/2507670865 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966076024 0 0) 0x7f31d0150940 con 0x7f31d00c68b0 2023-10-30T12:54:07.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d4cff640 1 -- 172.21.15.12:0/2507670865 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31bc005740 con 0x7f31d00c68b0 2023-10-30T12:54:07.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d4cff640 1 -- 172.21.15.12:0/2507670865 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 658561243 0 0) 0x7f31bc005740 con 0x7f31d00c68b0 2023-10-30T12:54:07.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d4cff640 1 -- 172.21.15.12:0/2507670865 >> v2:172.21.15.154:3300/0 conn(0x7f31d0150560 msgr2=0x7f31d006afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:07.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d4cff640 1 --2- 172.21.15.12:0/2507670865 >> v2:172.21.15.154:3300/0 conn(0x7f31d0150560 0x7f31d006afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d4cff640 1 -- 172.21.15.12:0/2507670865 >> v1:172.21.15.154:6789/0 conn(0x7f31d006d000 legacy=0x7f31d0160d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d4cff640 1 -- 172.21.15.12:0/2507670865 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d0156870 con 0x7f31d00c68b0 2023-10-30T12:54:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d4cff640 1 -- 172.21.15.12:0/2507670865 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f31c0001680 con 0x7f31d00c68b0 2023-10-30T12:54:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d4cff640 1 -- 172.21.15.12:0/2507670865 >> v1:172.21.15.12:6789/0 conn(0x7f31d00c68b0 legacy=0x7f31d006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d4cff640 1 --2- 172.21.15.12:0/2507670865 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31bc005b90 0x7f31bc005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:07.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d4cff640 1 --2- 172.21.15.12:0/2507670865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31bc0069e0 0x7f31bc006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:07.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d4cff640 1 --2- 172.21.15.12:0/2507670865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31bc0073a0 0x7f31bc00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:07.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d4cff640 1 -- 172.21.15.12:0/2507670865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d0156590 con 0x7f31bc0073a0 2023-10-30T12:54:07.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d4cff640 1 -- 172.21.15.12:0/2507670865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc00e240 con 0x7f31bc0069e0 2023-10-30T12:54:07.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d4cff640 1 -- 172.21.15.12:0/2507670865 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc00e400 con 0x7f31bc005b90 2023-10-30T12:54:07.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d6502640 1 --2- 172.21.15.12:0/2507670865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31bc0073a0 0x7f31bc00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:07.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d5500640 1 --2- 172.21.15.12:0/2507670865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31bc0069e0 0x7f31bc006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:07.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d5d01640 1 --2- 172.21.15.12:0/2507670865 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31bc005b90 0x7f31bc005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:07.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d6502640 1 -- 172.21.15.12:0/2507670865 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31bc005b90 msgr2=0x7f31bc005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:07.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d6502640 1 --2- 172.21.15.12:0/2507670865 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31bc005b90 0x7f31bc005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:07.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d6502640 1 -- 172.21.15.12:0/2507670865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31bc0069e0 msgr2=0x7f31bc006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:07.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d6502640 1 --2- 172.21.15.12:0/2507670865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31bc0069e0 0x7f31bc006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:07.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.723+0000 7f31d6502640 1 -- 172.21.15.12:0/2507670865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31bc005740 con 0x7f31bc0073a0 2023-10-30T12:54:07.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.727+0000 7f31d6502640 1 -- 172.21.15.12:0/2507670865 reap_dead start 2023-10-30T12:54:07.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.727+0000 7f31d6502640 1 --2- 172.21.15.12:0/2507670865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31bc0073a0 0x7f31bc00dba0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f31c8001e70 tx=0x7f31c801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b556938bb8a0a2c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:07.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.727+0000 7f31d4cff640 1 -- 172.21.15.12:0/2507670865 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8003b80 con 0x7f31bc0073a0 2023-10-30T12:54:07.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.727+0000 7f31d4cff640 1 -- 172.21.15.12:0/2507670865 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c8003d20 con 0x7f31bc0073a0 2023-10-30T12:54:07.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.727+0000 7f31d4cff640 1 -- 172.21.15.12:0/2507670865 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8005260 con 0x7f31bc0073a0 2023-10-30T12:54:07.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.727+0000 7f31d7f65640 1 -- 172.21.15.12:0/2507670865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31bc0073a0 msgr2=0x7f31bc00dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:07.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.727+0000 7f31d7f65640 1 --2- 172.21.15.12:0/2507670865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31bc0073a0 0x7f31bc00dba0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f31c8001e70 tx=0x7f31c801c760 comp rx=0 tx=0).stop 2023-10-30T12:54:07.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.727+0000 7f31d7f65640 1 -- 172.21.15.12:0/2507670865 shutdown_connections 2023-10-30T12:54:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.727+0000 7f31d7f65640 1 --2- 172.21.15.12:0/2507670865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31bc0073a0 0x7f31bc00dba0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.727+0000 7f31d7f65640 1 -- 172.21.15.12:0/2507670865 >> 172.21.15.12:0/2507670865 conn(0x7f31d00cbeb0 msgr2=0x7f31d00cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.727+0000 7f31d7f65640 1 -- 172.21.15.12:0/2507670865 shutdown_connections 2023-10-30T12:54:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.727+0000 7f31d7f65640 1 -- 172.21.15.12:0/2507670865 wait complete. 2023-10-30T12:54:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.727+0000 7f31d7f65640 1 Processor -- start 2023-10-30T12:54:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.727+0000 7f31d7f65640 1 -- start start 2023-10-30T12:54:07.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d7f65640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31bc0073a0 0x7f31d0146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:07.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d7f65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31d014aa90 0x7f31d01524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:07.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d7f65640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d0152a20 0x7f31d00cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:07.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d7f65640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d01512e0 con 0x7f31d014aa90 2023-10-30T12:54:07.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d7f65640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d0152ed0 con 0x7f31d0152a20 2023-10-30T12:54:07.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d7f65640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d0153c90 con 0x7f31bc0073a0 2023-10-30T12:54:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d5500640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31d014aa90 0x7f31d01524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d5500640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31d014aa90 0x7f31d01524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40564/0 (socket says 172.21.15.12:40564) 2023-10-30T12:54:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d5500640 1 -- 172.21.15.12:0/2853894451 learned_addr learned my addr 172.21.15.12:0/2853894451 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d5d01640 1 --2- 172.21.15.12:0/2853894451 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31bc0073a0 0x7f31d0146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d6502640 1 --2- 172.21.15.12:0/2853894451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d0152a20 0x7f31d00cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:07.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d5500640 1 -- 172.21.15.12:0/2853894451 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31bc0073a0 msgr2=0x7f31d0146360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:07.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d5500640 1 --2- 172.21.15.12:0/2853894451 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31bc0073a0 0x7f31d0146360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:07.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d5500640 1 -- 172.21.15.12:0/2853894451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d0152a20 msgr2=0x7f31d00cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:07.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d5500640 1 --2- 172.21.15.12:0/2853894451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d0152a20 0x7f31d00cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:07.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d5500640 1 -- 172.21.15.12:0/2853894451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d014c6f0 con 0x7f31d014aa90 2023-10-30T12:54:07.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d6502640 1 --2- 172.21.15.12:0/2853894451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d0152a20 0x7f31d00cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:07.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d5500640 1 --2- 172.21.15.12:0/2853894451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31d014aa90 0x7f31d01524e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f31c00019e0 tx=0x7f31c0007660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:07.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31c67fc640 1 -- 172.21.15.12:0/2853894451 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0007790 con 0x7f31d014aa90 2023-10-30T12:54:07.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31c67fc640 1 -- 172.21.15.12:0/2853894451 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c000f9c0 con 0x7f31d014aa90 2023-10-30T12:54:07.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31c67fc640 1 -- 172.21.15.12:0/2853894451 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c000fd30 con 0x7f31d014aa90 2023-10-30T12:54:07.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d7f65640 1 -- 172.21.15.12:0/2853894451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d014c9b0 con 0x7f31d014aa90 2023-10-30T12:54:07.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d7f65640 1 -- 172.21.15.12:0/2853894451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d014efc0 con 0x7f31d014aa90 2023-10-30T12:54:07.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31d7f65640 1 -- 172.21.15.12:0/2853894451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d0154420 con 0x7f31d014aa90 2023-10-30T12:54:07.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.731+0000 7f31c67fc640 1 -- 172.21.15.12:0/2853894451 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f31c001d460 con 0x7f31d014aa90 2023-10-30T12:54:07.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.735+0000 7f31c67fc640 1 --2- 172.21.15.12:0/2853894451 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31b40664d0 0x7f31b4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:07.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.735+0000 7f31c67fc640 1 -- 172.21.15.12:0/2853894451 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f31c0084430 con 0x7f31d014aa90 2023-10-30T12:54:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.735+0000 7f31d5d01640 1 --2- 172.21.15.12:0/2853894451 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31b40664d0 0x7f31b4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.735+0000 7f31d5d01640 1 --2- 172.21.15.12:0/2853894451 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31b40664d0 0x7f31b4068990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009b60 tx=0x7f31cc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.739+0000 7f31c67fc640 1 -- 172.21.15.12:0/2853894451 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f31c0051400 con 0x7f31d014aa90 2023-10-30T12:54:07.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.931+0000 7f31d7f65640 1 -- 172.21.15.12:0/2853894451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31d004e170 con 0x7f31d014aa90 2023-10-30T12:54:07.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.931+0000 7f31c67fc640 1 -- 172.21.15.12:0/2853894451 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+6534 (secure 0 0 0) 0x7f31c0054a20 con 0x7f31d014aa90 2023-10-30T12:54:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:07.935 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:04.516165+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:07.935 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 97 2023-10-30T12:54:07.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.931+0000 7f31d7f65640 1 -- 172.21.15.12:0/2853894451 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31b40664d0 msgr2=0x7f31b4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:07.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.931+0000 7f31d7f65640 1 --2- 172.21.15.12:0/2853894451 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31b40664d0 0x7f31b4068990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009b60 tx=0x7f31cc009190 comp rx=0 tx=0).stop 2023-10-30T12:54:07.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.931+0000 7f31d7f65640 1 -- 172.21.15.12:0/2853894451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31d014aa90 msgr2=0x7f31d01524e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:07.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.931+0000 7f31d7f65640 1 --2- 172.21.15.12:0/2853894451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31d014aa90 0x7f31d01524e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f31c00019e0 tx=0x7f31c0007660 comp rx=0 tx=0).stop 2023-10-30T12:54:07.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.931+0000 7f31d7f65640 1 -- 172.21.15.12:0/2853894451 shutdown_connections 2023-10-30T12:54:07.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.931+0000 7f31d7f65640 1 --2- 172.21.15.12:0/2853894451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d0152a20 0x7f31d00cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:07.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.931+0000 7f31d7f65640 1 --2- 172.21.15.12:0/2853894451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31d014aa90 0x7f31d01524e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:07.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.931+0000 7f31d7f65640 1 --2- 172.21.15.12:0/2853894451 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31b40664d0 0x7f31b4068990 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:07.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.931+0000 7f31d7f65640 1 --2- 172.21.15.12:0/2853894451 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31bc0073a0 0x7f31d0146360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:07.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.931+0000 7f31d7f65640 1 -- 172.21.15.12:0/2853894451 >> 172.21.15.12:0/2853894451 conn(0x7f31d00cbeb0 msgr2=0x7f31d0069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:07.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.931+0000 7f31d7f65640 1 -- 172.21.15.12:0/2853894451 shutdown_connections 2023-10-30T12:54:07.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:07.931+0000 7f31d7f65640 1 -- 172.21.15.12:0/2853894451 wait complete. 2023-10-30T12:54:07.956 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=1) 2023-10-30T12:54:07.956 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-30T12:54:07.956 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:54:07.970 INFO:tasks.ceph.mds.d:Stopped 2023-10-30T12:54:07.971 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:08.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.111+0000 7fc7f526b640 1 Processor -- start 2023-10-30T12:54:08.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.111+0000 7fc7f526b640 1 -- start start 2023-10-30T12:54:08.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.111+0000 7fc7f526b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc7f00c6b80 0x7fc7f00bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:08.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.111+0000 7fc7f526b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc7f013f4a0 0x7fc7f00c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:08.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.111+0000 7fc7f526b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f00c0a50 con 0x7fc7f00c6f60 2023-10-30T12:54:08.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.111+0000 7fc7f526b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f006f6e0 con 0x7fc7f013f4a0 2023-10-30T12:54:08.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.111+0000 7fc7f526b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f006f9c0 con 0x7fc7f00c6b80 2023-10-30T12:54:08.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.111+0000 7fc7ef7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc7f013f4a0 0x7fc7f00c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:08.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.111+0000 7fc7ef7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc7f013f4a0 0x7fc7f00c0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40570/0 (socket says 172.21.15.12:40570) 2023-10-30T12:54:08.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.111+0000 7fc7ef7fe640 1 -- 172.21.15.12:0/1016706197 learned_addr learned my addr 172.21.15.12:0/1016706197 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:08.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.111+0000 7fc7eeffd640 1 --2- 172.21.15.12:0/1016706197 >> v2:172.21.15.154:3300/0 conn(0x7fc7f00c6b80 0x7fc7f00bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:08.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.111+0000 7fc7ef7fe640 1 -- 172.21.15.12:0/1016706197 >> v2:172.21.15.154:3300/0 conn(0x7fc7f00c6b80 msgr2=0x7fc7f00bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:08.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.111+0000 7fc7ef7fe640 1 --2- 172.21.15.12:0/1016706197 >> v2:172.21.15.154:3300/0 conn(0x7fc7f00c6b80 0x7fc7f00bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:08.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.111+0000 7fc7ef7fe640 1 -- 172.21.15.12:0/1016706197 >> v1:172.21.15.154:6789/0 conn(0x7fc7f00c6f60 legacy=0x7fc7f013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:08.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.111+0000 7fc7ef7fe640 1 -- 172.21.15.12:0/1016706197 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f006fca0 con 0x7fc7f013f4a0 2023-10-30T12:54:08.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.115+0000 7fc7ef7fe640 1 --2- 172.21.15.12:0/1016706197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7f013f4a0 0x7fc7f00c0400 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc7f006f1b0 tx=0x7fc7e801ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cf26d68ec4cbc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:08.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.115+0000 7fc7edffb640 1 -- 172.21.15.12:0/1016706197 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e800b2d0 con 0x7fc7f013f4a0 2023-10-30T12:54:08.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.115+0000 7fc7edffb640 1 -- 172.21.15.12:0/1016706197 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e800b430 con 0x7fc7f013f4a0 2023-10-30T12:54:08.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.115+0000 7fc7edffb640 1 -- 172.21.15.12:0/1016706197 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e80a7bd0 con 0x7fc7f013f4a0 2023-10-30T12:54:08.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.115+0000 7fc7f526b640 1 -- 172.21.15.12:0/1016706197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7f013f4a0 msgr2=0x7fc7f00c0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:08.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.115+0000 7fc7f526b640 1 --2- 172.21.15.12:0/1016706197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7f013f4a0 0x7fc7f00c0400 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc7f006f1b0 tx=0x7fc7e801ebd0 comp rx=0 tx=0).stop 2023-10-30T12:54:08.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.115+0000 7fc7f526b640 1 -- 172.21.15.12:0/1016706197 shutdown_connections 2023-10-30T12:54:08.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.115+0000 7fc7f526b640 1 --2- 172.21.15.12:0/1016706197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7f013f4a0 0x7fc7f00c0400 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:08.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.115+0000 7fc7f526b640 1 --2- 172.21.15.12:0/1016706197 >> v2:172.21.15.154:3300/0 conn(0x7fc7f00c6b80 0x7fc7f00bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:08.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.115+0000 7fc7f526b640 1 -- 172.21.15.12:0/1016706197 >> 172.21.15.12:0/1016706197 conn(0x7fc7f00c7910 msgr2=0x7fc7f00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:08.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.115+0000 7fc7f526b640 1 -- 172.21.15.12:0/1016706197 shutdown_connections 2023-10-30T12:54:08.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.115+0000 7fc7f526b640 1 -- 172.21.15.12:0/1016706197 wait complete. 2023-10-30T12:54:08.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7f526b640 1 Processor -- start 2023-10-30T12:54:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7f526b640 1 -- start start 2023-10-30T12:54:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7f526b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f00c6b80 0x7fc7f013e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7f526b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7f00c6f60 0x7fc7f013d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7f526b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7f013f4a0 0x7fc7f00c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7f526b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00c9150 con 0x7fc7f013f4a0 2023-10-30T12:54:08.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7f526b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f015f6f0 con 0x7fc7f00c6f60 2023-10-30T12:54:08.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7f526b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f015f990 con 0x7fc7f00c6b80 2023-10-30T12:54:08.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7ef7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7f013f4a0 0x7fc7f00c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:08.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7ef7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7f013f4a0 0x7fc7f00c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40578/0 (socket says 172.21.15.12:40578) 2023-10-30T12:54:08.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7ef7fe640 1 -- 172.21.15.12:0/2375443406 learned_addr learned my addr 172.21.15.12:0/2375443406 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:08.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7eeffd640 1 --2- 172.21.15.12:0/2375443406 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f00c6b80 0x7fc7f013e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:08.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7ee7fc640 1 --2- 172.21.15.12:0/2375443406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7f00c6f60 0x7fc7f013d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:08.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7ef7fe640 1 -- 172.21.15.12:0/2375443406 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f00c6b80 msgr2=0x7fc7f013e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:08.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7ef7fe640 1 --2- 172.21.15.12:0/2375443406 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f00c6b80 0x7fc7f013e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:08.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7ef7fe640 1 -- 172.21.15.12:0/2375443406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7f00c6f60 msgr2=0x7fc7f013d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:08.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7ef7fe640 1 --2- 172.21.15.12:0/2375443406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7f00c6f60 0x7fc7f013d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:08.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7ef7fe640 1 -- 172.21.15.12:0/2375443406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e800bd40 con 0x7fc7f013f4a0 2023-10-30T12:54:08.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7eeffd640 1 --2- 172.21.15.12:0/2375443406 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f00c6b80 0x7fc7f013e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:08.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7ef7fe640 1 --2- 172.21.15.12:0/2375443406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7f013f4a0 0x7fc7f00c8c10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc7e801b7e0 tx=0x7fc7e801b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:08.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7d77fe640 1 -- 172.21.15.12:0/2375443406 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8009a10 con 0x7fc7f013f4a0 2023-10-30T12:54:08.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7d77fe640 1 -- 172.21.15.12:0/2375443406 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e8009b70 con 0x7fc7f013f4a0 2023-10-30T12:54:08.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7f526b640 1 -- 172.21.15.12:0/2375443406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f015fc70 con 0x7fc7f013f4a0 2023-10-30T12:54:08.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7d77fe640 1 -- 172.21.15.12:0/2375443406 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e800b720 con 0x7fc7f013f4a0 2023-10-30T12:54:08.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7f526b640 1 -- 172.21.15.12:0/2375443406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f013d4d0 con 0x7fc7f013f4a0 2023-10-30T12:54:08.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.119+0000 7fc7f526b640 1 -- 172.21.15.12:0/2375443406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f013d050 con 0x7fc7f013f4a0 2023-10-30T12:54:08.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.123+0000 7fc7d77fe640 1 -- 172.21.15.12:0/2375443406 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc7e80c3460 con 0x7fc7f013f4a0 2023-10-30T12:54:08.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.123+0000 7fc7d77fe640 1 --2- 172.21.15.12:0/2375443406 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc7dc0664a0 0x7fc7dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:08.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.123+0000 7fc7d77fe640 1 -- 172.21.15.12:0/2375443406 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fc7e811fb00 con 0x7fc7f013f4a0 2023-10-30T12:54:08.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.123+0000 7fc7eeffd640 1 --2- 172.21.15.12:0/2375443406 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc7dc0664a0 0x7fc7dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:08.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.123+0000 7fc7eeffd640 1 --2- 172.21.15.12:0/2375443406 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc7dc0664a0 0x7fc7dc068960 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0007a70 tx=0x7fc7e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:08.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.127+0000 7fc7d77fe640 1 -- 172.21.15.12:0/2375443406 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc7f013d050 con 0x7fc7f013f4a0 2023-10-30T12:54:08.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.315+0000 7fc7f526b640 1 -- 172.21.15.12:0/2375443406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7f013d050 con 0x7fc7f013f4a0 2023-10-30T12:54:08.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.319+0000 7fc7d77fe640 1 -- 172.21.15.12:0/2375443406 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+6534 (secure 0 0 0) 0x7fc7f013d050 con 0x7fc7f013f4a0 2023-10-30T12:54:08.322 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:08.323 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:04.516165+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:08.323 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 97 2023-10-30T12:54:08.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.319+0000 7fc7f526b640 1 -- 172.21.15.12:0/2375443406 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc7dc0664a0 msgr2=0x7fc7dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:08.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.319+0000 7fc7f526b640 1 --2- 172.21.15.12:0/2375443406 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc7dc0664a0 0x7fc7dc068960 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0007a70 tx=0x7fc7e0008040 comp rx=0 tx=0).stop 2023-10-30T12:54:08.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.319+0000 7fc7f526b640 1 -- 172.21.15.12:0/2375443406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7f013f4a0 msgr2=0x7fc7f00c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:08.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.319+0000 7fc7f526b640 1 --2- 172.21.15.12:0/2375443406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7f013f4a0 0x7fc7f00c8c10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc7e801b7e0 tx=0x7fc7e801b970 comp rx=0 tx=0).stop 2023-10-30T12:54:08.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.319+0000 7fc7f526b640 1 -- 172.21.15.12:0/2375443406 shutdown_connections 2023-10-30T12:54:08.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.319+0000 7fc7f526b640 1 --2- 172.21.15.12:0/2375443406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7f013f4a0 0x7fc7f00c8c10 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:08.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.319+0000 7fc7f526b640 1 --2- 172.21.15.12:0/2375443406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7f00c6f60 0x7fc7f013d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:08.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.319+0000 7fc7f526b640 1 --2- 172.21.15.12:0/2375443406 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc7dc0664a0 0x7fc7dc068960 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:08.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.319+0000 7fc7f526b640 1 --2- 172.21.15.12:0/2375443406 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f00c6b80 0x7fc7f013e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:08.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.319+0000 7fc7f526b640 1 -- 172.21.15.12:0/2375443406 >> 172.21.15.12:0/2375443406 conn(0x7fc7f00c7910 msgr2=0x7fc7f0139c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:08.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.319+0000 7fc7f526b640 1 -- 172.21.15.12:0/2375443406 shutdown_connections 2023-10-30T12:54:08.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:08.319+0000 7fc7f526b640 1 -- 172.21.15.12:0/2375443406 wait complete. 2023-10-30T12:54:08.343 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:54:10.344 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:10.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.491+0000 7f3f2c66b640 1 Processor -- start 2023-10-30T12:54:10.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.491+0000 7f3f2c66b640 1 -- start start 2023-10-30T12:54:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.491+0000 7f3f2c66b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3f24153d70 0x7f3f2406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.491+0000 7f3f2c66b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3f240c8490 0x7f3f240c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.491+0000 7f3f2c66b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2406afa0 con 0x7f3f2406e470 2023-10-30T12:54:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.491+0000 7f3f2c66b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f2406d780 con 0x7f3f240c8490 2023-10-30T12:54:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.491+0000 7f3f2c66b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f24154150 con 0x7f3f24153d70 2023-10-30T12:54:10.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.495+0000 7f3f29c06640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3f2406e470 0x7f3f240cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40200/0 (socket says 172.21.15.12:40200) 2023-10-30T12:54:10.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.495+0000 7f3f29c06640 1 -- 172.21.15.12:0/2760124758 learned_addr learned my addr 172.21.15.12:0/2760124758 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:54:10.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.495+0000 7f3f2ac08640 1 --2- 172.21.15.12:0/2760124758 >> v2:172.21.15.12:3300/0 conn(0x7f3f240c8490 0x7f3f240c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:10.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.495+0000 7f3f2a407640 1 --2- 172.21.15.12:0/2760124758 >> v2:172.21.15.154:3300/0 conn(0x7f3f24153d70 0x7f3f2406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:10.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.495+0000 7f3f29405640 1 -- 172.21.15.12:0/2760124758 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527543761 0 0) 0x7f3f2406afa0 con 0x7f3f2406e470 2023-10-30T12:54:10.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.495+0000 7f3f29405640 1 -- 172.21.15.12:0/2760124758 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f10003530 con 0x7f3f2406e470 2023-10-30T12:54:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.495+0000 7f3f2ac08640 1 -- 172.21.15.12:0/2760124758 >> v2:172.21.15.154:3300/0 conn(0x7f3f24153d70 msgr2=0x7f3f2406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.495+0000 7f3f2ac08640 1 --2- 172.21.15.12:0/2760124758 >> v2:172.21.15.154:3300/0 conn(0x7f3f24153d70 0x7f3f2406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.495+0000 7f3f2ac08640 1 -- 172.21.15.12:0/2760124758 >> v1:172.21.15.12:6789/0 conn(0x7f3f2406e470 legacy=0x7f3f240cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.495+0000 7f3f2ac08640 1 -- 172.21.15.12:0/2760124758 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2406e030 con 0x7f3f240c8490 2023-10-30T12:54:10.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.495+0000 7f3f2ac08640 1 --2- 172.21.15.12:0/2760124758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f240c8490 0x7f3f240c8cc0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3f2406d180 tx=0x7f3f1c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfffc08a9f783f70 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.495+0000 7f3f29405640 1 -- 172.21.15.12:0/2760124758 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c0a59a0 con 0x7f3f240c8490 2023-10-30T12:54:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.495+0000 7f3f29405640 1 -- 172.21.15.12:0/2760124758 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1c0a5b00 con 0x7f3f240c8490 2023-10-30T12:54:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.495+0000 7f3f29405640 1 -- 172.21.15.12:0/2760124758 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c0a5e90 con 0x7f3f240c8490 2023-10-30T12:54:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 -- 172.21.15.12:0/2760124758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f240c8490 msgr2=0x7f3f240c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 --2- 172.21.15.12:0/2760124758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f240c8490 0x7f3f240c8cc0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3f2406d180 tx=0x7f3f1c01c8e0 comp rx=0 tx=0).stop 2023-10-30T12:54:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 -- 172.21.15.12:0/2760124758 shutdown_connections 2023-10-30T12:54:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 --2- 172.21.15.12:0/2760124758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f240c8490 0x7f3f240c8cc0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 --2- 172.21.15.12:0/2760124758 >> v2:172.21.15.154:3300/0 conn(0x7f3f24153d70 0x7f3f2406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 -- 172.21.15.12:0/2760124758 >> 172.21.15.12:0/2760124758 conn(0x7f3f24071ea0 msgr2=0x7f3f24065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 -- 172.21.15.12:0/2760124758 shutdown_connections 2023-10-30T12:54:10.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 -- 172.21.15.12:0/2760124758 wait complete. 2023-10-30T12:54:10.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 Processor -- start 2023-10-30T12:54:10.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 -- start start 2023-10-30T12:54:10.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2406e470 0x7f3f240cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:10.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f240c8490 0x7f3f241f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:10.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f24153d70 0x7f3f241f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:10.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2413c0d0 con 0x7f3f24153d70 2023-10-30T12:54:10.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f241f5e30 con 0x7f3f2406e470 2023-10-30T12:54:10.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2c66b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f24145660 con 0x7f3f240c8490 2023-10-30T12:54:10.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2ac08640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f24153d70 0x7f3f241f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:10.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2ac08640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f24153d70 0x7f3f241f6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40590/0 (socket says 172.21.15.12:40590) 2023-10-30T12:54:10.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2ac08640 1 -- 172.21.15.12:0/3866033535 learned_addr learned my addr 172.21.15.12:0/3866033535 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:10.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f2a407640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2406e470 0x7f3f240cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:10.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.499+0000 7f3f29c06640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f240c8490 0x7f3f241f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:10.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f2ac08640 1 -- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f240c8490 msgr2=0x7f3f241f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:10.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f2ac08640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f240c8490 0x7f3f241f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:10.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f2ac08640 1 -- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2406e470 msgr2=0x7f3f240cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:10.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f2ac08640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2406e470 0x7f3f240cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:10.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f2ac08640 1 -- 172.21.15.12:0/3866033535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c009a70 con 0x7f3f24153d70 2023-10-30T12:54:10.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f29c06640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f240c8490 0x7f3f241f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:10.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f2a407640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2406e470 0x7f3f240cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f2ac08640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f24153d70 0x7f3f241f6800 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0b0040 tx=0x7f3f1c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f1affd640 1 -- 172.21.15.12:0/3866033535 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c0add50 con 0x7f3f24153d70 2023-10-30T12:54:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f1affd640 1 -- 172.21.15.12:0/3866033535 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1c009450 con 0x7f3f24153d70 2023-10-30T12:54:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f1affd640 1 -- 172.21.15.12:0/3866033535 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c0b9690 con 0x7f3f24153d70 2023-10-30T12:54:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f2c66b640 1 -- 172.21.15.12:0/3866033535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f24145940 con 0x7f3f24153d70 2023-10-30T12:54:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f2c66b640 1 -- 172.21.15.12:0/3866033535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f24145f00 con 0x7f3f24153d70 2023-10-30T12:54:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f2c66b640 1 -- 172.21.15.12:0/3866033535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f241f5fe0 con 0x7f3f24153d70 2023-10-30T12:54:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f1affd640 1 -- 172.21.15.12:0/3866033535 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3f1c0b97f0 con 0x7f3f24153d70 2023-10-30T12:54:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.503+0000 7f3f1affd640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f080664d0 0x7f3f08068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.507+0000 7f3f1affd640 1 -- 172.21.15.12:0/3866033535 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f3f1c11f370 con 0x7f3f24153d70 2023-10-30T12:54:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.507+0000 7f3f2a407640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f080664d0 0x7f3f08068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.507+0000 7f3f2a407640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f080664d0 0x7f3f08068990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3f20007a70 tx=0x7f3f20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:10.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.511+0000 7f3f1affd640 1 -- 172.21.15.12:0/3866033535 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3f241f5fe0 con 0x7f3f24153d70 2023-10-30T12:54:10.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.703+0000 7f3f2c66b640 1 -- 172.21.15.12:0/3866033535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f241f5fe0 con 0x7f3f24153d70 2023-10-30T12:54:10.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.703+0000 7f3f1affd640 1 -- 172.21.15.12:0/3866033535 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+6536 (secure 0 0 0) 0x7f3f241f5fe0 con 0x7f3f24153d70 2023-10-30T12:54:10.707 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:10.707 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:09.537303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:10.707 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 98 2023-10-30T12:54:10.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.703+0000 7f3f2c66b640 1 -- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f080664d0 msgr2=0x7f3f08068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:10.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.703+0000 7f3f2c66b640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f080664d0 0x7f3f08068990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3f20007a70 tx=0x7f3f20008040 comp rx=0 tx=0).stop 2023-10-30T12:54:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.703+0000 7f3f2c66b640 1 -- 172.21.15.12:0/3866033535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f24153d70 msgr2=0x7f3f241f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.703+0000 7f3f2c66b640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f24153d70 0x7f3f241f6800 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0b0040 tx=0x7f3f1c019680 comp rx=0 tx=0).stop 2023-10-30T12:54:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.703+0000 7f3f2c66b640 1 -- 172.21.15.12:0/3866033535 shutdown_connections 2023-10-30T12:54:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.703+0000 7f3f2c66b640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f24153d70 0x7f3f241f6800 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.703+0000 7f3f2c66b640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f080664d0 0x7f3f08068990 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.703+0000 7f3f2c66b640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f240c8490 0x7f3f241f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.703+0000 7f3f2c66b640 1 --2- 172.21.15.12:0/3866033535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2406e470 0x7f3f240cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.703+0000 7f3f2c66b640 1 -- 172.21.15.12:0/3866033535 >> 172.21.15.12:0/3866033535 conn(0x7f3f24071ea0 msgr2=0x7f3f24151d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.703+0000 7f3f2c66b640 1 -- 172.21.15.12:0/3866033535 shutdown_connections 2023-10-30T12:54:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:10.703+0000 7f3f2c66b640 1 -- 172.21.15.12:0/3866033535 wait complete. 2023-10-30T12:54:10.727 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:54:12.728 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:12.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.879+0000 7fa55e217640 1 Processor -- start 2023-10-30T12:54:12.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.879+0000 7fa55e217640 1 -- start start 2023-10-30T12:54:12.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.879+0000 7fa55e217640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa55806d290 0x7fa55806a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.879+0000 7fa55e217640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5580c6c10 con 0x7fa55815a410 2023-10-30T12:54:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.879+0000 7fa55e217640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5580c6ef0 con 0x7fa558069e90 2023-10-30T12:54:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.879+0000 7fa55e217640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5580c8570 con 0x7fa55806d290 2023-10-30T12:54:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.883+0000 7fa556ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa55815a410 0x7fa55806cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44382/0 (socket says 172.21.15.12:44382) 2023-10-30T12:54:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.883+0000 7fa556ffd640 1 -- 172.21.15.12:0/1829607747 learned_addr learned my addr 172.21.15.12:0/1829607747 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:54:12.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.883+0000 7fa557fff640 1 --2- 172.21.15.12:0/1829607747 >> v2:172.21.15.154:3300/0 conn(0x7fa55806d290 0x7fa55806a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:12.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.883+0000 7fa5567fc640 1 -- 172.21.15.12:0/1829607747 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263339964 0 0) 0x7fa5580c6c10 con 0x7fa55815a410 2023-10-30T12:54:12.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.883+0000 7fa5567fc640 1 -- 172.21.15.12:0/1829607747 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa540003530 con 0x7fa55815a410 2023-10-30T12:54:12.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.883+0000 7fa5567fc640 1 -- 172.21.15.12:0/1829607747 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749749233 0 0) 0x7fa5580c6ef0 con 0x7fa558069e90 2023-10-30T12:54:12.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.883+0000 7fa5567fc640 1 -- 172.21.15.12:0/1829607747 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5580c6c10 con 0x7fa558069e90 2023-10-30T12:54:12.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.883+0000 7fa557fff640 1 -- 172.21.15.12:0/1829607747 >> v1:172.21.15.154:6789/0 conn(0x7fa558069e90 legacy=0x7fa5580c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.883+0000 7fa557fff640 1 -- 172.21.15.12:0/1829607747 >> v1:172.21.15.12:6789/0 conn(0x7fa55815a410 legacy=0x7fa55806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.883+0000 7fa5567fc640 1 -- 172.21.15.12:0/1829607747 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2268811442 0 0) 0x7fa540003530 con 0x7fa55815a410 2023-10-30T12:54:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.883+0000 7fa557fff640 1 -- 172.21.15.12:0/1829607747 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5580c71d0 con 0x7fa55806d290 2023-10-30T12:54:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.883+0000 7fa557fff640 1 --2- 172.21.15.12:0/1829607747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa55806d290 0x7fa55806a270 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa550009d90 tx=0x7fa55001c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cfa8443121a2bfc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa5567fc640 1 -- 172.21.15.12:0/1829607747 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c001680 con 0x7fa55806d290 2023-10-30T12:54:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa5567fc640 1 -- 172.21.15.12:0/1829607747 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5500a64c0 con 0x7fa55806d290 2023-10-30T12:54:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa5567fc640 1 -- 172.21.15.12:0/1829607747 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5500a6850 con 0x7fa55806d290 2023-10-30T12:54:12.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa55e217640 1 -- 172.21.15.12:0/1829607747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa55806d290 msgr2=0x7fa55806a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:12.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa55e217640 1 --2- 172.21.15.12:0/1829607747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa55806d290 0x7fa55806a270 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa550009d90 tx=0x7fa55001c870 comp rx=0 tx=0).stop 2023-10-30T12:54:12.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa55e217640 1 -- 172.21.15.12:0/1829607747 shutdown_connections 2023-10-30T12:54:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa55e217640 1 --2- 172.21.15.12:0/1829607747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa55806d290 0x7fa55806a270 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa55e217640 1 -- 172.21.15.12:0/1829607747 >> 172.21.15.12:0/1829607747 conn(0x7fa55806dfd0 msgr2=0x7fa55806e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa55e217640 1 -- 172.21.15.12:0/1829607747 shutdown_connections 2023-10-30T12:54:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa55e217640 1 -- 172.21.15.12:0/1829607747 wait complete. 2023-10-30T12:54:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa55e217640 1 Processor -- start 2023-10-30T12:54:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa55e217640 1 -- start start 2023-10-30T12:54:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa55e217640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa558069e90 0x7fa55815c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa55e217640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa55806d290 0x7fa55814b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:12.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa55e217640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55815a410 0x7fa558145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:12.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa55e217640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55815b750 con 0x7fa55815a410 2023-10-30T12:54:12.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa55e217640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55815cb10 con 0x7fa558069e90 2023-10-30T12:54:12.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa55e217640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55815aa60 con 0x7fa55806d290 2023-10-30T12:54:12.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.887+0000 7fa557fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55815a410 0x7fa558145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:12.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa557fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55815a410 0x7fa558145750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33608/0 (socket says 172.21.15.12:33608) 2023-10-30T12:54:12.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa557fff640 1 -- 172.21.15.12:0/3449740539 learned_addr learned my addr 172.21.15.12:0/3449740539 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:12.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa5577fe640 1 --2- 172.21.15.12:0/3449740539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa558069e90 0x7fa55815c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:12.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa556ffd640 1 --2- 172.21.15.12:0/3449740539 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa55806d290 0x7fa55814b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:12.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa557fff640 1 -- 172.21.15.12:0/3449740539 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa55806d290 msgr2=0x7fa55814b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:12.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa557fff640 1 --2- 172.21.15.12:0/3449740539 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa55806d290 0x7fa55814b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:12.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa557fff640 1 -- 172.21.15.12:0/3449740539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa558069e90 msgr2=0x7fa55815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:12.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa557fff640 1 --2- 172.21.15.12:0/3449740539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa558069e90 0x7fa55815c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:12.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa557fff640 1 -- 172.21.15.12:0/3449740539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa550009a70 con 0x7fa55815a410 2023-10-30T12:54:12.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa557fff640 1 --2- 172.21.15.12:0/3449740539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55815a410 0x7fa558145750 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa5500b0040 tx=0x7fa550019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:12.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa53ffff640 1 -- 172.21.15.12:0/3449740539 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5500c0070 con 0x7fa55815a410 2023-10-30T12:54:12.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa53ffff640 1 -- 172.21.15.12:0/3449740539 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa550009450 con 0x7fa55815a410 2023-10-30T12:54:12.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa53ffff640 1 -- 172.21.15.12:0/3449740539 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5500b8630 con 0x7fa55815a410 2023-10-30T12:54:12.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa55e217640 1 -- 172.21.15.12:0/3449740539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55815ad40 con 0x7fa55815a410 2023-10-30T12:54:12.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa55e217640 1 -- 172.21.15.12:0/3449740539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5581f1c90 con 0x7fa55815a410 2023-10-30T12:54:12.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa55e217640 1 -- 172.21.15.12:0/3449740539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5581f1810 con 0x7fa55815a410 2023-10-30T12:54:12.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa53ffff640 1 -- 172.21.15.12:0/3449740539 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa5500b8790 con 0x7fa55815a410 2023-10-30T12:54:12.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.891+0000 7fa53ffff640 1 --2- 172.21.15.12:0/3449740539 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa5440664d0 0x7fa544068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:12.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.895+0000 7fa53ffff640 1 -- 172.21.15.12:0/3449740539 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fa5500aa070 con 0x7fa55815a410 2023-10-30T12:54:12.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.895+0000 7fa5577fe640 1 --2- 172.21.15.12:0/3449740539 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa5440664d0 0x7fa544068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:12.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.895+0000 7fa5577fe640 1 --2- 172.21.15.12:0/3449740539 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa5440664d0 0x7fa544068990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa5480011f0 tx=0x7fa548009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:12.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:12.895+0000 7fa53ffff640 1 -- 172.21.15.12:0/3449740539 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa5581f1810 con 0x7fa55815a410 2023-10-30T12:54:13.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:13.083+0000 7fa55e217640 1 -- 172.21.15.12:0/3449740539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5581f1810 con 0x7fa55815a410 2023-10-30T12:54:13.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:13.087+0000 7fa53ffff640 1 -- 172.21.15.12:0/3449740539 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+6536 (secure 0 0 0) 0x7fa5581f1810 con 0x7fa55815a410 2023-10-30T12:54:13.090 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:13.090 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:09.537303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:13.091 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 98 2023-10-30T12:54:13.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:13.087+0000 7fa55e217640 1 -- 172.21.15.12:0/3449740539 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa5440664d0 msgr2=0x7fa544068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:13.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:13.087+0000 7fa55e217640 1 --2- 172.21.15.12:0/3449740539 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa5440664d0 0x7fa544068990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa5480011f0 tx=0x7fa548009040 comp rx=0 tx=0).stop 2023-10-30T12:54:13.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:13.087+0000 7fa55e217640 1 -- 172.21.15.12:0/3449740539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55815a410 msgr2=0x7fa558145750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:13.087+0000 7fa55e217640 1 --2- 172.21.15.12:0/3449740539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55815a410 0x7fa558145750 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa5500b0040 tx=0x7fa550019610 comp rx=0 tx=0).stop 2023-10-30T12:54:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:13.087+0000 7fa55e217640 1 -- 172.21.15.12:0/3449740539 shutdown_connections 2023-10-30T12:54:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:13.087+0000 7fa55e217640 1 --2- 172.21.15.12:0/3449740539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa55815a410 0x7fa558145750 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:13.087+0000 7fa55e217640 1 --2- 172.21.15.12:0/3449740539 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa5440664d0 0x7fa544068990 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:13.087+0000 7fa55e217640 1 --2- 172.21.15.12:0/3449740539 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa55806d290 0x7fa55814b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:13.087+0000 7fa55e217640 1 --2- 172.21.15.12:0/3449740539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa558069e90 0x7fa55815c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:13.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:13.087+0000 7fa55e217640 1 -- 172.21.15.12:0/3449740539 >> 172.21.15.12:0/3449740539 conn(0x7fa55806dfd0 msgr2=0x7fa55814e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:13.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:13.087+0000 7fa55e217640 1 -- 172.21.15.12:0/3449740539 shutdown_connections 2023-10-30T12:54:13.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:13.087+0000 7fa55e217640 1 -- 172.21.15.12:0/3449740539 wait complete. 2023-10-30T12:54:13.111 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:54:15.112 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.263+0000 7f19fcfc5640 1 Processor -- start 2023-10-30T12:54:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.263+0000 7f19fcfc5640 1 -- start start 2023-10-30T12:54:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.263+0000 7f19fcfc5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19f8156f80 0x7f19f8159880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.263+0000 7f19fcfc5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19f8070650 0x7f19f8071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.263+0000 7f19fcfc5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f80c2f20 con 0x7f19f8070c90 2023-10-30T12:54:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.263+0000 7f19fcfc5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f8157360 con 0x7f19f8070650 2023-10-30T12:54:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.263+0000 7f19fcfc5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80c85c0 con 0x7f19f8156f80 2023-10-30T12:54:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.267+0000 7f19effff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19f8070650 0x7f19f8071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.267+0000 7f19effff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19f8070650 0x7f19f8071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33620/0 (socket says 172.21.15.12:33620) 2023-10-30T12:54:15.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.267+0000 7f19effff640 1 -- 172.21.15.12:0/2000278486 learned_addr learned my addr 172.21.15.12:0/2000278486 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:15.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.267+0000 7f19f659c640 1 --2- 172.21.15.12:0/2000278486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19f8156f80 0x7f19f8159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:15.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.267+0000 7f19effff640 1 -- 172.21.15.12:0/2000278486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19f8156f80 msgr2=0x7f19f8159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:15.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.267+0000 7f19effff640 1 --2- 172.21.15.12:0/2000278486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19f8156f80 0x7f19f8159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:15.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.267+0000 7f19effff640 1 -- 172.21.15.12:0/2000278486 >> v1:172.21.15.154:6789/0 conn(0x7f19f8070c90 legacy=0x7f19f815cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:15.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.267+0000 7f19effff640 1 -- 172.21.15.12:0/2000278486 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f8071230 con 0x7f19f8070650 2023-10-30T12:54:15.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.267+0000 7f19effff640 1 --2- 172.21.15.12:0/2000278486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f8070650 0x7f19f8071bb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009870 tx=0x7f19e001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5431e945e24f4d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:15.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.267+0000 7f19f5d9b640 1 -- 172.21.15.12:0/2000278486 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e00aa660 con 0x7f19f8070650 2023-10-30T12:54:15.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.267+0000 7f19f5d9b640 1 -- 172.21.15.12:0/2000278486 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e00aa7c0 con 0x7f19f8070650 2023-10-30T12:54:15.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.267+0000 7f19f5d9b640 1 -- 172.21.15.12:0/2000278486 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e00aab50 con 0x7f19f8070650 2023-10-30T12:54:15.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 -- 172.21.15.12:0/2000278486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f8070650 msgr2=0x7f19f8071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:15.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 --2- 172.21.15.12:0/2000278486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f8070650 0x7f19f8071bb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009870 tx=0x7f19e001c540 comp rx=0 tx=0).stop 2023-10-30T12:54:15.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 -- 172.21.15.12:0/2000278486 shutdown_connections 2023-10-30T12:54:15.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 --2- 172.21.15.12:0/2000278486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f8070650 0x7f19f8071bb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:15.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 --2- 172.21.15.12:0/2000278486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19f8156f80 0x7f19f8159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:15.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 -- 172.21.15.12:0/2000278486 >> 172.21.15.12:0/2000278486 conn(0x7f19f80cbeb0 msgr2=0x7f19f80cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:15.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 -- 172.21.15.12:0/2000278486 shutdown_connections 2023-10-30T12:54:15.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 -- 172.21.15.12:0/2000278486 wait complete. 2023-10-30T12:54:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 Processor -- start 2023-10-30T12:54:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 -- start start 2023-10-30T12:54:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19f8070650 0x7f19f8151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f8070c90 0x7f19f8153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19f8156f80 0x7f19f81f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:15.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80c63e0 con 0x7f19f8070c90 2023-10-30T12:54:15.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80c33b0 con 0x7f19f8070650 2023-10-30T12:54:15.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19fcfc5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f815c260 con 0x7f19f8156f80 2023-10-30T12:54:15.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19effff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f8070c90 0x7f19f8153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:15.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19f659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19f8070650 0x7f19f8151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:15.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19effff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f8070c90 0x7f19f8153bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33634/0 (socket says 172.21.15.12:33634) 2023-10-30T12:54:15.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19effff640 1 -- 172.21.15.12:0/3349523508 learned_addr learned my addr 172.21.15.12:0/3349523508 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:15.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19f6d9d640 1 --2- 172.21.15.12:0/3349523508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19f8156f80 0x7f19f81f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:15.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19effff640 1 -- 172.21.15.12:0/3349523508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19f8156f80 msgr2=0x7f19f81f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19effff640 1 --2- 172.21.15.12:0/3349523508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19f8156f80 0x7f19f81f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19effff640 1 -- 172.21.15.12:0/3349523508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19f8070650 msgr2=0x7f19f8151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19effff640 1 --2- 172.21.15.12:0/3349523508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19f8070650 0x7f19f8151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19effff640 1 -- 172.21.15.12:0/3349523508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e00094d0 con 0x7f19f8070c90 2023-10-30T12:54:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.271+0000 7f19f6d9d640 1 --2- 172.21.15.12:0/3349523508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19f8156f80 0x7f19f81f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.275+0000 7f19effff640 1 --2- 172.21.15.12:0/3349523508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f8070c90 0x7f19f8153bf0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f19e001a170 tx=0x7f19e001ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.275+0000 7f19eeffd640 1 -- 172.21.15.12:0/3349523508 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e00b3800 con 0x7f19f8070c90 2023-10-30T12:54:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.275+0000 7f19eeffd640 1 -- 172.21.15.12:0/3349523508 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e00b3960 con 0x7f19f8070c90 2023-10-30T12:54:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.275+0000 7f19eeffd640 1 -- 172.21.15.12:0/3349523508 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e00b3d10 con 0x7f19f8070c90 2023-10-30T12:54:15.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.275+0000 7f19fcfc5640 1 -- 172.21.15.12:0/3349523508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f815bf80 con 0x7f19f8070c90 2023-10-30T12:54:15.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.275+0000 7f19fcfc5640 1 -- 172.21.15.12:0/3349523508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f81f3c40 con 0x7f19f8070c90 2023-10-30T12:54:15.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.275+0000 7f19fcfc5640 1 -- 172.21.15.12:0/3349523508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f80c1d50 con 0x7f19f8070c90 2023-10-30T12:54:15.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.275+0000 7f19eeffd640 1 -- 172.21.15.12:0/3349523508 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f19e00b2380 con 0x7f19f8070c90 2023-10-30T12:54:15.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.275+0000 7f19eeffd640 1 --2- 172.21.15.12:0/3349523508 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19d80664d0 0x7f19d8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.275+0000 7f19eeffd640 1 -- 172.21.15.12:0/3349523508 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f19e00a6040 con 0x7f19f8070c90 2023-10-30T12:54:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.275+0000 7f19f659c640 1 --2- 172.21.15.12:0/3349523508 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19d80664d0 0x7f19d8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.279+0000 7f19f659c640 1 --2- 172.21.15.12:0/3349523508 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19d80664d0 0x7f19d8068990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f19e80079c0 tx=0x7f19e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.279+0000 7f19eeffd640 1 -- 172.21.15.12:0/3349523508 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f19e00ebad0 con 0x7f19f8070c90 2023-10-30T12:54:15.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.471+0000 7f19fcfc5640 1 -- 172.21.15.12:0/3349523508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19f804e170 con 0x7f19f8070c90 2023-10-30T12:54:15.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.471+0000 7f19eeffd640 1 -- 172.21.15.12:0/3349523508 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+6536 (secure 0 0 0) 0x7f19e00ef0f0 con 0x7f19f8070c90 2023-10-30T12:54:15.476 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:15.476 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:09.537303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:15.477 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 98 2023-10-30T12:54:15.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.471+0000 7f19fcfc5640 1 -- 172.21.15.12:0/3349523508 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19d80664d0 msgr2=0x7f19d8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:15.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.471+0000 7f19fcfc5640 1 --2- 172.21.15.12:0/3349523508 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19d80664d0 0x7f19d8068990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f19e80079c0 tx=0x7f19e8008040 comp rx=0 tx=0).stop 2023-10-30T12:54:15.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.471+0000 7f19fcfc5640 1 -- 172.21.15.12:0/3349523508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f8070c90 msgr2=0x7f19f8153bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:15.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.471+0000 7f19fcfc5640 1 --2- 172.21.15.12:0/3349523508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f8070c90 0x7f19f8153bf0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f19e001a170 tx=0x7f19e001ad80 comp rx=0 tx=0).stop 2023-10-30T12:54:15.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.471+0000 7f19fcfc5640 1 -- 172.21.15.12:0/3349523508 shutdown_connections 2023-10-30T12:54:15.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.475+0000 7f19fcfc5640 1 --2- 172.21.15.12:0/3349523508 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f19d80664d0 0x7f19d8068990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:15.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.475+0000 7f19fcfc5640 1 --2- 172.21.15.12:0/3349523508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19f8156f80 0x7f19f81f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:15.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.475+0000 7f19fcfc5640 1 --2- 172.21.15.12:0/3349523508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f8070c90 0x7f19f8153bf0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:15.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.475+0000 7f19fcfc5640 1 --2- 172.21.15.12:0/3349523508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19f8070650 0x7f19f8151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:15.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.475+0000 7f19fcfc5640 1 -- 172.21.15.12:0/3349523508 >> 172.21.15.12:0/3349523508 conn(0x7f19f80cbeb0 msgr2=0x7f19f80cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:15.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.475+0000 7f19fcfc5640 1 -- 172.21.15.12:0/3349523508 shutdown_connections 2023-10-30T12:54:15.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:15.475+0000 7f19fcfc5640 1 -- 172.21.15.12:0/3349523508 wait complete. 2023-10-30T12:54:15.497 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:54:17.497 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:17.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.643+0000 7f4ad770f640 1 Processor -- start 2023-10-30T12:54:17.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.643+0000 7f4ad770f640 1 -- start start 2023-10-30T12:54:17.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.647+0000 7f4ad770f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ad0150560 0x7f4ad006afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:17.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.647+0000 7f4ad770f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4ad00c68b0 0x7f4ad006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:17.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.647+0000 7f4ad770f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad0150940 con 0x7f4ad0070650 2023-10-30T12:54:17.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.647+0000 7f4ad770f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad01573f0 con 0x7f4ad00c68b0 2023-10-30T12:54:17.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.647+0000 7f4ad770f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0156590 con 0x7f4ad0150560 2023-10-30T12:54:17.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.647+0000 7f4ad4caa640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4ad00c68b0 0x7f4ad006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:17.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.647+0000 7f4ad4caa640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4ad00c68b0 0x7f4ad006b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33638/0 (socket says 172.21.15.12:33638) 2023-10-30T12:54:17.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.647+0000 7f4ad4caa640 1 -- 172.21.15.12:0/1575072125 learned_addr learned my addr 172.21.15.12:0/1575072125 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:17.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.647+0000 7f4ad54ab640 1 --2- 172.21.15.12:0/1575072125 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ad0150560 0x7f4ad006afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:17.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.647+0000 7f4ad4caa640 1 -- 172.21.15.12:0/1575072125 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ad0150560 msgr2=0x7f4ad006afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:17.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.647+0000 7f4ad4caa640 1 --2- 172.21.15.12:0/1575072125 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ad0150560 0x7f4ad006afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:17.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.647+0000 7f4ad4caa640 1 -- 172.21.15.12:0/1575072125 >> v1:172.21.15.154:6789/0 conn(0x7f4ad0070650 legacy=0x7f4ad006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:17.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.647+0000 7f4ad4caa640 1 -- 172.21.15.12:0/1575072125 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad0156870 con 0x7f4ad00c68b0 2023-10-30T12:54:17.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.647+0000 7f4ad54ab640 1 --2- 172.21.15.12:0/1575072125 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ad0150560 0x7f4ad006afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:17.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad4caa640 1 --2- 172.21.15.12:0/1575072125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad00c68b0 0x7f4ad006b4e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009870 tx=0x7f4ac001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b8d76e47ad4168 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:17.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ac7fff640 1 -- 172.21.15.12:0/1575072125 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac00ab610 con 0x7f4ad00c68b0 2023-10-30T12:54:17.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ac7fff640 1 -- 172.21.15.12:0/1575072125 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac00ab770 con 0x7f4ad00c68b0 2023-10-30T12:54:17.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ac7fff640 1 -- 172.21.15.12:0/1575072125 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac00abb00 con 0x7f4ad00c68b0 2023-10-30T12:54:17.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 -- 172.21.15.12:0/1575072125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad00c68b0 msgr2=0x7f4ad006b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:17.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 --2- 172.21.15.12:0/1575072125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad00c68b0 0x7f4ad006b4e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009870 tx=0x7f4ac001c510 comp rx=0 tx=0).stop 2023-10-30T12:54:17.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 -- 172.21.15.12:0/1575072125 shutdown_connections 2023-10-30T12:54:17.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 --2- 172.21.15.12:0/1575072125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad00c68b0 0x7f4ad006b4e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:17.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 --2- 172.21.15.12:0/1575072125 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ad0150560 0x7f4ad006afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:17.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 -- 172.21.15.12:0/1575072125 >> 172.21.15.12:0/1575072125 conn(0x7f4ad00cbeb0 msgr2=0x7f4ad00cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:17.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 -- 172.21.15.12:0/1575072125 shutdown_connections 2023-10-30T12:54:17.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 -- 172.21.15.12:0/1575072125 wait complete. 2023-10-30T12:54:17.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 Processor -- start 2023-10-30T12:54:17.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 -- start start 2023-10-30T12:54:17.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ad0070650 0x7f4ad0143a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:17.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad00c68b0 0x7f4ad0151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:17.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0150560 0x7f4ad0140810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:17.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0151080 con 0x7f4ad00c68b0 2023-10-30T12:54:17.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0153800 con 0x7f4ad0150560 2023-10-30T12:54:17.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad770f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0140380 con 0x7f4ad0070650 2023-10-30T12:54:17.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.651+0000 7f4ad4caa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad00c68b0 0x7f4ad0151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:17.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ad4caa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad00c68b0 0x7f4ad0151810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33652/0 (socket says 172.21.15.12:33652) 2023-10-30T12:54:17.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ad4caa640 1 -- 172.21.15.12:0/2104168928 learned_addr learned my addr 172.21.15.12:0/2104168928 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:17.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ad54ab640 1 --2- 172.21.15.12:0/2104168928 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ad0070650 0x7f4ad0143a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:17.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ad5cac640 1 --2- 172.21.15.12:0/2104168928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0150560 0x7f4ad0140810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:17.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ad4caa640 1 -- 172.21.15.12:0/2104168928 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ad0070650 msgr2=0x7f4ad0143a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:17.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ad4caa640 1 --2- 172.21.15.12:0/2104168928 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ad0070650 0x7f4ad0143a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:17.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ad4caa640 1 -- 172.21.15.12:0/2104168928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0150560 msgr2=0x7f4ad0140810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:17.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ad4caa640 1 --2- 172.21.15.12:0/2104168928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0150560 0x7f4ad0140810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:17.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ad4caa640 1 -- 172.21.15.12:0/2104168928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac00094d0 con 0x7f4ad00c68b0 2023-10-30T12:54:17.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ad4caa640 1 --2- 172.21.15.12:0/2104168928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad00c68b0 0x7f4ad0151810 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00ad040 tx=0x7f4ac001ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:17.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ac5ffb640 1 -- 172.21.15.12:0/2104168928 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac00b4800 con 0x7f4ad00c68b0 2023-10-30T12:54:17.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ac5ffb640 1 -- 172.21.15.12:0/2104168928 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac00b4980 con 0x7f4ad00c68b0 2023-10-30T12:54:17.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ac5ffb640 1 -- 172.21.15.12:0/2104168928 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac00b4d30 con 0x7f4ad00c68b0 2023-10-30T12:54:17.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ad770f640 1 -- 172.21.15.12:0/2104168928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad0152370 con 0x7f4ad00c68b0 2023-10-30T12:54:17.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ad770f640 1 -- 172.21.15.12:0/2104168928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad0150da0 con 0x7f4ad00c68b0 2023-10-30T12:54:17.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ad770f640 1 -- 172.21.15.12:0/2104168928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad00c6d40 con 0x7f4ad00c68b0 2023-10-30T12:54:17.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.655+0000 7f4ac5ffb640 1 -- 172.21.15.12:0/2104168928 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4ac00b3380 con 0x7f4ad00c68b0 2023-10-30T12:54:17.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.659+0000 7f4ac5ffb640 1 --2- 172.21.15.12:0/2104168928 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4ab40664a0 0x7f4ab4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:17.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.659+0000 7f4ac5ffb640 1 -- 172.21.15.12:0/2104168928 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f4ac011e930 con 0x7f4ad00c68b0 2023-10-30T12:54:17.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.659+0000 7f4ad54ab640 1 --2- 172.21.15.12:0/2104168928 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4ab40664a0 0x7f4ab4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:17.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.659+0000 7f4ad54ab640 1 --2- 172.21.15.12:0/2104168928 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4ab40664a0 0x7f4ab4068960 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00ed30 tx=0x7f4acc00e360 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:17.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.663+0000 7f4ac5ffb640 1 -- 172.21.15.12:0/2104168928 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4ac00ebad0 con 0x7f4ad00c68b0 2023-10-30T12:54:17.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.859+0000 7f4ad770f640 1 -- 172.21.15.12:0/2104168928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ad004e170 con 0x7f4ad00c68b0 2023-10-30T12:54:17.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.859+0000 7f4ac5ffb640 1 -- 172.21.15.12:0/2104168928 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+6536 (secure 0 0 0) 0x7f4ac011ce60 con 0x7f4ad00c68b0 2023-10-30T12:54:17.864 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:17.864 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:09.537303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:17.865 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 98 2023-10-30T12:54:17.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.863+0000 7f4ad770f640 1 -- 172.21.15.12:0/2104168928 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4ab40664a0 msgr2=0x7f4ab4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:17.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.863+0000 7f4ad770f640 1 --2- 172.21.15.12:0/2104168928 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4ab40664a0 0x7f4ab4068960 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00ed30 tx=0x7f4acc00e360 comp rx=0 tx=0).stop 2023-10-30T12:54:17.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.863+0000 7f4ad770f640 1 -- 172.21.15.12:0/2104168928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad00c68b0 msgr2=0x7f4ad0151810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:17.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.863+0000 7f4ad770f640 1 --2- 172.21.15.12:0/2104168928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad00c68b0 0x7f4ad0151810 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00ad040 tx=0x7f4ac001ad50 comp rx=0 tx=0).stop 2023-10-30T12:54:17.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.863+0000 7f4ad770f640 1 -- 172.21.15.12:0/2104168928 shutdown_connections 2023-10-30T12:54:17.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.863+0000 7f4ad770f640 1 --2- 172.21.15.12:0/2104168928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0150560 0x7f4ad0140810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:17.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.863+0000 7f4ad770f640 1 --2- 172.21.15.12:0/2104168928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad00c68b0 0x7f4ad0151810 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:17.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.863+0000 7f4ad770f640 1 --2- 172.21.15.12:0/2104168928 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4ab40664a0 0x7f4ab4068960 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:17.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.863+0000 7f4ad770f640 1 --2- 172.21.15.12:0/2104168928 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ad0070650 0x7f4ad0143a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:17.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.863+0000 7f4ad770f640 1 -- 172.21.15.12:0/2104168928 >> 172.21.15.12:0/2104168928 conn(0x7f4ad00cbeb0 msgr2=0x7f4ad014ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:17.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.863+0000 7f4ad770f640 1 -- 172.21.15.12:0/2104168928 shutdown_connections 2023-10-30T12:54:17.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:17.863+0000 7f4ad770f640 1 -- 172.21.15.12:0/2104168928 wait complete. 2023-10-30T12:54:17.888 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:54:19.890 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:20.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d16b0e640 1 Processor -- start 2023-10-30T12:54:20.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d16b0e640 1 -- start start 2023-10-30T12:54:20.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d16b0e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4d101538a0 0x7f4d1015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:20.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d16b0e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4d100ca0e0 0x7f4d1015a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:20.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d16b0e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d100cb800 con 0x7f4d100cf160 2023-10-30T12:54:20.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d16b0e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1015dc30 con 0x7f4d100ca0e0 2023-10-30T12:54:20.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d16b0e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d100cb330 con 0x7f4d101538a0 2023-10-30T12:54:20.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d150ab640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4d100cf160 0x7f4d100cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44390/0 (socket says 172.21.15.12:44390) 2023-10-30T12:54:20.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d150ab640 1 -- 172.21.15.12:0/1126256072 learned_addr learned my addr 172.21.15.12:0/1126256072 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:54:20.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d0ffff640 1 --2- 172.21.15.12:0/1126256072 >> v2:172.21.15.12:3300/0 conn(0x7f4d100ca0e0 0x7f4d1015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:20.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d148aa640 1 --2- 172.21.15.12:0/1126256072 >> v2:172.21.15.154:3300/0 conn(0x7f4d101538a0 0x7f4d1015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:20.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d0f7fe640 1 -- 172.21.15.12:0/1126256072 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541628761 0 0) 0x7f4d100cb800 con 0x7f4d100cf160 2023-10-30T12:54:20.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d0f7fe640 1 -- 172.21.15.12:0/1126256072 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d00003530 con 0x7f4d100cf160 2023-10-30T12:54:20.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d0ffff640 1 -- 172.21.15.12:0/1126256072 >> v2:172.21.15.154:3300/0 conn(0x7f4d101538a0 msgr2=0x7f4d1015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:20.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d0ffff640 1 --2- 172.21.15.12:0/1126256072 >> v2:172.21.15.154:3300/0 conn(0x7f4d101538a0 0x7f4d1015d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:20.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d0ffff640 1 -- 172.21.15.12:0/1126256072 >> v1:172.21.15.12:6789/0 conn(0x7f4d100cf160 legacy=0x7f4d100cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:20.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.051+0000 7f4d0ffff640 1 -- 172.21.15.12:0/1126256072 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d100cad70 con 0x7f4d100ca0e0 2023-10-30T12:54:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.055+0000 7f4d0ffff640 1 --2- 172.21.15.12:0/1126256072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d100ca0e0 0x7f4d1015a090 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4d100cfd10 tx=0x7f4cfc01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb06f33c9e7e5a43 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.055+0000 7f4d0f7fe640 1 -- 172.21.15.12:0/1126256072 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc0a8950 con 0x7f4d100ca0e0 2023-10-30T12:54:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.055+0000 7f4d0f7fe640 1 -- 172.21.15.12:0/1126256072 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc0a8ab0 con 0x7f4d100ca0e0 2023-10-30T12:54:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.055+0000 7f4d0f7fe640 1 -- 172.21.15.12:0/1126256072 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc0a8e40 con 0x7f4d100ca0e0 2023-10-30T12:54:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.055+0000 7f4d16b0e640 1 -- 172.21.15.12:0/1126256072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d100ca0e0 msgr2=0x7f4d1015a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.055+0000 7f4d16b0e640 1 --2- 172.21.15.12:0/1126256072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d100ca0e0 0x7f4d1015a090 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4d100cfd10 tx=0x7f4cfc01ec50 comp rx=0 tx=0).stop 2023-10-30T12:54:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.055+0000 7f4d16b0e640 1 -- 172.21.15.12:0/1126256072 shutdown_connections 2023-10-30T12:54:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.055+0000 7f4d16b0e640 1 --2- 172.21.15.12:0/1126256072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d100ca0e0 0x7f4d1015a090 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.055+0000 7f4d16b0e640 1 --2- 172.21.15.12:0/1126256072 >> v2:172.21.15.154:3300/0 conn(0x7f4d101538a0 0x7f4d1015d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.055+0000 7f4d16b0e640 1 -- 172.21.15.12:0/1126256072 >> 172.21.15.12:0/1126256072 conn(0x7f4d1006bfe0 msgr2=0x7f4d1006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.055+0000 7f4d16b0e640 1 -- 172.21.15.12:0/1126256072 shutdown_connections 2023-10-30T12:54:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.055+0000 7f4d16b0e640 1 -- 172.21.15.12:0/1126256072 wait complete. 2023-10-30T12:54:20.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d16b0e640 1 Processor -- start 2023-10-30T12:54:20.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d16b0e640 1 -- start start 2023-10-30T12:54:20.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d16b0e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d100ca0e0 0x7f4d1015a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:20.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d16b0e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d100cf160 0x7f4d100c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:20.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d16b0e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d101538a0 0x7f4d101f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:20.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d16b0e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100cd7a0 con 0x7f4d100ca0e0 2023-10-30T12:54:20.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d16b0e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1015adf0 con 0x7f4d101538a0 2023-10-30T12:54:20.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d16b0e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1015cb10 con 0x7f4d100cf160 2023-10-30T12:54:20.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d148aa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d100ca0e0 0x7f4d1015a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:20.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d148aa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d100ca0e0 0x7f4d1015a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33678/0 (socket says 172.21.15.12:33678) 2023-10-30T12:54:20.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d148aa640 1 -- 172.21.15.12:0/1340117009 learned_addr learned my addr 172.21.15.12:0/1340117009 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:20.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d150ab640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d101538a0 0x7f4d101f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:20.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d0ffff640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d100cf160 0x7f4d100c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:20.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d148aa640 1 -- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d100cf160 msgr2=0x7f4d100c4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:20.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d148aa640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d100cf160 0x7f4d100c4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:20.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d148aa640 1 -- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d101538a0 msgr2=0x7f4d101f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:20.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d148aa640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d101538a0 0x7f4d101f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d148aa640 1 -- 172.21.15.12:0/1340117009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc009420 con 0x7f4d100ca0e0 2023-10-30T12:54:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d150ab640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d101538a0 0x7f4d101f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:54:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d0ffff640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d100cf160 0x7f4d100c4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:54:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d148aa640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d100ca0e0 0x7f4d1015a8b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400e870 tx=0x7f4d0400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d0d7fa640 1 -- 172.21.15.12:0/1340117009 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d04009510 con 0x7f4d100ca0e0 2023-10-30T12:54:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d0d7fa640 1 -- 172.21.15.12:0/1340117009 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0400eec0 con 0x7f4d100ca0e0 2023-10-30T12:54:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d0d7fa640 1 -- 172.21.15.12:0/1340117009 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0400cb40 con 0x7f4d100ca0e0 2023-10-30T12:54:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d16b0e640 1 -- 172.21.15.12:0/1340117009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d10157600 con 0x7f4d100ca0e0 2023-10-30T12:54:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.059+0000 7f4d16b0e640 1 -- 172.21.15.12:0/1340117009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d100cd150 con 0x7f4d100ca0e0 2023-10-30T12:54:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.063+0000 7f4d16b0e640 1 -- 172.21.15.12:0/1340117009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d100b2a50 con 0x7f4d100ca0e0 2023-10-30T12:54:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.063+0000 7f4d0d7fa640 1 -- 172.21.15.12:0/1340117009 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4d0400cca0 con 0x7f4d100ca0e0 2023-10-30T12:54:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.063+0000 7f4d0d7fa640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4cf40664a0 0x7f4cf4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:20.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.063+0000 7f4d0d7fa640 1 -- 172.21.15.12:0/1340117009 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f4d04084460 con 0x7f4d100ca0e0 2023-10-30T12:54:20.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.063+0000 7f4d0ffff640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4cf40664a0 0x7f4cf4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:20.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.063+0000 7f4d0ffff640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4cf40664a0 0x7f4cf4068960 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0b2000 tx=0x7f4cfc0aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:20.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.067+0000 7f4d0d7fa640 1 -- 172.21.15.12:0/1340117009 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4d04051430 con 0x7f4d100ca0e0 2023-10-30T12:54:20.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.255+0000 7f4d16b0e640 1 -- 172.21.15.12:0/1340117009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d100ccfb0 con 0x7f4d100ca0e0 2023-10-30T12:54:20.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.255+0000 7f4d0d7fa640 1 -- 172.21.15.12:0/1340117009 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+6536 (secure 0 0 0) 0x7f4d100ccfb0 con 0x7f4d100ca0e0 2023-10-30T12:54:20.261 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:20.261 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:09.537303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 98 2023-10-30T12:54:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.259+0000 7f4d16b0e640 1 -- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4cf40664a0 msgr2=0x7f4cf4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.259+0000 7f4d16b0e640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4cf40664a0 0x7f4cf4068960 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0b2000 tx=0x7f4cfc0aa070 comp rx=0 tx=0).stop 2023-10-30T12:54:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.259+0000 7f4d16b0e640 1 -- 172.21.15.12:0/1340117009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d100ca0e0 msgr2=0x7f4d1015a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.259+0000 7f4d16b0e640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d100ca0e0 0x7f4d1015a8b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400e870 tx=0x7f4d0400ec70 comp rx=0 tx=0).stop 2023-10-30T12:54:20.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.259+0000 7f4d16b0e640 1 -- 172.21.15.12:0/1340117009 shutdown_connections 2023-10-30T12:54:20.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.259+0000 7f4d16b0e640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d101538a0 0x7f4d101f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:20.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.259+0000 7f4d16b0e640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4cf40664a0 0x7f4cf4068960 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:20.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.259+0000 7f4d16b0e640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d100cf160 0x7f4d100c4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:20.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.259+0000 7f4d16b0e640 1 --2- 172.21.15.12:0/1340117009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d100ca0e0 0x7f4d1015a8b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:20.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.259+0000 7f4d16b0e640 1 -- 172.21.15.12:0/1340117009 >> 172.21.15.12:0/1340117009 conn(0x7f4d1006bfe0 msgr2=0x7f4d10152b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:20.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.259+0000 7f4d16b0e640 1 -- 172.21.15.12:0/1340117009 shutdown_connections 2023-10-30T12:54:20.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:20.259+0000 7f4d16b0e640 1 -- 172.21.15.12:0/1340117009 wait complete. 2023-10-30T12:54:20.282 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:54:22.283 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:22.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.431+0000 7faf8b0d6640 1 Processor -- start 2023-10-30T12:54:22.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.431+0000 7faf8b0d6640 1 -- start start 2023-10-30T12:54:22.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.431+0000 7faf8b0d6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf841570f0 0x7faf8406ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:22.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.431+0000 7faf8b0d6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf840b2300 0x7faf84071b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:22.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.431+0000 7faf8b0d6640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf8406b3b0 con 0x7faf8406f370 2023-10-30T12:54:22.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.431+0000 7faf8b0d6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf840c38a0 con 0x7faf840b2300 2023-10-30T12:54:22.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.431+0000 7faf8b0d6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf840c3b20 con 0x7faf841570f0 2023-10-30T12:54:22.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.431+0000 7faf83fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf840b2300 0x7faf84071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:22.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.431+0000 7faf83fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf840b2300 0x7faf84071b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35124/0 (socket says 172.21.15.12:35124) 2023-10-30T12:54:22.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.431+0000 7faf83fff640 1 -- 172.21.15.12:0/1279992326 learned_addr learned my addr 172.21.15.12:0/1279992326 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:22.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.431+0000 7faf88e72640 1 --2- 172.21.15.12:0/1279992326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf841570f0 0x7faf8406ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:22.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.435+0000 7faf83fff640 1 -- 172.21.15.12:0/1279992326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf841570f0 msgr2=0x7faf8406ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:22.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.435+0000 7faf83fff640 1 --2- 172.21.15.12:0/1279992326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf841570f0 0x7faf8406ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:22.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.435+0000 7faf83fff640 1 -- 172.21.15.12:0/1279992326 >> v1:172.21.15.154:6789/0 conn(0x7faf8406f370 legacy=0x7faf840ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:22.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.435+0000 7faf83fff640 1 -- 172.21.15.12:0/1279992326 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf840ca9f0 con 0x7faf840b2300 2023-10-30T12:54:22.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.435+0000 7faf88e72640 1 --2- 172.21.15.12:0/1279992326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf841570f0 0x7faf8406ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:22.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.435+0000 7faf83fff640 1 --2- 172.21.15.12:0/1279992326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf840b2300 0x7faf84071b30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7faf70009870 tx=0x7faf7001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbda08acdf9505a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:22.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.435+0000 7faf837fe640 1 -- 172.21.15.12:0/1279992326 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf700a7070 con 0x7faf840b2300 2023-10-30T12:54:22.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.435+0000 7faf837fe640 1 -- 172.21.15.12:0/1279992326 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf700ab430 con 0x7faf840b2300 2023-10-30T12:54:22.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.435+0000 7faf837fe640 1 -- 172.21.15.12:0/1279992326 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf700ab7c0 con 0x7faf840b2300 2023-10-30T12:54:22.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.435+0000 7faf8b0d6640 1 -- 172.21.15.12:0/1279992326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf840b2300 msgr2=0x7faf84071b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:22.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.435+0000 7faf8b0d6640 1 --2- 172.21.15.12:0/1279992326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf840b2300 0x7faf84071b30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7faf70009870 tx=0x7faf7001c540 comp rx=0 tx=0).stop 2023-10-30T12:54:22.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.435+0000 7faf8b0d6640 1 -- 172.21.15.12:0/1279992326 shutdown_connections 2023-10-30T12:54:22.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.435+0000 7faf8b0d6640 1 --2- 172.21.15.12:0/1279992326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf840b2300 0x7faf84071b30 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:22.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.435+0000 7faf8b0d6640 1 --2- 172.21.15.12:0/1279992326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf841570f0 0x7faf8406ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:22.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.435+0000 7faf8b0d6640 1 -- 172.21.15.12:0/1279992326 >> 172.21.15.12:0/1279992326 conn(0x7faf84069e90 msgr2=0x7faf8406a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:22.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf8b0d6640 1 -- 172.21.15.12:0/1279992326 shutdown_connections 2023-10-30T12:54:22.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf8b0d6640 1 -- 172.21.15.12:0/1279992326 wait complete. 2023-10-30T12:54:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf8b0d6640 1 Processor -- start 2023-10-30T12:54:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf8b0d6640 1 -- start start 2023-10-30T12:54:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf8b0d6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf8406f370 0x7faf840cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf8b0d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf840b2300 0x7faf8414e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf8b0d6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf841570f0 0x7faf8414eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:22.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf8b0d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf840cc150 con 0x7faf840b2300 2023-10-30T12:54:22.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf8b0d6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf841442c0 con 0x7faf841570f0 2023-10-30T12:54:22.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf8b0d6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf84143760 con 0x7faf8406f370 2023-10-30T12:54:22.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf83fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf840b2300 0x7faf8414e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:22.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf83fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf840b2300 0x7faf8414e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35132/0 (socket says 172.21.15.12:35132) 2023-10-30T12:54:22.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf83fff640 1 -- 172.21.15.12:0/786542151 learned_addr learned my addr 172.21.15.12:0/786542151 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:22.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf88e72640 1 --2- 172.21.15.12:0/786542151 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf8406f370 0x7faf840cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:22.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf89673640 1 --2- 172.21.15.12:0/786542151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf841570f0 0x7faf8414eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:22.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf83fff640 1 -- 172.21.15.12:0/786542151 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf8406f370 msgr2=0x7faf840cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:22.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf83fff640 1 --2- 172.21.15.12:0/786542151 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf8406f370 0x7faf840cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf83fff640 1 -- 172.21.15.12:0/786542151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf841570f0 msgr2=0x7faf8414eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf83fff640 1 --2- 172.21.15.12:0/786542151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf841570f0 0x7faf8414eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf83fff640 1 -- 172.21.15.12:0/786542151 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf700094d0 con 0x7faf840b2300 2023-10-30T12:54:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf89673640 1 --2- 172.21.15.12:0/786542151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf841570f0 0x7faf8414eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.439+0000 7faf83fff640 1 --2- 172.21.15.12:0/786542151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf840b2300 0x7faf8414e5b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7faf7001a170 tx=0x7faf700192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:22.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.443+0000 7faf817fa640 1 -- 172.21.15.12:0/786542151 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf700ba070 con 0x7faf840b2300 2023-10-30T12:54:22.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.443+0000 7faf817fa640 1 -- 172.21.15.12:0/786542151 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf700abe10 con 0x7faf840b2300 2023-10-30T12:54:22.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.443+0000 7faf817fa640 1 -- 172.21.15.12:0/786542151 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf700b4870 con 0x7faf840b2300 2023-10-30T12:54:22.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.443+0000 7faf8b0d6640 1 -- 172.21.15.12:0/786542151 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf8414dc30 con 0x7faf840b2300 2023-10-30T12:54:22.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.443+0000 7faf8b0d6640 1 -- 172.21.15.12:0/786542151 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf84143d00 con 0x7faf840b2300 2023-10-30T12:54:22.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.443+0000 7faf8b0d6640 1 -- 172.21.15.12:0/786542151 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8404e170 con 0x7faf840b2300 2023-10-30T12:54:22.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.443+0000 7faf817fa640 1 -- 172.21.15.12:0/786542151 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faf700b49d0 con 0x7faf840b2300 2023-10-30T12:54:22.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.443+0000 7faf817fa640 1 --2- 172.21.15.12:0/786542151 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faf680664a0 0x7faf68068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:22.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.443+0000 7faf817fa640 1 -- 172.21.15.12:0/786542151 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7faf7011dc60 con 0x7faf840b2300 2023-10-30T12:54:22.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.443+0000 7faf88e72640 1 --2- 172.21.15.12:0/786542151 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faf680664a0 0x7faf68068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:22.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.447+0000 7faf88e72640 1 --2- 172.21.15.12:0/786542151 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faf680664a0 0x7faf68068960 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faf7800b9f0 tx=0x7faf78008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:22.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.447+0000 7faf817fa640 1 -- 172.21.15.12:0/786542151 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faf700ead50 con 0x7faf840b2300 2023-10-30T12:54:22.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.639+0000 7faf8b0d6640 1 -- 172.21.15.12:0/786542151 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf8406fbd0 con 0x7faf840b2300 2023-10-30T12:54:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.643+0000 7faf817fa640 1 -- 172.21.15.12:0/786542151 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+6536 (secure 0 0 0) 0x7faf700ee370 con 0x7faf840b2300 2023-10-30T12:54:22.646 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:22.647 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:09.537303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:22.647 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 98 2023-10-30T12:54:22.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.643+0000 7faf8b0d6640 1 -- 172.21.15.12:0/786542151 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faf680664a0 msgr2=0x7faf68068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:22.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.643+0000 7faf8b0d6640 1 --2- 172.21.15.12:0/786542151 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faf680664a0 0x7faf68068960 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faf7800b9f0 tx=0x7faf78008040 comp rx=0 tx=0).stop 2023-10-30T12:54:22.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.643+0000 7faf8b0d6640 1 -- 172.21.15.12:0/786542151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf840b2300 msgr2=0x7faf8414e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:22.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.643+0000 7faf8b0d6640 1 --2- 172.21.15.12:0/786542151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf840b2300 0x7faf8414e5b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7faf7001a170 tx=0x7faf700192e0 comp rx=0 tx=0).stop 2023-10-30T12:54:22.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.643+0000 7faf8b0d6640 1 -- 172.21.15.12:0/786542151 shutdown_connections 2023-10-30T12:54:22.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.643+0000 7faf8b0d6640 1 --2- 172.21.15.12:0/786542151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf841570f0 0x7faf8414eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:22.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.643+0000 7faf8b0d6640 1 --2- 172.21.15.12:0/786542151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf840b2300 0x7faf8414e5b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:22.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.643+0000 7faf8b0d6640 1 --2- 172.21.15.12:0/786542151 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7faf680664a0 0x7faf68068960 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:22.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.643+0000 7faf8b0d6640 1 --2- 172.21.15.12:0/786542151 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf8406f370 0x7faf840cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:22.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.643+0000 7faf8b0d6640 1 -- 172.21.15.12:0/786542151 >> 172.21.15.12:0/786542151 conn(0x7faf84069e90 msgr2=0x7faf841557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:22.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.643+0000 7faf8b0d6640 1 -- 172.21.15.12:0/786542151 shutdown_connections 2023-10-30T12:54:22.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:22.643+0000 7faf8b0d6640 1 -- 172.21.15.12:0/786542151 wait complete. 2023-10-30T12:54:22.667 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:54:24.668 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:24.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.819+0000 7feed5246640 1 Processor -- start 2023-10-30T12:54:24.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.819+0000 7feed5246640 1 -- start start 2023-10-30T12:54:24.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.819+0000 7feed5246640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feed01570f0 0x7feed006ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:24.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.823+0000 7feed5246640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7feed00b2300 0x7feed0071b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:24.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.823+0000 7feed5246640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed006b3b0 con 0x7feed006f370 2023-10-30T12:54:24.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.823+0000 7feed5246640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed00c38a0 con 0x7feed01570f0 2023-10-30T12:54:24.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.823+0000 7feed5246640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed00c3b20 con 0x7feed00b2300 2023-10-30T12:54:24.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.823+0000 7feeced9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feed01570f0 0x7feed006ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:24.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.823+0000 7feeced9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feed01570f0 0x7feed006ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35134/0 (socket says 172.21.15.12:35134) 2023-10-30T12:54:24.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.823+0000 7feeced9d640 1 -- 172.21.15.12:0/624489398 learned_addr learned my addr 172.21.15.12:0/624489398 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:24.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.823+0000 7feec7fff640 1 --2- 172.21.15.12:0/624489398 >> v2:172.21.15.154:3300/0 conn(0x7feed00b2300 0x7feed0071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:24.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.823+0000 7feeced9d640 1 -- 172.21.15.12:0/624489398 >> v2:172.21.15.154:3300/0 conn(0x7feed00b2300 msgr2=0x7feed0071b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:24.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.823+0000 7feeced9d640 1 --2- 172.21.15.12:0/624489398 >> v2:172.21.15.154:3300/0 conn(0x7feed00b2300 0x7feed0071b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:24.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.823+0000 7feeced9d640 1 -- 172.21.15.12:0/624489398 >> v1:172.21.15.154:6789/0 conn(0x7feed006f370 legacy=0x7feed00ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:24.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.823+0000 7feeced9d640 1 -- 172.21.15.12:0/624489398 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed00ca9f0 con 0x7feed01570f0 2023-10-30T12:54:24.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.823+0000 7feeced9d640 1 --2- 172.21.15.12:0/624489398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feed01570f0 0x7feed006ca30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7feed00ca8f0 tx=0x7feec001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa109722c64d24b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:24.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feece59c640 1 -- 172.21.15.12:0/624489398 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec00ab5f0 con 0x7feed01570f0 2023-10-30T12:54:24.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feece59c640 1 -- 172.21.15.12:0/624489398 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec00ab750 con 0x7feed01570f0 2023-10-30T12:54:24.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feece59c640 1 -- 172.21.15.12:0/624489398 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec00abae0 con 0x7feed01570f0 2023-10-30T12:54:24.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 -- 172.21.15.12:0/624489398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feed01570f0 msgr2=0x7feed006ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:24.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 --2- 172.21.15.12:0/624489398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feed01570f0 0x7feed006ca30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7feed00ca8f0 tx=0x7feec001c500 comp rx=0 tx=0).stop 2023-10-30T12:54:24.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 -- 172.21.15.12:0/624489398 shutdown_connections 2023-10-30T12:54:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 --2- 172.21.15.12:0/624489398 >> v2:172.21.15.154:3300/0 conn(0x7feed00b2300 0x7feed0071b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 --2- 172.21.15.12:0/624489398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feed01570f0 0x7feed006ca30 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 -- 172.21.15.12:0/624489398 >> 172.21.15.12:0/624489398 conn(0x7feed0069e90 msgr2=0x7feed006a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 -- 172.21.15.12:0/624489398 shutdown_connections 2023-10-30T12:54:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 -- 172.21.15.12:0/624489398 wait complete. 2023-10-30T12:54:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 Processor -- start 2023-10-30T12:54:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 -- start start 2023-10-30T12:54:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feed006f370 0x7feed0148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feed00b2300 0x7feed013fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feed01570f0 0x7feed0140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed013a330 con 0x7feed00b2300 2023-10-30T12:54:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed01f61c0 con 0x7feed01570f0 2023-10-30T12:54:24.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feed5246640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed01f58d0 con 0x7feed006f370 2023-10-30T12:54:24.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.827+0000 7feec7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feed00b2300 0x7feed013fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:24.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feec7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feed00b2300 0x7feed013fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35146/0 (socket says 172.21.15.12:35146) 2023-10-30T12:54:24.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feec7fff640 1 -- 172.21.15.12:0/2158534024 learned_addr learned my addr 172.21.15.12:0/2158534024 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:24.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feecf59e640 1 --2- 172.21.15.12:0/2158534024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feed01570f0 0x7feed0140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:24.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feeced9d640 1 --2- 172.21.15.12:0/2158534024 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feed006f370 0x7feed0148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:24.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feec7fff640 1 -- 172.21.15.12:0/2158534024 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feed006f370 msgr2=0x7feed0148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:24.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feec7fff640 1 --2- 172.21.15.12:0/2158534024 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feed006f370 0x7feed0148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:24.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feec7fff640 1 -- 172.21.15.12:0/2158534024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feed01570f0 msgr2=0x7feed0140060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:24.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feec7fff640 1 --2- 172.21.15.12:0/2158534024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feed01570f0 0x7feed0140060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:24.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feec7fff640 1 -- 172.21.15.12:0/2158534024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec00094d0 con 0x7feed00b2300 2023-10-30T12:54:24.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feecf59e640 1 --2- 172.21.15.12:0/2158534024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feed01570f0 0x7feed0140060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:24.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feec7fff640 1 --2- 172.21.15.12:0/2158534024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feed00b2300 0x7feed013fb20 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7feeb800d5f0 tx=0x7feeb800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:24.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feec77fe640 1 -- 172.21.15.12:0/2158534024 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb80125f0 con 0x7feed00b2300 2023-10-30T12:54:24.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feec77fe640 1 -- 172.21.15.12:0/2158534024 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb8012750 con 0x7feed00b2300 2023-10-30T12:54:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feec77fe640 1 -- 172.21.15.12:0/2158534024 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8012b00 con 0x7feed00b2300 2023-10-30T12:54:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feed5246640 1 -- 172.21.15.12:0/2158534024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed01f5b70 con 0x7feed00b2300 2023-10-30T12:54:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feed5246640 1 -- 172.21.15.12:0/2158534024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed0149460 con 0x7feed00b2300 2023-10-30T12:54:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feed5246640 1 -- 172.21.15.12:0/2158534024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed01f6300 con 0x7feed00b2300 2023-10-30T12:54:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feec77fe640 1 -- 172.21.15.12:0/2158534024 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feeb8012c60 con 0x7feed00b2300 2023-10-30T12:54:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.831+0000 7feec77fe640 1 --2- 172.21.15.12:0/2158534024 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feeb00664d0 0x7feeb0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.835+0000 7feec77fe640 1 -- 172.21.15.12:0/2158534024 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6599+0+0 (secure 0 0 0) 0x7feeb8084870 con 0x7feed00b2300 2023-10-30T12:54:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.835+0000 7feeced9d640 1 --2- 172.21.15.12:0/2158534024 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feeb00664d0 0x7feeb0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.835+0000 7feeced9d640 1 --2- 172.21.15.12:0/2158534024 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feeb00664d0 0x7feeb0068990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7feec00af000 tx=0x7feec00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:24.839+0000 7feec77fe640 1 -- 172.21.15.12:0/2158534024 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feed01f6300 con 0x7feed00b2300 2023-10-30T12:54:25.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:25.035+0000 7feed5246640 1 -- 172.21.15.12:0/2158534024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feed01f6300 con 0x7feed00b2300 2023-10-30T12:54:25.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:25.035+0000 7feec77fe640 1 -- 172.21.15.12:0/2158534024 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+6536 (secure 0 0 0) 0x7feed01f6300 con 0x7feed00b2300 2023-10-30T12:54:25.039 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:25.039 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:09.537303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":5758,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5758":{"gid":5758,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":30,"addr":"172.21.15.154:6839/2204547612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2204547612},{"type":"v1","addr":"172.21.15.154:6839","nonce":2204547612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:25.039 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 98 2023-10-30T12:54:25.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:25.035+0000 7feed5246640 1 -- 172.21.15.12:0/2158534024 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feeb00664d0 msgr2=0x7feeb0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:25.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:25.035+0000 7feed5246640 1 --2- 172.21.15.12:0/2158534024 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feeb00664d0 0x7feeb0068990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7feec00af000 tx=0x7feec00a7000 comp rx=0 tx=0).stop 2023-10-30T12:54:25.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:25.035+0000 7feed5246640 1 -- 172.21.15.12:0/2158534024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feed00b2300 msgr2=0x7feed013fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:25.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:25.035+0000 7feed5246640 1 --2- 172.21.15.12:0/2158534024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feed00b2300 0x7feed013fb20 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7feeb800d5f0 tx=0x7feeb800dab0 comp rx=0 tx=0).stop 2023-10-30T12:54:25.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:25.035+0000 7feed5246640 1 -- 172.21.15.12:0/2158534024 shutdown_connections 2023-10-30T12:54:25.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:25.035+0000 7feed5246640 1 --2- 172.21.15.12:0/2158534024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feed01570f0 0x7feed0140060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:25.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:25.035+0000 7feed5246640 1 --2- 172.21.15.12:0/2158534024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feed00b2300 0x7feed013fb20 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:25.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:25.035+0000 7feed5246640 1 --2- 172.21.15.12:0/2158534024 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feeb00664d0 0x7feeb0068990 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:25.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:25.035+0000 7feed5246640 1 --2- 172.21.15.12:0/2158534024 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feed006f370 0x7feed0148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:25.035+0000 7feed5246640 1 -- 172.21.15.12:0/2158534024 >> 172.21.15.12:0/2158534024 conn(0x7feed0069e90 msgr2=0x7feed00c8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:25.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:25.035+0000 7feed5246640 1 -- 172.21.15.12:0/2158534024 shutdown_connections 2023-10-30T12:54:25.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:25.035+0000 7feed5246640 1 -- 172.21.15.12:0/2158534024 wait complete. 2023-10-30T12:54:25.060 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-30T12:54:27.061 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:27.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.215+0000 7fdcb53f2640 1 Processor -- start 2023-10-30T12:54:27.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.215+0000 7fdcb53f2640 1 -- start start 2023-10-30T12:54:27.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.219+0000 7fdcb53f2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcb015cd50 0x7fdcb00c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:27.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.219+0000 7fdcb53f2640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb015a6c0 con 0x7fdcb00cefa0 2023-10-30T12:54:27.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.219+0000 7fdcb53f2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb015d300 con 0x7fdcb01538f0 2023-10-30T12:54:27.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.219+0000 7fdcb53f2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb015d5e0 con 0x7fdcb015cd50 2023-10-30T12:54:27.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.219+0000 7fdcb4bf1640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdcb00cefa0 0x7fdcb01640e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50142/0 (socket says 172.21.15.12:50142) 2023-10-30T12:54:27.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.219+0000 7fdcb4bf1640 1 -- 172.21.15.12:0/3631666442 learned_addr learned my addr 172.21.15.12:0/3631666442 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:54:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.219+0000 7fdcaf7fe640 1 --2- 172.21.15.12:0/3631666442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcb015cd50 0x7fdcb00c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.219+0000 7fdcaeffd640 1 -- 172.21.15.12:0/3631666442 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3118415474 0 0) 0x7fdcb015a6c0 con 0x7fdcb00cefa0 2023-10-30T12:54:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.219+0000 7fdcaeffd640 1 -- 172.21.15.12:0/3631666442 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc98003530 con 0x7fdcb00cefa0 2023-10-30T12:54:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.219+0000 7fdcaeffd640 1 -- 172.21.15.12:0/3631666442 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679319890 0 0) 0x7fdcb015d300 con 0x7fdcb01538f0 2023-10-30T12:54:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.219+0000 7fdcaeffd640 1 -- 172.21.15.12:0/3631666442 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb015a6c0 con 0x7fdcb01538f0 2023-10-30T12:54:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.219+0000 7fdcaf7fe640 1 -- 172.21.15.12:0/3631666442 >> v1:172.21.15.154:6789/0 conn(0x7fdcb01538f0 legacy=0x7fdcb0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.219+0000 7fdcaf7fe640 1 -- 172.21.15.12:0/3631666442 >> v1:172.21.15.12:6789/0 conn(0x7fdcb00cefa0 legacy=0x7fdcb01640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.219+0000 7fdcaf7fe640 1 -- 172.21.15.12:0/3631666442 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb015d8c0 con 0x7fdcb015cd50 2023-10-30T12:54:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.223+0000 7fdcaf7fe640 1 --2- 172.21.15.12:0/3631666442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcb015cd50 0x7fdcb00c9f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdca40097f0 tx=0x7fdca401eaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e9252d7b6b0db73 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.223+0000 7fdcaeffd640 1 -- 172.21.15.12:0/3631666442 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8001a00 con 0x7fdcb015cd50 2023-10-30T12:54:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.223+0000 7fdcaeffd640 1 -- 172.21.15.12:0/3631666442 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca40a85b0 con 0x7fdcb015cd50 2023-10-30T12:54:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.223+0000 7fdcaeffd640 1 -- 172.21.15.12:0/3631666442 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca40a8920 con 0x7fdcb015cd50 2023-10-30T12:54:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.223+0000 7fdcb53f2640 1 -- 172.21.15.12:0/3631666442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcb015cd50 msgr2=0x7fdcb00c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.223+0000 7fdcb53f2640 1 --2- 172.21.15.12:0/3631666442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcb015cd50 0x7fdcb00c9f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdca40097f0 tx=0x7fdca401eaf0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.223+0000 7fdcb53f2640 1 -- 172.21.15.12:0/3631666442 shutdown_connections 2023-10-30T12:54:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.223+0000 7fdcb53f2640 1 --2- 172.21.15.12:0/3631666442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcb015cd50 0x7fdcb00c9f90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.223+0000 7fdcb53f2640 1 -- 172.21.15.12:0/3631666442 >> 172.21.15.12:0/3631666442 conn(0x7fdcb006a640 msgr2=0x7fdcb0065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:27.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.223+0000 7fdcb53f2640 1 -- 172.21.15.12:0/3631666442 shutdown_connections 2023-10-30T12:54:27.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.223+0000 7fdcb53f2640 1 -- 172.21.15.12:0/3631666442 wait complete. 2023-10-30T12:54:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcb53f2640 1 Processor -- start 2023-10-30T12:54:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcb53f2640 1 -- start start 2023-10-30T12:54:27.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcb53f2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcb00cefa0 0x7fdcb01f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:27.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcb53f2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdcb01538f0 0x7fdcb0150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:27.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcb53f2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcb015cd50 0x7fdcb00b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:27.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcb53f2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb0152650 con 0x7fdcb00cefa0 2023-10-30T12:54:27.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcb53f2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb01ef850 con 0x7fdcb01538f0 2023-10-30T12:54:27.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcb53f2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb0159140 con 0x7fdcb015cd50 2023-10-30T12:54:27.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcaffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcb00cefa0 0x7fdcb01f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:27.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcaffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcb00cefa0 0x7fdcb01f2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35156/0 (socket says 172.21.15.12:35156) 2023-10-30T12:54:27.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcaffff640 1 -- 172.21.15.12:0/2110001647 learned_addr learned my addr 172.21.15.12:0/2110001647 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:27.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcb4bf1640 1 --2- 172.21.15.12:0/2110001647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcb015cd50 0x7fdcb00b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:27.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcaf7fe640 1 --2- 172.21.15.12:0/2110001647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdcb01538f0 0x7fdcb0150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:27.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcaffff640 1 -- 172.21.15.12:0/2110001647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcb015cd50 msgr2=0x7fdcb00b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:27.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcaffff640 1 --2- 172.21.15.12:0/2110001647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcb015cd50 0x7fdcb00b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcaffff640 1 -- 172.21.15.12:0/2110001647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdcb01538f0 msgr2=0x7fdcb0150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:27.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcaffff640 1 --2- 172.21.15.12:0/2110001647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdcb01538f0 0x7fdcb0150ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcaffff640 1 -- 172.21.15.12:0/2110001647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca4009420 con 0x7fdcb00cefa0 2023-10-30T12:54:27.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcaffff640 1 --2- 172.21.15.12:0/2110001647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcb00cefa0 0x7fdcb01f2e50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdca000c4c0 tx=0x7fdca000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:27.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcacff9640 1 -- 172.21.15.12:0/2110001647 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca00052d0 con 0x7fdcb00cefa0 2023-10-30T12:54:27.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcacff9640 1 -- 172.21.15.12:0/2110001647 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca0005430 con 0x7fdcb00cefa0 2023-10-30T12:54:27.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcacff9640 1 -- 172.21.15.12:0/2110001647 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0012870 con 0x7fdcb00cefa0 2023-10-30T12:54:27.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.227+0000 7fdcb53f2640 1 -- 172.21.15.12:0/2110001647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb0159390 con 0x7fdcb00cefa0 2023-10-30T12:54:27.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.231+0000 7fdcb53f2640 1 -- 172.21.15.12:0/2110001647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb01ef200 con 0x7fdcb00cefa0 2023-10-30T12:54:27.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.231+0000 7fdcb53f2640 1 -- 172.21.15.12:0/2110001647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb004e170 con 0x7fdcb00cefa0 2023-10-30T12:54:27.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.231+0000 7fdcacff9640 1 -- 172.21.15.12:0/2110001647 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdca00129d0 con 0x7fdcb00cefa0 2023-10-30T12:54:27.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.231+0000 7fdcacff9640 1 --2- 172.21.15.12:0/2110001647 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc9c0664d0 0x7fdc9c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:27.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.231+0000 7fdcacff9640 1 -- 172.21.15.12:0/2110001647 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fdca0083d20 con 0x7fdcb00cefa0 2023-10-30T12:54:27.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.231+0000 7fdcaf7fe640 1 --2- 172.21.15.12:0/2110001647 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc9c0664d0 0x7fdc9c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.235+0000 7fdcaf7fe640 1 --2- 172.21.15.12:0/2110001647 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc9c0664d0 0x7fdc9c068990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdca40b2000 tx=0x7fdca40aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.235+0000 7fdcacff9640 1 -- 172.21.15.12:0/2110001647 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdca0050de0 con 0x7fdcb00cefa0 2023-10-30T12:54:27.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.427+0000 7fdcb53f2640 1 -- 172.21.15.12:0/2110001647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcb00b2290 con 0x7fdcb00cefa0 2023-10-30T12:54:27.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.431+0000 7fdcacff9640 1 -- 172.21.15.12:0/2110001647 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+5741 (secure 0 0 0) 0x7fdca0054400 con 0x7fdcb00cefa0 2023-10-30T12:54:27.434 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:27.435 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":99,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:25.460406+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:27.435 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 99 2023-10-30T12:54:27.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.431+0000 7fdcb53f2640 1 -- 172.21.15.12:0/2110001647 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc9c0664d0 msgr2=0x7fdc9c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:27.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.431+0000 7fdcb53f2640 1 --2- 172.21.15.12:0/2110001647 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc9c0664d0 0x7fdc9c068990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdca40b2000 tx=0x7fdca40aa070 comp rx=0 tx=0).stop 2023-10-30T12:54:27.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.431+0000 7fdcb53f2640 1 -- 172.21.15.12:0/2110001647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcb00cefa0 msgr2=0x7fdcb01f2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:27.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.431+0000 7fdcb53f2640 1 --2- 172.21.15.12:0/2110001647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcb00cefa0 0x7fdcb01f2e50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdca000c4c0 tx=0x7fdca000c980 comp rx=0 tx=0).stop 2023-10-30T12:54:27.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.431+0000 7fdcb53f2640 1 -- 172.21.15.12:0/2110001647 shutdown_connections 2023-10-30T12:54:27.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.431+0000 7fdcb53f2640 1 --2- 172.21.15.12:0/2110001647 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fdc9c0664d0 0x7fdc9c068990 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.431+0000 7fdcb53f2640 1 --2- 172.21.15.12:0/2110001647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcb015cd50 0x7fdcb00b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.431+0000 7fdcb53f2640 1 --2- 172.21.15.12:0/2110001647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdcb01538f0 0x7fdcb0150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.431+0000 7fdcb53f2640 1 --2- 172.21.15.12:0/2110001647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcb00cefa0 0x7fdcb01f2e50 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.431+0000 7fdcb53f2640 1 -- 172.21.15.12:0/2110001647 >> 172.21.15.12:0/2110001647 conn(0x7fdcb006a640 msgr2=0x7fdcb00c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:27.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.431+0000 7fdcb53f2640 1 -- 172.21.15.12:0/2110001647 shutdown_connections 2023-10-30T12:54:27.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.431+0000 7fdcb53f2640 1 -- 172.21.15.12:0/2110001647 wait complete. 2023-10-30T12:54:27.455 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2023-10-30T12:54:27.456 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:54:27.456 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:27.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.603+0000 7f311a31d640 1 Processor -- start 2023-10-30T12:54:27.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.603+0000 7f311a31d640 1 -- start start 2023-10-30T12:54:27.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.603+0000 7f311a31d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f311415d6f0 0x7f311415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:27.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.603+0000 7f311a31d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31140cb800 con 0x7f31140cf160 2023-10-30T12:54:27.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.603+0000 7f311a31d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31140cb330 con 0x7f31141538a0 2023-10-30T12:54:27.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.603+0000 7f311a31d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31140cad70 con 0x7f311415d6f0 2023-10-30T12:54:27.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.603+0000 7f31188ba640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f31140cf160 0x7f3114164070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50156/0 (socket says 172.21.15.12:50156) 2023-10-30T12:54:27.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.603+0000 7f31188ba640 1 -- 172.21.15.12:0/778361665 learned_addr learned my addr 172.21.15.12:0/778361665 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:54:27.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.603+0000 7f31137fe640 1 --2- 172.21.15.12:0/778361665 >> v2:172.21.15.154:3300/0 conn(0x7f311415d6f0 0x7f311415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:27.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.603+0000 7f3112ffd640 1 -- 172.21.15.12:0/778361665 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152010268 0 0) 0x7f31140cb800 con 0x7f31140cf160 2023-10-30T12:54:27.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.603+0000 7f3112ffd640 1 -- 172.21.15.12:0/778361665 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30fc003530 con 0x7f31140cf160 2023-10-30T12:54:27.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.603+0000 7f31137fe640 1 -- 172.21.15.12:0/778361665 >> v1:172.21.15.154:6789/0 conn(0x7f31141538a0 legacy=0x7f31140ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:27.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.603+0000 7f3112ffd640 1 -- 172.21.15.12:0/778361665 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3182564740 0 0) 0x7f30fc003530 con 0x7f31140cf160 2023-10-30T12:54:27.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.603+0000 7f31137fe640 1 -- 172.21.15.12:0/778361665 >> v1:172.21.15.12:6789/0 conn(0x7f31140cf160 legacy=0x7f3114164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:27.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.603+0000 7f31137fe640 1 -- 172.21.15.12:0/778361665 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31140cb050 con 0x7f311415d6f0 2023-10-30T12:54:27.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.607+0000 7f31137fe640 1 --2- 172.21.15.12:0/778361665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f311415d6f0 0x7f311415a090 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f31080097c0 tx=0x7f310801eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ed7e4a8646ab37f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.611+0000 7f3112ffd640 1 -- 172.21.15.12:0/778361665 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c001a00 con 0x7f311415d6f0 2023-10-30T12:54:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.611+0000 7f3112ffd640 1 -- 172.21.15.12:0/778361665 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31080a8610 con 0x7f311415d6f0 2023-10-30T12:54:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.611+0000 7f3112ffd640 1 -- 172.21.15.12:0/778361665 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31080a8980 con 0x7f311415d6f0 2023-10-30T12:54:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.611+0000 7f311a31d640 1 -- 172.21.15.12:0/778361665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f311415d6f0 msgr2=0x7f311415a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.611+0000 7f311a31d640 1 --2- 172.21.15.12:0/778361665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f311415d6f0 0x7f311415a090 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f31080097c0 tx=0x7f310801eb90 comp rx=0 tx=0).stop 2023-10-30T12:54:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.611+0000 7f311a31d640 1 -- 172.21.15.12:0/778361665 shutdown_connections 2023-10-30T12:54:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.611+0000 7f311a31d640 1 --2- 172.21.15.12:0/778361665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f311415d6f0 0x7f311415a090 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.611+0000 7f311a31d640 1 -- 172.21.15.12:0/778361665 >> 172.21.15.12:0/778361665 conn(0x7f311406bfe0 msgr2=0x7f311406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:27.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.611+0000 7f311a31d640 1 -- 172.21.15.12:0/778361665 shutdown_connections 2023-10-30T12:54:27.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.611+0000 7f311a31d640 1 -- 172.21.15.12:0/778361665 wait complete. 2023-10-30T12:54:27.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.611+0000 7f311a31d640 1 Processor -- start 2023-10-30T12:54:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.611+0000 7f311a31d640 1 -- start start 2023-10-30T12:54:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f311a31d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31140cf160 0x7f31141410c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f311a31d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31141538a0 0x7f3114157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f311a31d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f311415d6f0 0x7f3114146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f311a31d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31141e5600 con 0x7f31140cf160 2023-10-30T12:54:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f311a31d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31140b3d80 con 0x7f31141538a0 2023-10-30T12:54:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f311a31d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31140b3aa0 con 0x7f311415d6f0 2023-10-30T12:54:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f3113fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31140cf160 0x7f31141410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f3113fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31140cf160 0x7f31141410c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35166/0 (socket says 172.21.15.12:35166) 2023-10-30T12:54:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f3113fff640 1 -- 172.21.15.12:0/4055653679 learned_addr learned my addr 172.21.15.12:0/4055653679 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f31188ba640 1 --2- 172.21.15.12:0/4055653679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f311415d6f0 0x7f3114146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f31137fe640 1 --2- 172.21.15.12:0/4055653679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31141538a0 0x7f3114157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f3113fff640 1 -- 172.21.15.12:0/4055653679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f311415d6f0 msgr2=0x7f3114146e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f3113fff640 1 --2- 172.21.15.12:0/4055653679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f311415d6f0 0x7f3114146e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f3113fff640 1 -- 172.21.15.12:0/4055653679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31141538a0 msgr2=0x7f3114157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f3113fff640 1 --2- 172.21.15.12:0/4055653679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31141538a0 0x7f3114157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f3113fff640 1 -- 172.21.15.12:0/4055653679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3108009420 con 0x7f31140cf160 2023-10-30T12:54:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f3113fff640 1 --2- 172.21.15.12:0/4055653679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31140cf160 0x7f31141410c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f310400c530 tx=0x7f310400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f3110ff9640 1 -- 172.21.15.12:0/4055653679 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104005220 con 0x7f31140cf160 2023-10-30T12:54:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f3110ff9640 1 -- 172.21.15.12:0/4055653679 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3104005380 con 0x7f31140cf160 2023-10-30T12:54:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f311a31d640 1 -- 172.21.15.12:0/4055653679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3114159310 con 0x7f31140cf160 2023-10-30T12:54:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f3110ff9640 1 -- 172.21.15.12:0/4055653679 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104007270 con 0x7f31140cf160 2023-10-30T12:54:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f311a31d640 1 -- 172.21.15.12:0/4055653679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31141e5290 con 0x7f31140cf160 2023-10-30T12:54:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.615+0000 7f311a31d640 1 -- 172.21.15.12:0/4055653679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31141e5770 con 0x7f31140cf160 2023-10-30T12:54:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.619+0000 7f3110ff9640 1 -- 172.21.15.12:0/4055653679 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f31040073d0 con 0x7f31140cf160 2023-10-30T12:54:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.619+0000 7f3110ff9640 1 --2- 172.21.15.12:0/4055653679 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31000664a0 0x7f3100068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:27.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.619+0000 7f3110ff9640 1 -- 172.21.15.12:0/4055653679 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f3104083a30 con 0x7f31140cf160 2023-10-30T12:54:27.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.619+0000 7f31137fe640 1 --2- 172.21.15.12:0/4055653679 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31000664a0 0x7f3100068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:27.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.619+0000 7f31137fe640 1 --2- 172.21.15.12:0/4055653679 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31000664a0 0x7f3100068960 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f31080b2000 tx=0x7f31080aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:27.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.623+0000 7f3110ff9640 1 -- 172.21.15.12:0/4055653679 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3104050b80 con 0x7f31140cf160 2023-10-30T12:54:27.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.819+0000 7f311a31d640 1 -- 172.21.15.12:0/4055653679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f311404e170 con 0x7f31140cf160 2023-10-30T12:54:27.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.819+0000 7f3110ff9640 1 -- 172.21.15.12:0/4055653679 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+5741 (secure 0 0 0) 0x7f31040541a0 con 0x7f31140cf160 2023-10-30T12:54:27.825 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:27.826 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":99,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:25.460406+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:27.826 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 99 2023-10-30T12:54:27.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.823+0000 7f311a31d640 1 -- 172.21.15.12:0/4055653679 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31000664a0 msgr2=0x7f3100068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:27.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.823+0000 7f311a31d640 1 --2- 172.21.15.12:0/4055653679 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31000664a0 0x7f3100068960 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f31080b2000 tx=0x7f31080aa070 comp rx=0 tx=0).stop 2023-10-30T12:54:27.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.823+0000 7f311a31d640 1 -- 172.21.15.12:0/4055653679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31140cf160 msgr2=0x7f31141410c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:27.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.823+0000 7f311a31d640 1 --2- 172.21.15.12:0/4055653679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31140cf160 0x7f31141410c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f310400c530 tx=0x7f310400c9f0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.823+0000 7f311a31d640 1 -- 172.21.15.12:0/4055653679 shutdown_connections 2023-10-30T12:54:27.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.823+0000 7f311a31d640 1 --2- 172.21.15.12:0/4055653679 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f31000664a0 0x7f3100068960 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.823+0000 7f311a31d640 1 --2- 172.21.15.12:0/4055653679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f311415d6f0 0x7f3114146e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.823+0000 7f311a31d640 1 --2- 172.21.15.12:0/4055653679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31141538a0 0x7f3114157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.823+0000 7f311a31d640 1 --2- 172.21.15.12:0/4055653679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31140cf160 0x7f31141410c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:27.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.823+0000 7f311a31d640 1 -- 172.21.15.12:0/4055653679 >> 172.21.15.12:0/4055653679 conn(0x7f311406bfe0 msgr2=0x7f3114152b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:27.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.823+0000 7f311a31d640 1 -- 172.21.15.12:0/4055653679 shutdown_connections 2023-10-30T12:54:27.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:27.823+0000 7f311a31d640 1 -- 172.21.15.12:0/4055653679 wait complete. 2023-10-30T12:54:27.846 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 99, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 99, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5877": { "addr": "172.21.15.12:6835/2306258157", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 2306258157, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 2306258157, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 5877, "incarnation": 81, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_6021": { "addr": "172.21.15.12:6836/3877967042", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 3877967042, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 3877967042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 6021, "incarnation": 89, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 42 }, "gid_6192": { "addr": "172.21.15.12:6837/2954662612", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2954662612, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 2954662612, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6192, "incarnation": 99, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 55, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:54:25.460406+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6021, "mds_1": 6192, "mds_2": 5877 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:54:29.847 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:29.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.987+0000 7f08fe468640 1 Processor -- start 2023-10-30T12:54:29.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.987+0000 7f08fe468640 1 -- start start 2023-10-30T12:54:29.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.987+0000 7f08fe468640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08f8153d70 0x7f08f806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:29.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.987+0000 7f08fe468640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f08f806e470 0x7f08f80cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:29.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.987+0000 7f08fe468640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f806afa0 con 0x7f08f80c8490 2023-10-30T12:54:29.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.987+0000 7f08fe468640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f806d780 con 0x7f08f806e470 2023-10-30T12:54:29.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.987+0000 7f08fe468640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f8154150 con 0x7f08f8153d70 2023-10-30T12:54:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.991+0000 7f08f77fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f08f806e470 0x7f08f80cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.991+0000 7f08f77fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f08f806e470 0x7f08f80cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35178/0 (socket says 172.21.15.12:35178) 2023-10-30T12:54:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.991+0000 7f08f77fe640 1 -- 172.21.15.12:0/850208144 learned_addr learned my addr 172.21.15.12:0/850208144 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.991+0000 7f08f7fff640 1 --2- 172.21.15.12:0/850208144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08f8153d70 0x7f08f806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.991+0000 7f08f77fe640 1 -- 172.21.15.12:0/850208144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08f8153d70 msgr2=0x7f08f806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.991+0000 7f08f77fe640 1 --2- 172.21.15.12:0/850208144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08f8153d70 0x7f08f806ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.991+0000 7f08f77fe640 1 -- 172.21.15.12:0/850208144 >> v1:172.21.15.154:6789/0 conn(0x7f08f80c8490 legacy=0x7f08f80c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:29.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.991+0000 7f08f77fe640 1 -- 172.21.15.12:0/850208144 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f806e030 con 0x7f08f806e470 2023-10-30T12:54:29.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.991+0000 7f08f77fe640 1 --2- 172.21.15.12:0/850208144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08f806e470 0x7f08f80cce00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009870 tx=0x7f08ec01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6bc39ea6418a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:29.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.991+0000 7f08f6ffd640 1 -- 172.21.15.12:0/850208144 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0aa670 con 0x7f08f806e470 2023-10-30T12:54:29.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.991+0000 7f08f6ffd640 1 -- 172.21.15.12:0/850208144 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec0aa7d0 con 0x7f08f806e470 2023-10-30T12:54:29.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.991+0000 7f08f6ffd640 1 -- 172.21.15.12:0/850208144 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0aab60 con 0x7f08f806e470 2023-10-30T12:54:29.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 -- 172.21.15.12:0/850208144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08f806e470 msgr2=0x7f08f80cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:29.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 --2- 172.21.15.12:0/850208144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08f806e470 0x7f08f80cce00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009870 tx=0x7f08ec01c510 comp rx=0 tx=0).stop 2023-10-30T12:54:29.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 -- 172.21.15.12:0/850208144 shutdown_connections 2023-10-30T12:54:29.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 --2- 172.21.15.12:0/850208144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08f806e470 0x7f08f80cce00 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:30.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 --2- 172.21.15.12:0/850208144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08f8153d70 0x7f08f806ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:30.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 -- 172.21.15.12:0/850208144 >> 172.21.15.12:0/850208144 conn(0x7f08f8071ea0 msgr2=0x7f08f8065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:30.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 -- 172.21.15.12:0/850208144 shutdown_connections 2023-10-30T12:54:30.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 -- 172.21.15.12:0/850208144 wait complete. 2023-10-30T12:54:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 Processor -- start 2023-10-30T12:54:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 -- start start 2023-10-30T12:54:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08f806e470 0x7f08f8154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08f80c8490 0x7f08f8155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08f8153d70 0x7f08f81f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f80b0210 con 0x7f08f80c8490 2023-10-30T12:54:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f81542f0 con 0x7f08f806e470 2023-10-30T12:54:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fe468640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f8154870 con 0x7f08f8153d70 2023-10-30T12:54:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08f77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08f80c8490 0x7f08f8155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08f77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08f80c8490 0x7f08f8155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35186/0 (socket says 172.21.15.12:35186) 2023-10-30T12:54:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08f77fe640 1 -- 172.21.15.12:0/2198825662 learned_addr learned my addr 172.21.15.12:0/2198825662 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08f7fff640 1 --2- 172.21.15.12:0/2198825662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08f806e470 0x7f08f8154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.995+0000 7f08fca05640 1 --2- 172.21.15.12:0/2198825662 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08f8153d70 0x7f08f81f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08f77fe640 1 -- 172.21.15.12:0/2198825662 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08f8153d70 msgr2=0x7f08f81f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08f77fe640 1 --2- 172.21.15.12:0/2198825662 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08f8153d70 0x7f08f81f60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08f77fe640 1 -- 172.21.15.12:0/2198825662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08f806e470 msgr2=0x7f08f8154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08f77fe640 1 --2- 172.21.15.12:0/2198825662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08f806e470 0x7f08f8154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08f77fe640 1 -- 172.21.15.12:0/2198825662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec0094d0 con 0x7f08f80c8490 2023-10-30T12:54:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08f7fff640 1 --2- 172.21.15.12:0/2198825662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08f806e470 0x7f08f8154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08f77fe640 1 --2- 172.21.15.12:0/2198825662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08f80c8490 0x7f08f8155240 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0ac040 tx=0x7f08ec01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08f4ff9640 1 -- 172.21.15.12:0/2198825662 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0b3800 con 0x7f08f80c8490 2023-10-30T12:54:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08f4ff9640 1 -- 172.21.15.12:0/2198825662 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec0b3960 con 0x7f08f80c8490 2023-10-30T12:54:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08f4ff9640 1 -- 172.21.15.12:0/2198825662 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0b3d10 con 0x7f08f80c8490 2023-10-30T12:54:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08fe468640 1 -- 172.21.15.12:0/2198825662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f80b17c0 con 0x7f08f80c8490 2023-10-30T12:54:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08fe468640 1 -- 172.21.15.12:0/2198825662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f8156f90 con 0x7f08f80c8490 2023-10-30T12:54:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08fe468640 1 -- 172.21.15.12:0/2198825662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f8157120 con 0x7f08f80c8490 2023-10-30T12:54:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08f4ff9640 1 -- 172.21.15.12:0/2198825662 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f08ec0b2380 con 0x7f08f80c8490 2023-10-30T12:54:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08f4ff9640 1 --2- 172.21.15.12:0/2198825662 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f08e40664a0 0x7f08e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:29.999+0000 7f08f4ff9640 1 -- 172.21.15.12:0/2198825662 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f08ec11ea50 con 0x7f08f80c8490 2023-10-30T12:54:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.003+0000 7f08f7fff640 1 --2- 172.21.15.12:0/2198825662 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f08e40664a0 0x7f08e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.003+0000 7f08f7fff640 1 --2- 172.21.15.12:0/2198825662 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f08e40664a0 0x7f08e4068960 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009b10 tx=0x7f08e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:30.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.007+0000 7f08f4ff9640 1 -- 172.21.15.12:0/2198825662 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f08ec0ebb10 con 0x7f08f80c8490 2023-10-30T12:54:30.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.199+0000 7f08fe468640 1 -- 172.21.15.12:0/2198825662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08f804e170 con 0x7f08f80c8490 2023-10-30T12:54:30.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.199+0000 7f08f4ff9640 1 -- 172.21.15.12:0/2198825662 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+5741 (secure 0 0 0) 0x7f08ec0ef130 con 0x7f08f80c8490 2023-10-30T12:54:30.203 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:30.203 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":99,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:25.460406+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:30.204 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 99 2023-10-30T12:54:30.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.199+0000 7f08fe468640 1 -- 172.21.15.12:0/2198825662 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f08e40664a0 msgr2=0x7f08e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:30.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.199+0000 7f08fe468640 1 --2- 172.21.15.12:0/2198825662 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f08e40664a0 0x7f08e4068960 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009b10 tx=0x7f08e8008040 comp rx=0 tx=0).stop 2023-10-30T12:54:30.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.199+0000 7f08fe468640 1 -- 172.21.15.12:0/2198825662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08f80c8490 msgr2=0x7f08f8155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:30.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.199+0000 7f08fe468640 1 --2- 172.21.15.12:0/2198825662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08f80c8490 0x7f08f8155240 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0ac040 tx=0x7f08ec01ad50 comp rx=0 tx=0).stop 2023-10-30T12:54:30.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.199+0000 7f08fe468640 1 -- 172.21.15.12:0/2198825662 shutdown_connections 2023-10-30T12:54:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.199+0000 7f08fe468640 1 --2- 172.21.15.12:0/2198825662 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f08e40664a0 0x7f08e4068960 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.199+0000 7f08fe468640 1 --2- 172.21.15.12:0/2198825662 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08f8153d70 0x7f08f81f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.199+0000 7f08fe468640 1 --2- 172.21.15.12:0/2198825662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08f80c8490 0x7f08f8155240 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.199+0000 7f08fe468640 1 --2- 172.21.15.12:0/2198825662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08f806e470 0x7f08f8154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.199+0000 7f08fe468640 1 -- 172.21.15.12:0/2198825662 >> 172.21.15.12:0/2198825662 conn(0x7f08f8071ea0 msgr2=0x7f08f80c7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:30.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.199+0000 7f08fe468640 1 -- 172.21.15.12:0/2198825662 shutdown_connections 2023-10-30T12:54:30.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:30.199+0000 7f08fe468640 1 -- 172.21.15.12:0/2198825662 wait complete. 2023-10-30T12:54:30.224 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:54:32.225 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2484ad6640 1 Processor -- start 2023-10-30T12:54:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2484ad6640 1 -- start start 2023-10-30T12:54:32.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2484ad6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24800c6b80 0x7f24800bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:32.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2484ad6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f24800c6f60 0x7f24800c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:32.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2484ad6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f24800c8430 0x7f248013ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:32.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2484ad6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24800c0a50 con 0x7f24800c6f60 2023-10-30T12:54:32.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2484ad6640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248006f6e0 con 0x7f24800c8430 2023-10-30T12:54:32.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2484ad6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248006f9c0 con 0x7f24800c6b80 2023-10-30T12:54:32.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2477fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f24800c6f60 0x7f24800c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:32.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2477fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f24800c6f60 0x7f24800c0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51572/0 (socket says 172.21.15.12:51572) 2023-10-30T12:54:32.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2477fff640 1 -- 172.21.15.12:0/549040659 learned_addr learned my addr 172.21.15.12:0/549040659 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:32.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f247e59c640 1 --2- 172.21.15.12:0/549040659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24800c6b80 0x7f24800bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:32.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f247ed9d640 1 --2- 172.21.15.12:0/549040659 >> v2:172.21.15.154:3300/0 conn(0x7f24800c8430 0x7f248013ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:32.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2477fff640 1 -- 172.21.15.12:0/549040659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24800c6b80 msgr2=0x7f24800bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:32.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2477fff640 1 --2- 172.21.15.12:0/549040659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24800c6b80 0x7f24800bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:32.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2477fff640 1 -- 172.21.15.12:0/549040659 >> v2:172.21.15.154:3300/0 conn(0x7f24800c8430 msgr2=0x7f248013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:32.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2477fff640 1 --2- 172.21.15.12:0/549040659 >> v2:172.21.15.154:3300/0 conn(0x7f24800c8430 0x7f248013ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:32.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f2477fff640 1 -- 172.21.15.12:0/549040659 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248006fca0 con 0x7f24800c6f60 2023-10-30T12:54:32.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.367+0000 7f247ed9d640 1 --2- 172.21.15.12:0/549040659 >> v2:172.21.15.154:3300/0 conn(0x7f24800c8430 0x7f248013ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.371+0000 7f2477fff640 1 --2- 172.21.15.12:0/549040659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24800c6f60 0x7f24800c0400 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f24800b0860 tx=0x7f246801ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5685b5f16ef71da server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.371+0000 7f247dd9b640 1 -- 172.21.15.12:0/549040659 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24680a7b70 con 0x7f24800c6f60 2023-10-30T12:54:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.371+0000 7f247dd9b640 1 -- 172.21.15.12:0/549040659 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24680a7cd0 con 0x7f24800c6f60 2023-10-30T12:54:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.371+0000 7f247dd9b640 1 -- 172.21.15.12:0/549040659 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24680af660 con 0x7f24800c6f60 2023-10-30T12:54:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.371+0000 7f2484ad6640 1 -- 172.21.15.12:0/549040659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24800c6f60 msgr2=0x7f24800c0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.371+0000 7f2484ad6640 1 --2- 172.21.15.12:0/549040659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24800c6f60 0x7f24800c0400 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f24800b0860 tx=0x7f246801ebd0 comp rx=0 tx=0).stop 2023-10-30T12:54:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.371+0000 7f2484ad6640 1 -- 172.21.15.12:0/549040659 shutdown_connections 2023-10-30T12:54:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.371+0000 7f2484ad6640 1 --2- 172.21.15.12:0/549040659 >> v2:172.21.15.154:3300/0 conn(0x7f24800c8430 0x7f248013ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.371+0000 7f2484ad6640 1 --2- 172.21.15.12:0/549040659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24800c6f60 0x7f24800c0400 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.371+0000 7f2484ad6640 1 --2- 172.21.15.12:0/549040659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24800c6b80 0x7f24800bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.371+0000 7f2484ad6640 1 -- 172.21.15.12:0/549040659 >> 172.21.15.12:0/549040659 conn(0x7f24800c7910 msgr2=0x7f24800c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.371+0000 7f2484ad6640 1 -- 172.21.15.12:0/549040659 shutdown_connections 2023-10-30T12:54:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.371+0000 7f2484ad6640 1 -- 172.21.15.12:0/549040659 wait complete. 2023-10-30T12:54:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f2484ad6640 1 Processor -- start 2023-10-30T12:54:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f2484ad6640 1 -- start start 2023-10-30T12:54:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f2484ad6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24800c6b80 0x7f2480159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:32.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f2484ad6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24800c6f60 0x7f24801f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:32.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f2484ad6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24800c8430 0x7f24801f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:32.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f2484ad6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24800c9640 con 0x7f24800c8430 2023-10-30T12:54:32.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f2484ad6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24800ca1a0 con 0x7f24800c6f60 2023-10-30T12:54:32.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f2484ad6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24801f5470 con 0x7f24800c6b80 2023-10-30T12:54:32.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f247ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24800c8430 0x7f24801f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:32.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f247ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24800c8430 0x7f24801f22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51578/0 (socket says 172.21.15.12:51578) 2023-10-30T12:54:32.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f247ed9d640 1 -- 172.21.15.12:0/2762228872 learned_addr learned my addr 172.21.15.12:0/2762228872 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:32.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f247e59c640 1 --2- 172.21.15.12:0/2762228872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24800c6b80 0x7f2480159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f2477fff640 1 --2- 172.21.15.12:0/2762228872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24800c6f60 0x7f24801f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f247ed9d640 1 -- 172.21.15.12:0/2762228872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24800c6b80 msgr2=0x7f2480159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f247ed9d640 1 --2- 172.21.15.12:0/2762228872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24800c6b80 0x7f2480159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f247ed9d640 1 -- 172.21.15.12:0/2762228872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24800c6f60 msgr2=0x7f24801f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f247ed9d640 1 --2- 172.21.15.12:0/2762228872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24800c6f60 0x7f24801f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.375+0000 7f247ed9d640 1 -- 172.21.15.12:0/2762228872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246800b770 con 0x7f24800c8430 2023-10-30T12:54:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.379+0000 7f247ed9d640 1 --2- 172.21.15.12:0/2762228872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24800c8430 0x7f24801f22c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2478007df0 tx=0x7f247800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.379+0000 7f2476ffd640 1 -- 172.21.15.12:0/2762228872 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24780039a0 con 0x7f24800c8430 2023-10-30T12:54:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.379+0000 7f2476ffd640 1 -- 172.21.15.12:0/2762228872 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247800f040 con 0x7f24800c8430 2023-10-30T12:54:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.379+0000 7f2476ffd640 1 -- 172.21.15.12:0/2762228872 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478003d40 con 0x7f24800c8430 2023-10-30T12:54:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.379+0000 7f2484ad6640 1 -- 172.21.15.12:0/2762228872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24800c9870 con 0x7f24800c8430 2023-10-30T12:54:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.379+0000 7f2484ad6640 1 -- 172.21.15.12:0/2762228872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24800c9e30 con 0x7f24800c8430 2023-10-30T12:54:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.379+0000 7f2484ad6640 1 -- 172.21.15.12:0/2762228872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248004e170 con 0x7f24800c8430 2023-10-30T12:54:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.379+0000 7f2476ffd640 1 -- 172.21.15.12:0/2762228872 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2478004030 con 0x7f24800c8430 2023-10-30T12:54:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.379+0000 7f2476ffd640 1 --2- 172.21.15.12:0/2762228872 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f24600664a0 0x7f2460068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.379+0000 7f2476ffd640 1 -- 172.21.15.12:0/2762228872 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f2478084480 con 0x7f24800c8430 2023-10-30T12:54:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.379+0000 7f247e59c640 1 --2- 172.21.15.12:0/2762228872 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f24600664a0 0x7f2460068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.383+0000 7f247e59c640 1 --2- 172.21.15.12:0/2762228872 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f24600664a0 0x7f2460068960 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2470009ae0 tx=0x7f2470009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.387+0000 7f2476ffd640 1 -- 172.21.15.12:0/2762228872 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f24780515e0 con 0x7f24800c8430 2023-10-30T12:54:32.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.575+0000 7f2484ad6640 1 -- 172.21.15.12:0/2762228872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24800ca2e0 con 0x7f24800c8430 2023-10-30T12:54:32.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.579+0000 7f2476ffd640 1 -- 172.21.15.12:0/2762228872 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+5741 (secure 0 0 0) 0x7f2478054c00 con 0x7f24800c8430 2023-10-30T12:54:32.582 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:32.582 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":99,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:25.460406+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:32.583 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 99 2023-10-30T12:54:32.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.579+0000 7f2484ad6640 1 -- 172.21.15.12:0/2762228872 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f24600664a0 msgr2=0x7f2460068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:32.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.579+0000 7f2484ad6640 1 --2- 172.21.15.12:0/2762228872 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f24600664a0 0x7f2460068960 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2470009ae0 tx=0x7f2470009190 comp rx=0 tx=0).stop 2023-10-30T12:54:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.579+0000 7f2484ad6640 1 -- 172.21.15.12:0/2762228872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24800c8430 msgr2=0x7f24801f22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.579+0000 7f2484ad6640 1 --2- 172.21.15.12:0/2762228872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24800c8430 0x7f24801f22c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2478007df0 tx=0x7f247800d4a0 comp rx=0 tx=0).stop 2023-10-30T12:54:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.579+0000 7f2484ad6640 1 -- 172.21.15.12:0/2762228872 shutdown_connections 2023-10-30T12:54:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.579+0000 7f2484ad6640 1 --2- 172.21.15.12:0/2762228872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24800c8430 0x7f24801f22c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:32.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.579+0000 7f2484ad6640 1 --2- 172.21.15.12:0/2762228872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24800c6f60 0x7f24801f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:32.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.579+0000 7f2484ad6640 1 --2- 172.21.15.12:0/2762228872 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f24600664a0 0x7f2460068960 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:32.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.579+0000 7f2484ad6640 1 --2- 172.21.15.12:0/2762228872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24800c6b80 0x7f2480159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:32.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.579+0000 7f2484ad6640 1 -- 172.21.15.12:0/2762228872 >> 172.21.15.12:0/2762228872 conn(0x7f24800c7910 msgr2=0x7f24800c16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:32.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.579+0000 7f2484ad6640 1 -- 172.21.15.12:0/2762228872 shutdown_connections 2023-10-30T12:54:32.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:32.579+0000 7f2484ad6640 1 -- 172.21.15.12:0/2762228872 wait complete. 2023-10-30T12:54:32.604 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:54:34.605 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:34.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.747+0000 7f2fbcf3e640 1 Processor -- start 2023-10-30T12:54:34.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.747+0000 7f2fbcf3e640 1 -- start start 2023-10-30T12:54:34.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.747+0000 7f2fbcf3e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2fb806f7a0 0x7f2fb806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:34.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.747+0000 7f2fbcf3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb8071110 0x7f2fb8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:34.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.747+0000 7f2fbcf3e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb8066cf0 con 0x7f2fb80c6bd0 2023-10-30T12:54:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.747+0000 7f2fbcf3e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb80700c0 con 0x7f2fb806f7a0 2023-10-30T12:54:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.747+0000 7f2fbcf3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb80c70d0 con 0x7f2fb8071110 2023-10-30T12:54:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.751+0000 7f2fb659c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2fb806f7a0 0x7f2fb806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.751+0000 7f2fb5d9b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2fb80c6bd0 0x7f2fb8069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.12:44104/0 (socket says 172.21.15.12:44104) 2023-10-30T12:54:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.751+0000 7f2fb6d9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb8071110 0x7f2fb8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.751+0000 7f2fb5d9b640 1 -- 172.21.15.12:0/56384606 learned_addr learned my addr 172.21.15.12:0/56384606 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:54:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.751+0000 7f2fb559a640 1 -- 172.21.15.12:0/56384606 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299594641 0 0) 0x7f2fb8066cf0 con 0x7f2fb80c6bd0 2023-10-30T12:54:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.751+0000 7f2fb559a640 1 -- 172.21.15.12:0/56384606 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fa0003530 con 0x7f2fb80c6bd0 2023-10-30T12:54:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.751+0000 7f2fb659c640 1 -- 172.21.15.12:0/56384606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb8071110 msgr2=0x7f2fb8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.751+0000 7f2fb659c640 1 --2- 172.21.15.12:0/56384606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb8071110 0x7f2fb8158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.751+0000 7f2fb659c640 1 -- 172.21.15.12:0/56384606 >> v1:172.21.15.154:6789/0 conn(0x7f2fb80c6bd0 legacy=0x7f2fb8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.751+0000 7f2fb659c640 1 -- 172.21.15.12:0/56384606 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb80c7390 con 0x7f2fb806f7a0 2023-10-30T12:54:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.751+0000 7f2fb6d9d640 1 --2- 172.21.15.12:0/56384606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb8071110 0x7f2fb8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:54:34.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.751+0000 7f2fb659c640 1 --2- 172.21.15.12:0/56384606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb806f7a0 0x7f2fb806fb80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80097c0 tx=0x7f2fa801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c1f56874a57cf2c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fb559a640 1 -- 172.21.15.12:0/56384606 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa80a8970 con 0x7f2fb806f7a0 2023-10-30T12:54:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fb559a640 1 -- 172.21.15.12:0/56384606 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa80a8ad0 con 0x7f2fb806f7a0 2023-10-30T12:54:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fb559a640 1 -- 172.21.15.12:0/56384606 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa80a8e60 con 0x7f2fb806f7a0 2023-10-30T12:54:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 -- 172.21.15.12:0/56384606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb806f7a0 msgr2=0x7f2fb806fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 --2- 172.21.15.12:0/56384606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb806f7a0 0x7f2fb806fb80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80097c0 tx=0x7f2fa801ec70 comp rx=0 tx=0).stop 2023-10-30T12:54:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 -- 172.21.15.12:0/56384606 shutdown_connections 2023-10-30T12:54:34.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 --2- 172.21.15.12:0/56384606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb8071110 0x7f2fb8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:34.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 --2- 172.21.15.12:0/56384606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb806f7a0 0x7f2fb806fb80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:34.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 -- 172.21.15.12:0/56384606 >> 172.21.15.12:0/56384606 conn(0x7f2fb80c7910 msgr2=0x7f2fb80c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:34.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 -- 172.21.15.12:0/56384606 shutdown_connections 2023-10-30T12:54:34.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 -- 172.21.15.12:0/56384606 wait complete. 2023-10-30T12:54:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 Processor -- start 2023-10-30T12:54:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 -- start start 2023-10-30T12:54:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb806f7a0 0x7f2fb8153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb8071110 0x7f2fb81f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb80c6bd0 0x7f2fb80c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb81f20c0 con 0x7f2fb806f7a0 2023-10-30T12:54:34.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb81f2680 con 0x7f2fb80c6bd0 2023-10-30T12:54:34.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.755+0000 7f2fbcf3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb8155df0 con 0x7f2fb8071110 2023-10-30T12:54:34.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fb659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb806f7a0 0x7f2fb8153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:34.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fb659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb806f7a0 0x7f2fb8153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51588/0 (socket says 172.21.15.12:51588) 2023-10-30T12:54:34.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fb659c640 1 -- 172.21.15.12:0/3399514089 learned_addr learned my addr 172.21.15.12:0/3399514089 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:34.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fb6d9d640 1 --2- 172.21.15.12:0/3399514089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb80c6bd0 0x7f2fb80c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:34.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fb5d9b640 1 --2- 172.21.15.12:0/3399514089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb8071110 0x7f2fb81f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:34.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fb659c640 1 -- 172.21.15.12:0/3399514089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb8071110 msgr2=0x7f2fb81f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:34.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fb659c640 1 --2- 172.21.15.12:0/3399514089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb8071110 0x7f2fb81f2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:34.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fb659c640 1 -- 172.21.15.12:0/3399514089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb80c6bd0 msgr2=0x7f2fb80c3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:34.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fb659c640 1 --2- 172.21.15.12:0/3399514089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb80c6bd0 0x7f2fb80c3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:34.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fb659c640 1 -- 172.21.15.12:0/3399514089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb0009a70 con 0x7f2fb806f7a0 2023-10-30T12:54:34.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fb659c640 1 --2- 172.21.15.12:0/3399514089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb806f7a0 0x7f2fb8153dd0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80b3040 tx=0x7f2fa801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:34.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fa6ffd640 1 -- 172.21.15.12:0/3399514089 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa80b0d60 con 0x7f2fb806f7a0 2023-10-30T12:54:34.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fa6ffd640 1 -- 172.21.15.12:0/3399514089 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa80b9430 con 0x7f2fb806f7a0 2023-10-30T12:54:34.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fa6ffd640 1 -- 172.21.15.12:0/3399514089 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa80b97e0 con 0x7f2fb806f7a0 2023-10-30T12:54:34.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fbcf3e640 1 -- 172.21.15.12:0/3399514089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa8009420 con 0x7f2fb806f7a0 2023-10-30T12:54:34.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fbcf3e640 1 -- 172.21.15.12:0/3399514089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb81560b0 con 0x7f2fb806f7a0 2023-10-30T12:54:34.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fbcf3e640 1 -- 172.21.15.12:0/3399514089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb804e170 con 0x7f2fb806f7a0 2023-10-30T12:54:34.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fa6ffd640 1 -- 172.21.15.12:0/3399514089 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2fa80b7040 con 0x7f2fb806f7a0 2023-10-30T12:54:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.759+0000 7f2fa6ffd640 1 --2- 172.21.15.12:0/3399514089 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f980664a0 0x7f2f98068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.763+0000 7f2fa6ffd640 1 -- 172.21.15.12:0/3399514089 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f2fa8121d30 con 0x7f2fb806f7a0 2023-10-30T12:54:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.763+0000 7f2fb5d9b640 1 --2- 172.21.15.12:0/3399514089 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f980664a0 0x7f2f98068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.763+0000 7f2fb5d9b640 1 --2- 172.21.15.12:0/3399514089 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f980664a0 0x7f2f98068960 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80ca050 tx=0x7f2fac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:34.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.767+0000 7f2fa6ffd640 1 -- 172.21.15.12:0/3399514089 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2fa80eedd0 con 0x7f2fb806f7a0 2023-10-30T12:54:34.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.951+0000 7f2fbcf3e640 1 -- 172.21.15.12:0/3399514089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fb80c33d0 con 0x7f2fb806f7a0 2023-10-30T12:54:34.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.955+0000 7f2fa6ffd640 1 -- 172.21.15.12:0/3399514089 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+5745 (secure 0 0 0) 0x7f2fa80f23f0 con 0x7f2fb806f7a0 2023-10-30T12:54:34.958 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:34.959 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:32.932542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:resolve","state_seq":11,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:34.960 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 100 2023-10-30T12:54:34.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.955+0000 7f2fbcf3e640 1 -- 172.21.15.12:0/3399514089 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f980664a0 msgr2=0x7f2f98068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:34.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.955+0000 7f2fbcf3e640 1 --2- 172.21.15.12:0/3399514089 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f980664a0 0x7f2f98068960 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80ca050 tx=0x7f2fac009040 comp rx=0 tx=0).stop 2023-10-30T12:54:34.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.955+0000 7f2fbcf3e640 1 -- 172.21.15.12:0/3399514089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb806f7a0 msgr2=0x7f2fb8153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:34.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.955+0000 7f2fbcf3e640 1 --2- 172.21.15.12:0/3399514089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb806f7a0 0x7f2fb8153dd0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80b3040 tx=0x7f2fa801ba10 comp rx=0 tx=0).stop 2023-10-30T12:54:34.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.955+0000 7f2fbcf3e640 1 -- 172.21.15.12:0/3399514089 shutdown_connections 2023-10-30T12:54:34.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.955+0000 7f2fbcf3e640 1 --2- 172.21.15.12:0/3399514089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fb80c6bd0 0x7f2fb80c3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:34.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.955+0000 7f2fbcf3e640 1 --2- 172.21.15.12:0/3399514089 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f2f980664a0 0x7f2f98068960 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:34.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.955+0000 7f2fbcf3e640 1 --2- 172.21.15.12:0/3399514089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fb8071110 0x7f2fb81f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:34.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.955+0000 7f2fbcf3e640 1 --2- 172.21.15.12:0/3399514089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb806f7a0 0x7f2fb8153dd0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:34.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.955+0000 7f2fbcf3e640 1 -- 172.21.15.12:0/3399514089 >> 172.21.15.12:0/3399514089 conn(0x7f2fb80c7910 msgr2=0x7f2fb80c9470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:34.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.955+0000 7f2fbcf3e640 1 -- 172.21.15.12:0/3399514089 shutdown_connections 2023-10-30T12:54:34.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:34.955+0000 7f2fbcf3e640 1 -- 172.21.15.12:0/3399514089 wait complete. 2023-10-30T12:54:34.979 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:54:36.980 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:37.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.131+0000 7fb8dca17640 1 Processor -- start 2023-10-30T12:54:37.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.131+0000 7fb8dca17640 1 -- start start 2023-10-30T12:54:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.135+0000 7fb8dca17640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb8d80c05d0 0x7fb8d813ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:37.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.135+0000 7fb8dca17640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d80c0a50 con 0x7fb8d80c6b80 2023-10-30T12:54:37.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.135+0000 7fb8dca17640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d806f6e0 con 0x7fb8d80c6f60 2023-10-30T12:54:37.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.135+0000 7fb8dca17640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d806f9c0 con 0x7fb8d80c05d0 2023-10-30T12:54:37.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.135+0000 7fb8cffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb8d80c05d0 0x7fb8d813ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:37.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.135+0000 7fb8cffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb8d80c05d0 0x7fb8d813ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51602/0 (socket says 172.21.15.12:51602) 2023-10-30T12:54:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.135+0000 7fb8cffff640 1 -- 172.21.15.12:0/3528831822 learned_addr learned my addr 172.21.15.12:0/3528831822 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.135+0000 7fb8cffff640 1 -- 172.21.15.12:0/3528831822 >> v1:172.21.15.154:6789/0 conn(0x7fb8d80c6f60 legacy=0x7fb8d81566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.135+0000 7fb8cffff640 1 -- 172.21.15.12:0/3528831822 >> v1:172.21.15.12:6789/0 conn(0x7fb8d80c6b80 legacy=0x7fb8d80bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.135+0000 7fb8cffff640 1 -- 172.21.15.12:0/3528831822 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d806fca0 con 0x7fb8d80c05d0 2023-10-30T12:54:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.135+0000 7fb8d5d9b640 1 -- 172.21.15.12:0/3528831822 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905653014 0 0) 0x7fb8d80c0a50 con 0x7fb8d80c6b80 2023-10-30T12:54:37.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.139+0000 7fb8cffff640 1 --2- 172.21.15.12:0/3528831822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d80c05d0 0x7fb8d813ed90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009870 tx=0x7fb8c001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51464c7ff778f6f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:37.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.139+0000 7fb8d5d9b640 1 -- 172.21.15.12:0/3528831822 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c00ab610 con 0x7fb8d80c05d0 2023-10-30T12:54:37.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.139+0000 7fb8d5d9b640 1 -- 172.21.15.12:0/3528831822 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c00ab770 con 0x7fb8d80c05d0 2023-10-30T12:54:37.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.139+0000 7fb8d5d9b640 1 -- 172.21.15.12:0/3528831822 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c00abb00 con 0x7fb8d80c05d0 2023-10-30T12:54:37.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.139+0000 7fb8dca17640 1 -- 172.21.15.12:0/3528831822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d80c05d0 msgr2=0x7fb8d813ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:37.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.139+0000 7fb8dca17640 1 --2- 172.21.15.12:0/3528831822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d80c05d0 0x7fb8d813ed90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009870 tx=0x7fb8c001c500 comp rx=0 tx=0).stop 2023-10-30T12:54:37.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.139+0000 7fb8dca17640 1 -- 172.21.15.12:0/3528831822 shutdown_connections 2023-10-30T12:54:37.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.139+0000 7fb8dca17640 1 --2- 172.21.15.12:0/3528831822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d80c05d0 0x7fb8d813ed90 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:37.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.139+0000 7fb8dca17640 1 -- 172.21.15.12:0/3528831822 >> 172.21.15.12:0/3528831822 conn(0x7fb8d80c7910 msgr2=0x7fb8d80c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:37.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.139+0000 7fb8dca17640 1 -- 172.21.15.12:0/3528831822 shutdown_connections 2023-10-30T12:54:37.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.139+0000 7fb8dca17640 1 -- 172.21.15.12:0/3528831822 wait complete. 2023-10-30T12:54:37.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.139+0000 7fb8dca17640 1 Processor -- start 2023-10-30T12:54:37.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.139+0000 7fb8dca17640 1 -- start start 2023-10-30T12:54:37.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8dca17640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d80c05d0 0x7fb8d80ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:37.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8dca17640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8d80c6b80 0x7fb8d8134780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:37.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8dca17640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8d80c6f60 0x7fb8d80c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:37.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8dca17640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d80ca1b0 con 0x7fb8d80c05d0 2023-10-30T12:54:37.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8dca17640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d813c650 con 0x7fb8d80c6b80 2023-10-30T12:54:37.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8dca17640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d806ab90 con 0x7fb8d80c6f60 2023-10-30T12:54:37.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8d659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d80c05d0 0x7fb8d80ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:37.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8d659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d80c05d0 0x7fb8d80ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51610/0 (socket says 172.21.15.12:51610) 2023-10-30T12:54:37.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8d659c640 1 -- 172.21.15.12:0/2346837116 learned_addr learned my addr 172.21.15.12:0/2346837116 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:37.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8cffff640 1 --2- 172.21.15.12:0/2346837116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8d80c6b80 0x7fb8d8134780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:37.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8d6d9d640 1 --2- 172.21.15.12:0/2346837116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8d80c6f60 0x7fb8d80c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:37.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8d659c640 1 -- 172.21.15.12:0/2346837116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8d80c6f60 msgr2=0x7fb8d80c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:37.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8d659c640 1 --2- 172.21.15.12:0/2346837116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8d80c6f60 0x7fb8d80c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:37.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8d659c640 1 -- 172.21.15.12:0/2346837116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8d80c6b80 msgr2=0x7fb8d8134780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:37.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8d659c640 1 --2- 172.21.15.12:0/2346837116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8d80c6b80 0x7fb8d8134780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:37.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8d659c640 1 -- 172.21.15.12:0/2346837116 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c00094d0 con 0x7fb8d80c05d0 2023-10-30T12:54:37.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8d659c640 1 --2- 172.21.15.12:0/2346837116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d80c05d0 0x7fb8d80ca610 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800c4d0 tx=0x7fb8c800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:37.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8ceffd640 1 -- 172.21.15.12:0/2346837116 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8005480 con 0x7fb8d80c05d0 2023-10-30T12:54:37.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8ceffd640 1 -- 172.21.15.12:0/2346837116 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c8003600 con 0x7fb8d80c05d0 2023-10-30T12:54:37.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8ceffd640 1 -- 172.21.15.12:0/2346837116 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c80039b0 con 0x7fb8d80c05d0 2023-10-30T12:54:37.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8dca17640 1 -- 172.21.15.12:0/2346837116 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d8154630 con 0x7fb8d80c05d0 2023-10-30T12:54:37.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8dca17640 1 -- 172.21.15.12:0/2346837116 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d80c00e0 con 0x7fb8d80c05d0 2023-10-30T12:54:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8dca17640 1 -- 172.21.15.12:0/2346837116 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d804e170 con 0x7fb8d80c05d0 2023-10-30T12:54:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.143+0000 7fb8ceffd640 1 -- 172.21.15.12:0/2346837116 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb8c8003b10 con 0x7fb8d80c05d0 2023-10-30T12:54:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.147+0000 7fb8ceffd640 1 --2- 172.21.15.12:0/2346837116 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb8b80664a0 0x7fb8b8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.147+0000 7fb8ceffd640 1 -- 172.21.15.12:0/2346837116 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fb8c80837e0 con 0x7fb8d80c05d0 2023-10-30T12:54:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.147+0000 7fb8cffff640 1 --2- 172.21.15.12:0/2346837116 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb8b80664a0 0x7fb8b8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:37.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.147+0000 7fb8cffff640 1 --2- 172.21.15.12:0/2346837116 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb8b80664a0 0x7fb8b8068960 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00af000 tx=0x7fb8c00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:37.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.151+0000 7fb8ceffd640 1 -- 172.21.15.12:0/2346837116 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb8c8050820 con 0x7fb8d80c05d0 2023-10-30T12:54:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.343+0000 7fb8dca17640 1 -- 172.21.15.12:0/2346837116 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8d8154630 con 0x7fb8d80c05d0 2023-10-30T12:54:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.343+0000 7fb8ceffd640 1 -- 172.21.15.12:0/2346837116 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+5745 (secure 0 0 0) 0x7fb8d8154630 con 0x7fb8d80c05d0 2023-10-30T12:54:37.349 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:37.349 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:32.932542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:resolve","state_seq":11,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:37.350 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 100 2023-10-30T12:54:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.347+0000 7fb8dca17640 1 -- 172.21.15.12:0/2346837116 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb8b80664a0 msgr2=0x7fb8b8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.347+0000 7fb8dca17640 1 --2- 172.21.15.12:0/2346837116 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb8b80664a0 0x7fb8b8068960 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00af000 tx=0x7fb8c00a7000 comp rx=0 tx=0).stop 2023-10-30T12:54:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.347+0000 7fb8dca17640 1 -- 172.21.15.12:0/2346837116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d80c05d0 msgr2=0x7fb8d80ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.347+0000 7fb8dca17640 1 --2- 172.21.15.12:0/2346837116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d80c05d0 0x7fb8d80ca610 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800c4d0 tx=0x7fb8c800c990 comp rx=0 tx=0).stop 2023-10-30T12:54:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.347+0000 7fb8dca17640 1 -- 172.21.15.12:0/2346837116 shutdown_connections 2023-10-30T12:54:37.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.347+0000 7fb8dca17640 1 --2- 172.21.15.12:0/2346837116 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb8b80664a0 0x7fb8b8068960 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:37.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.347+0000 7fb8dca17640 1 --2- 172.21.15.12:0/2346837116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8d80c6f60 0x7fb8d80c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:37.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.347+0000 7fb8dca17640 1 --2- 172.21.15.12:0/2346837116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8d80c6b80 0x7fb8d8134780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:37.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.347+0000 7fb8dca17640 1 --2- 172.21.15.12:0/2346837116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d80c05d0 0x7fb8d80ca610 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:37.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.347+0000 7fb8dca17640 1 -- 172.21.15.12:0/2346837116 >> 172.21.15.12:0/2346837116 conn(0x7fb8d80c7910 msgr2=0x7fb8d8135f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:37.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.347+0000 7fb8dca17640 1 -- 172.21.15.12:0/2346837116 shutdown_connections 2023-10-30T12:54:37.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:37.347+0000 7fb8dca17640 1 -- 172.21.15.12:0/2346837116 wait complete. 2023-10-30T12:54:37.371 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:54:37.520 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:54:37.524 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:54:39.371 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.519+0000 7fe15cba0640 1 Processor -- start 2023-10-30T12:54:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.519+0000 7fe15cba0640 1 -- start start 2023-10-30T12:54:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.519+0000 7fe15cba0640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe1581538f0 0x7fe158159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:39.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.523+0000 7fe15cba0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe1580cefa0 0x7fe1580cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:39.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.523+0000 7fe15cba0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1580ca6a0 con 0x7fe15815cd50 2023-10-30T12:54:39.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.523+0000 7fe15cba0640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1580cf8f0 con 0x7fe1581538f0 2023-10-30T12:54:39.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.523+0000 7fe15cba0640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15815d300 con 0x7fe1580cefa0 2023-10-30T12:54:39.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.523+0000 7fe1577fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe1581538f0 0x7fe158159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:39.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.523+0000 7fe1577fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe1581538f0 0x7fe158159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51612/0 (socket says 172.21.15.12:51612) 2023-10-30T12:54:39.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.523+0000 7fe1577fe640 1 -- 172.21.15.12:0/3779601778 learned_addr learned my addr 172.21.15.12:0/3779601778 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:39.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.523+0000 7fe157fff640 1 --2- 172.21.15.12:0/3779601778 >> v2:172.21.15.154:3300/0 conn(0x7fe1580cefa0 0x7fe1580cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:39.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.523+0000 7fe1577fe640 1 -- 172.21.15.12:0/3779601778 >> v2:172.21.15.154:3300/0 conn(0x7fe1580cefa0 msgr2=0x7fe1580cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:39.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.523+0000 7fe1577fe640 1 --2- 172.21.15.12:0/3779601778 >> v2:172.21.15.154:3300/0 conn(0x7fe1580cefa0 0x7fe1580cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:39.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.523+0000 7fe1577fe640 1 -- 172.21.15.12:0/3779601778 >> v1:172.21.15.154:6789/0 conn(0x7fe15815cd50 legacy=0x7fe1580c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:39.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.523+0000 7fe1577fe640 1 -- 172.21.15.12:0/3779601778 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15815d5e0 con 0x7fe1581538f0 2023-10-30T12:54:39.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.527+0000 7fe1577fe640 1 --2- 172.21.15.12:0/3779601778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1581538f0 0x7fe158159fb0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe158066a40 tx=0x7fe14801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13deb80fb691baa9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:39.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.527+0000 7fe1567fc640 1 -- 172.21.15.12:0/3779601778 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480a89f0 con 0x7fe1581538f0 2023-10-30T12:54:39.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.527+0000 7fe1567fc640 1 -- 172.21.15.12:0/3779601778 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1480a8b50 con 0x7fe1581538f0 2023-10-30T12:54:39.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.527+0000 7fe1567fc640 1 -- 172.21.15.12:0/3779601778 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480b0440 con 0x7fe1581538f0 2023-10-30T12:54:39.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.527+0000 7fe15cba0640 1 -- 172.21.15.12:0/3779601778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1581538f0 msgr2=0x7fe158159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:39.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.527+0000 7fe15cba0640 1 --2- 172.21.15.12:0/3779601778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1581538f0 0x7fe158159fb0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe158066a40 tx=0x7fe14801ecf0 comp rx=0 tx=0).stop 2023-10-30T12:54:39.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.527+0000 7fe15cba0640 1 -- 172.21.15.12:0/3779601778 shutdown_connections 2023-10-30T12:54:39.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.527+0000 7fe15cba0640 1 --2- 172.21.15.12:0/3779601778 >> v2:172.21.15.154:3300/0 conn(0x7fe1580cefa0 0x7fe1580cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:39.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.527+0000 7fe15cba0640 1 --2- 172.21.15.12:0/3779601778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1581538f0 0x7fe158159fb0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:39.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.527+0000 7fe15cba0640 1 -- 172.21.15.12:0/3779601778 >> 172.21.15.12:0/3779601778 conn(0x7fe15806a640 msgr2=0x7fe158065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:39.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.527+0000 7fe15cba0640 1 -- 172.21.15.12:0/3779601778 shutdown_connections 2023-10-30T12:54:39.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.527+0000 7fe15cba0640 1 -- 172.21.15.12:0/3779601778 wait complete. 2023-10-30T12:54:39.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe15cba0640 1 Processor -- start 2023-10-30T12:54:39.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe15cba0640 1 -- start start 2023-10-30T12:54:39.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe15cba0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1580cefa0 0x7fe15815bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:39.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe15cba0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1581538f0 0x7fe158146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:39.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe15cba0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe15815cd50 0x7fe158141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:39.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe15cba0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1581488c0 con 0x7fe1581538f0 2023-10-30T12:54:39.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe15cba0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe158143860 con 0x7fe15815cd50 2023-10-30T12:54:39.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe15cba0640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe158143430 con 0x7fe1580cefa0 2023-10-30T12:54:39.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe156ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1581538f0 0x7fe158146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:39.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe156ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1581538f0 0x7fe158146800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51628/0 (socket says 172.21.15.12:51628) 2023-10-30T12:54:39.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe156ffd640 1 -- 172.21.15.12:0/2555007053 learned_addr learned my addr 172.21.15.12:0/2555007053 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:39.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe157fff640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe15815cd50 0x7fe158141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:39.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe1577fe640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1580cefa0 0x7fe15815bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:39.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe156ffd640 1 -- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1580cefa0 msgr2=0x7fe15815bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:39.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe156ffd640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1580cefa0 0x7fe15815bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:39.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe156ffd640 1 -- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe15815cd50 msgr2=0x7fe158141a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:39.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe156ffd640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe15815cd50 0x7fe158141a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:39.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe156ffd640 1 -- 172.21.15.12:0/2555007053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1480094d0 con 0x7fe1581538f0 2023-10-30T12:54:39.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe1577fe640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1580cefa0 0x7fe15815bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:39.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe157fff640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe15815cd50 0x7fe158141a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:39.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe156ffd640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1581538f0 0x7fe158146800 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00c3c0 tx=0x7fe14c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:39.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe13ffff640 1 -- 172.21.15.12:0/2555007053 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0052d0 con 0x7fe1581538f0 2023-10-30T12:54:39.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe13ffff640 1 -- 172.21.15.12:0/2555007053 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14c005430 con 0x7fe1581538f0 2023-10-30T12:54:39.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe13ffff640 1 -- 172.21.15.12:0/2555007053 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c003840 con 0x7fe1581538f0 2023-10-30T12:54:39.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe15cba0640 1 -- 172.21.15.12:0/2555007053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1581479d0 con 0x7fe1581538f0 2023-10-30T12:54:39.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe15cba0640 1 -- 172.21.15.12:0/2555007053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe158147f90 con 0x7fe1581538f0 2023-10-30T12:54:39.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.531+0000 7fe15cba0640 1 -- 172.21.15.12:0/2555007053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15804e170 con 0x7fe1581538f0 2023-10-30T12:54:39.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.535+0000 7fe13ffff640 1 -- 172.21.15.12:0/2555007053 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe14c0039a0 con 0x7fe1581538f0 2023-10-30T12:54:39.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.535+0000 7fe13ffff640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe1440664a0 0x7fe144068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:39.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.535+0000 7fe13ffff640 1 -- 172.21.15.12:0/2555007053 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fe14c0838c0 con 0x7fe1581538f0 2023-10-30T12:54:39.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.535+0000 7fe1577fe640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe1440664a0 0x7fe144068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:39.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.535+0000 7fe1577fe640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe1440664a0 0x7fe144068960 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe1480b2000 tx=0x7fe148008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:39.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.539+0000 7fe13ffff640 1 -- 172.21.15.12:0/2555007053 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe14c050840 con 0x7fe1581538f0 2023-10-30T12:54:39.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.743+0000 7fe15cba0640 1 -- 172.21.15.12:0/2555007053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe158148a00 con 0x7fe1581538f0 2023-10-30T12:54:39.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.747+0000 7fe13ffff640 1 -- 172.21.15.12:0/2555007053 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5747 (secure 0 0 0) 0x7fe14c017090 con 0x7fe1581538f0 2023-10-30T12:54:39.750 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:39.751 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":101,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:38.560157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:39.751 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 101 2023-10-30T12:54:39.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.747+0000 7fe15cba0640 1 -- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe1440664a0 msgr2=0x7fe144068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:39.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.747+0000 7fe15cba0640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe1440664a0 0x7fe144068960 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe1480b2000 tx=0x7fe148008040 comp rx=0 tx=0).stop 2023-10-30T12:54:39.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.747+0000 7fe15cba0640 1 -- 172.21.15.12:0/2555007053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1581538f0 msgr2=0x7fe158146800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:39.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.747+0000 7fe15cba0640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1581538f0 0x7fe158146800 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00c3c0 tx=0x7fe14c00c880 comp rx=0 tx=0).stop 2023-10-30T12:54:39.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.747+0000 7fe15cba0640 1 -- 172.21.15.12:0/2555007053 shutdown_connections 2023-10-30T12:54:39.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.747+0000 7fe15cba0640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe15815cd50 0x7fe158141a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:39.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.747+0000 7fe15cba0640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1581538f0 0x7fe158146800 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:39.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.747+0000 7fe15cba0640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe1440664a0 0x7fe144068960 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:39.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.747+0000 7fe15cba0640 1 --2- 172.21.15.12:0/2555007053 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1580cefa0 0x7fe15815bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:39.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.747+0000 7fe15cba0640 1 -- 172.21.15.12:0/2555007053 >> 172.21.15.12:0/2555007053 conn(0x7fe15806a640 msgr2=0x7fe158065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:39.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.747+0000 7fe15cba0640 1 -- 172.21.15.12:0/2555007053 shutdown_connections 2023-10-30T12:54:39.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:39.747+0000 7fe15cba0640 1 -- 172.21.15.12:0/2555007053 wait complete. 2023-10-30T12:54:39.772 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 101, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:35:05.553867+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 101, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5877": { "addr": "172.21.15.12:6835/2306258157", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6832", "nonce": 2306258157, "type": "v2" }, { "addr": "172.21.15.12:6835", "nonce": 2306258157, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 5877, "incarnation": 81, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_6021": { "addr": "172.21.15.12:6836/3877967042", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6833", "nonce": 3877967042, "type": "v2" }, { "addr": "172.21.15.12:6836", "nonce": 3877967042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 6021, "incarnation": 89, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 42 }, "gid_6192": { "addr": "172.21.15.12:6837/2954662612", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2954662612, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 2954662612, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6192, "incarnation": 99, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:reconnect", "state_seq": 13 } }, "last_failure": 0, "last_failure_osd_epoch": 55, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:54:38.560157+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6021, "mds_1": 6192, "mds_2": 5877 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/665371837", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 665371837, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 665371837, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4253, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6838/1718052620", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1718052620, "type": "v2" }, { "addr": "172.21.15.154:6838", "nonce": 1718052620, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4256, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:54:41.773 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:41.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.923+0000 7f6b05aea640 1 Processor -- start 2023-10-30T12:54:41.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.923+0000 7f6b05aea640 1 -- start start 2023-10-30T12:54:41.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.923+0000 7f6b05aea640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b000c6b80 0x7f6b000bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:41.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.923+0000 7f6b05aea640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6b000c6f60 0x7f6b000c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:41.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.923+0000 7f6b05aea640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b000c0a50 con 0x7f6b000c8430 2023-10-30T12:54:41.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.923+0000 7f6b05aea640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0006f6e0 con 0x7f6b000c6f60 2023-10-30T12:54:41.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.923+0000 7f6b05aea640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0006f9c0 con 0x7f6b000c6b80 2023-10-30T12:54:41.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.923+0000 7f6afeffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6b000c6f60 0x7f6b000c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:41.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.923+0000 7f6afeffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6b000c6f60 0x7f6b000c0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37934/0 (socket says 172.21.15.12:37934) 2023-10-30T12:54:41.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.923+0000 7f6afeffd640 1 -- 172.21.15.12:0/3099572666 learned_addr learned my addr 172.21.15.12:0/3099572666 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:41.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.927+0000 7f6aff7fe640 1 --2- 172.21.15.12:0/3099572666 >> v2:172.21.15.154:3300/0 conn(0x7f6b000c6b80 0x7f6b000bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:41.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.927+0000 7f6afeffd640 1 -- 172.21.15.12:0/3099572666 >> v2:172.21.15.154:3300/0 conn(0x7f6b000c6b80 msgr2=0x7f6b000bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:41.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.927+0000 7f6afeffd640 1 --2- 172.21.15.12:0/3099572666 >> v2:172.21.15.154:3300/0 conn(0x7f6b000c6b80 0x7f6b000bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:41.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.927+0000 7f6afeffd640 1 -- 172.21.15.12:0/3099572666 >> v1:172.21.15.154:6789/0 conn(0x7f6b000c8430 legacy=0x7f6b0013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:41.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.927+0000 7f6afeffd640 1 -- 172.21.15.12:0/3099572666 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0006fca0 con 0x7f6b000c6f60 2023-10-30T12:54:41.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.927+0000 7f6afeffd640 1 --2- 172.21.15.12:0/3099572666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b000c6f60 0x7f6b000c0400 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6b0006f1b0 tx=0x7f6af401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa019263f2979c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:41.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.927+0000 7f6afe7fc640 1 -- 172.21.15.12:0/3099572666 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af40ab610 con 0x7f6b000c6f60 2023-10-30T12:54:41.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.927+0000 7f6afe7fc640 1 -- 172.21.15.12:0/3099572666 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af40ab770 con 0x7f6b000c6f60 2023-10-30T12:54:41.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.927+0000 7f6afe7fc640 1 -- 172.21.15.12:0/3099572666 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af40abb00 con 0x7f6b000c6f60 2023-10-30T12:54:41.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.927+0000 7f6b05aea640 1 -- 172.21.15.12:0/3099572666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b000c6f60 msgr2=0x7f6b000c0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:41.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.927+0000 7f6b05aea640 1 --2- 172.21.15.12:0/3099572666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b000c6f60 0x7f6b000c0400 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6b0006f1b0 tx=0x7f6af401c500 comp rx=0 tx=0).stop 2023-10-30T12:54:41.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6b05aea640 1 -- 172.21.15.12:0/3099572666 shutdown_connections 2023-10-30T12:54:41.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6b05aea640 1 --2- 172.21.15.12:0/3099572666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b000c6f60 0x7f6b000c0400 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:41.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6b05aea640 1 --2- 172.21.15.12:0/3099572666 >> v2:172.21.15.154:3300/0 conn(0x7f6b000c6b80 0x7f6b000bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:41.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6b05aea640 1 -- 172.21.15.12:0/3099572666 >> 172.21.15.12:0/3099572666 conn(0x7f6b000c7910 msgr2=0x7f6b000c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:41.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6b05aea640 1 -- 172.21.15.12:0/3099572666 shutdown_connections 2023-10-30T12:54:41.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6b05aea640 1 -- 172.21.15.12:0/3099572666 wait complete. 2023-10-30T12:54:41.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6b05aea640 1 Processor -- start 2023-10-30T12:54:41.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6b05aea640 1 -- start start 2023-10-30T12:54:41.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6b05aea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b000c6b80 0x7f6b001389b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:41.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6b05aea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b000c6f60 0x7f6b001f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:41.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6b05aea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b000c8430 0x7f6b001543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:41.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6b05aea640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b00155aa0 con 0x7f6b000c8430 2023-10-30T12:54:41.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6b05aea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b00160610 con 0x7f6b000c6f60 2023-10-30T12:54:41.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6b05aea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b00138ef0 con 0x7f6b000c6b80 2023-10-30T12:54:41.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6afffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b000c8430 0x7f6b001543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:41.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6afffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b000c8430 0x7f6b001543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37950/0 (socket says 172.21.15.12:37950) 2023-10-30T12:54:41.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6afffff640 1 -- 172.21.15.12:0/4112443085 learned_addr learned my addr 172.21.15.12:0/4112443085 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:41.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6afeffd640 1 --2- 172.21.15.12:0/4112443085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b000c6f60 0x7f6b001f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:41.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6aff7fe640 1 --2- 172.21.15.12:0/4112443085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b000c6b80 0x7f6b001389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:41.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6afffff640 1 -- 172.21.15.12:0/4112443085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b000c6b80 msgr2=0x7f6b001389b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:41.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6afffff640 1 --2- 172.21.15.12:0/4112443085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b000c6b80 0x7f6b001389b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:41.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6afffff640 1 -- 172.21.15.12:0/4112443085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b000c6f60 msgr2=0x7f6b001f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:41.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6afffff640 1 --2- 172.21.15.12:0/4112443085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b000c6f60 0x7f6b001f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:41.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.931+0000 7f6afffff640 1 -- 172.21.15.12:0/4112443085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af40094d0 con 0x7f6b000c8430 2023-10-30T12:54:41.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.935+0000 7f6afffff640 1 --2- 172.21.15.12:0/4112443085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b000c8430 0x7f6b001543e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6af800c760 tx=0x7f6af800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:41.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.935+0000 7f6ae7fff640 1 -- 172.21.15.12:0/4112443085 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8003b80 con 0x7f6b000c8430 2023-10-30T12:54:41.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.935+0000 7f6ae7fff640 1 -- 172.21.15.12:0/4112443085 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af8003ce0 con 0x7f6b000c8430 2023-10-30T12:54:41.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.935+0000 7f6ae7fff640 1 -- 172.21.15.12:0/4112443085 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af800aa30 con 0x7f6b000c8430 2023-10-30T12:54:41.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.935+0000 7f6b05aea640 1 -- 172.21.15.12:0/4112443085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0015f450 con 0x7f6b000c8430 2023-10-30T12:54:41.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.935+0000 7f6b05aea640 1 -- 172.21.15.12:0/4112443085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b001f1bc0 con 0x7f6b000c8430 2023-10-30T12:54:41.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.935+0000 7f6b05aea640 1 -- 172.21.15.12:0/4112443085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0004e170 con 0x7f6b000c8430 2023-10-30T12:54:41.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.935+0000 7f6ae7fff640 1 -- 172.21.15.12:0/4112443085 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6af800ab90 con 0x7f6b000c8430 2023-10-30T12:54:41.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.935+0000 7f6ae7fff640 1 --2- 172.21.15.12:0/4112443085 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6aec0664d0 0x7f6aec068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:41.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.935+0000 7f6ae7fff640 1 -- 172.21.15.12:0/4112443085 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f6af8083ea0 con 0x7f6b000c8430 2023-10-30T12:54:41.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.939+0000 7f6aff7fe640 1 --2- 172.21.15.12:0/4112443085 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6aec0664d0 0x7f6aec068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:41.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.939+0000 7f6aff7fe640 1 --2- 172.21.15.12:0/4112443085 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6aec0664d0 0x7f6aec068990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6af0007a60 tx=0x7f6af0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:41.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:41.943+0000 7f6ae7fff640 1 -- 172.21.15.12:0/4112443085 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6af8050ff0 con 0x7f6b000c8430 2023-10-30T12:54:42.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:42.131+0000 7f6b05aea640 1 -- 172.21.15.12:0/4112443085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b000723c0 con 0x7f6b000c8430 2023-10-30T12:54:42.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:42.131+0000 7f6ae7fff640 1 -- 172.21.15.12:0/4112443085 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5746 (secure 0 0 0) 0x7f6af8054610 con 0x7f6b000c8430 2023-10-30T12:54:42.137 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:42.137 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":102,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:39.616865+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:42.138 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 102 2023-10-30T12:54:42.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:42.135+0000 7f6b05aea640 1 -- 172.21.15.12:0/4112443085 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6aec0664d0 msgr2=0x7f6aec068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:42.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:42.135+0000 7f6b05aea640 1 --2- 172.21.15.12:0/4112443085 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6aec0664d0 0x7f6aec068990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6af0007a60 tx=0x7f6af0008040 comp rx=0 tx=0).stop 2023-10-30T12:54:42.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:42.135+0000 7f6b05aea640 1 -- 172.21.15.12:0/4112443085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b000c8430 msgr2=0x7f6b001543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:42.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:42.135+0000 7f6b05aea640 1 --2- 172.21.15.12:0/4112443085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b000c8430 0x7f6b001543e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6af800c760 tx=0x7f6af800cc20 comp rx=0 tx=0).stop 2023-10-30T12:54:42.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:42.135+0000 7f6b05aea640 1 -- 172.21.15.12:0/4112443085 shutdown_connections 2023-10-30T12:54:42.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:42.135+0000 7f6b05aea640 1 --2- 172.21.15.12:0/4112443085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b000c8430 0x7f6b001543e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:42.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:42.135+0000 7f6b05aea640 1 --2- 172.21.15.12:0/4112443085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b000c6f60 0x7f6b001f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:42.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:42.135+0000 7f6b05aea640 1 --2- 172.21.15.12:0/4112443085 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6aec0664d0 0x7f6aec068990 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:42.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:42.135+0000 7f6b05aea640 1 --2- 172.21.15.12:0/4112443085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b000c6b80 0x7f6b001389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:42.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:42.135+0000 7f6b05aea640 1 -- 172.21.15.12:0/4112443085 >> 172.21.15.12:0/4112443085 conn(0x7f6b000c7910 msgr2=0x7f6b000c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:42.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:42.135+0000 7f6b05aea640 1 -- 172.21.15.12:0/4112443085 shutdown_connections 2023-10-30T12:54:42.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:42.135+0000 7f6b05aea640 1 -- 172.21.15.12:0/4112443085 wait complete. 2023-10-30T12:54:42.158 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:54:44.159 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:44.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33fd817640 1 Processor -- start 2023-10-30T12:54:44.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33fd817640 1 -- start start 2023-10-30T12:54:44.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33fd817640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33f80c6fe0 0x7f33f8071260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:44.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33fd817640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f33f80c5f60 0x7f33f80cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:44.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33fd817640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f80700b0 con 0x7f33f8153bf0 2023-10-30T12:54:44.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33fd817640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f80c6810 con 0x7f33f80c5f60 2023-10-30T12:54:44.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33fd817640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f8159900 con 0x7f33f80c6fe0 2023-10-30T12:54:44.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f6ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f33f8153bf0 0x7f33f80c74a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43466/0 (socket says 172.21.15.12:43466) 2023-10-30T12:54:44.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f6ffd640 1 -- 172.21.15.12:0/44580113 learned_addr learned my addr 172.21.15.12:0/44580113 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:54:44.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33ee7fc640 1 --2- 172.21.15.12:0/44580113 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33f80c6fe0 0x7f33f8071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:44.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f77fe640 1 --2- 172.21.15.12:0/44580113 >> v2:172.21.15.154:3300/0 conn(0x7f33f80c5f60 0x7f33f80cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:44.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 -- 172.21.15.12:0/44580113 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640620490 0 0) 0x7f33f80700b0 con 0x7f33f8153bf0 2023-10-30T12:54:44.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 -- 172.21.15.12:0/44580113 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33e4003530 con 0x7f33f8153bf0 2023-10-30T12:54:44.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 -- 172.21.15.12:0/44580113 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3809935428 0 0) 0x7f33e4003530 con 0x7f33f8153bf0 2023-10-30T12:54:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 -- 172.21.15.12:0/44580113 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33f80c6fe0 msgr2=0x7f33f8071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 --2- 172.21.15.12:0/44580113 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33f80c6fe0 0x7f33f8071260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 -- 172.21.15.12:0/44580113 >> v2:172.21.15.154:3300/0 conn(0x7f33f80c5f60 msgr2=0x7f33f80cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 --2- 172.21.15.12:0/44580113 >> v2:172.21.15.154:3300/0 conn(0x7f33f80c5f60 0x7f33f80cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 -- 172.21.15.12:0/44580113 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f80c6530 con 0x7f33f8153bf0 2023-10-30T12:54:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 -- 172.21.15.12:0/44580113 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f33e8001680 con 0x7f33f8153bf0 2023-10-30T12:54:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 -- 172.21.15.12:0/44580113 >> v1:172.21.15.12:6789/0 conn(0x7f33f8153bf0 legacy=0x7f33f80c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 --2- 172.21.15.12:0/44580113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33e4003980 0x7f33e4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 --2- 172.21.15.12:0/44580113 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33f80c6fe0 0x7f33e40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 --2- 172.21.15.12:0/44580113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33e4004d10 0x7f33e4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:44.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 -- 172.21.15.12:0/44580113 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80c6810 con 0x7f33e4003980 2023-10-30T12:54:44.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 -- 172.21.15.12:0/44580113 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f8159900 con 0x7f33e4004d10 2023-10-30T12:54:44.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f67fc640 1 -- 172.21.15.12:0/44580113 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e400b990 con 0x7f33f80c6fe0 2023-10-30T12:54:44.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f6ffd640 1 --2- 172.21.15.12:0/44580113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33e4003980 0x7f33e4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:44.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33f77fe640 1 --2- 172.21.15.12:0/44580113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33e4004d10 0x7f33e4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:44.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.311+0000 7f33ee7fc640 1 --2- 172.21.15.12:0/44580113 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33f80c6fe0 0x7f33e40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:44.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33f6ffd640 1 -- 172.21.15.12:0/44580113 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33f80c6fe0 msgr2=0x7f33e40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:44.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33f6ffd640 1 --2- 172.21.15.12:0/44580113 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33f80c6fe0 0x7f33e40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:44.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33f6ffd640 1 -- 172.21.15.12:0/44580113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33e4004d10 msgr2=0x7f33e4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:44.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33f6ffd640 1 --2- 172.21.15.12:0/44580113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33e4004d10 0x7f33e4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:44.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33f6ffd640 1 -- 172.21.15.12:0/44580113 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e4003530 con 0x7f33e4003980 2023-10-30T12:54:44.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33f77fe640 1 --2- 172.21.15.12:0/44580113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33e4004d10 0x7f33e4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:44.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33f6ffd640 1 --2- 172.21.15.12:0/44580113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33e4003980 0x7f33e4003da0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f33e800c240 tx=0x7f33e801f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dc36e9afc514c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:44.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33f67fc640 1 -- 172.21.15.12:0/44580113 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8005ae0 con 0x7f33e4003980 2023-10-30T12:54:44.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33f67fc640 1 -- 172.21.15.12:0/44580113 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e80ab070 con 0x7f33e4003980 2023-10-30T12:54:44.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33f67fc640 1 -- 172.21.15.12:0/44580113 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8005ea0 con 0x7f33e4003980 2023-10-30T12:54:44.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33fd817640 1 -- 172.21.15.12:0/44580113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33e4003980 msgr2=0x7f33e4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:44.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33fd817640 1 --2- 172.21.15.12:0/44580113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33e4003980 0x7f33e4003da0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f33e800c240 tx=0x7f33e801f3d0 comp rx=0 tx=0).stop 2023-10-30T12:54:44.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33f77fe640 1 -- 172.21.15.12:0/44580113 reap_dead start 2023-10-30T12:54:44.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33fd817640 1 -- 172.21.15.12:0/44580113 shutdown_connections 2023-10-30T12:54:44.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33fd817640 1 -- 172.21.15.12:0/44580113 >> 172.21.15.12:0/44580113 conn(0x7f33f806c6e0 msgr2=0x7f33f8069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:44.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33fd817640 1 -- 172.21.15.12:0/44580113 shutdown_connections 2023-10-30T12:54:44.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.315+0000 7f33fd817640 1 -- 172.21.15.12:0/44580113 wait complete. 2023-10-30T12:54:44.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33fd817640 1 Processor -- start 2023-10-30T12:54:44.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33fd817640 1 -- start start 2023-10-30T12:54:44.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33fd817640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33e4003980 0x7f33f81552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:44.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33fd817640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33f8156c50 0x7f33f81529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:44.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33fd817640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33f81f32d0 0x7f33f81f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:44.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33fd817640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f81f3c90 con 0x7f33e4003980 2023-10-30T12:54:44.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33fd817640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f81f64f0 con 0x7f33f81f32d0 2023-10-30T12:54:44.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33fd817640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f81f67d0 con 0x7f33f8156c50 2023-10-30T12:54:44.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33e4003980 0x7f33f81552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:44.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33e4003980 0x7f33f81552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37964/0 (socket says 172.21.15.12:37964) 2023-10-30T12:54:44.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33f6ffd640 1 -- 172.21.15.12:0/2342481112 learned_addr learned my addr 172.21.15.12:0/2342481112 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:44.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33ee7fc640 1 --2- 172.21.15.12:0/2342481112 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33f8156c50 0x7f33f81529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:44.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33f77fe640 1 --2- 172.21.15.12:0/2342481112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33f81f32d0 0x7f33f81f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:44.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33f6ffd640 1 -- 172.21.15.12:0/2342481112 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33f8156c50 msgr2=0x7f33f81529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:44.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33f6ffd640 1 --2- 172.21.15.12:0/2342481112 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33f8156c50 0x7f33f81529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:44.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33f6ffd640 1 -- 172.21.15.12:0/2342481112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33f81f32d0 msgr2=0x7f33f81f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:44.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33f6ffd640 1 --2- 172.21.15.12:0/2342481112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33f81f32d0 0x7f33f81f3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:44.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33f6ffd640 1 -- 172.21.15.12:0/2342481112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f81f6ab0 con 0x7f33e4003980 2023-10-30T12:54:44.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33f6ffd640 1 --2- 172.21.15.12:0/2342481112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33e4003980 0x7f33f81552a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f33e800c240 tx=0x7f33e4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:44.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33f4ff9640 1 -- 172.21.15.12:0/2342481112 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e80b7070 con 0x7f33e4003980 2023-10-30T12:54:44.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33f4ff9640 1 -- 172.21.15.12:0/2342481112 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e80048b0 con 0x7f33e4003980 2023-10-30T12:54:44.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33f4ff9640 1 -- 172.21.15.12:0/2342481112 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8004c40 con 0x7f33e4003980 2023-10-30T12:54:44.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.319+0000 7f33fd817640 1 -- 172.21.15.12:0/2342481112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f81f6d90 con 0x7f33e4003980 2023-10-30T12:54:44.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.323+0000 7f33fd817640 1 -- 172.21.15.12:0/2342481112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f81f7350 con 0x7f33e4003980 2023-10-30T12:54:44.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.323+0000 7f33fd817640 1 -- 172.21.15.12:0/2342481112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f804e170 con 0x7f33e4003980 2023-10-30T12:54:44.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.323+0000 7f33f4ff9640 1 -- 172.21.15.12:0/2342481112 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f33e80ad020 con 0x7f33e4003980 2023-10-30T12:54:44.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.323+0000 7f33f4ff9640 1 --2- 172.21.15.12:0/2342481112 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f33d80664a0 0x7f33d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:44.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.323+0000 7f33f4ff9640 1 -- 172.21.15.12:0/2342481112 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f33e8120b90 con 0x7f33e4003980 2023-10-30T12:54:44.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.323+0000 7f33ee7fc640 1 --2- 172.21.15.12:0/2342481112 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f33d80664a0 0x7f33d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:44.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.323+0000 7f33ee7fc640 1 --2- 172.21.15.12:0/2342481112 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f33d80664a0 0x7f33d8068960 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f33f80cc560 tx=0x7f33e000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:44.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.327+0000 7f33f4ff9640 1 -- 172.21.15.12:0/2342481112 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f33e80f15a0 con 0x7f33e4003980 2023-10-30T12:54:44.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.531+0000 7f33fd817640 1 -- 172.21.15.12:0/2342481112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33f806fe90 con 0x7f33e4003980 2023-10-30T12:54:44.537 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:44.537 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:43.582549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:44.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.531+0000 7f33f4ff9640 1 -- 172.21.15.12:0/2342481112 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+5743 (secure 0 0 0) 0x7f33e80f1130 con 0x7f33e4003980 2023-10-30T12:54:44.538 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 103 2023-10-30T12:54:44.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.535+0000 7f33fd817640 1 -- 172.21.15.12:0/2342481112 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f33d80664a0 msgr2=0x7f33d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:44.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.535+0000 7f33fd817640 1 --2- 172.21.15.12:0/2342481112 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f33d80664a0 0x7f33d8068960 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f33f80cc560 tx=0x7f33e000b040 comp rx=0 tx=0).stop 2023-10-30T12:54:44.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.535+0000 7f33fd817640 1 -- 172.21.15.12:0/2342481112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33e4003980 msgr2=0x7f33f81552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:44.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.535+0000 7f33fd817640 1 --2- 172.21.15.12:0/2342481112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33e4003980 0x7f33f81552a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f33e800c240 tx=0x7f33e4005800 comp rx=0 tx=0).stop 2023-10-30T12:54:44.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.535+0000 7f33fd817640 1 -- 172.21.15.12:0/2342481112 shutdown_connections 2023-10-30T12:54:44.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.535+0000 7f33fd817640 1 --2- 172.21.15.12:0/2342481112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33f81f32d0 0x7f33f81f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:44.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.535+0000 7f33fd817640 1 --2- 172.21.15.12:0/2342481112 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f33d80664a0 0x7f33d8068960 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:44.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.535+0000 7f33fd817640 1 --2- 172.21.15.12:0/2342481112 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f33f8156c50 0x7f33f81529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:44.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.535+0000 7f33fd817640 1 --2- 172.21.15.12:0/2342481112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33e4003980 0x7f33f81552a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:44.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.535+0000 7f33fd817640 1 -- 172.21.15.12:0/2342481112 >> 172.21.15.12:0/2342481112 conn(0x7f33f806c6e0 msgr2=0x7f33f8157720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:44.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.535+0000 7f33fd817640 1 -- 172.21.15.12:0/2342481112 shutdown_connections 2023-10-30T12:54:44.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:44.535+0000 7f33fd817640 1 -- 172.21.15.12:0/2342481112 wait complete. 2023-10-30T12:54:44.572 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:54:46.574 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:46.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.723+0000 7f3b1c91d640 1 Processor -- start 2023-10-30T12:54:46.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b1c91d640 1 -- start start 2023-10-30T12:54:46.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b1c91d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b18153d70 0x7f3b1806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:46.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b1c91d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1806afa0 con 0x7f3b1806e470 2023-10-30T12:54:46.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b1c91d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b18154150 con 0x7f3b180c8490 2023-10-30T12:54:46.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b1c91d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1806d780 con 0x7f3b18153d70 2023-10-30T12:54:46.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b0ffff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3b1806e470 0x7f3b180cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43470/0 (socket says 172.21.15.12:43470) 2023-10-30T12:54:46.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b0ffff640 1 -- 172.21.15.12:0/1899900693 learned_addr learned my addr 172.21.15.12:0/1899900693 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:54:46.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b1659c640 1 --2- 172.21.15.12:0/1899900693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b18153d70 0x7f3b1806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:46.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b15d9b640 1 -- 172.21.15.12:0/1899900693 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884343814 0 0) 0x7f3b1806afa0 con 0x7f3b1806e470 2023-10-30T12:54:46.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b15d9b640 1 -- 172.21.15.12:0/1899900693 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b04003530 con 0x7f3b1806e470 2023-10-30T12:54:46.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b15d9b640 1 -- 172.21.15.12:0/1899900693 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2818010580 0 0) 0x7f3b04003530 con 0x7f3b1806e470 2023-10-30T12:54:46.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b15d9b640 1 -- 172.21.15.12:0/1899900693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b18153d70 msgr2=0x7f3b1806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:46.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b15d9b640 1 --2- 172.21.15.12:0/1899900693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b18153d70 0x7f3b1806ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:46.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b15d9b640 1 -- 172.21.15.12:0/1899900693 >> v1:172.21.15.154:6789/0 conn(0x7f3b180c8490 legacy=0x7f3b180c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:46.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b15d9b640 1 -- 172.21.15.12:0/1899900693 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1806e030 con 0x7f3b1806e470 2023-10-30T12:54:46.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b15d9b640 1 -- 172.21.15.12:0/1899900693 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f3b00001680 con 0x7f3b1806e470 2023-10-30T12:54:46.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b15d9b640 1 -- 172.21.15.12:0/1899900693 >> v1:172.21.15.12:6789/0 conn(0x7f3b1806e470 legacy=0x7f3b180cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:46.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b15d9b640 1 --2- 172.21.15.12:0/1899900693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b040039b0 0x7f3b04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:46.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b15d9b640 1 --2- 172.21.15.12:0/1899900693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b040047d0 0x7f3b04004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:46.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b15d9b640 1 --2- 172.21.15.12:0/1899900693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b18153d70 0x7f3b04005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:46.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b15d9b640 1 -- 172.21.15.12:0/1899900693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1806d780 con 0x7f3b040039b0 2023-10-30T12:54:46.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b15d9b640 1 -- 172.21.15.12:0/1899900693 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0400ba40 con 0x7f3b040047d0 2023-10-30T12:54:46.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b15d9b640 1 -- 172.21.15.12:0/1899900693 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0400bc00 con 0x7f3b18153d70 2023-10-30T12:54:46.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b1659c640 1 --2- 172.21.15.12:0/1899900693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b040039b0 0x7f3b04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:46.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b0ffff640 1 --2- 172.21.15.12:0/1899900693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b040047d0 0x7f3b04004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:46.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b16d9d640 1 --2- 172.21.15.12:0/1899900693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b18153d70 0x7f3b04005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:46.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b1659c640 1 -- 172.21.15.12:0/1899900693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b18153d70 msgr2=0x7f3b04005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:46.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b1659c640 1 --2- 172.21.15.12:0/1899900693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b18153d70 0x7f3b04005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:46.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b1659c640 1 -- 172.21.15.12:0/1899900693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b040047d0 msgr2=0x7f3b04004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:46.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b1659c640 1 --2- 172.21.15.12:0/1899900693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b040047d0 0x7f3b04004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:46.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b1659c640 1 -- 172.21.15.12:0/1899900693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b04003530 con 0x7f3b040039b0 2023-10-30T12:54:46.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.727+0000 7f3b16d9d640 1 --2- 172.21.15.12:0/1899900693 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b18153d70 0x7f3b04005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:46.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.731+0000 7f3b1659c640 1 --2- 172.21.15.12:0/1899900693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b040039b0 0x7f3b04003dd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009ab0 tx=0x7f3b0801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463c751d3d690006 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:46.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.731+0000 7f3b15d9b640 1 -- 172.21.15.12:0/1899900693 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080a55b0 con 0x7f3b040039b0 2023-10-30T12:54:46.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.731+0000 7f3b15d9b640 1 -- 172.21.15.12:0/1899900693 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b080a5710 con 0x7f3b040039b0 2023-10-30T12:54:46.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.731+0000 7f3b15d9b640 1 -- 172.21.15.12:0/1899900693 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080a5ac0 con 0x7f3b040039b0 2023-10-30T12:54:46.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.731+0000 7f3b1c91d640 1 -- 172.21.15.12:0/1899900693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b040039b0 msgr2=0x7f3b04003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:46.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.731+0000 7f3b1c91d640 1 --2- 172.21.15.12:0/1899900693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b040039b0 0x7f3b04003dd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009ab0 tx=0x7f3b0801c9e0 comp rx=0 tx=0).stop 2023-10-30T12:54:46.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.731+0000 7f3b16d9d640 1 -- 172.21.15.12:0/1899900693 reap_dead start 2023-10-30T12:54:46.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.731+0000 7f3b1c91d640 1 -- 172.21.15.12:0/1899900693 shutdown_connections 2023-10-30T12:54:46.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.731+0000 7f3b1c91d640 1 -- 172.21.15.12:0/1899900693 >> 172.21.15.12:0/1899900693 conn(0x7f3b18071ea0 msgr2=0x7f3b18065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.731+0000 7f3b1c91d640 1 -- 172.21.15.12:0/1899900693 shutdown_connections 2023-10-30T12:54:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.731+0000 7f3b1c91d640 1 -- 172.21.15.12:0/1899900693 wait complete. 2023-10-30T12:54:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b1c91d640 1 Processor -- start 2023-10-30T12:54:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b1c91d640 1 -- start start 2023-10-30T12:54:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b1c91d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b040039b0 0x7f3b1814d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b1c91d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b18161320 0x7f3b18156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b1c91d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b181f6360 0x7f3b181f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b1c91d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b181f27a0 con 0x7f3b181f6360 2023-10-30T12:54:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b1c91d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b181636c0 con 0x7f3b18161320 2023-10-30T12:54:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b1c91d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b181f5ed0 con 0x7f3b040039b0 2023-10-30T12:54:46.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b16d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b181f6360 0x7f3b181f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:46.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b16d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b181f6360 0x7f3b181f67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37988/0 (socket says 172.21.15.12:37988) 2023-10-30T12:54:46.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b16d9d640 1 -- 172.21.15.12:0/2015954747 learned_addr learned my addr 172.21.15.12:0/2015954747 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:46.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b0ffff640 1 --2- 172.21.15.12:0/2015954747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b18161320 0x7f3b18156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:46.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b1659c640 1 --2- 172.21.15.12:0/2015954747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b040039b0 0x7f3b1814d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:46.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b16d9d640 1 -- 172.21.15.12:0/2015954747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b040039b0 msgr2=0x7f3b1814d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:46.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b16d9d640 1 --2- 172.21.15.12:0/2015954747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b040039b0 0x7f3b1814d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:46.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b16d9d640 1 -- 172.21.15.12:0/2015954747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b18161320 msgr2=0x7f3b18156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:46.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b16d9d640 1 --2- 172.21.15.12:0/2015954747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b18161320 0x7f3b18156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:46.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b16d9d640 1 -- 172.21.15.12:0/2015954747 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b18162b40 con 0x7f3b181f6360 2023-10-30T12:54:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b1659c640 1 --2- 172.21.15.12:0/2015954747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b040039b0 0x7f3b1814d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b16d9d640 1 --2- 172.21.15.12:0/2015954747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b181f6360 0x7f3b181f67e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3b180cdcd0 tx=0x7f3b1000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b0effd640 1 -- 172.21.15.12:0/2015954747 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10017070 con 0x7f3b181f6360 2023-10-30T12:54:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b0effd640 1 -- 172.21.15.12:0/2015954747 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b10006b60 con 0x7f3b181f6360 2023-10-30T12:54:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b0effd640 1 -- 172.21.15.12:0/2015954747 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10006d10 con 0x7f3b181f6360 2023-10-30T12:54:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b1c91d640 1 -- 172.21.15.12:0/2015954747 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b18162e20 con 0x7f3b181f6360 2023-10-30T12:54:46.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b1c91d640 1 -- 172.21.15.12:0/2015954747 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b181633e0 con 0x7f3b181f6360 2023-10-30T12:54:46.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.735+0000 7f3b1c91d640 1 -- 172.21.15.12:0/2015954747 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1804e170 con 0x7f3b181f6360 2023-10-30T12:54:46.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.739+0000 7f3b0effd640 1 -- 172.21.15.12:0/2015954747 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3b10003980 con 0x7f3b181f6360 2023-10-30T12:54:46.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.739+0000 7f3b0effd640 1 --2- 172.21.15.12:0/2015954747 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3af80664a0 0x7f3af8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:46.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.739+0000 7f3b0effd640 1 -- 172.21.15.12:0/2015954747 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f3b100838a0 con 0x7f3b181f6360 2023-10-30T12:54:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.739+0000 7f3b1659c640 1 --2- 172.21.15.12:0/2015954747 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3af80664a0 0x7f3af8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.739+0000 7f3b1659c640 1 --2- 172.21.15.12:0/2015954747 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3af80664a0 0x7f3af8068960 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3b080af000 tx=0x7f3b080046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.743+0000 7f3b0effd640 1 -- 172.21.15.12:0/2015954747 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3b10050940 con 0x7f3b181f6360 2023-10-30T12:54:46.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.935+0000 7f3b1c91d640 1 -- 172.21.15.12:0/2015954747 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b181f6010 con 0x7f3b181f6360 2023-10-30T12:54:46.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.935+0000 7f3b0effd640 1 -- 172.21.15.12:0/2015954747 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+5743 (secure 0 0 0) 0x7f3b10053f60 con 0x7f3b181f6360 2023-10-30T12:54:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:46.941 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:43.582549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:46.942 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 103 2023-10-30T12:54:46.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.939+0000 7f3b1c91d640 1 -- 172.21.15.12:0/2015954747 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3af80664a0 msgr2=0x7f3af8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:46.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.939+0000 7f3b1c91d640 1 --2- 172.21.15.12:0/2015954747 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3af80664a0 0x7f3af8068960 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3b080af000 tx=0x7f3b080046d0 comp rx=0 tx=0).stop 2023-10-30T12:54:46.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.939+0000 7f3b1c91d640 1 -- 172.21.15.12:0/2015954747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b181f6360 msgr2=0x7f3b181f67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:46.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.939+0000 7f3b1c91d640 1 --2- 172.21.15.12:0/2015954747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b181f6360 0x7f3b181f67e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3b180cdcd0 tx=0x7f3b1000eb90 comp rx=0 tx=0).stop 2023-10-30T12:54:46.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.939+0000 7f3b1c91d640 1 -- 172.21.15.12:0/2015954747 shutdown_connections 2023-10-30T12:54:46.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.939+0000 7f3b1c91d640 1 --2- 172.21.15.12:0/2015954747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b181f6360 0x7f3b181f67e0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:46.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.939+0000 7f3b1c91d640 1 --2- 172.21.15.12:0/2015954747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b18161320 0x7f3b18156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:46.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.939+0000 7f3b1c91d640 1 --2- 172.21.15.12:0/2015954747 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3af80664a0 0x7f3af8068960 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:46.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.939+0000 7f3b1c91d640 1 --2- 172.21.15.12:0/2015954747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b040039b0 0x7f3b1814d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:46.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.939+0000 7f3b1c91d640 1 -- 172.21.15.12:0/2015954747 >> 172.21.15.12:0/2015954747 conn(0x7f3b18071ea0 msgr2=0x7f3b1814e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:46.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.939+0000 7f3b1c91d640 1 -- 172.21.15.12:0/2015954747 shutdown_connections 2023-10-30T12:54:46.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:46.939+0000 7f3b1c91d640 1 -- 172.21.15.12:0/2015954747 wait complete. 2023-10-30T12:54:46.963 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:54:48.964 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:49.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0f49a0640 1 Processor -- start 2023-10-30T12:54:49.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0f49a0640 1 -- start start 2023-10-30T12:54:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0f49a0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd0f015cd50 0x7fd0f00c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0f49a0640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd0f00cefa0 0x7fd0f00cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0f49a0640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f015a6c0 con 0x7fd0f01538f0 2023-10-30T12:54:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0f49a0640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f00cf8f0 con 0x7fd0f00cefa0 2023-10-30T12:54:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0f49a0640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f015d300 con 0x7fd0f015cd50 2023-10-30T12:54:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0effff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd0f00cefa0 0x7fd0f00cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:49.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0effff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd0f00cefa0 0x7fd0f00cf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37992/0 (socket says 172.21.15.12:37992) 2023-10-30T12:54:49.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0effff640 1 -- 172.21.15.12:0/3257090946 learned_addr learned my addr 172.21.15.12:0/3257090946 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:49.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0eeffd640 1 --2- 172.21.15.12:0/3257090946 >> v2:172.21.15.154:3300/0 conn(0x7fd0f015cd50 0x7fd0f00c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:49.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0effff640 1 -- 172.21.15.12:0/3257090946 >> v2:172.21.15.154:3300/0 conn(0x7fd0f015cd50 msgr2=0x7fd0f00c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0effff640 1 --2- 172.21.15.12:0/3257090946 >> v2:172.21.15.154:3300/0 conn(0x7fd0f015cd50 0x7fd0f00c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0effff640 1 -- 172.21.15.12:0/3257090946 >> v1:172.21.15.12:6789/0 conn(0x7fd0f01538f0 legacy=0x7fd0f0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0effff640 1 -- 172.21.15.12:0/3257090946 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f015d5e0 con 0x7fd0f00cefa0 2023-10-30T12:54:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.131+0000 7fd0ee7fc640 1 -- 172.21.15.12:0/3257090946 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690152512 0 0) 0x7fd0f015a6c0 con 0x7fd0f01538f0 2023-10-30T12:54:49.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.135+0000 7fd0effff640 1 --2- 172.21.15.12:0/3257090946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0f00cefa0 0x7fd0f00cf3b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0072e90 tx=0x7fd0e801f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c224f3c5876fdf39 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:49.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.135+0000 7fd0ee7fc640 1 -- 172.21.15.12:0/3257090946 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e80af400 con 0x7fd0f00cefa0 2023-10-30T12:54:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.135+0000 7fd0ee7fc640 1 -- 172.21.15.12:0/3257090946 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e80ab050 con 0x7fd0f00cefa0 2023-10-30T12:54:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.135+0000 7fd0ee7fc640 1 -- 172.21.15.12:0/3257090946 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e80af780 con 0x7fd0f00cefa0 2023-10-30T12:54:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.135+0000 7fd0f49a0640 1 -- 172.21.15.12:0/3257090946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0f00cefa0 msgr2=0x7fd0f00cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.135+0000 7fd0f49a0640 1 --2- 172.21.15.12:0/3257090946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0f00cefa0 0x7fd0f00cf3b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0072e90 tx=0x7fd0e801f1c0 comp rx=0 tx=0).stop 2023-10-30T12:54:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.135+0000 7fd0f49a0640 1 -- 172.21.15.12:0/3257090946 shutdown_connections 2023-10-30T12:54:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.135+0000 7fd0f49a0640 1 --2- 172.21.15.12:0/3257090946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0f00cefa0 0x7fd0f00cf3b0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.135+0000 7fd0f49a0640 1 --2- 172.21.15.12:0/3257090946 >> v2:172.21.15.154:3300/0 conn(0x7fd0f015cd50 0x7fd0f00c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.135+0000 7fd0f49a0640 1 -- 172.21.15.12:0/3257090946 >> 172.21.15.12:0/3257090946 conn(0x7fd0f006a640 msgr2=0x7fd0f0065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.135+0000 7fd0f49a0640 1 -- 172.21.15.12:0/3257090946 shutdown_connections 2023-10-30T12:54:49.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.135+0000 7fd0f49a0640 1 -- 172.21.15.12:0/3257090946 wait complete. 2023-10-30T12:54:49.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0f49a0640 1 Processor -- start 2023-10-30T12:54:49.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0f49a0640 1 -- start start 2023-10-30T12:54:49.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0f49a0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0f00cefa0 0x7fd0f01452c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:49.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0f49a0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0f01538f0 0x7fd0f0151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:49.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0f49a0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0f015cd50 0x7fd0f0152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:49.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0f49a0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f015cb40 con 0x7fd0f015cd50 2023-10-30T12:54:49.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0f49a0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f00b3f30 con 0x7fd0f00cefa0 2023-10-30T12:54:49.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0f49a0640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f00b4d60 con 0x7fd0f01538f0 2023-10-30T12:54:49.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0effff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0f015cd50 0x7fd0f0152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:49.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0effff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0f015cd50 0x7fd0f0152a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38004/0 (socket says 172.21.15.12:38004) 2023-10-30T12:54:49.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0effff640 1 -- 172.21.15.12:0/771369755 learned_addr learned my addr 172.21.15.12:0/771369755 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:49.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0ef7fe640 1 --2- 172.21.15.12:0/771369755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0f00cefa0 0x7fd0f01452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:49.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0eeffd640 1 --2- 172.21.15.12:0/771369755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0f01538f0 0x7fd0f0151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:49.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0effff640 1 -- 172.21.15.12:0/771369755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0f01538f0 msgr2=0x7fd0f0151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:49.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0effff640 1 --2- 172.21.15.12:0/771369755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0f01538f0 0x7fd0f0151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:49.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0effff640 1 -- 172.21.15.12:0/771369755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0f00cefa0 msgr2=0x7fd0f01452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:49.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0effff640 1 --2- 172.21.15.12:0/771369755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0f00cefa0 0x7fd0f01452c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:49.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0effff640 1 -- 172.21.15.12:0/771369755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e8009b30 con 0x7fd0f015cd50 2023-10-30T12:54:49.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0ef7fe640 1 --2- 172.21.15.12:0/771369755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0f00cefa0 0x7fd0f01452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:49.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0effff640 1 --2- 172.21.15.12:0/771369755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0f015cd50 0x7fd0f0152a70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80b2040 tx=0x7fd0e801bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:49.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0d7fff640 1 -- 172.21.15.12:0/771369755 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e80afde0 con 0x7fd0f015cd50 2023-10-30T12:54:49.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0d7fff640 1 -- 172.21.15.12:0/771369755 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e8009510 con 0x7fd0f015cd50 2023-10-30T12:54:49.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0d7fff640 1 -- 172.21.15.12:0/771369755 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e80ba660 con 0x7fd0f015cd50 2023-10-30T12:54:49.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0f49a0640 1 -- 172.21.15.12:0/771369755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f0144e30 con 0x7fd0f015cd50 2023-10-30T12:54:49.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.139+0000 7fd0f49a0640 1 -- 172.21.15.12:0/771369755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f015c600 con 0x7fd0f015cd50 2023-10-30T12:54:49.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.143+0000 7fd0f49a0640 1 -- 172.21.15.12:0/771369755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f0144fc0 con 0x7fd0f015cd50 2023-10-30T12:54:49.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.143+0000 7fd0d7fff640 1 -- 172.21.15.12:0/771369755 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd0e80b6040 con 0x7fd0f015cd50 2023-10-30T12:54:49.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.143+0000 7fd0d7fff640 1 --2- 172.21.15.12:0/771369755 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd0dc0664a0 0x7fd0dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:49.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.143+0000 7fd0d7fff640 1 -- 172.21.15.12:0/771369755 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fd0e81224f0 con 0x7fd0f015cd50 2023-10-30T12:54:49.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.143+0000 7fd0ef7fe640 1 --2- 172.21.15.12:0/771369755 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd0dc0664a0 0x7fd0dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:49.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.143+0000 7fd0ef7fe640 1 --2- 172.21.15.12:0/771369755 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd0dc0664a0 0x7fd0dc068960 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0000f70 tx=0x7fd0e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:49.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.147+0000 7fd0d7fff640 1 -- 172.21.15.12:0/771369755 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd0e80ef590 con 0x7fd0f015cd50 2023-10-30T12:54:49.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.335+0000 7fd0f49a0640 1 -- 172.21.15.12:0/771369755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0f004e170 con 0x7fd0f015cd50 2023-10-30T12:54:49.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.339+0000 7fd0d7fff640 1 -- 172.21.15.12:0/771369755 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+5743 (secure 0 0 0) 0x7fd0e80f2bb0 con 0x7fd0f015cd50 2023-10-30T12:54:49.342 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:49.342 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:43.582549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:49.343 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 103 2023-10-30T12:54:49.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.339+0000 7fd0f49a0640 1 -- 172.21.15.12:0/771369755 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd0dc0664a0 msgr2=0x7fd0dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:49.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.339+0000 7fd0f49a0640 1 --2- 172.21.15.12:0/771369755 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd0dc0664a0 0x7fd0dc068960 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0000f70 tx=0x7fd0e0009040 comp rx=0 tx=0).stop 2023-10-30T12:54:49.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.339+0000 7fd0f49a0640 1 -- 172.21.15.12:0/771369755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0f015cd50 msgr2=0x7fd0f0152a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:49.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.339+0000 7fd0f49a0640 1 --2- 172.21.15.12:0/771369755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0f015cd50 0x7fd0f0152a70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80b2040 tx=0x7fd0e801bf60 comp rx=0 tx=0).stop 2023-10-30T12:54:49.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.339+0000 7fd0f49a0640 1 -- 172.21.15.12:0/771369755 shutdown_connections 2023-10-30T12:54:49.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.339+0000 7fd0f49a0640 1 --2- 172.21.15.12:0/771369755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0f015cd50 0x7fd0f0152a70 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:49.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.339+0000 7fd0f49a0640 1 --2- 172.21.15.12:0/771369755 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd0dc0664a0 0x7fd0dc068960 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:49.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.339+0000 7fd0f49a0640 1 --2- 172.21.15.12:0/771369755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0f01538f0 0x7fd0f0151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:49.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.339+0000 7fd0f49a0640 1 --2- 172.21.15.12:0/771369755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0f00cefa0 0x7fd0f01452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:49.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.339+0000 7fd0f49a0640 1 -- 172.21.15.12:0/771369755 >> 172.21.15.12:0/771369755 conn(0x7fd0f006a640 msgr2=0x7fd0f00c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:49.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.339+0000 7fd0f49a0640 1 -- 172.21.15.12:0/771369755 shutdown_connections 2023-10-30T12:54:49.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:49.339+0000 7fd0f49a0640 1 -- 172.21.15.12:0/771369755 wait complete. 2023-10-30T12:54:49.366 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:54:51.367 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:51.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.515+0000 7f10b433e640 1 Processor -- start 2023-10-30T12:54:51.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.515+0000 7f10b433e640 1 -- start start 2023-10-30T12:54:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.519+0000 7f10b433e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f10ac0c6b80 0x7f10ac0bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.519+0000 7f10b433e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10ac13f4a0 0x7f10ac0c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.519+0000 7f10b433e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ac0c0a50 con 0x7f10ac0c6f60 2023-10-30T12:54:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.519+0000 7f10b433e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ac06f6e0 con 0x7f10ac0c6b80 2023-10-30T12:54:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.519+0000 7f10b433e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac06f9c0 con 0x7f10ac13f4a0 2023-10-30T12:54:51.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.519+0000 7f10b20da640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f10ac0c6b80 0x7f10ac0bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:51.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.519+0000 7f10b20da640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f10ac0c6b80 0x7f10ac0bfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35234/0 (socket says 172.21.15.12:35234) 2023-10-30T12:54:51.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.519+0000 7f10b20da640 1 -- 172.21.15.12:0/2301839556 learned_addr learned my addr 172.21.15.12:0/2301839556 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:51.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.519+0000 7f10b28db640 1 --2- 172.21.15.12:0/2301839556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10ac13f4a0 0x7f10ac0c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:51.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.519+0000 7f10b20da640 1 -- 172.21.15.12:0/2301839556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10ac13f4a0 msgr2=0x7f10ac0c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.519+0000 7f10b20da640 1 --2- 172.21.15.12:0/2301839556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10ac13f4a0 0x7f10ac0c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.519+0000 7f10b20da640 1 -- 172.21.15.12:0/2301839556 >> v1:172.21.15.154:6789/0 conn(0x7f10ac0c6f60 legacy=0x7f10ac13ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.519+0000 7f10b20da640 1 -- 172.21.15.12:0/2301839556 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac06fca0 con 0x7f10ac0c6b80 2023-10-30T12:54:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.523+0000 7f10b20da640 1 --2- 172.21.15.12:0/2301839556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10ac0c6b80 0x7f10ac0bfec0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009870 tx=0x7f10a801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997aad42a421c758 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.523+0000 7f10b10d8640 1 -- 172.21.15.12:0/2301839556 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a80ab650 con 0x7f10ac0c6b80 2023-10-30T12:54:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.523+0000 7f10b10d8640 1 -- 172.21.15.12:0/2301839556 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a80ab7b0 con 0x7f10ac0c6b80 2023-10-30T12:54:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.523+0000 7f10b10d8640 1 -- 172.21.15.12:0/2301839556 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a80abb40 con 0x7f10ac0c6b80 2023-10-30T12:54:51.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.523+0000 7f10b433e640 1 -- 172.21.15.12:0/2301839556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10ac0c6b80 msgr2=0x7f10ac0bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:51.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.523+0000 7f10b433e640 1 --2- 172.21.15.12:0/2301839556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10ac0c6b80 0x7f10ac0bfec0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009870 tx=0x7f10a801c540 comp rx=0 tx=0).stop 2023-10-30T12:54:51.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.523+0000 7f10b433e640 1 -- 172.21.15.12:0/2301839556 shutdown_connections 2023-10-30T12:54:51.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.523+0000 7f10b433e640 1 --2- 172.21.15.12:0/2301839556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10ac13f4a0 0x7f10ac0c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:51.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.523+0000 7f10b433e640 1 --2- 172.21.15.12:0/2301839556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10ac0c6b80 0x7f10ac0bfec0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:51.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.523+0000 7f10b433e640 1 -- 172.21.15.12:0/2301839556 >> 172.21.15.12:0/2301839556 conn(0x7f10ac0c7910 msgr2=0x7f10ac0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:51.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.523+0000 7f10b433e640 1 -- 172.21.15.12:0/2301839556 shutdown_connections 2023-10-30T12:54:51.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.523+0000 7f10b433e640 1 -- 172.21.15.12:0/2301839556 wait complete. 2023-10-30T12:54:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.523+0000 7f10b433e640 1 Processor -- start 2023-10-30T12:54:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.523+0000 7f10b433e640 1 -- start start 2023-10-30T12:54:51.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b433e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10ac0c6b80 0x7f10ac0ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:51.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b433e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10ac0c6f60 0x7f10ac1f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:51.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b433e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10ac13f4a0 0x7f10ac1f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:51.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b433e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac136390 con 0x7f10ac13f4a0 2023-10-30T12:54:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b433e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac154e60 con 0x7f10ac0c6f60 2023-10-30T12:54:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b433e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac13bce0 con 0x7f10ac0c6b80 2023-10-30T12:54:51.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b28db640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10ac13f4a0 0x7f10ac1f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:51.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b28db640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10ac13f4a0 0x7f10ac1f58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35248/0 (socket says 172.21.15.12:35248) 2023-10-30T12:54:51.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b28db640 1 -- 172.21.15.12:0/2187367571 learned_addr learned my addr 172.21.15.12:0/2187367571 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:51.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b20da640 1 --2- 172.21.15.12:0/2187367571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10ac0c6b80 0x7f10ac0ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:51.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b18d9640 1 --2- 172.21.15.12:0/2187367571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10ac0c6f60 0x7f10ac1f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:51.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b28db640 1 -- 172.21.15.12:0/2187367571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10ac0c6b80 msgr2=0x7f10ac0ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:51.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b28db640 1 --2- 172.21.15.12:0/2187367571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10ac0c6b80 0x7f10ac0ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:51.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b28db640 1 -- 172.21.15.12:0/2187367571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10ac0c6f60 msgr2=0x7f10ac1f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b28db640 1 --2- 172.21.15.12:0/2187367571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10ac0c6f60 0x7f10ac1f53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b28db640 1 -- 172.21.15.12:0/2187367571 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a80094d0 con 0x7f10ac13f4a0 2023-10-30T12:54:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b28db640 1 --2- 172.21.15.12:0/2187367571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10ac13f4a0 0x7f10ac1f58e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f10a4007e90 tx=0x7f10a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10a2ffd640 1 -- 172.21.15.12:0/2187367571 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4014420 con 0x7f10ac13f4a0 2023-10-30T12:54:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10a2ffd640 1 -- 172.21.15.12:0/2187367571 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a4014580 con 0x7f10ac13f4a0 2023-10-30T12:54:51.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10a2ffd640 1 -- 172.21.15.12:0/2187367571 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4014930 con 0x7f10ac13f4a0 2023-10-30T12:54:51.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b433e640 1 -- 172.21.15.12:0/2187367571 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ac1553c0 con 0x7f10ac13f4a0 2023-10-30T12:54:51.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b433e640 1 -- 172.21.15.12:0/2187367571 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ac135dd0 con 0x7f10ac13f4a0 2023-10-30T12:54:51.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10b433e640 1 -- 172.21.15.12:0/2187367571 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ac155550 con 0x7f10ac13f4a0 2023-10-30T12:54:51.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.527+0000 7f10a2ffd640 1 -- 172.21.15.12:0/2187367571 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f10a4014a90 con 0x7f10ac13f4a0 2023-10-30T12:54:51.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.531+0000 7f10a2ffd640 1 --2- 172.21.15.12:0/2187367571 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f10900664d0 0x7f1090068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:51.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.531+0000 7f10a2ffd640 1 -- 172.21.15.12:0/2187367571 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f10a4084a20 con 0x7f10ac13f4a0 2023-10-30T12:54:51.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.531+0000 7f10b20da640 1 --2- 172.21.15.12:0/2187367571 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f10900664d0 0x7f1090068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:51.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.531+0000 7f10b20da640 1 --2- 172.21.15.12:0/2187367571 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f10900664d0 0x7f1090068990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f10a80af000 tx=0x7f10a80b6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.535+0000 7f10a2ffd640 1 -- 172.21.15.12:0/2187367571 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f10a4051ae0 con 0x7f10ac13f4a0 2023-10-30T12:54:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.723+0000 7f10b433e640 1 -- 172.21.15.12:0/2187367571 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10ac04e170 con 0x7f10ac13f4a0 2023-10-30T12:54:51.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.727+0000 7f10a2ffd640 1 -- 172.21.15.12:0/2187367571 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+5743 (secure 0 0 0) 0x7f10a4055100 con 0x7f10ac13f4a0 2023-10-30T12:54:51.730 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:54:51.730 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":104,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:48.673781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 104 2023-10-30T12:54:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.727+0000 7f10b433e640 1 -- 172.21.15.12:0/2187367571 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f10900664d0 msgr2=0x7f1090068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.727+0000 7f10b433e640 1 --2- 172.21.15.12:0/2187367571 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f10900664d0 0x7f1090068990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f10a80af000 tx=0x7f10a80b6040 comp rx=0 tx=0).stop 2023-10-30T12:54:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.727+0000 7f10b433e640 1 -- 172.21.15.12:0/2187367571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10ac13f4a0 msgr2=0x7f10ac1f58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.727+0000 7f10b433e640 1 --2- 172.21.15.12:0/2187367571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10ac13f4a0 0x7f10ac1f58e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f10a4007e90 tx=0x7f10a400e4b0 comp rx=0 tx=0).stop 2023-10-30T12:54:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.727+0000 7f10b433e640 1 -- 172.21.15.12:0/2187367571 shutdown_connections 2023-10-30T12:54:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.727+0000 7f10b433e640 1 --2- 172.21.15.12:0/2187367571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10ac13f4a0 0x7f10ac1f58e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.727+0000 7f10b433e640 1 --2- 172.21.15.12:0/2187367571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10ac0c6f60 0x7f10ac1f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.727+0000 7f10b433e640 1 --2- 172.21.15.12:0/2187367571 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f10900664d0 0x7f1090068990 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.727+0000 7f10b433e640 1 --2- 172.21.15.12:0/2187367571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10ac0c6b80 0x7f10ac0ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:51.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.727+0000 7f10b433e640 1 -- 172.21.15.12:0/2187367571 >> 172.21.15.12:0/2187367571 conn(0x7f10ac0c7910 msgr2=0x7f10ac0c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:51.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.727+0000 7f10b433e640 1 -- 172.21.15.12:0/2187367571 shutdown_connections 2023-10-30T12:54:51.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:51.727+0000 7f10b433e640 1 -- 172.21.15.12:0/2187367571 wait complete. 2023-10-30T12:54:51.751 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=1, replacing gid=5758 2023-10-30T12:54:51.751 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 8 secs before reviving mds.d 2023-10-30T12:54:52.411 INFO:tasks.ceph.mds.e.smithi012.stderr:2023-10-30T12:54:52.407+0000 7f0f23052640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-30T12:54:59.753 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2023-10-30T12:54:59.753 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-30T12:54:59.754 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-30T12:54:59.757 INFO:tasks.ceph.mds.d:Started 2023-10-30T12:54:59.757 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:59.833 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2023-10-30T12:54:59.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f1adf8640 1 Processor -- start 2023-10-30T12:54:59.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f1adf8640 1 -- start start 2023-10-30T12:54:59.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f1adf8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0f141538a0 0x7f0f1415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:59.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f1adf8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0f140cf160 0x7f0f140cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:59.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f1adf8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f140cb800 con 0x7f0f140ca0e0 2023-10-30T12:54:59.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f1adf8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f1415dc30 con 0x7f0f140cf160 2023-10-30T12:54:59.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f1adf8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f140cb330 con 0x7f0f141538a0 2023-10-30T12:54:59.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f19395640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0f140cf160 0x7f0f140cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:59.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f19395640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0f140cf160 0x7f0f140cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35256/0 (socket says 172.21.15.12:35256) 2023-10-30T12:54:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f19395640 1 -- 172.21.15.12:0/3276328494 learned_addr learned my addr 172.21.15.12:0/3276328494 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f13fff640 1 --1- 172.21.15.12:0/3276328494 >> v1:172.21.15.12:6789/0 conn(0x7f0f140ca0e0 0x7f0f1415a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33540/0 (socket says 172.21.15.12:33540) 2023-10-30T12:54:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f18b94640 1 --2- 172.21.15.12:0/3276328494 >> v2:172.21.15.154:3300/0 conn(0x7f0f141538a0 0x7f0f1415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f19395640 1 -- 172.21.15.12:0/3276328494 >> v2:172.21.15.154:3300/0 conn(0x7f0f141538a0 msgr2=0x7f0f1415d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f19395640 1 --2- 172.21.15.12:0/3276328494 >> v2:172.21.15.154:3300/0 conn(0x7f0f141538a0 0x7f0f1415d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f19395640 1 -- 172.21.15.12:0/3276328494 >> v1:172.21.15.12:6789/0 conn(0x7f0f140ca0e0 legacy=0x7f0f1415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f19395640 1 -- 172.21.15.12:0/3276328494 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f140cad70 con 0x7f0f140cf160 2023-10-30T12:54:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.899+0000 7f0f137fe640 1 -- 172.21.15.12:0/3276328494 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697961536 0 0) 0x7f0f140cb800 con 0x7f0f140ca0e0 2023-10-30T12:54:59.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.903+0000 7f0f19395640 1 --2- 172.21.15.12:0/3276328494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f140cf160 0x7f0f140cf590 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0f140cfd10 tx=0x7f0f0c01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332cc7a6b51ce32 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:59.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.903+0000 7f0f137fe640 1 -- 172.21.15.12:0/3276328494 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c0af400 con 0x7f0f140cf160 2023-10-30T12:54:59.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.903+0000 7f0f137fe640 1 -- 172.21.15.12:0/3276328494 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0c0ab050 con 0x7f0f140cf160 2023-10-30T12:54:59.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.903+0000 7f0f137fe640 1 -- 172.21.15.12:0/3276328494 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c0af780 con 0x7f0f140cf160 2023-10-30T12:54:59.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.903+0000 7f0f1adf8640 1 -- 172.21.15.12:0/3276328494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f140cf160 msgr2=0x7f0f140cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:59.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.903+0000 7f0f1adf8640 1 --2- 172.21.15.12:0/3276328494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f140cf160 0x7f0f140cf590 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0f140cfd10 tx=0x7f0f0c01f1c0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.903+0000 7f0f1adf8640 1 -- 172.21.15.12:0/3276328494 shutdown_connections 2023-10-30T12:54:59.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.903+0000 7f0f1adf8640 1 --2- 172.21.15.12:0/3276328494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f140cf160 0x7f0f140cf590 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.903+0000 7f0f1adf8640 1 --2- 172.21.15.12:0/3276328494 >> v2:172.21.15.154:3300/0 conn(0x7f0f141538a0 0x7f0f1415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.903+0000 7f0f1adf8640 1 -- 172.21.15.12:0/3276328494 >> 172.21.15.12:0/3276328494 conn(0x7f0f1406bfe0 msgr2=0x7f0f1406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:59.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.903+0000 7f0f1adf8640 1 -- 172.21.15.12:0/3276328494 shutdown_connections 2023-10-30T12:54:59.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.903+0000 7f0f1adf8640 1 -- 172.21.15.12:0/3276328494 wait complete. 2023-10-30T12:54:59.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f1adf8640 1 Processor -- start 2023-10-30T12:54:59.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f1adf8640 1 -- start start 2023-10-30T12:54:59.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f1adf8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f140ca0e0 0x7f0f140b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:59.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f1adf8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f140cf160 0x7f0f141490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:59.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f1adf8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f141538a0 0x7f0f141efe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:59.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f1adf8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f140b4940 con 0x7f0f140cf160 2023-10-30T12:54:59.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f1adf8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f14148b30 con 0x7f0f140ca0e0 2023-10-30T12:54:59.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f1adf8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f140b3a50 con 0x7f0f141538a0 2023-10-30T12:54:59.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f13fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f140cf160 0x7f0f141490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:59.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f13fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f140cf160 0x7f0f141490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35270/0 (socket says 172.21.15.12:35270) 2023-10-30T12:54:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f13fff640 1 -- 172.21.15.12:0/78169991 learned_addr learned my addr 172.21.15.12:0/78169991 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:54:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f18b94640 1 --2- 172.21.15.12:0/78169991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f140ca0e0 0x7f0f140b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f19395640 1 --2- 172.21.15.12:0/78169991 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f141538a0 0x7f0f141efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f13fff640 1 -- 172.21.15.12:0/78169991 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f141538a0 msgr2=0x7f0f141efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f13fff640 1 --2- 172.21.15.12:0/78169991 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f141538a0 0x7f0f141efe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f13fff640 1 -- 172.21.15.12:0/78169991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f140ca0e0 msgr2=0x7f0f140b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f13fff640 1 --2- 172.21.15.12:0/78169991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f140ca0e0 0x7f0f140b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f13fff640 1 -- 172.21.15.12:0/78169991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c009b30 con 0x7f0f140cf160 2023-10-30T12:54:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f19395640 1 --2- 172.21.15.12:0/78169991 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f141538a0 0x7f0f141efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f13fff640 1 --2- 172.21.15.12:0/78169991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f140cf160 0x7f0f141490b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000c4b0 tx=0x7f0f0000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f117fa640 1 -- 172.21.15.12:0/78169991 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f000074e0 con 0x7f0f140cf160 2023-10-30T12:54:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f117fa640 1 -- 172.21.15.12:0/78169991 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f00007660 con 0x7f0f140cf160 2023-10-30T12:54:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f117fa640 1 -- 172.21.15.12:0/78169991 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00004a30 con 0x7f0f140cf160 2023-10-30T12:54:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f1adf8640 1 -- 172.21.15.12:0/78169991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f140b3d30 con 0x7f0f140cf160 2023-10-30T12:54:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.907+0000 7f0f1adf8640 1 -- 172.21.15.12:0/78169991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f140b42f0 con 0x7f0f140cf160 2023-10-30T12:54:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.911+0000 7f0f1adf8640 1 -- 172.21.15.12:0/78169991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f140b2a50 con 0x7f0f140cf160 2023-10-30T12:54:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.911+0000 7f0f117fa640 1 -- 172.21.15.12:0/78169991 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0f00004b90 con 0x7f0f140cf160 2023-10-30T12:54:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.911+0000 7f0f117fa640 1 --2- 172.21.15.12:0/78169991 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0ef80664a0 0x7f0ef8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.911+0000 7f0f117fa640 1 -- 172.21.15.12:0/78169991 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f0f00083450 con 0x7f0f140cf160 2023-10-30T12:54:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.911+0000 7f0f18b94640 1 --2- 172.21.15.12:0/78169991 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0ef80664a0 0x7f0ef8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.911+0000 7f0f18b94640 1 --2- 172.21.15.12:0/78169991 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0ef80664a0 0x7f0ef8068960 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009b60 tx=0x7f0f08009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:54:59.915+0000 7f0f117fa640 1 -- 172.21.15.12:0/78169991 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0f00053f10 con 0x7f0f140cf160 2023-10-30T12:55:00.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:00.103+0000 7f0f1adf8640 1 -- 172.21.15.12:0/78169991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f140cb020 con 0x7f0f140cf160 2023-10-30T12:55:00.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:00.103+0000 7f0f117fa640 1 -- 172.21.15.12:0/78169991 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+5743 (secure 0 0 0) 0x7f0f00018090 con 0x7f0f140cf160 2023-10-30T12:55:00.110 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:55:00.110 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":104,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:48.673781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:00.111 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 104 2023-10-30T12:55:00.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:00.107+0000 7f0f1adf8640 1 -- 172.21.15.12:0/78169991 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0ef80664a0 msgr2=0x7f0ef8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:00.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:00.107+0000 7f0f1adf8640 1 --2- 172.21.15.12:0/78169991 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0ef80664a0 0x7f0ef8068960 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009b60 tx=0x7f0f08009190 comp rx=0 tx=0).stop 2023-10-30T12:55:00.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:00.107+0000 7f0f1adf8640 1 -- 172.21.15.12:0/78169991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f140cf160 msgr2=0x7f0f141490b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:00.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:00.107+0000 7f0f1adf8640 1 --2- 172.21.15.12:0/78169991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f140cf160 0x7f0f141490b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000c4b0 tx=0x7f0f0000c970 comp rx=0 tx=0).stop 2023-10-30T12:55:00.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:00.107+0000 7f0f1adf8640 1 -- 172.21.15.12:0/78169991 shutdown_connections 2023-10-30T12:55:00.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:00.107+0000 7f0f1adf8640 1 --2- 172.21.15.12:0/78169991 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0ef80664a0 0x7f0ef8068960 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:00.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:00.107+0000 7f0f1adf8640 1 --2- 172.21.15.12:0/78169991 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f141538a0 0x7f0f141efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:00.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:00.107+0000 7f0f1adf8640 1 --2- 172.21.15.12:0/78169991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f140cf160 0x7f0f141490b0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:00.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:00.107+0000 7f0f1adf8640 1 --2- 172.21.15.12:0/78169991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f140ca0e0 0x7f0f140b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:00.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:00.107+0000 7f0f1adf8640 1 -- 172.21.15.12:0/78169991 >> 172.21.15.12:0/78169991 conn(0x7f0f1406bfe0 msgr2=0x7f0f14065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:00.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:00.107+0000 7f0f1adf8640 1 -- 172.21.15.12:0/78169991 shutdown_connections 2023-10-30T12:55:00.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:00.107+0000 7f0f1adf8640 1 -- 172.21.15.12:0/78169991 wait complete. 2023-10-30T12:55:00.130 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2023-10-30T12:55:02.131 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:02.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.287+0000 7f83fd0c9640 1 Processor -- start 2023-10-30T12:55:02.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.287+0000 7f83fd0c9640 1 -- start start 2023-10-30T12:55:02.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.287+0000 7f83fd0c9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f83f806e470 0x7f83f806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:02.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.287+0000 7f83fd0c9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f806afa0 con 0x7f83f80c8490 2023-10-30T12:55:02.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.287+0000 7f83fd0c9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f8154150 con 0x7f83f8153d70 2023-10-30T12:55:02.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.287+0000 7f83fd0c9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f806d780 con 0x7f83f806e470 2023-10-30T12:55:02.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.287+0000 7f83effff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f83f806e470 0x7f83f806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:02.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.287+0000 7f83f759e640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f83f80c8490 0x7f83f80c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:46192/0 (socket says 172.21.15.12:46192) 2023-10-30T12:55:02.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.287+0000 7f83f759e640 1 -- 172.21.15.12:0/3784917828 learned_addr learned my addr 172.21.15.12:0/3784917828 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:55:02.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.291+0000 7f83effff640 1 -- 172.21.15.12:0/3784917828 >> v1:172.21.15.154:6789/0 conn(0x7f83f8153d70 legacy=0x7f83f80cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:02.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.291+0000 7f83effff640 1 -- 172.21.15.12:0/3784917828 >> v1:172.21.15.12:6789/0 conn(0x7f83f80c8490 legacy=0x7f83f80c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:02.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.291+0000 7f83effff640 1 -- 172.21.15.12:0/3784917828 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f806e030 con 0x7f83f806e470 2023-10-30T12:55:02.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.291+0000 7f83effff640 1 --2- 172.21.15.12:0/3784917828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83f806e470 0x7f83f806ca80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f83e00097c0 tx=0x7f83e001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c102db73c122e05b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.291+0000 7f83f659c640 1 -- 172.21.15.12:0/3784917828 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e00ab650 con 0x7f83f806e470 2023-10-30T12:55:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.291+0000 7f83f659c640 1 -- 172.21.15.12:0/3784917828 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e00ab7b0 con 0x7f83f806e470 2023-10-30T12:55:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.291+0000 7f83f659c640 1 -- 172.21.15.12:0/3784917828 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e00abb40 con 0x7f83f806e470 2023-10-30T12:55:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.291+0000 7f83fd0c9640 1 -- 172.21.15.12:0/3784917828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83f806e470 msgr2=0x7f83f806ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.291+0000 7f83fd0c9640 1 --2- 172.21.15.12:0/3784917828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83f806e470 0x7f83f806ca80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f83e00097c0 tx=0x7f83e001c450 comp rx=0 tx=0).stop 2023-10-30T12:55:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83fd0c9640 1 -- 172.21.15.12:0/3784917828 shutdown_connections 2023-10-30T12:55:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83fd0c9640 1 --2- 172.21.15.12:0/3784917828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83f806e470 0x7f83f806ca80 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83fd0c9640 1 -- 172.21.15.12:0/3784917828 >> 172.21.15.12:0/3784917828 conn(0x7f83f8071ea0 msgr2=0x7f83f8065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83fd0c9640 1 -- 172.21.15.12:0/3784917828 shutdown_connections 2023-10-30T12:55:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83fd0c9640 1 -- 172.21.15.12:0/3784917828 wait complete. 2023-10-30T12:55:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83fd0c9640 1 Processor -- start 2023-10-30T12:55:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83fd0c9640 1 -- start start 2023-10-30T12:55:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83fd0c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f806e470 0x7f83f8154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83fd0c9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83f80c8490 0x7f83f8144260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83fd0c9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83f8153d70 0x7f83f81506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83fd0c9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f81465f0 con 0x7f83f80c8490 2023-10-30T12:55:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83fd0c9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f8143810 con 0x7f83f806e470 2023-10-30T12:55:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83fd0c9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f8143dd0 con 0x7f83f8153d70 2023-10-30T12:55:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83effff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83f80c8490 0x7f83f8144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:02.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83effff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83f80c8490 0x7f83f8144260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50544/0 (socket says 172.21.15.12:50544) 2023-10-30T12:55:02.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83effff640 1 -- 172.21.15.12:0/4114486691 learned_addr learned my addr 172.21.15.12:0/4114486691 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:55:02.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83f6d9d640 1 --2- 172.21.15.12:0/4114486691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f806e470 0x7f83f8154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:02.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83f759e640 1 --2- 172.21.15.12:0/4114486691 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83f8153d70 0x7f83f81506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:02.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83effff640 1 -- 172.21.15.12:0/4114486691 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83f8153d70 msgr2=0x7f83f81506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:02.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83effff640 1 --2- 172.21.15.12:0/4114486691 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83f8153d70 0x7f83f81506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83effff640 1 -- 172.21.15.12:0/4114486691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f806e470 msgr2=0x7f83f8154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:02.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83effff640 1 --2- 172.21.15.12:0/4114486691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f806e470 0x7f83f8154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83effff640 1 -- 172.21.15.12:0/4114486691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e0009420 con 0x7f83f80c8490 2023-10-30T12:55:02.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83f6d9d640 1 --2- 172.21.15.12:0/4114486691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f806e470 0x7f83f8154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:55:02.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.295+0000 7f83effff640 1 --2- 172.21.15.12:0/4114486691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83f80c8490 0x7f83f8144260 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f83e00ad040 tx=0x7f83e001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:02.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.299+0000 7f83ef7fe640 1 -- 172.21.15.12:0/4114486691 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e00b5810 con 0x7f83f80c8490 2023-10-30T12:55:02.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.299+0000 7f83ef7fe640 1 -- 172.21.15.12:0/4114486691 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e00b59b0 con 0x7f83f80c8490 2023-10-30T12:55:02.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.299+0000 7f83fd0c9640 1 -- 172.21.15.12:0/4114486691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f81447a0 con 0x7f83f80c8490 2023-10-30T12:55:02.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.299+0000 7f83ef7fe640 1 -- 172.21.15.12:0/4114486691 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e00b5d60 con 0x7f83f80c8490 2023-10-30T12:55:02.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.299+0000 7f83fd0c9640 1 -- 172.21.15.12:0/4114486691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f81547b0 con 0x7f83f80c8490 2023-10-30T12:55:02.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.299+0000 7f83fd0c9640 1 -- 172.21.15.12:0/4114486691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f80cd490 con 0x7f83f80c8490 2023-10-30T12:55:02.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.299+0000 7f83ef7fe640 1 -- 172.21.15.12:0/4114486691 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f83e00b4390 con 0x7f83f80c8490 2023-10-30T12:55:02.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.299+0000 7f83ef7fe640 1 --2- 172.21.15.12:0/4114486691 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f83d80664a0 0x7f83d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.299+0000 7f83ef7fe640 1 -- 172.21.15.12:0/4114486691 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f83e011efd0 con 0x7f83f80c8490 2023-10-30T12:55:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.299+0000 7f83f6d9d640 1 --2- 172.21.15.12:0/4114486691 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f83d80664a0 0x7f83d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.303+0000 7f83f6d9d640 1 --2- 172.21.15.12:0/4114486691 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f83d80664a0 0x7f83d8068960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f83e8000f70 tx=0x7f83e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:02.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.303+0000 7f83ef7fe640 1 -- 172.21.15.12:0/4114486691 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f83e00ebed0 con 0x7f83f80c8490 2023-10-30T12:55:02.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.507+0000 7f83fd0c9640 1 -- 172.21.15.12:0/4114486691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83f804e170 con 0x7f83f80c8490 2023-10-30T12:55:02.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.507+0000 7f83ef7fe640 1 -- 172.21.15.12:0/4114486691 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+6538 (secure 0 0 0) 0x7f83e00ef4f0 con 0x7f83f80c8490 2023-10-30T12:55:02.511 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:55:02.511 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":105,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1182202818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1182202818},{"type":"v1","addr":"172.21.15.154:6839","nonce":1182202818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:54:48.673781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:02.511 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 105 2023-10-30T12:55:02.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.507+0000 7f83fd0c9640 1 -- 172.21.15.12:0/4114486691 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f83d80664a0 msgr2=0x7f83d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:02.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.507+0000 7f83fd0c9640 1 --2- 172.21.15.12:0/4114486691 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f83d80664a0 0x7f83d8068960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f83e8000f70 tx=0x7f83e8009040 comp rx=0 tx=0).stop 2023-10-30T12:55:02.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.507+0000 7f83fd0c9640 1 -- 172.21.15.12:0/4114486691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83f80c8490 msgr2=0x7f83f8144260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:02.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.507+0000 7f83fd0c9640 1 --2- 172.21.15.12:0/4114486691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83f80c8490 0x7f83f8144260 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f83e00ad040 tx=0x7f83e001ac90 comp rx=0 tx=0).stop 2023-10-30T12:55:02.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.507+0000 7f83fd0c9640 1 -- 172.21.15.12:0/4114486691 shutdown_connections 2023-10-30T12:55:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.507+0000 7f83fd0c9640 1 --2- 172.21.15.12:0/4114486691 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f83d80664a0 0x7f83d8068960 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.507+0000 7f83fd0c9640 1 --2- 172.21.15.12:0/4114486691 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83f8153d70 0x7f83f81506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.507+0000 7f83fd0c9640 1 --2- 172.21.15.12:0/4114486691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83f80c8490 0x7f83f8144260 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.507+0000 7f83fd0c9640 1 --2- 172.21.15.12:0/4114486691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f806e470 0x7f83f8154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.507+0000 7f83fd0c9640 1 -- 172.21.15.12:0/4114486691 >> 172.21.15.12:0/4114486691 conn(0x7f83f8071ea0 msgr2=0x7f83f81530a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:02.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.507+0000 7f83fd0c9640 1 -- 172.21.15.12:0/4114486691 shutdown_connections 2023-10-30T12:55:02.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:55:02.507+0000 7f83fd0c9640 1 -- 172.21.15.12:0/4114486691 wait complete. 2023-10-30T12:55:02.532 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2023-10-30T12:55:02.532 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 102 secs before thrashing 2023-10-30T12:55:07.767 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:55:07.771 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:55:37.848 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:55:37.852 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:56:07.899 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:56:07.903 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:56:09.899 INFO:tasks.workunit.client.0.smithi012.stderr:Updating files: 1% (141/12071) Updating files: 2% (242/12071) Updating files: 3% (363/12071) Updating files: 3% (424/12071) Updating files: 4% (483/12071) Updating files: 4% (512/12071) Updating files: 4% (568/12071) Updating files: 5% (604/12071) Updating files: 5% (615/12071) Updating files: 5% (721/12071) Updating files: 6% (725/12071) Updating files: 6% (835/12071) Updating files: 7% (845/12071) Updating files: 8% (966/12071) Updating files: 8% (1079/12071) Updating files: 9% (1087/12071) Updating files: 9% (1111/12071) Updating files: 10% (1208/12071) Updating files: 11% (1328/12071) Updating files: 11% (1415/12071) Updating files: 12% (1449/12071) Updating files: 12% (1462/12071) Updating files: 12% (1530/12071) Updating files: 13% (1570/12071) Updating files: 13% (1603/12071) Updating files: 14% (1690/12071) Updating files: 14% (1764/12071) Updating files: 14% (1775/12071) Updating files: 15% (1811/12071) Updating files: 16% (1932/12071) Updating files: 16% (2029/12071) Updating files: 17% (2053/12071) Updating files: 18% (2173/12071) Updating files: 18% (2218/12071) Updating files: 19% (2294/12071) Updating files: 20% (2415/12071) Updating files: 21% (2535/12071) Updating files: 21% (2621/12071) Updating files: 22% (2656/12071) Updating files: 22% (2770/12071) Updating files: 23% (2777/12071) Updating files: 24% (2898/12071) Updating files: 24% (2900/12071) Updating files: 25% (3018/12071) Updating files: 25% (3055/12071) Updating files: 25% (3094/12071) Updating files: 26% (3139/12071) Updating files: 26% (3155/12071) Updating files: 26% (3214/12071) Updating files: 27% (3260/12071) Updating files: 27% (3296/12071) Updating files: 28% (3380/12071) Updating files: 29% (3501/12071) Updating files: 29% (3599/12071) Updating files: 30% (3622/12071) Updating files: 31% (3743/12071) Updating files: 31% (3819/12071) Updating files: 31% (3861/12071) Updating files: 32% (3863/12071) Updating files: 32% (3948/12071) Updating files: 33% (3984/12071) Updating files: 33% (4072/12071) Updating files: 33% (4096/12071) Updating files: 34% (4105/12071) Updating files: 34% (4177/12071) Updating files: 35% (4225/12071) Updating files: 35% (4266/12071) Updating files: 36% (4346/12071) Updating files: 36% (4420/12071) Updating files: 37% (4467/12071) Updating files: 38% (4587/12071) Updating files: 39% (4708/12071) Updating files: 39% (4714/12071) Updating files: 39% (4761/12071) Updating files: 40% (4829/12071) Updating files: 40% (4938/12071) Updating files: 41% (4950/12071) Updating files: 42% (5070/12071) Updating files: 42% (5083/12071) Updating files: 43% (5191/12071) Updating files: 43% (5232/12071) Updating files: 44% (5312/12071) Updating files: 44% (5376/12071) Updating files: 45% (5432/12071) Updating files: 46% (5553/12071) Updating files: 47% (5674/12071) Updating files: 47% (5678/12071) Updating files: 47% (5679/12071) Updating files: 47% (5690/12071) Updating files: 47% (5719/12071) Updating files: 47% (5731/12071) Updating files: 47% (5789/12071) Updating files: 48% (5795/12071) Updating files: 49% (5915/12071) Updating files: 50% (6036/12071) Updating files: 50% (6139/12071) Updating files: 51% (6157/12071) Updating files: 51% (6203/12071) Updating files: 51% (6255/12071) Updating files: 52% (6277/12071) Updating files: 52% (6299/12071) Updating files: 53% (6398/12071) Updating files: 53% (6437/12071) Updating files: 53% (6505/12071) Updating files: 53% (6506/12071) Updating files: 54% (6519/12071) Updating files: 54% (6572/12071) Updating files: 54% (6601/12071) Updating files: 55% (6640/12071) Updating files: 55% (6732/12071) Updating files: 56% (6760/12071) Updating files: 57% (6881/12071) Updating files: 57% (6894/12071) Updating files: 58% (7002/12071) Updating files: 58% (7075/12071) Updating files: 59% (7122/12071) Updating files: 60% (7243/12071) Updating files: 61% (7364/12071) Updating files: 61% (7470/12071) Updating files: 62% (7485/12071) Updating files: 62% (7582/12071) Updating files: 63% (7605/12071) Updating files: 63% (7701/12071) Updating files: 64% (7726/12071) Updating files: 65% (7847/12071) Updating files: 65% (7892/12071) Updating files: 66% (7967/12071) Updating files: 67% (8088/12071) Updating files: 67% (8109/12071) Updating files: 67% (8167/12071) Updating files: 67% (8204/12071) Updating files: 68% (8209/12071) Updating files: 69% (8329/12071) Updating files: 70% (8450/12071) Updating files: 70% (8514/12071) Updating files: 71% (8571/12071) Updating files: 71% (8680/12071) Updating files: 72% (8692/12071) Updating files: 72% (8782/12071) Updating files: 73% (8812/12071) Updating files: 73% (8931/12071) Updating files: 74% (8933/12071) Updating files: 74% (9047/12071) Updating files: 75% (9054/12071) Updating files: 76% (9174/12071) Updating files: 76% (9190/12071) Updating files: 76% (9242/12071) Updating files: 77% (9295/12071) Updating files: 77% (9393/12071) Updating files: 78% (9416/12071) Updating files: 78% (9490/12071) Updating files: 79% (9537/12071) Updating files: 80% (9657/12071) Updating files: 80% (9674/12071) Updating files: 80% (9702/12071) Updating files: 80% (9741/12071) Updating files: 80% (9772/12071) Updating files: 81% (9778/12071) Updating files: 81% (9804/12071) Updating files: 81% (9867/12071) Updating files: 82% (9899/12071) Updating files: 82% (9905/12071) Updating files: 83% (10019/12071) Updating files: 84% (10140/12071) Updating files: 85% (10261/12071) Updating files: 85% (10347/12071) Updating files: 86% (10382/12071) Updating files: 86% (10449/12071) Updating files: 87% (10502/12071) Updating files: 88% (10623/12071) Updating files: 88% (10647/12071) Updating files: 88% (10731/12071) Updating files: 89% (10744/12071) Updating files: 90% (10864/12071) Updating files: 90% (10870/12071) Updating files: 90% (10923/12071) Updating files: 91% (10985/12071) Updating files: 91% (11015/12071) Updating files: 92% (11106/12071) Updating files: 93% (11227/12071) Updating files: 94% (11347/12071) Updating files: 94% (11351/12071) Updating files: 95% (11468/12071) Updating files: 96% (11589/12071) Updating files: 96% (11603/12071) Updating files: 96% (11629/12071) Updating files: 96% (11659/12071) Updating files: 96% (11670/12071) Updating files: 96% (11700/12071) Updating files: 97% (11709/12071) Updating files: 98% (11830/12071) Updating files: 98% (11832/12071) Updating files: 99% (11951/12071) Updating files: 100% (12071/12071) Updating files: 100% (12071/12071), done. 2023-10-30T12:56:09.973 DEBUG:teuthology.orchestra.run:got remote process result: 128 2023-10-30T12:56:09.974 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.0... 2023-10-30T12:56:09.974 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-10-30T12:56:10.849 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi012 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2023-10-30T12:56:10.918 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=314390cf30fb4062bd25fa01f7bab155 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi012 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2023-10-30T12:56:10.921 DEBUG:teuthology.run_tasks:Unwinding manager mds_thrash 2023-10-30T12:56:10.931 INFO:tasks.mds_thrash:joining mds_thrasher 2023-10-30T12:56:10.934 INFO:tasks.mds_thrash.fs.[cephfs]:stat['max_mds'] = 0 2023-10-30T12:56:10.934 INFO:tasks.mds_thrash.fs.[cephfs]:stat['deactivate'] = 0 2023-10-30T12:56:10.935 INFO:tasks.mds_thrash.fs.[cephfs]:stat['kill'] = 10 2023-10-30T12:56:10.935 INFO:tasks.mds_thrash:done joining 2023-10-30T12:56:10.936 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-30T12:56:10.981 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-30T12:56:10.982 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:56:10.982 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-30T12:56:10.995 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-30T12:56:10.995 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-30T12:56:11.045 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:56:11.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.227+0000 7f9459f0e640 1 Processor -- start 2023-10-30T12:56:11.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.227+0000 7f9459f0e640 1 -- start start 2023-10-30T12:56:11.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.227+0000 7f9459f0e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9454157010 0x7f94540715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.227+0000 7f9459f0e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9454070650 0x7f9454159810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.227+0000 7f9459f0e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9454159d50 con 0x7f94540c8460 2023-10-30T12:56:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.227+0000 7f9459f0e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94540c21b0 con 0x7f9454070650 2023-10-30T12:56:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.227+0000 7f9459f0e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9454070ae0 con 0x7f9454157010 2023-10-30T12:56:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.227+0000 7f9453fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9454070650 0x7f9454159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.227+0000 7f9453fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9454070650 0x7f9454159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52072/0 (socket says 172.21.15.12:52072) 2023-10-30T12:56:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.227+0000 7f9453fff640 1 -- 172.21.15.12:0/2853185625 learned_addr learned my addr 172.21.15.12:0/2853185625 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.227+0000 7f94537fe640 1 --2- 172.21.15.12:0/2853185625 >> v2:172.21.15.154:3300/0 conn(0x7f9454157010 0x7f94540715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f9453fff640 1 -- 172.21.15.12:0/2853185625 >> v2:172.21.15.154:3300/0 conn(0x7f9454157010 msgr2=0x7f94540715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f9453fff640 1 --2- 172.21.15.12:0/2853185625 >> v2:172.21.15.154:3300/0 conn(0x7f9454157010 0x7f94540715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f9453fff640 1 -- 172.21.15.12:0/2853185625 >> v1:172.21.15.154:6789/0 conn(0x7f94540c8460 legacy=0x7f9454071bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f9453fff640 1 -- 172.21.15.12:0/2853185625 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94540c1530 con 0x7f9454070650 2023-10-30T12:56:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f94537fe640 1 --2- 172.21.15.12:0/2853185625 >> v2:172.21.15.154:3300/0 conn(0x7f9454157010 0x7f94540715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f9453fff640 1 --2- 172.21.15.12:0/2853185625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9454070650 0x7f9454159810 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f945406d550 tx=0x7f944c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d5509d57ecffcc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f94527fc640 1 -- 172.21.15.12:0/2853185625 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c0a5860 con 0x7f9454070650 2023-10-30T12:56:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f94527fc640 1 -- 172.21.15.12:0/2853185625 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944c0a59c0 con 0x7f9454070650 2023-10-30T12:56:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f94527fc640 1 -- 172.21.15.12:0/2853185625 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c0a5d50 con 0x7f9454070650 2023-10-30T12:56:11.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f9459f0e640 1 -- 172.21.15.12:0/2853185625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9454070650 msgr2=0x7f9454159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f9459f0e640 1 --2- 172.21.15.12:0/2853185625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9454070650 0x7f9454159810 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f945406d550 tx=0x7f944c01c9a0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f9459f0e640 1 -- 172.21.15.12:0/2853185625 shutdown_connections 2023-10-30T12:56:11.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f9459f0e640 1 --2- 172.21.15.12:0/2853185625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9454070650 0x7f9454159810 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f9459f0e640 1 --2- 172.21.15.12:0/2853185625 >> v2:172.21.15.154:3300/0 conn(0x7f9454157010 0x7f94540715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f9459f0e640 1 -- 172.21.15.12:0/2853185625 >> 172.21.15.12:0/2853185625 conn(0x7f94540cbeb0 msgr2=0x7f94540cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:11.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f9459f0e640 1 -- 172.21.15.12:0/2853185625 shutdown_connections 2023-10-30T12:56:11.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.231+0000 7f9459f0e640 1 -- 172.21.15.12:0/2853185625 wait complete. 2023-10-30T12:56:11.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9459f0e640 1 Processor -- start 2023-10-30T12:56:11.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9459f0e640 1 -- start start 2023-10-30T12:56:11.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9459f0e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9454070650 0x7f94541f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9459f0e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94540c8460 0x7f945406b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9459f0e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9454157010 0x7f9454143780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9459f0e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94541f2ad0 con 0x7f9454157010 2023-10-30T12:56:11.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9459f0e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94540ca960 con 0x7f9454070650 2023-10-30T12:56:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9459f0e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94540cac00 con 0x7f94540c8460 2023-10-30T12:56:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9453fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9454157010 0x7f9454143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9453fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9454157010 0x7f9454143780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52084/0 (socket says 172.21.15.12:52084) 2023-10-30T12:56:11.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9453fff640 1 -- 172.21.15.12:0/2514971782 learned_addr learned my addr 172.21.15.12:0/2514971782 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f94537fe640 1 --2- 172.21.15.12:0/2514971782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9454070650 0x7f94541f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9452ffd640 1 --2- 172.21.15.12:0/2514971782 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94540c8460 0x7f945406b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9453fff640 1 -- 172.21.15.12:0/2514971782 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94540c8460 msgr2=0x7f945406b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9453fff640 1 --2- 172.21.15.12:0/2514971782 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94540c8460 0x7f945406b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9453fff640 1 -- 172.21.15.12:0/2514971782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9454070650 msgr2=0x7f94541f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9453fff640 1 --2- 172.21.15.12:0/2514971782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9454070650 0x7f94541f2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9453fff640 1 -- 172.21.15.12:0/2514971782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944c009b30 con 0x7f9454157010 2023-10-30T12:56:11.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f9453fff640 1 --2- 172.21.15.12:0/2514971782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9454157010 0x7f9454143780 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f944c0af040 tx=0x7f944c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:11.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.235+0000 7f943bfff640 1 -- 172.21.15.12:0/2514971782 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c0b8670 con 0x7f9454157010 2023-10-30T12:56:11.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.239+0000 7f943bfff640 1 -- 172.21.15.12:0/2514971782 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944c0b87f0 con 0x7f9454157010 2023-10-30T12:56:11.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.239+0000 7f943bfff640 1 -- 172.21.15.12:0/2514971782 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c0b8ba0 con 0x7f9454157010 2023-10-30T12:56:11.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.239+0000 7f9459f0e640 1 -- 172.21.15.12:0/2514971782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94540caee0 con 0x7f9454157010 2023-10-30T12:56:11.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.239+0000 7f9459f0e640 1 -- 172.21.15.12:0/2514971782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94540cb4a0 con 0x7f9454157010 2023-10-30T12:56:11.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.239+0000 7f9459f0e640 1 -- 172.21.15.12:0/2514971782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f945404e170 con 0x7f9454157010 2023-10-30T12:56:11.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.239+0000 7f943bfff640 1 -- 172.21.15.12:0/2514971782 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f944c0b8d00 con 0x7f9454157010 2023-10-30T12:56:11.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.239+0000 7f943bfff640 1 --2- 172.21.15.12:0/2514971782 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94400664a0 0x7f9440068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.239+0000 7f943bfff640 1 -- 172.21.15.12:0/2514971782 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f944c11fa90 con 0x7f9454157010 2023-10-30T12:56:11.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.239+0000 7f94537fe640 1 --2- 172.21.15.12:0/2514971782 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94400664a0 0x7f9440068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.243+0000 7f94537fe640 1 --2- 172.21.15.12:0/2514971782 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94400664a0 0x7f9440068960 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f94440045f0 tx=0x7f9444008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:11.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.243+0000 7f943bfff640 1 -- 172.21.15.12:0/2514971782 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f944c0ecb40 con 0x7f9454157010 2023-10-30T12:56:11.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.403+0000 7f9459f0e640 1 -- 172.21.15.12:0/2514971782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94541567b0 con 0x7f9454157010 2023-10-30T12:56:11.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.403+0000 7f943bfff640 1 -- 172.21.15.12:0/2514971782 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+18079 (secure 0 0 0) 0x7f944c0f0160 con 0x7f9454157010 2023-10-30T12:56:11.408 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:56:11.409 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":55,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:54:25.460451+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:35:03.216053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:35:04.597162+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":46,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6836/2204547612":"2023-10-31T12:54:25.460390+0000","172.21.15.12:6837/2752088514":"2023-10-31T12:53:20.432410+0000","172.21.15.12:6834/2752088514":"2023-10-31T12:53:20.432410+0000","172.21.15.154:6839/2204547612":"2023-10-31T12:54:25.460390+0000","172.21.15.12:6836/2219276371":"2023-10-31T12:50:50.379953+0000","172.21.15.12:6833/2219276371":"2023-10-31T12:50:50.379953+0000","172.21.15.12:6835/559649789":"2023-10-31T12:49:35.359754+0000","172.21.15.12:6834/3836803994":"2023-10-31T12:37:55.139871+0000","172.21.15.154:6836/1517672481":"2023-10-31T12:43:13.483926+0000","172.21.15.12:6832/2676506149":"2023-10-31T12:40:40.195783+0000","172.21.15.154:6839/1517672481":"2023-10-31T12:43:13.483926+0000","172.21.15.12:6834/3124135109":"2023-10-31T12:46:40.313866+0000","172.21.15.12:6837/3836803994":"2023-10-31T12:37:55.139871+0000","172.21.15.12:6836/291392212":"2023-10-31T12:44:35.272569+0000","172.21.15.12:6837/3124135109":"2023-10-31T12:46:40.313866+0000","172.21.15.12:6833/291392212":"2023-10-31T12:44:35.272569+0000","172.21.15.154:6836/1635719967":"2023-10-31T12:47:45.331652+0000","172.21.15.12:6835/2676506149":"2023-10-31T12:40:40.195783+0000","172.21.15.154:6839/1635719967":"2023-10-31T12:47:45.331652+0000","172.21.15.12:6832/559649789":"2023-10-31T12:49:35.359754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:56:11.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.407+0000 7f9459f0e640 1 -- 172.21.15.12:0/2514971782 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94400664a0 msgr2=0x7f9440068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.407+0000 7f9459f0e640 1 --2- 172.21.15.12:0/2514971782 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94400664a0 0x7f9440068960 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f94440045f0 tx=0x7f9444008040 comp rx=0 tx=0).stop 2023-10-30T12:56:11.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.407+0000 7f9459f0e640 1 -- 172.21.15.12:0/2514971782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9454157010 msgr2=0x7f9454143780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.407+0000 7f9459f0e640 1 --2- 172.21.15.12:0/2514971782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9454157010 0x7f9454143780 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f944c0af040 tx=0x7f944c019740 comp rx=0 tx=0).stop 2023-10-30T12:56:11.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.407+0000 7f9459f0e640 1 -- 172.21.15.12:0/2514971782 shutdown_connections 2023-10-30T12:56:11.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.407+0000 7f9459f0e640 1 --2- 172.21.15.12:0/2514971782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9454157010 0x7f9454143780 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.407+0000 7f9459f0e640 1 --2- 172.21.15.12:0/2514971782 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f94400664a0 0x7f9440068960 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.407+0000 7f9459f0e640 1 --2- 172.21.15.12:0/2514971782 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94540c8460 0x7f945406b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.407+0000 7f9459f0e640 1 --2- 172.21.15.12:0/2514971782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9454070650 0x7f94541f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.407+0000 7f9459f0e640 1 -- 172.21.15.12:0/2514971782 >> 172.21.15.12:0/2514971782 conn(0x7f94540cbeb0 msgr2=0x7f94540cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:11.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.407+0000 7f9459f0e640 1 -- 172.21.15.12:0/2514971782 shutdown_connections 2023-10-30T12:56:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.407+0000 7f9459f0e640 1 -- 172.21.15.12:0/2514971782 wait complete. 2023-10-30T12:56:11.430 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:34:53.017612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-30T12:35:03.216053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-30T12:35:04.597162+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 505, 'snap_epoch': 46, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:56:11.431 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:56:11.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.575+0000 7fa0198d6640 1 Processor -- start 2023-10-30T12:56:11.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.575+0000 7fa0198d6640 1 -- start start 2023-10-30T12:56:11.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.575+0000 7fa0198d6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa014159930 0x7fa0140cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.575+0000 7fa0198d6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa0140cf6e0 0x7fa01415d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.575+0000 7fa0198d6640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01415a920 con 0x7fa014153980 2023-10-30T12:56:11.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.575+0000 7fa0198d6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0140cb930 con 0x7fa0140cf6e0 2023-10-30T12:56:11.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.575+0000 7fa0198d6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa014159dc0 con 0x7fa014159930 2023-10-30T12:56:11.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.575+0000 7fa0137fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa0140cf6e0 0x7fa01415d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.575+0000 7fa0137fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa0140cf6e0 0x7fa01415d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52100/0 (socket says 172.21.15.12:52100) 2023-10-30T12:56:11.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.575+0000 7fa0137fe640 1 -- 172.21.15.12:0/590815109 learned_addr learned my addr 172.21.15.12:0/590815109 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:11.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.575+0000 7fa00bfff640 1 --2- 172.21.15.12:0/590815109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa014159930 0x7fa0140cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.575+0000 7fa0137fe640 1 -- 172.21.15.12:0/590815109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa014159930 msgr2=0x7fa0140cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:11.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.575+0000 7fa0137fe640 1 --2- 172.21.15.12:0/590815109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa014159930 0x7fa0140cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.575+0000 7fa0137fe640 1 -- 172.21.15.12:0/590815109 >> v1:172.21.15.154:6789/0 conn(0x7fa014153980 legacy=0x7fa0140caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.575+0000 7fa0137fe640 1 -- 172.21.15.12:0/590815109 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01415a080 con 0x7fa0140cf6e0 2023-10-30T12:56:11.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.579+0000 7fa0137fe640 1 --2- 172.21.15.12:0/590815109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0140cf6e0 0x7fa01415d570 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009e50 tx=0x7fa00c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a63d8d5f8d3f5a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:11.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.579+0000 7fa0127fc640 1 -- 172.21.15.12:0/590815109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0a57c0 con 0x7fa0140cf6e0 2023-10-30T12:56:11.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.579+0000 7fa0127fc640 1 -- 172.21.15.12:0/590815109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c0a5920 con 0x7fa0140cf6e0 2023-10-30T12:56:11.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.579+0000 7fa0127fc640 1 -- 172.21.15.12:0/590815109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0a5cb0 con 0x7fa0140cf6e0 2023-10-30T12:56:11.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.579+0000 7fa0198d6640 1 -- 172.21.15.12:0/590815109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0140cf6e0 msgr2=0x7fa01415d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.579+0000 7fa0198d6640 1 --2- 172.21.15.12:0/590815109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0140cf6e0 0x7fa01415d570 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009e50 tx=0x7fa00c01c9e0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.579+0000 7fa0198d6640 1 -- 172.21.15.12:0/590815109 shutdown_connections 2023-10-30T12:56:11.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.579+0000 7fa0198d6640 1 --2- 172.21.15.12:0/590815109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0140cf6e0 0x7fa01415d570 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.579+0000 7fa0198d6640 1 --2- 172.21.15.12:0/590815109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa014159930 0x7fa0140cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.579+0000 7fa0198d6640 1 -- 172.21.15.12:0/590815109 >> 172.21.15.12:0/590815109 conn(0x7fa01406c0f0 msgr2=0x7fa01406c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:11.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.579+0000 7fa0198d6640 1 -- 172.21.15.12:0/590815109 shutdown_connections 2023-10-30T12:56:11.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.579+0000 7fa0198d6640 1 -- 172.21.15.12:0/590815109 wait complete. 2023-10-30T12:56:11.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0198d6640 1 Processor -- start 2023-10-30T12:56:11.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0198d6640 1 -- start start 2023-10-30T12:56:11.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0198d6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0140cf6e0 0x7fa01415c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0198d6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa014153980 0x7fa01415c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0198d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01415aaf0 0x7fa014148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0198d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140cee60 con 0x7fa01415aaf0 2023-10-30T12:56:11.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0198d6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa014145b20 con 0x7fa0140cf6e0 2023-10-30T12:56:11.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0198d6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0141477f0 con 0x7fa014153980 2023-10-30T12:56:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01415aaf0 0x7fa014148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01415aaf0 0x7fa014148240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52108/0 (socket says 172.21.15.12:52108) 2023-10-30T12:56:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0137fe640 1 -- 172.21.15.12:0/510210885 learned_addr learned my addr 172.21.15.12:0/510210885 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa00bfff640 1 --2- 172.21.15.12:0/510210885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa014153980 0x7fa01415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa012ffd640 1 --2- 172.21.15.12:0/510210885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0140cf6e0 0x7fa01415c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0137fe640 1 -- 172.21.15.12:0/510210885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa014153980 msgr2=0x7fa01415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0137fe640 1 --2- 172.21.15.12:0/510210885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa014153980 0x7fa01415c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0137fe640 1 -- 172.21.15.12:0/510210885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0140cf6e0 msgr2=0x7fa01415c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0137fe640 1 --2- 172.21.15.12:0/510210885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0140cf6e0 0x7fa01415c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0137fe640 1 -- 172.21.15.12:0/510210885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c009b30 con 0x7fa01415aaf0 2023-10-30T12:56:11.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa012ffd640 1 --2- 172.21.15.12:0/510210885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0140cf6e0 0x7fa01415c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:11.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0137fe640 1 --2- 172.21.15.12:0/510210885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01415aaf0 0x7fa014148240 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009e50 tx=0x7fa00c019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:11.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa010ff9640 1 -- 172.21.15.12:0/510210885 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0bd070 con 0x7fa01415aaf0 2023-10-30T12:56:11.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa010ff9640 1 -- 172.21.15.12:0/510210885 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c009510 con 0x7fa01415aaf0 2023-10-30T12:56:11.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa010ff9640 1 -- 172.21.15.12:0/510210885 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0b86b0 con 0x7fa01415aaf0 2023-10-30T12:56:11.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0198d6640 1 -- 172.21.15.12:0/510210885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa014147db0 con 0x7fa01415aaf0 2023-10-30T12:56:11.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.583+0000 7fa0198d6640 1 -- 172.21.15.12:0/510210885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa014152000 con 0x7fa01415aaf0 2023-10-30T12:56:11.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.587+0000 7fa0198d6640 1 -- 172.21.15.12:0/510210885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa014147f40 con 0x7fa01415aaf0 2023-10-30T12:56:11.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.587+0000 7fa010ff9640 1 -- 172.21.15.12:0/510210885 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa00c0b8810 con 0x7fa01415aaf0 2023-10-30T12:56:11.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.587+0000 7fa010ff9640 1 --2- 172.21.15.12:0/510210885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9ff40664d0 0x7f9ff4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.587+0000 7fa010ff9640 1 -- 172.21.15.12:0/510210885 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fa00c11f420 con 0x7fa01415aaf0 2023-10-30T12:56:11.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.587+0000 7fa012ffd640 1 --2- 172.21.15.12:0/510210885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9ff40664d0 0x7f9ff4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.591+0000 7fa012ffd640 1 --2- 172.21.15.12:0/510210885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9ff40664d0 0x7f9ff4068990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa004000f70 tx=0x7fa004009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:11.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.591+0000 7fa010ff9640 1 -- 172.21.15.12:0/510210885 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa00c0ec550 con 0x7fa01415aaf0 2023-10-30T12:56:11.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.751+0000 7fa0198d6640 1 -- 172.21.15.12:0/510210885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa01404e170 con 0x7fa01415aaf0 2023-10-30T12:56:11.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.751+0000 7fa010ff9640 1 -- 172.21.15.12:0/510210885 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa00c0efb70 con 0x7fa01415aaf0 2023-10-30T12:56:11.756 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-10-30T12:56:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.751+0000 7fa0198d6640 1 -- 172.21.15.12:0/510210885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9ff40664d0 msgr2=0x7f9ff4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.751+0000 7fa0198d6640 1 --2- 172.21.15.12:0/510210885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9ff40664d0 0x7f9ff4068990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa004000f70 tx=0x7fa004009040 comp rx=0 tx=0).stop 2023-10-30T12:56:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.751+0000 7fa0198d6640 1 -- 172.21.15.12:0/510210885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01415aaf0 msgr2=0x7fa014148240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.751+0000 7fa0198d6640 1 --2- 172.21.15.12:0/510210885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01415aaf0 0x7fa014148240 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009e50 tx=0x7fa00c019780 comp rx=0 tx=0).stop 2023-10-30T12:56:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.751+0000 7fa0198d6640 1 -- 172.21.15.12:0/510210885 shutdown_connections 2023-10-30T12:56:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.751+0000 7fa0198d6640 1 --2- 172.21.15.12:0/510210885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01415aaf0 0x7fa014148240 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.751+0000 7fa0198d6640 1 --2- 172.21.15.12:0/510210885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f9ff40664d0 0x7f9ff4068990 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.751+0000 7fa0198d6640 1 --2- 172.21.15.12:0/510210885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa014153980 0x7fa01415c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.751+0000 7fa0198d6640 1 --2- 172.21.15.12:0/510210885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0140cf6e0 0x7fa01415c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.751+0000 7fa0198d6640 1 -- 172.21.15.12:0/510210885 >> 172.21.15.12:0/510210885 conn(0x7fa01406c0f0 msgr2=0x7fa014158470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:11.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.751+0000 7fa0198d6640 1 -- 172.21.15.12:0/510210885 shutdown_connections 2023-10-30T12:56:11.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.751+0000 7fa0198d6640 1 -- 172.21.15.12:0/510210885 wait complete. 2023-10-30T12:56:11.777 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-30T12:56:11.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.919+0000 7fb7c7d80640 1 Processor -- start 2023-10-30T12:56:11.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.919+0000 7fb7c7d80640 1 -- start start 2023-10-30T12:56:11.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.919+0000 7fb7c7d80640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7c006bc50 0x7fb7c00be770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.919+0000 7fb7c7d80640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb7c015cd10 0x7fb7c015d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.919+0000 7fb7c7d80640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c00be140 con 0x7fb7c0153c90 2023-10-30T12:56:11.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.919+0000 7fb7c7d80640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c0070850 con 0x7fb7c015cd10 2023-10-30T12:56:11.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.919+0000 7fb7c7d80640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c006e8c0 con 0x7fb7c006bc50 2023-10-30T12:56:11.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.923+0000 7fb7c631d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb7c015cd10 0x7fb7c015d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.923+0000 7fb7c631d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb7c015cd10 0x7fb7c015d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:50102/0 (socket says 172.21.15.12:50102) 2023-10-30T12:56:11.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.923+0000 7fb7c631d640 1 -- 172.21.15.12:0/422109890 learned_addr learned my addr 172.21.15.12:0/422109890 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:11.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.923+0000 7fb7c531b640 1 --2- 172.21.15.12:0/422109890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7c006bc50 0x7fb7c00be770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.923+0000 7fb7c4b1a640 1 -- 172.21.15.12:0/422109890 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549060712 0 0) 0x7fb7c00be140 con 0x7fb7c0153c90 2023-10-30T12:56:11.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.923+0000 7fb7c4b1a640 1 -- 172.21.15.12:0/422109890 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7ac003530 con 0x7fb7c0153c90 2023-10-30T12:56:11.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.923+0000 7fb7c631d640 1 -- 172.21.15.12:0/422109890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7c006bc50 msgr2=0x7fb7c00be770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:11.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.923+0000 7fb7c631d640 1 --2- 172.21.15.12:0/422109890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7c006bc50 0x7fb7c00be770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.923+0000 7fb7c631d640 1 -- 172.21.15.12:0/422109890 >> v1:172.21.15.154:6789/0 conn(0x7fb7c0153c90 legacy=0x7fb7c006b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.923+0000 7fb7c631d640 1 -- 172.21.15.12:0/422109890 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c0070a40 con 0x7fb7c015cd10 2023-10-30T12:56:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.927+0000 7fb7c631d640 1 --2- 172.21.15.12:0/422109890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7c015cd10 0x7fb7c015d180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009e80 tx=0x7fb7b801f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da77b946ea6e541b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.927+0000 7fb7c4b1a640 1 -- 172.21.15.12:0/422109890 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80af400 con 0x7fb7c015cd10 2023-10-30T12:56:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.927+0000 7fb7c4b1a640 1 -- 172.21.15.12:0/422109890 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b80ab050 con 0x7fb7c015cd10 2023-10-30T12:56:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.927+0000 7fb7c4b1a640 1 -- 172.21.15.12:0/422109890 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80af780 con 0x7fb7c015cd10 2023-10-30T12:56:11.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.927+0000 7fb7c7d80640 1 -- 172.21.15.12:0/422109890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7c015cd10 msgr2=0x7fb7c015d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.927+0000 7fb7c7d80640 1 --2- 172.21.15.12:0/422109890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7c015cd10 0x7fb7c015d180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009e80 tx=0x7fb7b801f1e0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.927+0000 7fb7c7d80640 1 -- 172.21.15.12:0/422109890 shutdown_connections 2023-10-30T12:56:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.927+0000 7fb7c7d80640 1 --2- 172.21.15.12:0/422109890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7c015cd10 0x7fb7c015d180 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.927+0000 7fb7c7d80640 1 --2- 172.21.15.12:0/422109890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7c006bc50 0x7fb7c00be770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.927+0000 7fb7c7d80640 1 -- 172.21.15.12:0/422109890 >> 172.21.15.12:0/422109890 conn(0x7fb7c00c7fe0 msgr2=0x7fb7c00c83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:11.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.927+0000 7fb7c7d80640 1 -- 172.21.15.12:0/422109890 shutdown_connections 2023-10-30T12:56:11.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.927+0000 7fb7c7d80640 1 -- 172.21.15.12:0/422109890 wait complete. 2023-10-30T12:56:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.927+0000 7fb7c7d80640 1 Processor -- start 2023-10-30T12:56:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.927+0000 7fb7c7d80640 1 -- start start 2023-10-30T12:56:11.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c7d80640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7c006bc50 0x7fb7c0155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c7d80640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7c0153c90 0x7fb7c00be170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c7d80640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7c015cd10 0x7fb7c01ee950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c7d80640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c00c1240 con 0x7fb7c0153c90 2023-10-30T12:56:11.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c7d80640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c00ca6a0 con 0x7fb7c006bc50 2023-10-30T12:56:11.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c7d80640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c01f1e60 con 0x7fb7c015cd10 2023-10-30T12:56:11.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c531b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7c0153c90 0x7fb7c00be170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c531b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7c0153c90 0x7fb7c00be170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52118/0 (socket says 172.21.15.12:52118) 2023-10-30T12:56:11.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c531b640 1 -- 172.21.15.12:0/1963058100 learned_addr learned my addr 172.21.15.12:0/1963058100 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:11.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c5b1c640 1 --2- 172.21.15.12:0/1963058100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7c006bc50 0x7fb7c0155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c631d640 1 --2- 172.21.15.12:0/1963058100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7c015cd10 0x7fb7c01ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c531b640 1 -- 172.21.15.12:0/1963058100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7c015cd10 msgr2=0x7fb7c01ee950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c531b640 1 --2- 172.21.15.12:0/1963058100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7c015cd10 0x7fb7c01ee950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c531b640 1 -- 172.21.15.12:0/1963058100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7c006bc50 msgr2=0x7fb7c0155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:11.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c531b640 1 --2- 172.21.15.12:0/1963058100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7c006bc50 0x7fb7c0155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:11.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c531b640 1 -- 172.21.15.12:0/1963058100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b8009b30 con 0x7fb7c0153c90 2023-10-30T12:56:11.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c531b640 1 --2- 172.21.15.12:0/1963058100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7c0153c90 0x7fb7c00be170 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e870 tx=0x7fb7b000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:11.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7b67fc640 1 -- 172.21.15.12:0/1963058100 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0009510 con 0x7fb7c0153c90 2023-10-30T12:56:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7b67fc640 1 -- 172.21.15.12:0/1963058100 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b000eec0 con 0x7fb7c0153c90 2023-10-30T12:56:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7b67fc640 1 -- 172.21.15.12:0/1963058100 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000cb40 con 0x7fb7c0153c90 2023-10-30T12:56:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c7d80640 1 -- 172.21.15.12:0/1963058100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c01f2140 con 0x7fb7c0153c90 2023-10-30T12:56:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c7d80640 1 -- 172.21.15.12:0/1963058100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c01f2700 con 0x7fb7c0153c90 2023-10-30T12:56:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.931+0000 7fb7c7d80640 1 -- 172.21.15.12:0/1963058100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c004e170 con 0x7fb7c0153c90 2023-10-30T12:56:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.935+0000 7fb7b67fc640 1 -- 172.21.15.12:0/1963058100 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb7b000cca0 con 0x7fb7c0153c90 2023-10-30T12:56:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.935+0000 7fb7b67fc640 1 --2- 172.21.15.12:0/1963058100 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7a40664a0 0x7fb7a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.935+0000 7fb7b67fc640 1 -- 172.21.15.12:0/1963058100 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fb7b0084470 con 0x7fb7c0153c90 2023-10-30T12:56:11.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.935+0000 7fb7c5b1c640 1 --2- 172.21.15.12:0/1963058100 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7a40664a0 0x7fb7a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:11.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.935+0000 7fb7c5b1c640 1 --2- 172.21.15.12:0/1963058100 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7a40664a0 0x7fb7a4068960 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc007fd0 tx=0x7fb7bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:11.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:11.939+0000 7fb7b67fc640 1 -- 172.21.15.12:0/1963058100 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb7b0051480 con 0x7fb7c0153c90 2023-10-30T12:56:12.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.095+0000 7fb7c7d80640 1 -- 172.21.15.12:0/1963058100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7c00660f0 con 0x7fb7c0153c90 2023-10-30T12:56:12.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.095+0000 7fb7b67fc640 1 -- 172.21.15.12:0/1963058100 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7b0054aa0 con 0x7fb7c0153c90 2023-10-30T12:56:12.100 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-10-30T12:56:12.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.095+0000 7fb7c7d80640 1 -- 172.21.15.12:0/1963058100 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7a40664a0 msgr2=0x7fb7a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.095+0000 7fb7c7d80640 1 --2- 172.21.15.12:0/1963058100 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7a40664a0 0x7fb7a4068960 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc007fd0 tx=0x7fb7bc009040 comp rx=0 tx=0).stop 2023-10-30T12:56:12.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.095+0000 7fb7c7d80640 1 -- 172.21.15.12:0/1963058100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7c0153c90 msgr2=0x7fb7c00be170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.095+0000 7fb7c7d80640 1 --2- 172.21.15.12:0/1963058100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7c0153c90 0x7fb7c00be170 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e870 tx=0x7fb7b000ec70 comp rx=0 tx=0).stop 2023-10-30T12:56:12.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.095+0000 7fb7c7d80640 1 -- 172.21.15.12:0/1963058100 shutdown_connections 2023-10-30T12:56:12.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.095+0000 7fb7c7d80640 1 --2- 172.21.15.12:0/1963058100 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb7a40664a0 0x7fb7a4068960 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.095+0000 7fb7c7d80640 1 --2- 172.21.15.12:0/1963058100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb7c015cd10 0x7fb7c01ee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.095+0000 7fb7c7d80640 1 --2- 172.21.15.12:0/1963058100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7c0153c90 0x7fb7c00be170 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.095+0000 7fb7c7d80640 1 --2- 172.21.15.12:0/1963058100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7c006bc50 0x7fb7c0155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.095+0000 7fb7c7d80640 1 -- 172.21.15.12:0/1963058100 >> 172.21.15.12:0/1963058100 conn(0x7fb7c00c7fe0 msgr2=0x7fb7c00c67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.095+0000 7fb7c7d80640 1 -- 172.21.15.12:0/1963058100 shutdown_connections 2023-10-30T12:56:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.099+0000 7fb7c7d80640 1 -- 172.21.15.12:0/1963058100 wait complete. 2023-10-30T12:56:12.120 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-30T12:56:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12fa9c640 1 Processor -- start 2023-10-30T12:56:12.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12fa9c640 1 -- start start 2023-10-30T12:56:12.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12fa9c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1281508c0 0x7fa1280c6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:12.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12fa9c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1280c0a30 con 0x7fa1280c6e60 2023-10-30T12:56:12.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12fa9c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa12806f650 con 0x7fa1280722a0 2023-10-30T12:56:12.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12fa9c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280c85b0 con 0x7fa1281508c0 2023-10-30T12:56:12.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12e039640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa1280c6e60 0x7fa128156930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44364/0 (socket says 172.21.15.12:44364) 2023-10-30T12:56:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12e039640 1 -- 172.21.15.12:0/955800021 learned_addr learned my addr 172.21.15.12:0/955800021 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12d838640 1 --2- 172.21.15.12:0/955800021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1281508c0 0x7fa1280c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12c836640 1 -- 172.21.15.12:0/955800021 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93620989 0 0) 0x7fa1280c0a30 con 0x7fa1280c6e60 2023-10-30T12:56:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12c836640 1 -- 172.21.15.12:0/955800021 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa114003530 con 0x7fa1280c6e60 2023-10-30T12:56:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12c836640 1 -- 172.21.15.12:0/955800021 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3489428379 0 0) 0x7fa114003530 con 0x7fa1280c6e60 2023-10-30T12:56:12.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12c836640 1 -- 172.21.15.12:0/955800021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1281508c0 msgr2=0x7fa1280c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:12.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12c836640 1 --2- 172.21.15.12:0/955800021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1281508c0 0x7fa1280c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12c836640 1 -- 172.21.15.12:0/955800021 >> v1:172.21.15.154:6789/0 conn(0x7fa1280722a0 legacy=0x7fa1280bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12c836640 1 -- 172.21.15.12:0/955800021 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12806f930 con 0x7fa1280c6e60 2023-10-30T12:56:12.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12c836640 1 -- 172.21.15.12:0/955800021 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fa120001a00 con 0x7fa1280c6e60 2023-10-30T12:56:12.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12c836640 1 -- 172.21.15.12:0/955800021 >> v1:172.21.15.12:6789/0 conn(0x7fa1280c6e60 legacy=0x7fa128156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12c836640 1 --2- 172.21.15.12:0/955800021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1281508c0 0x7fa1140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:12.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12c836640 1 --2- 172.21.15.12:0/955800021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa114003ef0 0x7fa1140047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:12.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12c836640 1 --2- 172.21.15.12:0/955800021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa114004d10 0x7fa114005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:12.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12c836640 1 -- 172.21.15.12:0/955800021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280c85b0 con 0x7fa114003ef0 2023-10-30T12:56:12.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12c836640 1 -- 172.21.15.12:0/955800021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11400b9f0 con 0x7fa114004d10 2023-10-30T12:56:12.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12c836640 1 -- 172.21.15.12:0/955800021 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11400bbb0 con 0x7fa1281508c0 2023-10-30T12:56:12.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12d037640 1 --2- 172.21.15.12:0/955800021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa114003ef0 0x7fa1140047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:12.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12d838640 1 --2- 172.21.15.12:0/955800021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1281508c0 0x7fa1140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:12.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12e039640 1 --2- 172.21.15.12:0/955800021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa114004d10 0x7fa114005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:12.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12d037640 1 -- 172.21.15.12:0/955800021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1281508c0 msgr2=0x7fa1140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:12.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12d037640 1 --2- 172.21.15.12:0/955800021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1281508c0 0x7fa1140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12d037640 1 -- 172.21.15.12:0/955800021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa114004d10 msgr2=0x7fa114005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:12.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12d037640 1 --2- 172.21.15.12:0/955800021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa114004d10 0x7fa114005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.263+0000 7fa12d037640 1 -- 172.21.15.12:0/955800021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa114003530 con 0x7fa114003ef0 2023-10-30T12:56:12.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.267+0000 7fa12d037640 1 --2- 172.21.15.12:0/955800021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa114003ef0 0x7fa1140047d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa1280726c0 tx=0x7fa11801e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc72c79db55762c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:12.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.267+0000 7fa12c836640 1 -- 172.21.15.12:0/955800021 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1180ac5f0 con 0x7fa114003ef0 2023-10-30T12:56:12.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.267+0000 7fa12c836640 1 -- 172.21.15.12:0/955800021 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1180ac770 con 0x7fa114003ef0 2023-10-30T12:56:12.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.267+0000 7fa12c836640 1 -- 172.21.15.12:0/955800021 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1180acb20 con 0x7fa114003ef0 2023-10-30T12:56:12.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.267+0000 7fa12fa9c640 1 -- 172.21.15.12:0/955800021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa114003ef0 msgr2=0x7fa1140047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.267+0000 7fa12fa9c640 1 --2- 172.21.15.12:0/955800021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa114003ef0 0x7fa1140047d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa1280726c0 tx=0x7fa11801e670 comp rx=0 tx=0).stop 2023-10-30T12:56:12.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.267+0000 7fa12e039640 1 -- 172.21.15.12:0/955800021 reap_dead start 2023-10-30T12:56:12.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.267+0000 7fa12fa9c640 1 -- 172.21.15.12:0/955800021 shutdown_connections 2023-10-30T12:56:12.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.267+0000 7fa12fa9c640 1 -- 172.21.15.12:0/955800021 >> 172.21.15.12:0/955800021 conn(0x7fa1280c7910 msgr2=0x7fa1280c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:12.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.267+0000 7fa12fa9c640 1 -- 172.21.15.12:0/955800021 shutdown_connections 2023-10-30T12:56:12.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.267+0000 7fa12fa9c640 1 -- 172.21.15.12:0/955800021 wait complete. 2023-10-30T12:56:12.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12fa9c640 1 Processor -- start 2023-10-30T12:56:12.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12fa9c640 1 -- start start 2023-10-30T12:56:12.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12fa9c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa114003ef0 0x7fa128155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:12.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12fa9c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1281474e0 0x7fa1280ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:12.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12fa9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281f1ec0 0x7fa1281424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:12.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12fa9c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa128147fa0 con 0x7fa1281f1ec0 2023-10-30T12:56:12.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12fa9c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12814ad80 con 0x7fa1281474e0 2023-10-30T12:56:12.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12fa9c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12814b5e0 con 0x7fa114003ef0 2023-10-30T12:56:12.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12e039640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281f1ec0 0x7fa1281424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:12.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12e039640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281f1ec0 0x7fa1281424c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52142/0 (socket says 172.21.15.12:52142) 2023-10-30T12:56:12.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12e039640 1 -- 172.21.15.12:0/2766157783 learned_addr learned my addr 172.21.15.12:0/2766157783 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:12.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12d838640 1 --2- 172.21.15.12:0/2766157783 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa114003ef0 0x7fa128155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:12.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12d037640 1 --2- 172.21.15.12:0/2766157783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1281474e0 0x7fa1280ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:12.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12e039640 1 -- 172.21.15.12:0/2766157783 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa114003ef0 msgr2=0x7fa128155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12e039640 1 --2- 172.21.15.12:0/2766157783 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa114003ef0 0x7fa128155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12e039640 1 -- 172.21.15.12:0/2766157783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1281474e0 msgr2=0x7fa1280ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12e039640 1 --2- 172.21.15.12:0/2766157783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1281474e0 0x7fa1280ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12e039640 1 -- 172.21.15.12:0/2766157783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa128155ea0 con 0x7fa1281f1ec0 2023-10-30T12:56:12.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12e039640 1 --2- 172.21.15.12:0/2766157783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281f1ec0 0x7fa1281424c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa12806e720 tx=0x7fa12000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa11e7fc640 1 -- 172.21.15.12:0/2766157783 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12000ec40 con 0x7fa1281f1ec0 2023-10-30T12:56:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa11e7fc640 1 -- 172.21.15.12:0/2766157783 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12000eda0 con 0x7fa1281f1ec0 2023-10-30T12:56:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa11e7fc640 1 -- 172.21.15.12:0/2766157783 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120014650 con 0x7fa1281f1ec0 2023-10-30T12:56:12.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12fa9c640 1 -- 172.21.15.12:0/2766157783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa128156110 con 0x7fa1281f1ec0 2023-10-30T12:56:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.271+0000 7fa12fa9c640 1 -- 172.21.15.12:0/2766157783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12814af90 con 0x7fa1281f1ec0 2023-10-30T12:56:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.275+0000 7fa12fa9c640 1 -- 172.21.15.12:0/2766157783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1280b0e40 con 0x7fa1281f1ec0 2023-10-30T12:56:12.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.275+0000 7fa11e7fc640 1 -- 172.21.15.12:0/2766157783 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa1200147b0 con 0x7fa1281f1ec0 2023-10-30T12:56:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.275+0000 7fa11e7fc640 1 --2- 172.21.15.12:0/2766157783 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa10c0664a0 0x7fa10c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.275+0000 7fa11e7fc640 1 -- 172.21.15.12:0/2766157783 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fa120083f70 con 0x7fa1281f1ec0 2023-10-30T12:56:12.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.275+0000 7fa12d838640 1 --2- 172.21.15.12:0/2766157783 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa10c0664a0 0x7fa10c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:12.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.275+0000 7fa12d838640 1 --2- 172.21.15.12:0/2766157783 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa10c0664a0 0x7fa10c068960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa124009ad0 tx=0x7fa124008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:12.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.279+0000 7fa11e7fc640 1 -- 172.21.15.12:0/2766157783 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa120050fa0 con 0x7fa1281f1ec0 2023-10-30T12:56:12.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.435+0000 7fa12fa9c640 1 -- 172.21.15.12:0/2766157783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa12804e170 con 0x7fa1281f1ec0 2023-10-30T12:56:12.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.435+0000 7fa11e7fc640 1 -- 172.21.15.12:0/2766157783 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa1200545c0 con 0x7fa1281f1ec0 2023-10-30T12:56:12.441 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-10-30T12:56:12.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.439+0000 7fa12fa9c640 1 -- 172.21.15.12:0/2766157783 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa10c0664a0 msgr2=0x7fa10c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.439+0000 7fa12fa9c640 1 --2- 172.21.15.12:0/2766157783 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa10c0664a0 0x7fa10c068960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa124009ad0 tx=0x7fa124008040 comp rx=0 tx=0).stop 2023-10-30T12:56:12.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.439+0000 7fa12fa9c640 1 -- 172.21.15.12:0/2766157783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281f1ec0 msgr2=0x7fa1281424c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.439+0000 7fa12fa9c640 1 --2- 172.21.15.12:0/2766157783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281f1ec0 0x7fa1281424c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa12806e720 tx=0x7fa12000e530 comp rx=0 tx=0).stop 2023-10-30T12:56:12.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.439+0000 7fa12fa9c640 1 -- 172.21.15.12:0/2766157783 shutdown_connections 2023-10-30T12:56:12.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.439+0000 7fa12fa9c640 1 --2- 172.21.15.12:0/2766157783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281f1ec0 0x7fa1281424c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.439+0000 7fa12fa9c640 1 --2- 172.21.15.12:0/2766157783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1281474e0 0x7fa1280ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.439+0000 7fa12fa9c640 1 --2- 172.21.15.12:0/2766157783 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fa10c0664a0 0x7fa10c068960 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.439+0000 7fa12fa9c640 1 --2- 172.21.15.12:0/2766157783 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa114003ef0 0x7fa128155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.439+0000 7fa12fa9c640 1 -- 172.21.15.12:0/2766157783 >> 172.21.15.12:0/2766157783 conn(0x7fa1280c7910 msgr2=0x7fa12806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:12.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.439+0000 7fa12fa9c640 1 -- 172.21.15.12:0/2766157783 shutdown_connections 2023-10-30T12:56:12.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.439+0000 7fa12fa9c640 1 -- 172.21.15.12:0/2766157783 wait complete. 2023-10-30T12:56:12.463 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-30T12:56:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.603+0000 7fae2eef7640 1 Processor -- start 2023-10-30T12:56:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.603+0000 7fae2eef7640 1 -- start start 2023-10-30T12:56:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.603+0000 7fae2eef7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fae28151910 0x7fae28158120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.603+0000 7fae2eef7640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2806a4e0 con 0x7fae2806ade0 2023-10-30T12:56:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.603+0000 7fae2eef7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae28158660 con 0x7fae2806a970 2023-10-30T12:56:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.603+0000 7fae2eef7640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae280c6630 con 0x7fae28151910 2023-10-30T12:56:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.607+0000 7fae2d494640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fae2806ade0 0x7fae280c5f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44376/0 (socket says 172.21.15.12:44376) 2023-10-30T12:56:12.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.607+0000 7fae2d494640 1 -- 172.21.15.12:0/3597096644 learned_addr learned my addr 172.21.15.12:0/3597096644 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:12.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.607+0000 7fae2cc93640 1 --2- 172.21.15.12:0/3597096644 >> v2:172.21.15.154:3300/0 conn(0x7fae28151910 0x7fae28158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:12.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.607+0000 7fae277fe640 1 -- 172.21.15.12:0/3597096644 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196625434 0 0) 0x7fae2806a4e0 con 0x7fae2806ade0 2023-10-30T12:56:12.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.607+0000 7fae277fe640 1 -- 172.21.15.12:0/3597096644 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae18003530 con 0x7fae2806ade0 2023-10-30T12:56:12.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.607+0000 7fae277fe640 1 -- 172.21.15.12:0/3597096644 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1799147466 0 0) 0x7fae18003530 con 0x7fae2806ade0 2023-10-30T12:56:12.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.607+0000 7fae2cc93640 1 -- 172.21.15.12:0/3597096644 >> v1:172.21.15.154:6789/0 conn(0x7fae2806a970 legacy=0x7fae280c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.607+0000 7fae2cc93640 1 -- 172.21.15.12:0/3597096644 >> v1:172.21.15.12:6789/0 conn(0x7fae2806ade0 legacy=0x7fae280c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.607+0000 7fae2cc93640 1 -- 172.21.15.12:0/3597096644 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae280c6c50 con 0x7fae28151910 2023-10-30T12:56:12.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2cc93640 1 --2- 172.21.15.12:0/3597096644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae28151910 0x7fae28158120 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0097c0 tx=0x7fae1c01eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aee8fd05c8ea7a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:12.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae277fe640 1 -- 172.21.15.12:0/3597096644 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20001a00 con 0x7fae28151910 2023-10-30T12:56:12.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae277fe640 1 -- 172.21.15.12:0/3597096644 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c0a8610 con 0x7fae28151910 2023-10-30T12:56:12.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae277fe640 1 -- 172.21.15.12:0/3597096644 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0a8980 con 0x7fae28151910 2023-10-30T12:56:12.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2eef7640 1 -- 172.21.15.12:0/3597096644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae28151910 msgr2=0x7fae28158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2eef7640 1 --2- 172.21.15.12:0/3597096644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae28151910 0x7fae28158120 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0097c0 tx=0x7fae1c01eb90 comp rx=0 tx=0).stop 2023-10-30T12:56:12.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2eef7640 1 -- 172.21.15.12:0/3597096644 shutdown_connections 2023-10-30T12:56:12.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2eef7640 1 --2- 172.21.15.12:0/3597096644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae28151910 0x7fae28158120 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2eef7640 1 -- 172.21.15.12:0/3597096644 >> 172.21.15.12:0/3597096644 conn(0x7fae280c7910 msgr2=0x7fae280c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:12.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2eef7640 1 -- 172.21.15.12:0/3597096644 shutdown_connections 2023-10-30T12:56:12.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2eef7640 1 -- 172.21.15.12:0/3597096644 wait complete. 2023-10-30T12:56:12.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2eef7640 1 Processor -- start 2023-10-30T12:56:12.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2eef7640 1 -- start start 2023-10-30T12:56:12.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2eef7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2806a970 0x7fae280aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:12.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2eef7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae28151910 0x7fae28157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:12.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2eef7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae281550e0 0x7fae280c0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:12.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2eef7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae280aff90 con 0x7fae28151910 2023-10-30T12:56:12.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2eef7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae28158480 con 0x7fae281550e0 2023-10-30T12:56:12.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.611+0000 7fae2eef7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae280c0250 con 0x7fae2806a970 2023-10-30T12:56:12.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae27fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae28151910 0x7fae28157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:12.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae27fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae28151910 0x7fae28157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52158/0 (socket says 172.21.15.12:52158) 2023-10-30T12:56:12.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae27fff640 1 -- 172.21.15.12:0/1258221945 learned_addr learned my addr 172.21.15.12:0/1258221945 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:12.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae2d494640 1 --2- 172.21.15.12:0/1258221945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae281550e0 0x7fae280c0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:12.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae2cc93640 1 --2- 172.21.15.12:0/1258221945 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2806a970 0x7fae280aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:12.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae27fff640 1 -- 172.21.15.12:0/1258221945 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2806a970 msgr2=0x7fae280aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae27fff640 1 --2- 172.21.15.12:0/1258221945 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2806a970 0x7fae280aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae27fff640 1 -- 172.21.15.12:0/1258221945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae281550e0 msgr2=0x7fae280c0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae27fff640 1 --2- 172.21.15.12:0/1258221945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae281550e0 0x7fae280c0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae27fff640 1 -- 172.21.15.12:0/1258221945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c009420 con 0x7fae28151910 2023-10-30T12:56:12.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae27fff640 1 --2- 172.21.15.12:0/1258221945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae28151910 0x7fae28157f40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fae1400c530 tx=0x7fae1400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:12.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae257fa640 1 -- 172.21.15.12:0/1258221945 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14005220 con 0x7fae28151910 2023-10-30T12:56:12.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae257fa640 1 -- 172.21.15.12:0/1258221945 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae14005380 con 0x7fae28151910 2023-10-30T12:56:12.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae2eef7640 1 -- 172.21.15.12:0/1258221945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae280af330 con 0x7fae28151910 2023-10-30T12:56:12.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae257fa640 1 -- 172.21.15.12:0/1258221945 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14007270 con 0x7fae28151910 2023-10-30T12:56:12.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae2eef7640 1 -- 172.21.15.12:0/1258221945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae280adf20 con 0x7fae28151910 2023-10-30T12:56:12.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae2eef7640 1 -- 172.21.15.12:0/1258221945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2804e170 con 0x7fae28151910 2023-10-30T12:56:12.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae257fa640 1 -- 172.21.15.12:0/1258221945 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fae140073d0 con 0x7fae28151910 2023-10-30T12:56:12.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.615+0000 7fae257fa640 1 --2- 172.21.15.12:0/1258221945 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fae0c0664d0 0x7fae0c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:12.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.619+0000 7fae257fa640 1 -- 172.21.15.12:0/1258221945 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fae14083ad0 con 0x7fae28151910 2023-10-30T12:56:12.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.619+0000 7fae2cc93640 1 --2- 172.21.15.12:0/1258221945 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fae0c0664d0 0x7fae0c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:12.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.619+0000 7fae2cc93640 1 --2- 172.21.15.12:0/1258221945 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fae0c0664d0 0x7fae0c068990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0b2000 tx=0x7fae1c0aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:12.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.619+0000 7fae257fa640 1 -- 172.21.15.12:0/1258221945 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fae14050b90 con 0x7fae28151910 2023-10-30T12:56:12.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.787+0000 7fae2eef7640 1 -- 172.21.15.12:0/1258221945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fae280b09f0 con 0x7fae28151910 2023-10-30T12:56:12.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.791+0000 7fae257fa640 1 -- 172.21.15.12:0/1258221945 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 20 entries v55) v1 ==== 81+0+1222 (secure 0 0 0) 0x7fae140541b0 con 0x7fae28151910 2023-10-30T12:56:12.794 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.154:6836/2204547612 2023-10-31T12:54:25.460390+0000 2023-10-30T12:56:12.795 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6837/2752088514 2023-10-31T12:53:20.432410+0000 2023-10-30T12:56:12.795 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6834/2752088514 2023-10-31T12:53:20.432410+0000 2023-10-30T12:56:12.795 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.154:6839/2204547612 2023-10-31T12:54:25.460390+0000 2023-10-30T12:56:12.796 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6836/2219276371 2023-10-31T12:50:50.379953+0000 2023-10-30T12:56:12.796 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6833/2219276371 2023-10-31T12:50:50.379953+0000 2023-10-30T12:56:12.796 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6835/559649789 2023-10-31T12:49:35.359754+0000 2023-10-30T12:56:12.796 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6834/3836803994 2023-10-31T12:37:55.139871+0000 2023-10-30T12:56:12.797 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.154:6836/1517672481 2023-10-31T12:43:13.483926+0000 2023-10-30T12:56:12.797 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6832/2676506149 2023-10-31T12:40:40.195783+0000 2023-10-30T12:56:12.797 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.154:6839/1517672481 2023-10-31T12:43:13.483926+0000 2023-10-30T12:56:12.797 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6834/3124135109 2023-10-31T12:46:40.313866+0000 2023-10-30T12:56:12.798 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6837/3836803994 2023-10-31T12:37:55.139871+0000 2023-10-30T12:56:12.798 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6836/291392212 2023-10-31T12:44:35.272569+0000 2023-10-30T12:56:12.798 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6837/3124135109 2023-10-31T12:46:40.313866+0000 2023-10-30T12:56:12.799 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6833/291392212 2023-10-31T12:44:35.272569+0000 2023-10-30T12:56:12.799 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.154:6836/1635719967 2023-10-31T12:47:45.331652+0000 2023-10-30T12:56:12.799 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6835/2676506149 2023-10-31T12:40:40.195783+0000 2023-10-30T12:56:12.799 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.154:6839/1635719967 2023-10-31T12:47:45.331652+0000 2023-10-30T12:56:12.800 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6832/559649789 2023-10-31T12:49:35.359754+0000 2023-10-30T12:56:12.801 INFO:teuthology.orchestra.run.smithi012.stderr:listed 20 entries 2023-10-30T12:56:12.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.791+0000 7fae2eef7640 1 -- 172.21.15.12:0/1258221945 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fae0c0664d0 msgr2=0x7fae0c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.791+0000 7fae2eef7640 1 --2- 172.21.15.12:0/1258221945 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fae0c0664d0 0x7fae0c068990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0b2000 tx=0x7fae1c0aa070 comp rx=0 tx=0).stop 2023-10-30T12:56:12.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.791+0000 7fae2eef7640 1 -- 172.21.15.12:0/1258221945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae28151910 msgr2=0x7fae28157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:12.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.791+0000 7fae2eef7640 1 --2- 172.21.15.12:0/1258221945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae28151910 0x7fae28157f40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fae1400c530 tx=0x7fae1400c9f0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.791+0000 7fae2eef7640 1 -- 172.21.15.12:0/1258221945 shutdown_connections 2023-10-30T12:56:12.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.791+0000 7fae2eef7640 1 --2- 172.21.15.12:0/1258221945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae281550e0 0x7fae280c0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.791+0000 7fae2eef7640 1 --2- 172.21.15.12:0/1258221945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae28151910 0x7fae28157f40 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.791+0000 7fae2eef7640 1 --2- 172.21.15.12:0/1258221945 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fae0c0664d0 0x7fae0c068990 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.791+0000 7fae2eef7640 1 --2- 172.21.15.12:0/1258221945 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2806a970 0x7fae280aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:12.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.791+0000 7fae2eef7640 1 -- 172.21.15.12:0/1258221945 >> 172.21.15.12:0/1258221945 conn(0x7fae280c7910 msgr2=0x7fae280c91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:12.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.791+0000 7fae2eef7640 1 -- 172.21.15.12:0/1258221945 shutdown_connections 2023-10-30T12:56:12.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:12.791+0000 7fae2eef7640 1 -- 172.21.15.12:0/1258221945 wait complete. 2023-10-30T12:56:12.815 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-30T12:56:12.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:56:12.816 DEBUG:teuthology.orchestra.run.smithi012:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-30T12:56:14.417 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2023-10-30T12:56:14.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:56:14.419 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-30T12:56:14.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:56:14.434 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2023-10-30T12:56:14.486 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-30T12:56:14.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:56:14.487 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-30T12:56:14.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:56:14.551 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2023-10-30T12:56:14.760 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-30T12:56:14.769 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-30T12:56:14.770 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:56:14.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.907+0000 7f7ad3e6b640 1 Processor -- start 2023-10-30T12:56:14.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.911+0000 7f7ad3e6b640 1 -- start start 2023-10-30T12:56:14.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.911+0000 7f7ad3e6b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7acc0c6b80 0x7f7acc0bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:14.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.911+0000 7f7ad3e6b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7acc0c6f60 0x7f7acc0c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:14.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.911+0000 7f7ad3e6b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7acc0c0a50 con 0x7f7acc0c8430 2023-10-30T12:56:14.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.911+0000 7f7ad3e6b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7acc06f6e0 con 0x7f7acc0c6b80 2023-10-30T12:56:14.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.911+0000 7f7ad3e6b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7acc06f9c0 con 0x7f7acc0c6f60 2023-10-30T12:56:14.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.911+0000 7f7ad1c07640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7acc0c6b80 0x7f7acc0bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:14.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.911+0000 7f7ad1c07640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7acc0c6b80 0x7f7acc0bfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52172/0 (socket says 172.21.15.12:52172) 2023-10-30T12:56:14.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.911+0000 7f7ad1c07640 1 -- 172.21.15.12:0/1063499418 learned_addr learned my addr 172.21.15.12:0/1063499418 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:14.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.911+0000 7f7ad1406640 1 --2- 172.21.15.12:0/1063499418 >> v2:172.21.15.154:3300/0 conn(0x7f7acc0c6f60 0x7f7acc0c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:14.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.911+0000 7f7ad1c07640 1 -- 172.21.15.12:0/1063499418 >> v2:172.21.15.154:3300/0 conn(0x7f7acc0c6f60 msgr2=0x7f7acc0c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:14.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.911+0000 7f7ad1c07640 1 --2- 172.21.15.12:0/1063499418 >> v2:172.21.15.154:3300/0 conn(0x7f7acc0c6f60 0x7f7acc0c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:14.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.911+0000 7f7ad1c07640 1 -- 172.21.15.12:0/1063499418 >> v1:172.21.15.154:6789/0 conn(0x7f7acc0c8430 legacy=0x7f7acc13ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:14.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.911+0000 7f7ad1c07640 1 -- 172.21.15.12:0/1063499418 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc06fca0 con 0x7f7acc0c6b80 2023-10-30T12:56:14.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.915+0000 7f7ad1c07640 1 --2- 172.21.15.12:0/1063499418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7acc0c6b80 0x7f7acc0bfec0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f7acc159b90 tx=0x7f7ac801e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3923f2e8e9b23043 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:14.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.915+0000 7f7ad0c05640 1 -- 172.21.15.12:0/1063499418 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80aa610 con 0x7f7acc0c6b80 2023-10-30T12:56:14.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.915+0000 7f7ad0c05640 1 -- 172.21.15.12:0/1063499418 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac80aa770 con 0x7f7acc0c6b80 2023-10-30T12:56:14.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.915+0000 7f7ad0c05640 1 -- 172.21.15.12:0/1063499418 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80aab00 con 0x7f7acc0c6b80 2023-10-30T12:56:14.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.915+0000 7f7ad3e6b640 1 -- 172.21.15.12:0/1063499418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7acc0c6b80 msgr2=0x7f7acc0bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:14.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.915+0000 7f7ad3e6b640 1 --2- 172.21.15.12:0/1063499418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7acc0c6b80 0x7f7acc0bfec0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f7acc159b90 tx=0x7f7ac801e6d0 comp rx=0 tx=0).stop 2023-10-30T12:56:14.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.915+0000 7f7ad3e6b640 1 -- 172.21.15.12:0/1063499418 shutdown_connections 2023-10-30T12:56:14.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.915+0000 7f7ad3e6b640 1 --2- 172.21.15.12:0/1063499418 >> v2:172.21.15.154:3300/0 conn(0x7f7acc0c6f60 0x7f7acc0c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.915+0000 7f7ad3e6b640 1 --2- 172.21.15.12:0/1063499418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7acc0c6b80 0x7f7acc0bfec0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.915+0000 7f7ad3e6b640 1 -- 172.21.15.12:0/1063499418 >> 172.21.15.12:0/1063499418 conn(0x7f7acc0c7910 msgr2=0x7f7acc0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.915+0000 7f7ad3e6b640 1 -- 172.21.15.12:0/1063499418 shutdown_connections 2023-10-30T12:56:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.915+0000 7f7ad3e6b640 1 -- 172.21.15.12:0/1063499418 wait complete. 2023-10-30T12:56:14.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.915+0000 7f7ad3e6b640 1 Processor -- start 2023-10-30T12:56:14.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.915+0000 7f7ad3e6b640 1 -- start start 2023-10-30T12:56:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad3e6b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7acc0c6b80 0x7f7acc12f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad3e6b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7acc0c6f60 0x7f7acc12ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad3e6b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7acc0c8430 0x7f7acc1f1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad3e6b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc13b2f0 con 0x7f7acc0c6b80 2023-10-30T12:56:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad3e6b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc13e830 con 0x7f7acc0c8430 2023-10-30T12:56:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad3e6b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc13df90 con 0x7f7acc0c6f60 2023-10-30T12:56:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad1c07640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7acc0c6b80 0x7f7acc12f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad1c07640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7acc0c6b80 0x7f7acc12f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52174/0 (socket says 172.21.15.12:52174) 2023-10-30T12:56:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad1c07640 1 -- 172.21.15.12:0/1266236716 learned_addr learned my addr 172.21.15.12:0/1266236716 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad1406640 1 --2- 172.21.15.12:0/1266236716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7acc0c6f60 0x7f7acc12ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad2408640 1 --2- 172.21.15.12:0/1266236716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7acc0c8430 0x7f7acc1f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad1c07640 1 -- 172.21.15.12:0/1266236716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7acc0c6f60 msgr2=0x7f7acc12ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad1c07640 1 --2- 172.21.15.12:0/1266236716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7acc0c6f60 0x7f7acc12ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad1c07640 1 -- 172.21.15.12:0/1266236716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7acc0c8430 msgr2=0x7f7acc1f1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad1c07640 1 --2- 172.21.15.12:0/1266236716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7acc0c8430 0x7f7acc1f1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad1c07640 1 -- 172.21.15.12:0/1266236716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80094d0 con 0x7f7acc0c6b80 2023-10-30T12:56:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad1c07640 1 --2- 172.21.15.12:0/1266236716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7acc0c6b80 0x7f7acc12f9c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7ac801c300 tx=0x7f7ac801b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ac27fc640 1 -- 172.21.15.12:0/1266236716 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80b38a0 con 0x7f7acc0c6b80 2023-10-30T12:56:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ac27fc640 1 -- 172.21.15.12:0/1266236716 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac80b3a00 con 0x7f7acc0c6b80 2023-10-30T12:56:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ac27fc640 1 -- 172.21.15.12:0/1266236716 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80b3d70 con 0x7f7acc0c6b80 2023-10-30T12:56:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad3e6b640 1 -- 172.21.15.12:0/1266236716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7acc13e270 con 0x7f7acc0c6b80 2023-10-30T12:56:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad3e6b640 1 -- 172.21.15.12:0/1266236716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7acc12f4a0 con 0x7f7acc0c6b80 2023-10-30T12:56:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ad3e6b640 1 -- 172.21.15.12:0/1266236716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7acc12f300 con 0x7f7acc0c6b80 2023-10-30T12:56:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.919+0000 7f7ac27fc640 1 -- 172.21.15.12:0/1266236716 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7ac80a8040 con 0x7f7acc0c6b80 2023-10-30T12:56:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.923+0000 7f7ac27fc640 1 --2- 172.21.15.12:0/1266236716 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7ab00664a0 0x7f7ab0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.923+0000 7f7ac27fc640 1 -- 172.21.15.12:0/1266236716 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f7ac8120610 con 0x7f7acc0c6b80 2023-10-30T12:56:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.923+0000 7f7ad1406640 1 --2- 172.21.15.12:0/1266236716 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7ab00664a0 0x7f7ab0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.923+0000 7f7ad1406640 1 --2- 172.21.15.12:0/1266236716 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7ab00664a0 0x7f7ab0068960 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7acc134680 tx=0x7f7abc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:14.927+0000 7f7ac27fc640 1 -- 172.21.15.12:0/1266236716 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7acc12f300 con 0x7f7acc0c6b80 2023-10-30T12:56:15.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.083+0000 7f7ad3e6b640 1 -- 172.21.15.12:0/1266236716 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7acc12f300 con 0x7f7ab00664a0 2023-10-30T12:56:15.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.107+0000 7f7ac27fc640 1 -- 172.21.15.12:0/1266236716 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292643 (secure 0 0 0) 0x7f7acc12f300 con 0x7f7ab00664a0 2023-10-30T12:56:15.113 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:56:15.114 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-10-30T12:56:15.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.107+0000 7f7ad3e6b640 1 -- 172.21.15.12:0/1266236716 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7ab00664a0 msgr2=0x7f7ab0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:15.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.107+0000 7f7ad3e6b640 1 --2- 172.21.15.12:0/1266236716 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7ab00664a0 0x7f7ab0068960 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7acc134680 tx=0x7f7abc008040 comp rx=0 tx=0).stop 2023-10-30T12:56:15.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.107+0000 7f7ad3e6b640 1 -- 172.21.15.12:0/1266236716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7acc0c6b80 msgr2=0x7f7acc12f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:15.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.107+0000 7f7ad3e6b640 1 --2- 172.21.15.12:0/1266236716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7acc0c6b80 0x7f7acc12f9c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7ac801c300 tx=0x7f7ac801b470 comp rx=0 tx=0).stop 2023-10-30T12:56:15.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.111+0000 7f7ad3e6b640 1 -- 172.21.15.12:0/1266236716 shutdown_connections 2023-10-30T12:56:15.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.111+0000 7f7ad3e6b640 1 --2- 172.21.15.12:0/1266236716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7acc0c8430 0x7f7acc1f1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.111+0000 7f7ad3e6b640 1 --2- 172.21.15.12:0/1266236716 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f7ab00664a0 0x7f7ab0068960 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.111+0000 7f7ad3e6b640 1 --2- 172.21.15.12:0/1266236716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7acc0c6f60 0x7f7acc12ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.111+0000 7f7ad3e6b640 1 --2- 172.21.15.12:0/1266236716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7acc0c6b80 0x7f7acc12f9c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.111+0000 7f7ad3e6b640 1 -- 172.21.15.12:0/1266236716 >> 172.21.15.12:0/1266236716 conn(0x7f7acc0c7910 msgr2=0x7f7acc0c11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:15.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.111+0000 7f7ad3e6b640 1 -- 172.21.15.12:0/1266236716 shutdown_connections 2023-10-30T12:56:15.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.111+0000 7f7ad3e6b640 1 -- 172.21.15.12:0/1266236716 wait complete. 2023-10-30T12:56:15.133 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":689,"stamp":"2023-10-30T12:56:13.408800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1271508492,"num_objects":9804,"num_object_clones":0,"num_object_copies":19608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9804,"num_whiteouts":0,"num_read":5687,"num_read_kb":1040706,"num_write":52424,"num_write_kb":2300570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55711,"ondisk_log_size":55711,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3201824,"kb_used_data":2536096,"kb_used_omap":12,"kb_used_meta":665651,"kb_avail":751772896,"statfs":{"total":773094113280,"available":769815445504,"internally_reserved":0,"allocated":2596962304,"data_stored":2543947272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12729,"internal_metadata":681627207},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":66,"apply_latency_ms":66,"commit_latency_ns":66000000,"apply_latency_ns":66000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":113516058,"num_objects":2524,"num_object_clones":0,"num_object_copies":5048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":111850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004108"},"pg_stats":[{"pgid":"3.3f","version":"55'704","reported_seq":2869,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.052574+0000","last_change":"2023-10-30T12:42:02.472363+0000","last_active":"2023-10-30T12:56:07.052574+0000","last_peered":"2023-10-30T12:56:07.052574+0000","last_clean":"2023-10-30T12:56:07.052574+0000","last_became_active":"2023-10-30T12:35:06.222724+0000","last_became_peered":"2023-10-30T12:35:06.222724+0000","last_unstale":"2023-10-30T12:56:07.052574+0000","last_undegraded":"2023-10-30T12:56:07.052574+0000","last_fullsized":"2023-10-30T12:56:07.052574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:27:39.311531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059524979999999996,"stat_sum":{"num_bytes":14367774,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":73,"num_read_kb":8207,"num_write":683,"num_write_kb":25650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"55'686","reported_seq":2848,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.014947+0000","last_change":"2023-10-30T12:42:02.711331+0000","last_active":"2023-10-30T12:56:07.014947+0000","last_peered":"2023-10-30T12:56:07.014947+0000","last_clean":"2023-10-30T12:56:07.014947+0000","last_became_active":"2023-10-30T12:35:06.182441+0000","last_became_peered":"2023-10-30T12:35:06.182441+0000","last_unstale":"2023-10-30T12:56:07.014947+0000","last_undegraded":"2023-10-30T12:56:07.014947+0000","last_fullsized":"2023-10-30T12:56:07.014947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:00:23.194349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026942979999999999,"stat_sum":{"num_bytes":14474399,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":70,"num_read_kb":10,"num_write":665,"num_write_kb":36839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"55'669","reported_seq":2841,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.004471+0000","last_change":"2023-10-30T12:42:02.724489+0000","last_active":"2023-10-30T12:56:07.004471+0000","last_peered":"2023-10-30T12:56:07.004471+0000","last_clean":"2023-10-30T12:56:07.004471+0000","last_became_active":"2023-10-30T12:35:06.180132+0000","last_became_peered":"2023-10-30T12:35:06.180132+0000","last_unstale":"2023-10-30T12:56:07.004471+0000","last_undegraded":"2023-10-30T12:56:07.004471+0000","last_fullsized":"2023-10-30T12:56:07.004471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:12.922855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003537278,"stat_sum":{"num_bytes":22206444,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":80,"num_read_kb":12305,"num_write":639,"num_write_kb":42559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"55'658","reported_seq":2807,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.150943+0000","last_change":"2023-10-30T12:42:02.742917+0000","last_active":"2023-10-30T12:56:07.150943+0000","last_peered":"2023-10-30T12:56:07.150943+0000","last_clean":"2023-10-30T12:56:07.150943+0000","last_became_active":"2023-10-30T12:35:06.213987+0000","last_became_peered":"2023-10-30T12:35:06.213987+0000","last_unstale":"2023-10-30T12:56:07.150943+0000","last_undegraded":"2023-10-30T12:56:07.150943+0000","last_fullsized":"2023-10-30T12:56:07.150943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:39:55.675219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013528057,"stat_sum":{"num_bytes":18334048,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":57,"num_read_kb":8205,"num_write":618,"num_write_kb":28313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"55'637","reported_seq":2778,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.021805+0000","last_change":"2023-10-30T12:42:02.470142+0000","last_active":"2023-10-30T12:56:07.021805+0000","last_peered":"2023-10-30T12:56:07.021805+0000","last_clean":"2023-10-30T12:56:07.021805+0000","last_became_active":"2023-10-30T12:35:06.214964+0000","last_became_peered":"2023-10-30T12:35:06.214964+0000","last_unstale":"2023-10-30T12:56:07.021805+0000","last_undegraded":"2023-10-30T12:56:07.021805+0000","last_fullsized":"2023-10-30T12:56:07.021805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:56:51.594442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046834600000000004,"stat_sum":{"num_bytes":8551162,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":51,"num_read_kb":4112,"num_write":602,"num_write_kb":25707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"55'662","reported_seq":2813,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.149801+0000","last_change":"2023-10-30T12:42:02.747177+0000","last_active":"2023-10-30T12:56:07.149801+0000","last_peered":"2023-10-30T12:56:07.149801+0000","last_clean":"2023-10-30T12:56:07.149801+0000","last_became_active":"2023-10-30T12:35:06.214219+0000","last_became_peered":"2023-10-30T12:35:06.214219+0000","last_unstale":"2023-10-30T12:56:07.149801+0000","last_undegraded":"2023-10-30T12:56:07.149801+0000","last_fullsized":"2023-10-30T12:56:07.149801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:36:56.070585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015862434000000002,"stat_sum":{"num_bytes":14711969,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":61,"num_read_kb":16,"num_write":631,"num_write_kb":33150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"55'668","reported_seq":2809,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.029181+0000","last_change":"2023-10-30T12:42:02.467959+0000","last_active":"2023-10-30T12:56:07.029181+0000","last_peered":"2023-10-30T12:56:07.029181+0000","last_clean":"2023-10-30T12:56:07.029181+0000","last_became_active":"2023-10-30T12:35:06.216468+0000","last_became_peered":"2023-10-30T12:35:06.216468+0000","last_unstale":"2023-10-30T12:56:07.029181+0000","last_undegraded":"2023-10-30T12:56:07.029181+0000","last_fullsized":"2023-10-30T12:56:07.029181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:19:43.632233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031309480000000002,"stat_sum":{"num_bytes":9417687,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":52,"num_read_kb":12216,"num_write":636,"num_write_kb":35921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"55'605","reported_seq":2751,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.153162+0000","last_change":"2023-10-30T12:42:02.241155+0000","last_active":"2023-10-30T12:56:07.153162+0000","last_peered":"2023-10-30T12:56:07.153162+0000","last_clean":"2023-10-30T12:56:07.153162+0000","last_became_active":"2023-10-30T12:35:06.214384+0000","last_became_peered":"2023-10-30T12:35:06.214384+0000","last_unstale":"2023-10-30T12:56:07.153162+0000","last_undegraded":"2023-10-30T12:56:07.153162+0000","last_fullsized":"2023-10-30T12:56:07.153162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:44:07.845577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045552559999999997,"stat_sum":{"num_bytes":9683176,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":56,"num_read_kb":15,"num_write":587,"num_write_kb":23249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"55'660","reported_seq":2822,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.060712+0000","last_change":"2023-10-30T12:42:02.473886+0000","last_active":"2023-10-30T12:56:07.060712+0000","last_peered":"2023-10-30T12:56:07.060712+0000","last_clean":"2023-10-30T12:56:07.060712+0000","last_became_active":"2023-10-30T12:35:06.222692+0000","last_became_peered":"2023-10-30T12:35:06.222692+0000","last_unstale":"2023-10-30T12:56:07.060712+0000","last_undegraded":"2023-10-30T12:56:07.060712+0000","last_fullsized":"2023-10-30T12:56:07.060712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:12:07.552579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074617599999999996,"stat_sum":{"num_bytes":5453765,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":72,"num_read_kb":4115,"num_write":623,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"55'653","reported_seq":2807,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.014218+0000","last_change":"2023-10-30T12:42:02.739661+0000","last_active":"2023-10-30T12:56:07.014218+0000","last_peered":"2023-10-30T12:56:07.014218+0000","last_clean":"2023-10-30T12:56:07.014218+0000","last_became_active":"2023-10-30T12:35:06.215384+0000","last_became_peered":"2023-10-30T12:35:06.215384+0000","last_unstale":"2023-10-30T12:56:07.014218+0000","last_undegraded":"2023-10-30T12:56:07.014218+0000","last_fullsized":"2023-10-30T12:56:07.014218+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:02:59.748923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080059940000000007,"stat_sum":{"num_bytes":17838264,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":64,"num_read_kb":28,"num_write":624,"num_write_kb":19661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"55'661","reported_seq":2818,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.991403+0000","last_change":"2023-10-30T12:42:02.739462+0000","last_active":"2023-10-30T12:56:06.991403+0000","last_peered":"2023-10-30T12:56:06.991403+0000","last_clean":"2023-10-30T12:56:06.991403+0000","last_became_active":"2023-10-30T12:35:06.215355+0000","last_became_peered":"2023-10-30T12:35:06.215355+0000","last_unstale":"2023-10-30T12:56:06.991403+0000","last_undegraded":"2023-10-30T12:56:06.991403+0000","last_fullsized":"2023-10-30T12:56:06.991403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:04:51.314044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011778769,"stat_sum":{"num_bytes":10379280,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":67,"num_read_kb":8209,"num_write":639,"num_write_kb":25731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"55'640","reported_seq":2795,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.010644+0000","last_change":"2023-10-30T12:42:02.253484+0000","last_active":"2023-10-30T12:56:07.010644+0000","last_peered":"2023-10-30T12:56:07.010644+0000","last_clean":"2023-10-30T12:56:07.010644+0000","last_became_active":"2023-10-30T12:35:06.196503+0000","last_became_peered":"2023-10-30T12:35:06.196503+0000","last_unstale":"2023-10-30T12:56:07.010644+0000","last_undegraded":"2023-10-30T12:56:07.010644+0000","last_fullsized":"2023-10-30T12:56:07.010644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:33:54.826649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030385149999999999,"stat_sum":{"num_bytes":16049079,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":65,"num_read_kb":7648,"num_write":614,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"55'689","reported_seq":2856,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.137866+0000","last_change":"2023-10-30T12:42:02.241621+0000","last_active":"2023-10-30T12:56:07.137866+0000","last_peered":"2023-10-30T12:56:07.137866+0000","last_clean":"2023-10-30T12:56:07.137866+0000","last_became_active":"2023-10-30T12:35:06.195408+0000","last_became_peered":"2023-10-30T12:35:06.195408+0000","last_unstale":"2023-10-30T12:56:07.137866+0000","last_undegraded":"2023-10-30T12:56:07.137866+0000","last_fullsized":"2023-10-30T12:56:07.137866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:06:33.968324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004560812,"stat_sum":{"num_bytes":29038845,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":77,"num_read_kb":9808,"num_write":655,"num_write_kb":48120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"55'657","reported_seq":2819,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.112877+0000","last_change":"2023-10-30T12:42:02.722894+0000","last_active":"2023-10-30T12:56:07.112877+0000","last_peered":"2023-10-30T12:56:07.112877+0000","last_clean":"2023-10-30T12:56:07.112877+0000","last_became_active":"2023-10-30T12:35:06.182573+0000","last_became_peered":"2023-10-30T12:35:06.182573+0000","last_unstale":"2023-10-30T12:56:07.112877+0000","last_undegraded":"2023-10-30T12:56:07.112877+0000","last_fullsized":"2023-10-30T12:56:07.112877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:42:24.313362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003020062,"stat_sum":{"num_bytes":14507519,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":71,"num_read_kb":4113,"num_write":633,"num_write_kb":31547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"55'673","reported_seq":2827,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.046529+0000","last_change":"2023-10-30T12:42:02.710474+0000","last_active":"2023-10-30T12:56:07.046529+0000","last_peered":"2023-10-30T12:56:07.046529+0000","last_clean":"2023-10-30T12:56:07.046529+0000","last_became_active":"2023-10-30T12:35:06.185290+0000","last_became_peered":"2023-10-30T12:35:06.185290+0000","last_unstale":"2023-10-30T12:56:07.046529+0000","last_undegraded":"2023-10-30T12:56:07.046529+0000","last_fullsized":"2023-10-30T12:56:07.046529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:18:23.116561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002282292,"stat_sum":{"num_bytes":14565906,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":63,"num_read_kb":11730,"num_write":637,"num_write_kb":27497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"55'678","reported_seq":2828,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.129521+0000","last_change":"2023-10-30T12:42:02.748073+0000","last_active":"2023-10-30T12:56:07.129521+0000","last_peered":"2023-10-30T12:56:07.129521+0000","last_clean":"2023-10-30T12:56:07.129521+0000","last_became_active":"2023-10-30T12:35:06.213627+0000","last_became_peered":"2023-10-30T12:35:06.213627+0000","last_unstale":"2023-10-30T12:56:07.129521+0000","last_undegraded":"2023-10-30T12:56:07.129521+0000","last_fullsized":"2023-10-30T12:56:07.129521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:56:05.792085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014095760000000001,"stat_sum":{"num_bytes":9785326,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":59,"num_read_kb":4114,"num_write":638,"num_write_kb":31646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"55'710","reported_seq":2863,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.066514+0000","last_change":"2023-10-30T12:42:02.745783+0000","last_active":"2023-10-30T12:56:07.066514+0000","last_peered":"2023-10-30T12:56:07.066514+0000","last_clean":"2023-10-30T12:56:07.066514+0000","last_became_active":"2023-10-30T12:35:06.221696+0000","last_became_peered":"2023-10-30T12:35:06.221696+0000","last_unstale":"2023-10-30T12:56:07.066514+0000","last_undegraded":"2023-10-30T12:56:07.066514+0000","last_fullsized":"2023-10-30T12:56:07.066514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:26:40.837242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015743672,"stat_sum":{"num_bytes":28263392,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":62,"num_read_kb":16401,"num_write":680,"num_write_kb":59327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"55'660","reported_seq":2833,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.992539+0000","last_change":"2023-10-30T12:42:02.242356+0000","last_active":"2023-10-30T12:56:06.992539+0000","last_peered":"2023-10-30T12:56:06.992539+0000","last_clean":"2023-10-30T12:56:06.992539+0000","last_became_active":"2023-10-30T12:35:06.198074+0000","last_became_peered":"2023-10-30T12:35:06.198074+0000","last_unstale":"2023-10-30T12:56:06.992539+0000","last_undegraded":"2023-10-30T12:56:06.992539+0000","last_fullsized":"2023-10-30T12:56:06.992539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:55:38.021093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023389270000000002,"stat_sum":{"num_bytes":28742059,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":82,"num_read_kb":27,"num_write":629,"num_write_kb":39704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"55'675","reported_seq":2826,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.023088+0000","last_change":"2023-10-30T12:42:02.473536+0000","last_active":"2023-10-30T12:56:07.023088+0000","last_peered":"2023-10-30T12:56:07.023088+0000","last_clean":"2023-10-30T12:56:07.023088+0000","last_became_active":"2023-10-30T12:35:06.215935+0000","last_became_peered":"2023-10-30T12:35:06.215935+0000","last_unstale":"2023-10-30T12:56:07.023088+0000","last_undegraded":"2023-10-30T12:56:07.023088+0000","last_fullsized":"2023-10-30T12:56:07.023088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:39:23.851788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062537060000000004,"stat_sum":{"num_bytes":21375724,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":60,"num_read_kb":12305,"num_write":641,"num_write_kb":39315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"55'672","reported_seq":2838,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.992738+0000","last_change":"2023-10-30T12:42:02.739007+0000","last_active":"2023-10-30T12:56:06.992738+0000","last_peered":"2023-10-30T12:56:06.992738+0000","last_clean":"2023-10-30T12:56:06.992738+0000","last_became_active":"2023-10-30T12:35:06.199812+0000","last_became_peered":"2023-10-30T12:35:06.199812+0000","last_unstale":"2023-10-30T12:56:06.992738+0000","last_undegraded":"2023-10-30T12:56:06.992738+0000","last_fullsized":"2023-10-30T12:56:06.992738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:07:11.762372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011375481999999999,"stat_sum":{"num_bytes":6241435,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":75,"num_read_kb":20150,"num_write":638,"num_write_kb":36226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"55'680","reported_seq":2849,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.113501+0000","last_change":"2023-10-30T12:42:02.725299+0000","last_active":"2023-10-30T12:56:07.113501+0000","last_peered":"2023-10-30T12:56:07.113501+0000","last_clean":"2023-10-30T12:56:07.113501+0000","last_became_active":"2023-10-30T12:35:06.216861+0000","last_became_peered":"2023-10-30T12:35:06.216861+0000","last_unstale":"2023-10-30T12:56:07.113501+0000","last_undegraded":"2023-10-30T12:56:07.113501+0000","last_fullsized":"2023-10-30T12:56:07.113501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:58:16.568819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040841849999999997,"stat_sum":{"num_bytes":12253674,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":78,"num_read_kb":8213,"num_write":653,"num_write_kb":26257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"55'705","reported_seq":2854,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.001507+0000","last_change":"2023-10-30T12:42:02.241419+0000","last_active":"2023-10-30T12:56:07.001507+0000","last_peered":"2023-10-30T12:56:07.001507+0000","last_clean":"2023-10-30T12:56:07.001507+0000","last_became_active":"2023-10-30T12:35:46.122517+0000","last_became_peered":"2023-10-30T12:35:46.122517+0000","last_unstale":"2023-10-30T12:56:07.001507+0000","last_undegraded":"2023-10-30T12:56:07.001507+0000","last_fullsized":"2023-10-30T12:56:07.001507+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:10:07.253977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020435240000000001,"stat_sum":{"num_bytes":19343344,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":61,"num_read_kb":21,"num_write":677,"num_write_kb":29921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"55'676","reported_seq":2847,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.095881+0000","last_change":"2023-10-30T12:42:02.241859+0000","last_active":"2023-10-30T12:56:07.095881+0000","last_peered":"2023-10-30T12:56:07.095881+0000","last_clean":"2023-10-30T12:56:07.095881+0000","last_became_active":"2023-10-30T12:35:06.182314+0000","last_became_peered":"2023-10-30T12:35:06.182314+0000","last_unstale":"2023-10-30T12:56:07.095881+0000","last_undegraded":"2023-10-30T12:56:07.095881+0000","last_fullsized":"2023-10-30T12:56:07.095881+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:20:19.153859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051111200000000002,"stat_sum":{"num_bytes":5339400,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":80,"num_read_kb":15696,"num_write":644,"num_write_kb":25369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"55'681","reported_seq":2839,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.120096+0000","last_change":"2023-10-30T12:42:02.748079+0000","last_active":"2023-10-30T12:56:07.120096+0000","last_peered":"2023-10-30T12:56:07.120096+0000","last_clean":"2023-10-30T12:56:07.120096+0000","last_became_active":"2023-10-30T12:35:06.213402+0000","last_became_peered":"2023-10-30T12:35:06.213402+0000","last_unstale":"2023-10-30T12:56:07.120096+0000","last_undegraded":"2023-10-30T12:56:07.120096+0000","last_fullsized":"2023-10-30T12:56:07.120096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:11.732115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014055767,"stat_sum":{"num_bytes":10176680,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":67,"num_read_kb":4112,"num_write":643,"num_write_kb":20459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"55'660","reported_seq":2810,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.118693+0000","last_change":"2023-10-30T12:42:02.242578+0000","last_active":"2023-10-30T12:56:07.118693+0000","last_peered":"2023-10-30T12:56:07.118693+0000","last_clean":"2023-10-30T12:56:07.118693+0000","last_became_active":"2023-10-30T12:35:06.188298+0000","last_became_peered":"2023-10-30T12:35:06.188298+0000","last_unstale":"2023-10-30T12:56:07.118693+0000","last_undegraded":"2023-10-30T12:56:07.118693+0000","last_fullsized":"2023-10-30T12:56:07.118693+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:56:11.006974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053580889999999999,"stat_sum":{"num_bytes":4997964,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":59,"num_read_kb":8204,"num_write":612,"num_write_kb":44089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"55'741","reported_seq":2919,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.030017+0000","last_change":"2023-10-30T12:42:02.740157+0000","last_active":"2023-10-30T12:56:07.030017+0000","last_peered":"2023-10-30T12:56:07.030017+0000","last_clean":"2023-10-30T12:56:07.030017+0000","last_became_active":"2023-10-30T12:35:06.202507+0000","last_became_peered":"2023-10-30T12:35:06.202507+0000","last_unstale":"2023-10-30T12:56:07.030017+0000","last_undegraded":"2023-10-30T12:56:07.030017+0000","last_fullsized":"2023-10-30T12:56:07.030017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:55.774699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082500649999999991,"stat_sum":{"num_bytes":6483256,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":87,"num_read_kb":1892,"num_write":707,"num_write_kb":28689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"55'657","reported_seq":2811,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.123179+0000","last_change":"2023-10-30T12:42:02.710762+0000","last_active":"2023-10-30T12:56:07.123179+0000","last_peered":"2023-10-30T12:56:07.123179+0000","last_clean":"2023-10-30T12:56:07.123179+0000","last_became_active":"2023-10-30T12:35:06.170030+0000","last_became_peered":"2023-10-30T12:35:06.170030+0000","last_unstale":"2023-10-30T12:56:07.123179+0000","last_undegraded":"2023-10-30T12:56:07.123179+0000","last_fullsized":"2023-10-30T12:56:07.123179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:04.189974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024132049999999999,"stat_sum":{"num_bytes":15276436,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":63,"num_read_kb":15,"num_write":628,"num_write_kb":22058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"55'623","reported_seq":2792,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.158356+0000","last_change":"2023-10-30T12:42:02.242975+0000","last_active":"2023-10-30T12:56:07.158356+0000","last_peered":"2023-10-30T12:56:07.158356+0000","last_clean":"2023-10-30T12:56:07.158356+0000","last_became_active":"2023-10-30T12:35:06.189171+0000","last_became_peered":"2023-10-30T12:35:06.189171+0000","last_unstale":"2023-10-30T12:56:07.158356+0000","last_undegraded":"2023-10-30T12:56:07.158356+0000","last_fullsized":"2023-10-30T12:56:07.158356+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:00:28.317415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056904870000000001,"stat_sum":{"num_bytes":13986941,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":78,"num_read_kb":12372,"num_write":595,"num_write_kb":31185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"55'639","reported_seq":2791,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.133710+0000","last_change":"2023-10-30T12:42:02.725392+0000","last_active":"2023-10-30T12:56:07.133710+0000","last_peered":"2023-10-30T12:56:07.133710+0000","last_clean":"2023-10-30T12:56:07.133710+0000","last_became_active":"2023-10-30T12:35:06.183348+0000","last_became_peered":"2023-10-30T12:35:06.183348+0000","last_unstale":"2023-10-30T12:56:07.133710+0000","last_undegraded":"2023-10-30T12:56:07.133710+0000","last_fullsized":"2023-10-30T12:56:07.133710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:15:11.322585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041317979999999999,"stat_sum":{"num_bytes":14536653,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":60,"num_read_kb":7678,"num_write":614,"num_write_kb":36281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"55'712","reported_seq":2879,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.104341+0000","last_change":"2023-10-30T12:42:02.747343+0000","last_active":"2023-10-30T12:56:07.104341+0000","last_peered":"2023-10-30T12:56:07.104341+0000","last_clean":"2023-10-30T12:56:07.104341+0000","last_became_active":"2023-10-30T12:35:06.196368+0000","last_became_peered":"2023-10-30T12:35:06.196368+0000","last_unstale":"2023-10-30T12:56:07.104341+0000","last_undegraded":"2023-10-30T12:56:07.104341+0000","last_fullsized":"2023-10-30T12:56:07.104341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:53:23.269916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015569382999999999,"stat_sum":{"num_bytes":14455388,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":75,"num_read_kb":9352,"num_write":686,"num_write_kb":32708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"55'735","reported_seq":2899,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.035413+0000","last_change":"2023-10-30T12:42:02.241047+0000","last_active":"2023-10-30T12:56:07.035413+0000","last_peered":"2023-10-30T12:56:07.035413+0000","last_clean":"2023-10-30T12:56:07.035413+0000","last_became_active":"2023-10-30T12:35:06.213994+0000","last_became_peered":"2023-10-30T12:35:06.213994+0000","last_unstale":"2023-10-30T12:56:07.035413+0000","last_undegraded":"2023-10-30T12:56:07.035413+0000","last_fullsized":"2023-10-30T12:56:07.035413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:49:43.203436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001873502,"stat_sum":{"num_bytes":23237117,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":72,"num_read_kb":4118,"num_write":703,"num_write_kb":32259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"55'672","reported_seq":2834,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.181318+0000","last_change":"2023-10-30T12:42:02.738216+0000","last_active":"2023-10-30T12:56:07.181318+0000","last_peered":"2023-10-30T12:56:07.181318+0000","last_clean":"2023-10-30T12:56:07.181318+0000","last_became_active":"2023-10-30T12:35:06.197647+0000","last_became_peered":"2023-10-30T12:35:06.197647+0000","last_unstale":"2023-10-30T12:56:07.181318+0000","last_undegraded":"2023-10-30T12:56:07.181318+0000","last_fullsized":"2023-10-30T12:56:07.181318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:21:00.884446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011378932,"stat_sum":{"num_bytes":18305029,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":70,"num_read_kb":8212,"num_write":638,"num_write_kb":28193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"55'634","reported_seq":2786,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.949834+0000","last_change":"2023-10-30T12:42:02.251618+0000","last_active":"2023-10-30T12:56:06.949834+0000","last_peered":"2023-10-30T12:56:06.949834+0000","last_clean":"2023-10-30T12:56:06.949834+0000","last_became_active":"2023-10-30T12:35:06.167558+0000","last_became_peered":"2023-10-30T12:35:06.167558+0000","last_unstale":"2023-10-30T12:56:06.949834+0000","last_undegraded":"2023-10-30T12:56:06.949834+0000","last_fullsized":"2023-10-30T12:56:06.949834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:11:30.490780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048907120000000002,"stat_sum":{"num_bytes":9666676,"num_objects":132,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":0,"num_read":62,"num_read_kb":17,"num_write":598,"num_write_kb":15613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"55'233","reported_seq":298,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:07.513824+0000","last_change":"2023-10-30T12:35:05.200844+0000","last_active":"2023-10-30T12:55:07.513824+0000","last_peered":"2023-10-30T12:55:07.513824+0000","last_clean":"2023-10-30T12:55:07.513824+0000","last_became_active":"2023-10-30T12:35:05.200169+0000","last_became_peered":"2023-10-30T12:35:05.200169+0000","last_unstale":"2023-10-30T12:55:07.513824+0000","last_undegraded":"2023-10-30T12:55:07.513824+0000","last_fullsized":"2023-10-30T12:55:07.513824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:07:19.571357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":36867,"num_write":183,"num_write_kb":12445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"55'646","reported_seq":2797,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.123456+0000","last_change":"2023-10-30T12:42:02.725641+0000","last_active":"2023-10-30T12:56:07.123456+0000","last_peered":"2023-10-30T12:56:07.123456+0000","last_clean":"2023-10-30T12:56:07.123456+0000","last_became_active":"2023-10-30T12:35:06.194689+0000","last_became_peered":"2023-10-30T12:35:06.194689+0000","last_unstale":"2023-10-30T12:56:07.123456+0000","last_undegraded":"2023-10-30T12:56:07.123456+0000","last_fullsized":"2023-10-30T12:56:07.123456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:38:22.949407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004291034,"stat_sum":{"num_bytes":10877495,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":61,"num_read_kb":18,"num_write":619,"num_write_kb":24664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"55'974","reported_seq":1165,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:34.540942+0000","last_change":"2023-10-30T12:35:05.183569+0000","last_active":"2023-10-30T12:55:34.540942+0000","last_peered":"2023-10-30T12:55:34.540942+0000","last_clean":"2023-10-30T12:55:34.540942+0000","last_became_active":"2023-10-30T12:35:05.182761+0000","last_became_peered":"2023-10-30T12:35:05.182761+0000","last_unstale":"2023-10-30T12:55:34.540942+0000","last_undegraded":"2023-10-30T12:55:34.540942+0000","last_fullsized":"2023-10-30T12:55:34.540942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:49:44.311814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25625406,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":150,"num_read_kb":45229,"num_write":895,"num_write_kb":25503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"55'430","reported_seq":635,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.685247+0000","last_change":"2023-10-30T12:35:05.181278+0000","last_active":"2023-10-30T12:56:09.685247+0000","last_peered":"2023-10-30T12:56:09.685247+0000","last_clean":"2023-10-30T12:56:09.685247+0000","last_became_active":"2023-10-30T12:35:05.180730+0000","last_became_peered":"2023-10-30T12:35:05.180730+0000","last_unstale":"2023-10-30T12:56:09.685247+0000","last_undegraded":"2023-10-30T12:56:09.685247+0000","last_fullsized":"2023-10-30T12:56:09.685247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:07:10.828707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8578635,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":162,"num_read_kb":8759,"num_write":365,"num_write_kb":8612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"55'760","reported_seq":2929,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.115297+0000","last_change":"2023-10-30T12:42:02.240966+0000","last_active":"2023-10-30T12:56:07.115297+0000","last_peered":"2023-10-30T12:56:07.115297+0000","last_clean":"2023-10-30T12:56:07.115297+0000","last_became_active":"2023-10-30T12:35:46.051057+0000","last_became_peered":"2023-10-30T12:35:46.051057+0000","last_unstale":"2023-10-30T12:56:07.115297+0000","last_undegraded":"2023-10-30T12:56:07.115297+0000","last_fullsized":"2023-10-30T12:56:07.115297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:39:59.687314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018351520000000001,"stat_sum":{"num_bytes":18874907,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":82,"num_read_kb":16408,"num_write":717,"num_write_kb":52822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"55'983","reported_seq":1049,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:04.638735+0000","last_change":"2023-10-30T12:35:05.202539+0000","last_active":"2023-10-30T12:55:04.638735+0000","last_peered":"2023-10-30T12:55:04.638735+0000","last_clean":"2023-10-30T12:55:04.638735+0000","last_became_active":"2023-10-30T12:35:05.201853+0000","last_became_peered":"2023-10-30T12:35:05.201853+0000","last_unstale":"2023-10-30T12:55:04.638735+0000","last_undegraded":"2023-10-30T12:55:04.638735+0000","last_fullsized":"2023-10-30T12:55:04.638735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:33:33.151433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":38325,"num_write":914,"num_write_kb":16753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"55'707","reported_seq":2870,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.090289+0000","last_change":"2023-10-30T12:42:02.710378+0000","last_active":"2023-10-30T12:56:07.090289+0000","last_peered":"2023-10-30T12:56:07.090289+0000","last_clean":"2023-10-30T12:56:07.090289+0000","last_became_active":"2023-10-30T12:35:06.170179+0000","last_became_peered":"2023-10-30T12:35:06.170179+0000","last_unstale":"2023-10-30T12:56:07.090289+0000","last_undegraded":"2023-10-30T12:56:07.090289+0000","last_fullsized":"2023-10-30T12:56:07.090289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:21:27.326814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021794980000000002,"stat_sum":{"num_bytes":14241209,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":73,"num_read_kb":12307,"num_write":678,"num_write_kb":28673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"55'145","reported_seq":204,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:54.873264+0000","last_change":"2023-10-30T12:35:05.198546+0000","last_active":"2023-10-30T12:55:54.873264+0000","last_peered":"2023-10-30T12:55:54.873264+0000","last_clean":"2023-10-30T12:55:54.873264+0000","last_became_active":"2023-10-30T12:35:05.197606+0000","last_became_peered":"2023-10-30T12:35:05.197606+0000","last_unstale":"2023-10-30T12:55:54.873264+0000","last_undegraded":"2023-10-30T12:55:54.873264+0000","last_fullsized":"2023-10-30T12:55:54.873264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:26:57.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":90,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"55'666","reported_seq":2833,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.026011+0000","last_change":"2023-10-30T12:42:02.745501+0000","last_active":"2023-10-30T12:56:07.026011+0000","last_peered":"2023-10-30T12:56:07.026011+0000","last_clean":"2023-10-30T12:56:07.026011+0000","last_became_active":"2023-10-30T12:35:06.225392+0000","last_became_peered":"2023-10-30T12:35:06.225392+0000","last_unstale":"2023-10-30T12:56:07.026011+0000","last_undegraded":"2023-10-30T12:56:07.026011+0000","last_fullsized":"2023-10-30T12:56:07.026011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:06:30.011008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01552166,"stat_sum":{"num_bytes":9602206,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":77,"num_read_kb":18,"num_write":639,"num_write_kb":18963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"55'480","reported_seq":541,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.722504+0000","last_change":"2023-10-30T12:35:05.197614+0000","last_active":"2023-10-30T12:56:09.722504+0000","last_peered":"2023-10-30T12:56:09.722504+0000","last_clean":"2023-10-30T12:56:09.722504+0000","last_became_active":"2023-10-30T12:35:05.196899+0000","last_became_peered":"2023-10-30T12:35:05.196899+0000","last_unstale":"2023-10-30T12:56:09.722504+0000","last_undegraded":"2023-10-30T12:56:09.722504+0000","last_fullsized":"2023-10-30T12:56:09.722504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:21:00.152426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14221472,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":14173,"num_write":412,"num_write_kb":14118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"55'672","reported_seq":2825,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.087588+0000","last_change":"2023-10-30T12:42:02.724824+0000","last_active":"2023-10-30T12:56:07.087588+0000","last_peered":"2023-10-30T12:56:07.087588+0000","last_clean":"2023-10-30T12:56:07.087588+0000","last_became_active":"2023-10-30T12:35:06.222057+0000","last_became_peered":"2023-10-30T12:35:06.222057+0000","last_unstale":"2023-10-30T12:56:07.087588+0000","last_undegraded":"2023-10-30T12:56:07.087588+0000","last_fullsized":"2023-10-30T12:56:07.087588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:13:07.261307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004210915,"stat_sum":{"num_bytes":31275308,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":63,"num_read_kb":12308,"num_write":645,"num_write_kb":50366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"55'312","reported_seq":375,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:08.116088+0000","last_change":"2023-10-30T12:35:05.197501+0000","last_active":"2023-10-30T12:56:08.116088+0000","last_peered":"2023-10-30T12:56:08.116088+0000","last_clean":"2023-10-30T12:56:08.116088+0000","last_became_active":"2023-10-30T12:35:05.197190+0000","last_became_peered":"2023-10-30T12:35:05.197190+0000","last_unstale":"2023-10-30T12:56:08.116088+0000","last_undegraded":"2023-10-30T12:56:08.116088+0000","last_fullsized":"2023-10-30T12:56:08.116088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:53:31.121721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":4111,"num_write":258,"num_write_kb":8429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"55'651","reported_seq":2802,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.024707+0000","last_change":"2023-10-30T12:42:02.253699+0000","last_active":"2023-10-30T12:56:07.024707+0000","last_peered":"2023-10-30T12:56:07.024707+0000","last_clean":"2023-10-30T12:56:07.024707+0000","last_became_active":"2023-10-30T12:35:06.204122+0000","last_became_peered":"2023-10-30T12:35:06.204122+0000","last_unstale":"2023-10-30T12:56:07.024707+0000","last_undegraded":"2023-10-30T12:56:07.024707+0000","last_fullsized":"2023-10-30T12:56:07.024707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:59:50.717088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030324050000000002,"stat_sum":{"num_bytes":15521520,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":60,"num_read_kb":8209,"num_write":621,"num_write_kb":41873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"55'359","reported_seq":431,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.533756+0000","last_change":"2023-10-30T12:35:05.180005+0000","last_active":"2023-10-30T12:56:09.533756+0000","last_peered":"2023-10-30T12:56:09.533756+0000","last_clean":"2023-10-30T12:56:09.533756+0000","last_became_active":"2023-10-30T12:35:05.179547+0000","last_became_peered":"2023-10-30T12:35:05.179547+0000","last_unstale":"2023-10-30T12:56:09.533756+0000","last_undegraded":"2023-10-30T12:56:09.533756+0000","last_fullsized":"2023-10-30T12:56:09.533756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:13:03.137433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":24600,"num_write":299,"num_write_kb":12510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"55'628","reported_seq":2779,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.133221+0000","last_change":"2023-10-30T12:42:02.473160+0000","last_active":"2023-10-30T12:56:07.133221+0000","last_peered":"2023-10-30T12:56:07.133221+0000","last_clean":"2023-10-30T12:56:07.133221+0000","last_became_active":"2023-10-30T12:35:06.215128+0000","last_became_peered":"2023-10-30T12:35:06.215128+0000","last_unstale":"2023-10-30T12:56:07.133221+0000","last_undegraded":"2023-10-30T12:56:07.133221+0000","last_fullsized":"2023-10-30T12:56:07.133221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:47:38.300886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064662909999999999,"stat_sum":{"num_bytes":22716254,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":61,"num_read_kb":11,"num_write":594,"num_write_kb":27628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"55'209","reported_seq":283,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.553645+0000","last_change":"2023-10-30T12:35:05.190041+0000","last_active":"2023-10-30T12:56:04.553645+0000","last_peered":"2023-10-30T12:56:04.553645+0000","last_clean":"2023-10-30T12:56:04.553645+0000","last_became_active":"2023-10-30T12:35:05.189573+0000","last_became_peered":"2023-10-30T12:35:05.189573+0000","last_unstale":"2023-10-30T12:56:04.553645+0000","last_undegraded":"2023-10-30T12:56:04.553645+0000","last_fullsized":"2023-10-30T12:56:04.553645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:02:57.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":24624,"num_write":154,"num_write_kb":8340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"55'702","reported_seq":2870,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.111628+0000","last_change":"2023-10-30T12:42:02.724827+0000","last_active":"2023-10-30T12:56:07.111628+0000","last_peered":"2023-10-30T12:56:07.111628+0000","last_clean":"2023-10-30T12:56:07.111628+0000","last_became_active":"2023-10-30T12:35:06.216885+0000","last_became_peered":"2023-10-30T12:35:06.216885+0000","last_unstale":"2023-10-30T12:56:07.111628+0000","last_undegraded":"2023-10-30T12:56:07.111628+0000","last_fullsized":"2023-10-30T12:56:07.111628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:50:48.181232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042006370000000001,"stat_sum":{"num_bytes":18982563,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":77,"num_read_kb":16402,"num_write":665,"num_write_kb":45070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"55'232","reported_seq":293,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.565577+0000","last_change":"2023-10-30T12:35:05.203198+0000","last_active":"2023-10-30T12:56:04.565577+0000","last_peered":"2023-10-30T12:56:04.565577+0000","last_clean":"2023-10-30T12:56:04.565577+0000","last_became_active":"2023-10-30T12:35:05.202612+0000","last_became_peered":"2023-10-30T12:35:05.202612+0000","last_unstale":"2023-10-30T12:56:04.565577+0000","last_undegraded":"2023-10-30T12:56:04.565577+0000","last_fullsized":"2023-10-30T12:56:04.565577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:55.986236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":181,"num_write_kb":8338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"55'665","reported_seq":2831,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.009342+0000","last_change":"2023-10-30T12:42:02.471602+0000","last_active":"2023-10-30T12:56:07.009342+0000","last_peered":"2023-10-30T12:56:07.009342+0000","last_clean":"2023-10-30T12:56:07.009342+0000","last_became_active":"2023-10-30T12:35:06.215553+0000","last_became_peered":"2023-10-30T12:35:06.215553+0000","last_unstale":"2023-10-30T12:56:07.009342+0000","last_undegraded":"2023-10-30T12:56:07.009342+0000","last_fullsized":"2023-10-30T12:56:07.009342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:58:00.184959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060418609999999999,"stat_sum":{"num_bytes":18953195,"num_objects":130,"num_object_clones":0,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":0,"num_read":76,"num_read_kb":33,"num_write":640,"num_write_kb":32599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"55'289","reported_seq":360,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:27.641002+0000","last_change":"2023-10-30T12:35:05.181882+0000","last_active":"2023-10-30T12:55:27.641002+0000","last_peered":"2023-10-30T12:55:27.641002+0000","last_clean":"2023-10-30T12:55:27.641002+0000","last_became_active":"2023-10-30T12:35:05.181115+0000","last_became_peered":"2023-10-30T12:35:05.181115+0000","last_unstale":"2023-10-30T12:55:27.641002+0000","last_undegraded":"2023-10-30T12:55:27.641002+0000","last_fullsized":"2023-10-30T12:55:27.641002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:18:55.496940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":43239,"num_write":227,"num_write_kb":16591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"55'646","reported_seq":2797,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.181423+0000","last_change":"2023-10-30T12:42:02.738621+0000","last_active":"2023-10-30T12:56:07.181423+0000","last_peered":"2023-10-30T12:56:07.181423+0000","last_clean":"2023-10-30T12:56:07.181423+0000","last_became_active":"2023-10-30T12:35:06.197722+0000","last_became_peered":"2023-10-30T12:35:06.197722+0000","last_unstale":"2023-10-30T12:56:07.181423+0000","last_undegraded":"2023-10-30T12:56:07.181423+0000","last_fullsized":"2023-10-30T12:56:07.181423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:56:33.480519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011777173,"stat_sum":{"num_bytes":11054792,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":60,"num_read_kb":16,"num_write":607,"num_write_kb":21388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"55'481","reported_seq":572,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.552101+0000","last_change":"2023-10-30T12:35:05.169455+0000","last_active":"2023-10-30T12:56:04.552101+0000","last_peered":"2023-10-30T12:56:04.552101+0000","last_clean":"2023-10-30T12:56:04.552101+0000","last_became_active":"2023-10-30T12:35:05.169130+0000","last_became_peered":"2023-10-30T12:35:05.169130+0000","last_unstale":"2023-10-30T12:56:04.552101+0000","last_undegraded":"2023-10-30T12:56:04.552101+0000","last_fullsized":"2023-10-30T12:56:04.552101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:59:14.096315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583024,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":18341,"num_write":415,"num_write_kb":12551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"55'687","reported_seq":2869,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.010092+0000","last_change":"2023-10-30T12:42:02.470189+0000","last_active":"2023-10-30T12:56:07.010092+0000","last_peered":"2023-10-30T12:56:07.010092+0000","last_clean":"2023-10-30T12:56:07.010092+0000","last_became_active":"2023-10-30T12:35:06.214839+0000","last_became_peered":"2023-10-30T12:35:06.214839+0000","last_unstale":"2023-10-30T12:56:07.010092+0000","last_undegraded":"2023-10-30T12:56:07.010092+0000","last_fullsized":"2023-10-30T12:56:07.010092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:09:56.796250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049147749999999997,"stat_sum":{"num_bytes":12025229,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":91,"num_read_kb":16402,"num_write":654,"num_write_kb":29636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"55'312","reported_seq":377,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.648665+0000","last_change":"2023-10-30T12:35:05.189361+0000","last_active":"2023-10-30T12:56:04.648665+0000","last_peered":"2023-10-30T12:56:04.648665+0000","last_clean":"2023-10-30T12:56:04.648665+0000","last_became_active":"2023-10-30T12:35:05.188862+0000","last_became_peered":"2023-10-30T12:35:05.188862+0000","last_unstale":"2023-10-30T12:56:04.648665+0000","last_undegraded":"2023-10-30T12:56:04.648665+0000","last_fullsized":"2023-10-30T12:56:04.648665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:13.319732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":12343,"num_write":386,"num_write_kb":4750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"55'644","reported_seq":2800,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.941496+0000","last_change":"2023-10-30T12:42:02.472755+0000","last_active":"2023-10-30T12:56:06.941496+0000","last_peered":"2023-10-30T12:56:06.941496+0000","last_clean":"2023-10-30T12:56:06.941496+0000","last_became_active":"2023-10-30T12:35:06.216533+0000","last_became_peered":"2023-10-30T12:35:06.216533+0000","last_unstale":"2023-10-30T12:56:06.941496+0000","last_undegraded":"2023-10-30T12:56:06.941496+0000","last_fullsized":"2023-10-30T12:56:06.941496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:13:01.401936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060962430000000003,"stat_sum":{"num_bytes":10676108,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":65,"num_read_kb":4109,"num_write":617,"num_write_kb":24051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"55'234","reported_seq":295,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:04.569141+0000","last_change":"2023-10-30T12:35:05.177609+0000","last_active":"2023-10-30T12:55:04.569141+0000","last_peered":"2023-10-30T12:55:04.569141+0000","last_clean":"2023-10-30T12:55:04.569141+0000","last_became_active":"2023-10-30T12:35:05.177316+0000","last_became_peered":"2023-10-30T12:35:05.177316+0000","last_unstale":"2023-10-30T12:55:04.569141+0000","last_undegraded":"2023-10-30T12:55:04.569141+0000","last_fullsized":"2023-10-30T12:55:04.569141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:46.820925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":180,"num_write_kb":8325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"55'668","reported_seq":2828,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.003575+0000","last_change":"2023-10-30T12:42:02.241911+0000","last_active":"2023-10-30T12:56:07.003575+0000","last_peered":"2023-10-30T12:56:07.003575+0000","last_clean":"2023-10-30T12:56:07.003575+0000","last_became_active":"2023-10-30T12:35:46.049464+0000","last_became_peered":"2023-10-30T12:35:46.049464+0000","last_unstale":"2023-10-30T12:56:07.003575+0000","last_undegraded":"2023-10-30T12:56:07.003575+0000","last_fullsized":"2023-10-30T12:56:07.003575+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:06:39.487053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023101760000000002,"stat_sum":{"num_bytes":6776001,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":644,"num_write_kb":12631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"55'211","reported_seq":272,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:28.291856+0000","last_change":"2023-10-30T12:35:05.171787+0000","last_active":"2023-10-30T12:55:28.291856+0000","last_peered":"2023-10-30T12:55:28.291856+0000","last_clean":"2023-10-30T12:55:28.291856+0000","last_became_active":"2023-10-30T12:35:05.171491+0000","last_became_peered":"2023-10-30T12:35:05.171491+0000","last_unstale":"2023-10-30T12:55:28.291856+0000","last_undegraded":"2023-10-30T12:55:28.291856+0000","last_fullsized":"2023-10-30T12:55:28.291856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:43:15.331175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":16387,"num_write":154,"num_write_kb":4208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"55'739","reported_seq":2900,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.151055+0000","last_change":"2023-10-30T12:42:02.746323+0000","last_active":"2023-10-30T12:56:07.151055+0000","last_peered":"2023-10-30T12:56:07.151055+0000","last_clean":"2023-10-30T12:56:07.151055+0000","last_became_active":"2023-10-30T12:35:06.213493+0000","last_became_peered":"2023-10-30T12:35:06.213493+0000","last_unstale":"2023-10-30T12:56:07.151055+0000","last_undegraded":"2023-10-30T12:56:07.151055+0000","last_fullsized":"2023-10-30T12:56:07.151055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:38:05.007816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015847370999999999,"stat_sum":{"num_bytes":18167539,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":70,"num_read_kb":4114,"num_write":710,"num_write_kb":33439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"55'305","reported_seq":368,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:04.570822+0000","last_change":"2023-10-30T12:35:05.197811+0000","last_active":"2023-10-30T12:55:04.570822+0000","last_peered":"2023-10-30T12:55:04.570822+0000","last_clean":"2023-10-30T12:55:04.570822+0000","last_became_active":"2023-10-30T12:35:05.197453+0000","last_became_peered":"2023-10-30T12:35:05.197453+0000","last_unstale":"2023-10-30T12:55:04.570822+0000","last_undegraded":"2023-10-30T12:55:04.570822+0000","last_fullsized":"2023-10-30T12:55:04.570822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:16:14.550887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":240,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"55'660","reported_seq":2812,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.010112+0000","last_change":"2023-10-30T12:42:02.472032+0000","last_active":"2023-10-30T12:56:07.010112+0000","last_peered":"2023-10-30T12:56:07.010112+0000","last_clean":"2023-10-30T12:56:07.010112+0000","last_became_active":"2023-10-30T12:35:06.219526+0000","last_became_peered":"2023-10-30T12:35:06.219526+0000","last_unstale":"2023-10-30T12:56:07.010112+0000","last_undegraded":"2023-10-30T12:56:07.010112+0000","last_fullsized":"2023-10-30T12:56:07.010112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:01:04.257663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060180570000000003,"stat_sum":{"num_bytes":14243175,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":61,"num_read_kb":19722,"num_write":630,"num_write_kb":44881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":208,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:54:25.564818+0000","last_change":"2023-10-30T12:34:55.206976+0000","last_active":"2023-10-30T12:54:25.564818+0000","last_peered":"2023-10-30T12:54:25.564818+0000","last_clean":"2023-10-30T12:54:25.564818+0000","last_became_active":"2023-10-30T12:34:55.205923+0000","last_became_peered":"2023-10-30T12:34:55.205923+0000","last_unstale":"2023-10-30T12:54:25.564818+0000","last_undegraded":"2023-10-30T12:54:25.564818+0000","last_fullsized":"2023-10-30T12:54:25.564818+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_clean_scrub_stamp":"2023-10-30T12:34:53.444020+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:25:28.782111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"53'311","reported_seq":380,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:54:54.680160+0000","last_change":"2023-10-30T12:35:05.197974+0000","last_active":"2023-10-30T12:54:54.680160+0000","last_peered":"2023-10-30T12:54:54.680160+0000","last_clean":"2023-10-30T12:54:54.680160+0000","last_became_active":"2023-10-30T12:35:05.197201+0000","last_became_peered":"2023-10-30T12:35:05.197201+0000","last_unstale":"2023-10-30T12:54:54.680160+0000","last_undegraded":"2023-10-30T12:54:54.680160+0000","last_fullsized":"2023-10-30T12:54:54.680160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:39:47.119091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":23,"num_read_kb":16415,"num_write":260,"num_write_kb":8374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"55'616","reported_seq":2759,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.118778+0000","last_change":"2023-10-30T12:42:02.243413+0000","last_active":"2023-10-30T12:56:07.118778+0000","last_peered":"2023-10-30T12:56:07.118778+0000","last_clean":"2023-10-30T12:56:07.118778+0000","last_became_active":"2023-10-30T12:35:06.217687+0000","last_became_peered":"2023-10-30T12:35:06.217687+0000","last_unstale":"2023-10-30T12:56:07.118778+0000","last_undegraded":"2023-10-30T12:56:07.118778+0000","last_fullsized":"2023-10-30T12:56:07.118778+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:47:42.301163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059801699999999999,"stat_sum":{"num_bytes":18764071,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":51,"num_read_kb":8203,"num_write":589,"num_write_kb":32793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"55'386","reported_seq":458,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:31.185445+0000","last_change":"2023-10-30T12:35:05.208339+0000","last_active":"2023-10-30T12:55:31.185445+0000","last_peered":"2023-10-30T12:55:31.185445+0000","last_clean":"2023-10-30T12:55:31.185445+0000","last_became_active":"2023-10-30T12:35:05.207828+0000","last_became_peered":"2023-10-30T12:35:05.207828+0000","last_unstale":"2023-10-30T12:55:31.185445+0000","last_undegraded":"2023-10-30T12:55:31.185445+0000","last_fullsized":"2023-10-30T12:55:31.185445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:00.649048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":45074,"num_write":317,"num_write_kb":20666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"55'680","reported_seq":2830,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.087169+0000","last_change":"2023-10-30T12:42:02.724463+0000","last_active":"2023-10-30T12:56:07.087169+0000","last_peered":"2023-10-30T12:56:07.087169+0000","last_clean":"2023-10-30T12:56:07.087169+0000","last_became_active":"2023-10-30T12:35:06.194791+0000","last_became_peered":"2023-10-30T12:35:06.194791+0000","last_unstale":"2023-10-30T12:56:07.087169+0000","last_undegraded":"2023-10-30T12:56:07.087169+0000","last_fullsized":"2023-10-30T12:56:07.087169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:13:52.749170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043414500000000002,"stat_sum":{"num_bytes":10508371,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":58,"num_read_kb":3682,"num_write":647,"num_write_kb":34521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"55'607","reported_seq":965,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:57.501324+0000","last_change":"2023-10-30T12:35:05.176296+0000","last_active":"2023-10-30T12:55:57.501324+0000","last_peered":"2023-10-30T12:55:57.501324+0000","last_clean":"2023-10-30T12:55:57.501324+0000","last_became_active":"2023-10-30T12:35:05.175959+0000","last_became_peered":"2023-10-30T12:35:05.175959+0000","last_unstale":"2023-10-30T12:55:57.501324+0000","last_undegraded":"2023-10-30T12:55:57.501324+0000","last_fullsized":"2023-10-30T12:55:57.501324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:53.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9398249,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":313,"num_read_kb":20888,"num_write":566,"num_write_kb":9626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"55'674","reported_seq":2834,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.973362+0000","last_change":"2023-10-30T12:42:02.711348+0000","last_active":"2023-10-30T12:56:06.973362+0000","last_peered":"2023-10-30T12:56:06.973362+0000","last_clean":"2023-10-30T12:56:06.973362+0000","last_became_active":"2023-10-30T12:35:06.164711+0000","last_became_peered":"2023-10-30T12:35:06.164711+0000","last_unstale":"2023-10-30T12:56:06.973362+0000","last_undegraded":"2023-10-30T12:56:06.973362+0000","last_fullsized":"2023-10-30T12:56:06.973362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:21:46.319142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002714419,"stat_sum":{"num_bytes":18594076,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":68,"num_read_kb":8213,"num_write":640,"num_write_kb":33748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"55'497","reported_seq":561,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:56.727539+0000","last_change":"2023-10-30T12:35:05.198964+0000","last_active":"2023-10-30T12:55:56.727539+0000","last_peered":"2023-10-30T12:55:56.727539+0000","last_clean":"2023-10-30T12:55:56.727539+0000","last_became_active":"2023-10-30T12:35:05.196873+0000","last_became_peered":"2023-10-30T12:35:05.196873+0000","last_unstale":"2023-10-30T12:55:56.727539+0000","last_undegraded":"2023-10-30T12:55:56.727539+0000","last_fullsized":"2023-10-30T12:55:56.727539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:47.793585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":431,"num_write_kb":16687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"55'673","reported_seq":2837,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.990872+0000","last_change":"2023-10-30T12:42:02.250101+0000","last_active":"2023-10-30T12:56:06.990872+0000","last_peered":"2023-10-30T12:56:06.990872+0000","last_clean":"2023-10-30T12:56:06.990872+0000","last_became_active":"2023-10-30T12:35:06.183225+0000","last_became_peered":"2023-10-30T12:35:06.183225+0000","last_unstale":"2023-10-30T12:56:06.990872+0000","last_undegraded":"2023-10-30T12:56:06.990872+0000","last_fullsized":"2023-10-30T12:56:06.990872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:02:51.357971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032989500000000001,"stat_sum":{"num_bytes":17999573,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":72,"num_read_kb":12,"num_write":642,"num_write_kb":36248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"55'183","reported_seq":239,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:27.599652+0000","last_change":"2023-10-30T12:35:05.177571+0000","last_active":"2023-10-30T12:55:27.599652+0000","last_peered":"2023-10-30T12:55:27.599652+0000","last_clean":"2023-10-30T12:55:27.599652+0000","last_became_active":"2023-10-30T12:35:05.177266+0000","last_became_peered":"2023-10-30T12:35:05.177266+0000","last_unstale":"2023-10-30T12:55:27.599652+0000","last_undegraded":"2023-10-30T12:55:27.599652+0000","last_fullsized":"2023-10-30T12:55:27.599652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:14.601934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":129,"num_write_kb":8259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"55'655","reported_seq":2816,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.161469+0000","last_change":"2023-10-30T12:42:02.745017+0000","last_active":"2023-10-30T12:56:07.161469+0000","last_peered":"2023-10-30T12:56:07.161469+0000","last_clean":"2023-10-30T12:56:07.161469+0000","last_became_active":"2023-10-30T12:35:06.196307+0000","last_became_peered":"2023-10-30T12:35:06.196307+0000","last_unstale":"2023-10-30T12:56:07.161469+0000","last_undegraded":"2023-10-30T12:56:07.161469+0000","last_fullsized":"2023-10-30T12:56:07.161469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:09:56.936899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015197106,"stat_sum":{"num_bytes":5743717,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":69,"num_read_kb":13,"num_write":620,"num_write_kb":24446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"55'489","reported_seq":552,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.635330+0000","last_change":"2023-10-30T12:35:05.207594+0000","last_active":"2023-10-30T12:56:09.635330+0000","last_peered":"2023-10-30T12:56:09.635330+0000","last_clean":"2023-10-30T12:56:09.635330+0000","last_became_active":"2023-10-30T12:35:05.206930+0000","last_became_peered":"2023-10-30T12:35:05.206930+0000","last_unstale":"2023-10-30T12:56:09.635330+0000","last_undegraded":"2023-10-30T12:56:09.635330+0000","last_fullsized":"2023-10-30T12:56:09.635330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:23:46.519077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15572411,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":8,"num_read_kb":9517,"num_write":429,"num_write_kb":15473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"55'662","reported_seq":2826,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.964738+0000","last_change":"2023-10-30T12:42:02.710925+0000","last_active":"2023-10-30T12:56:06.964738+0000","last_peered":"2023-10-30T12:56:06.964738+0000","last_clean":"2023-10-30T12:56:06.964738+0000","last_became_active":"2023-10-30T12:35:06.164733+0000","last_became_peered":"2023-10-30T12:35:06.164733+0000","last_unstale":"2023-10-30T12:56:06.964738+0000","last_undegraded":"2023-10-30T12:56:06.964738+0000","last_fullsized":"2023-10-30T12:56:06.964738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:04:21.927512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024278020000000002,"stat_sum":{"num_bytes":19053289,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":637,"num_write_kb":32762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"55'374","reported_seq":441,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.541877+0000","last_change":"2023-10-30T12:35:05.185034+0000","last_active":"2023-10-30T12:56:04.541877+0000","last_peered":"2023-10-30T12:56:04.541877+0000","last_clean":"2023-10-30T12:56:04.541877+0000","last_became_active":"2023-10-30T12:35:05.184695+0000","last_became_peered":"2023-10-30T12:35:05.184695+0000","last_unstale":"2023-10-30T12:56:04.541877+0000","last_undegraded":"2023-10-30T12:56:04.541877+0000","last_fullsized":"2023-10-30T12:56:04.541877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:01:42.019128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":8489,"num_write":571,"num_write_kb":5858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"55'646","reported_seq":2799,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.108475+0000","last_change":"2023-10-30T12:42:02.242088+0000","last_active":"2023-10-30T12:56:07.108475+0000","last_peered":"2023-10-30T12:56:07.108475+0000","last_clean":"2023-10-30T12:56:07.108475+0000","last_became_active":"2023-10-30T12:35:06.204422+0000","last_became_peered":"2023-10-30T12:35:06.204422+0000","last_unstale":"2023-10-30T12:56:07.108475+0000","last_undegraded":"2023-10-30T12:56:07.108475+0000","last_fullsized":"2023-10-30T12:56:07.108475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:38:09.854923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048535030000000003,"stat_sum":{"num_bytes":22678810,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":62,"num_read_kb":4110,"num_write":624,"num_write_kb":32700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"55'230","reported_seq":292,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:28.060181+0000","last_change":"2023-10-30T12:35:05.200871+0000","last_active":"2023-10-30T12:55:28.060181+0000","last_peered":"2023-10-30T12:55:28.060181+0000","last_clean":"2023-10-30T12:55:28.060181+0000","last_became_active":"2023-10-30T12:35:05.200449+0000","last_became_peered":"2023-10-30T12:35:05.200449+0000","last_unstale":"2023-10-30T12:55:28.060181+0000","last_undegraded":"2023-10-30T12:55:28.060181+0000","last_fullsized":"2023-10-30T12:55:28.060181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:39:20.867587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":4117,"num_write":187,"num_write_kb":4299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"55'643","reported_seq":2803,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.060907+0000","last_change":"2023-10-30T12:42:02.724106+0000","last_active":"2023-10-30T12:56:07.060907+0000","last_peered":"2023-10-30T12:56:07.060907+0000","last_clean":"2023-10-30T12:56:07.060907+0000","last_became_active":"2023-10-30T12:35:06.179412+0000","last_became_peered":"2023-10-30T12:35:06.179412+0000","last_unstale":"2023-10-30T12:56:07.060907+0000","last_undegraded":"2023-10-30T12:56:07.060907+0000","last_fullsized":"2023-10-30T12:56:07.060907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:15:21.046958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003661568,"stat_sum":{"num_bytes":1280771,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":69,"num_read_kb":22,"num_write":604,"num_write_kb":12079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"55'552","reported_seq":617,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:26.056739+0000","last_change":"2023-10-30T12:35:05.207392+0000","last_active":"2023-10-30T12:55:26.056739+0000","last_peered":"2023-10-30T12:55:26.056739+0000","last_clean":"2023-10-30T12:55:26.056739+0000","last_became_active":"2023-10-30T12:35:05.206607+0000","last_became_peered":"2023-10-30T12:35:05.206607+0000","last_unstale":"2023-10-30T12:55:26.056739+0000","last_undegraded":"2023-10-30T12:55:26.056739+0000","last_fullsized":"2023-10-30T12:55:26.056739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:36.758740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":17,"num_read_kb":23179,"num_write":497,"num_write_kb":12662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"55'665","reported_seq":2826,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.023360+0000","last_change":"2023-10-30T12:42:02.251623+0000","last_active":"2023-10-30T12:56:07.023360+0000","last_peered":"2023-10-30T12:56:07.023360+0000","last_clean":"2023-10-30T12:56:07.023360+0000","last_became_active":"2023-10-30T12:35:06.181256+0000","last_became_peered":"2023-10-30T12:35:06.181256+0000","last_unstale":"2023-10-30T12:56:07.023360+0000","last_undegraded":"2023-10-30T12:56:07.023360+0000","last_fullsized":"2023-10-30T12:56:07.023360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:30:13.806672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030387220000000002,"stat_sum":{"num_bytes":10400130,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":70,"num_read_kb":4103,"num_write":631,"num_write_kb":29164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"55'369","reported_seq":432,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:02.291363+0000","last_change":"2023-10-30T12:35:05.201024+0000","last_active":"2023-10-30T12:56:02.291363+0000","last_peered":"2023-10-30T12:56:02.291363+0000","last_clean":"2023-10-30T12:56:02.291363+0000","last_became_active":"2023-10-30T12:35:05.200724+0000","last_became_peered":"2023-10-30T12:35:05.200724+0000","last_unstale":"2023-10-30T12:56:02.291363+0000","last_undegraded":"2023-10-30T12:56:02.291363+0000","last_fullsized":"2023-10-30T12:56:02.291363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:24.746469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388698,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":12300,"num_write":322,"num_write_kb":8548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"55'675","reported_seq":2824,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.153249+0000","last_change":"2023-10-30T12:42:02.241265+0000","last_active":"2023-10-30T12:56:07.153249+0000","last_peered":"2023-10-30T12:56:07.153249+0000","last_clean":"2023-10-30T12:56:07.153249+0000","last_became_active":"2023-10-30T12:35:06.173996+0000","last_became_peered":"2023-10-30T12:35:06.173996+0000","last_unstale":"2023-10-30T12:56:07.153249+0000","last_undegraded":"2023-10-30T12:56:07.153249+0000","last_fullsized":"2023-10-30T12:56:07.153249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:34:09.185233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045562010000000002,"stat_sum":{"num_bytes":5657110,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":58,"num_read_kb":18,"num_write":633,"num_write_kb":16227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"55'873","reported_seq":934,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.676168+0000","last_change":"2023-10-30T12:35:05.164079+0000","last_active":"2023-10-30T12:56:04.676168+0000","last_peered":"2023-10-30T12:56:04.676168+0000","last_clean":"2023-10-30T12:56:04.676168+0000","last_became_active":"2023-10-30T12:35:05.163794+0000","last_became_peered":"2023-10-30T12:35:05.163794+0000","last_unstale":"2023-10-30T12:56:04.676168+0000","last_undegraded":"2023-10-30T12:56:04.676168+0000","last_fullsized":"2023-10-30T12:56:04.676168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:19.795656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":10,"num_read_kb":8454,"num_write":952,"num_write_kb":13780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"55'645","reported_seq":2805,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.134005+0000","last_change":"2023-10-30T12:42:02.242567+0000","last_active":"2023-10-30T12:56:07.134005+0000","last_peered":"2023-10-30T12:56:07.134005+0000","last_clean":"2023-10-30T12:56:07.134005+0000","last_became_active":"2023-10-30T12:35:06.183226+0000","last_became_peered":"2023-10-30T12:35:06.183226+0000","last_unstale":"2023-10-30T12:56:07.134005+0000","last_undegraded":"2023-10-30T12:56:07.134005+0000","last_fullsized":"2023-10-30T12:56:07.134005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:30:48.627756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052521379999999999,"stat_sum":{"num_bytes":5772919,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":69,"num_read_kb":4112,"num_write":611,"num_write_kb":16444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"55'336","reported_seq":402,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:36.391405+0000","last_change":"2023-10-30T12:35:05.206995+0000","last_active":"2023-10-30T12:55:36.391405+0000","last_peered":"2023-10-30T12:55:36.391405+0000","last_clean":"2023-10-30T12:55:36.391405+0000","last_became_active":"2023-10-30T12:35:05.206518+0000","last_became_peered":"2023-10-30T12:35:05.206518+0000","last_unstale":"2023-10-30T12:55:36.391405+0000","last_undegraded":"2023-10-30T12:55:36.391405+0000","last_fullsized":"2023-10-30T12:55:36.391405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:41:33.703261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":268,"num_write_kb":12461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"55'637","reported_seq":2796,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.957805+0000","last_change":"2023-10-30T12:42:02.737782+0000","last_active":"2023-10-30T12:56:06.957805+0000","last_peered":"2023-10-30T12:56:06.957805+0000","last_clean":"2023-10-30T12:56:06.957805+0000","last_became_active":"2023-10-30T12:35:06.197861+0000","last_became_peered":"2023-10-30T12:35:06.197861+0000","last_unstale":"2023-10-30T12:56:06.957805+0000","last_undegraded":"2023-10-30T12:56:06.957805+0000","last_fullsized":"2023-10-30T12:56:06.957805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:38:17.960141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011118161,"stat_sum":{"num_bytes":14676487,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":68,"num_read_kb":15,"num_write":602,"num_write_kb":24516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"55'655","reported_seq":2826,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.074252+0000","last_change":"2023-10-30T12:42:02.744356+0000","last_active":"2023-10-30T12:56:07.074252+0000","last_peered":"2023-10-30T12:56:07.074252+0000","last_clean":"2023-10-30T12:56:07.074252+0000","last_became_active":"2023-10-30T12:35:06.212548+0000","last_became_peered":"2023-10-30T12:35:06.212548+0000","last_unstale":"2023-10-30T12:56:07.074252+0000","last_undegraded":"2023-10-30T12:56:07.074252+0000","last_fullsized":"2023-10-30T12:56:07.074252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:58:19.323075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015114931999999999,"stat_sum":{"num_bytes":19302518,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":81,"num_read_kb":7349,"num_write":635,"num_write_kb":32674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"55'363","reported_seq":423,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:07.528197+0000","last_change":"2023-10-30T12:35:05.206980+0000","last_active":"2023-10-30T12:55:07.528197+0000","last_peered":"2023-10-30T12:55:07.528197+0000","last_clean":"2023-10-30T12:55:07.528197+0000","last_became_active":"2023-10-30T12:35:05.206559+0000","last_became_peered":"2023-10-30T12:35:05.206559+0000","last_unstale":"2023-10-30T12:55:07.528197+0000","last_undegraded":"2023-10-30T12:55:07.528197+0000","last_fullsized":"2023-10-30T12:55:07.528197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:40.259622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":312,"num_write_kb":8389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"55'654","reported_seq":2814,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.072742+0000","last_change":"2023-10-30T12:42:02.744472+0000","last_active":"2023-10-30T12:56:07.072742+0000","last_peered":"2023-10-30T12:56:07.072742+0000","last_clean":"2023-10-30T12:56:07.072742+0000","last_became_active":"2023-10-30T12:35:06.212505+0000","last_became_peered":"2023-10-30T12:35:06.212505+0000","last_unstale":"2023-10-30T12:56:07.072742+0000","last_undegraded":"2023-10-30T12:56:07.072742+0000","last_fullsized":"2023-10-30T12:56:07.072742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:28:56.182227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014553393,"stat_sum":{"num_bytes":9479151,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":70,"num_read_kb":12304,"num_write":628,"num_write_kb":36199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"55'377","reported_seq":448,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:59.997488+0000","last_change":"2023-10-30T12:35:05.207851+0000","last_active":"2023-10-30T12:55:59.997488+0000","last_peered":"2023-10-30T12:55:59.997488+0000","last_clean":"2023-10-30T12:55:59.997488+0000","last_became_active":"2023-10-30T12:35:05.206879+0000","last_became_peered":"2023-10-30T12:35:05.206879+0000","last_unstale":"2023-10-30T12:55:59.997488+0000","last_undegraded":"2023-10-30T12:55:59.997488+0000","last_fullsized":"2023-10-30T12:55:59.997488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:43:15.717507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":32,"num_read_kb":45093,"num_write":322,"num_write_kb":20730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"55'623","reported_seq":2785,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.101531+0000","last_change":"2023-10-30T12:42:02.723663+0000","last_active":"2023-10-30T12:56:07.101531+0000","last_peered":"2023-10-30T12:56:07.101531+0000","last_clean":"2023-10-30T12:56:07.101531+0000","last_became_active":"2023-10-30T12:35:46.275077+0000","last_became_peered":"2023-10-30T12:35:46.275077+0000","last_unstale":"2023-10-30T12:56:07.101531+0000","last_undegraded":"2023-10-30T12:56:07.101531+0000","last_fullsized":"2023-10-30T12:56:07.101531+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:31:10.982688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032416139999999999,"stat_sum":{"num_bytes":6183106,"num_objects":128,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":0,"num_read":63,"num_read_kb":4113,"num_write":600,"num_write_kb":9129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"55'508","reported_seq":569,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:35.795056+0000","last_change":"2023-10-30T12:35:05.174319+0000","last_active":"2023-10-30T12:55:35.795056+0000","last_peered":"2023-10-30T12:55:35.795056+0000","last_clean":"2023-10-30T12:55:35.795056+0000","last_became_active":"2023-10-30T12:35:05.173984+0000","last_became_peered":"2023-10-30T12:35:05.173984+0000","last_unstale":"2023-10-30T12:55:35.795056+0000","last_undegraded":"2023-10-30T12:55:35.795056+0000","last_fullsized":"2023-10-30T12:55:35.795056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:11.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":430,"num_write_kb":12596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"55'666","reported_seq":2819,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.075370+0000","last_change":"2023-10-30T12:42:02.243078+0000","last_active":"2023-10-30T12:56:07.075370+0000","last_peered":"2023-10-30T12:56:07.075370+0000","last_clean":"2023-10-30T12:56:07.075370+0000","last_became_active":"2023-10-30T12:35:06.198504+0000","last_became_peered":"2023-10-30T12:35:06.198504+0000","last_unstale":"2023-10-30T12:56:07.075370+0000","last_undegraded":"2023-10-30T12:56:07.075370+0000","last_fullsized":"2023-10-30T12:56:07.075370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:44:32.447527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055292249999999996,"stat_sum":{"num_bytes":14373178,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":61,"num_read_kb":1995,"num_write":639,"num_write_kb":19071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"55'208","reported_seq":290,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:28.291837+0000","last_change":"2023-10-30T12:35:05.200926+0000","last_active":"2023-10-30T12:55:28.291837+0000","last_peered":"2023-10-30T12:55:28.291837+0000","last_clean":"2023-10-30T12:55:28.291837+0000","last_became_active":"2023-10-30T12:35:05.200623+0000","last_became_peered":"2023-10-30T12:35:05.200623+0000","last_unstale":"2023-10-30T12:55:28.291837+0000","last_undegraded":"2023-10-30T12:55:28.291837+0000","last_fullsized":"2023-10-30T12:55:28.291837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:38.942922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":63,"num_read_kb":24633,"num_write":147,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":916500589,"num_objects":9593,"num_object_clones":0,"num_object_copies":19186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9593,"num_whiteouts":0,"num_read":4339,"num_read_kb":400205,"num_write":40772,"num_write_kb":1941314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1881726976,"data_stored":1831461082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42742,"ondisk_log_size":42742,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":354417535,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1232,"num_read_kb":640401,"num_write":11493,"num_write_kb":356840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":709001216,"data_stored":708835070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12852,"ondisk_log_size":12852,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542405,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487968,"kb_used_data":398488,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":93883872,"statfs":{"total":96636764160,"available":96137084928,"internally_reserved":0,"allocated":408051712,"data_stored":400479873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":91617741},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.48599999999999999,"5min":0.438,"15min":0.42899999999999999},"max":{"1min":1.2330000000000001,"5min":1.371,"15min":1.371},"last":1.121},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.82099999999999995,"15min":0.82399999999999995},"min":{"1min":0.54200000000000004,"5min":0.437,"15min":0.41699999999999998},"max":{"1min":1.137,"5min":1.45,"15min":1.45},"last":0.91300000000000003}]},{"osd":1,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82699999999999996,"15min":0.85299999999999998},"min":{"1min":0.52700000000000002,"5min":0.46700000000000003,"15min":0.44600000000000001},"max":{"1min":1.222,"5min":1.222,"15min":1.496},"last":1.095},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.86599999999999999,"15min":0.89600000000000002},"min":{"1min":0.55700000000000005,"5min":0.53500000000000003,"15min":0.51800000000000002},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.98999999999999999}]},{"osd":2,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.873,"15min":0.82799999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.1659999999999999,"5min":1.242,"15min":1.9059999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.84199999999999997,"15min":0.89700000000000002},"min":{"1min":0.60299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.1830000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":0.92600000000000005,"15min":0.91100000000000003},"min":{"1min":0.76300000000000001,"5min":0.54200000000000004,"15min":0.54000000000000004},"max":{"1min":1.474,"5min":1.607,"15min":1.9810000000000001},"last":1.278},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.004,"15min":1},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.1559999999999999}]},{"osd":4,"last update":"Mon Oct 30 12:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90100000000000002,"15min":0.92800000000000005},"min":{"1min":0.71599999999999997,"5min":0.52500000000000002,"15min":0.46899999999999997},"max":{"1min":1.329,"5min":1.4059999999999999,"15min":2.7309999999999999},"last":1.05},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.016,"15min":0.99399999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.41999999999999998},"max":{"1min":1.3600000000000001,"5min":1.694,"15min":1.8859999999999999},"last":0.95299999999999996}]},{"osd":5,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.048,"15min":1.0700000000000001},"min":{"1min":0.72099999999999997,"5min":0.59399999999999997,"15min":0.49099999999999999},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":2.2120000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.218,"15min":1.143},"min":{"1min":1.0089999999999999,"5min":0.72599999999999998,"15min":0.69899999999999995},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.2310000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0229999999999999,"15min":1.087},"min":{"1min":0.79400000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.456,"5min":1.478,"15min":1.802},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.1359999999999999,"15min":1.1930000000000001},"min":{"1min":0.92800000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.4870000000000001,"5min":1.6639999999999999,"15min":1.881},"last":1.014}]}]},{"osd":6,"up_from":14,"seq":60129542404,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":378044,"kb_used_data":292788,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":93993796,"statfs":{"total":96636764160,"available":96249647104,"internally_reserved":0,"allocated":299814912,"data_stored":294791815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":87292361},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.83199999999999996,"15min":0.82399999999999995},"min":{"1min":0.496,"5min":0.45600000000000002,"15min":0.38600000000000001},"max":{"1min":1.056,"5min":5.8760000000000003,"15min":5.8760000000000003},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.60899999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.161,"5min":1.508,"15min":1.673},"last":0.85499999999999998}]},{"osd":1,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.85699999999999998,"15min":0.92100000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.113,"5min":2.2679999999999998,"15min":5.0949999999999998},"last":1.012},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.91100000000000003,"15min":0.92700000000000005},"min":{"1min":0.65000000000000002,"5min":0.54800000000000004,"15min":0.45500000000000002},"max":{"1min":1.282,"5min":1.423,"15min":5.0519999999999996},"last":1.0840000000000001}]},{"osd":2,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.86499999999999999,"15min":0.84699999999999998},"min":{"1min":0.621,"5min":0.52900000000000003,"15min":0.51800000000000002},"max":{"1min":1.403,"5min":2.649,"15min":2.649},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.069,"5min":0.97099999999999997,"15min":0.92500000000000004},"min":{"1min":0.74099999999999999,"5min":0.56899999999999995,"15min":0.44700000000000001},"max":{"1min":1.528,"5min":2.282,"15min":2.282},"last":1.143}]},{"osd":3,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.88500000000000001,"15min":0.94599999999999995},"min":{"1min":0.62,"5min":0.51700000000000002,"15min":0.40200000000000002},"max":{"1min":1.1120000000000001,"5min":1.929,"15min":1.929},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.97499999999999998,"15min":1},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.42699999999999999},"max":{"1min":1.246,"5min":1.5640000000000001,"15min":1.7589999999999999},"last":0.64300000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.03,"15min":1.0009999999999999},"min":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.27600000000000002},"max":{"1min":1.617,"5min":1.619,"15min":1.9219999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.05,"15min":1.0329999999999999},"min":{"1min":0.65600000000000003,"5min":0.64200000000000002,"15min":0.32400000000000001},"max":{"1min":1.3160000000000001,"5min":1.5700000000000001,"15min":1.625},"last":0.97899999999999998}]},{"osd":5,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":1.0109999999999999,"15min":1.0349999999999999},"min":{"1min":0.66600000000000004,"5min":0.66300000000000003,"15min":0.29599999999999999},"max":{"1min":1.194,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.1040000000000001,"15min":1.133},"min":{"1min":0.78500000000000003,"5min":0.65300000000000002,"15min":0.34399999999999997},"max":{"1min":1.3089999999999999,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":0.94499999999999995}]},{"osd":7,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.179,"15min":1.149},"min":{"1min":0.82999999999999996,"5min":0.79300000000000004,"15min":0.37},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":2.0350000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2030000000000001,"15min":1.1950000000000001},"min":{"1min":0.90200000000000002,"5min":0.82099999999999995,"15min":0.46800000000000003},"max":{"1min":1.754,"5min":2.274,"15min":2.274},"last":1.2410000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410352,"kb_used_data":316712,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":93961488,"statfs":{"total":96636764160,"available":96216563712,"internally_reserved":0,"allocated":324313088,"data_stored":316662693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79500000000000004,"15min":0.80900000000000005},"min":{"1min":0.53200000000000003,"5min":0.47199999999999998,"15min":0.437},"max":{"1min":1.1819999999999999,"5min":1.3700000000000001,"15min":1.5649999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.83299999999999996,"15min":0.84099999999999997},"min":{"1min":0.58899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2390000000000001,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.76000000000000001}]},{"osd":1,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.53300000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.034,"5min":1.3069999999999999,"15min":2.4910000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.86299999999999999,"15min":0.84999999999999998},"min":{"1min":0.63,"5min":0.433,"15min":0.433},"max":{"1min":1.2030000000000001,"5min":1.54,"15min":1.9730000000000001},"last":0.64000000000000001}]},{"osd":2,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.67300000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1950000000000001,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":1.0600000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.55600000000000005},"max":{"1min":1.2170000000000001,"5min":4.6879999999999997,"15min":4.6879999999999997},"last":0.89100000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.92500000000000004,"15min":0.94399999999999995},"min":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.284,"5min":2.266,"15min":2.5819999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.94999999999999996,"15min":0.90700000000000003},"min":{"1min":0.78200000000000003,"5min":0.54300000000000004,"15min":0.53500000000000003},"max":{"1min":1.411,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.94799999999999995}]},{"osd":5,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.97099999999999997,"15min":1.0029999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.129,"5min":2.1779999999999999,"15min":2.3180000000000001},"last":1.075},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.0880000000000001,"15min":1.0620000000000001},"min":{"1min":1.0660000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3959999999999999,"5min":3.173,"15min":3.173},"last":1.2150000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":1.0509999999999999,"15min":1.0880000000000001},"min":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.59299999999999997},"max":{"1min":1.131,"5min":2.2709999999999999,"15min":2.73},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.093,"5min":1.1080000000000001,"15min":1.0640000000000001},"min":{"1min":0.83999999999999997,"5min":0.748,"15min":0.56499999999999995},"max":{"1min":1.262,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":1.123}]},{"osd":7,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.087,"15min":1.0700000000000001},"min":{"1min":0.70699999999999996,"5min":0.68899999999999995,"15min":0.64900000000000002},"max":{"1min":1.196,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.02},{"interface":"front","average":{"1min":1.204,"5min":1.175,"15min":1.131},"min":{"1min":0.92500000000000004,"5min":0.70299999999999996,"15min":0.65300000000000002},"max":{"1min":1.4950000000000001,"5min":2.375,"15min":3.0649999999999999},"last":1.1699999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542404,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":296824,"kb_used_data":239280,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94075016,"statfs":{"total":96636764160,"available":96332816384,"internally_reserved":0,"allocated":245022720,"data_stored":240236236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.48599999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.22700000000000001},"max":{"1min":1.6930000000000001,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.58199999999999996,"15min":0.53200000000000003},"min":{"1min":0.40000000000000002,"5min":0.33200000000000002,"15min":0.28699999999999998},"max":{"1min":2.0619999999999998,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":0.47399999999999998}]},{"osd":1,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.33000000000000002,"5min":0.317,"15min":0.27900000000000003},"max":{"1min":3.02,"5min":3.5059999999999998,"15min":4.5019999999999998},"last":0.497},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.39100000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.0299999999999998,"5min":3.5430000000000001,"15min":3.5430000000000001},"last":0.38200000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.76000000000000001,"15min":0.60399999999999998},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26100000000000001},"max":{"1min":8.9550000000000001,"5min":18.747,"15min":18.747},"last":0.36599999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":0.85499999999999998,"15min":0.70099999999999996},"min":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38300000000000001},"max":{"1min":8.9900000000000002,"5min":18.863,"15min":18.863},"last":0.45400000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85999999999999999},"min":{"1min":0.626,"5min":0.56999999999999995,"15min":0.53600000000000003},"max":{"1min":3.0760000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.321,"5min":0.96299999999999997,"15min":0.89700000000000002},"min":{"1min":0.73299999999999998,"5min":0.51100000000000001,"15min":0.48299999999999998},"max":{"1min":9.6289999999999996,"5min":9.6289999999999996,"15min":9.6289999999999996},"last":0.78200000000000003}]},{"osd":5,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":0.876,"15min":0.85199999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":9.7289999999999992,"5min":9.7289999999999992,"15min":9.7289999999999992},"last":0.82599999999999996},{"interface":"front","average":{"1min":5.577,"5min":1.9490000000000001,"15min":1.2},"min":{"1min":0.626,"5min":0.60399999999999998,"15min":0.51900000000000002},"max":{"1min":91.072999999999993,"5min":91.072999999999993,"15min":91.072999999999993},"last":0.64800000000000002}]},{"osd":6,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.048,"15min":0.91700000000000004},"min":{"1min":0.59299999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":5.4640000000000004,"5min":19.126000000000001,"15min":19.126000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.2509999999999999,"15min":1.006},"min":{"1min":0.72499999999999998,"5min":0.60299999999999998,"15min":0.59399999999999997},"max":{"1min":21.469999999999999,"5min":21.469999999999999,"15min":21.469999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":0.94699999999999995,"15min":0.89600000000000002},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.47099999999999997},"max":{"1min":9.6869999999999994,"5min":9.6869999999999994,"15min":9.6869999999999994},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.1479999999999999,"15min":0.96799999999999997},"min":{"1min":0.80800000000000005,"5min":0.69799999999999995,"15min":0.53200000000000003},"max":{"1min":5.524,"5min":19.050999999999998,"15min":19.050999999999998},"last":0.90900000000000003}]}]},{"osd":0,"up_from":14,"seq":60129542405,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427176,"kb_used_data":337696,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":93944664,"statfs":{"total":96636764160,"available":96199335936,"internally_reserved":0,"allocated":345800704,"data_stored":338053520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":91617736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.46600000000000003,"15min":0.48399999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.248},"max":{"1min":0.69099999999999995,"5min":0.78700000000000003,"15min":8.8059999999999992},"last":0.376},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.504,"15min":0.61099999999999999},"min":{"1min":0.30099999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69299999999999995,"5min":0.83899999999999997,"15min":35.084000000000003},"last":0.55900000000000005}]},{"osd":2,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51500000000000001,"15min":0.63500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.28999999999999998},"max":{"1min":0.68000000000000005,"5min":0.93100000000000005,"15min":34.741},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58099999999999996,"15min":0.69299999999999995},"min":{"1min":0.42999999999999999,"5min":0.37,"15min":0.35299999999999998},"max":{"1min":0.72999999999999998,"5min":1.0049999999999999,"15min":35.058999999999997},"last":0.47099999999999997}]},{"osd":3,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.35899999999999999,"5min":0.32000000000000001,"15min":0.29499999999999998},"max":{"1min":0.76000000000000001,"5min":0.877,"15min":8.8469999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.629},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.88800000000000001,"15min":11.916},"last":0.52900000000000003}]},{"osd":4,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.85499999999999998,"15min":0.79700000000000004},"min":{"1min":0.66600000000000004,"5min":0.55500000000000005,"15min":0.373},"max":{"1min":1.0389999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.82499999999999996,"15min":0.92800000000000005},"min":{"1min":0.63900000000000001,"5min":0.57099999999999995,"15min":0.39900000000000002},"max":{"1min":1.1419999999999999,"5min":1.175,"15min":34.902999999999999},"last":0.76300000000000001}]},{"osd":5,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82099999999999995,"15min":0.84799999999999998},"min":{"1min":0.59499999999999997,"5min":0.51800000000000002,"15min":0.5},"max":{"1min":1.0820000000000001,"5min":1.171,"15min":8.4900000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.90600000000000003,"15min":0.97499999999999998},"min":{"1min":0.78500000000000003,"5min":0.66900000000000004,"15min":0.48799999999999999},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":34.698},"last":0.81200000000000006}]},{"osd":6,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81000000000000005},"min":{"1min":0.64200000000000002,"5min":0.58099999999999996,"15min":0.47799999999999998},"max":{"1min":1.097,"5min":1.1439999999999999,"15min":1.2789999999999999},"last":0.75},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.878,"15min":0.88700000000000001},"min":{"1min":0.64100000000000001,"5min":0.629,"15min":0.50900000000000001},"max":{"1min":1.304,"5min":1.304,"15min":8.8800000000000008},"last":0.70599999999999996}]},{"osd":7,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82499999999999996,"15min":0.84099999999999997},"min":{"1min":0.65000000000000002,"5min":0.53000000000000003,"15min":0.437},"max":{"1min":1.042,"5min":1.1930000000000001,"15min":1.304},"last":0.875},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.88900000000000001,"15min":0.92900000000000005},"min":{"1min":0.70699999999999996,"5min":0.60599999999999998,"15min":0.55900000000000005},"max":{"1min":1.1479999999999999,"5min":1.323,"15min":8.5549999999999997},"last":0.79200000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542403,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462712,"kb_used_data":373232,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":93909128,"statfs":{"total":96636764160,"available":96162947072,"internally_reserved":0,"allocated":382189568,"data_stored":374550682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":91617737},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.88300000000000001,"15min":0.84399999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.41899999999999998},"max":{"1min":1.3129999999999999,"5min":1.627,"15min":1.627},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.87,"5min":0.88900000000000001,"15min":0.84999999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.495},"max":{"1min":1.2649999999999999,"5min":1.468,"15min":1.468},"last":0.79400000000000004}]},{"osd":1,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.82899999999999996,"15min":0.91300000000000003},"min":{"1min":0.52800000000000002,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.3089999999999999,"5min":1.363,"15min":1.746},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.82399999999999995,"15min":0.88},"min":{"1min":0.52700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2969999999999999,"5min":1.3169999999999999,"15min":1.337},"last":0.92300000000000004}]},{"osd":2,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89700000000000002,"15min":0.872},"min":{"1min":0.60299999999999998,"5min":0.53800000000000003,"15min":0.501},"max":{"1min":1.4139999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.21},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.86199999999999999,"15min":0.83899999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.442},"max":{"1min":1.1699999999999999,"5min":1.29,"15min":1.4770000000000001},"last":1.0369999999999999}]},{"osd":3,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.9850000000000001,"15min":1.3180000000000001},"min":{"1min":0.63900000000000001,"5min":0.54400000000000004,"15min":0.504},"max":{"1min":1.3640000000000001,"5min":104.88500000000001,"15min":104.88500000000001},"last":1.252},{"interface":"front","average":{"1min":1.034,"5min":2.0619999999999998,"15min":1.3700000000000001},"min":{"1min":0.71399999999999997,"5min":0.624,"15min":0.60699999999999998},"max":{"1min":1.405,"5min":105.042,"15min":105.042},"last":0.96099999999999997}]},{"osd":4,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0349999999999999,"15min":0.94099999999999995},"min":{"1min":0.69899999999999995,"5min":0.58699999999999997,"15min":0.38500000000000001},"max":{"1min":1.262,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.88},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.077,"15min":1.1220000000000001},"min":{"1min":0.68100000000000005,"5min":0.59799999999999998,"15min":0.57399999999999995},"max":{"1min":1.5229999999999999,"5min":1.542,"15min":1.712},"last":0.59699999999999998}]},{"osd":6,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.96199999999999997,"15min":0.95599999999999996},"min":{"1min":0.77800000000000002,"5min":0.627,"15min":0.35899999999999999},"max":{"1min":1.389,"5min":1.3899999999999999,"15min":1.9410000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.0649999999999999,"15min":1.056},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.48899999999999999},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":2.0299999999999998},"last":0.83799999999999997}]},{"osd":7,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.1080000000000001,"15min":1.095},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.40600000000000003},"max":{"1min":1.357,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.113,"15min":1.1120000000000001},"min":{"1min":0.88,"5min":0.70499999999999996,"15min":0.53300000000000003},"max":{"1min":1.5149999999999999,"5min":1.5660000000000001,"15min":1.7689999999999999},"last":0.99299999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542405,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442132,"kb_used_data":348492,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":93929708,"statfs":{"total":96636764160,"available":96184020992,"internally_reserved":0,"allocated":356855808,"data_stored":348975142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":95877578},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:29 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42999999999999999,"15min":0.436},"min":{"1min":0.26600000000000001,"5min":0.25600000000000001,"15min":0.224},"max":{"1min":0.68100000000000005,"5min":0.77100000000000002,"15min":2.1459999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.46700000000000003,"15min":0.54800000000000004},"min":{"1min":0.32100000000000001,"5min":0.28399999999999997,"15min":0.27200000000000002},"max":{"1min":0.59999999999999998,"5min":0.77900000000000003,"15min":21.495999999999999},"last":0.48199999999999998}]},{"osd":1,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.52100000000000002,"15min":0.53200000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72999999999999998,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48999999999999999,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.314,"15min":0.30399999999999999},"max":{"1min":0.68500000000000005,"5min":1.2450000000000001,"15min":15.102},"last":0.53000000000000003}]},{"osd":2,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53900000000000003,"15min":0.60899999999999999},"min":{"1min":0.33300000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.75800000000000001,"5min":1.044,"15min":21.321000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.495,"5min":0.39000000000000001,"15min":0.379},"max":{"1min":0.77000000000000002,"5min":0.97099999999999997,"15min":1.681},"last":0.45700000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.80600000000000005,"15min":0.90000000000000002},"min":{"1min":0.57999999999999996,"5min":0.49199999999999999,"15min":0.48499999999999999},"max":{"1min":1.21,"5min":1.21,"15min":21.382999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.879,"15min":0.91900000000000004},"min":{"1min":0.73799999999999999,"5min":0.60399999999999998,"15min":0.46700000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":15.231999999999999},"last":0.71299999999999997}]},{"osd":5,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.84399999999999997,"15min":0.89500000000000002},"min":{"1min":0.58699999999999997,"5min":0.54200000000000004,"15min":0.47799999999999998},"max":{"1min":1.0600000000000001,"5min":1.1879999999999999,"15min":15.273999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.90500000000000003,"15min":0.95399999999999996},"min":{"1min":0.69499999999999995,"5min":0.67700000000000005,"15min":0.57499999999999996},"max":{"1min":1.1619999999999999,"5min":1.2170000000000001,"15min":15.359999999999999},"last":0.84399999999999997}]},{"osd":6,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79400000000000004,"15min":0.81699999999999995},"min":{"1min":0.61099999999999999,"5min":0.53200000000000003,"15min":0.51100000000000001},"max":{"1min":1.099,"5min":1.296,"15min":1.5620000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81599999999999995,"15min":0.89100000000000001},"min":{"1min":0.66000000000000003,"5min":0.55700000000000005,"15min":0.498},"max":{"1min":1.04,"5min":1.2210000000000001,"15min":15.313000000000001},"last":0.88400000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.85199999999999998,"15min":0.92700000000000005},"min":{"1min":0.65000000000000002,"5min":0.58799999999999997,"15min":0.55200000000000005},"max":{"1min":1.0620000000000001,"5min":1.137,"15min":21.428999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.85299999999999998,"15min":0.879},"min":{"1min":0.81699999999999995,"5min":0.56899999999999995,"15min":0.56799999999999995},"max":{"1min":1.1180000000000001,"5min":1.28,"15min":1.3100000000000001},"last":0.80900000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542403,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":296616,"kb_used_data":229408,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94075224,"statfs":{"total":96636764160,"available":96333029376,"internally_reserved":0,"allocated":234913792,"data_stored":230197311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.68500000000000005,"15min":0.54400000000000004},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.62,"5min":15.343999999999999,"15min":15.343999999999999},"last":0.434},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.73299999999999998,"15min":0.59999999999999998},"min":{"1min":0.34100000000000003,"5min":0.313,"15min":0.29899999999999999},"max":{"1min":0.66800000000000004,"5min":14.68,"15min":14.68},"last":0.45200000000000001}]},{"osd":2,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.66700000000000004,"15min":0.59799999999999998},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68999999999999995,"5min":8.1359999999999992,"15min":8.1359999999999992},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.33900000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69899999999999995,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.65100000000000002,"15min":0.59599999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.81200000000000006,"5min":8.0920000000000005,"15min":8.0920000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59399999999999997,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.35199999999999998,"15min":0.32000000000000001},"max":{"1min":0.79500000000000004,"5min":5.9199999999999999,"15min":5.9199999999999999},"last":0.504}]},{"osd":4,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":1.133,"15min":0.94099999999999995},"min":{"1min":0.629,"5min":0.60499999999999998,"15min":0.41799999999999998},"max":{"1min":1.2010000000000001,"5min":21.27,"15min":21.27},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.93899999999999995,"15min":0.90700000000000003},"min":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.53600000000000003},"max":{"1min":1.3420000000000001,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.82199999999999995}]},{"osd":5,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.83099999999999996,"15min":0.82599999999999996},"min":{"1min":0.63900000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0089999999999999,"5min":1.355,"15min":1.355},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.872,"15min":0.85599999999999998},"min":{"1min":0.68300000000000005,"5min":0.499,"15min":0.46200000000000002},"max":{"1min":1.1599999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.66300000000000003}]},{"osd":6,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85599999999999998,"15min":0.83999999999999997},"min":{"1min":0.70999999999999996,"5min":0.61399999999999999,"15min":0.51300000000000001},"max":{"1min":1.111,"5min":1.29,"15min":1.4450000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.88400000000000001,"15min":0.88500000000000001},"min":{"1min":0.64600000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.246,"5min":1.323,"15min":1.468},"last":0.75600000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.871,"15min":0.85299999999999998},"min":{"1min":0.67800000000000005,"5min":0.629,"15min":0.496},"max":{"1min":1.304,"5min":1.304,"15min":1.4299999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.86399999999999999,"15min":0.874},"min":{"1min":0.76200000000000001,"5min":0.63100000000000001,"15min":0.443},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2410000000000001},"last":0.88200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":74309632,"data_stored":74293305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96493568,"data_stored":96469936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42975232,"data_stored":42953409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":85110784,"data_stored":85086454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":88555520,"data_stored":88540092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":85549056,"data_stored":85526018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":107171840,"data_stored":107146531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":128835584,"data_stored":128819325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":270266368,"data_stored":262861049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":137789440,"data_stored":133418577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":201416704,"data_stored":196974029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":271114240,"data_stored":263579890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":235126784,"data_stored":227813803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":296009728,"data_stored":288715866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":192012288,"data_stored":187336486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":277991424,"data_stored":270761382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:56:15.140 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:56:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb29a46640 1 Processor -- start 2023-10-30T12:56:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb29a46640 1 -- start start 2023-10-30T12:56:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb29a46640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feb2415d6f0 0x7feb2415a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb29a46640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb240cf160 0x7feb240cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb29a46640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb240cb800 con 0x7feb241538a0 2023-10-30T12:56:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb29a46640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb240cb330 con 0x7feb2415d6f0 2023-10-30T12:56:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb29a46640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb240cad70 con 0x7feb240cf160 2023-10-30T12:56:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb1bfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feb2415d6f0 0x7feb2415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb1bfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feb2415d6f0 0x7feb2415a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52190/0 (socket says 172.21.15.12:52190) 2023-10-30T12:56:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb1bfff640 1 -- 172.21.15.12:0/2223658603 learned_addr learned my addr 172.21.15.12:0/2223658603 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb237fe640 1 --2- 172.21.15.12:0/2223658603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb240cf160 0x7feb240cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb1bfff640 1 -- 172.21.15.12:0/2223658603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb240cf160 msgr2=0x7feb240cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb1bfff640 1 --2- 172.21.15.12:0/2223658603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb240cf160 0x7feb240cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb1bfff640 1 -- 172.21.15.12:0/2223658603 >> v1:172.21.15.154:6789/0 conn(0x7feb241538a0 legacy=0x7feb240ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb1bfff640 1 -- 172.21.15.12:0/2223658603 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb240cb050 con 0x7feb2415d6f0 2023-10-30T12:56:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.315+0000 7feb237fe640 1 --2- 172.21.15.12:0/2223658603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb240cf160 0x7feb240cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:56:15.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.319+0000 7feb1bfff640 1 --2- 172.21.15.12:0/2223658603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb2415d6f0 0x7feb2415a090 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009870 tx=0x7feb0c01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be32b184eba6a07 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:15.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.319+0000 7feb227fc640 1 -- 172.21.15.12:0/2223658603 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0a89b0 con 0x7feb2415d6f0 2023-10-30T12:56:15.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.319+0000 7feb227fc640 1 -- 172.21.15.12:0/2223658603 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0c0a8b10 con 0x7feb2415d6f0 2023-10-30T12:56:15.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.319+0000 7feb227fc640 1 -- 172.21.15.12:0/2223658603 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0a8ea0 con 0x7feb2415d6f0 2023-10-30T12:56:15.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.319+0000 7feb29a46640 1 -- 172.21.15.12:0/2223658603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb2415d6f0 msgr2=0x7feb2415a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:15.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.319+0000 7feb29a46640 1 --2- 172.21.15.12:0/2223658603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb2415d6f0 0x7feb2415a090 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009870 tx=0x7feb0c01ecf0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.319+0000 7feb29a46640 1 -- 172.21.15.12:0/2223658603 shutdown_connections 2023-10-30T12:56:15.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.319+0000 7feb29a46640 1 --2- 172.21.15.12:0/2223658603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb240cf160 0x7feb240cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.319+0000 7feb29a46640 1 --2- 172.21.15.12:0/2223658603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb2415d6f0 0x7feb2415a090 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.319+0000 7feb29a46640 1 -- 172.21.15.12:0/2223658603 >> 172.21.15.12:0/2223658603 conn(0x7feb2406bfe0 msgr2=0x7feb2406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:15.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.319+0000 7feb29a46640 1 -- 172.21.15.12:0/2223658603 shutdown_connections 2023-10-30T12:56:15.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.319+0000 7feb29a46640 1 -- 172.21.15.12:0/2223658603 wait complete. 2023-10-30T12:56:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb29a46640 1 Processor -- start 2023-10-30T12:56:15.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb29a46640 1 -- start start 2023-10-30T12:56:15.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb29a46640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb240cf160 0x7feb240b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:15.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb29a46640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb241538a0 0x7feb241e3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:15.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb29a46640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb2415d6f0 0x7feb240ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:15.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb29a46640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb240b3da0 con 0x7feb2415d6f0 2023-10-30T12:56:15.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb29a46640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb241ef410 con 0x7feb241538a0 2023-10-30T12:56:15.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb29a46640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb240b3190 con 0x7feb240cf160 2023-10-30T12:56:15.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb237fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb2415d6f0 0x7feb240ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:15.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb237fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb2415d6f0 0x7feb240ce260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52196/0 (socket says 172.21.15.12:52196) 2023-10-30T12:56:15.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb237fe640 1 -- 172.21.15.12:0/1273668199 learned_addr learned my addr 172.21.15.12:0/1273668199 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:15.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb22ffd640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb240cf160 0x7feb240b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:15.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb1bfff640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb241538a0 0x7feb241e3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:15.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb237fe640 1 -- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb240cf160 msgr2=0x7feb240b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:15.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb237fe640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb240cf160 0x7feb240b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb237fe640 1 -- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb241538a0 msgr2=0x7feb241e3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:15.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb237fe640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb241538a0 0x7feb241e3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb237fe640 1 -- 172.21.15.12:0/1273668199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c0094d0 con 0x7feb2415d6f0 2023-10-30T12:56:15.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb22ffd640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb240cf160 0x7feb240b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:15.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb1bfff640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb241538a0 0x7feb241e3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:15.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb237fe640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb2415d6f0 0x7feb240ce260 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00ce40 tx=0x7feb1c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:15.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb20ff9640 1 -- 172.21.15.12:0/1273668199 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c00d840 con 0x7feb2415d6f0 2023-10-30T12:56:15.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb20ff9640 1 -- 172.21.15.12:0/1273668199 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c00d9a0 con 0x7feb2415d6f0 2023-10-30T12:56:15.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb20ff9640 1 -- 172.21.15.12:0/1273668199 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c00dd50 con 0x7feb2415d6f0 2023-10-30T12:56:15.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb29a46640 1 -- 172.21.15.12:0/1273668199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb240b3470 con 0x7feb2415d6f0 2023-10-30T12:56:15.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.323+0000 7feb29a46640 1 -- 172.21.15.12:0/1273668199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb240b3a30 con 0x7feb2415d6f0 2023-10-30T12:56:15.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.327+0000 7feb29a46640 1 -- 172.21.15.12:0/1273668199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb240b2a50 con 0x7feb2415d6f0 2023-10-30T12:56:15.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.327+0000 7feb20ff9640 1 -- 172.21.15.12:0/1273668199 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feb1c01d360 con 0x7feb2415d6f0 2023-10-30T12:56:15.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.327+0000 7feb20ff9640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb040664a0 0x7feb04068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:15.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.327+0000 7feb20ff9640 1 -- 172.21.15.12:0/1273668199 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7feb1c084d00 con 0x7feb2415d6f0 2023-10-30T12:56:15.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.327+0000 7feb22ffd640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb040664a0 0x7feb04068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:15.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.331+0000 7feb22ffd640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb040664a0 0x7feb04068960 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7feb14003ec0 tx=0x7feb14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:15.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.331+0000 7feb20ff9640 1 -- 172.21.15.12:0/1273668199 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feb1c051d10 con 0x7feb2415d6f0 2023-10-30T12:56:15.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.487+0000 7feb29a46640 1 -- 172.21.15.12:0/1273668199 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb24159b70 con 0x7feb040664a0 2023-10-30T12:56:15.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.507+0000 7feb20ff9640 1 -- 172.21.15.12:0/1273668199 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292641 (secure 0 0 0) 0x7feb14003ab0 con 0x7feb040664a0 2023-10-30T12:56:15.515 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:56:15.517 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-10-30T12:56:15.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.511+0000 7feb29a46640 1 -- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb040664a0 msgr2=0x7feb04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:15.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.511+0000 7feb29a46640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb040664a0 0x7feb04068960 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7feb14003ec0 tx=0x7feb14009040 comp rx=0 tx=0).stop 2023-10-30T12:56:15.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.511+0000 7feb29a46640 1 -- 172.21.15.12:0/1273668199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb2415d6f0 msgr2=0x7feb240ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:15.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.511+0000 7feb29a46640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb2415d6f0 0x7feb240ce260 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00ce40 tx=0x7feb1c00a710 comp rx=0 tx=0).stop 2023-10-30T12:56:15.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.511+0000 7feb29a46640 1 -- 172.21.15.12:0/1273668199 shutdown_connections 2023-10-30T12:56:15.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.511+0000 7feb29a46640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb2415d6f0 0x7feb240ce260 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.511+0000 7feb29a46640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feb241538a0 0x7feb241e3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.511+0000 7feb29a46640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7feb040664a0 0x7feb04068960 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.511+0000 7feb29a46640 1 --2- 172.21.15.12:0/1273668199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feb240cf160 0x7feb240b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.511+0000 7feb29a46640 1 -- 172.21.15.12:0/1273668199 >> 172.21.15.12:0/1273668199 conn(0x7feb2406bfe0 msgr2=0x7feb240cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:15.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.511+0000 7feb29a46640 1 -- 172.21.15.12:0/1273668199 shutdown_connections 2023-10-30T12:56:15.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.511+0000 7feb29a46640 1 -- 172.21.15.12:0/1273668199 wait complete. 2023-10-30T12:56:15.536 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":690,"stamp":"2023-10-30T12:56:15.409746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1271508492,"num_objects":9804,"num_object_clones":0,"num_object_copies":19608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9804,"num_whiteouts":0,"num_read":5687,"num_read_kb":1040706,"num_write":52424,"num_write_kb":2300570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55711,"ondisk_log_size":55711,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3201824,"kb_used_data":2536096,"kb_used_omap":12,"kb_used_meta":665651,"kb_avail":751772896,"statfs":{"total":773094113280,"available":769815445504,"internally_reserved":0,"allocated":2596962304,"data_stored":2543947272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12729,"internal_metadata":681627207},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":66,"apply_latency_ms":66,"commit_latency_ns":66000000,"apply_latency_ns":66000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50689313,"num_objects":1715,"num_object_clones":0,"num_object_copies":3430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1826,"num_write_kb":50621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004527"},"pg_stats":[{"pgid":"3.3f","version":"55'704","reported_seq":2869,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.052574+0000","last_change":"2023-10-30T12:42:02.472363+0000","last_active":"2023-10-30T12:56:07.052574+0000","last_peered":"2023-10-30T12:56:07.052574+0000","last_clean":"2023-10-30T12:56:07.052574+0000","last_became_active":"2023-10-30T12:35:06.222724+0000","last_became_peered":"2023-10-30T12:35:06.222724+0000","last_unstale":"2023-10-30T12:56:07.052574+0000","last_undegraded":"2023-10-30T12:56:07.052574+0000","last_fullsized":"2023-10-30T12:56:07.052574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:27:39.311531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059524979999999996,"stat_sum":{"num_bytes":14367774,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":73,"num_read_kb":8207,"num_write":683,"num_write_kb":25650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"55'686","reported_seq":2848,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.014947+0000","last_change":"2023-10-30T12:42:02.711331+0000","last_active":"2023-10-30T12:56:07.014947+0000","last_peered":"2023-10-30T12:56:07.014947+0000","last_clean":"2023-10-30T12:56:07.014947+0000","last_became_active":"2023-10-30T12:35:06.182441+0000","last_became_peered":"2023-10-30T12:35:06.182441+0000","last_unstale":"2023-10-30T12:56:07.014947+0000","last_undegraded":"2023-10-30T12:56:07.014947+0000","last_fullsized":"2023-10-30T12:56:07.014947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:00:23.194349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026942979999999999,"stat_sum":{"num_bytes":14474399,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":70,"num_read_kb":10,"num_write":665,"num_write_kb":36839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"55'669","reported_seq":2841,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.004471+0000","last_change":"2023-10-30T12:42:02.724489+0000","last_active":"2023-10-30T12:56:07.004471+0000","last_peered":"2023-10-30T12:56:07.004471+0000","last_clean":"2023-10-30T12:56:07.004471+0000","last_became_active":"2023-10-30T12:35:06.180132+0000","last_became_peered":"2023-10-30T12:35:06.180132+0000","last_unstale":"2023-10-30T12:56:07.004471+0000","last_undegraded":"2023-10-30T12:56:07.004471+0000","last_fullsized":"2023-10-30T12:56:07.004471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:12.922855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003537278,"stat_sum":{"num_bytes":22206444,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":80,"num_read_kb":12305,"num_write":639,"num_write_kb":42559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"55'658","reported_seq":2807,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.150943+0000","last_change":"2023-10-30T12:42:02.742917+0000","last_active":"2023-10-30T12:56:07.150943+0000","last_peered":"2023-10-30T12:56:07.150943+0000","last_clean":"2023-10-30T12:56:07.150943+0000","last_became_active":"2023-10-30T12:35:06.213987+0000","last_became_peered":"2023-10-30T12:35:06.213987+0000","last_unstale":"2023-10-30T12:56:07.150943+0000","last_undegraded":"2023-10-30T12:56:07.150943+0000","last_fullsized":"2023-10-30T12:56:07.150943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:39:55.675219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013528057,"stat_sum":{"num_bytes":18334048,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":57,"num_read_kb":8205,"num_write":618,"num_write_kb":28313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"55'637","reported_seq":2778,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.021805+0000","last_change":"2023-10-30T12:42:02.470142+0000","last_active":"2023-10-30T12:56:07.021805+0000","last_peered":"2023-10-30T12:56:07.021805+0000","last_clean":"2023-10-30T12:56:07.021805+0000","last_became_active":"2023-10-30T12:35:06.214964+0000","last_became_peered":"2023-10-30T12:35:06.214964+0000","last_unstale":"2023-10-30T12:56:07.021805+0000","last_undegraded":"2023-10-30T12:56:07.021805+0000","last_fullsized":"2023-10-30T12:56:07.021805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:56:51.594442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046834600000000004,"stat_sum":{"num_bytes":8551162,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":51,"num_read_kb":4112,"num_write":602,"num_write_kb":25707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"55'662","reported_seq":2813,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.149801+0000","last_change":"2023-10-30T12:42:02.747177+0000","last_active":"2023-10-30T12:56:07.149801+0000","last_peered":"2023-10-30T12:56:07.149801+0000","last_clean":"2023-10-30T12:56:07.149801+0000","last_became_active":"2023-10-30T12:35:06.214219+0000","last_became_peered":"2023-10-30T12:35:06.214219+0000","last_unstale":"2023-10-30T12:56:07.149801+0000","last_undegraded":"2023-10-30T12:56:07.149801+0000","last_fullsized":"2023-10-30T12:56:07.149801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:36:56.070585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015862434000000002,"stat_sum":{"num_bytes":14711969,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":61,"num_read_kb":16,"num_write":631,"num_write_kb":33150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"55'668","reported_seq":2809,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.029181+0000","last_change":"2023-10-30T12:42:02.467959+0000","last_active":"2023-10-30T12:56:07.029181+0000","last_peered":"2023-10-30T12:56:07.029181+0000","last_clean":"2023-10-30T12:56:07.029181+0000","last_became_active":"2023-10-30T12:35:06.216468+0000","last_became_peered":"2023-10-30T12:35:06.216468+0000","last_unstale":"2023-10-30T12:56:07.029181+0000","last_undegraded":"2023-10-30T12:56:07.029181+0000","last_fullsized":"2023-10-30T12:56:07.029181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:19:43.632233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031309480000000002,"stat_sum":{"num_bytes":9417687,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":52,"num_read_kb":12216,"num_write":636,"num_write_kb":35921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"55'605","reported_seq":2751,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.153162+0000","last_change":"2023-10-30T12:42:02.241155+0000","last_active":"2023-10-30T12:56:07.153162+0000","last_peered":"2023-10-30T12:56:07.153162+0000","last_clean":"2023-10-30T12:56:07.153162+0000","last_became_active":"2023-10-30T12:35:06.214384+0000","last_became_peered":"2023-10-30T12:35:06.214384+0000","last_unstale":"2023-10-30T12:56:07.153162+0000","last_undegraded":"2023-10-30T12:56:07.153162+0000","last_fullsized":"2023-10-30T12:56:07.153162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:44:07.845577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045552559999999997,"stat_sum":{"num_bytes":9683176,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":56,"num_read_kb":15,"num_write":587,"num_write_kb":23249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"55'660","reported_seq":2822,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.060712+0000","last_change":"2023-10-30T12:42:02.473886+0000","last_active":"2023-10-30T12:56:07.060712+0000","last_peered":"2023-10-30T12:56:07.060712+0000","last_clean":"2023-10-30T12:56:07.060712+0000","last_became_active":"2023-10-30T12:35:06.222692+0000","last_became_peered":"2023-10-30T12:35:06.222692+0000","last_unstale":"2023-10-30T12:56:07.060712+0000","last_undegraded":"2023-10-30T12:56:07.060712+0000","last_fullsized":"2023-10-30T12:56:07.060712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:12:07.552579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074617599999999996,"stat_sum":{"num_bytes":5453765,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":72,"num_read_kb":4115,"num_write":623,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"55'653","reported_seq":2807,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.014218+0000","last_change":"2023-10-30T12:42:02.739661+0000","last_active":"2023-10-30T12:56:07.014218+0000","last_peered":"2023-10-30T12:56:07.014218+0000","last_clean":"2023-10-30T12:56:07.014218+0000","last_became_active":"2023-10-30T12:35:06.215384+0000","last_became_peered":"2023-10-30T12:35:06.215384+0000","last_unstale":"2023-10-30T12:56:07.014218+0000","last_undegraded":"2023-10-30T12:56:07.014218+0000","last_fullsized":"2023-10-30T12:56:07.014218+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:02:59.748923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080059940000000007,"stat_sum":{"num_bytes":17838264,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":64,"num_read_kb":28,"num_write":624,"num_write_kb":19661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"55'661","reported_seq":2818,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.991403+0000","last_change":"2023-10-30T12:42:02.739462+0000","last_active":"2023-10-30T12:56:06.991403+0000","last_peered":"2023-10-30T12:56:06.991403+0000","last_clean":"2023-10-30T12:56:06.991403+0000","last_became_active":"2023-10-30T12:35:06.215355+0000","last_became_peered":"2023-10-30T12:35:06.215355+0000","last_unstale":"2023-10-30T12:56:06.991403+0000","last_undegraded":"2023-10-30T12:56:06.991403+0000","last_fullsized":"2023-10-30T12:56:06.991403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:04:51.314044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011778769,"stat_sum":{"num_bytes":10379280,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":67,"num_read_kb":8209,"num_write":639,"num_write_kb":25731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"55'640","reported_seq":2795,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.010644+0000","last_change":"2023-10-30T12:42:02.253484+0000","last_active":"2023-10-30T12:56:07.010644+0000","last_peered":"2023-10-30T12:56:07.010644+0000","last_clean":"2023-10-30T12:56:07.010644+0000","last_became_active":"2023-10-30T12:35:06.196503+0000","last_became_peered":"2023-10-30T12:35:06.196503+0000","last_unstale":"2023-10-30T12:56:07.010644+0000","last_undegraded":"2023-10-30T12:56:07.010644+0000","last_fullsized":"2023-10-30T12:56:07.010644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:33:54.826649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030385149999999999,"stat_sum":{"num_bytes":16049079,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":65,"num_read_kb":7648,"num_write":614,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"55'689","reported_seq":2856,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.137866+0000","last_change":"2023-10-30T12:42:02.241621+0000","last_active":"2023-10-30T12:56:07.137866+0000","last_peered":"2023-10-30T12:56:07.137866+0000","last_clean":"2023-10-30T12:56:07.137866+0000","last_became_active":"2023-10-30T12:35:06.195408+0000","last_became_peered":"2023-10-30T12:35:06.195408+0000","last_unstale":"2023-10-30T12:56:07.137866+0000","last_undegraded":"2023-10-30T12:56:07.137866+0000","last_fullsized":"2023-10-30T12:56:07.137866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:06:33.968324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004560812,"stat_sum":{"num_bytes":29038845,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":77,"num_read_kb":9808,"num_write":655,"num_write_kb":48120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"55'657","reported_seq":2819,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.112877+0000","last_change":"2023-10-30T12:42:02.722894+0000","last_active":"2023-10-30T12:56:07.112877+0000","last_peered":"2023-10-30T12:56:07.112877+0000","last_clean":"2023-10-30T12:56:07.112877+0000","last_became_active":"2023-10-30T12:35:06.182573+0000","last_became_peered":"2023-10-30T12:35:06.182573+0000","last_unstale":"2023-10-30T12:56:07.112877+0000","last_undegraded":"2023-10-30T12:56:07.112877+0000","last_fullsized":"2023-10-30T12:56:07.112877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:42:24.313362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003020062,"stat_sum":{"num_bytes":14507519,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":71,"num_read_kb":4113,"num_write":633,"num_write_kb":31547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"55'673","reported_seq":2827,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.046529+0000","last_change":"2023-10-30T12:42:02.710474+0000","last_active":"2023-10-30T12:56:07.046529+0000","last_peered":"2023-10-30T12:56:07.046529+0000","last_clean":"2023-10-30T12:56:07.046529+0000","last_became_active":"2023-10-30T12:35:06.185290+0000","last_became_peered":"2023-10-30T12:35:06.185290+0000","last_unstale":"2023-10-30T12:56:07.046529+0000","last_undegraded":"2023-10-30T12:56:07.046529+0000","last_fullsized":"2023-10-30T12:56:07.046529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:18:23.116561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002282292,"stat_sum":{"num_bytes":14565906,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":63,"num_read_kb":11730,"num_write":637,"num_write_kb":27497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"55'678","reported_seq":2828,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.129521+0000","last_change":"2023-10-30T12:42:02.748073+0000","last_active":"2023-10-30T12:56:07.129521+0000","last_peered":"2023-10-30T12:56:07.129521+0000","last_clean":"2023-10-30T12:56:07.129521+0000","last_became_active":"2023-10-30T12:35:06.213627+0000","last_became_peered":"2023-10-30T12:35:06.213627+0000","last_unstale":"2023-10-30T12:56:07.129521+0000","last_undegraded":"2023-10-30T12:56:07.129521+0000","last_fullsized":"2023-10-30T12:56:07.129521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:56:05.792085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014095760000000001,"stat_sum":{"num_bytes":9785326,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":59,"num_read_kb":4114,"num_write":638,"num_write_kb":31646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"55'710","reported_seq":2863,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.066514+0000","last_change":"2023-10-30T12:42:02.745783+0000","last_active":"2023-10-30T12:56:07.066514+0000","last_peered":"2023-10-30T12:56:07.066514+0000","last_clean":"2023-10-30T12:56:07.066514+0000","last_became_active":"2023-10-30T12:35:06.221696+0000","last_became_peered":"2023-10-30T12:35:06.221696+0000","last_unstale":"2023-10-30T12:56:07.066514+0000","last_undegraded":"2023-10-30T12:56:07.066514+0000","last_fullsized":"2023-10-30T12:56:07.066514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:26:40.837242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015743672,"stat_sum":{"num_bytes":28263392,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":62,"num_read_kb":16401,"num_write":680,"num_write_kb":59327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"55'660","reported_seq":2833,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.992539+0000","last_change":"2023-10-30T12:42:02.242356+0000","last_active":"2023-10-30T12:56:06.992539+0000","last_peered":"2023-10-30T12:56:06.992539+0000","last_clean":"2023-10-30T12:56:06.992539+0000","last_became_active":"2023-10-30T12:35:06.198074+0000","last_became_peered":"2023-10-30T12:35:06.198074+0000","last_unstale":"2023-10-30T12:56:06.992539+0000","last_undegraded":"2023-10-30T12:56:06.992539+0000","last_fullsized":"2023-10-30T12:56:06.992539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:55:38.021093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023389270000000002,"stat_sum":{"num_bytes":28742059,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":82,"num_read_kb":27,"num_write":629,"num_write_kb":39704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"55'675","reported_seq":2826,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.023088+0000","last_change":"2023-10-30T12:42:02.473536+0000","last_active":"2023-10-30T12:56:07.023088+0000","last_peered":"2023-10-30T12:56:07.023088+0000","last_clean":"2023-10-30T12:56:07.023088+0000","last_became_active":"2023-10-30T12:35:06.215935+0000","last_became_peered":"2023-10-30T12:35:06.215935+0000","last_unstale":"2023-10-30T12:56:07.023088+0000","last_undegraded":"2023-10-30T12:56:07.023088+0000","last_fullsized":"2023-10-30T12:56:07.023088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:39:23.851788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062537060000000004,"stat_sum":{"num_bytes":21375724,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":60,"num_read_kb":12305,"num_write":641,"num_write_kb":39315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"55'672","reported_seq":2838,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.992738+0000","last_change":"2023-10-30T12:42:02.739007+0000","last_active":"2023-10-30T12:56:06.992738+0000","last_peered":"2023-10-30T12:56:06.992738+0000","last_clean":"2023-10-30T12:56:06.992738+0000","last_became_active":"2023-10-30T12:35:06.199812+0000","last_became_peered":"2023-10-30T12:35:06.199812+0000","last_unstale":"2023-10-30T12:56:06.992738+0000","last_undegraded":"2023-10-30T12:56:06.992738+0000","last_fullsized":"2023-10-30T12:56:06.992738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:07:11.762372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011375481999999999,"stat_sum":{"num_bytes":6241435,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":75,"num_read_kb":20150,"num_write":638,"num_write_kb":36226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"55'680","reported_seq":2849,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.113501+0000","last_change":"2023-10-30T12:42:02.725299+0000","last_active":"2023-10-30T12:56:07.113501+0000","last_peered":"2023-10-30T12:56:07.113501+0000","last_clean":"2023-10-30T12:56:07.113501+0000","last_became_active":"2023-10-30T12:35:06.216861+0000","last_became_peered":"2023-10-30T12:35:06.216861+0000","last_unstale":"2023-10-30T12:56:07.113501+0000","last_undegraded":"2023-10-30T12:56:07.113501+0000","last_fullsized":"2023-10-30T12:56:07.113501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:58:16.568819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040841849999999997,"stat_sum":{"num_bytes":12253674,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":78,"num_read_kb":8213,"num_write":653,"num_write_kb":26257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"55'705","reported_seq":2854,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.001507+0000","last_change":"2023-10-30T12:42:02.241419+0000","last_active":"2023-10-30T12:56:07.001507+0000","last_peered":"2023-10-30T12:56:07.001507+0000","last_clean":"2023-10-30T12:56:07.001507+0000","last_became_active":"2023-10-30T12:35:46.122517+0000","last_became_peered":"2023-10-30T12:35:46.122517+0000","last_unstale":"2023-10-30T12:56:07.001507+0000","last_undegraded":"2023-10-30T12:56:07.001507+0000","last_fullsized":"2023-10-30T12:56:07.001507+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:10:07.253977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020435240000000001,"stat_sum":{"num_bytes":19343344,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":61,"num_read_kb":21,"num_write":677,"num_write_kb":29921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"55'676","reported_seq":2847,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.095881+0000","last_change":"2023-10-30T12:42:02.241859+0000","last_active":"2023-10-30T12:56:07.095881+0000","last_peered":"2023-10-30T12:56:07.095881+0000","last_clean":"2023-10-30T12:56:07.095881+0000","last_became_active":"2023-10-30T12:35:06.182314+0000","last_became_peered":"2023-10-30T12:35:06.182314+0000","last_unstale":"2023-10-30T12:56:07.095881+0000","last_undegraded":"2023-10-30T12:56:07.095881+0000","last_fullsized":"2023-10-30T12:56:07.095881+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:20:19.153859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051111200000000002,"stat_sum":{"num_bytes":5339400,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":80,"num_read_kb":15696,"num_write":644,"num_write_kb":25369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"55'681","reported_seq":2839,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.120096+0000","last_change":"2023-10-30T12:42:02.748079+0000","last_active":"2023-10-30T12:56:07.120096+0000","last_peered":"2023-10-30T12:56:07.120096+0000","last_clean":"2023-10-30T12:56:07.120096+0000","last_became_active":"2023-10-30T12:35:06.213402+0000","last_became_peered":"2023-10-30T12:35:06.213402+0000","last_unstale":"2023-10-30T12:56:07.120096+0000","last_undegraded":"2023-10-30T12:56:07.120096+0000","last_fullsized":"2023-10-30T12:56:07.120096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:11.732115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014055767,"stat_sum":{"num_bytes":10176680,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":67,"num_read_kb":4112,"num_write":643,"num_write_kb":20459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"55'660","reported_seq":2810,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.118693+0000","last_change":"2023-10-30T12:42:02.242578+0000","last_active":"2023-10-30T12:56:07.118693+0000","last_peered":"2023-10-30T12:56:07.118693+0000","last_clean":"2023-10-30T12:56:07.118693+0000","last_became_active":"2023-10-30T12:35:06.188298+0000","last_became_peered":"2023-10-30T12:35:06.188298+0000","last_unstale":"2023-10-30T12:56:07.118693+0000","last_undegraded":"2023-10-30T12:56:07.118693+0000","last_fullsized":"2023-10-30T12:56:07.118693+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:56:11.006974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053580889999999999,"stat_sum":{"num_bytes":4997964,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":59,"num_read_kb":8204,"num_write":612,"num_write_kb":44089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"55'741","reported_seq":2919,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.030017+0000","last_change":"2023-10-30T12:42:02.740157+0000","last_active":"2023-10-30T12:56:07.030017+0000","last_peered":"2023-10-30T12:56:07.030017+0000","last_clean":"2023-10-30T12:56:07.030017+0000","last_became_active":"2023-10-30T12:35:06.202507+0000","last_became_peered":"2023-10-30T12:35:06.202507+0000","last_unstale":"2023-10-30T12:56:07.030017+0000","last_undegraded":"2023-10-30T12:56:07.030017+0000","last_fullsized":"2023-10-30T12:56:07.030017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:55.774699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082500649999999991,"stat_sum":{"num_bytes":6483256,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":87,"num_read_kb":1892,"num_write":707,"num_write_kb":28689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"55'657","reported_seq":2811,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.123179+0000","last_change":"2023-10-30T12:42:02.710762+0000","last_active":"2023-10-30T12:56:07.123179+0000","last_peered":"2023-10-30T12:56:07.123179+0000","last_clean":"2023-10-30T12:56:07.123179+0000","last_became_active":"2023-10-30T12:35:06.170030+0000","last_became_peered":"2023-10-30T12:35:06.170030+0000","last_unstale":"2023-10-30T12:56:07.123179+0000","last_undegraded":"2023-10-30T12:56:07.123179+0000","last_fullsized":"2023-10-30T12:56:07.123179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:04.189974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024132049999999999,"stat_sum":{"num_bytes":15276436,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":63,"num_read_kb":15,"num_write":628,"num_write_kb":22058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"55'623","reported_seq":2792,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.158356+0000","last_change":"2023-10-30T12:42:02.242975+0000","last_active":"2023-10-30T12:56:07.158356+0000","last_peered":"2023-10-30T12:56:07.158356+0000","last_clean":"2023-10-30T12:56:07.158356+0000","last_became_active":"2023-10-30T12:35:06.189171+0000","last_became_peered":"2023-10-30T12:35:06.189171+0000","last_unstale":"2023-10-30T12:56:07.158356+0000","last_undegraded":"2023-10-30T12:56:07.158356+0000","last_fullsized":"2023-10-30T12:56:07.158356+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:00:28.317415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056904870000000001,"stat_sum":{"num_bytes":13986941,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":78,"num_read_kb":12372,"num_write":595,"num_write_kb":31185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"55'639","reported_seq":2791,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.133710+0000","last_change":"2023-10-30T12:42:02.725392+0000","last_active":"2023-10-30T12:56:07.133710+0000","last_peered":"2023-10-30T12:56:07.133710+0000","last_clean":"2023-10-30T12:56:07.133710+0000","last_became_active":"2023-10-30T12:35:06.183348+0000","last_became_peered":"2023-10-30T12:35:06.183348+0000","last_unstale":"2023-10-30T12:56:07.133710+0000","last_undegraded":"2023-10-30T12:56:07.133710+0000","last_fullsized":"2023-10-30T12:56:07.133710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:15:11.322585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041317979999999999,"stat_sum":{"num_bytes":14536653,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":60,"num_read_kb":7678,"num_write":614,"num_write_kb":36281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"55'712","reported_seq":2879,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.104341+0000","last_change":"2023-10-30T12:42:02.747343+0000","last_active":"2023-10-30T12:56:07.104341+0000","last_peered":"2023-10-30T12:56:07.104341+0000","last_clean":"2023-10-30T12:56:07.104341+0000","last_became_active":"2023-10-30T12:35:06.196368+0000","last_became_peered":"2023-10-30T12:35:06.196368+0000","last_unstale":"2023-10-30T12:56:07.104341+0000","last_undegraded":"2023-10-30T12:56:07.104341+0000","last_fullsized":"2023-10-30T12:56:07.104341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:53:23.269916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015569382999999999,"stat_sum":{"num_bytes":14455388,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":75,"num_read_kb":9352,"num_write":686,"num_write_kb":32708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"55'735","reported_seq":2899,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.035413+0000","last_change":"2023-10-30T12:42:02.241047+0000","last_active":"2023-10-30T12:56:07.035413+0000","last_peered":"2023-10-30T12:56:07.035413+0000","last_clean":"2023-10-30T12:56:07.035413+0000","last_became_active":"2023-10-30T12:35:06.213994+0000","last_became_peered":"2023-10-30T12:35:06.213994+0000","last_unstale":"2023-10-30T12:56:07.035413+0000","last_undegraded":"2023-10-30T12:56:07.035413+0000","last_fullsized":"2023-10-30T12:56:07.035413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:49:43.203436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001873502,"stat_sum":{"num_bytes":23237117,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":72,"num_read_kb":4118,"num_write":703,"num_write_kb":32259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"55'672","reported_seq":2834,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.181318+0000","last_change":"2023-10-30T12:42:02.738216+0000","last_active":"2023-10-30T12:56:07.181318+0000","last_peered":"2023-10-30T12:56:07.181318+0000","last_clean":"2023-10-30T12:56:07.181318+0000","last_became_active":"2023-10-30T12:35:06.197647+0000","last_became_peered":"2023-10-30T12:35:06.197647+0000","last_unstale":"2023-10-30T12:56:07.181318+0000","last_undegraded":"2023-10-30T12:56:07.181318+0000","last_fullsized":"2023-10-30T12:56:07.181318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:21:00.884446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011378932,"stat_sum":{"num_bytes":18305029,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":70,"num_read_kb":8212,"num_write":638,"num_write_kb":28193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"55'634","reported_seq":2786,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.949834+0000","last_change":"2023-10-30T12:42:02.251618+0000","last_active":"2023-10-30T12:56:06.949834+0000","last_peered":"2023-10-30T12:56:06.949834+0000","last_clean":"2023-10-30T12:56:06.949834+0000","last_became_active":"2023-10-30T12:35:06.167558+0000","last_became_peered":"2023-10-30T12:35:06.167558+0000","last_unstale":"2023-10-30T12:56:06.949834+0000","last_undegraded":"2023-10-30T12:56:06.949834+0000","last_fullsized":"2023-10-30T12:56:06.949834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:11:30.490780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048907120000000002,"stat_sum":{"num_bytes":9666676,"num_objects":132,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":0,"num_read":62,"num_read_kb":17,"num_write":598,"num_write_kb":15613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"55'233","reported_seq":298,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:07.513824+0000","last_change":"2023-10-30T12:35:05.200844+0000","last_active":"2023-10-30T12:55:07.513824+0000","last_peered":"2023-10-30T12:55:07.513824+0000","last_clean":"2023-10-30T12:55:07.513824+0000","last_became_active":"2023-10-30T12:35:05.200169+0000","last_became_peered":"2023-10-30T12:35:05.200169+0000","last_unstale":"2023-10-30T12:55:07.513824+0000","last_undegraded":"2023-10-30T12:55:07.513824+0000","last_fullsized":"2023-10-30T12:55:07.513824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:07:19.571357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":36867,"num_write":183,"num_write_kb":12445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"55'646","reported_seq":2797,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.123456+0000","last_change":"2023-10-30T12:42:02.725641+0000","last_active":"2023-10-30T12:56:07.123456+0000","last_peered":"2023-10-30T12:56:07.123456+0000","last_clean":"2023-10-30T12:56:07.123456+0000","last_became_active":"2023-10-30T12:35:06.194689+0000","last_became_peered":"2023-10-30T12:35:06.194689+0000","last_unstale":"2023-10-30T12:56:07.123456+0000","last_undegraded":"2023-10-30T12:56:07.123456+0000","last_fullsized":"2023-10-30T12:56:07.123456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:38:22.949407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004291034,"stat_sum":{"num_bytes":10877495,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":61,"num_read_kb":18,"num_write":619,"num_write_kb":24664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"55'974","reported_seq":1165,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:34.540942+0000","last_change":"2023-10-30T12:35:05.183569+0000","last_active":"2023-10-30T12:55:34.540942+0000","last_peered":"2023-10-30T12:55:34.540942+0000","last_clean":"2023-10-30T12:55:34.540942+0000","last_became_active":"2023-10-30T12:35:05.182761+0000","last_became_peered":"2023-10-30T12:35:05.182761+0000","last_unstale":"2023-10-30T12:55:34.540942+0000","last_undegraded":"2023-10-30T12:55:34.540942+0000","last_fullsized":"2023-10-30T12:55:34.540942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:49:44.311814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25625406,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":150,"num_read_kb":45229,"num_write":895,"num_write_kb":25503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"55'430","reported_seq":635,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.685247+0000","last_change":"2023-10-30T12:35:05.181278+0000","last_active":"2023-10-30T12:56:09.685247+0000","last_peered":"2023-10-30T12:56:09.685247+0000","last_clean":"2023-10-30T12:56:09.685247+0000","last_became_active":"2023-10-30T12:35:05.180730+0000","last_became_peered":"2023-10-30T12:35:05.180730+0000","last_unstale":"2023-10-30T12:56:09.685247+0000","last_undegraded":"2023-10-30T12:56:09.685247+0000","last_fullsized":"2023-10-30T12:56:09.685247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:07:10.828707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8578635,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":162,"num_read_kb":8759,"num_write":365,"num_write_kb":8612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"55'760","reported_seq":2929,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.115297+0000","last_change":"2023-10-30T12:42:02.240966+0000","last_active":"2023-10-30T12:56:07.115297+0000","last_peered":"2023-10-30T12:56:07.115297+0000","last_clean":"2023-10-30T12:56:07.115297+0000","last_became_active":"2023-10-30T12:35:46.051057+0000","last_became_peered":"2023-10-30T12:35:46.051057+0000","last_unstale":"2023-10-30T12:56:07.115297+0000","last_undegraded":"2023-10-30T12:56:07.115297+0000","last_fullsized":"2023-10-30T12:56:07.115297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:39:59.687314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018351520000000001,"stat_sum":{"num_bytes":18874907,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":82,"num_read_kb":16408,"num_write":717,"num_write_kb":52822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"55'983","reported_seq":1049,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:04.638735+0000","last_change":"2023-10-30T12:35:05.202539+0000","last_active":"2023-10-30T12:55:04.638735+0000","last_peered":"2023-10-30T12:55:04.638735+0000","last_clean":"2023-10-30T12:55:04.638735+0000","last_became_active":"2023-10-30T12:35:05.201853+0000","last_became_peered":"2023-10-30T12:35:05.201853+0000","last_unstale":"2023-10-30T12:55:04.638735+0000","last_undegraded":"2023-10-30T12:55:04.638735+0000","last_fullsized":"2023-10-30T12:55:04.638735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:33:33.151433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":38325,"num_write":914,"num_write_kb":16753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"55'707","reported_seq":2870,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.090289+0000","last_change":"2023-10-30T12:42:02.710378+0000","last_active":"2023-10-30T12:56:07.090289+0000","last_peered":"2023-10-30T12:56:07.090289+0000","last_clean":"2023-10-30T12:56:07.090289+0000","last_became_active":"2023-10-30T12:35:06.170179+0000","last_became_peered":"2023-10-30T12:35:06.170179+0000","last_unstale":"2023-10-30T12:56:07.090289+0000","last_undegraded":"2023-10-30T12:56:07.090289+0000","last_fullsized":"2023-10-30T12:56:07.090289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:21:27.326814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021794980000000002,"stat_sum":{"num_bytes":14241209,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":73,"num_read_kb":12307,"num_write":678,"num_write_kb":28673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"55'145","reported_seq":204,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:54.873264+0000","last_change":"2023-10-30T12:35:05.198546+0000","last_active":"2023-10-30T12:55:54.873264+0000","last_peered":"2023-10-30T12:55:54.873264+0000","last_clean":"2023-10-30T12:55:54.873264+0000","last_became_active":"2023-10-30T12:35:05.197606+0000","last_became_peered":"2023-10-30T12:35:05.197606+0000","last_unstale":"2023-10-30T12:55:54.873264+0000","last_undegraded":"2023-10-30T12:55:54.873264+0000","last_fullsized":"2023-10-30T12:55:54.873264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:26:57.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":90,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"55'666","reported_seq":2833,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.026011+0000","last_change":"2023-10-30T12:42:02.745501+0000","last_active":"2023-10-30T12:56:07.026011+0000","last_peered":"2023-10-30T12:56:07.026011+0000","last_clean":"2023-10-30T12:56:07.026011+0000","last_became_active":"2023-10-30T12:35:06.225392+0000","last_became_peered":"2023-10-30T12:35:06.225392+0000","last_unstale":"2023-10-30T12:56:07.026011+0000","last_undegraded":"2023-10-30T12:56:07.026011+0000","last_fullsized":"2023-10-30T12:56:07.026011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:06:30.011008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01552166,"stat_sum":{"num_bytes":9602206,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":77,"num_read_kb":18,"num_write":639,"num_write_kb":18963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"55'480","reported_seq":541,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.722504+0000","last_change":"2023-10-30T12:35:05.197614+0000","last_active":"2023-10-30T12:56:09.722504+0000","last_peered":"2023-10-30T12:56:09.722504+0000","last_clean":"2023-10-30T12:56:09.722504+0000","last_became_active":"2023-10-30T12:35:05.196899+0000","last_became_peered":"2023-10-30T12:35:05.196899+0000","last_unstale":"2023-10-30T12:56:09.722504+0000","last_undegraded":"2023-10-30T12:56:09.722504+0000","last_fullsized":"2023-10-30T12:56:09.722504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:21:00.152426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14221472,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":14173,"num_write":412,"num_write_kb":14118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"55'672","reported_seq":2825,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.087588+0000","last_change":"2023-10-30T12:42:02.724824+0000","last_active":"2023-10-30T12:56:07.087588+0000","last_peered":"2023-10-30T12:56:07.087588+0000","last_clean":"2023-10-30T12:56:07.087588+0000","last_became_active":"2023-10-30T12:35:06.222057+0000","last_became_peered":"2023-10-30T12:35:06.222057+0000","last_unstale":"2023-10-30T12:56:07.087588+0000","last_undegraded":"2023-10-30T12:56:07.087588+0000","last_fullsized":"2023-10-30T12:56:07.087588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:13:07.261307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004210915,"stat_sum":{"num_bytes":31275308,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":63,"num_read_kb":12308,"num_write":645,"num_write_kb":50366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"55'312","reported_seq":375,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:08.116088+0000","last_change":"2023-10-30T12:35:05.197501+0000","last_active":"2023-10-30T12:56:08.116088+0000","last_peered":"2023-10-30T12:56:08.116088+0000","last_clean":"2023-10-30T12:56:08.116088+0000","last_became_active":"2023-10-30T12:35:05.197190+0000","last_became_peered":"2023-10-30T12:35:05.197190+0000","last_unstale":"2023-10-30T12:56:08.116088+0000","last_undegraded":"2023-10-30T12:56:08.116088+0000","last_fullsized":"2023-10-30T12:56:08.116088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:53:31.121721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":4111,"num_write":258,"num_write_kb":8429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"55'651","reported_seq":2802,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.024707+0000","last_change":"2023-10-30T12:42:02.253699+0000","last_active":"2023-10-30T12:56:07.024707+0000","last_peered":"2023-10-30T12:56:07.024707+0000","last_clean":"2023-10-30T12:56:07.024707+0000","last_became_active":"2023-10-30T12:35:06.204122+0000","last_became_peered":"2023-10-30T12:35:06.204122+0000","last_unstale":"2023-10-30T12:56:07.024707+0000","last_undegraded":"2023-10-30T12:56:07.024707+0000","last_fullsized":"2023-10-30T12:56:07.024707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:59:50.717088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030324050000000002,"stat_sum":{"num_bytes":15521520,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":60,"num_read_kb":8209,"num_write":621,"num_write_kb":41873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"55'359","reported_seq":431,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.533756+0000","last_change":"2023-10-30T12:35:05.180005+0000","last_active":"2023-10-30T12:56:09.533756+0000","last_peered":"2023-10-30T12:56:09.533756+0000","last_clean":"2023-10-30T12:56:09.533756+0000","last_became_active":"2023-10-30T12:35:05.179547+0000","last_became_peered":"2023-10-30T12:35:05.179547+0000","last_unstale":"2023-10-30T12:56:09.533756+0000","last_undegraded":"2023-10-30T12:56:09.533756+0000","last_fullsized":"2023-10-30T12:56:09.533756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:13:03.137433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":24600,"num_write":299,"num_write_kb":12510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"55'628","reported_seq":2779,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.133221+0000","last_change":"2023-10-30T12:42:02.473160+0000","last_active":"2023-10-30T12:56:07.133221+0000","last_peered":"2023-10-30T12:56:07.133221+0000","last_clean":"2023-10-30T12:56:07.133221+0000","last_became_active":"2023-10-30T12:35:06.215128+0000","last_became_peered":"2023-10-30T12:35:06.215128+0000","last_unstale":"2023-10-30T12:56:07.133221+0000","last_undegraded":"2023-10-30T12:56:07.133221+0000","last_fullsized":"2023-10-30T12:56:07.133221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:47:38.300886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064662909999999999,"stat_sum":{"num_bytes":22716254,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":61,"num_read_kb":11,"num_write":594,"num_write_kb":27628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"55'209","reported_seq":283,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.553645+0000","last_change":"2023-10-30T12:35:05.190041+0000","last_active":"2023-10-30T12:56:04.553645+0000","last_peered":"2023-10-30T12:56:04.553645+0000","last_clean":"2023-10-30T12:56:04.553645+0000","last_became_active":"2023-10-30T12:35:05.189573+0000","last_became_peered":"2023-10-30T12:35:05.189573+0000","last_unstale":"2023-10-30T12:56:04.553645+0000","last_undegraded":"2023-10-30T12:56:04.553645+0000","last_fullsized":"2023-10-30T12:56:04.553645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:02:57.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":24624,"num_write":154,"num_write_kb":8340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"55'702","reported_seq":2870,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.111628+0000","last_change":"2023-10-30T12:42:02.724827+0000","last_active":"2023-10-30T12:56:07.111628+0000","last_peered":"2023-10-30T12:56:07.111628+0000","last_clean":"2023-10-30T12:56:07.111628+0000","last_became_active":"2023-10-30T12:35:06.216885+0000","last_became_peered":"2023-10-30T12:35:06.216885+0000","last_unstale":"2023-10-30T12:56:07.111628+0000","last_undegraded":"2023-10-30T12:56:07.111628+0000","last_fullsized":"2023-10-30T12:56:07.111628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:50:48.181232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042006370000000001,"stat_sum":{"num_bytes":18982563,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":77,"num_read_kb":16402,"num_write":665,"num_write_kb":45070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"55'232","reported_seq":293,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.565577+0000","last_change":"2023-10-30T12:35:05.203198+0000","last_active":"2023-10-30T12:56:04.565577+0000","last_peered":"2023-10-30T12:56:04.565577+0000","last_clean":"2023-10-30T12:56:04.565577+0000","last_became_active":"2023-10-30T12:35:05.202612+0000","last_became_peered":"2023-10-30T12:35:05.202612+0000","last_unstale":"2023-10-30T12:56:04.565577+0000","last_undegraded":"2023-10-30T12:56:04.565577+0000","last_fullsized":"2023-10-30T12:56:04.565577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:55.986236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":181,"num_write_kb":8338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"55'665","reported_seq":2831,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.009342+0000","last_change":"2023-10-30T12:42:02.471602+0000","last_active":"2023-10-30T12:56:07.009342+0000","last_peered":"2023-10-30T12:56:07.009342+0000","last_clean":"2023-10-30T12:56:07.009342+0000","last_became_active":"2023-10-30T12:35:06.215553+0000","last_became_peered":"2023-10-30T12:35:06.215553+0000","last_unstale":"2023-10-30T12:56:07.009342+0000","last_undegraded":"2023-10-30T12:56:07.009342+0000","last_fullsized":"2023-10-30T12:56:07.009342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:58:00.184959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060418609999999999,"stat_sum":{"num_bytes":18953195,"num_objects":130,"num_object_clones":0,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":0,"num_read":76,"num_read_kb":33,"num_write":640,"num_write_kb":32599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"55'289","reported_seq":360,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:27.641002+0000","last_change":"2023-10-30T12:35:05.181882+0000","last_active":"2023-10-30T12:55:27.641002+0000","last_peered":"2023-10-30T12:55:27.641002+0000","last_clean":"2023-10-30T12:55:27.641002+0000","last_became_active":"2023-10-30T12:35:05.181115+0000","last_became_peered":"2023-10-30T12:35:05.181115+0000","last_unstale":"2023-10-30T12:55:27.641002+0000","last_undegraded":"2023-10-30T12:55:27.641002+0000","last_fullsized":"2023-10-30T12:55:27.641002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:18:55.496940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":43239,"num_write":227,"num_write_kb":16591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"55'646","reported_seq":2797,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.181423+0000","last_change":"2023-10-30T12:42:02.738621+0000","last_active":"2023-10-30T12:56:07.181423+0000","last_peered":"2023-10-30T12:56:07.181423+0000","last_clean":"2023-10-30T12:56:07.181423+0000","last_became_active":"2023-10-30T12:35:06.197722+0000","last_became_peered":"2023-10-30T12:35:06.197722+0000","last_unstale":"2023-10-30T12:56:07.181423+0000","last_undegraded":"2023-10-30T12:56:07.181423+0000","last_fullsized":"2023-10-30T12:56:07.181423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:56:33.480519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011777173,"stat_sum":{"num_bytes":11054792,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":60,"num_read_kb":16,"num_write":607,"num_write_kb":21388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"55'481","reported_seq":572,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.552101+0000","last_change":"2023-10-30T12:35:05.169455+0000","last_active":"2023-10-30T12:56:04.552101+0000","last_peered":"2023-10-30T12:56:04.552101+0000","last_clean":"2023-10-30T12:56:04.552101+0000","last_became_active":"2023-10-30T12:35:05.169130+0000","last_became_peered":"2023-10-30T12:35:05.169130+0000","last_unstale":"2023-10-30T12:56:04.552101+0000","last_undegraded":"2023-10-30T12:56:04.552101+0000","last_fullsized":"2023-10-30T12:56:04.552101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:59:14.096315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583024,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":18341,"num_write":415,"num_write_kb":12551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"55'687","reported_seq":2869,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.010092+0000","last_change":"2023-10-30T12:42:02.470189+0000","last_active":"2023-10-30T12:56:07.010092+0000","last_peered":"2023-10-30T12:56:07.010092+0000","last_clean":"2023-10-30T12:56:07.010092+0000","last_became_active":"2023-10-30T12:35:06.214839+0000","last_became_peered":"2023-10-30T12:35:06.214839+0000","last_unstale":"2023-10-30T12:56:07.010092+0000","last_undegraded":"2023-10-30T12:56:07.010092+0000","last_fullsized":"2023-10-30T12:56:07.010092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:09:56.796250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049147749999999997,"stat_sum":{"num_bytes":12025229,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":91,"num_read_kb":16402,"num_write":654,"num_write_kb":29636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"55'312","reported_seq":377,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.648665+0000","last_change":"2023-10-30T12:35:05.189361+0000","last_active":"2023-10-30T12:56:04.648665+0000","last_peered":"2023-10-30T12:56:04.648665+0000","last_clean":"2023-10-30T12:56:04.648665+0000","last_became_active":"2023-10-30T12:35:05.188862+0000","last_became_peered":"2023-10-30T12:35:05.188862+0000","last_unstale":"2023-10-30T12:56:04.648665+0000","last_undegraded":"2023-10-30T12:56:04.648665+0000","last_fullsized":"2023-10-30T12:56:04.648665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:13.319732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":12343,"num_write":386,"num_write_kb":4750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"55'644","reported_seq":2800,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.941496+0000","last_change":"2023-10-30T12:42:02.472755+0000","last_active":"2023-10-30T12:56:06.941496+0000","last_peered":"2023-10-30T12:56:06.941496+0000","last_clean":"2023-10-30T12:56:06.941496+0000","last_became_active":"2023-10-30T12:35:06.216533+0000","last_became_peered":"2023-10-30T12:35:06.216533+0000","last_unstale":"2023-10-30T12:56:06.941496+0000","last_undegraded":"2023-10-30T12:56:06.941496+0000","last_fullsized":"2023-10-30T12:56:06.941496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:13:01.401936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060962430000000003,"stat_sum":{"num_bytes":10676108,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":65,"num_read_kb":4109,"num_write":617,"num_write_kb":24051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"55'234","reported_seq":295,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:04.569141+0000","last_change":"2023-10-30T12:35:05.177609+0000","last_active":"2023-10-30T12:55:04.569141+0000","last_peered":"2023-10-30T12:55:04.569141+0000","last_clean":"2023-10-30T12:55:04.569141+0000","last_became_active":"2023-10-30T12:35:05.177316+0000","last_became_peered":"2023-10-30T12:35:05.177316+0000","last_unstale":"2023-10-30T12:55:04.569141+0000","last_undegraded":"2023-10-30T12:55:04.569141+0000","last_fullsized":"2023-10-30T12:55:04.569141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:46.820925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":180,"num_write_kb":8325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"55'668","reported_seq":2828,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.003575+0000","last_change":"2023-10-30T12:42:02.241911+0000","last_active":"2023-10-30T12:56:07.003575+0000","last_peered":"2023-10-30T12:56:07.003575+0000","last_clean":"2023-10-30T12:56:07.003575+0000","last_became_active":"2023-10-30T12:35:46.049464+0000","last_became_peered":"2023-10-30T12:35:46.049464+0000","last_unstale":"2023-10-30T12:56:07.003575+0000","last_undegraded":"2023-10-30T12:56:07.003575+0000","last_fullsized":"2023-10-30T12:56:07.003575+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:06:39.487053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023101760000000002,"stat_sum":{"num_bytes":6776001,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":644,"num_write_kb":12631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"55'211","reported_seq":272,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:28.291856+0000","last_change":"2023-10-30T12:35:05.171787+0000","last_active":"2023-10-30T12:55:28.291856+0000","last_peered":"2023-10-30T12:55:28.291856+0000","last_clean":"2023-10-30T12:55:28.291856+0000","last_became_active":"2023-10-30T12:35:05.171491+0000","last_became_peered":"2023-10-30T12:35:05.171491+0000","last_unstale":"2023-10-30T12:55:28.291856+0000","last_undegraded":"2023-10-30T12:55:28.291856+0000","last_fullsized":"2023-10-30T12:55:28.291856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:43:15.331175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":16387,"num_write":154,"num_write_kb":4208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"55'739","reported_seq":2900,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.151055+0000","last_change":"2023-10-30T12:42:02.746323+0000","last_active":"2023-10-30T12:56:07.151055+0000","last_peered":"2023-10-30T12:56:07.151055+0000","last_clean":"2023-10-30T12:56:07.151055+0000","last_became_active":"2023-10-30T12:35:06.213493+0000","last_became_peered":"2023-10-30T12:35:06.213493+0000","last_unstale":"2023-10-30T12:56:07.151055+0000","last_undegraded":"2023-10-30T12:56:07.151055+0000","last_fullsized":"2023-10-30T12:56:07.151055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:38:05.007816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015847370999999999,"stat_sum":{"num_bytes":18167539,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":70,"num_read_kb":4114,"num_write":710,"num_write_kb":33439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"55'305","reported_seq":368,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:04.570822+0000","last_change":"2023-10-30T12:35:05.197811+0000","last_active":"2023-10-30T12:55:04.570822+0000","last_peered":"2023-10-30T12:55:04.570822+0000","last_clean":"2023-10-30T12:55:04.570822+0000","last_became_active":"2023-10-30T12:35:05.197453+0000","last_became_peered":"2023-10-30T12:35:05.197453+0000","last_unstale":"2023-10-30T12:55:04.570822+0000","last_undegraded":"2023-10-30T12:55:04.570822+0000","last_fullsized":"2023-10-30T12:55:04.570822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:16:14.550887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":240,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"55'660","reported_seq":2812,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.010112+0000","last_change":"2023-10-30T12:42:02.472032+0000","last_active":"2023-10-30T12:56:07.010112+0000","last_peered":"2023-10-30T12:56:07.010112+0000","last_clean":"2023-10-30T12:56:07.010112+0000","last_became_active":"2023-10-30T12:35:06.219526+0000","last_became_peered":"2023-10-30T12:35:06.219526+0000","last_unstale":"2023-10-30T12:56:07.010112+0000","last_undegraded":"2023-10-30T12:56:07.010112+0000","last_fullsized":"2023-10-30T12:56:07.010112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:01:04.257663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060180570000000003,"stat_sum":{"num_bytes":14243175,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":61,"num_read_kb":19722,"num_write":630,"num_write_kb":44881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":208,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:54:25.564818+0000","last_change":"2023-10-30T12:34:55.206976+0000","last_active":"2023-10-30T12:54:25.564818+0000","last_peered":"2023-10-30T12:54:25.564818+0000","last_clean":"2023-10-30T12:54:25.564818+0000","last_became_active":"2023-10-30T12:34:55.205923+0000","last_became_peered":"2023-10-30T12:34:55.205923+0000","last_unstale":"2023-10-30T12:54:25.564818+0000","last_undegraded":"2023-10-30T12:54:25.564818+0000","last_fullsized":"2023-10-30T12:54:25.564818+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_clean_scrub_stamp":"2023-10-30T12:34:53.444020+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:25:28.782111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"53'311","reported_seq":380,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:54:54.680160+0000","last_change":"2023-10-30T12:35:05.197974+0000","last_active":"2023-10-30T12:54:54.680160+0000","last_peered":"2023-10-30T12:54:54.680160+0000","last_clean":"2023-10-30T12:54:54.680160+0000","last_became_active":"2023-10-30T12:35:05.197201+0000","last_became_peered":"2023-10-30T12:35:05.197201+0000","last_unstale":"2023-10-30T12:54:54.680160+0000","last_undegraded":"2023-10-30T12:54:54.680160+0000","last_fullsized":"2023-10-30T12:54:54.680160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:39:47.119091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":23,"num_read_kb":16415,"num_write":260,"num_write_kb":8374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"55'616","reported_seq":2759,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.118778+0000","last_change":"2023-10-30T12:42:02.243413+0000","last_active":"2023-10-30T12:56:07.118778+0000","last_peered":"2023-10-30T12:56:07.118778+0000","last_clean":"2023-10-30T12:56:07.118778+0000","last_became_active":"2023-10-30T12:35:06.217687+0000","last_became_peered":"2023-10-30T12:35:06.217687+0000","last_unstale":"2023-10-30T12:56:07.118778+0000","last_undegraded":"2023-10-30T12:56:07.118778+0000","last_fullsized":"2023-10-30T12:56:07.118778+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:47:42.301163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059801699999999999,"stat_sum":{"num_bytes":18764071,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":51,"num_read_kb":8203,"num_write":589,"num_write_kb":32793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"55'386","reported_seq":458,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:31.185445+0000","last_change":"2023-10-30T12:35:05.208339+0000","last_active":"2023-10-30T12:55:31.185445+0000","last_peered":"2023-10-30T12:55:31.185445+0000","last_clean":"2023-10-30T12:55:31.185445+0000","last_became_active":"2023-10-30T12:35:05.207828+0000","last_became_peered":"2023-10-30T12:35:05.207828+0000","last_unstale":"2023-10-30T12:55:31.185445+0000","last_undegraded":"2023-10-30T12:55:31.185445+0000","last_fullsized":"2023-10-30T12:55:31.185445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:00.649048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":45074,"num_write":317,"num_write_kb":20666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"55'680","reported_seq":2830,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.087169+0000","last_change":"2023-10-30T12:42:02.724463+0000","last_active":"2023-10-30T12:56:07.087169+0000","last_peered":"2023-10-30T12:56:07.087169+0000","last_clean":"2023-10-30T12:56:07.087169+0000","last_became_active":"2023-10-30T12:35:06.194791+0000","last_became_peered":"2023-10-30T12:35:06.194791+0000","last_unstale":"2023-10-30T12:56:07.087169+0000","last_undegraded":"2023-10-30T12:56:07.087169+0000","last_fullsized":"2023-10-30T12:56:07.087169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:13:52.749170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043414500000000002,"stat_sum":{"num_bytes":10508371,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":58,"num_read_kb":3682,"num_write":647,"num_write_kb":34521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"55'607","reported_seq":965,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:57.501324+0000","last_change":"2023-10-30T12:35:05.176296+0000","last_active":"2023-10-30T12:55:57.501324+0000","last_peered":"2023-10-30T12:55:57.501324+0000","last_clean":"2023-10-30T12:55:57.501324+0000","last_became_active":"2023-10-30T12:35:05.175959+0000","last_became_peered":"2023-10-30T12:35:05.175959+0000","last_unstale":"2023-10-30T12:55:57.501324+0000","last_undegraded":"2023-10-30T12:55:57.501324+0000","last_fullsized":"2023-10-30T12:55:57.501324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:53.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9398249,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":313,"num_read_kb":20888,"num_write":566,"num_write_kb":9626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"55'674","reported_seq":2834,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.973362+0000","last_change":"2023-10-30T12:42:02.711348+0000","last_active":"2023-10-30T12:56:06.973362+0000","last_peered":"2023-10-30T12:56:06.973362+0000","last_clean":"2023-10-30T12:56:06.973362+0000","last_became_active":"2023-10-30T12:35:06.164711+0000","last_became_peered":"2023-10-30T12:35:06.164711+0000","last_unstale":"2023-10-30T12:56:06.973362+0000","last_undegraded":"2023-10-30T12:56:06.973362+0000","last_fullsized":"2023-10-30T12:56:06.973362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:21:46.319142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002714419,"stat_sum":{"num_bytes":18594076,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":68,"num_read_kb":8213,"num_write":640,"num_write_kb":33748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"55'497","reported_seq":561,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:56.727539+0000","last_change":"2023-10-30T12:35:05.198964+0000","last_active":"2023-10-30T12:55:56.727539+0000","last_peered":"2023-10-30T12:55:56.727539+0000","last_clean":"2023-10-30T12:55:56.727539+0000","last_became_active":"2023-10-30T12:35:05.196873+0000","last_became_peered":"2023-10-30T12:35:05.196873+0000","last_unstale":"2023-10-30T12:55:56.727539+0000","last_undegraded":"2023-10-30T12:55:56.727539+0000","last_fullsized":"2023-10-30T12:55:56.727539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:47.793585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":431,"num_write_kb":16687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"55'673","reported_seq":2837,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.990872+0000","last_change":"2023-10-30T12:42:02.250101+0000","last_active":"2023-10-30T12:56:06.990872+0000","last_peered":"2023-10-30T12:56:06.990872+0000","last_clean":"2023-10-30T12:56:06.990872+0000","last_became_active":"2023-10-30T12:35:06.183225+0000","last_became_peered":"2023-10-30T12:35:06.183225+0000","last_unstale":"2023-10-30T12:56:06.990872+0000","last_undegraded":"2023-10-30T12:56:06.990872+0000","last_fullsized":"2023-10-30T12:56:06.990872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:02:51.357971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032989500000000001,"stat_sum":{"num_bytes":17999573,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":72,"num_read_kb":12,"num_write":642,"num_write_kb":36248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"55'183","reported_seq":239,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:27.599652+0000","last_change":"2023-10-30T12:35:05.177571+0000","last_active":"2023-10-30T12:55:27.599652+0000","last_peered":"2023-10-30T12:55:27.599652+0000","last_clean":"2023-10-30T12:55:27.599652+0000","last_became_active":"2023-10-30T12:35:05.177266+0000","last_became_peered":"2023-10-30T12:35:05.177266+0000","last_unstale":"2023-10-30T12:55:27.599652+0000","last_undegraded":"2023-10-30T12:55:27.599652+0000","last_fullsized":"2023-10-30T12:55:27.599652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:14.601934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":129,"num_write_kb":8259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"55'655","reported_seq":2816,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.161469+0000","last_change":"2023-10-30T12:42:02.745017+0000","last_active":"2023-10-30T12:56:07.161469+0000","last_peered":"2023-10-30T12:56:07.161469+0000","last_clean":"2023-10-30T12:56:07.161469+0000","last_became_active":"2023-10-30T12:35:06.196307+0000","last_became_peered":"2023-10-30T12:35:06.196307+0000","last_unstale":"2023-10-30T12:56:07.161469+0000","last_undegraded":"2023-10-30T12:56:07.161469+0000","last_fullsized":"2023-10-30T12:56:07.161469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:09:56.936899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015197106,"stat_sum":{"num_bytes":5743717,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":69,"num_read_kb":13,"num_write":620,"num_write_kb":24446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"55'489","reported_seq":552,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.635330+0000","last_change":"2023-10-30T12:35:05.207594+0000","last_active":"2023-10-30T12:56:09.635330+0000","last_peered":"2023-10-30T12:56:09.635330+0000","last_clean":"2023-10-30T12:56:09.635330+0000","last_became_active":"2023-10-30T12:35:05.206930+0000","last_became_peered":"2023-10-30T12:35:05.206930+0000","last_unstale":"2023-10-30T12:56:09.635330+0000","last_undegraded":"2023-10-30T12:56:09.635330+0000","last_fullsized":"2023-10-30T12:56:09.635330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:23:46.519077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15572411,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":8,"num_read_kb":9517,"num_write":429,"num_write_kb":15473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"55'662","reported_seq":2826,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.964738+0000","last_change":"2023-10-30T12:42:02.710925+0000","last_active":"2023-10-30T12:56:06.964738+0000","last_peered":"2023-10-30T12:56:06.964738+0000","last_clean":"2023-10-30T12:56:06.964738+0000","last_became_active":"2023-10-30T12:35:06.164733+0000","last_became_peered":"2023-10-30T12:35:06.164733+0000","last_unstale":"2023-10-30T12:56:06.964738+0000","last_undegraded":"2023-10-30T12:56:06.964738+0000","last_fullsized":"2023-10-30T12:56:06.964738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:04:21.927512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024278020000000002,"stat_sum":{"num_bytes":19053289,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":637,"num_write_kb":32762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"55'374","reported_seq":441,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.541877+0000","last_change":"2023-10-30T12:35:05.185034+0000","last_active":"2023-10-30T12:56:04.541877+0000","last_peered":"2023-10-30T12:56:04.541877+0000","last_clean":"2023-10-30T12:56:04.541877+0000","last_became_active":"2023-10-30T12:35:05.184695+0000","last_became_peered":"2023-10-30T12:35:05.184695+0000","last_unstale":"2023-10-30T12:56:04.541877+0000","last_undegraded":"2023-10-30T12:56:04.541877+0000","last_fullsized":"2023-10-30T12:56:04.541877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:01:42.019128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":8489,"num_write":571,"num_write_kb":5858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"55'646","reported_seq":2799,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.108475+0000","last_change":"2023-10-30T12:42:02.242088+0000","last_active":"2023-10-30T12:56:07.108475+0000","last_peered":"2023-10-30T12:56:07.108475+0000","last_clean":"2023-10-30T12:56:07.108475+0000","last_became_active":"2023-10-30T12:35:06.204422+0000","last_became_peered":"2023-10-30T12:35:06.204422+0000","last_unstale":"2023-10-30T12:56:07.108475+0000","last_undegraded":"2023-10-30T12:56:07.108475+0000","last_fullsized":"2023-10-30T12:56:07.108475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:38:09.854923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048535030000000003,"stat_sum":{"num_bytes":22678810,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":62,"num_read_kb":4110,"num_write":624,"num_write_kb":32700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"55'230","reported_seq":292,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:28.060181+0000","last_change":"2023-10-30T12:35:05.200871+0000","last_active":"2023-10-30T12:55:28.060181+0000","last_peered":"2023-10-30T12:55:28.060181+0000","last_clean":"2023-10-30T12:55:28.060181+0000","last_became_active":"2023-10-30T12:35:05.200449+0000","last_became_peered":"2023-10-30T12:35:05.200449+0000","last_unstale":"2023-10-30T12:55:28.060181+0000","last_undegraded":"2023-10-30T12:55:28.060181+0000","last_fullsized":"2023-10-30T12:55:28.060181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:39:20.867587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":4117,"num_write":187,"num_write_kb":4299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"55'643","reported_seq":2803,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.060907+0000","last_change":"2023-10-30T12:42:02.724106+0000","last_active":"2023-10-30T12:56:07.060907+0000","last_peered":"2023-10-30T12:56:07.060907+0000","last_clean":"2023-10-30T12:56:07.060907+0000","last_became_active":"2023-10-30T12:35:06.179412+0000","last_became_peered":"2023-10-30T12:35:06.179412+0000","last_unstale":"2023-10-30T12:56:07.060907+0000","last_undegraded":"2023-10-30T12:56:07.060907+0000","last_fullsized":"2023-10-30T12:56:07.060907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:15:21.046958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003661568,"stat_sum":{"num_bytes":1280771,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":69,"num_read_kb":22,"num_write":604,"num_write_kb":12079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"55'552","reported_seq":617,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:26.056739+0000","last_change":"2023-10-30T12:35:05.207392+0000","last_active":"2023-10-30T12:55:26.056739+0000","last_peered":"2023-10-30T12:55:26.056739+0000","last_clean":"2023-10-30T12:55:26.056739+0000","last_became_active":"2023-10-30T12:35:05.206607+0000","last_became_peered":"2023-10-30T12:35:05.206607+0000","last_unstale":"2023-10-30T12:55:26.056739+0000","last_undegraded":"2023-10-30T12:55:26.056739+0000","last_fullsized":"2023-10-30T12:55:26.056739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:36.758740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":17,"num_read_kb":23179,"num_write":497,"num_write_kb":12662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"55'665","reported_seq":2826,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.023360+0000","last_change":"2023-10-30T12:42:02.251623+0000","last_active":"2023-10-30T12:56:07.023360+0000","last_peered":"2023-10-30T12:56:07.023360+0000","last_clean":"2023-10-30T12:56:07.023360+0000","last_became_active":"2023-10-30T12:35:06.181256+0000","last_became_peered":"2023-10-30T12:35:06.181256+0000","last_unstale":"2023-10-30T12:56:07.023360+0000","last_undegraded":"2023-10-30T12:56:07.023360+0000","last_fullsized":"2023-10-30T12:56:07.023360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:30:13.806672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030387220000000002,"stat_sum":{"num_bytes":10400130,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":70,"num_read_kb":4103,"num_write":631,"num_write_kb":29164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"55'369","reported_seq":432,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:02.291363+0000","last_change":"2023-10-30T12:35:05.201024+0000","last_active":"2023-10-30T12:56:02.291363+0000","last_peered":"2023-10-30T12:56:02.291363+0000","last_clean":"2023-10-30T12:56:02.291363+0000","last_became_active":"2023-10-30T12:35:05.200724+0000","last_became_peered":"2023-10-30T12:35:05.200724+0000","last_unstale":"2023-10-30T12:56:02.291363+0000","last_undegraded":"2023-10-30T12:56:02.291363+0000","last_fullsized":"2023-10-30T12:56:02.291363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:24.746469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388698,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":12300,"num_write":322,"num_write_kb":8548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"55'675","reported_seq":2824,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.153249+0000","last_change":"2023-10-30T12:42:02.241265+0000","last_active":"2023-10-30T12:56:07.153249+0000","last_peered":"2023-10-30T12:56:07.153249+0000","last_clean":"2023-10-30T12:56:07.153249+0000","last_became_active":"2023-10-30T12:35:06.173996+0000","last_became_peered":"2023-10-30T12:35:06.173996+0000","last_unstale":"2023-10-30T12:56:07.153249+0000","last_undegraded":"2023-10-30T12:56:07.153249+0000","last_fullsized":"2023-10-30T12:56:07.153249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:34:09.185233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045562010000000002,"stat_sum":{"num_bytes":5657110,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":58,"num_read_kb":18,"num_write":633,"num_write_kb":16227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"55'873","reported_seq":934,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.676168+0000","last_change":"2023-10-30T12:35:05.164079+0000","last_active":"2023-10-30T12:56:04.676168+0000","last_peered":"2023-10-30T12:56:04.676168+0000","last_clean":"2023-10-30T12:56:04.676168+0000","last_became_active":"2023-10-30T12:35:05.163794+0000","last_became_peered":"2023-10-30T12:35:05.163794+0000","last_unstale":"2023-10-30T12:56:04.676168+0000","last_undegraded":"2023-10-30T12:56:04.676168+0000","last_fullsized":"2023-10-30T12:56:04.676168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:19.795656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":10,"num_read_kb":8454,"num_write":952,"num_write_kb":13780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"55'645","reported_seq":2805,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.134005+0000","last_change":"2023-10-30T12:42:02.242567+0000","last_active":"2023-10-30T12:56:07.134005+0000","last_peered":"2023-10-30T12:56:07.134005+0000","last_clean":"2023-10-30T12:56:07.134005+0000","last_became_active":"2023-10-30T12:35:06.183226+0000","last_became_peered":"2023-10-30T12:35:06.183226+0000","last_unstale":"2023-10-30T12:56:07.134005+0000","last_undegraded":"2023-10-30T12:56:07.134005+0000","last_fullsized":"2023-10-30T12:56:07.134005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:30:48.627756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052521379999999999,"stat_sum":{"num_bytes":5772919,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":69,"num_read_kb":4112,"num_write":611,"num_write_kb":16444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"55'336","reported_seq":402,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:36.391405+0000","last_change":"2023-10-30T12:35:05.206995+0000","last_active":"2023-10-30T12:55:36.391405+0000","last_peered":"2023-10-30T12:55:36.391405+0000","last_clean":"2023-10-30T12:55:36.391405+0000","last_became_active":"2023-10-30T12:35:05.206518+0000","last_became_peered":"2023-10-30T12:35:05.206518+0000","last_unstale":"2023-10-30T12:55:36.391405+0000","last_undegraded":"2023-10-30T12:55:36.391405+0000","last_fullsized":"2023-10-30T12:55:36.391405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:41:33.703261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":268,"num_write_kb":12461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"55'637","reported_seq":2796,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.957805+0000","last_change":"2023-10-30T12:42:02.737782+0000","last_active":"2023-10-30T12:56:06.957805+0000","last_peered":"2023-10-30T12:56:06.957805+0000","last_clean":"2023-10-30T12:56:06.957805+0000","last_became_active":"2023-10-30T12:35:06.197861+0000","last_became_peered":"2023-10-30T12:35:06.197861+0000","last_unstale":"2023-10-30T12:56:06.957805+0000","last_undegraded":"2023-10-30T12:56:06.957805+0000","last_fullsized":"2023-10-30T12:56:06.957805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:38:17.960141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011118161,"stat_sum":{"num_bytes":14676487,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":68,"num_read_kb":15,"num_write":602,"num_write_kb":24516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"55'655","reported_seq":2826,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.074252+0000","last_change":"2023-10-30T12:42:02.744356+0000","last_active":"2023-10-30T12:56:07.074252+0000","last_peered":"2023-10-30T12:56:07.074252+0000","last_clean":"2023-10-30T12:56:07.074252+0000","last_became_active":"2023-10-30T12:35:06.212548+0000","last_became_peered":"2023-10-30T12:35:06.212548+0000","last_unstale":"2023-10-30T12:56:07.074252+0000","last_undegraded":"2023-10-30T12:56:07.074252+0000","last_fullsized":"2023-10-30T12:56:07.074252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:58:19.323075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015114931999999999,"stat_sum":{"num_bytes":19302518,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":81,"num_read_kb":7349,"num_write":635,"num_write_kb":32674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"55'363","reported_seq":423,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:07.528197+0000","last_change":"2023-10-30T12:35:05.206980+0000","last_active":"2023-10-30T12:55:07.528197+0000","last_peered":"2023-10-30T12:55:07.528197+0000","last_clean":"2023-10-30T12:55:07.528197+0000","last_became_active":"2023-10-30T12:35:05.206559+0000","last_became_peered":"2023-10-30T12:35:05.206559+0000","last_unstale":"2023-10-30T12:55:07.528197+0000","last_undegraded":"2023-10-30T12:55:07.528197+0000","last_fullsized":"2023-10-30T12:55:07.528197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:40.259622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":312,"num_write_kb":8389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"55'654","reported_seq":2814,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.072742+0000","last_change":"2023-10-30T12:42:02.744472+0000","last_active":"2023-10-30T12:56:07.072742+0000","last_peered":"2023-10-30T12:56:07.072742+0000","last_clean":"2023-10-30T12:56:07.072742+0000","last_became_active":"2023-10-30T12:35:06.212505+0000","last_became_peered":"2023-10-30T12:35:06.212505+0000","last_unstale":"2023-10-30T12:56:07.072742+0000","last_undegraded":"2023-10-30T12:56:07.072742+0000","last_fullsized":"2023-10-30T12:56:07.072742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:28:56.182227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014553393,"stat_sum":{"num_bytes":9479151,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":70,"num_read_kb":12304,"num_write":628,"num_write_kb":36199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"55'377","reported_seq":448,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:59.997488+0000","last_change":"2023-10-30T12:35:05.207851+0000","last_active":"2023-10-30T12:55:59.997488+0000","last_peered":"2023-10-30T12:55:59.997488+0000","last_clean":"2023-10-30T12:55:59.997488+0000","last_became_active":"2023-10-30T12:35:05.206879+0000","last_became_peered":"2023-10-30T12:35:05.206879+0000","last_unstale":"2023-10-30T12:55:59.997488+0000","last_undegraded":"2023-10-30T12:55:59.997488+0000","last_fullsized":"2023-10-30T12:55:59.997488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:43:15.717507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":32,"num_read_kb":45093,"num_write":322,"num_write_kb":20730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"55'623","reported_seq":2785,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.101531+0000","last_change":"2023-10-30T12:42:02.723663+0000","last_active":"2023-10-30T12:56:07.101531+0000","last_peered":"2023-10-30T12:56:07.101531+0000","last_clean":"2023-10-30T12:56:07.101531+0000","last_became_active":"2023-10-30T12:35:46.275077+0000","last_became_peered":"2023-10-30T12:35:46.275077+0000","last_unstale":"2023-10-30T12:56:07.101531+0000","last_undegraded":"2023-10-30T12:56:07.101531+0000","last_fullsized":"2023-10-30T12:56:07.101531+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:31:10.982688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032416139999999999,"stat_sum":{"num_bytes":6183106,"num_objects":128,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":0,"num_read":63,"num_read_kb":4113,"num_write":600,"num_write_kb":9129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"55'508","reported_seq":569,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:35.795056+0000","last_change":"2023-10-30T12:35:05.174319+0000","last_active":"2023-10-30T12:55:35.795056+0000","last_peered":"2023-10-30T12:55:35.795056+0000","last_clean":"2023-10-30T12:55:35.795056+0000","last_became_active":"2023-10-30T12:35:05.173984+0000","last_became_peered":"2023-10-30T12:35:05.173984+0000","last_unstale":"2023-10-30T12:55:35.795056+0000","last_undegraded":"2023-10-30T12:55:35.795056+0000","last_fullsized":"2023-10-30T12:55:35.795056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:11.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":430,"num_write_kb":12596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"55'666","reported_seq":2819,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.075370+0000","last_change":"2023-10-30T12:42:02.243078+0000","last_active":"2023-10-30T12:56:07.075370+0000","last_peered":"2023-10-30T12:56:07.075370+0000","last_clean":"2023-10-30T12:56:07.075370+0000","last_became_active":"2023-10-30T12:35:06.198504+0000","last_became_peered":"2023-10-30T12:35:06.198504+0000","last_unstale":"2023-10-30T12:56:07.075370+0000","last_undegraded":"2023-10-30T12:56:07.075370+0000","last_fullsized":"2023-10-30T12:56:07.075370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:44:32.447527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055292249999999996,"stat_sum":{"num_bytes":14373178,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":61,"num_read_kb":1995,"num_write":639,"num_write_kb":19071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"55'208","reported_seq":290,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:28.291837+0000","last_change":"2023-10-30T12:35:05.200926+0000","last_active":"2023-10-30T12:55:28.291837+0000","last_peered":"2023-10-30T12:55:28.291837+0000","last_clean":"2023-10-30T12:55:28.291837+0000","last_became_active":"2023-10-30T12:35:05.200623+0000","last_became_peered":"2023-10-30T12:35:05.200623+0000","last_unstale":"2023-10-30T12:55:28.291837+0000","last_undegraded":"2023-10-30T12:55:28.291837+0000","last_fullsized":"2023-10-30T12:55:28.291837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:38.942922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":63,"num_read_kb":24633,"num_write":147,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":916500589,"num_objects":9593,"num_object_clones":0,"num_object_copies":19186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9593,"num_whiteouts":0,"num_read":4339,"num_read_kb":400205,"num_write":40772,"num_write_kb":1941314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1881726976,"data_stored":1831461082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42742,"ondisk_log_size":42742,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":354417535,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1232,"num_read_kb":640401,"num_write":11493,"num_write_kb":356840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":709001216,"data_stored":708835070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12852,"ondisk_log_size":12852,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542405,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487968,"kb_used_data":398488,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":93883872,"statfs":{"total":96636764160,"available":96137084928,"internally_reserved":0,"allocated":408051712,"data_stored":400479873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":91617741},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.48599999999999999,"5min":0.438,"15min":0.42899999999999999},"max":{"1min":1.2330000000000001,"5min":1.371,"15min":1.371},"last":1.121},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.82099999999999995,"15min":0.82399999999999995},"min":{"1min":0.54200000000000004,"5min":0.437,"15min":0.41699999999999998},"max":{"1min":1.137,"5min":1.45,"15min":1.45},"last":0.91300000000000003}]},{"osd":1,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82699999999999996,"15min":0.85299999999999998},"min":{"1min":0.52700000000000002,"5min":0.46700000000000003,"15min":0.44600000000000001},"max":{"1min":1.222,"5min":1.222,"15min":1.496},"last":1.095},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.86599999999999999,"15min":0.89600000000000002},"min":{"1min":0.55700000000000005,"5min":0.53500000000000003,"15min":0.51800000000000002},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.98999999999999999}]},{"osd":2,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.873,"15min":0.82799999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.1659999999999999,"5min":1.242,"15min":1.9059999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.84199999999999997,"15min":0.89700000000000002},"min":{"1min":0.60299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.1830000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":0.92600000000000005,"15min":0.91100000000000003},"min":{"1min":0.76300000000000001,"5min":0.54200000000000004,"15min":0.54000000000000004},"max":{"1min":1.474,"5min":1.607,"15min":1.9810000000000001},"last":1.278},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.004,"15min":1},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.1559999999999999}]},{"osd":4,"last update":"Mon Oct 30 12:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90100000000000002,"15min":0.92800000000000005},"min":{"1min":0.71599999999999997,"5min":0.52500000000000002,"15min":0.46899999999999997},"max":{"1min":1.329,"5min":1.4059999999999999,"15min":2.7309999999999999},"last":1.05},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.016,"15min":0.99399999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.41999999999999998},"max":{"1min":1.3600000000000001,"5min":1.694,"15min":1.8859999999999999},"last":0.95299999999999996}]},{"osd":5,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.048,"15min":1.0700000000000001},"min":{"1min":0.72099999999999997,"5min":0.59399999999999997,"15min":0.49099999999999999},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":2.2120000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.218,"15min":1.143},"min":{"1min":1.0089999999999999,"5min":0.72599999999999998,"15min":0.69899999999999995},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.2310000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0229999999999999,"15min":1.087},"min":{"1min":0.79400000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.456,"5min":1.478,"15min":1.802},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.1359999999999999,"15min":1.1930000000000001},"min":{"1min":0.92800000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.4870000000000001,"5min":1.6639999999999999,"15min":1.881},"last":1.014}]}]},{"osd":6,"up_from":14,"seq":60129542404,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":378044,"kb_used_data":292788,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":93993796,"statfs":{"total":96636764160,"available":96249647104,"internally_reserved":0,"allocated":299814912,"data_stored":294791815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":87292361},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.83199999999999996,"15min":0.82399999999999995},"min":{"1min":0.496,"5min":0.45600000000000002,"15min":0.38600000000000001},"max":{"1min":1.056,"5min":5.8760000000000003,"15min":5.8760000000000003},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.60899999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.161,"5min":1.508,"15min":1.673},"last":0.85499999999999998}]},{"osd":1,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.85699999999999998,"15min":0.92100000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.113,"5min":2.2679999999999998,"15min":5.0949999999999998},"last":1.012},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.91100000000000003,"15min":0.92700000000000005},"min":{"1min":0.65000000000000002,"5min":0.54800000000000004,"15min":0.45500000000000002},"max":{"1min":1.282,"5min":1.423,"15min":5.0519999999999996},"last":1.0840000000000001}]},{"osd":2,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.86499999999999999,"15min":0.84699999999999998},"min":{"1min":0.621,"5min":0.52900000000000003,"15min":0.51800000000000002},"max":{"1min":1.403,"5min":2.649,"15min":2.649},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.069,"5min":0.97099999999999997,"15min":0.92500000000000004},"min":{"1min":0.74099999999999999,"5min":0.56899999999999995,"15min":0.44700000000000001},"max":{"1min":1.528,"5min":2.282,"15min":2.282},"last":1.143}]},{"osd":3,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.88500000000000001,"15min":0.94599999999999995},"min":{"1min":0.62,"5min":0.51700000000000002,"15min":0.40200000000000002},"max":{"1min":1.1120000000000001,"5min":1.929,"15min":1.929},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.97499999999999998,"15min":1},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.42699999999999999},"max":{"1min":1.246,"5min":1.5640000000000001,"15min":1.7589999999999999},"last":0.64300000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.03,"15min":1.0009999999999999},"min":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.27600000000000002},"max":{"1min":1.617,"5min":1.619,"15min":1.9219999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.05,"15min":1.0329999999999999},"min":{"1min":0.65600000000000003,"5min":0.64200000000000002,"15min":0.32400000000000001},"max":{"1min":1.3160000000000001,"5min":1.5700000000000001,"15min":1.625},"last":0.97899999999999998}]},{"osd":5,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":1.0109999999999999,"15min":1.0349999999999999},"min":{"1min":0.66600000000000004,"5min":0.66300000000000003,"15min":0.29599999999999999},"max":{"1min":1.194,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.1040000000000001,"15min":1.133},"min":{"1min":0.78500000000000003,"5min":0.65300000000000002,"15min":0.34399999999999997},"max":{"1min":1.3089999999999999,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":0.94499999999999995}]},{"osd":7,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.179,"15min":1.149},"min":{"1min":0.82999999999999996,"5min":0.79300000000000004,"15min":0.37},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":2.0350000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2030000000000001,"15min":1.1950000000000001},"min":{"1min":0.90200000000000002,"5min":0.82099999999999995,"15min":0.46800000000000003},"max":{"1min":1.754,"5min":2.274,"15min":2.274},"last":1.2410000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410352,"kb_used_data":316712,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":93961488,"statfs":{"total":96636764160,"available":96216563712,"internally_reserved":0,"allocated":324313088,"data_stored":316662693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79500000000000004,"15min":0.80900000000000005},"min":{"1min":0.53200000000000003,"5min":0.47199999999999998,"15min":0.437},"max":{"1min":1.1819999999999999,"5min":1.3700000000000001,"15min":1.5649999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.83299999999999996,"15min":0.84099999999999997},"min":{"1min":0.58899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2390000000000001,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.76000000000000001}]},{"osd":1,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.53300000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.034,"5min":1.3069999999999999,"15min":2.4910000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.86299999999999999,"15min":0.84999999999999998},"min":{"1min":0.63,"5min":0.433,"15min":0.433},"max":{"1min":1.2030000000000001,"5min":1.54,"15min":1.9730000000000001},"last":0.64000000000000001}]},{"osd":2,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.67300000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1950000000000001,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":1.0600000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.55600000000000005},"max":{"1min":1.2170000000000001,"5min":4.6879999999999997,"15min":4.6879999999999997},"last":0.89100000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.92500000000000004,"15min":0.94399999999999995},"min":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.284,"5min":2.266,"15min":2.5819999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.94999999999999996,"15min":0.90700000000000003},"min":{"1min":0.78200000000000003,"5min":0.54300000000000004,"15min":0.53500000000000003},"max":{"1min":1.411,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.94799999999999995}]},{"osd":5,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.97099999999999997,"15min":1.0029999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.129,"5min":2.1779999999999999,"15min":2.3180000000000001},"last":1.075},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.0880000000000001,"15min":1.0620000000000001},"min":{"1min":1.0660000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3959999999999999,"5min":3.173,"15min":3.173},"last":1.2150000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":1.0509999999999999,"15min":1.0880000000000001},"min":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.59299999999999997},"max":{"1min":1.131,"5min":2.2709999999999999,"15min":2.73},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.093,"5min":1.1080000000000001,"15min":1.0640000000000001},"min":{"1min":0.83999999999999997,"5min":0.748,"15min":0.56499999999999995},"max":{"1min":1.262,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":1.123}]},{"osd":7,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.087,"15min":1.0700000000000001},"min":{"1min":0.70699999999999996,"5min":0.68899999999999995,"15min":0.64900000000000002},"max":{"1min":1.196,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.02},{"interface":"front","average":{"1min":1.204,"5min":1.175,"15min":1.131},"min":{"1min":0.92500000000000004,"5min":0.70299999999999996,"15min":0.65300000000000002},"max":{"1min":1.4950000000000001,"5min":2.375,"15min":3.0649999999999999},"last":1.1699999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542404,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":296824,"kb_used_data":239280,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94075016,"statfs":{"total":96636764160,"available":96332816384,"internally_reserved":0,"allocated":245022720,"data_stored":240236236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.48599999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.22700000000000001},"max":{"1min":1.6930000000000001,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.58199999999999996,"15min":0.53200000000000003},"min":{"1min":0.40000000000000002,"5min":0.33200000000000002,"15min":0.28699999999999998},"max":{"1min":2.0619999999999998,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":0.47399999999999998}]},{"osd":1,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.33000000000000002,"5min":0.317,"15min":0.27900000000000003},"max":{"1min":3.02,"5min":3.5059999999999998,"15min":4.5019999999999998},"last":0.497},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.39100000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.0299999999999998,"5min":3.5430000000000001,"15min":3.5430000000000001},"last":0.38200000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.76000000000000001,"15min":0.60399999999999998},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26100000000000001},"max":{"1min":8.9550000000000001,"5min":18.747,"15min":18.747},"last":0.36599999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":0.85499999999999998,"15min":0.70099999999999996},"min":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38300000000000001},"max":{"1min":8.9900000000000002,"5min":18.863,"15min":18.863},"last":0.45400000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85999999999999999},"min":{"1min":0.626,"5min":0.56999999999999995,"15min":0.53600000000000003},"max":{"1min":3.0760000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.321,"5min":0.96299999999999997,"15min":0.89700000000000002},"min":{"1min":0.73299999999999998,"5min":0.51100000000000001,"15min":0.48299999999999998},"max":{"1min":9.6289999999999996,"5min":9.6289999999999996,"15min":9.6289999999999996},"last":0.78200000000000003}]},{"osd":5,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":0.876,"15min":0.85199999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":9.7289999999999992,"5min":9.7289999999999992,"15min":9.7289999999999992},"last":0.82599999999999996},{"interface":"front","average":{"1min":5.577,"5min":1.9490000000000001,"15min":1.2},"min":{"1min":0.626,"5min":0.60399999999999998,"15min":0.51900000000000002},"max":{"1min":91.072999999999993,"5min":91.072999999999993,"15min":91.072999999999993},"last":0.64800000000000002}]},{"osd":6,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.048,"15min":0.91700000000000004},"min":{"1min":0.59299999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":5.4640000000000004,"5min":19.126000000000001,"15min":19.126000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.2509999999999999,"15min":1.006},"min":{"1min":0.72499999999999998,"5min":0.60299999999999998,"15min":0.59399999999999997},"max":{"1min":21.469999999999999,"5min":21.469999999999999,"15min":21.469999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":0.94699999999999995,"15min":0.89600000000000002},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.47099999999999997},"max":{"1min":9.6869999999999994,"5min":9.6869999999999994,"15min":9.6869999999999994},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.1479999999999999,"15min":0.96799999999999997},"min":{"1min":0.80800000000000005,"5min":0.69799999999999995,"15min":0.53200000000000003},"max":{"1min":5.524,"5min":19.050999999999998,"15min":19.050999999999998},"last":0.90900000000000003}]}]},{"osd":0,"up_from":14,"seq":60129542405,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427176,"kb_used_data":337696,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":93944664,"statfs":{"total":96636764160,"available":96199335936,"internally_reserved":0,"allocated":345800704,"data_stored":338053520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":91617736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.46600000000000003,"15min":0.48399999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.248},"max":{"1min":0.69099999999999995,"5min":0.78700000000000003,"15min":8.8059999999999992},"last":0.376},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.504,"15min":0.61099999999999999},"min":{"1min":0.30099999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69299999999999995,"5min":0.83899999999999997,"15min":35.084000000000003},"last":0.55900000000000005}]},{"osd":2,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51500000000000001,"15min":0.63500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.28999999999999998},"max":{"1min":0.68000000000000005,"5min":0.93100000000000005,"15min":34.741},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58099999999999996,"15min":0.69299999999999995},"min":{"1min":0.42999999999999999,"5min":0.37,"15min":0.35299999999999998},"max":{"1min":0.72999999999999998,"5min":1.0049999999999999,"15min":35.058999999999997},"last":0.47099999999999997}]},{"osd":3,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.35899999999999999,"5min":0.32000000000000001,"15min":0.29499999999999998},"max":{"1min":0.76000000000000001,"5min":0.877,"15min":8.8469999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.629},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.88800000000000001,"15min":11.916},"last":0.52900000000000003}]},{"osd":4,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.85499999999999998,"15min":0.79700000000000004},"min":{"1min":0.66600000000000004,"5min":0.55500000000000005,"15min":0.373},"max":{"1min":1.0389999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.82499999999999996,"15min":0.92800000000000005},"min":{"1min":0.63900000000000001,"5min":0.57099999999999995,"15min":0.39900000000000002},"max":{"1min":1.1419999999999999,"5min":1.175,"15min":34.902999999999999},"last":0.76300000000000001}]},{"osd":5,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82099999999999995,"15min":0.84799999999999998},"min":{"1min":0.59499999999999997,"5min":0.51800000000000002,"15min":0.5},"max":{"1min":1.0820000000000001,"5min":1.171,"15min":8.4900000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.90600000000000003,"15min":0.97499999999999998},"min":{"1min":0.78500000000000003,"5min":0.66900000000000004,"15min":0.48799999999999999},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":34.698},"last":0.81200000000000006}]},{"osd":6,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81000000000000005},"min":{"1min":0.64200000000000002,"5min":0.58099999999999996,"15min":0.47799999999999998},"max":{"1min":1.097,"5min":1.1439999999999999,"15min":1.2789999999999999},"last":0.75},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.878,"15min":0.88700000000000001},"min":{"1min":0.64100000000000001,"5min":0.629,"15min":0.50900000000000001},"max":{"1min":1.304,"5min":1.304,"15min":8.8800000000000008},"last":0.70599999999999996}]},{"osd":7,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82499999999999996,"15min":0.84099999999999997},"min":{"1min":0.65000000000000002,"5min":0.53000000000000003,"15min":0.437},"max":{"1min":1.042,"5min":1.1930000000000001,"15min":1.304},"last":0.875},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.88900000000000001,"15min":0.92900000000000005},"min":{"1min":0.70699999999999996,"5min":0.60599999999999998,"15min":0.55900000000000005},"max":{"1min":1.1479999999999999,"5min":1.323,"15min":8.5549999999999997},"last":0.79200000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542403,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462712,"kb_used_data":373232,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":93909128,"statfs":{"total":96636764160,"available":96162947072,"internally_reserved":0,"allocated":382189568,"data_stored":374550682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":91617737},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.88300000000000001,"15min":0.84399999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.41899999999999998},"max":{"1min":1.3129999999999999,"5min":1.627,"15min":1.627},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.87,"5min":0.88900000000000001,"15min":0.84999999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.495},"max":{"1min":1.2649999999999999,"5min":1.468,"15min":1.468},"last":0.79400000000000004}]},{"osd":1,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.82899999999999996,"15min":0.91300000000000003},"min":{"1min":0.52800000000000002,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.3089999999999999,"5min":1.363,"15min":1.746},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.82399999999999995,"15min":0.88},"min":{"1min":0.52700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2969999999999999,"5min":1.3169999999999999,"15min":1.337},"last":0.92300000000000004}]},{"osd":2,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89700000000000002,"15min":0.872},"min":{"1min":0.60299999999999998,"5min":0.53800000000000003,"15min":0.501},"max":{"1min":1.4139999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.21},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.86199999999999999,"15min":0.83899999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.442},"max":{"1min":1.1699999999999999,"5min":1.29,"15min":1.4770000000000001},"last":1.0369999999999999}]},{"osd":3,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.9850000000000001,"15min":1.3180000000000001},"min":{"1min":0.63900000000000001,"5min":0.54400000000000004,"15min":0.504},"max":{"1min":1.3640000000000001,"5min":104.88500000000001,"15min":104.88500000000001},"last":1.252},{"interface":"front","average":{"1min":1.034,"5min":2.0619999999999998,"15min":1.3700000000000001},"min":{"1min":0.71399999999999997,"5min":0.624,"15min":0.60699999999999998},"max":{"1min":1.405,"5min":105.042,"15min":105.042},"last":0.96099999999999997}]},{"osd":4,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0349999999999999,"15min":0.94099999999999995},"min":{"1min":0.69899999999999995,"5min":0.58699999999999997,"15min":0.38500000000000001},"max":{"1min":1.262,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.88},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.077,"15min":1.1220000000000001},"min":{"1min":0.68100000000000005,"5min":0.59799999999999998,"15min":0.57399999999999995},"max":{"1min":1.5229999999999999,"5min":1.542,"15min":1.712},"last":0.59699999999999998}]},{"osd":6,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.96199999999999997,"15min":0.95599999999999996},"min":{"1min":0.77800000000000002,"5min":0.627,"15min":0.35899999999999999},"max":{"1min":1.389,"5min":1.3899999999999999,"15min":1.9410000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.0649999999999999,"15min":1.056},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.48899999999999999},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":2.0299999999999998},"last":0.83799999999999997}]},{"osd":7,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.1080000000000001,"15min":1.095},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.40600000000000003},"max":{"1min":1.357,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.113,"15min":1.1120000000000001},"min":{"1min":0.88,"5min":0.70499999999999996,"15min":0.53300000000000003},"max":{"1min":1.5149999999999999,"5min":1.5660000000000001,"15min":1.7689999999999999},"last":0.99299999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542405,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442132,"kb_used_data":348492,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":93929708,"statfs":{"total":96636764160,"available":96184020992,"internally_reserved":0,"allocated":356855808,"data_stored":348975142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":95877578},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:29 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42999999999999999,"15min":0.436},"min":{"1min":0.26600000000000001,"5min":0.25600000000000001,"15min":0.224},"max":{"1min":0.68100000000000005,"5min":0.77100000000000002,"15min":2.1459999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.46700000000000003,"15min":0.54800000000000004},"min":{"1min":0.32100000000000001,"5min":0.28399999999999997,"15min":0.27200000000000002},"max":{"1min":0.59999999999999998,"5min":0.77900000000000003,"15min":21.495999999999999},"last":0.48199999999999998}]},{"osd":1,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.52100000000000002,"15min":0.53200000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72999999999999998,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48999999999999999,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.314,"15min":0.30399999999999999},"max":{"1min":0.68500000000000005,"5min":1.2450000000000001,"15min":15.102},"last":0.53000000000000003}]},{"osd":2,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53900000000000003,"15min":0.60899999999999999},"min":{"1min":0.33300000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.75800000000000001,"5min":1.044,"15min":21.321000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.495,"5min":0.39000000000000001,"15min":0.379},"max":{"1min":0.77000000000000002,"5min":0.97099999999999997,"15min":1.681},"last":0.45700000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.80600000000000005,"15min":0.90000000000000002},"min":{"1min":0.57999999999999996,"5min":0.49199999999999999,"15min":0.48499999999999999},"max":{"1min":1.21,"5min":1.21,"15min":21.382999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.879,"15min":0.91900000000000004},"min":{"1min":0.73799999999999999,"5min":0.60399999999999998,"15min":0.46700000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":15.231999999999999},"last":0.71299999999999997}]},{"osd":5,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.84399999999999997,"15min":0.89500000000000002},"min":{"1min":0.58699999999999997,"5min":0.54200000000000004,"15min":0.47799999999999998},"max":{"1min":1.0600000000000001,"5min":1.1879999999999999,"15min":15.273999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.90500000000000003,"15min":0.95399999999999996},"min":{"1min":0.69499999999999995,"5min":0.67700000000000005,"15min":0.57499999999999996},"max":{"1min":1.1619999999999999,"5min":1.2170000000000001,"15min":15.359999999999999},"last":0.84399999999999997}]},{"osd":6,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79400000000000004,"15min":0.81699999999999995},"min":{"1min":0.61099999999999999,"5min":0.53200000000000003,"15min":0.51100000000000001},"max":{"1min":1.099,"5min":1.296,"15min":1.5620000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81599999999999995,"15min":0.89100000000000001},"min":{"1min":0.66000000000000003,"5min":0.55700000000000005,"15min":0.498},"max":{"1min":1.04,"5min":1.2210000000000001,"15min":15.313000000000001},"last":0.88400000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.85199999999999998,"15min":0.92700000000000005},"min":{"1min":0.65000000000000002,"5min":0.58799999999999997,"15min":0.55200000000000005},"max":{"1min":1.0620000000000001,"5min":1.137,"15min":21.428999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.85299999999999998,"15min":0.879},"min":{"1min":0.81699999999999995,"5min":0.56899999999999995,"15min":0.56799999999999995},"max":{"1min":1.1180000000000001,"5min":1.28,"15min":1.3100000000000001},"last":0.80900000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542403,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":296616,"kb_used_data":229408,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94075224,"statfs":{"total":96636764160,"available":96333029376,"internally_reserved":0,"allocated":234913792,"data_stored":230197311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.68500000000000005,"15min":0.54400000000000004},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.62,"5min":15.343999999999999,"15min":15.343999999999999},"last":0.434},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.73299999999999998,"15min":0.59999999999999998},"min":{"1min":0.34100000000000003,"5min":0.313,"15min":0.29899999999999999},"max":{"1min":0.66800000000000004,"5min":14.68,"15min":14.68},"last":0.45200000000000001}]},{"osd":2,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.66700000000000004,"15min":0.59799999999999998},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68999999999999995,"5min":8.1359999999999992,"15min":8.1359999999999992},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.33900000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69899999999999995,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.65100000000000002,"15min":0.59599999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.81200000000000006,"5min":8.0920000000000005,"15min":8.0920000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59399999999999997,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.35199999999999998,"15min":0.32000000000000001},"max":{"1min":0.79500000000000004,"5min":5.9199999999999999,"15min":5.9199999999999999},"last":0.504}]},{"osd":4,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":1.133,"15min":0.94099999999999995},"min":{"1min":0.629,"5min":0.60499999999999998,"15min":0.41799999999999998},"max":{"1min":1.2010000000000001,"5min":21.27,"15min":21.27},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.93899999999999995,"15min":0.90700000000000003},"min":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.53600000000000003},"max":{"1min":1.3420000000000001,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.82199999999999995}]},{"osd":5,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.83099999999999996,"15min":0.82599999999999996},"min":{"1min":0.63900000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0089999999999999,"5min":1.355,"15min":1.355},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.872,"15min":0.85599999999999998},"min":{"1min":0.68300000000000005,"5min":0.499,"15min":0.46200000000000002},"max":{"1min":1.1599999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.66300000000000003}]},{"osd":6,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85599999999999998,"15min":0.83999999999999997},"min":{"1min":0.70999999999999996,"5min":0.61399999999999999,"15min":0.51300000000000001},"max":{"1min":1.111,"5min":1.29,"15min":1.4450000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.88400000000000001,"15min":0.88500000000000001},"min":{"1min":0.64600000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.246,"5min":1.323,"15min":1.468},"last":0.75600000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.871,"15min":0.85299999999999998},"min":{"1min":0.67800000000000005,"5min":0.629,"15min":0.496},"max":{"1min":1.304,"5min":1.304,"15min":1.4299999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.86399999999999999,"15min":0.874},"min":{"1min":0.76200000000000001,"5min":0.63100000000000001,"15min":0.443},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2410000000000001},"last":0.88200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":74309632,"data_stored":74293305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96493568,"data_stored":96469936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42975232,"data_stored":42953409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":85110784,"data_stored":85086454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":88555520,"data_stored":88540092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":85549056,"data_stored":85526018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":107171840,"data_stored":107146531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":128835584,"data_stored":128819325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":270266368,"data_stored":262861049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":137789440,"data_stored":133418577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":201416704,"data_stored":196974029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":271114240,"data_stored":263579890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":235126784,"data_stored":227813803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":296009728,"data_stored":288715866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":192012288,"data_stored":187336486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":277991424,"data_stored":270761382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:56:15.543 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-30T12:56:15.544 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:56:15.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.715+0000 7fea7b5c4640 1 Processor -- start 2023-10-30T12:56:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.715+0000 7fea7b5c4640 1 -- start start 2023-10-30T12:56:15.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.715+0000 7fea7b5c4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea741580e0 0x7fea74070740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:15.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea7b5c4640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea740c5810 con 0x7fea7415e700 2023-10-30T12:56:15.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea7b5c4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea7415eb90 con 0x7fea740c6540 2023-10-30T12:56:15.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea7b5c4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7406acb0 con 0x7fea741580e0 2023-10-30T12:56:15.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea79b61640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fea7415e700 0x7fea7406a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44384/0 (socket says 172.21.15.12:44384) 2023-10-30T12:56:15.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea79b61640 1 -- 172.21.15.12:0/418761012 learned_addr learned my addr 172.21.15.12:0/418761012 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:15.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea79360640 1 --2- 172.21.15.12:0/418761012 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea741580e0 0x7fea74070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea6bfff640 1 -- 172.21.15.12:0/418761012 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025637650 0 0) 0x7fea740c5810 con 0x7fea7415e700 2023-10-30T12:56:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea6bfff640 1 -- 172.21.15.12:0/418761012 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea60003530 con 0x7fea7415e700 2023-10-30T12:56:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea6bfff640 1 -- 172.21.15.12:0/418761012 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 431712035 0 0) 0x7fea60003530 con 0x7fea7415e700 2023-10-30T12:56:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea6bfff640 1 -- 172.21.15.12:0/418761012 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea741580e0 msgr2=0x7fea74070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea6bfff640 1 --2- 172.21.15.12:0/418761012 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea741580e0 0x7fea74070740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea6bfff640 1 -- 172.21.15.12:0/418761012 >> v1:172.21.15.154:6789/0 conn(0x7fea740c6540 legacy=0x7fea74069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea6bfff640 1 -- 172.21.15.12:0/418761012 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea740c6960 con 0x7fea7415e700 2023-10-30T12:56:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea6bfff640 1 -- 172.21.15.12:0/418761012 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fea6c001a00 con 0x7fea7415e700 2023-10-30T12:56:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea79360640 1 --2- 172.21.15.12:0/418761012 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea741580e0 0x7fea74070740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:56:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea6bfff640 1 -- 172.21.15.12:0/418761012 >> v1:172.21.15.12:6789/0 conn(0x7fea7415e700 legacy=0x7fea7406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:15.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea6bfff640 1 --2- 172.21.15.12:0/418761012 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea741580e0 0x7fea600039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:15.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea6bfff640 1 --2- 172.21.15.12:0/418761012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea60003ef0 0x7fea600047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:15.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea6bfff640 1 --2- 172.21.15.12:0/418761012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea60004d10 0x7fea60005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:15.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea6bfff640 1 -- 172.21.15.12:0/418761012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6000ba40 con 0x7fea60004d10 2023-10-30T12:56:15.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea6bfff640 1 -- 172.21.15.12:0/418761012 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6000bb80 con 0x7fea60003ef0 2023-10-30T12:56:15.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea6bfff640 1 -- 172.21.15.12:0/418761012 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6000bd40 con 0x7fea741580e0 2023-10-30T12:56:15.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea79b61640 1 --2- 172.21.15.12:0/418761012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea60004d10 0x7fea60005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:15.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea79360640 1 --2- 172.21.15.12:0/418761012 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea741580e0 0x7fea600039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:15.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea78b5f640 1 --2- 172.21.15.12:0/418761012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea60003ef0 0x7fea600047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:15.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea79b61640 1 -- 172.21.15.12:0/418761012 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea741580e0 msgr2=0x7fea600039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:15.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea79b61640 1 --2- 172.21.15.12:0/418761012 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea741580e0 0x7fea600039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea79b61640 1 -- 172.21.15.12:0/418761012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea60003ef0 msgr2=0x7fea600047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:15.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea79b61640 1 --2- 172.21.15.12:0/418761012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea60003ef0 0x7fea600047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.719+0000 7fea79b61640 1 -- 172.21.15.12:0/418761012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea70009420 con 0x7fea60004d10 2023-10-30T12:56:15.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.723+0000 7fea79b61640 1 --2- 172.21.15.12:0/418761012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea60004d10 0x7fea60005190 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00a820 tx=0x7fea6c01d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0919c780d2b64fe server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:15.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.723+0000 7fea6bfff640 1 -- 172.21.15.12:0/418761012 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0072d0 con 0x7fea60004d10 2023-10-30T12:56:15.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.723+0000 7fea6bfff640 1 -- 172.21.15.12:0/418761012 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6c0a8070 con 0x7fea60004d10 2023-10-30T12:56:15.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.723+0000 7fea6bfff640 1 -- 172.21.15.12:0/418761012 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0076f0 con 0x7fea60004d10 2023-10-30T12:56:15.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.723+0000 7fea7b5c4640 1 -- 172.21.15.12:0/418761012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea60004d10 msgr2=0x7fea60005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:15.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.723+0000 7fea7b5c4640 1 --2- 172.21.15.12:0/418761012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea60004d10 0x7fea60005190 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00a820 tx=0x7fea6c01d320 comp rx=0 tx=0).stop 2023-10-30T12:56:15.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.723+0000 7fea79b61640 1 -- 172.21.15.12:0/418761012 reap_dead start 2023-10-30T12:56:15.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.723+0000 7fea7b5c4640 1 -- 172.21.15.12:0/418761012 shutdown_connections 2023-10-30T12:56:15.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.723+0000 7fea7b5c4640 1 -- 172.21.15.12:0/418761012 >> 172.21.15.12:0/418761012 conn(0x7fea740c7910 msgr2=0x7fea740c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:15.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.723+0000 7fea7b5c4640 1 -- 172.21.15.12:0/418761012 shutdown_connections 2023-10-30T12:56:15.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.723+0000 7fea7b5c4640 1 -- 172.21.15.12:0/418761012 wait complete. 2023-10-30T12:56:15.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.723+0000 7fea7b5c4640 1 Processor -- start 2023-10-30T12:56:15.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.723+0000 7fea7b5c4640 1 -- start start 2023-10-30T12:56:15.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea7b5c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea60004d10 0x7fea7406c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:15.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea7b5c4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea7414d0e0 0x7fea74153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:15.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea7b5c4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea741543c0 0x7fea74152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:15.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea7b5c4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea741f3160 con 0x7fea60004d10 2023-10-30T12:56:15.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea7b5c4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea741f3350 con 0x7fea741543c0 2023-10-30T12:56:15.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea7b5c4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea741f3630 con 0x7fea7414d0e0 2023-10-30T12:56:15.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea79360640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea60004d10 0x7fea7406c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:15.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea79360640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea60004d10 0x7fea7406c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52226/0 (socket says 172.21.15.12:52226) 2023-10-30T12:56:15.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea79360640 1 -- 172.21.15.12:0/2374560554 learned_addr learned my addr 172.21.15.12:0/2374560554 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:15.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea78b5f640 1 --2- 172.21.15.12:0/2374560554 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea7414d0e0 0x7fea74153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:15.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea79b61640 1 --2- 172.21.15.12:0/2374560554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea741543c0 0x7fea74152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:15.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea79360640 1 -- 172.21.15.12:0/2374560554 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea7414d0e0 msgr2=0x7fea74153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:15.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea79360640 1 --2- 172.21.15.12:0/2374560554 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea7414d0e0 0x7fea74153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea79360640 1 -- 172.21.15.12:0/2374560554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea741543c0 msgr2=0x7fea74152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:15.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea79360640 1 --2- 172.21.15.12:0/2374560554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea741543c0 0x7fea74152770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea79360640 1 -- 172.21.15.12:0/2374560554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea741f3910 con 0x7fea60004d10 2023-10-30T12:56:15.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea79360640 1 --2- 172.21.15.12:0/2374560554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea60004d10 0x7fea7406c020 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fea70011900 tx=0x7fea70011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:15.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea69ffb640 1 -- 172.21.15.12:0/2374560554 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7000fbd0 con 0x7fea60004d10 2023-10-30T12:56:15.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea69ffb640 1 -- 172.21.15.12:0/2374560554 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7000fd30 con 0x7fea60004d10 2023-10-30T12:56:15.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea7b5c4640 1 -- 172.21.15.12:0/2374560554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea741f3bf0 con 0x7fea60004d10 2023-10-30T12:56:15.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea69ffb640 1 -- 172.21.15.12:0/2374560554 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70015660 con 0x7fea60004d10 2023-10-30T12:56:15.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea7b5c4640 1 -- 172.21.15.12:0/2374560554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea740c1b80 con 0x7fea60004d10 2023-10-30T12:56:15.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea7b5c4640 1 -- 172.21.15.12:0/2374560554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea741f4110 con 0x7fea60004d10 2023-10-30T12:56:15.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.727+0000 7fea69ffb640 1 -- 172.21.15.12:0/2374560554 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fea700157c0 con 0x7fea60004d10 2023-10-30T12:56:15.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.731+0000 7fea69ffb640 1 --2- 172.21.15.12:0/2374560554 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fea580664a0 0x7fea58068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:15.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.731+0000 7fea69ffb640 1 -- 172.21.15.12:0/2374560554 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fea70084d10 con 0x7fea60004d10 2023-10-30T12:56:15.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.731+0000 7fea78b5f640 1 --2- 172.21.15.12:0/2374560554 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fea580664a0 0x7fea58068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:15.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.731+0000 7fea78b5f640 1 --2- 172.21.15.12:0/2374560554 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fea580664a0 0x7fea58068960 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fea741f7790 tx=0x7fea64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:15.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.735+0000 7fea69ffb640 1 -- 172.21.15.12:0/2374560554 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fea70051dc0 con 0x7fea60004d10 2023-10-30T12:56:15.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.891+0000 7fea7b5c4640 1 -- 172.21.15.12:0/2374560554 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fea740c6e70 con 0x7fea580664a0 2023-10-30T12:56:15.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.911+0000 7fea69ffb640 1 -- 172.21.15.12:0/2374560554 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292641 (secure 0 0 0) 0x7fea64014400 con 0x7fea580664a0 2023-10-30T12:56:15.918 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:56:15.919 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-10-30T12:56:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.915+0000 7fea7b5c4640 1 -- 172.21.15.12:0/2374560554 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fea580664a0 msgr2=0x7fea58068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.915+0000 7fea7b5c4640 1 --2- 172.21.15.12:0/2374560554 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fea580664a0 0x7fea58068960 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fea741f7790 tx=0x7fea64009040 comp rx=0 tx=0).stop 2023-10-30T12:56:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.915+0000 7fea7b5c4640 1 -- 172.21.15.12:0/2374560554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea60004d10 msgr2=0x7fea7406c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.915+0000 7fea7b5c4640 1 --2- 172.21.15.12:0/2374560554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea60004d10 0x7fea7406c020 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fea70011900 tx=0x7fea70011dc0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.915+0000 7fea7b5c4640 1 -- 172.21.15.12:0/2374560554 shutdown_connections 2023-10-30T12:56:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.915+0000 7fea7b5c4640 1 --2- 172.21.15.12:0/2374560554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea741543c0 0x7fea74152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.915+0000 7fea7b5c4640 1 --2- 172.21.15.12:0/2374560554 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fea580664a0 0x7fea58068960 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.915+0000 7fea7b5c4640 1 --2- 172.21.15.12:0/2374560554 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea7414d0e0 0x7fea74153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.915+0000 7fea7b5c4640 1 --2- 172.21.15.12:0/2374560554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea60004d10 0x7fea7406c020 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.915+0000 7fea7b5c4640 1 -- 172.21.15.12:0/2374560554 >> 172.21.15.12:0/2374560554 conn(0x7fea740c7910 msgr2=0x7fea74156400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.915+0000 7fea7b5c4640 1 -- 172.21.15.12:0/2374560554 shutdown_connections 2023-10-30T12:56:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:15.915+0000 7fea7b5c4640 1 -- 172.21.15.12:0/2374560554 wait complete. 2023-10-30T12:56:15.938 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":690,"stamp":"2023-10-30T12:56:15.409746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1271508492,"num_objects":9804,"num_object_clones":0,"num_object_copies":19608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9804,"num_whiteouts":0,"num_read":5687,"num_read_kb":1040706,"num_write":52424,"num_write_kb":2300570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55711,"ondisk_log_size":55711,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3201824,"kb_used_data":2536096,"kb_used_omap":12,"kb_used_meta":665651,"kb_avail":751772896,"statfs":{"total":773094113280,"available":769815445504,"internally_reserved":0,"allocated":2596962304,"data_stored":2543947272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12729,"internal_metadata":681627207},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":66,"apply_latency_ms":66,"commit_latency_ns":66000000,"apply_latency_ns":66000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50689313,"num_objects":1715,"num_object_clones":0,"num_object_copies":3430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1826,"num_write_kb":50621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004527"},"pg_stats":[{"pgid":"3.3f","version":"55'704","reported_seq":2869,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.052574+0000","last_change":"2023-10-30T12:42:02.472363+0000","last_active":"2023-10-30T12:56:07.052574+0000","last_peered":"2023-10-30T12:56:07.052574+0000","last_clean":"2023-10-30T12:56:07.052574+0000","last_became_active":"2023-10-30T12:35:06.222724+0000","last_became_peered":"2023-10-30T12:35:06.222724+0000","last_unstale":"2023-10-30T12:56:07.052574+0000","last_undegraded":"2023-10-30T12:56:07.052574+0000","last_fullsized":"2023-10-30T12:56:07.052574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:27:39.311531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059524979999999996,"stat_sum":{"num_bytes":14367774,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":73,"num_read_kb":8207,"num_write":683,"num_write_kb":25650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"55'686","reported_seq":2848,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.014947+0000","last_change":"2023-10-30T12:42:02.711331+0000","last_active":"2023-10-30T12:56:07.014947+0000","last_peered":"2023-10-30T12:56:07.014947+0000","last_clean":"2023-10-30T12:56:07.014947+0000","last_became_active":"2023-10-30T12:35:06.182441+0000","last_became_peered":"2023-10-30T12:35:06.182441+0000","last_unstale":"2023-10-30T12:56:07.014947+0000","last_undegraded":"2023-10-30T12:56:07.014947+0000","last_fullsized":"2023-10-30T12:56:07.014947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:00:23.194349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026942979999999999,"stat_sum":{"num_bytes":14474399,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":70,"num_read_kb":10,"num_write":665,"num_write_kb":36839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"55'669","reported_seq":2841,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.004471+0000","last_change":"2023-10-30T12:42:02.724489+0000","last_active":"2023-10-30T12:56:07.004471+0000","last_peered":"2023-10-30T12:56:07.004471+0000","last_clean":"2023-10-30T12:56:07.004471+0000","last_became_active":"2023-10-30T12:35:06.180132+0000","last_became_peered":"2023-10-30T12:35:06.180132+0000","last_unstale":"2023-10-30T12:56:07.004471+0000","last_undegraded":"2023-10-30T12:56:07.004471+0000","last_fullsized":"2023-10-30T12:56:07.004471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:12.922855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003537278,"stat_sum":{"num_bytes":22206444,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":80,"num_read_kb":12305,"num_write":639,"num_write_kb":42559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"55'658","reported_seq":2807,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.150943+0000","last_change":"2023-10-30T12:42:02.742917+0000","last_active":"2023-10-30T12:56:07.150943+0000","last_peered":"2023-10-30T12:56:07.150943+0000","last_clean":"2023-10-30T12:56:07.150943+0000","last_became_active":"2023-10-30T12:35:06.213987+0000","last_became_peered":"2023-10-30T12:35:06.213987+0000","last_unstale":"2023-10-30T12:56:07.150943+0000","last_undegraded":"2023-10-30T12:56:07.150943+0000","last_fullsized":"2023-10-30T12:56:07.150943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:39:55.675219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013528057,"stat_sum":{"num_bytes":18334048,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":57,"num_read_kb":8205,"num_write":618,"num_write_kb":28313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"55'637","reported_seq":2778,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.021805+0000","last_change":"2023-10-30T12:42:02.470142+0000","last_active":"2023-10-30T12:56:07.021805+0000","last_peered":"2023-10-30T12:56:07.021805+0000","last_clean":"2023-10-30T12:56:07.021805+0000","last_became_active":"2023-10-30T12:35:06.214964+0000","last_became_peered":"2023-10-30T12:35:06.214964+0000","last_unstale":"2023-10-30T12:56:07.021805+0000","last_undegraded":"2023-10-30T12:56:07.021805+0000","last_fullsized":"2023-10-30T12:56:07.021805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:56:51.594442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046834600000000004,"stat_sum":{"num_bytes":8551162,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":51,"num_read_kb":4112,"num_write":602,"num_write_kb":25707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"55'662","reported_seq":2813,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.149801+0000","last_change":"2023-10-30T12:42:02.747177+0000","last_active":"2023-10-30T12:56:07.149801+0000","last_peered":"2023-10-30T12:56:07.149801+0000","last_clean":"2023-10-30T12:56:07.149801+0000","last_became_active":"2023-10-30T12:35:06.214219+0000","last_became_peered":"2023-10-30T12:35:06.214219+0000","last_unstale":"2023-10-30T12:56:07.149801+0000","last_undegraded":"2023-10-30T12:56:07.149801+0000","last_fullsized":"2023-10-30T12:56:07.149801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:36:56.070585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015862434000000002,"stat_sum":{"num_bytes":14711969,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":61,"num_read_kb":16,"num_write":631,"num_write_kb":33150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"55'668","reported_seq":2809,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.029181+0000","last_change":"2023-10-30T12:42:02.467959+0000","last_active":"2023-10-30T12:56:07.029181+0000","last_peered":"2023-10-30T12:56:07.029181+0000","last_clean":"2023-10-30T12:56:07.029181+0000","last_became_active":"2023-10-30T12:35:06.216468+0000","last_became_peered":"2023-10-30T12:35:06.216468+0000","last_unstale":"2023-10-30T12:56:07.029181+0000","last_undegraded":"2023-10-30T12:56:07.029181+0000","last_fullsized":"2023-10-30T12:56:07.029181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:19:43.632233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031309480000000002,"stat_sum":{"num_bytes":9417687,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":52,"num_read_kb":12216,"num_write":636,"num_write_kb":35921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"55'605","reported_seq":2751,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.153162+0000","last_change":"2023-10-30T12:42:02.241155+0000","last_active":"2023-10-30T12:56:07.153162+0000","last_peered":"2023-10-30T12:56:07.153162+0000","last_clean":"2023-10-30T12:56:07.153162+0000","last_became_active":"2023-10-30T12:35:06.214384+0000","last_became_peered":"2023-10-30T12:35:06.214384+0000","last_unstale":"2023-10-30T12:56:07.153162+0000","last_undegraded":"2023-10-30T12:56:07.153162+0000","last_fullsized":"2023-10-30T12:56:07.153162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:44:07.845577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045552559999999997,"stat_sum":{"num_bytes":9683176,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":56,"num_read_kb":15,"num_write":587,"num_write_kb":23249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"55'660","reported_seq":2822,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.060712+0000","last_change":"2023-10-30T12:42:02.473886+0000","last_active":"2023-10-30T12:56:07.060712+0000","last_peered":"2023-10-30T12:56:07.060712+0000","last_clean":"2023-10-30T12:56:07.060712+0000","last_became_active":"2023-10-30T12:35:06.222692+0000","last_became_peered":"2023-10-30T12:35:06.222692+0000","last_unstale":"2023-10-30T12:56:07.060712+0000","last_undegraded":"2023-10-30T12:56:07.060712+0000","last_fullsized":"2023-10-30T12:56:07.060712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:12:07.552579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074617599999999996,"stat_sum":{"num_bytes":5453765,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":72,"num_read_kb":4115,"num_write":623,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"55'653","reported_seq":2807,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.014218+0000","last_change":"2023-10-30T12:42:02.739661+0000","last_active":"2023-10-30T12:56:07.014218+0000","last_peered":"2023-10-30T12:56:07.014218+0000","last_clean":"2023-10-30T12:56:07.014218+0000","last_became_active":"2023-10-30T12:35:06.215384+0000","last_became_peered":"2023-10-30T12:35:06.215384+0000","last_unstale":"2023-10-30T12:56:07.014218+0000","last_undegraded":"2023-10-30T12:56:07.014218+0000","last_fullsized":"2023-10-30T12:56:07.014218+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:02:59.748923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080059940000000007,"stat_sum":{"num_bytes":17838264,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":64,"num_read_kb":28,"num_write":624,"num_write_kb":19661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"55'661","reported_seq":2818,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.991403+0000","last_change":"2023-10-30T12:42:02.739462+0000","last_active":"2023-10-30T12:56:06.991403+0000","last_peered":"2023-10-30T12:56:06.991403+0000","last_clean":"2023-10-30T12:56:06.991403+0000","last_became_active":"2023-10-30T12:35:06.215355+0000","last_became_peered":"2023-10-30T12:35:06.215355+0000","last_unstale":"2023-10-30T12:56:06.991403+0000","last_undegraded":"2023-10-30T12:56:06.991403+0000","last_fullsized":"2023-10-30T12:56:06.991403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:04:51.314044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011778769,"stat_sum":{"num_bytes":10379280,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":67,"num_read_kb":8209,"num_write":639,"num_write_kb":25731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"55'640","reported_seq":2795,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.010644+0000","last_change":"2023-10-30T12:42:02.253484+0000","last_active":"2023-10-30T12:56:07.010644+0000","last_peered":"2023-10-30T12:56:07.010644+0000","last_clean":"2023-10-30T12:56:07.010644+0000","last_became_active":"2023-10-30T12:35:06.196503+0000","last_became_peered":"2023-10-30T12:35:06.196503+0000","last_unstale":"2023-10-30T12:56:07.010644+0000","last_undegraded":"2023-10-30T12:56:07.010644+0000","last_fullsized":"2023-10-30T12:56:07.010644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:33:54.826649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030385149999999999,"stat_sum":{"num_bytes":16049079,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":65,"num_read_kb":7648,"num_write":614,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"55'689","reported_seq":2856,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.137866+0000","last_change":"2023-10-30T12:42:02.241621+0000","last_active":"2023-10-30T12:56:07.137866+0000","last_peered":"2023-10-30T12:56:07.137866+0000","last_clean":"2023-10-30T12:56:07.137866+0000","last_became_active":"2023-10-30T12:35:06.195408+0000","last_became_peered":"2023-10-30T12:35:06.195408+0000","last_unstale":"2023-10-30T12:56:07.137866+0000","last_undegraded":"2023-10-30T12:56:07.137866+0000","last_fullsized":"2023-10-30T12:56:07.137866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:06:33.968324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004560812,"stat_sum":{"num_bytes":29038845,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":77,"num_read_kb":9808,"num_write":655,"num_write_kb":48120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"55'657","reported_seq":2819,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.112877+0000","last_change":"2023-10-30T12:42:02.722894+0000","last_active":"2023-10-30T12:56:07.112877+0000","last_peered":"2023-10-30T12:56:07.112877+0000","last_clean":"2023-10-30T12:56:07.112877+0000","last_became_active":"2023-10-30T12:35:06.182573+0000","last_became_peered":"2023-10-30T12:35:06.182573+0000","last_unstale":"2023-10-30T12:56:07.112877+0000","last_undegraded":"2023-10-30T12:56:07.112877+0000","last_fullsized":"2023-10-30T12:56:07.112877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:42:24.313362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003020062,"stat_sum":{"num_bytes":14507519,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":71,"num_read_kb":4113,"num_write":633,"num_write_kb":31547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"55'673","reported_seq":2827,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.046529+0000","last_change":"2023-10-30T12:42:02.710474+0000","last_active":"2023-10-30T12:56:07.046529+0000","last_peered":"2023-10-30T12:56:07.046529+0000","last_clean":"2023-10-30T12:56:07.046529+0000","last_became_active":"2023-10-30T12:35:06.185290+0000","last_became_peered":"2023-10-30T12:35:06.185290+0000","last_unstale":"2023-10-30T12:56:07.046529+0000","last_undegraded":"2023-10-30T12:56:07.046529+0000","last_fullsized":"2023-10-30T12:56:07.046529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:18:23.116561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002282292,"stat_sum":{"num_bytes":14565906,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":63,"num_read_kb":11730,"num_write":637,"num_write_kb":27497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"55'678","reported_seq":2828,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.129521+0000","last_change":"2023-10-30T12:42:02.748073+0000","last_active":"2023-10-30T12:56:07.129521+0000","last_peered":"2023-10-30T12:56:07.129521+0000","last_clean":"2023-10-30T12:56:07.129521+0000","last_became_active":"2023-10-30T12:35:06.213627+0000","last_became_peered":"2023-10-30T12:35:06.213627+0000","last_unstale":"2023-10-30T12:56:07.129521+0000","last_undegraded":"2023-10-30T12:56:07.129521+0000","last_fullsized":"2023-10-30T12:56:07.129521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:56:05.792085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014095760000000001,"stat_sum":{"num_bytes":9785326,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":59,"num_read_kb":4114,"num_write":638,"num_write_kb":31646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"55'710","reported_seq":2863,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.066514+0000","last_change":"2023-10-30T12:42:02.745783+0000","last_active":"2023-10-30T12:56:07.066514+0000","last_peered":"2023-10-30T12:56:07.066514+0000","last_clean":"2023-10-30T12:56:07.066514+0000","last_became_active":"2023-10-30T12:35:06.221696+0000","last_became_peered":"2023-10-30T12:35:06.221696+0000","last_unstale":"2023-10-30T12:56:07.066514+0000","last_undegraded":"2023-10-30T12:56:07.066514+0000","last_fullsized":"2023-10-30T12:56:07.066514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:26:40.837242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015743672,"stat_sum":{"num_bytes":28263392,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":62,"num_read_kb":16401,"num_write":680,"num_write_kb":59327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"55'660","reported_seq":2833,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.992539+0000","last_change":"2023-10-30T12:42:02.242356+0000","last_active":"2023-10-30T12:56:06.992539+0000","last_peered":"2023-10-30T12:56:06.992539+0000","last_clean":"2023-10-30T12:56:06.992539+0000","last_became_active":"2023-10-30T12:35:06.198074+0000","last_became_peered":"2023-10-30T12:35:06.198074+0000","last_unstale":"2023-10-30T12:56:06.992539+0000","last_undegraded":"2023-10-30T12:56:06.992539+0000","last_fullsized":"2023-10-30T12:56:06.992539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:55:38.021093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023389270000000002,"stat_sum":{"num_bytes":28742059,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":82,"num_read_kb":27,"num_write":629,"num_write_kb":39704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"55'675","reported_seq":2826,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.023088+0000","last_change":"2023-10-30T12:42:02.473536+0000","last_active":"2023-10-30T12:56:07.023088+0000","last_peered":"2023-10-30T12:56:07.023088+0000","last_clean":"2023-10-30T12:56:07.023088+0000","last_became_active":"2023-10-30T12:35:06.215935+0000","last_became_peered":"2023-10-30T12:35:06.215935+0000","last_unstale":"2023-10-30T12:56:07.023088+0000","last_undegraded":"2023-10-30T12:56:07.023088+0000","last_fullsized":"2023-10-30T12:56:07.023088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:39:23.851788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062537060000000004,"stat_sum":{"num_bytes":21375724,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":60,"num_read_kb":12305,"num_write":641,"num_write_kb":39315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"55'672","reported_seq":2838,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.992738+0000","last_change":"2023-10-30T12:42:02.739007+0000","last_active":"2023-10-30T12:56:06.992738+0000","last_peered":"2023-10-30T12:56:06.992738+0000","last_clean":"2023-10-30T12:56:06.992738+0000","last_became_active":"2023-10-30T12:35:06.199812+0000","last_became_peered":"2023-10-30T12:35:06.199812+0000","last_unstale":"2023-10-30T12:56:06.992738+0000","last_undegraded":"2023-10-30T12:56:06.992738+0000","last_fullsized":"2023-10-30T12:56:06.992738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:07:11.762372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011375481999999999,"stat_sum":{"num_bytes":6241435,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":75,"num_read_kb":20150,"num_write":638,"num_write_kb":36226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"55'680","reported_seq":2849,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.113501+0000","last_change":"2023-10-30T12:42:02.725299+0000","last_active":"2023-10-30T12:56:07.113501+0000","last_peered":"2023-10-30T12:56:07.113501+0000","last_clean":"2023-10-30T12:56:07.113501+0000","last_became_active":"2023-10-30T12:35:06.216861+0000","last_became_peered":"2023-10-30T12:35:06.216861+0000","last_unstale":"2023-10-30T12:56:07.113501+0000","last_undegraded":"2023-10-30T12:56:07.113501+0000","last_fullsized":"2023-10-30T12:56:07.113501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:58:16.568819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040841849999999997,"stat_sum":{"num_bytes":12253674,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":78,"num_read_kb":8213,"num_write":653,"num_write_kb":26257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"55'705","reported_seq":2854,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.001507+0000","last_change":"2023-10-30T12:42:02.241419+0000","last_active":"2023-10-30T12:56:07.001507+0000","last_peered":"2023-10-30T12:56:07.001507+0000","last_clean":"2023-10-30T12:56:07.001507+0000","last_became_active":"2023-10-30T12:35:46.122517+0000","last_became_peered":"2023-10-30T12:35:46.122517+0000","last_unstale":"2023-10-30T12:56:07.001507+0000","last_undegraded":"2023-10-30T12:56:07.001507+0000","last_fullsized":"2023-10-30T12:56:07.001507+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:10:07.253977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020435240000000001,"stat_sum":{"num_bytes":19343344,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":61,"num_read_kb":21,"num_write":677,"num_write_kb":29921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"55'676","reported_seq":2847,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.095881+0000","last_change":"2023-10-30T12:42:02.241859+0000","last_active":"2023-10-30T12:56:07.095881+0000","last_peered":"2023-10-30T12:56:07.095881+0000","last_clean":"2023-10-30T12:56:07.095881+0000","last_became_active":"2023-10-30T12:35:06.182314+0000","last_became_peered":"2023-10-30T12:35:06.182314+0000","last_unstale":"2023-10-30T12:56:07.095881+0000","last_undegraded":"2023-10-30T12:56:07.095881+0000","last_fullsized":"2023-10-30T12:56:07.095881+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:20:19.153859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051111200000000002,"stat_sum":{"num_bytes":5339400,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":80,"num_read_kb":15696,"num_write":644,"num_write_kb":25369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"55'681","reported_seq":2839,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.120096+0000","last_change":"2023-10-30T12:42:02.748079+0000","last_active":"2023-10-30T12:56:07.120096+0000","last_peered":"2023-10-30T12:56:07.120096+0000","last_clean":"2023-10-30T12:56:07.120096+0000","last_became_active":"2023-10-30T12:35:06.213402+0000","last_became_peered":"2023-10-30T12:35:06.213402+0000","last_unstale":"2023-10-30T12:56:07.120096+0000","last_undegraded":"2023-10-30T12:56:07.120096+0000","last_fullsized":"2023-10-30T12:56:07.120096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:11.732115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014055767,"stat_sum":{"num_bytes":10176680,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":67,"num_read_kb":4112,"num_write":643,"num_write_kb":20459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"55'660","reported_seq":2810,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.118693+0000","last_change":"2023-10-30T12:42:02.242578+0000","last_active":"2023-10-30T12:56:07.118693+0000","last_peered":"2023-10-30T12:56:07.118693+0000","last_clean":"2023-10-30T12:56:07.118693+0000","last_became_active":"2023-10-30T12:35:06.188298+0000","last_became_peered":"2023-10-30T12:35:06.188298+0000","last_unstale":"2023-10-30T12:56:07.118693+0000","last_undegraded":"2023-10-30T12:56:07.118693+0000","last_fullsized":"2023-10-30T12:56:07.118693+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:56:11.006974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053580889999999999,"stat_sum":{"num_bytes":4997964,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":59,"num_read_kb":8204,"num_write":612,"num_write_kb":44089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"55'741","reported_seq":2919,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.030017+0000","last_change":"2023-10-30T12:42:02.740157+0000","last_active":"2023-10-30T12:56:07.030017+0000","last_peered":"2023-10-30T12:56:07.030017+0000","last_clean":"2023-10-30T12:56:07.030017+0000","last_became_active":"2023-10-30T12:35:06.202507+0000","last_became_peered":"2023-10-30T12:35:06.202507+0000","last_unstale":"2023-10-30T12:56:07.030017+0000","last_undegraded":"2023-10-30T12:56:07.030017+0000","last_fullsized":"2023-10-30T12:56:07.030017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:55.774699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082500649999999991,"stat_sum":{"num_bytes":6483256,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":87,"num_read_kb":1892,"num_write":707,"num_write_kb":28689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"55'657","reported_seq":2811,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.123179+0000","last_change":"2023-10-30T12:42:02.710762+0000","last_active":"2023-10-30T12:56:07.123179+0000","last_peered":"2023-10-30T12:56:07.123179+0000","last_clean":"2023-10-30T12:56:07.123179+0000","last_became_active":"2023-10-30T12:35:06.170030+0000","last_became_peered":"2023-10-30T12:35:06.170030+0000","last_unstale":"2023-10-30T12:56:07.123179+0000","last_undegraded":"2023-10-30T12:56:07.123179+0000","last_fullsized":"2023-10-30T12:56:07.123179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:04.189974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024132049999999999,"stat_sum":{"num_bytes":15276436,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":63,"num_read_kb":15,"num_write":628,"num_write_kb":22058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"55'623","reported_seq":2792,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.158356+0000","last_change":"2023-10-30T12:42:02.242975+0000","last_active":"2023-10-30T12:56:07.158356+0000","last_peered":"2023-10-30T12:56:07.158356+0000","last_clean":"2023-10-30T12:56:07.158356+0000","last_became_active":"2023-10-30T12:35:06.189171+0000","last_became_peered":"2023-10-30T12:35:06.189171+0000","last_unstale":"2023-10-30T12:56:07.158356+0000","last_undegraded":"2023-10-30T12:56:07.158356+0000","last_fullsized":"2023-10-30T12:56:07.158356+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:00:28.317415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056904870000000001,"stat_sum":{"num_bytes":13986941,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":78,"num_read_kb":12372,"num_write":595,"num_write_kb":31185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"55'639","reported_seq":2791,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.133710+0000","last_change":"2023-10-30T12:42:02.725392+0000","last_active":"2023-10-30T12:56:07.133710+0000","last_peered":"2023-10-30T12:56:07.133710+0000","last_clean":"2023-10-30T12:56:07.133710+0000","last_became_active":"2023-10-30T12:35:06.183348+0000","last_became_peered":"2023-10-30T12:35:06.183348+0000","last_unstale":"2023-10-30T12:56:07.133710+0000","last_undegraded":"2023-10-30T12:56:07.133710+0000","last_fullsized":"2023-10-30T12:56:07.133710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:15:11.322585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041317979999999999,"stat_sum":{"num_bytes":14536653,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":60,"num_read_kb":7678,"num_write":614,"num_write_kb":36281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"55'712","reported_seq":2879,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.104341+0000","last_change":"2023-10-30T12:42:02.747343+0000","last_active":"2023-10-30T12:56:07.104341+0000","last_peered":"2023-10-30T12:56:07.104341+0000","last_clean":"2023-10-30T12:56:07.104341+0000","last_became_active":"2023-10-30T12:35:06.196368+0000","last_became_peered":"2023-10-30T12:35:06.196368+0000","last_unstale":"2023-10-30T12:56:07.104341+0000","last_undegraded":"2023-10-30T12:56:07.104341+0000","last_fullsized":"2023-10-30T12:56:07.104341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:53:23.269916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015569382999999999,"stat_sum":{"num_bytes":14455388,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":75,"num_read_kb":9352,"num_write":686,"num_write_kb":32708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"55'735","reported_seq":2899,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.035413+0000","last_change":"2023-10-30T12:42:02.241047+0000","last_active":"2023-10-30T12:56:07.035413+0000","last_peered":"2023-10-30T12:56:07.035413+0000","last_clean":"2023-10-30T12:56:07.035413+0000","last_became_active":"2023-10-30T12:35:06.213994+0000","last_became_peered":"2023-10-30T12:35:06.213994+0000","last_unstale":"2023-10-30T12:56:07.035413+0000","last_undegraded":"2023-10-30T12:56:07.035413+0000","last_fullsized":"2023-10-30T12:56:07.035413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:49:43.203436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001873502,"stat_sum":{"num_bytes":23237117,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":72,"num_read_kb":4118,"num_write":703,"num_write_kb":32259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"55'672","reported_seq":2834,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.181318+0000","last_change":"2023-10-30T12:42:02.738216+0000","last_active":"2023-10-30T12:56:07.181318+0000","last_peered":"2023-10-30T12:56:07.181318+0000","last_clean":"2023-10-30T12:56:07.181318+0000","last_became_active":"2023-10-30T12:35:06.197647+0000","last_became_peered":"2023-10-30T12:35:06.197647+0000","last_unstale":"2023-10-30T12:56:07.181318+0000","last_undegraded":"2023-10-30T12:56:07.181318+0000","last_fullsized":"2023-10-30T12:56:07.181318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:21:00.884446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011378932,"stat_sum":{"num_bytes":18305029,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":70,"num_read_kb":8212,"num_write":638,"num_write_kb":28193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"55'634","reported_seq":2786,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.949834+0000","last_change":"2023-10-30T12:42:02.251618+0000","last_active":"2023-10-30T12:56:06.949834+0000","last_peered":"2023-10-30T12:56:06.949834+0000","last_clean":"2023-10-30T12:56:06.949834+0000","last_became_active":"2023-10-30T12:35:06.167558+0000","last_became_peered":"2023-10-30T12:35:06.167558+0000","last_unstale":"2023-10-30T12:56:06.949834+0000","last_undegraded":"2023-10-30T12:56:06.949834+0000","last_fullsized":"2023-10-30T12:56:06.949834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:11:30.490780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048907120000000002,"stat_sum":{"num_bytes":9666676,"num_objects":132,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":0,"num_read":62,"num_read_kb":17,"num_write":598,"num_write_kb":15613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"55'233","reported_seq":298,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:07.513824+0000","last_change":"2023-10-30T12:35:05.200844+0000","last_active":"2023-10-30T12:55:07.513824+0000","last_peered":"2023-10-30T12:55:07.513824+0000","last_clean":"2023-10-30T12:55:07.513824+0000","last_became_active":"2023-10-30T12:35:05.200169+0000","last_became_peered":"2023-10-30T12:35:05.200169+0000","last_unstale":"2023-10-30T12:55:07.513824+0000","last_undegraded":"2023-10-30T12:55:07.513824+0000","last_fullsized":"2023-10-30T12:55:07.513824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:07:19.571357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":36867,"num_write":183,"num_write_kb":12445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"55'646","reported_seq":2797,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.123456+0000","last_change":"2023-10-30T12:42:02.725641+0000","last_active":"2023-10-30T12:56:07.123456+0000","last_peered":"2023-10-30T12:56:07.123456+0000","last_clean":"2023-10-30T12:56:07.123456+0000","last_became_active":"2023-10-30T12:35:06.194689+0000","last_became_peered":"2023-10-30T12:35:06.194689+0000","last_unstale":"2023-10-30T12:56:07.123456+0000","last_undegraded":"2023-10-30T12:56:07.123456+0000","last_fullsized":"2023-10-30T12:56:07.123456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:38:22.949407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004291034,"stat_sum":{"num_bytes":10877495,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":61,"num_read_kb":18,"num_write":619,"num_write_kb":24664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"55'974","reported_seq":1165,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:34.540942+0000","last_change":"2023-10-30T12:35:05.183569+0000","last_active":"2023-10-30T12:55:34.540942+0000","last_peered":"2023-10-30T12:55:34.540942+0000","last_clean":"2023-10-30T12:55:34.540942+0000","last_became_active":"2023-10-30T12:35:05.182761+0000","last_became_peered":"2023-10-30T12:35:05.182761+0000","last_unstale":"2023-10-30T12:55:34.540942+0000","last_undegraded":"2023-10-30T12:55:34.540942+0000","last_fullsized":"2023-10-30T12:55:34.540942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:49:44.311814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25625406,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":150,"num_read_kb":45229,"num_write":895,"num_write_kb":25503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"55'430","reported_seq":635,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.685247+0000","last_change":"2023-10-30T12:35:05.181278+0000","last_active":"2023-10-30T12:56:09.685247+0000","last_peered":"2023-10-30T12:56:09.685247+0000","last_clean":"2023-10-30T12:56:09.685247+0000","last_became_active":"2023-10-30T12:35:05.180730+0000","last_became_peered":"2023-10-30T12:35:05.180730+0000","last_unstale":"2023-10-30T12:56:09.685247+0000","last_undegraded":"2023-10-30T12:56:09.685247+0000","last_fullsized":"2023-10-30T12:56:09.685247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:07:10.828707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8578635,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":162,"num_read_kb":8759,"num_write":365,"num_write_kb":8612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"55'760","reported_seq":2929,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.115297+0000","last_change":"2023-10-30T12:42:02.240966+0000","last_active":"2023-10-30T12:56:07.115297+0000","last_peered":"2023-10-30T12:56:07.115297+0000","last_clean":"2023-10-30T12:56:07.115297+0000","last_became_active":"2023-10-30T12:35:46.051057+0000","last_became_peered":"2023-10-30T12:35:46.051057+0000","last_unstale":"2023-10-30T12:56:07.115297+0000","last_undegraded":"2023-10-30T12:56:07.115297+0000","last_fullsized":"2023-10-30T12:56:07.115297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:39:59.687314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018351520000000001,"stat_sum":{"num_bytes":18874907,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":82,"num_read_kb":16408,"num_write":717,"num_write_kb":52822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"55'983","reported_seq":1049,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:04.638735+0000","last_change":"2023-10-30T12:35:05.202539+0000","last_active":"2023-10-30T12:55:04.638735+0000","last_peered":"2023-10-30T12:55:04.638735+0000","last_clean":"2023-10-30T12:55:04.638735+0000","last_became_active":"2023-10-30T12:35:05.201853+0000","last_became_peered":"2023-10-30T12:35:05.201853+0000","last_unstale":"2023-10-30T12:55:04.638735+0000","last_undegraded":"2023-10-30T12:55:04.638735+0000","last_fullsized":"2023-10-30T12:55:04.638735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:33:33.151433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":38325,"num_write":914,"num_write_kb":16753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"55'707","reported_seq":2870,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.090289+0000","last_change":"2023-10-30T12:42:02.710378+0000","last_active":"2023-10-30T12:56:07.090289+0000","last_peered":"2023-10-30T12:56:07.090289+0000","last_clean":"2023-10-30T12:56:07.090289+0000","last_became_active":"2023-10-30T12:35:06.170179+0000","last_became_peered":"2023-10-30T12:35:06.170179+0000","last_unstale":"2023-10-30T12:56:07.090289+0000","last_undegraded":"2023-10-30T12:56:07.090289+0000","last_fullsized":"2023-10-30T12:56:07.090289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:21:27.326814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021794980000000002,"stat_sum":{"num_bytes":14241209,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":73,"num_read_kb":12307,"num_write":678,"num_write_kb":28673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"55'145","reported_seq":204,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:54.873264+0000","last_change":"2023-10-30T12:35:05.198546+0000","last_active":"2023-10-30T12:55:54.873264+0000","last_peered":"2023-10-30T12:55:54.873264+0000","last_clean":"2023-10-30T12:55:54.873264+0000","last_became_active":"2023-10-30T12:35:05.197606+0000","last_became_peered":"2023-10-30T12:35:05.197606+0000","last_unstale":"2023-10-30T12:55:54.873264+0000","last_undegraded":"2023-10-30T12:55:54.873264+0000","last_fullsized":"2023-10-30T12:55:54.873264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:26:57.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":90,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"55'666","reported_seq":2833,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.026011+0000","last_change":"2023-10-30T12:42:02.745501+0000","last_active":"2023-10-30T12:56:07.026011+0000","last_peered":"2023-10-30T12:56:07.026011+0000","last_clean":"2023-10-30T12:56:07.026011+0000","last_became_active":"2023-10-30T12:35:06.225392+0000","last_became_peered":"2023-10-30T12:35:06.225392+0000","last_unstale":"2023-10-30T12:56:07.026011+0000","last_undegraded":"2023-10-30T12:56:07.026011+0000","last_fullsized":"2023-10-30T12:56:07.026011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:06:30.011008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01552166,"stat_sum":{"num_bytes":9602206,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":77,"num_read_kb":18,"num_write":639,"num_write_kb":18963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"55'480","reported_seq":541,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.722504+0000","last_change":"2023-10-30T12:35:05.197614+0000","last_active":"2023-10-30T12:56:09.722504+0000","last_peered":"2023-10-30T12:56:09.722504+0000","last_clean":"2023-10-30T12:56:09.722504+0000","last_became_active":"2023-10-30T12:35:05.196899+0000","last_became_peered":"2023-10-30T12:35:05.196899+0000","last_unstale":"2023-10-30T12:56:09.722504+0000","last_undegraded":"2023-10-30T12:56:09.722504+0000","last_fullsized":"2023-10-30T12:56:09.722504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:21:00.152426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14221472,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":14173,"num_write":412,"num_write_kb":14118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"55'672","reported_seq":2825,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.087588+0000","last_change":"2023-10-30T12:42:02.724824+0000","last_active":"2023-10-30T12:56:07.087588+0000","last_peered":"2023-10-30T12:56:07.087588+0000","last_clean":"2023-10-30T12:56:07.087588+0000","last_became_active":"2023-10-30T12:35:06.222057+0000","last_became_peered":"2023-10-30T12:35:06.222057+0000","last_unstale":"2023-10-30T12:56:07.087588+0000","last_undegraded":"2023-10-30T12:56:07.087588+0000","last_fullsized":"2023-10-30T12:56:07.087588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:13:07.261307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004210915,"stat_sum":{"num_bytes":31275308,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":63,"num_read_kb":12308,"num_write":645,"num_write_kb":50366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"55'312","reported_seq":375,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:08.116088+0000","last_change":"2023-10-30T12:35:05.197501+0000","last_active":"2023-10-30T12:56:08.116088+0000","last_peered":"2023-10-30T12:56:08.116088+0000","last_clean":"2023-10-30T12:56:08.116088+0000","last_became_active":"2023-10-30T12:35:05.197190+0000","last_became_peered":"2023-10-30T12:35:05.197190+0000","last_unstale":"2023-10-30T12:56:08.116088+0000","last_undegraded":"2023-10-30T12:56:08.116088+0000","last_fullsized":"2023-10-30T12:56:08.116088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:53:31.121721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":4111,"num_write":258,"num_write_kb":8429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"55'651","reported_seq":2802,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.024707+0000","last_change":"2023-10-30T12:42:02.253699+0000","last_active":"2023-10-30T12:56:07.024707+0000","last_peered":"2023-10-30T12:56:07.024707+0000","last_clean":"2023-10-30T12:56:07.024707+0000","last_became_active":"2023-10-30T12:35:06.204122+0000","last_became_peered":"2023-10-30T12:35:06.204122+0000","last_unstale":"2023-10-30T12:56:07.024707+0000","last_undegraded":"2023-10-30T12:56:07.024707+0000","last_fullsized":"2023-10-30T12:56:07.024707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:59:50.717088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030324050000000002,"stat_sum":{"num_bytes":15521520,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":60,"num_read_kb":8209,"num_write":621,"num_write_kb":41873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"55'359","reported_seq":431,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.533756+0000","last_change":"2023-10-30T12:35:05.180005+0000","last_active":"2023-10-30T12:56:09.533756+0000","last_peered":"2023-10-30T12:56:09.533756+0000","last_clean":"2023-10-30T12:56:09.533756+0000","last_became_active":"2023-10-30T12:35:05.179547+0000","last_became_peered":"2023-10-30T12:35:05.179547+0000","last_unstale":"2023-10-30T12:56:09.533756+0000","last_undegraded":"2023-10-30T12:56:09.533756+0000","last_fullsized":"2023-10-30T12:56:09.533756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:13:03.137433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":24600,"num_write":299,"num_write_kb":12510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"55'628","reported_seq":2779,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.133221+0000","last_change":"2023-10-30T12:42:02.473160+0000","last_active":"2023-10-30T12:56:07.133221+0000","last_peered":"2023-10-30T12:56:07.133221+0000","last_clean":"2023-10-30T12:56:07.133221+0000","last_became_active":"2023-10-30T12:35:06.215128+0000","last_became_peered":"2023-10-30T12:35:06.215128+0000","last_unstale":"2023-10-30T12:56:07.133221+0000","last_undegraded":"2023-10-30T12:56:07.133221+0000","last_fullsized":"2023-10-30T12:56:07.133221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:47:38.300886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064662909999999999,"stat_sum":{"num_bytes":22716254,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":61,"num_read_kb":11,"num_write":594,"num_write_kb":27628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"55'209","reported_seq":283,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.553645+0000","last_change":"2023-10-30T12:35:05.190041+0000","last_active":"2023-10-30T12:56:04.553645+0000","last_peered":"2023-10-30T12:56:04.553645+0000","last_clean":"2023-10-30T12:56:04.553645+0000","last_became_active":"2023-10-30T12:35:05.189573+0000","last_became_peered":"2023-10-30T12:35:05.189573+0000","last_unstale":"2023-10-30T12:56:04.553645+0000","last_undegraded":"2023-10-30T12:56:04.553645+0000","last_fullsized":"2023-10-30T12:56:04.553645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:02:57.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":24624,"num_write":154,"num_write_kb":8340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"55'702","reported_seq":2870,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.111628+0000","last_change":"2023-10-30T12:42:02.724827+0000","last_active":"2023-10-30T12:56:07.111628+0000","last_peered":"2023-10-30T12:56:07.111628+0000","last_clean":"2023-10-30T12:56:07.111628+0000","last_became_active":"2023-10-30T12:35:06.216885+0000","last_became_peered":"2023-10-30T12:35:06.216885+0000","last_unstale":"2023-10-30T12:56:07.111628+0000","last_undegraded":"2023-10-30T12:56:07.111628+0000","last_fullsized":"2023-10-30T12:56:07.111628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:50:48.181232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042006370000000001,"stat_sum":{"num_bytes":18982563,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":77,"num_read_kb":16402,"num_write":665,"num_write_kb":45070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"55'232","reported_seq":293,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.565577+0000","last_change":"2023-10-30T12:35:05.203198+0000","last_active":"2023-10-30T12:56:04.565577+0000","last_peered":"2023-10-30T12:56:04.565577+0000","last_clean":"2023-10-30T12:56:04.565577+0000","last_became_active":"2023-10-30T12:35:05.202612+0000","last_became_peered":"2023-10-30T12:35:05.202612+0000","last_unstale":"2023-10-30T12:56:04.565577+0000","last_undegraded":"2023-10-30T12:56:04.565577+0000","last_fullsized":"2023-10-30T12:56:04.565577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:55.986236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":181,"num_write_kb":8338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"55'665","reported_seq":2831,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.009342+0000","last_change":"2023-10-30T12:42:02.471602+0000","last_active":"2023-10-30T12:56:07.009342+0000","last_peered":"2023-10-30T12:56:07.009342+0000","last_clean":"2023-10-30T12:56:07.009342+0000","last_became_active":"2023-10-30T12:35:06.215553+0000","last_became_peered":"2023-10-30T12:35:06.215553+0000","last_unstale":"2023-10-30T12:56:07.009342+0000","last_undegraded":"2023-10-30T12:56:07.009342+0000","last_fullsized":"2023-10-30T12:56:07.009342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:58:00.184959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060418609999999999,"stat_sum":{"num_bytes":18953195,"num_objects":130,"num_object_clones":0,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":0,"num_read":76,"num_read_kb":33,"num_write":640,"num_write_kb":32599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"55'289","reported_seq":360,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:27.641002+0000","last_change":"2023-10-30T12:35:05.181882+0000","last_active":"2023-10-30T12:55:27.641002+0000","last_peered":"2023-10-30T12:55:27.641002+0000","last_clean":"2023-10-30T12:55:27.641002+0000","last_became_active":"2023-10-30T12:35:05.181115+0000","last_became_peered":"2023-10-30T12:35:05.181115+0000","last_unstale":"2023-10-30T12:55:27.641002+0000","last_undegraded":"2023-10-30T12:55:27.641002+0000","last_fullsized":"2023-10-30T12:55:27.641002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:18:55.496940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":43239,"num_write":227,"num_write_kb":16591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"55'646","reported_seq":2797,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.181423+0000","last_change":"2023-10-30T12:42:02.738621+0000","last_active":"2023-10-30T12:56:07.181423+0000","last_peered":"2023-10-30T12:56:07.181423+0000","last_clean":"2023-10-30T12:56:07.181423+0000","last_became_active":"2023-10-30T12:35:06.197722+0000","last_became_peered":"2023-10-30T12:35:06.197722+0000","last_unstale":"2023-10-30T12:56:07.181423+0000","last_undegraded":"2023-10-30T12:56:07.181423+0000","last_fullsized":"2023-10-30T12:56:07.181423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:56:33.480519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011777173,"stat_sum":{"num_bytes":11054792,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":60,"num_read_kb":16,"num_write":607,"num_write_kb":21388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"55'481","reported_seq":572,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.552101+0000","last_change":"2023-10-30T12:35:05.169455+0000","last_active":"2023-10-30T12:56:04.552101+0000","last_peered":"2023-10-30T12:56:04.552101+0000","last_clean":"2023-10-30T12:56:04.552101+0000","last_became_active":"2023-10-30T12:35:05.169130+0000","last_became_peered":"2023-10-30T12:35:05.169130+0000","last_unstale":"2023-10-30T12:56:04.552101+0000","last_undegraded":"2023-10-30T12:56:04.552101+0000","last_fullsized":"2023-10-30T12:56:04.552101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:59:14.096315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583024,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":18341,"num_write":415,"num_write_kb":12551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"55'687","reported_seq":2869,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.010092+0000","last_change":"2023-10-30T12:42:02.470189+0000","last_active":"2023-10-30T12:56:07.010092+0000","last_peered":"2023-10-30T12:56:07.010092+0000","last_clean":"2023-10-30T12:56:07.010092+0000","last_became_active":"2023-10-30T12:35:06.214839+0000","last_became_peered":"2023-10-30T12:35:06.214839+0000","last_unstale":"2023-10-30T12:56:07.010092+0000","last_undegraded":"2023-10-30T12:56:07.010092+0000","last_fullsized":"2023-10-30T12:56:07.010092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:09:56.796250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049147749999999997,"stat_sum":{"num_bytes":12025229,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":91,"num_read_kb":16402,"num_write":654,"num_write_kb":29636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"55'312","reported_seq":377,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.648665+0000","last_change":"2023-10-30T12:35:05.189361+0000","last_active":"2023-10-30T12:56:04.648665+0000","last_peered":"2023-10-30T12:56:04.648665+0000","last_clean":"2023-10-30T12:56:04.648665+0000","last_became_active":"2023-10-30T12:35:05.188862+0000","last_became_peered":"2023-10-30T12:35:05.188862+0000","last_unstale":"2023-10-30T12:56:04.648665+0000","last_undegraded":"2023-10-30T12:56:04.648665+0000","last_fullsized":"2023-10-30T12:56:04.648665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:13.319732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":12343,"num_write":386,"num_write_kb":4750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"55'644","reported_seq":2800,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.941496+0000","last_change":"2023-10-30T12:42:02.472755+0000","last_active":"2023-10-30T12:56:06.941496+0000","last_peered":"2023-10-30T12:56:06.941496+0000","last_clean":"2023-10-30T12:56:06.941496+0000","last_became_active":"2023-10-30T12:35:06.216533+0000","last_became_peered":"2023-10-30T12:35:06.216533+0000","last_unstale":"2023-10-30T12:56:06.941496+0000","last_undegraded":"2023-10-30T12:56:06.941496+0000","last_fullsized":"2023-10-30T12:56:06.941496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:13:01.401936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060962430000000003,"stat_sum":{"num_bytes":10676108,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":65,"num_read_kb":4109,"num_write":617,"num_write_kb":24051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"55'234","reported_seq":295,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:04.569141+0000","last_change":"2023-10-30T12:35:05.177609+0000","last_active":"2023-10-30T12:55:04.569141+0000","last_peered":"2023-10-30T12:55:04.569141+0000","last_clean":"2023-10-30T12:55:04.569141+0000","last_became_active":"2023-10-30T12:35:05.177316+0000","last_became_peered":"2023-10-30T12:35:05.177316+0000","last_unstale":"2023-10-30T12:55:04.569141+0000","last_undegraded":"2023-10-30T12:55:04.569141+0000","last_fullsized":"2023-10-30T12:55:04.569141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:46.820925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":180,"num_write_kb":8325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"55'668","reported_seq":2828,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.003575+0000","last_change":"2023-10-30T12:42:02.241911+0000","last_active":"2023-10-30T12:56:07.003575+0000","last_peered":"2023-10-30T12:56:07.003575+0000","last_clean":"2023-10-30T12:56:07.003575+0000","last_became_active":"2023-10-30T12:35:46.049464+0000","last_became_peered":"2023-10-30T12:35:46.049464+0000","last_unstale":"2023-10-30T12:56:07.003575+0000","last_undegraded":"2023-10-30T12:56:07.003575+0000","last_fullsized":"2023-10-30T12:56:07.003575+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:06:39.487053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023101760000000002,"stat_sum":{"num_bytes":6776001,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":644,"num_write_kb":12631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"55'211","reported_seq":272,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:28.291856+0000","last_change":"2023-10-30T12:35:05.171787+0000","last_active":"2023-10-30T12:55:28.291856+0000","last_peered":"2023-10-30T12:55:28.291856+0000","last_clean":"2023-10-30T12:55:28.291856+0000","last_became_active":"2023-10-30T12:35:05.171491+0000","last_became_peered":"2023-10-30T12:35:05.171491+0000","last_unstale":"2023-10-30T12:55:28.291856+0000","last_undegraded":"2023-10-30T12:55:28.291856+0000","last_fullsized":"2023-10-30T12:55:28.291856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:43:15.331175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":16387,"num_write":154,"num_write_kb":4208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"55'739","reported_seq":2900,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.151055+0000","last_change":"2023-10-30T12:42:02.746323+0000","last_active":"2023-10-30T12:56:07.151055+0000","last_peered":"2023-10-30T12:56:07.151055+0000","last_clean":"2023-10-30T12:56:07.151055+0000","last_became_active":"2023-10-30T12:35:06.213493+0000","last_became_peered":"2023-10-30T12:35:06.213493+0000","last_unstale":"2023-10-30T12:56:07.151055+0000","last_undegraded":"2023-10-30T12:56:07.151055+0000","last_fullsized":"2023-10-30T12:56:07.151055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:38:05.007816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015847370999999999,"stat_sum":{"num_bytes":18167539,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":70,"num_read_kb":4114,"num_write":710,"num_write_kb":33439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"55'305","reported_seq":368,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:04.570822+0000","last_change":"2023-10-30T12:35:05.197811+0000","last_active":"2023-10-30T12:55:04.570822+0000","last_peered":"2023-10-30T12:55:04.570822+0000","last_clean":"2023-10-30T12:55:04.570822+0000","last_became_active":"2023-10-30T12:35:05.197453+0000","last_became_peered":"2023-10-30T12:35:05.197453+0000","last_unstale":"2023-10-30T12:55:04.570822+0000","last_undegraded":"2023-10-30T12:55:04.570822+0000","last_fullsized":"2023-10-30T12:55:04.570822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:16:14.550887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":240,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"55'660","reported_seq":2812,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.010112+0000","last_change":"2023-10-30T12:42:02.472032+0000","last_active":"2023-10-30T12:56:07.010112+0000","last_peered":"2023-10-30T12:56:07.010112+0000","last_clean":"2023-10-30T12:56:07.010112+0000","last_became_active":"2023-10-30T12:35:06.219526+0000","last_became_peered":"2023-10-30T12:35:06.219526+0000","last_unstale":"2023-10-30T12:56:07.010112+0000","last_undegraded":"2023-10-30T12:56:07.010112+0000","last_fullsized":"2023-10-30T12:56:07.010112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:01:04.257663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060180570000000003,"stat_sum":{"num_bytes":14243175,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":61,"num_read_kb":19722,"num_write":630,"num_write_kb":44881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":208,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:54:25.564818+0000","last_change":"2023-10-30T12:34:55.206976+0000","last_active":"2023-10-30T12:54:25.564818+0000","last_peered":"2023-10-30T12:54:25.564818+0000","last_clean":"2023-10-30T12:54:25.564818+0000","last_became_active":"2023-10-30T12:34:55.205923+0000","last_became_peered":"2023-10-30T12:34:55.205923+0000","last_unstale":"2023-10-30T12:54:25.564818+0000","last_undegraded":"2023-10-30T12:54:25.564818+0000","last_fullsized":"2023-10-30T12:54:25.564818+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_clean_scrub_stamp":"2023-10-30T12:34:53.444020+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:25:28.782111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"53'311","reported_seq":380,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:54:54.680160+0000","last_change":"2023-10-30T12:35:05.197974+0000","last_active":"2023-10-30T12:54:54.680160+0000","last_peered":"2023-10-30T12:54:54.680160+0000","last_clean":"2023-10-30T12:54:54.680160+0000","last_became_active":"2023-10-30T12:35:05.197201+0000","last_became_peered":"2023-10-30T12:35:05.197201+0000","last_unstale":"2023-10-30T12:54:54.680160+0000","last_undegraded":"2023-10-30T12:54:54.680160+0000","last_fullsized":"2023-10-30T12:54:54.680160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:39:47.119091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":23,"num_read_kb":16415,"num_write":260,"num_write_kb":8374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"55'616","reported_seq":2759,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.118778+0000","last_change":"2023-10-30T12:42:02.243413+0000","last_active":"2023-10-30T12:56:07.118778+0000","last_peered":"2023-10-30T12:56:07.118778+0000","last_clean":"2023-10-30T12:56:07.118778+0000","last_became_active":"2023-10-30T12:35:06.217687+0000","last_became_peered":"2023-10-30T12:35:06.217687+0000","last_unstale":"2023-10-30T12:56:07.118778+0000","last_undegraded":"2023-10-30T12:56:07.118778+0000","last_fullsized":"2023-10-30T12:56:07.118778+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:47:42.301163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059801699999999999,"stat_sum":{"num_bytes":18764071,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":51,"num_read_kb":8203,"num_write":589,"num_write_kb":32793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"55'386","reported_seq":458,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:31.185445+0000","last_change":"2023-10-30T12:35:05.208339+0000","last_active":"2023-10-30T12:55:31.185445+0000","last_peered":"2023-10-30T12:55:31.185445+0000","last_clean":"2023-10-30T12:55:31.185445+0000","last_became_active":"2023-10-30T12:35:05.207828+0000","last_became_peered":"2023-10-30T12:35:05.207828+0000","last_unstale":"2023-10-30T12:55:31.185445+0000","last_undegraded":"2023-10-30T12:55:31.185445+0000","last_fullsized":"2023-10-30T12:55:31.185445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:00.649048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":45074,"num_write":317,"num_write_kb":20666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"55'680","reported_seq":2830,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.087169+0000","last_change":"2023-10-30T12:42:02.724463+0000","last_active":"2023-10-30T12:56:07.087169+0000","last_peered":"2023-10-30T12:56:07.087169+0000","last_clean":"2023-10-30T12:56:07.087169+0000","last_became_active":"2023-10-30T12:35:06.194791+0000","last_became_peered":"2023-10-30T12:35:06.194791+0000","last_unstale":"2023-10-30T12:56:07.087169+0000","last_undegraded":"2023-10-30T12:56:07.087169+0000","last_fullsized":"2023-10-30T12:56:07.087169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:13:52.749170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043414500000000002,"stat_sum":{"num_bytes":10508371,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":58,"num_read_kb":3682,"num_write":647,"num_write_kb":34521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"55'607","reported_seq":965,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:57.501324+0000","last_change":"2023-10-30T12:35:05.176296+0000","last_active":"2023-10-30T12:55:57.501324+0000","last_peered":"2023-10-30T12:55:57.501324+0000","last_clean":"2023-10-30T12:55:57.501324+0000","last_became_active":"2023-10-30T12:35:05.175959+0000","last_became_peered":"2023-10-30T12:35:05.175959+0000","last_unstale":"2023-10-30T12:55:57.501324+0000","last_undegraded":"2023-10-30T12:55:57.501324+0000","last_fullsized":"2023-10-30T12:55:57.501324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:53.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9398249,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":313,"num_read_kb":20888,"num_write":566,"num_write_kb":9626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"55'674","reported_seq":2834,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.973362+0000","last_change":"2023-10-30T12:42:02.711348+0000","last_active":"2023-10-30T12:56:06.973362+0000","last_peered":"2023-10-30T12:56:06.973362+0000","last_clean":"2023-10-30T12:56:06.973362+0000","last_became_active":"2023-10-30T12:35:06.164711+0000","last_became_peered":"2023-10-30T12:35:06.164711+0000","last_unstale":"2023-10-30T12:56:06.973362+0000","last_undegraded":"2023-10-30T12:56:06.973362+0000","last_fullsized":"2023-10-30T12:56:06.973362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:21:46.319142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002714419,"stat_sum":{"num_bytes":18594076,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":68,"num_read_kb":8213,"num_write":640,"num_write_kb":33748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"55'497","reported_seq":561,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:56.727539+0000","last_change":"2023-10-30T12:35:05.198964+0000","last_active":"2023-10-30T12:55:56.727539+0000","last_peered":"2023-10-30T12:55:56.727539+0000","last_clean":"2023-10-30T12:55:56.727539+0000","last_became_active":"2023-10-30T12:35:05.196873+0000","last_became_peered":"2023-10-30T12:35:05.196873+0000","last_unstale":"2023-10-30T12:55:56.727539+0000","last_undegraded":"2023-10-30T12:55:56.727539+0000","last_fullsized":"2023-10-30T12:55:56.727539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:47.793585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":431,"num_write_kb":16687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"55'673","reported_seq":2837,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.990872+0000","last_change":"2023-10-30T12:42:02.250101+0000","last_active":"2023-10-30T12:56:06.990872+0000","last_peered":"2023-10-30T12:56:06.990872+0000","last_clean":"2023-10-30T12:56:06.990872+0000","last_became_active":"2023-10-30T12:35:06.183225+0000","last_became_peered":"2023-10-30T12:35:06.183225+0000","last_unstale":"2023-10-30T12:56:06.990872+0000","last_undegraded":"2023-10-30T12:56:06.990872+0000","last_fullsized":"2023-10-30T12:56:06.990872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:02:51.357971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032989500000000001,"stat_sum":{"num_bytes":17999573,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":72,"num_read_kb":12,"num_write":642,"num_write_kb":36248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"55'183","reported_seq":239,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:27.599652+0000","last_change":"2023-10-30T12:35:05.177571+0000","last_active":"2023-10-30T12:55:27.599652+0000","last_peered":"2023-10-30T12:55:27.599652+0000","last_clean":"2023-10-30T12:55:27.599652+0000","last_became_active":"2023-10-30T12:35:05.177266+0000","last_became_peered":"2023-10-30T12:35:05.177266+0000","last_unstale":"2023-10-30T12:55:27.599652+0000","last_undegraded":"2023-10-30T12:55:27.599652+0000","last_fullsized":"2023-10-30T12:55:27.599652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:14.601934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":129,"num_write_kb":8259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"55'655","reported_seq":2816,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.161469+0000","last_change":"2023-10-30T12:42:02.745017+0000","last_active":"2023-10-30T12:56:07.161469+0000","last_peered":"2023-10-30T12:56:07.161469+0000","last_clean":"2023-10-30T12:56:07.161469+0000","last_became_active":"2023-10-30T12:35:06.196307+0000","last_became_peered":"2023-10-30T12:35:06.196307+0000","last_unstale":"2023-10-30T12:56:07.161469+0000","last_undegraded":"2023-10-30T12:56:07.161469+0000","last_fullsized":"2023-10-30T12:56:07.161469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:09:56.936899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015197106,"stat_sum":{"num_bytes":5743717,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":69,"num_read_kb":13,"num_write":620,"num_write_kb":24446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"55'489","reported_seq":552,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.635330+0000","last_change":"2023-10-30T12:35:05.207594+0000","last_active":"2023-10-30T12:56:09.635330+0000","last_peered":"2023-10-30T12:56:09.635330+0000","last_clean":"2023-10-30T12:56:09.635330+0000","last_became_active":"2023-10-30T12:35:05.206930+0000","last_became_peered":"2023-10-30T12:35:05.206930+0000","last_unstale":"2023-10-30T12:56:09.635330+0000","last_undegraded":"2023-10-30T12:56:09.635330+0000","last_fullsized":"2023-10-30T12:56:09.635330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:23:46.519077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15572411,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":8,"num_read_kb":9517,"num_write":429,"num_write_kb":15473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"55'662","reported_seq":2826,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.964738+0000","last_change":"2023-10-30T12:42:02.710925+0000","last_active":"2023-10-30T12:56:06.964738+0000","last_peered":"2023-10-30T12:56:06.964738+0000","last_clean":"2023-10-30T12:56:06.964738+0000","last_became_active":"2023-10-30T12:35:06.164733+0000","last_became_peered":"2023-10-30T12:35:06.164733+0000","last_unstale":"2023-10-30T12:56:06.964738+0000","last_undegraded":"2023-10-30T12:56:06.964738+0000","last_fullsized":"2023-10-30T12:56:06.964738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:04:21.927512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024278020000000002,"stat_sum":{"num_bytes":19053289,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":637,"num_write_kb":32762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"55'374","reported_seq":441,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.541877+0000","last_change":"2023-10-30T12:35:05.185034+0000","last_active":"2023-10-30T12:56:04.541877+0000","last_peered":"2023-10-30T12:56:04.541877+0000","last_clean":"2023-10-30T12:56:04.541877+0000","last_became_active":"2023-10-30T12:35:05.184695+0000","last_became_peered":"2023-10-30T12:35:05.184695+0000","last_unstale":"2023-10-30T12:56:04.541877+0000","last_undegraded":"2023-10-30T12:56:04.541877+0000","last_fullsized":"2023-10-30T12:56:04.541877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:01:42.019128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":8489,"num_write":571,"num_write_kb":5858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"55'646","reported_seq":2799,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.108475+0000","last_change":"2023-10-30T12:42:02.242088+0000","last_active":"2023-10-30T12:56:07.108475+0000","last_peered":"2023-10-30T12:56:07.108475+0000","last_clean":"2023-10-30T12:56:07.108475+0000","last_became_active":"2023-10-30T12:35:06.204422+0000","last_became_peered":"2023-10-30T12:35:06.204422+0000","last_unstale":"2023-10-30T12:56:07.108475+0000","last_undegraded":"2023-10-30T12:56:07.108475+0000","last_fullsized":"2023-10-30T12:56:07.108475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:38:09.854923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048535030000000003,"stat_sum":{"num_bytes":22678810,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":62,"num_read_kb":4110,"num_write":624,"num_write_kb":32700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"55'230","reported_seq":292,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:28.060181+0000","last_change":"2023-10-30T12:35:05.200871+0000","last_active":"2023-10-30T12:55:28.060181+0000","last_peered":"2023-10-30T12:55:28.060181+0000","last_clean":"2023-10-30T12:55:28.060181+0000","last_became_active":"2023-10-30T12:35:05.200449+0000","last_became_peered":"2023-10-30T12:35:05.200449+0000","last_unstale":"2023-10-30T12:55:28.060181+0000","last_undegraded":"2023-10-30T12:55:28.060181+0000","last_fullsized":"2023-10-30T12:55:28.060181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:39:20.867587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":4117,"num_write":187,"num_write_kb":4299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"55'643","reported_seq":2803,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.060907+0000","last_change":"2023-10-30T12:42:02.724106+0000","last_active":"2023-10-30T12:56:07.060907+0000","last_peered":"2023-10-30T12:56:07.060907+0000","last_clean":"2023-10-30T12:56:07.060907+0000","last_became_active":"2023-10-30T12:35:06.179412+0000","last_became_peered":"2023-10-30T12:35:06.179412+0000","last_unstale":"2023-10-30T12:56:07.060907+0000","last_undegraded":"2023-10-30T12:56:07.060907+0000","last_fullsized":"2023-10-30T12:56:07.060907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:15:21.046958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003661568,"stat_sum":{"num_bytes":1280771,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":69,"num_read_kb":22,"num_write":604,"num_write_kb":12079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"55'552","reported_seq":617,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:26.056739+0000","last_change":"2023-10-30T12:35:05.207392+0000","last_active":"2023-10-30T12:55:26.056739+0000","last_peered":"2023-10-30T12:55:26.056739+0000","last_clean":"2023-10-30T12:55:26.056739+0000","last_became_active":"2023-10-30T12:35:05.206607+0000","last_became_peered":"2023-10-30T12:35:05.206607+0000","last_unstale":"2023-10-30T12:55:26.056739+0000","last_undegraded":"2023-10-30T12:55:26.056739+0000","last_fullsized":"2023-10-30T12:55:26.056739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:36.758740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":17,"num_read_kb":23179,"num_write":497,"num_write_kb":12662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"55'665","reported_seq":2826,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.023360+0000","last_change":"2023-10-30T12:42:02.251623+0000","last_active":"2023-10-30T12:56:07.023360+0000","last_peered":"2023-10-30T12:56:07.023360+0000","last_clean":"2023-10-30T12:56:07.023360+0000","last_became_active":"2023-10-30T12:35:06.181256+0000","last_became_peered":"2023-10-30T12:35:06.181256+0000","last_unstale":"2023-10-30T12:56:07.023360+0000","last_undegraded":"2023-10-30T12:56:07.023360+0000","last_fullsized":"2023-10-30T12:56:07.023360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:30:13.806672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030387220000000002,"stat_sum":{"num_bytes":10400130,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":70,"num_read_kb":4103,"num_write":631,"num_write_kb":29164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"55'369","reported_seq":432,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:02.291363+0000","last_change":"2023-10-30T12:35:05.201024+0000","last_active":"2023-10-30T12:56:02.291363+0000","last_peered":"2023-10-30T12:56:02.291363+0000","last_clean":"2023-10-30T12:56:02.291363+0000","last_became_active":"2023-10-30T12:35:05.200724+0000","last_became_peered":"2023-10-30T12:35:05.200724+0000","last_unstale":"2023-10-30T12:56:02.291363+0000","last_undegraded":"2023-10-30T12:56:02.291363+0000","last_fullsized":"2023-10-30T12:56:02.291363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:24.746469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388698,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":12300,"num_write":322,"num_write_kb":8548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"55'675","reported_seq":2824,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.153249+0000","last_change":"2023-10-30T12:42:02.241265+0000","last_active":"2023-10-30T12:56:07.153249+0000","last_peered":"2023-10-30T12:56:07.153249+0000","last_clean":"2023-10-30T12:56:07.153249+0000","last_became_active":"2023-10-30T12:35:06.173996+0000","last_became_peered":"2023-10-30T12:35:06.173996+0000","last_unstale":"2023-10-30T12:56:07.153249+0000","last_undegraded":"2023-10-30T12:56:07.153249+0000","last_fullsized":"2023-10-30T12:56:07.153249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:34:09.185233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045562010000000002,"stat_sum":{"num_bytes":5657110,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":58,"num_read_kb":18,"num_write":633,"num_write_kb":16227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"55'873","reported_seq":934,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.676168+0000","last_change":"2023-10-30T12:35:05.164079+0000","last_active":"2023-10-30T12:56:04.676168+0000","last_peered":"2023-10-30T12:56:04.676168+0000","last_clean":"2023-10-30T12:56:04.676168+0000","last_became_active":"2023-10-30T12:35:05.163794+0000","last_became_peered":"2023-10-30T12:35:05.163794+0000","last_unstale":"2023-10-30T12:56:04.676168+0000","last_undegraded":"2023-10-30T12:56:04.676168+0000","last_fullsized":"2023-10-30T12:56:04.676168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:19.795656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":10,"num_read_kb":8454,"num_write":952,"num_write_kb":13780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"55'645","reported_seq":2805,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.134005+0000","last_change":"2023-10-30T12:42:02.242567+0000","last_active":"2023-10-30T12:56:07.134005+0000","last_peered":"2023-10-30T12:56:07.134005+0000","last_clean":"2023-10-30T12:56:07.134005+0000","last_became_active":"2023-10-30T12:35:06.183226+0000","last_became_peered":"2023-10-30T12:35:06.183226+0000","last_unstale":"2023-10-30T12:56:07.134005+0000","last_undegraded":"2023-10-30T12:56:07.134005+0000","last_fullsized":"2023-10-30T12:56:07.134005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:30:48.627756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052521379999999999,"stat_sum":{"num_bytes":5772919,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":69,"num_read_kb":4112,"num_write":611,"num_write_kb":16444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"55'336","reported_seq":402,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:36.391405+0000","last_change":"2023-10-30T12:35:05.206995+0000","last_active":"2023-10-30T12:55:36.391405+0000","last_peered":"2023-10-30T12:55:36.391405+0000","last_clean":"2023-10-30T12:55:36.391405+0000","last_became_active":"2023-10-30T12:35:05.206518+0000","last_became_peered":"2023-10-30T12:35:05.206518+0000","last_unstale":"2023-10-30T12:55:36.391405+0000","last_undegraded":"2023-10-30T12:55:36.391405+0000","last_fullsized":"2023-10-30T12:55:36.391405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:41:33.703261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":268,"num_write_kb":12461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"55'637","reported_seq":2796,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:06.957805+0000","last_change":"2023-10-30T12:42:02.737782+0000","last_active":"2023-10-30T12:56:06.957805+0000","last_peered":"2023-10-30T12:56:06.957805+0000","last_clean":"2023-10-30T12:56:06.957805+0000","last_became_active":"2023-10-30T12:35:06.197861+0000","last_became_peered":"2023-10-30T12:35:06.197861+0000","last_unstale":"2023-10-30T12:56:06.957805+0000","last_undegraded":"2023-10-30T12:56:06.957805+0000","last_fullsized":"2023-10-30T12:56:06.957805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:38:17.960141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011118161,"stat_sum":{"num_bytes":14676487,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":68,"num_read_kb":15,"num_write":602,"num_write_kb":24516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"55'655","reported_seq":2826,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.074252+0000","last_change":"2023-10-30T12:42:02.744356+0000","last_active":"2023-10-30T12:56:07.074252+0000","last_peered":"2023-10-30T12:56:07.074252+0000","last_clean":"2023-10-30T12:56:07.074252+0000","last_became_active":"2023-10-30T12:35:06.212548+0000","last_became_peered":"2023-10-30T12:35:06.212548+0000","last_unstale":"2023-10-30T12:56:07.074252+0000","last_undegraded":"2023-10-30T12:56:07.074252+0000","last_fullsized":"2023-10-30T12:56:07.074252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:58:19.323075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015114931999999999,"stat_sum":{"num_bytes":19302518,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":81,"num_read_kb":7349,"num_write":635,"num_write_kb":32674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"55'363","reported_seq":423,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:07.528197+0000","last_change":"2023-10-30T12:35:05.206980+0000","last_active":"2023-10-30T12:55:07.528197+0000","last_peered":"2023-10-30T12:55:07.528197+0000","last_clean":"2023-10-30T12:55:07.528197+0000","last_became_active":"2023-10-30T12:35:05.206559+0000","last_became_peered":"2023-10-30T12:35:05.206559+0000","last_unstale":"2023-10-30T12:55:07.528197+0000","last_undegraded":"2023-10-30T12:55:07.528197+0000","last_fullsized":"2023-10-30T12:55:07.528197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:40.259622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":312,"num_write_kb":8389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"55'654","reported_seq":2814,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.072742+0000","last_change":"2023-10-30T12:42:02.744472+0000","last_active":"2023-10-30T12:56:07.072742+0000","last_peered":"2023-10-30T12:56:07.072742+0000","last_clean":"2023-10-30T12:56:07.072742+0000","last_became_active":"2023-10-30T12:35:06.212505+0000","last_became_peered":"2023-10-30T12:35:06.212505+0000","last_unstale":"2023-10-30T12:56:07.072742+0000","last_undegraded":"2023-10-30T12:56:07.072742+0000","last_fullsized":"2023-10-30T12:56:07.072742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:28:56.182227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014553393,"stat_sum":{"num_bytes":9479151,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":70,"num_read_kb":12304,"num_write":628,"num_write_kb":36199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"55'377","reported_seq":448,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:59.997488+0000","last_change":"2023-10-30T12:35:05.207851+0000","last_active":"2023-10-30T12:55:59.997488+0000","last_peered":"2023-10-30T12:55:59.997488+0000","last_clean":"2023-10-30T12:55:59.997488+0000","last_became_active":"2023-10-30T12:35:05.206879+0000","last_became_peered":"2023-10-30T12:35:05.206879+0000","last_unstale":"2023-10-30T12:55:59.997488+0000","last_undegraded":"2023-10-30T12:55:59.997488+0000","last_fullsized":"2023-10-30T12:55:59.997488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:43:15.717507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":32,"num_read_kb":45093,"num_write":322,"num_write_kb":20730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"55'623","reported_seq":2785,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.101531+0000","last_change":"2023-10-30T12:42:02.723663+0000","last_active":"2023-10-30T12:56:07.101531+0000","last_peered":"2023-10-30T12:56:07.101531+0000","last_clean":"2023-10-30T12:56:07.101531+0000","last_became_active":"2023-10-30T12:35:46.275077+0000","last_became_peered":"2023-10-30T12:35:46.275077+0000","last_unstale":"2023-10-30T12:56:07.101531+0000","last_undegraded":"2023-10-30T12:56:07.101531+0000","last_fullsized":"2023-10-30T12:56:07.101531+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:31:10.982688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032416139999999999,"stat_sum":{"num_bytes":6183106,"num_objects":128,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":0,"num_read":63,"num_read_kb":4113,"num_write":600,"num_write_kb":9129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"55'508","reported_seq":569,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:35.795056+0000","last_change":"2023-10-30T12:35:05.174319+0000","last_active":"2023-10-30T12:55:35.795056+0000","last_peered":"2023-10-30T12:55:35.795056+0000","last_clean":"2023-10-30T12:55:35.795056+0000","last_became_active":"2023-10-30T12:35:05.173984+0000","last_became_peered":"2023-10-30T12:35:05.173984+0000","last_unstale":"2023-10-30T12:55:35.795056+0000","last_undegraded":"2023-10-30T12:55:35.795056+0000","last_fullsized":"2023-10-30T12:55:35.795056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:11.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":430,"num_write_kb":12596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"55'666","reported_seq":2819,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:07.075370+0000","last_change":"2023-10-30T12:42:02.243078+0000","last_active":"2023-10-30T12:56:07.075370+0000","last_peered":"2023-10-30T12:56:07.075370+0000","last_clean":"2023-10-30T12:56:07.075370+0000","last_became_active":"2023-10-30T12:35:06.198504+0000","last_became_peered":"2023-10-30T12:35:06.198504+0000","last_unstale":"2023-10-30T12:56:07.075370+0000","last_undegraded":"2023-10-30T12:56:07.075370+0000","last_fullsized":"2023-10-30T12:56:07.075370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:44:32.447527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055292249999999996,"stat_sum":{"num_bytes":14373178,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":61,"num_read_kb":1995,"num_write":639,"num_write_kb":19071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"55'208","reported_seq":290,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:28.291837+0000","last_change":"2023-10-30T12:35:05.200926+0000","last_active":"2023-10-30T12:55:28.291837+0000","last_peered":"2023-10-30T12:55:28.291837+0000","last_clean":"2023-10-30T12:55:28.291837+0000","last_became_active":"2023-10-30T12:35:05.200623+0000","last_became_peered":"2023-10-30T12:35:05.200623+0000","last_unstale":"2023-10-30T12:55:28.291837+0000","last_undegraded":"2023-10-30T12:55:28.291837+0000","last_fullsized":"2023-10-30T12:55:28.291837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:38.942922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":63,"num_read_kb":24633,"num_write":147,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":916500589,"num_objects":9593,"num_object_clones":0,"num_object_copies":19186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9593,"num_whiteouts":0,"num_read":4339,"num_read_kb":400205,"num_write":40772,"num_write_kb":1941314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1881726976,"data_stored":1831461082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42742,"ondisk_log_size":42742,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":354417535,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1232,"num_read_kb":640401,"num_write":11493,"num_write_kb":356840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":709001216,"data_stored":708835070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12852,"ondisk_log_size":12852,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542405,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487968,"kb_used_data":398488,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":93883872,"statfs":{"total":96636764160,"available":96137084928,"internally_reserved":0,"allocated":408051712,"data_stored":400479873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":91617741},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.48599999999999999,"5min":0.438,"15min":0.42899999999999999},"max":{"1min":1.2330000000000001,"5min":1.371,"15min":1.371},"last":1.121},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.82099999999999995,"15min":0.82399999999999995},"min":{"1min":0.54200000000000004,"5min":0.437,"15min":0.41699999999999998},"max":{"1min":1.137,"5min":1.45,"15min":1.45},"last":0.91300000000000003}]},{"osd":1,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82699999999999996,"15min":0.85299999999999998},"min":{"1min":0.52700000000000002,"5min":0.46700000000000003,"15min":0.44600000000000001},"max":{"1min":1.222,"5min":1.222,"15min":1.496},"last":1.095},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.86599999999999999,"15min":0.89600000000000002},"min":{"1min":0.55700000000000005,"5min":0.53500000000000003,"15min":0.51800000000000002},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.98999999999999999}]},{"osd":2,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.873,"15min":0.82799999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.1659999999999999,"5min":1.242,"15min":1.9059999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.84199999999999997,"15min":0.89700000000000002},"min":{"1min":0.60299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.1830000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":0.92600000000000005,"15min":0.91100000000000003},"min":{"1min":0.76300000000000001,"5min":0.54200000000000004,"15min":0.54000000000000004},"max":{"1min":1.474,"5min":1.607,"15min":1.9810000000000001},"last":1.278},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.004,"15min":1},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.1559999999999999}]},{"osd":4,"last update":"Mon Oct 30 12:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90100000000000002,"15min":0.92800000000000005},"min":{"1min":0.71599999999999997,"5min":0.52500000000000002,"15min":0.46899999999999997},"max":{"1min":1.329,"5min":1.4059999999999999,"15min":2.7309999999999999},"last":1.05},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.016,"15min":0.99399999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.41999999999999998},"max":{"1min":1.3600000000000001,"5min":1.694,"15min":1.8859999999999999},"last":0.95299999999999996}]},{"osd":5,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.048,"15min":1.0700000000000001},"min":{"1min":0.72099999999999997,"5min":0.59399999999999997,"15min":0.49099999999999999},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":2.2120000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.218,"15min":1.143},"min":{"1min":1.0089999999999999,"5min":0.72599999999999998,"15min":0.69899999999999995},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.2310000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0229999999999999,"15min":1.087},"min":{"1min":0.79400000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.456,"5min":1.478,"15min":1.802},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.1359999999999999,"15min":1.1930000000000001},"min":{"1min":0.92800000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.4870000000000001,"5min":1.6639999999999999,"15min":1.881},"last":1.014}]}]},{"osd":6,"up_from":14,"seq":60129542404,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":378044,"kb_used_data":292788,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":93993796,"statfs":{"total":96636764160,"available":96249647104,"internally_reserved":0,"allocated":299814912,"data_stored":294791815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":87292361},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.83199999999999996,"15min":0.82399999999999995},"min":{"1min":0.496,"5min":0.45600000000000002,"15min":0.38600000000000001},"max":{"1min":1.056,"5min":5.8760000000000003,"15min":5.8760000000000003},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.60899999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.161,"5min":1.508,"15min":1.673},"last":0.85499999999999998}]},{"osd":1,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.85699999999999998,"15min":0.92100000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.113,"5min":2.2679999999999998,"15min":5.0949999999999998},"last":1.012},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.91100000000000003,"15min":0.92700000000000005},"min":{"1min":0.65000000000000002,"5min":0.54800000000000004,"15min":0.45500000000000002},"max":{"1min":1.282,"5min":1.423,"15min":5.0519999999999996},"last":1.0840000000000001}]},{"osd":2,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.86499999999999999,"15min":0.84699999999999998},"min":{"1min":0.621,"5min":0.52900000000000003,"15min":0.51800000000000002},"max":{"1min":1.403,"5min":2.649,"15min":2.649},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.069,"5min":0.97099999999999997,"15min":0.92500000000000004},"min":{"1min":0.74099999999999999,"5min":0.56899999999999995,"15min":0.44700000000000001},"max":{"1min":1.528,"5min":2.282,"15min":2.282},"last":1.143}]},{"osd":3,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.88500000000000001,"15min":0.94599999999999995},"min":{"1min":0.62,"5min":0.51700000000000002,"15min":0.40200000000000002},"max":{"1min":1.1120000000000001,"5min":1.929,"15min":1.929},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.97499999999999998,"15min":1},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.42699999999999999},"max":{"1min":1.246,"5min":1.5640000000000001,"15min":1.7589999999999999},"last":0.64300000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.03,"15min":1.0009999999999999},"min":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.27600000000000002},"max":{"1min":1.617,"5min":1.619,"15min":1.9219999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.05,"15min":1.0329999999999999},"min":{"1min":0.65600000000000003,"5min":0.64200000000000002,"15min":0.32400000000000001},"max":{"1min":1.3160000000000001,"5min":1.5700000000000001,"15min":1.625},"last":0.97899999999999998}]},{"osd":5,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":1.0109999999999999,"15min":1.0349999999999999},"min":{"1min":0.66600000000000004,"5min":0.66300000000000003,"15min":0.29599999999999999},"max":{"1min":1.194,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.1040000000000001,"15min":1.133},"min":{"1min":0.78500000000000003,"5min":0.65300000000000002,"15min":0.34399999999999997},"max":{"1min":1.3089999999999999,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":0.94499999999999995}]},{"osd":7,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.179,"15min":1.149},"min":{"1min":0.82999999999999996,"5min":0.79300000000000004,"15min":0.37},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":2.0350000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2030000000000001,"15min":1.1950000000000001},"min":{"1min":0.90200000000000002,"5min":0.82099999999999995,"15min":0.46800000000000003},"max":{"1min":1.754,"5min":2.274,"15min":2.274},"last":1.2410000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410352,"kb_used_data":316712,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":93961488,"statfs":{"total":96636764160,"available":96216563712,"internally_reserved":0,"allocated":324313088,"data_stored":316662693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79500000000000004,"15min":0.80900000000000005},"min":{"1min":0.53200000000000003,"5min":0.47199999999999998,"15min":0.437},"max":{"1min":1.1819999999999999,"5min":1.3700000000000001,"15min":1.5649999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.83299999999999996,"15min":0.84099999999999997},"min":{"1min":0.58899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2390000000000001,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.76000000000000001}]},{"osd":1,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.53300000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.034,"5min":1.3069999999999999,"15min":2.4910000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.86299999999999999,"15min":0.84999999999999998},"min":{"1min":0.63,"5min":0.433,"15min":0.433},"max":{"1min":1.2030000000000001,"5min":1.54,"15min":1.9730000000000001},"last":0.64000000000000001}]},{"osd":2,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.67300000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1950000000000001,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":1.0600000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.55600000000000005},"max":{"1min":1.2170000000000001,"5min":4.6879999999999997,"15min":4.6879999999999997},"last":0.89100000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.92500000000000004,"15min":0.94399999999999995},"min":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.284,"5min":2.266,"15min":2.5819999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.94999999999999996,"15min":0.90700000000000003},"min":{"1min":0.78200000000000003,"5min":0.54300000000000004,"15min":0.53500000000000003},"max":{"1min":1.411,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.94799999999999995}]},{"osd":5,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.97099999999999997,"15min":1.0029999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.129,"5min":2.1779999999999999,"15min":2.3180000000000001},"last":1.075},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.0880000000000001,"15min":1.0620000000000001},"min":{"1min":1.0660000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3959999999999999,"5min":3.173,"15min":3.173},"last":1.2150000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":1.0509999999999999,"15min":1.0880000000000001},"min":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.59299999999999997},"max":{"1min":1.131,"5min":2.2709999999999999,"15min":2.73},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.093,"5min":1.1080000000000001,"15min":1.0640000000000001},"min":{"1min":0.83999999999999997,"5min":0.748,"15min":0.56499999999999995},"max":{"1min":1.262,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":1.123}]},{"osd":7,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.087,"15min":1.0700000000000001},"min":{"1min":0.70699999999999996,"5min":0.68899999999999995,"15min":0.64900000000000002},"max":{"1min":1.196,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.02},{"interface":"front","average":{"1min":1.204,"5min":1.175,"15min":1.131},"min":{"1min":0.92500000000000004,"5min":0.70299999999999996,"15min":0.65300000000000002},"max":{"1min":1.4950000000000001,"5min":2.375,"15min":3.0649999999999999},"last":1.1699999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542404,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":296824,"kb_used_data":239280,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94075016,"statfs":{"total":96636764160,"available":96332816384,"internally_reserved":0,"allocated":245022720,"data_stored":240236236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.48599999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.22700000000000001},"max":{"1min":1.6930000000000001,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.58199999999999996,"15min":0.53200000000000003},"min":{"1min":0.40000000000000002,"5min":0.33200000000000002,"15min":0.28699999999999998},"max":{"1min":2.0619999999999998,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":0.47399999999999998}]},{"osd":1,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.33000000000000002,"5min":0.317,"15min":0.27900000000000003},"max":{"1min":3.02,"5min":3.5059999999999998,"15min":4.5019999999999998},"last":0.497},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.39100000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.0299999999999998,"5min":3.5430000000000001,"15min":3.5430000000000001},"last":0.38200000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.76000000000000001,"15min":0.60399999999999998},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26100000000000001},"max":{"1min":8.9550000000000001,"5min":18.747,"15min":18.747},"last":0.36599999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":0.85499999999999998,"15min":0.70099999999999996},"min":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38300000000000001},"max":{"1min":8.9900000000000002,"5min":18.863,"15min":18.863},"last":0.45400000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85999999999999999},"min":{"1min":0.626,"5min":0.56999999999999995,"15min":0.53600000000000003},"max":{"1min":3.0760000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.321,"5min":0.96299999999999997,"15min":0.89700000000000002},"min":{"1min":0.73299999999999998,"5min":0.51100000000000001,"15min":0.48299999999999998},"max":{"1min":9.6289999999999996,"5min":9.6289999999999996,"15min":9.6289999999999996},"last":0.78200000000000003}]},{"osd":5,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":0.876,"15min":0.85199999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":9.7289999999999992,"5min":9.7289999999999992,"15min":9.7289999999999992},"last":0.82599999999999996},{"interface":"front","average":{"1min":5.577,"5min":1.9490000000000001,"15min":1.2},"min":{"1min":0.626,"5min":0.60399999999999998,"15min":0.51900000000000002},"max":{"1min":91.072999999999993,"5min":91.072999999999993,"15min":91.072999999999993},"last":0.64800000000000002}]},{"osd":6,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.048,"15min":0.91700000000000004},"min":{"1min":0.59299999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":5.4640000000000004,"5min":19.126000000000001,"15min":19.126000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.2509999999999999,"15min":1.006},"min":{"1min":0.72499999999999998,"5min":0.60299999999999998,"15min":0.59399999999999997},"max":{"1min":21.469999999999999,"5min":21.469999999999999,"15min":21.469999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":0.94699999999999995,"15min":0.89600000000000002},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.47099999999999997},"max":{"1min":9.6869999999999994,"5min":9.6869999999999994,"15min":9.6869999999999994},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.1479999999999999,"15min":0.96799999999999997},"min":{"1min":0.80800000000000005,"5min":0.69799999999999995,"15min":0.53200000000000003},"max":{"1min":5.524,"5min":19.050999999999998,"15min":19.050999999999998},"last":0.90900000000000003}]}]},{"osd":0,"up_from":14,"seq":60129542405,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427176,"kb_used_data":337696,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":93944664,"statfs":{"total":96636764160,"available":96199335936,"internally_reserved":0,"allocated":345800704,"data_stored":338053520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":91617736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.46600000000000003,"15min":0.48399999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.248},"max":{"1min":0.69099999999999995,"5min":0.78700000000000003,"15min":8.8059999999999992},"last":0.376},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.504,"15min":0.61099999999999999},"min":{"1min":0.30099999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69299999999999995,"5min":0.83899999999999997,"15min":35.084000000000003},"last":0.55900000000000005}]},{"osd":2,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51500000000000001,"15min":0.63500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.28999999999999998},"max":{"1min":0.68000000000000005,"5min":0.93100000000000005,"15min":34.741},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58099999999999996,"15min":0.69299999999999995},"min":{"1min":0.42999999999999999,"5min":0.37,"15min":0.35299999999999998},"max":{"1min":0.72999999999999998,"5min":1.0049999999999999,"15min":35.058999999999997},"last":0.47099999999999997}]},{"osd":3,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.35899999999999999,"5min":0.32000000000000001,"15min":0.29499999999999998},"max":{"1min":0.76000000000000001,"5min":0.877,"15min":8.8469999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.629},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.88800000000000001,"15min":11.916},"last":0.52900000000000003}]},{"osd":4,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.85499999999999998,"15min":0.79700000000000004},"min":{"1min":0.66600000000000004,"5min":0.55500000000000005,"15min":0.373},"max":{"1min":1.0389999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.82499999999999996,"15min":0.92800000000000005},"min":{"1min":0.63900000000000001,"5min":0.57099999999999995,"15min":0.39900000000000002},"max":{"1min":1.1419999999999999,"5min":1.175,"15min":34.902999999999999},"last":0.76300000000000001}]},{"osd":5,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82099999999999995,"15min":0.84799999999999998},"min":{"1min":0.59499999999999997,"5min":0.51800000000000002,"15min":0.5},"max":{"1min":1.0820000000000001,"5min":1.171,"15min":8.4900000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.90600000000000003,"15min":0.97499999999999998},"min":{"1min":0.78500000000000003,"5min":0.66900000000000004,"15min":0.48799999999999999},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":34.698},"last":0.81200000000000006}]},{"osd":6,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81000000000000005},"min":{"1min":0.64200000000000002,"5min":0.58099999999999996,"15min":0.47799999999999998},"max":{"1min":1.097,"5min":1.1439999999999999,"15min":1.2789999999999999},"last":0.75},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.878,"15min":0.88700000000000001},"min":{"1min":0.64100000000000001,"5min":0.629,"15min":0.50900000000000001},"max":{"1min":1.304,"5min":1.304,"15min":8.8800000000000008},"last":0.70599999999999996}]},{"osd":7,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82499999999999996,"15min":0.84099999999999997},"min":{"1min":0.65000000000000002,"5min":0.53000000000000003,"15min":0.437},"max":{"1min":1.042,"5min":1.1930000000000001,"15min":1.304},"last":0.875},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.88900000000000001,"15min":0.92900000000000005},"min":{"1min":0.70699999999999996,"5min":0.60599999999999998,"15min":0.55900000000000005},"max":{"1min":1.1479999999999999,"5min":1.323,"15min":8.5549999999999997},"last":0.79200000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542403,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462712,"kb_used_data":373232,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":93909128,"statfs":{"total":96636764160,"available":96162947072,"internally_reserved":0,"allocated":382189568,"data_stored":374550682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":91617737},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.88300000000000001,"15min":0.84399999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.41899999999999998},"max":{"1min":1.3129999999999999,"5min":1.627,"15min":1.627},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.87,"5min":0.88900000000000001,"15min":0.84999999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.495},"max":{"1min":1.2649999999999999,"5min":1.468,"15min":1.468},"last":0.79400000000000004}]},{"osd":1,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.82899999999999996,"15min":0.91300000000000003},"min":{"1min":0.52800000000000002,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.3089999999999999,"5min":1.363,"15min":1.746},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.82399999999999995,"15min":0.88},"min":{"1min":0.52700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2969999999999999,"5min":1.3169999999999999,"15min":1.337},"last":0.92300000000000004}]},{"osd":2,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89700000000000002,"15min":0.872},"min":{"1min":0.60299999999999998,"5min":0.53800000000000003,"15min":0.501},"max":{"1min":1.4139999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.21},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.86199999999999999,"15min":0.83899999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.442},"max":{"1min":1.1699999999999999,"5min":1.29,"15min":1.4770000000000001},"last":1.0369999999999999}]},{"osd":3,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.9850000000000001,"15min":1.3180000000000001},"min":{"1min":0.63900000000000001,"5min":0.54400000000000004,"15min":0.504},"max":{"1min":1.3640000000000001,"5min":104.88500000000001,"15min":104.88500000000001},"last":1.252},{"interface":"front","average":{"1min":1.034,"5min":2.0619999999999998,"15min":1.3700000000000001},"min":{"1min":0.71399999999999997,"5min":0.624,"15min":0.60699999999999998},"max":{"1min":1.405,"5min":105.042,"15min":105.042},"last":0.96099999999999997}]},{"osd":4,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0349999999999999,"15min":0.94099999999999995},"min":{"1min":0.69899999999999995,"5min":0.58699999999999997,"15min":0.38500000000000001},"max":{"1min":1.262,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.88},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.077,"15min":1.1220000000000001},"min":{"1min":0.68100000000000005,"5min":0.59799999999999998,"15min":0.57399999999999995},"max":{"1min":1.5229999999999999,"5min":1.542,"15min":1.712},"last":0.59699999999999998}]},{"osd":6,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.96199999999999997,"15min":0.95599999999999996},"min":{"1min":0.77800000000000002,"5min":0.627,"15min":0.35899999999999999},"max":{"1min":1.389,"5min":1.3899999999999999,"15min":1.9410000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.0649999999999999,"15min":1.056},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.48899999999999999},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":2.0299999999999998},"last":0.83799999999999997}]},{"osd":7,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.1080000000000001,"15min":1.095},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.40600000000000003},"max":{"1min":1.357,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.113,"15min":1.1120000000000001},"min":{"1min":0.88,"5min":0.70499999999999996,"15min":0.53300000000000003},"max":{"1min":1.5149999999999999,"5min":1.5660000000000001,"15min":1.7689999999999999},"last":0.99299999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542405,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442132,"kb_used_data":348492,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":93929708,"statfs":{"total":96636764160,"available":96184020992,"internally_reserved":0,"allocated":356855808,"data_stored":348975142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":95877578},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:29 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42999999999999999,"15min":0.436},"min":{"1min":0.26600000000000001,"5min":0.25600000000000001,"15min":0.224},"max":{"1min":0.68100000000000005,"5min":0.77100000000000002,"15min":2.1459999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.46700000000000003,"15min":0.54800000000000004},"min":{"1min":0.32100000000000001,"5min":0.28399999999999997,"15min":0.27200000000000002},"max":{"1min":0.59999999999999998,"5min":0.77900000000000003,"15min":21.495999999999999},"last":0.48199999999999998}]},{"osd":1,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.52100000000000002,"15min":0.53200000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72999999999999998,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48999999999999999,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.314,"15min":0.30399999999999999},"max":{"1min":0.68500000000000005,"5min":1.2450000000000001,"15min":15.102},"last":0.53000000000000003}]},{"osd":2,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53900000000000003,"15min":0.60899999999999999},"min":{"1min":0.33300000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.75800000000000001,"5min":1.044,"15min":21.321000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.495,"5min":0.39000000000000001,"15min":0.379},"max":{"1min":0.77000000000000002,"5min":0.97099999999999997,"15min":1.681},"last":0.45700000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.80600000000000005,"15min":0.90000000000000002},"min":{"1min":0.57999999999999996,"5min":0.49199999999999999,"15min":0.48499999999999999},"max":{"1min":1.21,"5min":1.21,"15min":21.382999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.879,"15min":0.91900000000000004},"min":{"1min":0.73799999999999999,"5min":0.60399999999999998,"15min":0.46700000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":15.231999999999999},"last":0.71299999999999997}]},{"osd":5,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.84399999999999997,"15min":0.89500000000000002},"min":{"1min":0.58699999999999997,"5min":0.54200000000000004,"15min":0.47799999999999998},"max":{"1min":1.0600000000000001,"5min":1.1879999999999999,"15min":15.273999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.90500000000000003,"15min":0.95399999999999996},"min":{"1min":0.69499999999999995,"5min":0.67700000000000005,"15min":0.57499999999999996},"max":{"1min":1.1619999999999999,"5min":1.2170000000000001,"15min":15.359999999999999},"last":0.84399999999999997}]},{"osd":6,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79400000000000004,"15min":0.81699999999999995},"min":{"1min":0.61099999999999999,"5min":0.53200000000000003,"15min":0.51100000000000001},"max":{"1min":1.099,"5min":1.296,"15min":1.5620000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81599999999999995,"15min":0.89100000000000001},"min":{"1min":0.66000000000000003,"5min":0.55700000000000005,"15min":0.498},"max":{"1min":1.04,"5min":1.2210000000000001,"15min":15.313000000000001},"last":0.88400000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.85199999999999998,"15min":0.92700000000000005},"min":{"1min":0.65000000000000002,"5min":0.58799999999999997,"15min":0.55200000000000005},"max":{"1min":1.0620000000000001,"5min":1.137,"15min":21.428999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.85299999999999998,"15min":0.879},"min":{"1min":0.81699999999999995,"5min":0.56899999999999995,"15min":0.56799999999999995},"max":{"1min":1.1180000000000001,"5min":1.28,"15min":1.3100000000000001},"last":0.80900000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542403,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":296616,"kb_used_data":229408,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94075224,"statfs":{"total":96636764160,"available":96333029376,"internally_reserved":0,"allocated":234913792,"data_stored":230197311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.68500000000000005,"15min":0.54400000000000004},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.62,"5min":15.343999999999999,"15min":15.343999999999999},"last":0.434},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.73299999999999998,"15min":0.59999999999999998},"min":{"1min":0.34100000000000003,"5min":0.313,"15min":0.29899999999999999},"max":{"1min":0.66800000000000004,"5min":14.68,"15min":14.68},"last":0.45200000000000001}]},{"osd":2,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.66700000000000004,"15min":0.59799999999999998},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68999999999999995,"5min":8.1359999999999992,"15min":8.1359999999999992},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.33900000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69899999999999995,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.65100000000000002,"15min":0.59599999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.81200000000000006,"5min":8.0920000000000005,"15min":8.0920000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59399999999999997,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.35199999999999998,"15min":0.32000000000000001},"max":{"1min":0.79500000000000004,"5min":5.9199999999999999,"15min":5.9199999999999999},"last":0.504}]},{"osd":4,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":1.133,"15min":0.94099999999999995},"min":{"1min":0.629,"5min":0.60499999999999998,"15min":0.41799999999999998},"max":{"1min":1.2010000000000001,"5min":21.27,"15min":21.27},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.93899999999999995,"15min":0.90700000000000003},"min":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.53600000000000003},"max":{"1min":1.3420000000000001,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.82199999999999995}]},{"osd":5,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.83099999999999996,"15min":0.82599999999999996},"min":{"1min":0.63900000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0089999999999999,"5min":1.355,"15min":1.355},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.872,"15min":0.85599999999999998},"min":{"1min":0.68300000000000005,"5min":0.499,"15min":0.46200000000000002},"max":{"1min":1.1599999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.66300000000000003}]},{"osd":6,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85599999999999998,"15min":0.83999999999999997},"min":{"1min":0.70999999999999996,"5min":0.61399999999999999,"15min":0.51300000000000001},"max":{"1min":1.111,"5min":1.29,"15min":1.4450000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.88400000000000001,"15min":0.88500000000000001},"min":{"1min":0.64600000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.246,"5min":1.323,"15min":1.468},"last":0.75600000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.871,"15min":0.85299999999999998},"min":{"1min":0.67800000000000005,"5min":0.629,"15min":0.496},"max":{"1min":1.304,"5min":1.304,"15min":1.4299999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.86399999999999999,"15min":0.874},"min":{"1min":0.76200000000000001,"5min":0.63100000000000001,"15min":0.443},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2410000000000001},"last":0.88200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":74309632,"data_stored":74293305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96493568,"data_stored":96469936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42975232,"data_stored":42953409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":85110784,"data_stored":85086454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":88555520,"data_stored":88540092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":85549056,"data_stored":85526018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":107171840,"data_stored":107146531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":128835584,"data_stored":128819325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":270266368,"data_stored":262861049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":137789440,"data_stored":133418577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":201416704,"data_stored":196974029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":271114240,"data_stored":263579890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":235126784,"data_stored":227813803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":296009728,"data_stored":288715866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":192012288,"data_stored":187336486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":277991424,"data_stored":270761382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:56:15.944 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:56:16.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.123+0000 7fe9b81ab640 1 Processor -- start 2023-10-30T12:56:16.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.123+0000 7fe9b81ab640 1 -- start start 2023-10-30T12:56:16.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.123+0000 7fe9b81ab640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9b00c05d0 0x7fe9b013ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:16.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.123+0000 7fe9b81ab640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b00c0a50 con 0x7fe9b00c6f60 2023-10-30T12:56:16.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.123+0000 7fe9b81ab640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b006f6e0 con 0x7fe9b00c6b80 2023-10-30T12:56:16.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.123+0000 7fe9b81ab640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b006f9c0 con 0x7fe9b00c05d0 2023-10-30T12:56:16.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.123+0000 7fe9b6748640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe9b00c6f60 0x7fe9b01566a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44390/0 (socket says 172.21.15.12:44390) 2023-10-30T12:56:16.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.123+0000 7fe9b6748640 1 -- 172.21.15.12:0/1979752591 learned_addr learned my addr 172.21.15.12:0/1979752591 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:16.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b5746640 1 --2- 172.21.15.12:0/1979752591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9b00c05d0 0x7fe9b013ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:16.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b4f45640 1 -- 172.21.15.12:0/1979752591 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228351669 0 0) 0x7fe9b00c0a50 con 0x7fe9b00c6f60 2023-10-30T12:56:16.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b4f45640 1 -- 172.21.15.12:0/1979752591 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe99c003530 con 0x7fe9b00c6f60 2023-10-30T12:56:16.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b4f45640 1 -- 172.21.15.12:0/1979752591 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2508108899 0 0) 0x7fe99c003530 con 0x7fe9b00c6f60 2023-10-30T12:56:16.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b4f45640 1 -- 172.21.15.12:0/1979752591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9b00c05d0 msgr2=0x7fe9b013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:16.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b4f45640 1 --2- 172.21.15.12:0/1979752591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9b00c05d0 0x7fe9b013ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:16.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b4f45640 1 -- 172.21.15.12:0/1979752591 >> v1:172.21.15.154:6789/0 conn(0x7fe9b00c6b80 legacy=0x7fe9b00bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:16.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b4f45640 1 -- 172.21.15.12:0/1979752591 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b006fca0 con 0x7fe9b00c6f60 2023-10-30T12:56:16.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b4f45640 1 -- 172.21.15.12:0/1979752591 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fe9a8001a00 con 0x7fe9b00c6f60 2023-10-30T12:56:16.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b4f45640 1 -- 172.21.15.12:0/1979752591 >> v1:172.21.15.12:6789/0 conn(0x7fe9b00c6f60 legacy=0x7fe9b01566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:16.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b4f45640 1 --2- 172.21.15.12:0/1979752591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe99c0039b0 0x7fe99c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:16.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b4f45640 1 --2- 172.21.15.12:0/1979752591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe99c004360 0x7fe99c004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:16.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b4f45640 1 --2- 172.21.15.12:0/1979752591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe99c004fe0 0x7fe99c00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:16.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b4f45640 1 -- 172.21.15.12:0/1979752591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b006f9c0 con 0x7fe99c0039b0 2023-10-30T12:56:16.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b4f45640 1 -- 172.21.15.12:0/1979752591 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c00e110 con 0x7fe99c004fe0 2023-10-30T12:56:16.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b4f45640 1 -- 172.21.15.12:0/1979752591 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c00e2d0 con 0x7fe99c004360 2023-10-30T12:56:16.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b5746640 1 --2- 172.21.15.12:0/1979752591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9b00c05d0 0x7fe9b013ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:56:16.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b5f47640 1 --2- 172.21.15.12:0/1979752591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe99c0039b0 0x7fe99c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:16.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b6748640 1 --2- 172.21.15.12:0/1979752591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe99c004fe0 0x7fe99c00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:16.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b5746640 1 --2- 172.21.15.12:0/1979752591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe99c004360 0x7fe99c004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b5f47640 1 -- 172.21.15.12:0/1979752591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe99c004360 msgr2=0x7fe99c004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b5f47640 1 --2- 172.21.15.12:0/1979752591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe99c004360 0x7fe99c004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b5f47640 1 -- 172.21.15.12:0/1979752591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe99c004fe0 msgr2=0x7fe99c00dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b5f47640 1 --2- 172.21.15.12:0/1979752591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe99c004fe0 0x7fe99c00dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:16.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.127+0000 7fe9b5f47640 1 -- 172.21.15.12:0/1979752591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c003530 con 0x7fe99c0039b0 2023-10-30T12:56:16.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b5f47640 1 --2- 172.21.15.12:0/1979752591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe99c0039b0 0x7fe99c003d90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac001910 tx=0x7fe9ac01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b9617ada797c0f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:16.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b4f45640 1 -- 172.21.15.12:0/1979752591 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0ad640 con 0x7fe99c0039b0 2023-10-30T12:56:16.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b4f45640 1 -- 172.21.15.12:0/1979752591 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac0ad7c0 con 0x7fe99c0039b0 2023-10-30T12:56:16.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b4f45640 1 -- 172.21.15.12:0/1979752591 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0adb70 con 0x7fe99c0039b0 2023-10-30T12:56:16.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b81ab640 1 -- 172.21.15.12:0/1979752591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe99c0039b0 msgr2=0x7fe99c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:16.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b81ab640 1 --2- 172.21.15.12:0/1979752591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe99c0039b0 0x7fe99c003d90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac001910 tx=0x7fe9ac01c560 comp rx=0 tx=0).stop 2023-10-30T12:56:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b6748640 1 -- 172.21.15.12:0/1979752591 reap_dead start 2023-10-30T12:56:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b81ab640 1 -- 172.21.15.12:0/1979752591 shutdown_connections 2023-10-30T12:56:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b81ab640 1 -- 172.21.15.12:0/1979752591 >> 172.21.15.12:0/1979752591 conn(0x7fe9b00c7910 msgr2=0x7fe9b00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b81ab640 1 -- 172.21.15.12:0/1979752591 shutdown_connections 2023-10-30T12:56:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b81ab640 1 -- 172.21.15.12:0/1979752591 wait complete. 2023-10-30T12:56:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b81ab640 1 Processor -- start 2023-10-30T12:56:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b81ab640 1 -- start start 2023-10-30T12:56:16.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b81ab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe99c0039b0 0x7fe9b01e4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:16.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b81ab640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9b013ca00 0x7fe9b00ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:16.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b81ab640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01544a0 0x7fe9b0138520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b81ab640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b01e4f70 con 0x7fe9b01544a0 2023-10-30T12:56:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b81ab640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b013d220 con 0x7fe99c0039b0 2023-10-30T12:56:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.131+0000 7fe9b81ab640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b01f6370 con 0x7fe9b013ca00 2023-10-30T12:56:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9b6748640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01544a0 0x7fe9b0138520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:16.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9b6748640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01544a0 0x7fe9b0138520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52252/0 (socket says 172.21.15.12:52252) 2023-10-30T12:56:16.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9b6748640 1 -- 172.21.15.12:0/1740604922 learned_addr learned my addr 172.21.15.12:0/1740604922 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:16.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9b5746640 1 --2- 172.21.15.12:0/1740604922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9b013ca00 0x7fe9b00ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9b5f47640 1 --2- 172.21.15.12:0/1740604922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe99c0039b0 0x7fe9b01e4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9b6748640 1 -- 172.21.15.12:0/1740604922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9b013ca00 msgr2=0x7fe9b00ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9b6748640 1 --2- 172.21.15.12:0/1740604922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9b013ca00 0x7fe9b00ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9b6748640 1 -- 172.21.15.12:0/1740604922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe99c0039b0 msgr2=0x7fe9b01e4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:16.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9b6748640 1 --2- 172.21.15.12:0/1740604922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe99c0039b0 0x7fe9b01e4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:16.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9b6748640 1 -- 172.21.15.12:0/1740604922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b01f6800 con 0x7fe9b01544a0 2023-10-30T12:56:16.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9b6748640 1 --2- 172.21.15.12:0/1740604922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01544a0 0x7fe9b0138520 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00c7340 tx=0x7fe9a800a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:16.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9a67fc640 1 -- 172.21.15.12:0/1740604922 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a800aca0 con 0x7fe9b01544a0 2023-10-30T12:56:16.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9a67fc640 1 -- 172.21.15.12:0/1740604922 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a800ae00 con 0x7fe9b01544a0 2023-10-30T12:56:16.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9a67fc640 1 -- 172.21.15.12:0/1740604922 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8013610 con 0x7fe9b01544a0 2023-10-30T12:56:16.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9b81ab640 1 -- 172.21.15.12:0/1740604922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b013cf40 con 0x7fe9b01544a0 2023-10-30T12:56:16.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9b81ab640 1 -- 172.21.15.12:0/1740604922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b01f5d20 con 0x7fe9b01544a0 2023-10-30T12:56:16.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9b81ab640 1 -- 172.21.15.12:0/1740604922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b004e170 con 0x7fe9b01544a0 2023-10-30T12:56:16.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9a67fc640 1 -- 172.21.15.12:0/1740604922 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe9a8013770 con 0x7fe9b01544a0 2023-10-30T12:56:16.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.135+0000 7fe9a67fc640 1 --2- 172.21.15.12:0/1740604922 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe9940664a0 0x7fe994068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:16.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.139+0000 7fe9a67fc640 1 -- 172.21.15.12:0/1740604922 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fe9a8084c90 con 0x7fe9b01544a0 2023-10-30T12:56:16.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.139+0000 7fe9b5f47640 1 --2- 172.21.15.12:0/1740604922 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe9940664a0 0x7fe994068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:16.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.139+0000 7fe9b5f47640 1 --2- 172.21.15.12:0/1740604922 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe9940664a0 0x7fe994068960 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0af000 tx=0x7fe9ac001330 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:16.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.139+0000 7fe9a67fc640 1 -- 172.21.15.12:0/1740604922 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe9a8051d30 con 0x7fe9b01544a0 2023-10-30T12:56:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.299+0000 7fe9b81ab640 1 -- 172.21.15.12:0/1740604922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9b00c54f0 con 0x7fe9b01544a0 2023-10-30T12:56:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.303+0000 7fe9a67fc640 1 -- 172.21.15.12:0/1740604922 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+18079 (secure 0 0 0) 0x7fe9a8017070 con 0x7fe9b01544a0 2023-10-30T12:56:16.306 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:56:16.306 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":55,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:54:25.460451+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:35:03.216053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:35:04.597162+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":46,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6836/2204547612":"2023-10-31T12:54:25.460390+0000","172.21.15.12:6837/2752088514":"2023-10-31T12:53:20.432410+0000","172.21.15.12:6834/2752088514":"2023-10-31T12:53:20.432410+0000","172.21.15.154:6839/2204547612":"2023-10-31T12:54:25.460390+0000","172.21.15.12:6836/2219276371":"2023-10-31T12:50:50.379953+0000","172.21.15.12:6833/2219276371":"2023-10-31T12:50:50.379953+0000","172.21.15.12:6835/559649789":"2023-10-31T12:49:35.359754+0000","172.21.15.12:6834/3836803994":"2023-10-31T12:37:55.139871+0000","172.21.15.154:6836/1517672481":"2023-10-31T12:43:13.483926+0000","172.21.15.12:6832/2676506149":"2023-10-31T12:40:40.195783+0000","172.21.15.154:6839/1517672481":"2023-10-31T12:43:13.483926+0000","172.21.15.12:6834/3124135109":"2023-10-31T12:46:40.313866+0000","172.21.15.12:6837/3836803994":"2023-10-31T12:37:55.139871+0000","172.21.15.12:6836/291392212":"2023-10-31T12:44:35.272569+0000","172.21.15.12:6837/3124135109":"2023-10-31T12:46:40.313866+0000","172.21.15.12:6833/291392212":"2023-10-31T12:44:35.272569+0000","172.21.15.154:6836/1635719967":"2023-10-31T12:47:45.331652+0000","172.21.15.12:6835/2676506149":"2023-10-31T12:40:40.195783+0000","172.21.15.154:6839/1635719967":"2023-10-31T12:47:45.331652+0000","172.21.15.12:6832/559649789":"2023-10-31T12:49:35.359754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:56:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.303+0000 7fe9b81ab640 1 -- 172.21.15.12:0/1740604922 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe9940664a0 msgr2=0x7fe994068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.303+0000 7fe9b81ab640 1 --2- 172.21.15.12:0/1740604922 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe9940664a0 0x7fe994068960 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0af000 tx=0x7fe9ac001330 comp rx=0 tx=0).stop 2023-10-30T12:56:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.303+0000 7fe9b81ab640 1 -- 172.21.15.12:0/1740604922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01544a0 msgr2=0x7fe9b0138520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.303+0000 7fe9b81ab640 1 --2- 172.21.15.12:0/1740604922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01544a0 0x7fe9b0138520 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00c7340 tx=0x7fe9a800a6a0 comp rx=0 tx=0).stop 2023-10-30T12:56:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.303+0000 7fe9b81ab640 1 -- 172.21.15.12:0/1740604922 shutdown_connections 2023-10-30T12:56:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.303+0000 7fe9b81ab640 1 --2- 172.21.15.12:0/1740604922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9b01544a0 0x7fe9b0138520 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.303+0000 7fe9b81ab640 1 --2- 172.21.15.12:0/1740604922 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe9940664a0 0x7fe994068960 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.303+0000 7fe9b81ab640 1 --2- 172.21.15.12:0/1740604922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9b013ca00 0x7fe9b00ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.303+0000 7fe9b81ab640 1 --2- 172.21.15.12:0/1740604922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe99c0039b0 0x7fe9b01e4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.303+0000 7fe9b81ab640 1 -- 172.21.15.12:0/1740604922 >> 172.21.15.12:0/1740604922 conn(0x7fe9b00c7910 msgr2=0x7fe9b0139310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.303+0000 7fe9b81ab640 1 -- 172.21.15.12:0/1740604922 shutdown_connections 2023-10-30T12:56:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:16.303+0000 7fe9b81ab640 1 -- 172.21.15.12:0/1740604922 wait complete. 2023-10-30T12:56:17.328 INFO:tasks.ceph:Scrubbing osd.0 2023-10-30T12:56:17.329 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:56:17.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b1efaa640 1 Processor -- start 2023-10-30T12:56:17.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b1efaa640 1 -- start start 2023-10-30T12:56:17.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b1efaa640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0b18158b10 0x7f0b18157620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:17.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b1efaa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b180c6490 0x7f0b180c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:17.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b1efaa640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b1806c810 con 0x7f0b181517f0 2023-10-30T12:56:17.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b1efaa640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b1815bbf0 con 0x7f0b18158b10 2023-10-30T12:56:17.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b1efaa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b18071ed0 con 0x7f0b180c6490 2023-10-30T12:56:17.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b17fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0b18158b10 0x7f0b18157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:17.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b17fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0b18158b10 0x7f0b18157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52268/0 (socket says 172.21.15.12:52268) 2023-10-30T12:56:17.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b1cd46640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0b181517f0 0x7f0b1815ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44400/0 (socket says 172.21.15.12:44400) 2023-10-30T12:56:17.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b17fff640 1 -- 172.21.15.12:0/1236952081 learned_addr learned my addr 172.21.15.12:0/1236952081 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:17.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b1d547640 1 --2- 172.21.15.12:0/1236952081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b180c6490 0x7f0b180c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:17.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b17fff640 1 -- 172.21.15.12:0/1236952081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b180c6490 msgr2=0x7f0b180c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:17.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b17fff640 1 --2- 172.21.15.12:0/1236952081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b180c6490 0x7f0b180c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b17fff640 1 -- 172.21.15.12:0/1236952081 >> v1:172.21.15.12:6789/0 conn(0x7f0b181517f0 legacy=0x7f0b1815ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:17.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b17fff640 1 -- 172.21.15.12:0/1236952081 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1815b630 con 0x7f0b18158b10 2023-10-30T12:56:17.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.487+0000 7f0b177fe640 1 -- 172.21.15.12:0/1236952081 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230281990 0 0) 0x7f0b1806c810 con 0x7f0b181517f0 2023-10-30T12:56:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.491+0000 7f0b17fff640 1 --2- 172.21.15.12:0/1236952081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b18158b10 0x7f0b18157620 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009870 tx=0x7f0b0401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda41a892b078c7f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:17.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.491+0000 7f0b177fe640 1 -- 172.21.15.12:0/1236952081 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b040ab620 con 0x7f0b18158b10 2023-10-30T12:56:17.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.491+0000 7f0b177fe640 1 -- 172.21.15.12:0/1236952081 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b040ab780 con 0x7f0b18158b10 2023-10-30T12:56:17.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.491+0000 7f0b177fe640 1 -- 172.21.15.12:0/1236952081 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b040abb10 con 0x7f0b18158b10 2023-10-30T12:56:17.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.491+0000 7f0b1efaa640 1 -- 172.21.15.12:0/1236952081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b18158b10 msgr2=0x7f0b18157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:17.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.491+0000 7f0b1efaa640 1 --2- 172.21.15.12:0/1236952081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b18158b10 0x7f0b18157620 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009870 tx=0x7f0b0401c510 comp rx=0 tx=0).stop 2023-10-30T12:56:17.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.491+0000 7f0b1efaa640 1 -- 172.21.15.12:0/1236952081 shutdown_connections 2023-10-30T12:56:17.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.491+0000 7f0b1efaa640 1 --2- 172.21.15.12:0/1236952081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b180c6490 0x7f0b180c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.491+0000 7f0b1efaa640 1 --2- 172.21.15.12:0/1236952081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b18158b10 0x7f0b18157620 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.491+0000 7f0b1efaa640 1 -- 172.21.15.12:0/1236952081 >> 172.21.15.12:0/1236952081 conn(0x7f0b180c7910 msgr2=0x7f0b180c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:17.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.491+0000 7f0b1efaa640 1 -- 172.21.15.12:0/1236952081 shutdown_connections 2023-10-30T12:56:17.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.491+0000 7f0b1efaa640 1 -- 172.21.15.12:0/1236952081 wait complete. 2023-10-30T12:56:17.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.491+0000 7f0b1efaa640 1 Processor -- start 2023-10-30T12:56:17.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.491+0000 7f0b1efaa640 1 -- start start 2023-10-30T12:56:17.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1efaa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b180c6490 0x7f0b181f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:17.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1efaa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b181517f0 0x7f0b1806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:17.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1efaa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b18158b10 0x7f0b1814f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:17.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1efaa640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b181f5c10 con 0x7f0b18158b10 2023-10-30T12:56:17.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1efaa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b180c1f10 con 0x7f0b180c6490 2023-10-30T12:56:17.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1efaa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b180c01b0 con 0x7f0b181517f0 2023-10-30T12:56:17.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1d547640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b18158b10 0x7f0b1814f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:17.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1d547640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b18158b10 0x7f0b1814f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52280/0 (socket says 172.21.15.12:52280) 2023-10-30T12:56:17.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1d547640 1 -- 172.21.15.12:0/2036279473 learned_addr learned my addr 172.21.15.12:0/2036279473 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:17.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1cd46640 1 --2- 172.21.15.12:0/2036279473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b180c6490 0x7f0b181f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:17.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b17fff640 1 --2- 172.21.15.12:0/2036279473 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b181517f0 0x7f0b1806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:17.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1d547640 1 -- 172.21.15.12:0/2036279473 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b181517f0 msgr2=0x7f0b1806b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:17.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1d547640 1 --2- 172.21.15.12:0/2036279473 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0b181517f0 0x7f0b1806b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1d547640 1 -- 172.21.15.12:0/2036279473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b180c6490 msgr2=0x7f0b181f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:17.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1d547640 1 --2- 172.21.15.12:0/2036279473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b180c6490 0x7f0b181f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1d547640 1 -- 172.21.15.12:0/2036279473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b040094d0 con 0x7f0b18158b10 2023-10-30T12:56:17.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1cd46640 1 --2- 172.21.15.12:0/2036279473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b180c6490 0x7f0b181f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:17.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1d547640 1 --2- 172.21.15.12:0/2036279473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b18158b10 0x7f0b1814f620 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000efc0 tx=0x7f0b1000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:17.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b157fa640 1 -- 172.21.15.12:0/2036279473 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10014480 con 0x7f0b18158b10 2023-10-30T12:56:17.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b157fa640 1 -- 172.21.15.12:0/2036279473 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1000f040 con 0x7f0b18158b10 2023-10-30T12:56:17.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b157fa640 1 -- 172.21.15.12:0/2036279473 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10014820 con 0x7f0b18158b10 2023-10-30T12:56:17.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1efaa640 1 -- 172.21.15.12:0/2036279473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b180c1950 con 0x7f0b18158b10 2023-10-30T12:56:17.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1efaa640 1 -- 172.21.15.12:0/2036279473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b181f5830 con 0x7f0b18158b10 2023-10-30T12:56:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.495+0000 7f0b1efaa640 1 -- 172.21.15.12:0/2036279473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0b180c5f70 con 0x7f0b18158b10 2023-10-30T12:56:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.499+0000 7f0b157fa640 1 -- 172.21.15.12:0/2036279473 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0b10014980 con 0x7f0b18158b10 2023-10-30T12:56:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.499+0000 7f0b157fa640 1 --2- 172.21.15.12:0/2036279473 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0afc0664a0 0x7f0afc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.499+0000 7f0b157fa640 1 -- 172.21.15.12:0/2036279473 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f0b10085120 con 0x7f0b18158b10 2023-10-30T12:56:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.499+0000 7f0b157fa640 1 --2- 172.21.15.12:0/2036279473 >> [v2:172.21.15.12:6808/2303962477,v1:172.21.15.12:6809/2303962477] conn(0x7f0afc06ef60 0x7f0afc0713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.499+0000 7f0b157fa640 1 -- 172.21.15.12:0/2036279473 --> [v2:172.21.15.12:6808/2303962477,v1:172.21.15.12:6809/2303962477] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0afc071ab0 con 0x7f0afc06ef60 2023-10-30T12:56:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.499+0000 7f0b157fa640 1 -- 172.21.15.12:0/2036279473 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b10087700 con 0x7f0b18158b10 2023-10-30T12:56:17.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.499+0000 7f0b1cd46640 1 --2- 172.21.15.12:0/2036279473 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0afc0664a0 0x7f0afc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:17.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.499+0000 7f0b17fff640 1 --2- 172.21.15.12:0/2036279473 >> [v2:172.21.15.12:6808/2303962477,v1:172.21.15.12:6809/2303962477] conn(0x7f0afc06ef60 0x7f0afc0713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:17.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.499+0000 7f0b17fff640 1 --2- 172.21.15.12:0/2036279473 >> [v2:172.21.15.12:6808/2303962477,v1:172.21.15.12:6809/2303962477] conn(0x7f0afc06ef60 0x7f0afc0713e0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:17.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.499+0000 7f0b1cd46640 1 --2- 172.21.15.12:0/2036279473 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0afc0664a0 0x7f0afc068960 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c007fd0 tx=0x7f0b0c007c20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:17.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.499+0000 7f0b157fa640 1 -- 172.21.15.12:0/2036279473 <== osd.0 v2:172.21.15.12:6808/2303962477 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f0afc071ab0 con 0x7f0afc06ef60 2023-10-30T12:56:17.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.511+0000 7f0b1efaa640 1 -- 172.21.15.12:0/2036279473 --> [v2:172.21.15.12:6808/2303962477,v1:172.21.15.12:6809/2303962477] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0b180c5c90 con 0x7f0afc06ef60 2023-10-30T12:56:17.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.515+0000 7f0b157fa640 1 -- 172.21.15.12:0/2036279473 <== osd.0 v2:172.21.15.12:6808/2303962477 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0b040ab5a0 con 0x7f0afc06ef60 2023-10-30T12:56:17.518 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-30T12:56:17.518 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:56:17.519 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-30T12:56:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.515+0000 7f0b1efaa640 1 -- 172.21.15.12:0/2036279473 >> [v2:172.21.15.12:6808/2303962477,v1:172.21.15.12:6809/2303962477] conn(0x7f0afc06ef60 msgr2=0x7f0afc0713e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.515+0000 7f0b1efaa640 1 --2- 172.21.15.12:0/2036279473 >> [v2:172.21.15.12:6808/2303962477,v1:172.21.15.12:6809/2303962477] conn(0x7f0afc06ef60 0x7f0afc0713e0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.515+0000 7f0b1efaa640 1 -- 172.21.15.12:0/2036279473 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0afc0664a0 msgr2=0x7f0afc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.515+0000 7f0b1efaa640 1 --2- 172.21.15.12:0/2036279473 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0afc0664a0 0x7f0afc068960 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c007fd0 tx=0x7f0b0c007c20 comp rx=0 tx=0).stop 2023-10-30T12:56:17.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.515+0000 7f0b1efaa640 1 -- 172.21.15.12:0/2036279473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b18158b10 msgr2=0x7f0b1814f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:17.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.515+0000 7f0b1efaa640 1 --2- 172.21.15.12:0/2036279473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b18158b10 0x7f0b1814f620 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000efc0 tx=0x7f0b1000c490 comp rx=0 tx=0).stop 2023-10-30T12:56:17.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.515+0000 7f0b1d547640 1 -- 172.21.15.12:0/2036279473 reap_dead start 2023-10-30T12:56:17.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.515+0000 7f0b1efaa640 1 -- 172.21.15.12:0/2036279473 shutdown_connections 2023-10-30T12:56:17.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.515+0000 7f0b1efaa640 1 -- 172.21.15.12:0/2036279473 >> 172.21.15.12:0/2036279473 conn(0x7f0b180c7910 msgr2=0x7f0b1806fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:17.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.515+0000 7f0b1efaa640 1 -- 172.21.15.12:0/2036279473 shutdown_connections 2023-10-30T12:56:17.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.515+0000 7f0b1efaa640 1 -- 172.21.15.12:0/2036279473 wait complete. 2023-10-30T12:56:17.537 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-30T12:56:17.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbefd572640 1 Processor -- start 2023-10-30T12:56:17.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbefd572640 1 -- start start 2023-10-30T12:56:17.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbefd572640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbef80c6b80 0x7fbef8071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:17.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbefd572640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbef80c6f60 0x7fbef813f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:17.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbefd572640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef8070010 con 0x7fbef80c08d0 2023-10-30T12:56:17.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbefd572640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef8072260 con 0x7fbef80c6f60 2023-10-30T12:56:17.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbefd572640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef806f6e0 con 0x7fbef80c6b80 2023-10-30T12:56:17.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbeeffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbef80c6f60 0x7fbef813f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:17.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbef77fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbef80c08d0 0x7fbef81565a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44410/0 (socket says 172.21.15.12:44410) 2023-10-30T12:56:17.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbef77fe640 1 -- 172.21.15.12:0/884124395 learned_addr learned my addr 172.21.15.12:0/884124395 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:17.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbef6ffd640 1 --2- 172.21.15.12:0/884124395 >> v2:172.21.15.154:3300/0 conn(0x7fbef80c6b80 0x7fbef8071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:17.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbeeffff640 1 -- 172.21.15.12:0/884124395 >> v2:172.21.15.154:3300/0 conn(0x7fbef80c6b80 msgr2=0x7fbef8071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:17.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbeeffff640 1 --2- 172.21.15.12:0/884124395 >> v2:172.21.15.154:3300/0 conn(0x7fbef80c6b80 0x7fbef8071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbeeffff640 1 -- 172.21.15.12:0/884124395 >> v1:172.21.15.12:6789/0 conn(0x7fbef80c08d0 legacy=0x7fbef81565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:17.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbeeffff640 1 -- 172.21.15.12:0/884124395 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef806f9c0 con 0x7fbef80c6f60 2023-10-30T12:56:17.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.679+0000 7fbef67fc640 1 -- 172.21.15.12:0/884124395 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851041466 0 0) 0x7fbef8070010 con 0x7fbef80c08d0 2023-10-30T12:56:17.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.683+0000 7fbeeffff640 1 --2- 172.21.15.12:0/884124395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbef80c6f60 0x7fbef813f600 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbef806eea0 tx=0x7fbee001eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ac7e1a00656df3 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:17.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.683+0000 7fbef67fc640 1 -- 172.21.15.12:0/884124395 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee00a7af0 con 0x7fbef80c6f60 2023-10-30T12:56:17.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.683+0000 7fbef67fc640 1 -- 172.21.15.12:0/884124395 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee00a7c50 con 0x7fbef80c6f60 2023-10-30T12:56:17.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.683+0000 7fbef67fc640 1 -- 172.21.15.12:0/884124395 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee00af440 con 0x7fbef80c6f60 2023-10-30T12:56:17.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.683+0000 7fbefd572640 1 -- 172.21.15.12:0/884124395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbef80c6f60 msgr2=0x7fbef813f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:17.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.683+0000 7fbefd572640 1 --2- 172.21.15.12:0/884124395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbef80c6f60 0x7fbef813f600 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbef806eea0 tx=0x7fbee001eb00 comp rx=0 tx=0).stop 2023-10-30T12:56:17.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.683+0000 7fbefd572640 1 -- 172.21.15.12:0/884124395 shutdown_connections 2023-10-30T12:56:17.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.683+0000 7fbefd572640 1 --2- 172.21.15.12:0/884124395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbef80c6f60 0x7fbef813f600 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.683+0000 7fbefd572640 1 --2- 172.21.15.12:0/884124395 >> v2:172.21.15.154:3300/0 conn(0x7fbef80c6b80 0x7fbef8071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.683+0000 7fbefd572640 1 -- 172.21.15.12:0/884124395 >> 172.21.15.12:0/884124395 conn(0x7fbef80c7910 msgr2=0x7fbef80c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:17.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.683+0000 7fbefd572640 1 -- 172.21.15.12:0/884124395 shutdown_connections 2023-10-30T12:56:17.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.683+0000 7fbefd572640 1 -- 172.21.15.12:0/884124395 wait complete. 2023-10-30T12:56:17.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.683+0000 7fbefd572640 1 Processor -- start 2023-10-30T12:56:17.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.683+0000 7fbefd572640 1 -- start start 2023-10-30T12:56:17.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbefd572640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbef80c08d0 0x7fbef81399a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:17.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbefd572640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbef80c6b80 0x7fbef80c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:17.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbefd572640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbef80c6f60 0x7fbef80c2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:17.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbefd572640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef81522b0 con 0x7fbef80c6b80 2023-10-30T12:56:17.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbefd572640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef813cf20 con 0x7fbef80c08d0 2023-10-30T12:56:17.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbefd572640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef8151610 con 0x7fbef80c6f60 2023-10-30T12:56:17.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbeeffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbef80c6b80 0x7fbef80c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:17.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbeeffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbef80c6b80 0x7fbef80c3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52290/0 (socket says 172.21.15.12:52290) 2023-10-30T12:56:17.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbeeffff640 1 -- 172.21.15.12:0/3315748956 learned_addr learned my addr 172.21.15.12:0/3315748956 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:17.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbef6ffd640 1 --2- 172.21.15.12:0/3315748956 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbef80c08d0 0x7fbef81399a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:17.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbef77fe640 1 --2- 172.21.15.12:0/3315748956 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbef80c6f60 0x7fbef80c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbeeffff640 1 -- 172.21.15.12:0/3315748956 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbef80c6f60 msgr2=0x7fbef80c2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbeeffff640 1 --2- 172.21.15.12:0/3315748956 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbef80c6f60 0x7fbef80c2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbeeffff640 1 -- 172.21.15.12:0/3315748956 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbef80c08d0 msgr2=0x7fbef81399a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbeeffff640 1 --2- 172.21.15.12:0/3315748956 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbef80c08d0 0x7fbef81399a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbeeffff640 1 -- 172.21.15.12:0/3315748956 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee000b6c0 con 0x7fbef80c6b80 2023-10-30T12:56:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbeeffff640 1 --2- 172.21.15.12:0/3315748956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbef80c6b80 0x7fbef80c3430 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbee00b2040 tx=0x7fbee001b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:17.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbeef7fe640 1 -- 172.21.15.12:0/3315748956 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee00b9600 con 0x7fbef80c6b80 2023-10-30T12:56:17.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbeef7fe640 1 -- 172.21.15.12:0/3315748956 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee00b9780 con 0x7fbef80c6b80 2023-10-30T12:56:17.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbefd572640 1 -- 172.21.15.12:0/3315748956 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef813af00 con 0x7fbef80c6b80 2023-10-30T12:56:17.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbeef7fe640 1 -- 172.21.15.12:0/3315748956 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee00b9b30 con 0x7fbef80c6b80 2023-10-30T12:56:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbefd572640 1 -- 172.21.15.12:0/3315748956 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef813d410 con 0x7fbef80c6b80 2023-10-30T12:56:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbefd572640 1 -- 172.21.15.12:0/3315748956 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef80b0f40 con 0x7fbef80c6b80 2023-10-30T12:56:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.687+0000 7fbeef7fe640 1 -- 172.21.15.12:0/3315748956 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbee00b9c90 con 0x7fbef80c6b80 2023-10-30T12:56:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.691+0000 7fbeef7fe640 1 --2- 172.21.15.12:0/3315748956 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbed80664a0 0x7fbed8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.691+0000 7fbeef7fe640 1 -- 172.21.15.12:0/3315748956 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fbee0121ad0 con 0x7fbef80c6b80 2023-10-30T12:56:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.691+0000 7fbef6ffd640 1 --2- 172.21.15.12:0/3315748956 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbed80664a0 0x7fbed8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.691+0000 7fbef6ffd640 1 --2- 172.21.15.12:0/3315748956 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbed80664a0 0x7fbed8068960 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbee800b9d0 tx=0x7fbee8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.695+0000 7fbeef7fe640 1 -- 172.21.15.12:0/3315748956 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbee00eeb70 con 0x7fbef80c6b80 2023-10-30T12:56:17.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.851+0000 7fbefd572640 1 -- 172.21.15.12:0/3315748956 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fbef813ff50 con 0x7fbed80664a0 2023-10-30T12:56:17.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.851+0000 7fbeef7fe640 1 -- 172.21.15.12:0/3315748956 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbee800b670 con 0x7fbed80664a0 2023-10-30T12:56:17.857 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 0 to deep-scrub 2023-10-30T12:56:17.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.855+0000 7fbefd572640 1 -- 172.21.15.12:0/3315748956 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbed80664a0 msgr2=0x7fbed8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:17.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.855+0000 7fbefd572640 1 --2- 172.21.15.12:0/3315748956 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbed80664a0 0x7fbed8068960 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbee800b9d0 tx=0x7fbee8008040 comp rx=0 tx=0).stop 2023-10-30T12:56:17.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.855+0000 7fbefd572640 1 -- 172.21.15.12:0/3315748956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbef80c6b80 msgr2=0x7fbef80c3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:17.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.855+0000 7fbefd572640 1 --2- 172.21.15.12:0/3315748956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbef80c6b80 0x7fbef80c3430 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbee00b2040 tx=0x7fbee001b8a0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.855+0000 7fbefd572640 1 -- 172.21.15.12:0/3315748956 shutdown_connections 2023-10-30T12:56:17.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.855+0000 7fbefd572640 1 --2- 172.21.15.12:0/3315748956 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbed80664a0 0x7fbed8068960 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.855+0000 7fbefd572640 1 --2- 172.21.15.12:0/3315748956 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbef80c6f60 0x7fbef80c2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.855+0000 7fbefd572640 1 --2- 172.21.15.12:0/3315748956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbef80c6b80 0x7fbef80c3430 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.855+0000 7fbefd572640 1 --2- 172.21.15.12:0/3315748956 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbef80c08d0 0x7fbef81399a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:17.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.855+0000 7fbefd572640 1 -- 172.21.15.12:0/3315748956 >> 172.21.15.12:0/3315748956 conn(0x7fbef80c7910 msgr2=0x7fbef815a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:17.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.855+0000 7fbefd572640 1 -- 172.21.15.12:0/3315748956 shutdown_connections 2023-10-30T12:56:17.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:17.855+0000 7fbefd572640 1 -- 172.21.15.12:0/3315748956 wait complete. 2023-10-30T12:56:17.878 INFO:tasks.ceph:Scrubbing osd.1 2023-10-30T12:56:17.879 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:56:18.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff026994640 1 Processor -- start 2023-10-30T12:56:18.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff026994640 1 -- start start 2023-10-30T12:56:18.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff026994640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff020153d90 0x7ff0200c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff026994640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff02015a7e0 con 0x7ff0200c66f0 2023-10-30T12:56:18.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff026994640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0200c6e00 con 0x7ff0200c79a0 2023-10-30T12:56:18.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff026994640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff020159c60 con 0x7ff020153d90 2023-10-30T12:56:18.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff01f7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff0200c66f0 0x7ff0200c7290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44414/0 (socket says 172.21.15.12:44414) 2023-10-30T12:56:18.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff01f7fe640 1 -- 172.21.15.12:0/2247667020 learned_addr learned my addr 172.21.15.12:0/2247667020 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:18.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff01ffff640 1 --2- 172.21.15.12:0/2247667020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff020153d90 0x7ff0200c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2070558325 0 0) 0x7ff02015a7e0 con 0x7ff0200c66f0 2023-10-30T12:56:18.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff008003530 con 0x7ff0200c66f0 2023-10-30T12:56:18.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231191406 0 0) 0x7ff0200c6e00 con 0x7ff0200c79a0 2023-10-30T12:56:18.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff02015a7e0 con 0x7ff0200c79a0 2023-10-30T12:56:18.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2977267785 0 0) 0x7ff008003530 con 0x7ff0200c66f0 2023-10-30T12:56:18.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff020153d90 msgr2=0x7ff0200c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:18.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.023+0000 7ff01effd640 1 --2- 172.21.15.12:0/2247667020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff020153d90 0x7ff0200c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 >> v1:172.21.15.154:6789/0 conn(0x7ff0200c79a0 legacy=0x7ff020164100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff020159f40 con 0x7ff0200c66f0 2023-10-30T12:56:18.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7ff014001680 con 0x7ff0200c66f0 2023-10-30T12:56:18.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 >> v1:172.21.15.12:6789/0 conn(0x7ff0200c66f0 legacy=0x7ff0200c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01effd640 1 --2- 172.21.15.12:0/2247667020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff008003b40 0x7ff008003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01effd640 1 --2- 172.21.15.12:0/2247667020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0080047d0 0x7ff008004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01effd640 1 --2- 172.21.15.12:0/2247667020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff008005190 0x7ff00800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff020159c60 con 0x7ff008003b40 2023-10-30T12:56:18.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00800bf80 con 0x7ff0080047d0 2023-10-30T12:56:18.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00800c140 con 0x7ff008005190 2023-10-30T12:56:18.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01ffff640 1 --2- 172.21.15.12:0/2247667020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff020153d90 0x7ff0200c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:56:18.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01ffff640 1 --2- 172.21.15.12:0/2247667020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff008003b40 0x7ff008003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff024f31640 1 --2- 172.21.15.12:0/2247667020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff008005190 0x7ff00800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01f7fe640 1 --2- 172.21.15.12:0/2247667020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0080047d0 0x7ff008004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01ffff640 1 -- 172.21.15.12:0/2247667020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff008005190 msgr2=0x7ff00800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:18.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01ffff640 1 --2- 172.21.15.12:0/2247667020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff008005190 0x7ff00800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01ffff640 1 -- 172.21.15.12:0/2247667020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0080047d0 msgr2=0x7ff008004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:18.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01ffff640 1 --2- 172.21.15.12:0/2247667020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0080047d0 0x7ff008004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01ffff640 1 -- 172.21.15.12:0/2247667020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff008003530 con 0x7ff008003b40 2023-10-30T12:56:18.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.027+0000 7ff01ffff640 1 --2- 172.21.15.12:0/2247667020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff008003b40 0x7ff008003f40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff02006d0f0 tx=0x7ff01001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5da154329ee77bd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:18.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff010004030 con 0x7ff008003b40 2023-10-30T12:56:18.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff010004190 con 0x7ff008003b40 2023-10-30T12:56:18.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff01effd640 1 -- 172.21.15.12:0/2247667020 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff010004520 con 0x7ff008003b40 2023-10-30T12:56:18.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff026994640 1 -- 172.21.15.12:0/2247667020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff008003b40 msgr2=0x7ff008003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff026994640 1 --2- 172.21.15.12:0/2247667020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff008003b40 0x7ff008003f40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff02006d0f0 tx=0x7ff01001ca60 comp rx=0 tx=0).stop 2023-10-30T12:56:18.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff024f31640 1 -- 172.21.15.12:0/2247667020 reap_dead start 2023-10-30T12:56:18.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff026994640 1 -- 172.21.15.12:0/2247667020 shutdown_connections 2023-10-30T12:56:18.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff026994640 1 -- 172.21.15.12:0/2247667020 >> 172.21.15.12:0/2247667020 conn(0x7ff0200cf060 msgr2=0x7ff0200cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:18.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff026994640 1 -- 172.21.15.12:0/2247667020 shutdown_connections 2023-10-30T12:56:18.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff026994640 1 -- 172.21.15.12:0/2247667020 wait complete. 2023-10-30T12:56:18.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff026994640 1 Processor -- start 2023-10-30T12:56:18.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff026994640 1 -- start start 2023-10-30T12:56:18.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff026994640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff008003b40 0x7ff0201f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff026994640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff02015a9c0 0x7ff0201f3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff026994640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0200c55f0 0x7ff020147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff026994640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02014fa10 con 0x7ff0200c55f0 2023-10-30T12:56:18.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff026994640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02015af20 con 0x7ff008003b40 2023-10-30T12:56:18.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.031+0000 7ff026994640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02014f0e0 con 0x7ff02015a9c0 2023-10-30T12:56:18.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff024f31640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0200c55f0 0x7ff020147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff024f31640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0200c55f0 0x7ff020147900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52300/0 (socket says 172.21.15.12:52300) 2023-10-30T12:56:18.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff024f31640 1 -- 172.21.15.12:0/3317890028 learned_addr learned my addr 172.21.15.12:0/3317890028 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:18.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff01ffff640 1 --2- 172.21.15.12:0/3317890028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff008003b40 0x7ff0201f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff01f7fe640 1 --2- 172.21.15.12:0/3317890028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff02015a9c0 0x7ff0201f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff024f31640 1 -- 172.21.15.12:0/3317890028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff02015a9c0 msgr2=0x7ff0201f3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff024f31640 1 --2- 172.21.15.12:0/3317890028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff02015a9c0 0x7ff0201f3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff024f31640 1 -- 172.21.15.12:0/3317890028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff008003b40 msgr2=0x7ff0201f32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff024f31640 1 --2- 172.21.15.12:0/3317890028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff008003b40 0x7ff0201f32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff024f31640 1 -- 172.21.15.12:0/3317890028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02014f3c0 con 0x7ff0200c55f0 2023-10-30T12:56:18.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff024f31640 1 --2- 172.21.15.12:0/3317890028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0200c55f0 0x7ff020147900 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff020067630 tx=0x7ff01800ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:18.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff01cff9640 1 -- 172.21.15.12:0/3317890028 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff018006d70 con 0x7ff0200c55f0 2023-10-30T12:56:18.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff01cff9640 1 -- 172.21.15.12:0/3317890028 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff018003980 con 0x7ff0200c55f0 2023-10-30T12:56:18.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff026994640 1 -- 172.21.15.12:0/3317890028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02014f6a0 con 0x7ff0200c55f0 2023-10-30T12:56:18.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff01cff9640 1 -- 172.21.15.12:0/3317890028 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff018003d30 con 0x7ff0200c55f0 2023-10-30T12:56:18.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff026994640 1 -- 172.21.15.12:0/3317890028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff020148010 con 0x7ff0200c55f0 2023-10-30T12:56:18.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff026994640 1 -- 172.21.15.12:0/3317890028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff02014f250 con 0x7ff0200c55f0 2023-10-30T12:56:18.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff01cff9640 1 -- 172.21.15.12:0/3317890028 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff01800d740 con 0x7ff0200c55f0 2023-10-30T12:56:18.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff01cff9640 1 --2- 172.21.15.12:0/3317890028 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff00c0664a0 0x7ff00c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.035+0000 7ff01cff9640 1 -- 172.21.15.12:0/3317890028 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7ff018083980 con 0x7ff0200c55f0 2023-10-30T12:56:18.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.039+0000 7ff01cff9640 1 --2- 172.21.15.12:0/3317890028 >> [v2:172.21.15.12:6816/2335476277,v1:172.21.15.12:6817/2335476277] conn(0x7ff00c06ef60 0x7ff00c0713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.039+0000 7ff01cff9640 1 -- 172.21.15.12:0/3317890028 --> [v2:172.21.15.12:6816/2335476277,v1:172.21.15.12:6817/2335476277] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff00c071ab0 con 0x7ff00c06ef60 2023-10-30T12:56:18.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.039+0000 7ff01cff9640 1 -- 172.21.15.12:0/3317890028 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0180882c0 con 0x7ff0200c55f0 2023-10-30T12:56:18.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.039+0000 7ff01f7fe640 1 --2- 172.21.15.12:0/3317890028 >> [v2:172.21.15.12:6816/2335476277,v1:172.21.15.12:6817/2335476277] conn(0x7ff00c06ef60 0x7ff00c0713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.039+0000 7ff01ffff640 1 --2- 172.21.15.12:0/3317890028 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff00c0664a0 0x7ff00c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.039+0000 7ff01f7fe640 1 --2- 172.21.15.12:0/3317890028 >> [v2:172.21.15.12:6816/2335476277,v1:172.21.15.12:6817/2335476277] conn(0x7ff00c06ef60 0x7ff00c0713e0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:18.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.039+0000 7ff01ffff640 1 --2- 172.21.15.12:0/3317890028 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff00c0664a0 0x7ff00c068960 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff0100ad000 tx=0x7ff0100046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:18.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.039+0000 7ff01cff9640 1 -- 172.21.15.12:0/3317890028 <== osd.1 v2:172.21.15.12:6816/2335476277 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7ff00c071ab0 con 0x7ff00c06ef60 2023-10-30T12:56:18.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.051+0000 7ff026994640 1 -- 172.21.15.12:0/3317890028 --> [v2:172.21.15.12:6816/2335476277,v1:172.21.15.12:6817/2335476277] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff0200b4e50 con 0x7ff00c06ef60 2023-10-30T12:56:18.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.051+0000 7ff01cff9640 1 -- 172.21.15.12:0/3317890028 <== osd.1 v2:172.21.15.12:6816/2335476277 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff014007d20 con 0x7ff00c06ef60 2023-10-30T12:56:18.058 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-30T12:56:18.058 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:56:18.058 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-30T12:56:18.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.051+0000 7ff026994640 1 -- 172.21.15.12:0/3317890028 >> [v2:172.21.15.12:6816/2335476277,v1:172.21.15.12:6817/2335476277] conn(0x7ff00c06ef60 msgr2=0x7ff00c0713e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.051+0000 7ff026994640 1 --2- 172.21.15.12:0/3317890028 >> [v2:172.21.15.12:6816/2335476277,v1:172.21.15.12:6817/2335476277] conn(0x7ff00c06ef60 0x7ff00c0713e0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.051+0000 7ff026994640 1 -- 172.21.15.12:0/3317890028 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff00c0664a0 msgr2=0x7ff00c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.051+0000 7ff026994640 1 --2- 172.21.15.12:0/3317890028 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff00c0664a0 0x7ff00c068960 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff0100ad000 tx=0x7ff0100046d0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.051+0000 7ff026994640 1 -- 172.21.15.12:0/3317890028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0200c55f0 msgr2=0x7ff020147900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.051+0000 7ff026994640 1 --2- 172.21.15.12:0/3317890028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0200c55f0 0x7ff020147900 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff020067630 tx=0x7ff01800ff70 comp rx=0 tx=0).stop 2023-10-30T12:56:18.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.051+0000 7ff024f31640 1 -- 172.21.15.12:0/3317890028 reap_dead start 2023-10-30T12:56:18.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.051+0000 7ff026994640 1 -- 172.21.15.12:0/3317890028 shutdown_connections 2023-10-30T12:56:18.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.051+0000 7ff026994640 1 -- 172.21.15.12:0/3317890028 >> 172.21.15.12:0/3317890028 conn(0x7ff0200cf060 msgr2=0x7ff020069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:18.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.051+0000 7ff026994640 1 -- 172.21.15.12:0/3317890028 shutdown_connections 2023-10-30T12:56:18.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.051+0000 7ff026994640 1 -- 172.21.15.12:0/3317890028 wait complete. 2023-10-30T12:56:18.075 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-30T12:56:18.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb91002640 1 Processor -- start 2023-10-30T12:56:18.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb91002640 1 -- start start 2023-10-30T12:56:18.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb91002640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb8c0c90d0 0x7fcb8c0cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb91002640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcb8c0cf4d0 0x7fcb8c15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb91002640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8c15ce30 con 0x7fcb8c0c8cf0 2023-10-30T12:56:18.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb91002640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8c159880 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb91002640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c159ae0 con 0x7fcb8c0c90d0 2023-10-30T12:56:18.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb8b59e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcb8c0cf4d0 0x7fcb8c15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb8ad9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fcb8c0c8cf0 0x7fcb8c0c9e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44426/0 (socket says 172.21.15.12:44426) 2023-10-30T12:56:18.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb8ad9d640 1 -- 172.21.15.12:0/1658764882 learned_addr learned my addr 172.21.15.12:0/1658764882 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:18.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb83fff640 1 --2- 172.21.15.12:0/1658764882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb8c0c90d0 0x7fcb8c0cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb8b59e640 1 -- 172.21.15.12:0/1658764882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb8c0c90d0 msgr2=0x7fcb8c0cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:18.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb8b59e640 1 --2- 172.21.15.12:0/1658764882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb8c0c90d0 0x7fcb8c0cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb8b59e640 1 -- 172.21.15.12:0/1658764882 >> v1:172.21.15.12:6789/0 conn(0x7fcb8c0c8cf0 legacy=0x7fcb8c0c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb8b59e640 1 -- 172.21.15.12:0/1658764882 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c159dc0 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.215+0000 7fcb8a59c640 1 -- 172.21.15.12:0/1658764882 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323705924 0 0) 0x7fcb8c15ce30 con 0x7fcb8c0c8cf0 2023-10-30T12:56:18.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.219+0000 7fcb8b59e640 1 --2- 172.21.15.12:0/1658764882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c0cf4d0 0x7fcb8c15d2c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009d90 tx=0x7fcb8401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20482db9c443c888 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:18.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.219+0000 7fcb8a59c640 1 -- 172.21.15.12:0/1658764882 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb840a59e0 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.219+0000 7fcb8a59c640 1 -- 172.21.15.12:0/1658764882 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb840a5b40 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.219+0000 7fcb8a59c640 1 -- 172.21.15.12:0/1658764882 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb840ad440 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.219+0000 7fcb91002640 1 -- 172.21.15.12:0/1658764882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c0cf4d0 msgr2=0x7fcb8c15d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.219+0000 7fcb91002640 1 --2- 172.21.15.12:0/1658764882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c0cf4d0 0x7fcb8c15d2c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009d90 tx=0x7fcb8401c920 comp rx=0 tx=0).stop 2023-10-30T12:56:18.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.219+0000 7fcb91002640 1 -- 172.21.15.12:0/1658764882 shutdown_connections 2023-10-30T12:56:18.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.219+0000 7fcb91002640 1 --2- 172.21.15.12:0/1658764882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c0cf4d0 0x7fcb8c15d2c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.219+0000 7fcb91002640 1 --2- 172.21.15.12:0/1658764882 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb8c0c90d0 0x7fcb8c0cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.219+0000 7fcb91002640 1 -- 172.21.15.12:0/1658764882 >> 172.21.15.12:0/1658764882 conn(0x7fcb8c06be40 msgr2=0x7fcb8c06c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.219+0000 7fcb91002640 1 -- 172.21.15.12:0/1658764882 shutdown_connections 2023-10-30T12:56:18.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.219+0000 7fcb91002640 1 -- 172.21.15.12:0/1658764882 wait complete. 2023-10-30T12:56:18.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.223+0000 7fcb91002640 1 Processor -- start 2023-10-30T12:56:18.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.223+0000 7fcb91002640 1 -- start start 2023-10-30T12:56:18.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.223+0000 7fcb91002640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb8c0c8cf0 0x7fcb8c154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.223+0000 7fcb91002640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb8c0c90d0 0x7fcb8c1ef830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.223+0000 7fcb91002640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c0cf4d0 0x7fcb8c157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.223+0000 7fcb91002640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c1f44f0 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.223+0000 7fcb91002640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c1efd70 con 0x7fcb8c0c90d0 2023-10-30T12:56:18.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.223+0000 7fcb91002640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c1f38e0 con 0x7fcb8c0c8cf0 2023-10-30T12:56:18.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.223+0000 7fcb8b59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c0cf4d0 0x7fcb8c157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.223+0000 7fcb8b59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c0cf4d0 0x7fcb8c157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52308/0 (socket says 172.21.15.12:52308) 2023-10-30T12:56:18.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.223+0000 7fcb8b59e640 1 -- 172.21.15.12:0/3432120456 learned_addr learned my addr 172.21.15.12:0/3432120456 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:18.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.223+0000 7fcb8ad9d640 1 --2- 172.21.15.12:0/3432120456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb8c0c8cf0 0x7fcb8c154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.223+0000 7fcb83fff640 1 --2- 172.21.15.12:0/3432120456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb8c0c90d0 0x7fcb8c1ef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.227+0000 7fcb8b59e640 1 -- 172.21.15.12:0/3432120456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb8c0c8cf0 msgr2=0x7fcb8c154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.227+0000 7fcb8b59e640 1 --2- 172.21.15.12:0/3432120456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb8c0c8cf0 0x7fcb8c154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.227+0000 7fcb8b59e640 1 -- 172.21.15.12:0/3432120456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb8c0c90d0 msgr2=0x7fcb8c1ef830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.227+0000 7fcb8b59e640 1 --2- 172.21.15.12:0/3432120456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb8c0c90d0 0x7fcb8c1ef830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.227+0000 7fcb8b59e640 1 -- 172.21.15.12:0/3432120456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb84009a70 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.227+0000 7fcb8b59e640 1 --2- 172.21.15.12:0/3432120456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c0cf4d0 0x7fcb8c157a30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcb840b0040 tx=0x7fcb840196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.227+0000 7fcb837fe640 1 -- 172.21.15.12:0/3432120456 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb840ade40 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.227+0000 7fcb837fe640 1 -- 172.21.15.12:0/3432120456 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb84009450 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.227+0000 7fcb837fe640 1 -- 172.21.15.12:0/3432120456 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb840b7630 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.227+0000 7fcb91002640 1 -- 172.21.15.12:0/3432120456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c1f3bc0 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.227+0000 7fcb91002640 1 -- 172.21.15.12:0/3432120456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8c1f4180 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.227+0000 7fcb91002640 1 -- 172.21.15.12:0/3432120456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8c1f4660 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.227+0000 7fcb837fe640 1 -- 172.21.15.12:0/3432120456 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcb840b4050 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.227+0000 7fcb837fe640 1 --2- 172.21.15.12:0/3432120456 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcb6c0664a0 0x7fcb6c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.231+0000 7fcb837fe640 1 -- 172.21.15.12:0/3432120456 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fcb8411f460 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.231+0000 7fcb8ad9d640 1 --2- 172.21.15.12:0/3432120456 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcb6c0664a0 0x7fcb6c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.231+0000 7fcb8ad9d640 1 --2- 172.21.15.12:0/3432120456 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcb6c0664a0 0x7fcb6c068960 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c000f70 tx=0x7fcb7c007df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:18.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.235+0000 7fcb837fe640 1 -- 172.21.15.12:0/3432120456 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcb840ec520 con 0x7fcb8c0cf4d0 2023-10-30T12:56:18.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.395+0000 7fcb91002640 1 -- 172.21.15.12:0/3432120456 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb8c15dc10 con 0x7fcb6c0664a0 2023-10-30T12:56:18.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.399+0000 7fcb837fe640 1 -- 172.21.15.12:0/3432120456 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcb7c0072a0 con 0x7fcb6c0664a0 2023-10-30T12:56:18.402 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 1 to deep-scrub 2023-10-30T12:56:18.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.399+0000 7fcb91002640 1 -- 172.21.15.12:0/3432120456 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcb6c0664a0 msgr2=0x7fcb6c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.399+0000 7fcb91002640 1 --2- 172.21.15.12:0/3432120456 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcb6c0664a0 0x7fcb6c068960 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c000f70 tx=0x7fcb7c007df0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.399+0000 7fcb91002640 1 -- 172.21.15.12:0/3432120456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c0cf4d0 msgr2=0x7fcb8c157a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.399+0000 7fcb91002640 1 --2- 172.21.15.12:0/3432120456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c0cf4d0 0x7fcb8c157a30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcb840b0040 tx=0x7fcb840196c0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.399+0000 7fcb91002640 1 -- 172.21.15.12:0/3432120456 shutdown_connections 2023-10-30T12:56:18.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.399+0000 7fcb91002640 1 --2- 172.21.15.12:0/3432120456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c0cf4d0 0x7fcb8c157a30 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.399+0000 7fcb91002640 1 --2- 172.21.15.12:0/3432120456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb8c0c90d0 0x7fcb8c1ef830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.399+0000 7fcb91002640 1 --2- 172.21.15.12:0/3432120456 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fcb6c0664a0 0x7fcb6c068960 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.399+0000 7fcb91002640 1 --2- 172.21.15.12:0/3432120456 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb8c0c8cf0 0x7fcb8c154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.399+0000 7fcb91002640 1 -- 172.21.15.12:0/3432120456 >> 172.21.15.12:0/3432120456 conn(0x7fcb8c06be40 msgr2=0x7fcb8c0b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:18.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.399+0000 7fcb91002640 1 -- 172.21.15.12:0/3432120456 shutdown_connections 2023-10-30T12:56:18.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.399+0000 7fcb91002640 1 -- 172.21.15.12:0/3432120456 wait complete. 2023-10-30T12:56:18.425 INFO:tasks.ceph:Scrubbing osd.2 2023-10-30T12:56:18.425 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:56:18.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.571+0000 7fbb0371d640 1 Processor -- start 2023-10-30T12:56:18.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.571+0000 7fbb0371d640 1 -- start start 2023-10-30T12:56:18.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.571+0000 7fbb0371d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbafc157620 0x7fbafc0c6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.571+0000 7fbb0371d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbafc06c810 con 0x7fbafc158b10 2023-10-30T12:56:18.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.571+0000 7fbb0371d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbafc15b630 con 0x7fbafc1517f0 2023-10-30T12:56:18.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.571+0000 7fbb0371d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc15bbf0 con 0x7fbafc157620 2023-10-30T12:56:18.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.571+0000 7fbb00cb8640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbafc158b10 0x7fbafc0c6b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44436/0 (socket says 172.21.15.12:44436) 2023-10-30T12:56:18.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.571+0000 7fbb00cb8640 1 -- 172.21.15.12:0/2140974027 learned_addr learned my addr 172.21.15.12:0/2140974027 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:18.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.571+0000 7fbb01cba640 1 --2- 172.21.15.12:0/2140974027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbafc157620 0x7fbafc0c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbaf3fff640 1 -- 172.21.15.12:0/2140974027 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253779589 0 0) 0x7fbafc06c810 con 0x7fbafc158b10 2023-10-30T12:56:18.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbaf3fff640 1 -- 172.21.15.12:0/2140974027 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbae8003530 con 0x7fbafc158b10 2023-10-30T12:56:18.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbaf3fff640 1 -- 172.21.15.12:0/2140974027 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1331520757 0 0) 0x7fbae8003530 con 0x7fbafc158b10 2023-10-30T12:56:18.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbaf3fff640 1 -- 172.21.15.12:0/2140974027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbafc157620 msgr2=0x7fbafc0c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:18.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbaf3fff640 1 --2- 172.21.15.12:0/2140974027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbafc157620 0x7fbafc0c6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbaf3fff640 1 -- 172.21.15.12:0/2140974027 >> v1:172.21.15.154:6789/0 conn(0x7fbafc1517f0 legacy=0x7fbafc15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbaf3fff640 1 -- 172.21.15.12:0/2140974027 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc15b910 con 0x7fbafc158b10 2023-10-30T12:56:18.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbaf3fff640 1 -- 172.21.15.12:0/2140974027 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fbaec001680 con 0x7fbafc158b10 2023-10-30T12:56:18.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbaf3fff640 1 -- 172.21.15.12:0/2140974027 >> v1:172.21.15.12:6789/0 conn(0x7fbafc158b10 legacy=0x7fbafc0c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbaf3fff640 1 --2- 172.21.15.12:0/2140974027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbafc157620 0x7fbae80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbaf3fff640 1 --2- 172.21.15.12:0/2140974027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbae8003ef0 0x7fbae80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbaf3fff640 1 --2- 172.21.15.12:0/2140974027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbae8004d10 0x7fbae8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbaf3fff640 1 -- 172.21.15.12:0/2140974027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc15bbf0 con 0x7fbae8003ef0 2023-10-30T12:56:18.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbaf3fff640 1 -- 172.21.15.12:0/2140974027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae800b9f0 con 0x7fbae8004d10 2023-10-30T12:56:18.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbaf3fff640 1 -- 172.21.15.12:0/2140974027 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae800bbb0 con 0x7fbafc157620 2023-10-30T12:56:18.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbb00cb8640 1 --2- 172.21.15.12:0/2140974027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbae8003ef0 0x7fbae80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbb014b9640 1 --2- 172.21.15.12:0/2140974027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbafc157620 0x7fbae80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbb01cba640 1 --2- 172.21.15.12:0/2140974027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbae8004d10 0x7fbae8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbb00cb8640 1 -- 172.21.15.12:0/2140974027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbafc157620 msgr2=0x7fbae80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:18.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbb00cb8640 1 --2- 172.21.15.12:0/2140974027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbafc157620 0x7fbae80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbb00cb8640 1 -- 172.21.15.12:0/2140974027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbae8004d10 msgr2=0x7fbae8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:18.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbb00cb8640 1 --2- 172.21.15.12:0/2140974027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbae8004d10 0x7fbae8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.575+0000 7fbb00cb8640 1 -- 172.21.15.12:0/2140974027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae8003530 con 0x7fbae8003ef0 2023-10-30T12:56:18.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.579+0000 7fbb00cb8640 1 --2- 172.21.15.12:0/2140974027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbae8003ef0 0x7fbae80047d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00c160 tx=0x7fbaec01cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fddaa2364262ad server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:18.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.579+0000 7fbaf3fff640 1 -- 172.21.15.12:0/2140974027 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec005ae0 con 0x7fbae8003ef0 2023-10-30T12:56:18.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.579+0000 7fbaf3fff640 1 -- 172.21.15.12:0/2140974027 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaec005c60 con 0x7fbae8003ef0 2023-10-30T12:56:18.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.579+0000 7fbaf3fff640 1 -- 172.21.15.12:0/2140974027 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0a5a20 con 0x7fbae8003ef0 2023-10-30T12:56:18.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.579+0000 7fbb0371d640 1 -- 172.21.15.12:0/2140974027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbae8003ef0 msgr2=0x7fbae80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.579+0000 7fbb0371d640 1 --2- 172.21.15.12:0/2140974027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbae8003ef0 0x7fbae80047d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00c160 tx=0x7fbaec01cae0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.579+0000 7fbb01cba640 1 -- 172.21.15.12:0/2140974027 reap_dead start 2023-10-30T12:56:18.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.579+0000 7fbb0371d640 1 -- 172.21.15.12:0/2140974027 shutdown_connections 2023-10-30T12:56:18.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.579+0000 7fbb0371d640 1 -- 172.21.15.12:0/2140974027 >> 172.21.15.12:0/2140974027 conn(0x7fbafc0c7910 msgr2=0x7fbafc0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:18.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.579+0000 7fbb0371d640 1 -- 172.21.15.12:0/2140974027 shutdown_connections 2023-10-30T12:56:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.579+0000 7fbb0371d640 1 -- 172.21.15.12:0/2140974027 wait complete. 2023-10-30T12:56:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.579+0000 7fbb0371d640 1 Processor -- start 2023-10-30T12:56:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.579+0000 7fbb0371d640 1 -- start start 2023-10-30T12:56:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.579+0000 7fbb0371d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbae8003ef0 0x7fbafc14e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.579+0000 7fbb0371d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbafc14eb50 0x7fbafc14bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb0371d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbafc14c1e0 0x7fbafc1f2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb0371d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc06f7d0 con 0x7fbafc14c1e0 2023-10-30T12:56:18.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb0371d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc14e180 con 0x7fbae8003ef0 2023-10-30T12:56:18.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb0371d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc070660 con 0x7fbafc14eb50 2023-10-30T12:56:18.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb01cba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbafc14c1e0 0x7fbafc1f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb01cba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbafc14c1e0 0x7fbafc1f2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52328/0 (socket says 172.21.15.12:52328) 2023-10-30T12:56:18.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb01cba640 1 -- 172.21.15.12:0/463248313 learned_addr learned my addr 172.21.15.12:0/463248313 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:18.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb014b9640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbae8003ef0 0x7fbafc14e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb00cb8640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbafc14eb50 0x7fbafc14bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb01cba640 1 -- 172.21.15.12:0/463248313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbafc14eb50 msgr2=0x7fbafc14bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb01cba640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbafc14eb50 0x7fbafc14bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb01cba640 1 -- 172.21.15.12:0/463248313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbae8003ef0 msgr2=0x7fbafc14e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb01cba640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbae8003ef0 0x7fbafc14e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb01cba640 1 -- 172.21.15.12:0/463248313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc070260 con 0x7fbafc14c1e0 2023-10-30T12:56:18.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb014b9640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbae8003ef0 0x7fbafc14e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:18.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb00cb8640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbafc14eb50 0x7fbafc14bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:18.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb01cba640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbafc14c1e0 0x7fbafc1f2220 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fbafc1551f0 tx=0x7fbaf400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:18.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbaf1ffb640 1 -- 172.21.15.12:0/463248313 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf400e040 con 0x7fbafc14c1e0 2023-10-30T12:56:18.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbaf1ffb640 1 -- 172.21.15.12:0/463248313 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf400f040 con 0x7fbafc14c1e0 2023-10-30T12:56:18.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbaf1ffb640 1 -- 172.21.15.12:0/463248313 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4015430 con 0x7fbafc14c1e0 2023-10-30T12:56:18.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb0371d640 1 -- 172.21.15.12:0/463248313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbafc0c1cf0 con 0x7fbafc14c1e0 2023-10-30T12:56:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb0371d640 1 -- 172.21.15.12:0/463248313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbafc14d850 con 0x7fbafc14c1e0 2023-10-30T12:56:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbb0371d640 1 -- 172.21.15.12:0/463248313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbafc0c5c90 con 0x7fbafc14c1e0 2023-10-30T12:56:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.583+0000 7fbaf1ffb640 1 -- 172.21.15.12:0/463248313 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbaf4015590 con 0x7fbafc14c1e0 2023-10-30T12:56:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.587+0000 7fbaf1ffb640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbae00664d0 0x7fbae0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.587+0000 7fbaf1ffb640 1 -- 172.21.15.12:0/463248313 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fbaf4084b10 con 0x7fbafc14c1e0 2023-10-30T12:56:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.587+0000 7fbaf1ffb640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.12:6800/3034215139,v1:172.21.15.12:6801/3034215139] conn(0x7fbae006ef90 0x7fbae0071410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.587+0000 7fbaf1ffb640 1 -- 172.21.15.12:0/463248313 --> [v2:172.21.15.12:6800/3034215139,v1:172.21.15.12:6801/3034215139] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbae0071ae0 con 0x7fbae006ef90 2023-10-30T12:56:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.587+0000 7fbaf1ffb640 1 -- 172.21.15.12:0/463248313 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaf40892a0 con 0x7fbafc14c1e0 2023-10-30T12:56:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.587+0000 7fbb014b9640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbae00664d0 0x7fbae0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.587+0000 7fbb00cb8640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.12:6800/3034215139,v1:172.21.15.12:6801/3034215139] conn(0x7fbae006ef90 0x7fbae0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.587+0000 7fbb00cb8640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.12:6800/3034215139,v1:172.21.15.12:6801/3034215139] conn(0x7fbae006ef90 0x7fbae0071410 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.587+0000 7fbb014b9640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbae00664d0 0x7fbae0068990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8007fd0 tx=0x7fbaf8007cd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.587+0000 7fbaf1ffb640 1 -- 172.21.15.12:0/463248313 <== osd.2 v2:172.21.15.12:6800/3034215139 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fbae0071ae0 con 0x7fbae006ef90 2023-10-30T12:56:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.599+0000 7fbb0371d640 1 -- 172.21.15.12:0/463248313 --> [v2:172.21.15.12:6800/3034215139,v1:172.21.15.12:6801/3034215139] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbafc0c59b0 con 0x7fbae006ef90 2023-10-30T12:56:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.599+0000 7fbaf1ffb640 1 -- 172.21.15.12:0/463248313 <== osd.2 v2:172.21.15.12:6800/3034215139 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbaec0b5040 con 0x7fbae006ef90 2023-10-30T12:56:18.605 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-30T12:56:18.606 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:56:18.606 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-30T12:56:18.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.599+0000 7fbb0371d640 1 -- 172.21.15.12:0/463248313 >> [v2:172.21.15.12:6800/3034215139,v1:172.21.15.12:6801/3034215139] conn(0x7fbae006ef90 msgr2=0x7fbae0071410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.599+0000 7fbb0371d640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.12:6800/3034215139,v1:172.21.15.12:6801/3034215139] conn(0x7fbae006ef90 0x7fbae0071410 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.603+0000 7fbb0371d640 1 -- 172.21.15.12:0/463248313 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbae00664d0 msgr2=0x7fbae0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.603+0000 7fbb0371d640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbae00664d0 0x7fbae0068990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8007fd0 tx=0x7fbaf8007cd0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.603+0000 7fbb0371d640 1 -- 172.21.15.12:0/463248313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbafc14c1e0 msgr2=0x7fbafc1f2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.603+0000 7fbb0371d640 1 --2- 172.21.15.12:0/463248313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbafc14c1e0 0x7fbafc1f2220 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fbafc1551f0 tx=0x7fbaf400f580 comp rx=0 tx=0).stop 2023-10-30T12:56:18.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.603+0000 7fbb01cba640 1 -- 172.21.15.12:0/463248313 reap_dead start 2023-10-30T12:56:18.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.603+0000 7fbb0371d640 1 -- 172.21.15.12:0/463248313 shutdown_connections 2023-10-30T12:56:18.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.603+0000 7fbb0371d640 1 -- 172.21.15.12:0/463248313 >> 172.21.15.12:0/463248313 conn(0x7fbafc0c7910 msgr2=0x7fbafc14f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:18.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.603+0000 7fbb0371d640 1 -- 172.21.15.12:0/463248313 shutdown_connections 2023-10-30T12:56:18.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.603+0000 7fbb0371d640 1 -- 172.21.15.12:0/463248313 wait complete. 2023-10-30T12:56:18.624 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-30T12:56:18.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.763+0000 7fc924bc5640 1 Processor -- start 2023-10-30T12:56:18.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.763+0000 7fc924bc5640 1 -- start start 2023-10-30T12:56:18.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.767+0000 7fc924bc5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc920153c60 0x7fc9200ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.767+0000 7fc924bc5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9200cb780 con 0x7fc920159830 2023-10-30T12:56:18.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.767+0000 7fc924bc5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92015a2e0 con 0x7fc9200c4430 2023-10-30T12:56:18.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.767+0000 7fc924bc5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc920154040 con 0x7fc920153c60 2023-10-30T12:56:18.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.767+0000 7fc91ed9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc920159830 0x7fc9200cac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44438/0 (socket says 172.21.15.12:44438) 2023-10-30T12:56:18.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.767+0000 7fc91ed9d640 1 -- 172.21.15.12:0/1056937100 learned_addr learned my addr 172.21.15.12:0/1056937100 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:18.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.767+0000 7fc91e59c640 1 --2- 172.21.15.12:0/1056937100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc920153c60 0x7fc9200ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.767+0000 7fc91dd9b640 1 -- 172.21.15.12:0/1056937100 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823841451 0 0) 0x7fc9200cb780 con 0x7fc920159830 2023-10-30T12:56:18.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.767+0000 7fc91dd9b640 1 -- 172.21.15.12:0/1056937100 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc90c003530 con 0x7fc920159830 2023-10-30T12:56:18.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.767+0000 7fc91dd9b640 1 -- 172.21.15.12:0/1056937100 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653483729 0 0) 0x7fc92015a2e0 con 0x7fc9200c4430 2023-10-30T12:56:18.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.767+0000 7fc91dd9b640 1 -- 172.21.15.12:0/1056937100 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9200cb780 con 0x7fc9200c4430 2023-10-30T12:56:18.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.767+0000 7fc91e59c640 1 -- 172.21.15.12:0/1056937100 >> v1:172.21.15.154:6789/0 conn(0x7fc9200c4430 legacy=0x7fc9200cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.767+0000 7fc91dd9b640 1 -- 172.21.15.12:0/1056937100 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1999693798 0 0) 0x7fc90c003530 con 0x7fc920159830 2023-10-30T12:56:18.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.767+0000 7fc91e59c640 1 -- 172.21.15.12:0/1056937100 >> v1:172.21.15.12:6789/0 conn(0x7fc920159830 legacy=0x7fc9200cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.767+0000 7fc91e59c640 1 -- 172.21.15.12:0/1056937100 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92015a5c0 con 0x7fc920153c60 2023-10-30T12:56:18.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.771+0000 7fc91e59c640 1 --2- 172.21.15.12:0/1056937100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc920153c60 0x7fc9200ca360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc9100097f0 tx=0x7fc91001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6bc0c30ec9c57fa server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:18.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.771+0000 7fc91dd9b640 1 -- 172.21.15.12:0/1056937100 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918001a00 con 0x7fc920153c60 2023-10-30T12:56:18.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.771+0000 7fc91dd9b640 1 -- 172.21.15.12:0/1056937100 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9100a7050 con 0x7fc920153c60 2023-10-30T12:56:18.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.771+0000 7fc91dd9b640 1 -- 172.21.15.12:0/1056937100 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9100ab620 con 0x7fc920153c60 2023-10-30T12:56:18.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.771+0000 7fc924bc5640 1 -- 172.21.15.12:0/1056937100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc920153c60 msgr2=0x7fc9200ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.771+0000 7fc924bc5640 1 --2- 172.21.15.12:0/1056937100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc920153c60 0x7fc9200ca360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc9100097f0 tx=0x7fc91001c3e0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.771+0000 7fc924bc5640 1 -- 172.21.15.12:0/1056937100 shutdown_connections 2023-10-30T12:56:18.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.771+0000 7fc924bc5640 1 --2- 172.21.15.12:0/1056937100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc920153c60 0x7fc9200ca360 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.771+0000 7fc924bc5640 1 -- 172.21.15.12:0/1056937100 >> 172.21.15.12:0/1056937100 conn(0x7fc92006bfe0 msgr2=0x7fc92006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:18.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.771+0000 7fc924bc5640 1 -- 172.21.15.12:0/1056937100 shutdown_connections 2023-10-30T12:56:18.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.771+0000 7fc924bc5640 1 -- 172.21.15.12:0/1056937100 wait complete. 2023-10-30T12:56:18.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.771+0000 7fc924bc5640 1 Processor -- start 2023-10-30T12:56:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc924bc5640 1 -- start start 2023-10-30T12:56:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc924bc5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9200c4430 0x7fc92014b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc924bc5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc920153c60 0x7fc920156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc924bc5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc920159830 0x7fc9201f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc924bc5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9201f2dd0 con 0x7fc920153c60 2023-10-30T12:56:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc924bc5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9200c9540 con 0x7fc920159830 2023-10-30T12:56:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc924bc5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc920159050 con 0x7fc9200c4430 2023-10-30T12:56:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc917fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc920153c60 0x7fc920156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc917fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc920153c60 0x7fc920156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52344/0 (socket says 172.21.15.12:52344) 2023-10-30T12:56:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc917fff640 1 -- 172.21.15.12:0/3626726688 learned_addr learned my addr 172.21.15.12:0/3626726688 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc91ed9d640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc920159830 0x7fc9201f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc91e59c640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9200c4430 0x7fc92014b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc917fff640 1 -- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9200c4430 msgr2=0x7fc92014b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc917fff640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9200c4430 0x7fc92014b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc917fff640 1 -- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc920159830 msgr2=0x7fc9201f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc917fff640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc920159830 0x7fc9201f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc917fff640 1 -- 172.21.15.12:0/3626726688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc910009420 con 0x7fc920153c60 2023-10-30T12:56:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc91ed9d640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc920159830 0x7fc9201f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc91e59c640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9200c4430 0x7fc92014b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc917fff640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc920153c60 0x7fc920156100 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc90800c500 tx=0x7fc90800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc916ffd640 1 -- 172.21.15.12:0/3626726688 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9080052d0 con 0x7fc920153c60 2023-10-30T12:56:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc916ffd640 1 -- 172.21.15.12:0/3626726688 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc908005430 con 0x7fc920153c60 2023-10-30T12:56:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc916ffd640 1 -- 172.21.15.12:0/3626726688 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908012870 con 0x7fc920153c60 2023-10-30T12:56:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc924bc5640 1 -- 172.21.15.12:0/3626726688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9200c8f80 con 0x7fc920153c60 2023-10-30T12:56:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc924bc5640 1 -- 172.21.15.12:0/3626726688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92014aaa0 con 0x7fc920153c60 2023-10-30T12:56:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.775+0000 7fc924bc5640 1 -- 172.21.15.12:0/3626726688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92004e170 con 0x7fc920153c60 2023-10-30T12:56:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.779+0000 7fc916ffd640 1 -- 172.21.15.12:0/3626726688 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc9080129d0 con 0x7fc920153c60 2023-10-30T12:56:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.779+0000 7fc916ffd640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9000664a0 0x7fc900068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.779+0000 7fc916ffd640 1 -- 172.21.15.12:0/3626726688 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fc908083f70 con 0x7fc920153c60 2023-10-30T12:56:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.779+0000 7fc91e59c640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9000664a0 0x7fc900068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.779+0000 7fc91e59c640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9000664a0 0x7fc900068960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc9100ad000 tx=0x7fc9100a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.783+0000 7fc916ffd640 1 -- 172.21.15.12:0/3626726688 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc908050f90 con 0x7fc920153c60 2023-10-30T12:56:18.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.939+0000 7fc924bc5640 1 -- 172.21.15.12:0/3626726688 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9200cbf90 con 0x7fc9000664a0 2023-10-30T12:56:18.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.939+0000 7fc916ffd640 1 -- 172.21.15.12:0/3626726688 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc9100bc090 con 0x7fc9000664a0 2023-10-30T12:56:18.944 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 2 to deep-scrub 2023-10-30T12:56:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.943+0000 7fc924bc5640 1 -- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9000664a0 msgr2=0x7fc900068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.943+0000 7fc924bc5640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9000664a0 0x7fc900068960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc9100ad000 tx=0x7fc9100a6000 comp rx=0 tx=0).stop 2023-10-30T12:56:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.943+0000 7fc924bc5640 1 -- 172.21.15.12:0/3626726688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc920153c60 msgr2=0x7fc920156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.943+0000 7fc924bc5640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc920153c60 0x7fc920156100 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc90800c500 tx=0x7fc90800c9c0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.943+0000 7fc924bc5640 1 -- 172.21.15.12:0/3626726688 shutdown_connections 2023-10-30T12:56:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.943+0000 7fc924bc5640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc920159830 0x7fc9201f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.943+0000 7fc924bc5640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc920153c60 0x7fc920156100 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.943+0000 7fc924bc5640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fc9000664a0 0x7fc900068960 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.943+0000 7fc924bc5640 1 --2- 172.21.15.12:0/3626726688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9200c4430 0x7fc92014b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.943+0000 7fc924bc5640 1 -- 172.21.15.12:0/3626726688 >> 172.21.15.12:0/3626726688 conn(0x7fc92006bfe0 msgr2=0x7fc920152bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.943+0000 7fc924bc5640 1 -- 172.21.15.12:0/3626726688 shutdown_connections 2023-10-30T12:56:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:18.943+0000 7fc924bc5640 1 -- 172.21.15.12:0/3626726688 wait complete. 2023-10-30T12:56:18.965 INFO:tasks.ceph:Scrubbing osd.3 2023-10-30T12:56:18.966 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:56:19.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2f013e640 1 Processor -- start 2023-10-30T12:56:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2f013e640 1 -- start start 2023-10-30T12:56:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2f013e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2e8158070 0x7ff2e80c08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2f013e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff2e806a960 0x7ff2e815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2f013e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e8158450 con 0x7ff2e80c17f0 2023-10-30T12:56:19.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2f013e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e80c13d0 con 0x7ff2e806a960 2023-10-30T12:56:19.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2f013e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e815e1d0 con 0x7ff2e8158070 2023-10-30T12:56:19.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2ed6d9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff2e806a960 0x7ff2e815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2ee6db640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff2e80c17f0 0x7ff2e8069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44440/0 (socket says 172.21.15.12:44440) 2023-10-30T12:56:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2ee6db640 1 -- 172.21.15.12:0/2938414296 learned_addr learned my addr 172.21.15.12:0/2938414296 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2ededa640 1 --2- 172.21.15.12:0/2938414296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2e8158070 0x7ff2e80c08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2ed6d9640 1 -- 172.21.15.12:0/2938414296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2e8158070 msgr2=0x7ff2e80c08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2ed6d9640 1 --2- 172.21.15.12:0/2938414296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2e8158070 0x7ff2e80c08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2ed6d9640 1 -- 172.21.15.12:0/2938414296 >> v1:172.21.15.12:6789/0 conn(0x7ff2e80c17f0 legacy=0x7ff2e8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2ed6d9640 1 -- 172.21.15.12:0/2938414296 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e815e460 con 0x7ff2e806a960 2023-10-30T12:56:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.111+0000 7ff2eced8640 1 -- 172.21.15.12:0/2938414296 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930187715 0 0) 0x7ff2e8158450 con 0x7ff2e80c17f0 2023-10-30T12:56:19.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.115+0000 7ff2ed6d9640 1 --2- 172.21.15.12:0/2938414296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e806a960 0x7ff2e815dc90 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80097c0 tx=0x7ff2d801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc69773fee8506f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:19.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.115+0000 7ff2eced8640 1 -- 172.21.15.12:0/2938414296 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d80ab600 con 0x7ff2e806a960 2023-10-30T12:56:19.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.115+0000 7ff2eced8640 1 -- 172.21.15.12:0/2938414296 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d80a7050 con 0x7ff2e806a960 2023-10-30T12:56:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.115+0000 7ff2eced8640 1 -- 172.21.15.12:0/2938414296 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d80ab980 con 0x7ff2e806a960 2023-10-30T12:56:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.115+0000 7ff2f013e640 1 -- 172.21.15.12:0/2938414296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e806a960 msgr2=0x7ff2e815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.115+0000 7ff2f013e640 1 --2- 172.21.15.12:0/2938414296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e806a960 0x7ff2e815dc90 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80097c0 tx=0x7ff2d801c3f0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.115+0000 7ff2f013e640 1 -- 172.21.15.12:0/2938414296 shutdown_connections 2023-10-30T12:56:19.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.115+0000 7ff2f013e640 1 --2- 172.21.15.12:0/2938414296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e806a960 0x7ff2e815dc90 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.115+0000 7ff2f013e640 1 --2- 172.21.15.12:0/2938414296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2e8158070 0x7ff2e80c08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.115+0000 7ff2f013e640 1 -- 172.21.15.12:0/2938414296 >> 172.21.15.12:0/2938414296 conn(0x7ff2e80c7910 msgr2=0x7ff2e80c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:19.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.115+0000 7ff2f013e640 1 -- 172.21.15.12:0/2938414296 shutdown_connections 2023-10-30T12:56:19.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.115+0000 7ff2f013e640 1 -- 172.21.15.12:0/2938414296 wait complete. 2023-10-30T12:56:19.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2f013e640 1 Processor -- start 2023-10-30T12:56:19.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2f013e640 1 -- start start 2023-10-30T12:56:19.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2f013e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2e806a960 0x7ff2e814e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2f013e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e80c17f0 0x7ff2e81f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2f013e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2e8158070 0x7ff2e8149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2f013e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e814e800 con 0x7ff2e80c17f0 2023-10-30T12:56:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2f013e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e81f70d0 con 0x7ff2e806a960 2023-10-30T12:56:19.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2f013e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e81521e0 con 0x7ff2e8158070 2023-10-30T12:56:19.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2ed6d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e80c17f0 0x7ff2e81f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2ed6d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e80c17f0 0x7ff2e81f6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52364/0 (socket says 172.21.15.12:52364) 2023-10-30T12:56:19.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2ed6d9640 1 -- 172.21.15.12:0/2929239150 learned_addr learned my addr 172.21.15.12:0/2929239150 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:19.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2ededa640 1 --2- 172.21.15.12:0/2929239150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2e806a960 0x7ff2e814e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2ee6db640 1 --2- 172.21.15.12:0/2929239150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2e8158070 0x7ff2e8149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2ed6d9640 1 -- 172.21.15.12:0/2929239150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2e8158070 msgr2=0x7ff2e8149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2ed6d9640 1 --2- 172.21.15.12:0/2929239150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff2e8158070 0x7ff2e8149e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2ed6d9640 1 -- 172.21.15.12:0/2929239150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2e806a960 msgr2=0x7ff2e814e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2ed6d9640 1 --2- 172.21.15.12:0/2929239150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2e806a960 0x7ff2e814e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2ed6d9640 1 -- 172.21.15.12:0/2929239150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d8009420 con 0x7ff2e80c17f0 2023-10-30T12:56:19.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2ededa640 1 --2- 172.21.15.12:0/2929239150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2e806a960 0x7ff2e814e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:19.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2ed6d9640 1 --2- 172.21.15.12:0/2929239150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e80c17f0 0x7ff2e81f6b90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff2d801a020 tx=0x7ff2d8019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:19.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2de7fc640 1 -- 172.21.15.12:0/2929239150 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d80b4840 con 0x7ff2e80c17f0 2023-10-30T12:56:19.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2de7fc640 1 -- 172.21.15.12:0/2929239150 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d80b49c0 con 0x7ff2e80c17f0 2023-10-30T12:56:19.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2f013e640 1 -- 172.21.15.12:0/2929239150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e81524c0 con 0x7ff2e80c17f0 2023-10-30T12:56:19.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2de7fc640 1 -- 172.21.15.12:0/2929239150 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d80b4d70 con 0x7ff2e80c17f0 2023-10-30T12:56:19.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2f013e640 1 -- 172.21.15.12:0/2929239150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e8152a80 con 0x7ff2e80c17f0 2023-10-30T12:56:19.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.119+0000 7ff2f013e640 1 -- 172.21.15.12:0/2929239150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff2e80c0ff0 con 0x7ff2e80c17f0 2023-10-30T12:56:19.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.123+0000 7ff2de7fc640 1 -- 172.21.15.12:0/2929239150 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff2d80b2040 con 0x7ff2e80c17f0 2023-10-30T12:56:19.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.123+0000 7ff2de7fc640 1 --2- 172.21.15.12:0/2929239150 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2cc0664a0 0x7ff2cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.123+0000 7ff2de7fc640 1 -- 172.21.15.12:0/2929239150 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7ff2d811e4a0 con 0x7ff2e80c17f0 2023-10-30T12:56:19.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.123+0000 7ff2de7fc640 1 --2- 172.21.15.12:0/2929239150 >> [v2:172.21.15.12:6824/3921928345,v1:172.21.15.12:6825/3921928345] conn(0x7ff2cc06ef60 0x7ff2cc0713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.123+0000 7ff2de7fc640 1 -- 172.21.15.12:0/2929239150 --> [v2:172.21.15.12:6824/3921928345,v1:172.21.15.12:6825/3921928345] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2cc071ab0 con 0x7ff2cc06ef60 2023-10-30T12:56:19.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.123+0000 7ff2de7fc640 1 -- 172.21.15.12:0/2929239150 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2d8120a80 con 0x7ff2e80c17f0 2023-10-30T12:56:19.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.123+0000 7ff2ededa640 1 --2- 172.21.15.12:0/2929239150 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2cc0664a0 0x7ff2cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.123+0000 7ff2ee6db640 1 --2- 172.21.15.12:0/2929239150 >> [v2:172.21.15.12:6824/3921928345,v1:172.21.15.12:6825/3921928345] conn(0x7ff2cc06ef60 0x7ff2cc0713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.123+0000 7ff2ee6db640 1 --2- 172.21.15.12:0/2929239150 >> [v2:172.21.15.12:6824/3921928345,v1:172.21.15.12:6825/3921928345] conn(0x7ff2cc06ef60 0x7ff2cc0713e0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:19.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.123+0000 7ff2ededa640 1 --2- 172.21.15.12:0/2929239150 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2cc0664a0 0x7ff2cc068960 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400b9f0 tx=0x7ff2e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:19.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.123+0000 7ff2de7fc640 1 -- 172.21.15.12:0/2929239150 <== osd.3 v2:172.21.15.12:6824/3921928345 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7ff2cc071ab0 con 0x7ff2cc06ef60 2023-10-30T12:56:19.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.135+0000 7ff2f013e640 1 -- 172.21.15.12:0/2929239150 --> [v2:172.21.15.12:6824/3921928345,v1:172.21.15.12:6825/3921928345] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff2e806a5a0 con 0x7ff2cc06ef60 2023-10-30T12:56:19.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.139+0000 7ff2de7fc640 1 -- 172.21.15.12:0/2929239150 <== osd.3 v2:172.21.15.12:6824/3921928345 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff2e00075d0 con 0x7ff2cc06ef60 2023-10-30T12:56:19.142 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-30T12:56:19.142 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:56:19.142 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-30T12:56:19.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.139+0000 7ff2f013e640 1 -- 172.21.15.12:0/2929239150 >> [v2:172.21.15.12:6824/3921928345,v1:172.21.15.12:6825/3921928345] conn(0x7ff2cc06ef60 msgr2=0x7ff2cc0713e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.139+0000 7ff2f013e640 1 --2- 172.21.15.12:0/2929239150 >> [v2:172.21.15.12:6824/3921928345,v1:172.21.15.12:6825/3921928345] conn(0x7ff2cc06ef60 0x7ff2cc0713e0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.139+0000 7ff2f013e640 1 -- 172.21.15.12:0/2929239150 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2cc0664a0 msgr2=0x7ff2cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.139+0000 7ff2f013e640 1 --2- 172.21.15.12:0/2929239150 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff2cc0664a0 0x7ff2cc068960 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400b9f0 tx=0x7ff2e4008040 comp rx=0 tx=0).stop 2023-10-30T12:56:19.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.139+0000 7ff2f013e640 1 -- 172.21.15.12:0/2929239150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e80c17f0 msgr2=0x7ff2e81f6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.139+0000 7ff2f013e640 1 --2- 172.21.15.12:0/2929239150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e80c17f0 0x7ff2e81f6b90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff2d801a020 tx=0x7ff2d8019190 comp rx=0 tx=0).stop 2023-10-30T12:56:19.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.139+0000 7ff2ee6db640 1 -- 172.21.15.12:0/2929239150 reap_dead start 2023-10-30T12:56:19.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.139+0000 7ff2f013e640 1 -- 172.21.15.12:0/2929239150 shutdown_connections 2023-10-30T12:56:19.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.139+0000 7ff2f013e640 1 -- 172.21.15.12:0/2929239150 >> 172.21.15.12:0/2929239150 conn(0x7ff2e80c7910 msgr2=0x7ff2e80c9240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:19.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.139+0000 7ff2f013e640 1 -- 172.21.15.12:0/2929239150 shutdown_connections 2023-10-30T12:56:19.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.139+0000 7ff2f013e640 1 -- 172.21.15.12:0/2929239150 wait complete. 2023-10-30T12:56:19.160 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-30T12:56:19.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.307+0000 7f74e6f3e640 1 Processor -- start 2023-10-30T12:56:19.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.307+0000 7f74e6f3e640 1 -- start start 2023-10-30T12:56:19.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74e6f3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74e006ade0 0x7f74e00c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74e6f3e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e006a4e0 con 0x7f74e0151910 2023-10-30T12:56:19.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74e6f3e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e00c6c50 con 0x7f74e006a970 2023-10-30T12:56:19.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74e6f3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00c6f30 con 0x7f74e006ade0 2023-10-30T12:56:19.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74e4cda640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f74e0151910 0x7f74e0158120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44448/0 (socket says 172.21.15.12:44448) 2023-10-30T12:56:19.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74e4cda640 1 -- 172.21.15.12:0/4071341412 learned_addr learned my addr 172.21.15.12:0/4071341412 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:19.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74e54db640 1 --2- 172.21.15.12:0/4071341412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74e006ade0 0x7f74e00c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045673597 0 0) 0x7f74e006a4e0 con 0x7f74e0151910 2023-10-30T12:56:19.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74d0003530 con 0x7f74e0151910 2023-10-30T12:56:19.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048060819 0 0) 0x7f74e00c6c50 con 0x7f74e006a970 2023-10-30T12:56:19.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74e006a4e0 con 0x7f74e006a970 2023-10-30T12:56:19.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3324976651 0 0) 0x7f74d0003530 con 0x7f74e0151910 2023-10-30T12:56:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74e006ade0 msgr2=0x7f74e00c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 --2- 172.21.15.12:0/4071341412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74e006ade0 0x7f74e00c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 >> v1:172.21.15.154:6789/0 conn(0x7f74e006a970 legacy=0x7f74e00c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e00c7210 con 0x7f74e0151910 2023-10-30T12:56:19.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f74d4001680 con 0x7f74e0151910 2023-10-30T12:56:19.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 >> v1:172.21.15.12:6789/0 conn(0x7f74e0151910 legacy=0x7f74e0158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 --2- 172.21.15.12:0/4071341412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74d0003b40 0x7f74d0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 --2- 172.21.15.12:0/4071341412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74d00047d0 0x7f74d0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 --2- 172.21.15.12:0/4071341412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74d0005150 0x7f74d000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00c6f30 con 0x7f74d0005150 2023-10-30T12:56:19.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d000bed0 con 0x7f74d0003b40 2023-10-30T12:56:19.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d000c010 con 0x7f74d00047d0 2023-10-30T12:56:19.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74e54db640 1 --2- 172.21.15.12:0/4071341412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74d0005150 0x7f74d000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74e4cda640 1 --2- 172.21.15.12:0/4071341412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74d0003b40 0x7f74d0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74dffff640 1 --2- 172.21.15.12:0/4071341412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74d00047d0 0x7f74d0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74e54db640 1 -- 172.21.15.12:0/4071341412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74d00047d0 msgr2=0x7f74d0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:19.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74e54db640 1 --2- 172.21.15.12:0/4071341412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74d00047d0 0x7f74d0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74e54db640 1 -- 172.21.15.12:0/4071341412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74d0003b40 msgr2=0x7f74d0003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:19.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74e54db640 1 --2- 172.21.15.12:0/4071341412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74d0003b40 0x7f74d0003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74e54db640 1 -- 172.21.15.12:0/4071341412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d0003530 con 0x7f74d0005150 2023-10-30T12:56:19.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74dffff640 1 --2- 172.21.15.12:0/4071341412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74d00047d0 0x7f74d0004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:19.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.311+0000 7f74e4cda640 1 --2- 172.21.15.12:0/4071341412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74d0003b40 0x7f74d0003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:19.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.315+0000 7f74e54db640 1 --2- 172.21.15.12:0/4071341412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74d0005150 0x7f74d000b990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009f90 tx=0x7f74d801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6199d8caf069f664 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:19.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.315+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d80a5c30 con 0x7f74d0005150 2023-10-30T12:56:19.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.315+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d800f040 con 0x7f74d0005150 2023-10-30T12:56:19.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.315+0000 7f74df7fe640 1 -- 172.21.15.12:0/4071341412 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d80ad440 con 0x7f74d0005150 2023-10-30T12:56:19.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.315+0000 7f74e6f3e640 1 -- 172.21.15.12:0/4071341412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74d0005150 msgr2=0x7f74d000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.315+0000 7f74e6f3e640 1 --2- 172.21.15.12:0/4071341412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74d0005150 0x7f74d000b990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009f90 tx=0x7f74d801cc20 comp rx=0 tx=0).stop 2023-10-30T12:56:19.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.315+0000 7f74e54db640 1 -- 172.21.15.12:0/4071341412 reap_dead start 2023-10-30T12:56:19.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.315+0000 7f74e6f3e640 1 -- 172.21.15.12:0/4071341412 shutdown_connections 2023-10-30T12:56:19.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.315+0000 7f74e6f3e640 1 -- 172.21.15.12:0/4071341412 >> 172.21.15.12:0/4071341412 conn(0x7f74e00c7910 msgr2=0x7f74e00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:19.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.315+0000 7f74e6f3e640 1 -- 172.21.15.12:0/4071341412 shutdown_connections 2023-10-30T12:56:19.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.315+0000 7f74e6f3e640 1 -- 172.21.15.12:0/4071341412 wait complete. 2023-10-30T12:56:19.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e6f3e640 1 Processor -- start 2023-10-30T12:56:19.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e6f3e640 1 -- start start 2023-10-30T12:56:19.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e6f3e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74d0005150 0x7f74e01442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e6f3e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74e0142b50 0x7f74e0147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e6f3e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74e01482a0 0x7f74e01ecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e6f3e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e0156c90 con 0x7f74e01482a0 2023-10-30T12:56:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e6f3e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e0158290 con 0x7f74d0005150 2023-10-30T12:56:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e6f3e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e0158530 con 0x7f74e0142b50 2023-10-30T12:56:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e54db640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74e01482a0 0x7f74e01ecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e54db640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74e01482a0 0x7f74e01ecfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52386/0 (socket says 172.21.15.12:52386) 2023-10-30T12:56:19.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e54db640 1 -- 172.21.15.12:0/2915896755 learned_addr learned my addr 172.21.15.12:0/2915896755 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:19.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e4cda640 1 --2- 172.21.15.12:0/2915896755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74d0005150 0x7f74e01442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74dffff640 1 --2- 172.21.15.12:0/2915896755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74e0142b50 0x7f74e0147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e54db640 1 -- 172.21.15.12:0/2915896755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74e0142b50 msgr2=0x7f74e0147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e54db640 1 --2- 172.21.15.12:0/2915896755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74e0142b50 0x7f74e0147d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e54db640 1 -- 172.21.15.12:0/2915896755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74d0005150 msgr2=0x7f74e01442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e54db640 1 --2- 172.21.15.12:0/2915896755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74d0005150 0x7f74e01442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e54db640 1 -- 172.21.15.12:0/2915896755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e0155af0 con 0x7f74e01482a0 2023-10-30T12:56:19.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74dffff640 1 --2- 172.21.15.12:0/2915896755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74e0142b50 0x7f74e0147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:19.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e54db640 1 --2- 172.21.15.12:0/2915896755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74e01482a0 0x7f74e01ecfe0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f74d80b0040 tx=0x7f74d8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:19.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74dd7fa640 1 -- 172.21.15.12:0/2915896755 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d80ae890 con 0x7f74e01482a0 2023-10-30T12:56:19.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74dd7fa640 1 -- 172.21.15.12:0/2915896755 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d80ae9f0 con 0x7f74e01482a0 2023-10-30T12:56:19.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74dd7fa640 1 -- 172.21.15.12:0/2915896755 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d80aed80 con 0x7f74e01482a0 2023-10-30T12:56:19.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e6f3e640 1 -- 172.21.15.12:0/2915896755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e0156640 con 0x7f74e01482a0 2023-10-30T12:56:19.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e6f3e640 1 -- 172.21.15.12:0/2915896755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e014fd00 con 0x7f74e01482a0 2023-10-30T12:56:19.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.319+0000 7f74e6f3e640 1 -- 172.21.15.12:0/2915896755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e01564a0 con 0x7f74e01482a0 2023-10-30T12:56:19.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.323+0000 7f74dd7fa640 1 -- 172.21.15.12:0/2915896755 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f74d80b4040 con 0x7f74e01482a0 2023-10-30T12:56:19.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.323+0000 7f74dd7fa640 1 --2- 172.21.15.12:0/2915896755 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f74c40664a0 0x7f74c4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.323+0000 7f74dd7fa640 1 -- 172.21.15.12:0/2915896755 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f74d811eb40 con 0x7f74e01482a0 2023-10-30T12:56:19.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.323+0000 7f74e4cda640 1 --2- 172.21.15.12:0/2915896755 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f74c40664a0 0x7f74c4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.323+0000 7f74e4cda640 1 --2- 172.21.15.12:0/2915896755 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f74c40664a0 0x7f74c4068960 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f74d4000c10 tx=0x7f74d4007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.327+0000 7f74dd7fa640 1 -- 172.21.15.12:0/2915896755 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f74e01564a0 con 0x7f74e01482a0 2023-10-30T12:56:19.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.483+0000 7f74e6f3e640 1 -- 172.21.15.12:0/2915896755 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f74e006a170 con 0x7f74c40664a0 2023-10-30T12:56:19.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.483+0000 7f74dd7fa640 1 -- 172.21.15.12:0/2915896755 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f74d4011a40 con 0x7f74c40664a0 2023-10-30T12:56:19.489 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 3 to deep-scrub 2023-10-30T12:56:19.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.487+0000 7f74e6f3e640 1 -- 172.21.15.12:0/2915896755 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f74c40664a0 msgr2=0x7f74c4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.487+0000 7f74e6f3e640 1 --2- 172.21.15.12:0/2915896755 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f74c40664a0 0x7f74c4068960 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f74d4000c10 tx=0x7f74d4007040 comp rx=0 tx=0).stop 2023-10-30T12:56:19.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.487+0000 7f74e6f3e640 1 -- 172.21.15.12:0/2915896755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74e01482a0 msgr2=0x7f74e01ecfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.487+0000 7f74e6f3e640 1 --2- 172.21.15.12:0/2915896755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74e01482a0 0x7f74e01ecfe0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f74d80b0040 tx=0x7f74d8009750 comp rx=0 tx=0).stop 2023-10-30T12:56:19.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.487+0000 7f74e6f3e640 1 -- 172.21.15.12:0/2915896755 shutdown_connections 2023-10-30T12:56:19.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.487+0000 7f74e6f3e640 1 --2- 172.21.15.12:0/2915896755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74e01482a0 0x7f74e01ecfe0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.487+0000 7f74e6f3e640 1 --2- 172.21.15.12:0/2915896755 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f74c40664a0 0x7f74c4068960 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.487+0000 7f74e6f3e640 1 --2- 172.21.15.12:0/2915896755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74e0142b50 0x7f74e0147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.487+0000 7f74e6f3e640 1 --2- 172.21.15.12:0/2915896755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74d0005150 0x7f74e01442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.487+0000 7f74e6f3e640 1 -- 172.21.15.12:0/2915896755 >> 172.21.15.12:0/2915896755 conn(0x7f74e00c7910 msgr2=0x7f74e006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:19.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.487+0000 7f74e6f3e640 1 -- 172.21.15.12:0/2915896755 shutdown_connections 2023-10-30T12:56:19.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.487+0000 7f74e6f3e640 1 -- 172.21.15.12:0/2915896755 wait complete. 2023-10-30T12:56:19.511 INFO:tasks.ceph:Scrubbing osd.4 2023-10-30T12:56:19.512 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:56:19.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.687+0000 7f4e64780640 1 Processor -- start 2023-10-30T12:56:19.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.687+0000 7f4e64780640 1 -- start start 2023-10-30T12:56:19.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.687+0000 7f4e64780640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e5c15d9d0 0x7f4e5c0cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.687+0000 7f4e64780640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4e5c0c4000 0x7f4e5c0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.687+0000 7f4e64780640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e5c15a210 con 0x7f4e5c1536d0 2023-10-30T12:56:19.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.687+0000 7f4e64780640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e5c15a4f0 con 0x7f4e5c0c4000 2023-10-30T12:56:19.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.687+0000 7f4e64780640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c15cef0 con 0x7f4e5c15d9d0 2023-10-30T12:56:19.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.687+0000 7f4e62d1d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4e5c0c4000 0x7f4e5c0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.687+0000 7f4e61d1b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e5c15d9d0 0x7f4e5c0cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.687+0000 7f4e62d1d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4e5c0c4000 0x7f4e5c0cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52392/0 (socket says 172.21.15.12:52392) 2023-10-30T12:56:19.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.687+0000 7f4e62d1d640 1 -- 172.21.15.12:0/682720550 learned_addr learned my addr 172.21.15.12:0/682720550 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:19.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.687+0000 7f4e6251c640 1 --1- 172.21.15.12:0/682720550 >> v1:172.21.15.12:6789/0 conn(0x7f4e5c1536d0 0x7f4e5c159760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44450/0 (socket says 172.21.15.12:44450) 2023-10-30T12:56:19.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.691+0000 7f4e62d1d640 1 -- 172.21.15.12:0/682720550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e5c15d9d0 msgr2=0x7f4e5c0cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:19.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.691+0000 7f4e62d1d640 1 --2- 172.21.15.12:0/682720550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e5c15d9d0 0x7f4e5c0cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.691+0000 7f4e62d1d640 1 -- 172.21.15.12:0/682720550 >> v1:172.21.15.12:6789/0 conn(0x7f4e5c1536d0 legacy=0x7f4e5c159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.691+0000 7f4e62d1d640 1 -- 172.21.15.12:0/682720550 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c15d1d0 con 0x7f4e5c0c4000 2023-10-30T12:56:19.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.691+0000 7f4e62d1d640 1 --2- 172.21.15.12:0/682720550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e5c0c4000 0x7f4e5c0cf090 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009e50 tx=0x7f4e5401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d99a2b02cbebfc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:19.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.691+0000 7f4e6151a640 1 -- 172.21.15.12:0/682720550 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e540a56e0 con 0x7f4e5c0c4000 2023-10-30T12:56:19.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.691+0000 7f4e6151a640 1 -- 172.21.15.12:0/682720550 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e540a5840 con 0x7f4e5c0c4000 2023-10-30T12:56:19.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.691+0000 7f4e6151a640 1 -- 172.21.15.12:0/682720550 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e540a5bd0 con 0x7f4e5c0c4000 2023-10-30T12:56:19.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 -- 172.21.15.12:0/682720550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e5c0c4000 msgr2=0x7f4e5c0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 --2- 172.21.15.12:0/682720550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e5c0c4000 0x7f4e5c0cf090 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009e50 tx=0x7f4e5401ca10 comp rx=0 tx=0).stop 2023-10-30T12:56:19.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 -- 172.21.15.12:0/682720550 shutdown_connections 2023-10-30T12:56:19.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 --2- 172.21.15.12:0/682720550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e5c0c4000 0x7f4e5c0cf090 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 --2- 172.21.15.12:0/682720550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e5c15d9d0 0x7f4e5c0cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 -- 172.21.15.12:0/682720550 >> 172.21.15.12:0/682720550 conn(0x7f4e5c06bf70 msgr2=0x7f4e5c06c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:19.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 -- 172.21.15.12:0/682720550 shutdown_connections 2023-10-30T12:56:19.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 -- 172.21.15.12:0/682720550 wait complete. 2023-10-30T12:56:19.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 Processor -- start 2023-10-30T12:56:19.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 -- start start 2023-10-30T12:56:19.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c0c4000 0x7f4e5c1f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e5c1536d0 0x7f4e5c14ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e5c15d9d0 0x7f4e5c148b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c15c7e0 con 0x7f4e5c1536d0 2023-10-30T12:56:19.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c14e230 con 0x7f4e5c0c4000 2023-10-30T12:56:19.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.695+0000 7f4e64780640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c14dc70 con 0x7f4e5c15d9d0 2023-10-30T12:56:19.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e6251c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c0c4000 0x7f4e5c1f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e61d1b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e5c1536d0 0x7f4e5c14ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e6251c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c0c4000 0x7f4e5c1f1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:50342/0 (socket says 172.21.15.12:50342) 2023-10-30T12:56:19.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e61d1b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e5c1536d0 0x7f4e5c14ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52398/0 (socket says 172.21.15.12:52398) 2023-10-30T12:56:19.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e6251c640 1 -- 172.21.15.12:0/1852597721 learned_addr learned my addr 172.21.15.12:0/1852597721 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:19.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e62d1d640 1 --2- 172.21.15.12:0/1852597721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e5c15d9d0 0x7f4e5c148b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e61d1b640 1 -- 172.21.15.12:0/1852597721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e5c15d9d0 msgr2=0x7f4e5c148b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e61d1b640 1 --2- 172.21.15.12:0/1852597721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e5c15d9d0 0x7f4e5c148b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e61d1b640 1 -- 172.21.15.12:0/1852597721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c0c4000 msgr2=0x7f4e5c1f1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e61d1b640 1 --2- 172.21.15.12:0/1852597721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c0c4000 0x7f4e5c1f1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e61d1b640 1 -- 172.21.15.12:0/1852597721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e54009b30 con 0x7f4e5c1536d0 2023-10-30T12:56:19.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e61d1b640 1 --2- 172.21.15.12:0/1852597721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e5c1536d0 0x7f4e5c14ef20 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00e870 tx=0x7f4e4c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:19.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e52ffd640 1 -- 172.21.15.12:0/1852597721 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c0095d0 con 0x7f4e5c1536d0 2023-10-30T12:56:19.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e52ffd640 1 -- 172.21.15.12:0/1852597721 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4c00c940 con 0x7f4e5c1536d0 2023-10-30T12:56:19.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e64780640 1 -- 172.21.15.12:0/1852597721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5c14df50 con 0x7f4e5c1536d0 2023-10-30T12:56:19.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e52ffd640 1 -- 172.21.15.12:0/1852597721 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c00ccb0 con 0x7f4e5c1536d0 2023-10-30T12:56:19.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e64780640 1 -- 172.21.15.12:0/1852597721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5c1f58b0 con 0x7f4e5c1536d0 2023-10-30T12:56:19.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.699+0000 7f4e64780640 1 -- 172.21.15.12:0/1852597721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4e5c153ab0 con 0x7f4e5c1536d0 2023-10-30T12:56:19.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.703+0000 7f4e52ffd640 1 -- 172.21.15.12:0/1852597721 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4e4c013470 con 0x7f4e5c1536d0 2023-10-30T12:56:19.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.703+0000 7f4e52ffd640 1 --2- 172.21.15.12:0/1852597721 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4e400664a0 0x7f4e40068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.703+0000 7f4e52ffd640 1 -- 172.21.15.12:0/1852597721 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f4e4c0844d0 con 0x7f4e5c1536d0 2023-10-30T12:56:19.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.703+0000 7f4e52ffd640 1 --2- 172.21.15.12:0/1852597721 >> [v2:172.21.15.154:6808/3824067880,v1:172.21.15.154:6809/3824067880] conn(0x7f4e4006ef60 0x7f4e400713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.703+0000 7f4e52ffd640 1 -- 172.21.15.12:0/1852597721 --> [v2:172.21.15.154:6808/3824067880,v1:172.21.15.154:6809/3824067880] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e40071ab0 con 0x7f4e4006ef60 2023-10-30T12:56:19.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.703+0000 7f4e52ffd640 1 -- 172.21.15.12:0/1852597721 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e4c086ab0 con 0x7f4e5c1536d0 2023-10-30T12:56:19.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.703+0000 7f4e6251c640 1 --2- 172.21.15.12:0/1852597721 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4e400664a0 0x7f4e40068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.703+0000 7f4e62d1d640 1 --2- 172.21.15.12:0/1852597721 >> [v2:172.21.15.154:6808/3824067880,v1:172.21.15.154:6809/3824067880] conn(0x7f4e4006ef60 0x7f4e400713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.703+0000 7f4e6251c640 1 --2- 172.21.15.12:0/1852597721 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4e400664a0 0x7f4e40068960 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4e58000f70 tx=0x7f4e58007f10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:19.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.703+0000 7f4e62d1d640 1 --2- 172.21.15.12:0/1852597721 >> [v2:172.21.15.154:6808/3824067880,v1:172.21.15.154:6809/3824067880] conn(0x7f4e4006ef60 0x7f4e400713e0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:19.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.707+0000 7f4e52ffd640 1 -- 172.21.15.12:0/1852597721 <== osd.4 v2:172.21.15.154:6808/3824067880 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f4e40071ab0 con 0x7f4e4006ef60 2023-10-30T12:56:19.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.719+0000 7f4e64780640 1 -- 172.21.15.12:0/1852597721 --> [v2:172.21.15.154:6808/3824067880,v1:172.21.15.154:6809/3824067880] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4e5c1f5430 con 0x7f4e4006ef60 2023-10-30T12:56:19.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.719+0000 7f4e52ffd640 1 -- 172.21.15.12:0/1852597721 <== osd.4 v2:172.21.15.154:6808/3824067880 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4e5c1f5430 con 0x7f4e4006ef60 2023-10-30T12:56:19.725 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-30T12:56:19.726 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:56:19.726 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-30T12:56:19.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.719+0000 7f4e64780640 1 -- 172.21.15.12:0/1852597721 >> [v2:172.21.15.154:6808/3824067880,v1:172.21.15.154:6809/3824067880] conn(0x7f4e4006ef60 msgr2=0x7f4e400713e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.719+0000 7f4e64780640 1 --2- 172.21.15.12:0/1852597721 >> [v2:172.21.15.154:6808/3824067880,v1:172.21.15.154:6809/3824067880] conn(0x7f4e4006ef60 0x7f4e400713e0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.723+0000 7f4e64780640 1 -- 172.21.15.12:0/1852597721 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4e400664a0 msgr2=0x7f4e40068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.723+0000 7f4e64780640 1 --2- 172.21.15.12:0/1852597721 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4e400664a0 0x7f4e40068960 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4e58000f70 tx=0x7f4e58007f10 comp rx=0 tx=0).stop 2023-10-30T12:56:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.723+0000 7f4e64780640 1 -- 172.21.15.12:0/1852597721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e5c1536d0 msgr2=0x7f4e5c14ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.723+0000 7f4e64780640 1 --2- 172.21.15.12:0/1852597721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e5c1536d0 0x7f4e5c14ef20 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00e870 tx=0x7f4e4c00ed30 comp rx=0 tx=0).stop 2023-10-30T12:56:19.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.723+0000 7f4e62d1d640 1 -- 172.21.15.12:0/1852597721 reap_dead start 2023-10-30T12:56:19.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.723+0000 7f4e64780640 1 -- 172.21.15.12:0/1852597721 shutdown_connections 2023-10-30T12:56:19.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.723+0000 7f4e64780640 1 -- 172.21.15.12:0/1852597721 >> 172.21.15.12:0/1852597721 conn(0x7f4e5c06bf70 msgr2=0x7f4e5c14cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:19.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.723+0000 7f4e64780640 1 -- 172.21.15.12:0/1852597721 shutdown_connections 2023-10-30T12:56:19.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.723+0000 7f4e64780640 1 -- 172.21.15.12:0/1852597721 wait complete. 2023-10-30T12:56:19.753 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-30T12:56:19.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.911+0000 7f3f7d668640 1 Processor -- start 2023-10-30T12:56:19.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.911+0000 7f3f7d668640 1 -- start start 2023-10-30T12:56:19.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f7d668640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f780ca360 0x7f3f780cac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f7d668640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f780cb780 con 0x7f3f780c4430 2023-10-30T12:56:19.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f7d668640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f780ca810 con 0x7f3f78153c60 2023-10-30T12:56:19.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f7d668640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f78154040 con 0x7f3f780ca360 2023-10-30T12:56:19.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f767fc640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3f780c4430 0x7f3f78159830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44456/0 (socket says 172.21.15.12:44456) 2023-10-30T12:56:19.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f767fc640 1 -- 172.21.15.12:0/3938845672 learned_addr learned my addr 172.21.15.12:0/3938845672 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:19.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f777fe640 1 --2- 172.21.15.12:0/3938845672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f780ca360 0x7f3f780cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f75ffb640 1 -- 172.21.15.12:0/3938845672 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15084120 0 0) 0x7f3f780cb780 con 0x7f3f780c4430 2023-10-30T12:56:19.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f75ffb640 1 -- 172.21.15.12:0/3938845672 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f60003530 con 0x7f3f780c4430 2023-10-30T12:56:19.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f75ffb640 1 -- 172.21.15.12:0/3938845672 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 413423610 0 0) 0x7f3f60003530 con 0x7f3f780c4430 2023-10-30T12:56:19.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f75ffb640 1 -- 172.21.15.12:0/3938845672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f780ca360 msgr2=0x7f3f780cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:19.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f75ffb640 1 --2- 172.21.15.12:0/3938845672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f780ca360 0x7f3f780cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f75ffb640 1 -- 172.21.15.12:0/3938845672 >> v1:172.21.15.154:6789/0 conn(0x7f3f78153c60 legacy=0x7f3f780cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f75ffb640 1 -- 172.21.15.12:0/3938845672 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7815a2e0 con 0x7f3f780c4430 2023-10-30T12:56:19.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f75ffb640 1 -- 172.21.15.12:0/3938845672 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f3f6c001680 con 0x7f3f780c4430 2023-10-30T12:56:19.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f75ffb640 1 -- 172.21.15.12:0/3938845672 >> v1:172.21.15.12:6789/0 conn(0x7f3f780c4430 legacy=0x7f3f78159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f75ffb640 1 --2- 172.21.15.12:0/3938845672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f600039b0 0x7f3f60003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f75ffb640 1 --2- 172.21.15.12:0/3938845672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f60004360 0x7f3f60004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f75ffb640 1 --2- 172.21.15.12:0/3938845672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f780ca360 0x7f3f60005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f75ffb640 1 -- 172.21.15.12:0/3938845672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f78154040 con 0x7f3f60004360 2023-10-30T12:56:19.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f75ffb640 1 -- 172.21.15.12:0/3938845672 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6000b990 con 0x7f3f600039b0 2023-10-30T12:56:19.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f75ffb640 1 -- 172.21.15.12:0/3938845672 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6000bb50 con 0x7f3f780ca360 2023-10-30T12:56:19.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f767fc640 1 --2- 172.21.15.12:0/3938845672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f60004360 0x7f3f60004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f76ffd640 1 --2- 172.21.15.12:0/3938845672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f600039b0 0x7f3f60003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f777fe640 1 --2- 172.21.15.12:0/3938845672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f780ca360 0x7f3f60005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f767fc640 1 -- 172.21.15.12:0/3938845672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f780ca360 msgr2=0x7f3f60005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:19.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f767fc640 1 --2- 172.21.15.12:0/3938845672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f780ca360 0x7f3f60005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f767fc640 1 -- 172.21.15.12:0/3938845672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f600039b0 msgr2=0x7f3f60003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:19.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f767fc640 1 --2- 172.21.15.12:0/3938845672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f600039b0 0x7f3f60003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f767fc640 1 -- 172.21.15.12:0/3938845672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f60003530 con 0x7f3f60004360 2023-10-30T12:56:19.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f76ffd640 1 --2- 172.21.15.12:0/3938845672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f600039b0 0x7f3f60003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:19.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.915+0000 7f3f777fe640 1 --2- 172.21.15.12:0/3938845672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f780ca360 0x7f3f60005090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:19.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.919+0000 7f3f767fc640 1 --2- 172.21.15.12:0/3938845672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f60004360 0x7f3f60004830 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3f780c4850 tx=0x7f3f6c01cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c8dbc32285a8d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:19.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.919+0000 7f3f75ffb640 1 -- 172.21.15.12:0/3938845672 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0a5b00 con 0x7f3f60004360 2023-10-30T12:56:19.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.919+0000 7f3f75ffb640 1 -- 172.21.15.12:0/3938845672 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0a5c80 con 0x7f3f60004360 2023-10-30T12:56:19.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.919+0000 7f3f75ffb640 1 -- 172.21.15.12:0/3938845672 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0058e0 con 0x7f3f60004360 2023-10-30T12:56:19.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.919+0000 7f3f7d668640 1 -- 172.21.15.12:0/3938845672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f60004360 msgr2=0x7f3f60004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.919+0000 7f3f7d668640 1 --2- 172.21.15.12:0/3938845672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f60004360 0x7f3f60004830 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3f780c4850 tx=0x7f3f6c01cc00 comp rx=0 tx=0).stop 2023-10-30T12:56:19.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.919+0000 7f3f777fe640 1 -- 172.21.15.12:0/3938845672 reap_dead start 2023-10-30T12:56:19.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.919+0000 7f3f7d668640 1 -- 172.21.15.12:0/3938845672 shutdown_connections 2023-10-30T12:56:19.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.919+0000 7f3f7d668640 1 -- 172.21.15.12:0/3938845672 >> 172.21.15.12:0/3938845672 conn(0x7f3f7806bfe0 msgr2=0x7f3f7806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:19.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.919+0000 7f3f7d668640 1 -- 172.21.15.12:0/3938845672 shutdown_connections 2023-10-30T12:56:19.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.919+0000 7f3f7d668640 1 -- 172.21.15.12:0/3938845672 wait complete. 2023-10-30T12:56:19.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.919+0000 7f3f7d668640 1 Processor -- start 2023-10-30T12:56:19.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f7d668640 1 -- start start 2023-10-30T12:56:19.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f7d668640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f60004360 0x7f3f781f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f7d668640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f780c92e0 0x7f3f78150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f7d668640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f781ee5b0 0x7f3f780c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f7d668640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f781f1fc0 con 0x7f3f780c92e0 2023-10-30T12:56:19.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f7d668640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f781581e0 con 0x7f3f781ee5b0 2023-10-30T12:56:19.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f7d668640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f78158d20 con 0x7f3f60004360 2023-10-30T12:56:19.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f767fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f780c92e0 0x7f3f78150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f767fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f780c92e0 0x7f3f78150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52408/0 (socket says 172.21.15.12:52408) 2023-10-30T12:56:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f767fc640 1 -- 172.21.15.12:0/286152766 learned_addr learned my addr 172.21.15.12:0/286152766 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f76ffd640 1 --2- 172.21.15.12:0/286152766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f60004360 0x7f3f781f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f777fe640 1 --2- 172.21.15.12:0/286152766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f781ee5b0 0x7f3f780c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f767fc640 1 -- 172.21.15.12:0/286152766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f60004360 msgr2=0x7f3f781f2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f767fc640 1 --2- 172.21.15.12:0/286152766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f60004360 0x7f3f781f2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f767fc640 1 -- 172.21.15.12:0/286152766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f781ee5b0 msgr2=0x7f3f780c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:19.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f767fc640 1 --2- 172.21.15.12:0/286152766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f781ee5b0 0x7f3f780c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:19.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f767fc640 1 -- 172.21.15.12:0/286152766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f78158480 con 0x7f3f780c92e0 2023-10-30T12:56:19.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f767fc640 1 --2- 172.21.15.12:0/286152766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f780c92e0 0x7f3f78150690 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c098620 tx=0x7f3f60005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:19.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f5f7fe640 1 -- 172.21.15.12:0/286152766 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0b4070 con 0x7f3f780c92e0 2023-10-30T12:56:19.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f5f7fe640 1 -- 172.21.15.12:0/286152766 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0a5b10 con 0x7f3f780c92e0 2023-10-30T12:56:19.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f5f7fe640 1 -- 172.21.15.12:0/286152766 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0049f0 con 0x7f3f780c92e0 2023-10-30T12:56:19.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f7d668640 1 -- 172.21.15.12:0/286152766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f78158760 con 0x7f3f780c92e0 2023-10-30T12:56:19.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f7d668640 1 -- 172.21.15.12:0/286152766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f781f1c50 con 0x7f3f780c92e0 2023-10-30T12:56:19.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.923+0000 7f3f7d668640 1 -- 172.21.15.12:0/286152766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f781f1e10 con 0x7f3f780c92e0 2023-10-30T12:56:19.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.927+0000 7f3f5f7fe640 1 -- 172.21.15.12:0/286152766 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3f6c004b50 con 0x7f3f780c92e0 2023-10-30T12:56:19.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.927+0000 7f3f5f7fe640 1 --2- 172.21.15.12:0/286152766 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f640664a0 0x7f3f64068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:19.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.927+0000 7f3f5f7fe640 1 -- 172.21.15.12:0/286152766 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f3f6c11db60 con 0x7f3f780c92e0 2023-10-30T12:56:19.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.927+0000 7f3f76ffd640 1 --2- 172.21.15.12:0/286152766 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f640664a0 0x7f3f64068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:19.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.927+0000 7f3f76ffd640 1 --2- 172.21.15.12:0/286152766 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f640664a0 0x7f3f64068960 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3f68007bd0 tx=0x7f3f68007b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:19.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:19.931+0000 7f3f5f7fe640 1 -- 172.21.15.12:0/286152766 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3f6c0eab70 con 0x7f3f780c92e0 2023-10-30T12:56:20.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.091+0000 7f3f7d668640 1 -- 172.21.15.12:0/286152766 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f780cb9d0 con 0x7f3f640664a0 2023-10-30T12:56:20.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.091+0000 7f3f5f7fe640 1 -- 172.21.15.12:0/286152766 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3f6800d040 con 0x7f3f640664a0 2023-10-30T12:56:20.096 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 4 to deep-scrub 2023-10-30T12:56:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.095+0000 7f3f7d668640 1 -- 172.21.15.12:0/286152766 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f640664a0 msgr2=0x7f3f64068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.095+0000 7f3f7d668640 1 --2- 172.21.15.12:0/286152766 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f640664a0 0x7f3f64068960 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3f68007bd0 tx=0x7f3f68007b20 comp rx=0 tx=0).stop 2023-10-30T12:56:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.095+0000 7f3f7d668640 1 -- 172.21.15.12:0/286152766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f780c92e0 msgr2=0x7f3f78150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.095+0000 7f3f7d668640 1 --2- 172.21.15.12:0/286152766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f780c92e0 0x7f3f78150690 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c098620 tx=0x7f3f60005800 comp rx=0 tx=0).stop 2023-10-30T12:56:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.095+0000 7f3f7d668640 1 -- 172.21.15.12:0/286152766 shutdown_connections 2023-10-30T12:56:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.095+0000 7f3f7d668640 1 --2- 172.21.15.12:0/286152766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f781ee5b0 0x7f3f780c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.095+0000 7f3f7d668640 1 --2- 172.21.15.12:0/286152766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f780c92e0 0x7f3f78150690 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.095+0000 7f3f7d668640 1 --2- 172.21.15.12:0/286152766 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f3f640664a0 0x7f3f64068960 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.095+0000 7f3f7d668640 1 --2- 172.21.15.12:0/286152766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f60004360 0x7f3f781f2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.095+0000 7f3f7d668640 1 -- 172.21.15.12:0/286152766 >> 172.21.15.12:0/286152766 conn(0x7f3f7806bfe0 msgr2=0x7f3f780b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.095+0000 7f3f7d668640 1 -- 172.21.15.12:0/286152766 shutdown_connections 2023-10-30T12:56:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.095+0000 7f3f7d668640 1 -- 172.21.15.12:0/286152766 wait complete. 2023-10-30T12:56:20.117 INFO:tasks.ceph:Scrubbing osd.5 2023-10-30T12:56:20.118 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:56:20.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.263+0000 7f0a74eea640 1 Processor -- start 2023-10-30T12:56:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.263+0000 7f0a74eea640 1 -- start start 2023-10-30T12:56:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.263+0000 7f0a74eea640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0a7006f010 0x7f0a7006f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.263+0000 7f0a74eea640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a700c6a50 con 0x7f0a70156ed0 2023-10-30T12:56:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.263+0000 7f0a74eea640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a700c3750 con 0x7f0a700c6eb0 2023-10-30T12:56:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.263+0000 7f0a74eea640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a700c6630 con 0x7f0a7006f010 2023-10-30T12:56:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.263+0000 7f0a6ed9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0a7006f010 0x7f0a7006f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.263+0000 7f0a6ed9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0a7006f010 0x7f0a7006f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52410/0 (socket says 172.21.15.12:52410) 2023-10-30T12:56:20.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.263+0000 7f0a6ed9d640 1 -- 172.21.15.12:0/3955888261 learned_addr learned my addr 172.21.15.12:0/3955888261 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:20.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.263+0000 7f0a6e59c640 1 --1- 172.21.15.12:0/3955888261 >> v1:172.21.15.12:6789/0 conn(0x7f0a70156ed0 0x7f0a700c43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44468/0 (socket says 172.21.15.12:44468) 2023-10-30T12:56:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.263+0000 7f0a6ed9d640 1 -- 172.21.15.12:0/3955888261 >> v1:172.21.15.154:6789/0 conn(0x7f0a700c6eb0 legacy=0x7f0a700c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.263+0000 7f0a6ed9d640 1 -- 172.21.15.12:0/3955888261 >> v1:172.21.15.12:6789/0 conn(0x7f0a70156ed0 legacy=0x7f0a700c43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.263+0000 7f0a6ed9d640 1 -- 172.21.15.12:0/3955888261 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a701572b0 con 0x7f0a7006f010 2023-10-30T12:56:20.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.267+0000 7f0a6ed9d640 1 --2- 172.21.15.12:0/3955888261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7006f010 0x7f0a7006f4c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009e50 tx=0x7f0a6801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8bb597ea50054a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:20.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.267+0000 7f0a6d59a640 1 -- 172.21.15.12:0/3955888261 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a680a5860 con 0x7f0a7006f010 2023-10-30T12:56:20.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.267+0000 7f0a6d59a640 1 -- 172.21.15.12:0/3955888261 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a680a59c0 con 0x7f0a7006f010 2023-10-30T12:56:20.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.267+0000 7f0a6d59a640 1 -- 172.21.15.12:0/3955888261 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a680a5d50 con 0x7f0a7006f010 2023-10-30T12:56:20.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.267+0000 7f0a74eea640 1 -- 172.21.15.12:0/3955888261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7006f010 msgr2=0x7f0a7006f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.267+0000 7f0a74eea640 1 --2- 172.21.15.12:0/3955888261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7006f010 0x7f0a7006f4c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009e50 tx=0x7f0a6801c9a0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.267+0000 7f0a74eea640 1 -- 172.21.15.12:0/3955888261 shutdown_connections 2023-10-30T12:56:20.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.267+0000 7f0a74eea640 1 --2- 172.21.15.12:0/3955888261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7006f010 0x7f0a7006f4c0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.267+0000 7f0a74eea640 1 -- 172.21.15.12:0/3955888261 >> 172.21.15.12:0/3955888261 conn(0x7f0a700ccf80 msgr2=0x7f0a700cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:20.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.267+0000 7f0a74eea640 1 -- 172.21.15.12:0/3955888261 shutdown_connections 2023-10-30T12:56:20.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.267+0000 7f0a74eea640 1 -- 172.21.15.12:0/3955888261 wait complete. 2023-10-30T12:56:20.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a74eea640 1 Processor -- start 2023-10-30T12:56:20.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a74eea640 1 -- start start 2023-10-30T12:56:20.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a74eea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7006f010 0x7f0a70158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a74eea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a700c6eb0 0x7f0a701f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a74eea640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a70156ed0 0x7f0a70157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a74eea640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a70158470 con 0x7f0a7006f010 2023-10-30T12:56:20.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a74eea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a70148cd0 con 0x7f0a700c6eb0 2023-10-30T12:56:20.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a74eea640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a70159200 con 0x7f0a70156ed0 2023-10-30T12:56:20.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a6e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7006f010 0x7f0a70158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a6e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7006f010 0x7f0a70158cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52422/0 (socket says 172.21.15.12:52422) 2023-10-30T12:56:20.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a6e59c640 1 -- 172.21.15.12:0/3865308312 learned_addr learned my addr 172.21.15.12:0/3865308312 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:20.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a6dd9b640 1 --2- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a700c6eb0 0x7f0a701f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a6ed9d640 1 --2- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a70156ed0 0x7f0a70157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a6e59c640 1 -- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a70156ed0 msgr2=0x7f0a70157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a6e59c640 1 --2- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a70156ed0 0x7f0a70157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a6e59c640 1 -- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a700c6eb0 msgr2=0x7f0a701f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a6e59c640 1 --2- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a700c6eb0 0x7f0a701f23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a6e59c640 1 -- 172.21.15.12:0/3865308312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a68009b30 con 0x7f0a7006f010 2023-10-30T12:56:20.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a6e59c640 1 --2- 172.21.15.12:0/3865308312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7006f010 0x7f0a70158cc0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000c490 tx=0x7f0a6000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:20.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a5effd640 1 -- 172.21.15.12:0/3865308312 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60007480 con 0x7f0a7006f010 2023-10-30T12:56:20.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a5effd640 1 -- 172.21.15.12:0/3865308312 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a600075e0 con 0x7f0a7006f010 2023-10-30T12:56:20.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a74eea640 1 -- 172.21.15.12:0/3865308312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a700b2b30 con 0x7f0a7006f010 2023-10-30T12:56:20.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a5effd640 1 -- 172.21.15.12:0/3865308312 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60004a10 con 0x7f0a7006f010 2023-10-30T12:56:20.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a74eea640 1 -- 172.21.15.12:0/3865308312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a700b2dd0 con 0x7f0a7006f010 2023-10-30T12:56:20.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.271+0000 7f0a74eea640 1 -- 172.21.15.12:0/3865308312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a700c6250 con 0x7f0a7006f010 2023-10-30T12:56:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.275+0000 7f0a5effd640 1 -- 172.21.15.12:0/3865308312 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0a60004b70 con 0x7f0a7006f010 2023-10-30T12:56:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.275+0000 7f0a5effd640 1 --2- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a500664a0 0x7f0a50068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.275+0000 7f0a5effd640 1 -- 172.21.15.12:0/3865308312 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f0a60083690 con 0x7f0a7006f010 2023-10-30T12:56:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.275+0000 7f0a5effd640 1 --2- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:6800/2056450204,v1:172.21.15.154:6801/2056450204] conn(0x7f0a5006ef60 0x7f0a500713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.275+0000 7f0a5effd640 1 -- 172.21.15.12:0/3865308312 --> [v2:172.21.15.154:6800/2056450204,v1:172.21.15.154:6801/2056450204] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a50071ab0 con 0x7f0a5006ef60 2023-10-30T12:56:20.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.275+0000 7f0a5effd640 1 -- 172.21.15.12:0/3865308312 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a60085c70 con 0x7f0a7006f010 2023-10-30T12:56:20.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.275+0000 7f0a6dd9b640 1 --2- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a500664a0 0x7f0a50068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.275+0000 7f0a6ed9d640 1 --2- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:6800/2056450204,v1:172.21.15.154:6801/2056450204] conn(0x7f0a5006ef60 0x7f0a500713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.275+0000 7f0a6dd9b640 1 --2- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a500664a0 0x7f0a50068960 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0a700c3230 tx=0x7f0a64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:20.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.275+0000 7f0a6ed9d640 1 --2- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:6800/2056450204,v1:172.21.15.154:6801/2056450204] conn(0x7f0a5006ef60 0x7f0a500713e0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:20.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.279+0000 7f0a5effd640 1 -- 172.21.15.12:0/3865308312 <== osd.5 v2:172.21.15.154:6800/2056450204 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f0a50071ab0 con 0x7f0a5006ef60 2023-10-30T12:56:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.291+0000 7f0a74eea640 1 -- 172.21.15.12:0/3865308312 --> [v2:172.21.15.154:6800/2056450204,v1:172.21.15.154:6801/2056450204] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0a700c3fd0 con 0x7f0a5006ef60 2023-10-30T12:56:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.291+0000 7f0a5effd640 1 -- 172.21.15.12:0/3865308312 <== osd.5 v2:172.21.15.154:6800/2056450204 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0a680091d0 con 0x7f0a5006ef60 2023-10-30T12:56:20.296 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-30T12:56:20.296 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:56:20.297 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-30T12:56:20.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.291+0000 7f0a74eea640 1 -- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:6800/2056450204,v1:172.21.15.154:6801/2056450204] conn(0x7f0a5006ef60 msgr2=0x7f0a500713e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.291+0000 7f0a74eea640 1 --2- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:6800/2056450204,v1:172.21.15.154:6801/2056450204] conn(0x7f0a5006ef60 0x7f0a500713e0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.291+0000 7f0a74eea640 1 -- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a500664a0 msgr2=0x7f0a50068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.291+0000 7f0a74eea640 1 --2- 172.21.15.12:0/3865308312 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f0a500664a0 0x7f0a50068960 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0a700c3230 tx=0x7f0a64009040 comp rx=0 tx=0).stop 2023-10-30T12:56:20.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.291+0000 7f0a74eea640 1 -- 172.21.15.12:0/3865308312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7006f010 msgr2=0x7f0a70158cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.291+0000 7f0a74eea640 1 --2- 172.21.15.12:0/3865308312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7006f010 0x7f0a70158cc0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000c490 tx=0x7f0a6000c950 comp rx=0 tx=0).stop 2023-10-30T12:56:20.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.291+0000 7f0a6ed9d640 1 -- 172.21.15.12:0/3865308312 reap_dead start 2023-10-30T12:56:20.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.291+0000 7f0a74eea640 1 -- 172.21.15.12:0/3865308312 shutdown_connections 2023-10-30T12:56:20.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.291+0000 7f0a74eea640 1 -- 172.21.15.12:0/3865308312 >> 172.21.15.12:0/3865308312 conn(0x7f0a700ccf80 msgr2=0x7f0a700cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:20.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.291+0000 7f0a74eea640 1 -- 172.21.15.12:0/3865308312 shutdown_connections 2023-10-30T12:56:20.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.291+0000 7f0a74eea640 1 -- 172.21.15.12:0/3865308312 wait complete. 2023-10-30T12:56:20.314 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-30T12:56:20.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.455+0000 7fbcb886b640 1 Processor -- start 2023-10-30T12:56:20.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.455+0000 7fbcb886b640 1 -- start start 2023-10-30T12:56:20.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.455+0000 7fbcb886b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbcb4153c60 0x7fbcb40ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.455+0000 7fbcb886b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcb40c4430 0x7fbcb40cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.455+0000 7fbcb886b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbcb4159830 0x7fbcb4159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.455+0000 7fbcb886b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb4154040 con 0x7fbcb4153c60 2023-10-30T12:56:20.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.455+0000 7fbcb886b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb40cb780 con 0x7fbcb4159830 2023-10-30T12:56:20.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.455+0000 7fbcb886b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb40cc9f0 con 0x7fbcb40c4430 2023-10-30T12:56:20.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.455+0000 7fbcb259c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbcb4153c60 0x7fbcb40ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.455+0000 7fbcb259c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbcb4153c60 0x7fbcb40ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52432/0 (socket says 172.21.15.12:52432) 2023-10-30T12:56:20.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.455+0000 7fbcb259c640 1 -- 172.21.15.12:0/2598591994 learned_addr learned my addr 172.21.15.12:0/2598591994 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:20.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.455+0000 7fbcb1d9b640 1 --2- 172.21.15.12:0/2598591994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcb40c4430 0x7fbcb40cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.459+0000 7fbcb2d9d640 1 --2- 172.21.15.12:0/2598591994 >> v2:172.21.15.154:3300/0 conn(0x7fbcb4159830 0x7fbcb4159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.459+0000 7fbcb259c640 1 -- 172.21.15.12:0/2598591994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcb40c4430 msgr2=0x7fbcb40cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:20.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.459+0000 7fbcb259c640 1 --2- 172.21.15.12:0/2598591994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcb40c4430 0x7fbcb40cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.459+0000 7fbcb259c640 1 -- 172.21.15.12:0/2598591994 >> v2:172.21.15.154:3300/0 conn(0x7fbcb4159830 msgr2=0x7fbcb4159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:20.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.459+0000 7fbcb259c640 1 --2- 172.21.15.12:0/2598591994 >> v2:172.21.15.154:3300/0 conn(0x7fbcb4159830 0x7fbcb4159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.459+0000 7fbcb259c640 1 -- 172.21.15.12:0/2598591994 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb415a2e0 con 0x7fbcb4153c60 2023-10-30T12:56:20.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.459+0000 7fbcb2d9d640 1 --2- 172.21.15.12:0/2598591994 >> v2:172.21.15.154:3300/0 conn(0x7fbcb4159830 0x7fbcb4159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:56:20.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.459+0000 7fbcb259c640 1 --2- 172.21.15.12:0/2598591994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4153c60 0x7fbcb40ca360 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fbcb408d1a0 tx=0x7fbca401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7057719539b050 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:20.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.459+0000 7fbcb159a640 1 -- 172.21.15.12:0/2598591994 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca40ac410 con 0x7fbcb4153c60 2023-10-30T12:56:20.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.459+0000 7fbcb159a640 1 -- 172.21.15.12:0/2598591994 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca40ac570 con 0x7fbcb4153c60 2023-10-30T12:56:20.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.459+0000 7fbcb159a640 1 -- 172.21.15.12:0/2598591994 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca40ac900 con 0x7fbcb4153c60 2023-10-30T12:56:20.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.459+0000 7fbcb886b640 1 -- 172.21.15.12:0/2598591994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4153c60 msgr2=0x7fbcb40ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.459+0000 7fbcb886b640 1 --2- 172.21.15.12:0/2598591994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4153c60 0x7fbcb40ca360 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fbcb408d1a0 tx=0x7fbca401c500 comp rx=0 tx=0).stop 2023-10-30T12:56:20.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb886b640 1 -- 172.21.15.12:0/2598591994 shutdown_connections 2023-10-30T12:56:20.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb886b640 1 --2- 172.21.15.12:0/2598591994 >> v2:172.21.15.154:3300/0 conn(0x7fbcb4159830 0x7fbcb4159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb886b640 1 --2- 172.21.15.12:0/2598591994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcb40c4430 0x7fbcb40cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb886b640 1 --2- 172.21.15.12:0/2598591994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4153c60 0x7fbcb40ca360 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb886b640 1 -- 172.21.15.12:0/2598591994 >> 172.21.15.12:0/2598591994 conn(0x7fbcb406bfe0 msgr2=0x7fbcb406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:20.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb886b640 1 -- 172.21.15.12:0/2598591994 shutdown_connections 2023-10-30T12:56:20.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb886b640 1 -- 172.21.15.12:0/2598591994 wait complete. 2023-10-30T12:56:20.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb886b640 1 Processor -- start 2023-10-30T12:56:20.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb886b640 1 -- start start 2023-10-30T12:56:20.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb886b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcb40c4430 0x7fbcb40cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb886b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4153c60 0x7fbcb4147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb886b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcb4159830 0x7fbcb4148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb886b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb414bb90 con 0x7fbcb4153c60 2023-10-30T12:56:20.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb886b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb4158010 con 0x7fbcb40c4430 2023-10-30T12:56:20.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb886b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb41f5ff0 con 0x7fbcb4159830 2023-10-30T12:56:20.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4153c60 0x7fbcb4147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4153c60 0x7fbcb4147fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52438/0 (socket says 172.21.15.12:52438) 2023-10-30T12:56:20.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcb40c4430 0x7fbcb40cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb1d9b640 1 -- 172.21.15.12:0/1171933410 learned_addr learned my addr 172.21.15.12:0/1171933410 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:20.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb2d9d640 1 --2- 172.21.15.12:0/1171933410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcb4159830 0x7fbcb4148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb1d9b640 1 -- 172.21.15.12:0/1171933410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcb4159830 msgr2=0x7fbcb4148510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb1d9b640 1 --2- 172.21.15.12:0/1171933410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcb4159830 0x7fbcb4148510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb1d9b640 1 -- 172.21.15.12:0/1171933410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcb40c4430 msgr2=0x7fbcb40cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb1d9b640 1 --2- 172.21.15.12:0/1171933410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcb40c4430 0x7fbcb40cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.463+0000 7fbcb1d9b640 1 -- 172.21.15.12:0/1171933410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca40094d0 con 0x7fbcb4153c60 2023-10-30T12:56:20.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.467+0000 7fbcb1d9b640 1 --2- 172.21.15.12:0/1171933410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4153c60 0x7fbcb4147fd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbca800d7d0 tx=0x7fbca800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:20.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.467+0000 7fbca2ffd640 1 -- 172.21.15.12:0/1171933410 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8004020 con 0x7fbcb4153c60 2023-10-30T12:56:20.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.467+0000 7fbca2ffd640 1 -- 172.21.15.12:0/1171933410 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca8004180 con 0x7fbcb4153c60 2023-10-30T12:56:20.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.467+0000 7fbca2ffd640 1 -- 172.21.15.12:0/1171933410 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8004330 con 0x7fbcb4153c60 2023-10-30T12:56:20.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.467+0000 7fbcb886b640 1 -- 172.21.15.12:0/1171933410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb41f5a90 con 0x7fbcb4153c60 2023-10-30T12:56:20.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.467+0000 7fbcb886b640 1 -- 172.21.15.12:0/1171933410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb41f5d10 con 0x7fbcb4153c60 2023-10-30T12:56:20.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.467+0000 7fbcb886b640 1 -- 172.21.15.12:0/1171933410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb404e170 con 0x7fbcb4153c60 2023-10-30T12:56:20.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.467+0000 7fbca2ffd640 1 -- 172.21.15.12:0/1171933410 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbca80096a0 con 0x7fbcb4153c60 2023-10-30T12:56:20.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.467+0000 7fbca2ffd640 1 --2- 172.21.15.12:0/1171933410 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbc940664d0 0x7fbc94068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.467+0000 7fbca2ffd640 1 -- 172.21.15.12:0/1171933410 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fbca8084460 con 0x7fbcb4153c60 2023-10-30T12:56:20.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.467+0000 7fbcb259c640 1 --2- 172.21.15.12:0/1171933410 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbc940664d0 0x7fbc94068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.471+0000 7fbcb259c640 1 --2- 172.21.15.12:0/1171933410 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbc940664d0 0x7fbc94068990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbca40ae000 tx=0x7fbca4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:20.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.471+0000 7fbca2ffd640 1 -- 172.21.15.12:0/1171933410 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbca8051580 con 0x7fbcb4153c60 2023-10-30T12:56:20.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.639+0000 7fbcb886b640 1 -- 172.21.15.12:0/1171933410 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcb40cbf90 con 0x7fbc940664d0 2023-10-30T12:56:20.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.639+0000 7fbca2ffd640 1 -- 172.21.15.12:0/1171933410 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbca40bc090 con 0x7fbc940664d0 2023-10-30T12:56:20.643 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 5 to deep-scrub 2023-10-30T12:56:20.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.639+0000 7fbcb886b640 1 -- 172.21.15.12:0/1171933410 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbc940664d0 msgr2=0x7fbc94068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.639+0000 7fbcb886b640 1 --2- 172.21.15.12:0/1171933410 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbc940664d0 0x7fbc94068990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbca40ae000 tx=0x7fbca4008040 comp rx=0 tx=0).stop 2023-10-30T12:56:20.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.639+0000 7fbcb886b640 1 -- 172.21.15.12:0/1171933410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4153c60 msgr2=0x7fbcb4147fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.639+0000 7fbcb886b640 1 --2- 172.21.15.12:0/1171933410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4153c60 0x7fbcb4147fd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbca800d7d0 tx=0x7fbca800dc90 comp rx=0 tx=0).stop 2023-10-30T12:56:20.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.639+0000 7fbcb886b640 1 -- 172.21.15.12:0/1171933410 shutdown_connections 2023-10-30T12:56:20.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.639+0000 7fbcb886b640 1 --2- 172.21.15.12:0/1171933410 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbc940664d0 0x7fbc94068990 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.639+0000 7fbcb886b640 1 --2- 172.21.15.12:0/1171933410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcb4159830 0x7fbcb4148510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.639+0000 7fbcb886b640 1 --2- 172.21.15.12:0/1171933410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4153c60 0x7fbcb4147fd0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.639+0000 7fbcb886b640 1 --2- 172.21.15.12:0/1171933410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcb40c4430 0x7fbcb40cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.639+0000 7fbcb886b640 1 -- 172.21.15.12:0/1171933410 >> 172.21.15.12:0/1171933410 conn(0x7fbcb406bfe0 msgr2=0x7fbcb4151ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:20.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.639+0000 7fbcb886b640 1 -- 172.21.15.12:0/1171933410 shutdown_connections 2023-10-30T12:56:20.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.639+0000 7fbcb886b640 1 -- 172.21.15.12:0/1171933410 wait complete. 2023-10-30T12:56:20.665 INFO:tasks.ceph:Scrubbing osd.6 2023-10-30T12:56:20.665 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:56:20.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.807+0000 7fbcb24c9640 1 Processor -- start 2023-10-30T12:56:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.807+0000 7fbcb24c9640 1 -- start start 2023-10-30T12:56:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.807+0000 7fbcb24c9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbcac158b10 0x7fbcac157620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.807+0000 7fbcb24c9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcac06c810 con 0x7fbcac0c6490 2023-10-30T12:56:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.807+0000 7fbcb24c9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcac15b630 con 0x7fbcac1517f0 2023-10-30T12:56:20.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.807+0000 7fbcb24c9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcac15bbf0 con 0x7fbcac158b10 2023-10-30T12:56:20.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcb0a66640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbcac0c6490 0x7fbcac0c6b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43986/0 (socket says 172.21.15.12:43986) 2023-10-30T12:56:20.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcb0a66640 1 -- 172.21.15.12:0/2105665195 learned_addr learned my addr 172.21.15.12:0/2105665195 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:20.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcab7fe640 1 --2- 172.21.15.12:0/2105665195 >> v2:172.21.15.154:3300/0 conn(0x7fbcac158b10 0x7fbcac157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692288812 0 0) 0x7fbcac06c810 con 0x7fbcac0c6490 2023-10-30T12:56:20.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc94003530 con 0x7fbcac0c6490 2023-10-30T12:56:20.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3135566882 0 0) 0x7fbcac15b630 con 0x7fbcac1517f0 2023-10-30T12:56:20.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcac06c810 con 0x7fbcac1517f0 2023-10-30T12:56:20.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2877551726 0 0) 0x7fbc94003530 con 0x7fbcac0c6490 2023-10-30T12:56:20.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 >> v2:172.21.15.154:3300/0 conn(0x7fbcac158b10 msgr2=0x7fbcac157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:20.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 --2- 172.21.15.12:0/2105665195 >> v2:172.21.15.154:3300/0 conn(0x7fbcac158b10 0x7fbcac157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 >> v1:172.21.15.154:6789/0 conn(0x7fbcac1517f0 legacy=0x7fbcac15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac15b910 con 0x7fbcac0c6490 2023-10-30T12:56:20.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7fbca4001a00 con 0x7fbcac0c6490 2023-10-30T12:56:20.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcab7fe640 1 --2- 172.21.15.12:0/2105665195 >> v2:172.21.15.154:3300/0 conn(0x7fbcac158b10 0x7fbcac157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:56:20.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 >> v1:172.21.15.12:6789/0 conn(0x7fbcac0c6490 legacy=0x7fbcac0c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 --2- 172.21.15.12:0/2105665195 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc94003b10 0x7fbc94003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 --2- 172.21.15.12:0/2105665195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc940047d0 0x7fbc94004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 --2- 172.21.15.12:0/2105665195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc94005190 0x7fbc9400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac15bbf0 con 0x7fbc940047d0 2023-10-30T12:56:20.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9400c010 con 0x7fbc94005190 2023-10-30T12:56:20.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9400c1d0 con 0x7fbc94003b10 2023-10-30T12:56:20.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcab7fe640 1 --2- 172.21.15.12:0/2105665195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc940047d0 0x7fbc94004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcb0a66640 1 --2- 172.21.15.12:0/2105665195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc94005190 0x7fbc9400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcabfff640 1 --2- 172.21.15.12:0/2105665195 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc94003b10 0x7fbc94003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcab7fe640 1 -- 172.21.15.12:0/2105665195 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc94003b10 msgr2=0x7fbc94003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcab7fe640 1 --2- 172.21.15.12:0/2105665195 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbc94003b10 0x7fbc94003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcab7fe640 1 -- 172.21.15.12:0/2105665195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc94005190 msgr2=0x7fbc9400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcab7fe640 1 --2- 172.21.15.12:0/2105665195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc94005190 0x7fbc9400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcab7fe640 1 -- 172.21.15.12:0/2105665195 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc94003530 con 0x7fbc940047d0 2023-10-30T12:56:20.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.811+0000 7fbcb0a66640 1 -- 172.21.15.12:0/2105665195 reap_dead start 2023-10-30T12:56:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.815+0000 7fbcab7fe640 1 --2- 172.21.15.12:0/2105665195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc940047d0 0x7fbc94004c50 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009cb0 tx=0x7fbca001cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999cc5b6207ff35e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.815+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca00a58b0 con 0x7fbc940047d0 2023-10-30T12:56:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.815+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca00a5a10 con 0x7fbc940047d0 2023-10-30T12:56:20.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.815+0000 7fbcaaffd640 1 -- 172.21.15.12:0/2105665195 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca00a5e00 con 0x7fbc940047d0 2023-10-30T12:56:20.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.815+0000 7fbcb24c9640 1 -- 172.21.15.12:0/2105665195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc940047d0 msgr2=0x7fbc94004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.815+0000 7fbcb24c9640 1 --2- 172.21.15.12:0/2105665195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc940047d0 0x7fbc94004c50 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009cb0 tx=0x7fbca001cbf0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.815+0000 7fbcb24c9640 1 -- 172.21.15.12:0/2105665195 shutdown_connections 2023-10-30T12:56:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.815+0000 7fbcb24c9640 1 --2- 172.21.15.12:0/2105665195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc940047d0 0x7fbc94004c50 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.815+0000 7fbcb24c9640 1 -- 172.21.15.12:0/2105665195 >> 172.21.15.12:0/2105665195 conn(0x7fbcac0c7910 msgr2=0x7fbcac0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.815+0000 7fbcb24c9640 1 -- 172.21.15.12:0/2105665195 shutdown_connections 2023-10-30T12:56:20.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.815+0000 7fbcb24c9640 1 -- 172.21.15.12:0/2105665195 wait complete. 2023-10-30T12:56:20.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.815+0000 7fbcb24c9640 1 Processor -- start 2023-10-30T12:56:20.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.815+0000 7fbcb24c9640 1 -- start start 2023-10-30T12:56:20.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb24c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc940047d0 0x7fbcac0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb24c9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcac0af830 0x7fbcac1efa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb24c9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcac14d7a0 0x7fbcac14c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb24c9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac14ebf0 con 0x7fbcac14d7a0 2023-10-30T12:56:20.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb24c9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac1ef5e0 con 0x7fbc940047d0 2023-10-30T12:56:20.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb24c9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac14b970 con 0x7fbcac0af830 2023-10-30T12:56:20.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb0a66640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcac14d7a0 0x7fbcac14c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb0a66640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcac14d7a0 0x7fbcac14c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50618/0 (socket says 172.21.15.12:50618) 2023-10-30T12:56:20.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb0a66640 1 -- 172.21.15.12:0/1650905203 learned_addr learned my addr 172.21.15.12:0/1650905203 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:20.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcabfff640 1 --2- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc940047d0 0x7fbcac0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcab7fe640 1 --2- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcac0af830 0x7fbcac1efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb0a66640 1 -- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcac0af830 msgr2=0x7fbcac1efa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb0a66640 1 --2- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcac0af830 0x7fbcac1efa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb0a66640 1 -- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc940047d0 msgr2=0x7fbcac0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb0a66640 1 --2- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc940047d0 0x7fbcac0701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb0a66640 1 -- 172.21.15.12:0/1650905203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac06f600 con 0x7fbcac14d7a0 2023-10-30T12:56:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcabfff640 1 --2- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc940047d0 0x7fbcac0701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb0a66640 1 --2- 172.21.15.12:0/1650905203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcac14d7a0 0x7fbcac14c3e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbcac072500 tx=0x7fbca400e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbca8ff9640 1 -- 172.21.15.12:0/1650905203 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca400eb10 con 0x7fbcac14d7a0 2023-10-30T12:56:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbca8ff9640 1 -- 172.21.15.12:0/1650905203 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca400ec70 con 0x7fbcac14d7a0 2023-10-30T12:56:20.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb24c9640 1 -- 172.21.15.12:0/1650905203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcac1eefb0 con 0x7fbcac14d7a0 2023-10-30T12:56:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbca8ff9640 1 -- 172.21.15.12:0/1650905203 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4013420 con 0x7fbcac14d7a0 2023-10-30T12:56:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb24c9640 1 -- 172.21.15.12:0/1650905203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcac1ef270 con 0x7fbcac14d7a0 2023-10-30T12:56:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbcb24c9640 1 -- 172.21.15.12:0/1650905203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbcac0c5c90 con 0x7fbcac14d7a0 2023-10-30T12:56:20.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.819+0000 7fbca8ff9640 1 -- 172.21.15.12:0/1650905203 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbca4013580 con 0x7fbcac14d7a0 2023-10-30T12:56:20.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.823+0000 7fbca8ff9640 1 --2- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbc980664d0 0x7fbc98068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.823+0000 7fbca8ff9640 1 -- 172.21.15.12:0/1650905203 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fbca4084650 con 0x7fbcac14d7a0 2023-10-30T12:56:20.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.823+0000 7fbca8ff9640 1 --2- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:6817/1102176766,v1:172.21.15.154:6819/1102176766] conn(0x7fbc9806ef90 0x7fbc98071410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:20.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.823+0000 7fbca8ff9640 1 -- 172.21.15.12:0/1650905203 --> [v2:172.21.15.154:6817/1102176766,v1:172.21.15.154:6819/1102176766] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbc98071ae0 con 0x7fbc9806ef90 2023-10-30T12:56:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.823+0000 7fbca8ff9640 1 -- 172.21.15.12:0/1650905203 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbca4086c30 con 0x7fbcac14d7a0 2023-10-30T12:56:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.823+0000 7fbcabfff640 1 --2- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbc980664d0 0x7fbc98068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.823+0000 7fbcab7fe640 1 --2- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:6817/1102176766,v1:172.21.15.154:6819/1102176766] conn(0x7fbc9806ef90 0x7fbc98071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.823+0000 7fbcabfff640 1 --2- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbc980664d0 0x7fbc98068990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c005630 tx=0x7fbc9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.823+0000 7fbcab7fe640 1 --2- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:6817/1102176766,v1:172.21.15.154:6819/1102176766] conn(0x7fbc9806ef90 0x7fbc98071410 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.823+0000 7fbca8ff9640 1 -- 172.21.15.12:0/1650905203 <== osd.6 v2:172.21.15.154:6817/1102176766 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fbc98071ae0 con 0x7fbc9806ef90 2023-10-30T12:56:20.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.839+0000 7fbcb24c9640 1 -- 172.21.15.12:0/1650905203 --> [v2:172.21.15.154:6817/1102176766,v1:172.21.15.154:6819/1102176766] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbcac0c59b0 con 0x7fbc9806ef90 2023-10-30T12:56:20.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.839+0000 7fbca8ff9640 1 -- 172.21.15.12:0/1650905203 <== osd.6 v2:172.21.15.154:6817/1102176766 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbca00a5b80 con 0x7fbc9806ef90 2023-10-30T12:56:20.843 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-30T12:56:20.843 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:56:20.844 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-30T12:56:20.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.839+0000 7fbcb24c9640 1 -- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:6817/1102176766,v1:172.21.15.154:6819/1102176766] conn(0x7fbc9806ef90 msgr2=0x7fbc98071410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.839+0000 7fbcb24c9640 1 --2- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:6817/1102176766,v1:172.21.15.154:6819/1102176766] conn(0x7fbc9806ef90 0x7fbc98071410 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:20.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.839+0000 7fbcb24c9640 1 -- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbc980664d0 msgr2=0x7fbc98068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.839+0000 7fbcb24c9640 1 --2- 172.21.15.12:0/1650905203 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fbc980664d0 0x7fbc98068990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c005630 tx=0x7fbc9c009040 comp rx=0 tx=0).stop 2023-10-30T12:56:20.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.839+0000 7fbcb24c9640 1 -- 172.21.15.12:0/1650905203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcac14d7a0 msgr2=0x7fbcac14c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:20.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.839+0000 7fbcb24c9640 1 --2- 172.21.15.12:0/1650905203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcac14d7a0 0x7fbcac14c3e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbcac072500 tx=0x7fbca400e160 comp rx=0 tx=0).stop 2023-10-30T12:56:20.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.839+0000 7fbcb0a66640 1 -- 172.21.15.12:0/1650905203 reap_dead start 2023-10-30T12:56:20.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.839+0000 7fbcb24c9640 1 -- 172.21.15.12:0/1650905203 shutdown_connections 2023-10-30T12:56:20.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.839+0000 7fbcb24c9640 1 -- 172.21.15.12:0/1650905203 >> 172.21.15.12:0/1650905203 conn(0x7fbcac0c7910 msgr2=0x7fbcac14f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:20.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.839+0000 7fbcb24c9640 1 -- 172.21.15.12:0/1650905203 shutdown_connections 2023-10-30T12:56:20.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:20.839+0000 7fbcb24c9640 1 -- 172.21.15.12:0/1650905203 wait complete. 2023-10-30T12:56:20.862 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-30T12:56:21.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.003+0000 7fe0761a7640 1 Processor -- start 2023-10-30T12:56:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.003+0000 7fe0761a7640 1 -- start start 2023-10-30T12:56:21.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.003+0000 7fe0761a7640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe0700cb050 0x7fe0700cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.003+0000 7fe0761a7640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0700cba30 con 0x7fe0700cab00 2023-10-30T12:56:21.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.003+0000 7fe0761a7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0700ca290 con 0x7fe0700c5550 2023-10-30T12:56:21.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.003+0000 7fe0761a7640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0700c9cf0 con 0x7fe0700cb050 2023-10-30T12:56:21.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.003+0000 7fe0749a4640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe0700cb050 0x7fe0700cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.003+0000 7fe0749a4640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe0700cb050 0x7fe0700cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50632/0 (socket says 172.21.15.12:50632) 2023-10-30T12:56:21.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.003+0000 7fe0749a4640 1 -- 172.21.15.12:0/885529205 learned_addr learned my addr 172.21.15.12:0/885529205 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:21.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.003+0000 7fe0749a4640 1 -- 172.21.15.12:0/885529205 >> v1:172.21.15.154:6789/0 conn(0x7fe0700c5550 legacy=0x7fe0700cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.003+0000 7fe0749a4640 1 -- 172.21.15.12:0/885529205 >> v1:172.21.15.12:6789/0 conn(0x7fe0700cab00 legacy=0x7fe070163ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.003+0000 7fe0749a4640 1 -- 172.21.15.12:0/885529205 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07015d990 con 0x7fe0700cb050 2023-10-30T12:56:21.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.007+0000 7fe0749a4640 1 --2- 172.21.15.12:0/885529205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0700cb050 0x7fe0700cbec0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe060006fd0 tx=0x7fe06001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6935726a430104cd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:21.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.007+0000 7fe067fff640 1 -- 172.21.15.12:0/885529205 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0600ab600 con 0x7fe0700cb050 2023-10-30T12:56:21.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.007+0000 7fe067fff640 1 -- 172.21.15.12:0/885529205 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0600a7050 con 0x7fe0700cb050 2023-10-30T12:56:21.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.007+0000 7fe067fff640 1 -- 172.21.15.12:0/885529205 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0600ab980 con 0x7fe0700cb050 2023-10-30T12:56:21.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.007+0000 7fe0761a7640 1 -- 172.21.15.12:0/885529205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0700cb050 msgr2=0x7fe0700cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.007+0000 7fe0761a7640 1 --2- 172.21.15.12:0/885529205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0700cb050 0x7fe0700cbec0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe060006fd0 tx=0x7fe06001c520 comp rx=0 tx=0).stop 2023-10-30T12:56:21.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.007+0000 7fe0761a7640 1 -- 172.21.15.12:0/885529205 shutdown_connections 2023-10-30T12:56:21.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.007+0000 7fe0761a7640 1 --2- 172.21.15.12:0/885529205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0700cb050 0x7fe0700cbec0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.007+0000 7fe0761a7640 1 -- 172.21.15.12:0/885529205 >> 172.21.15.12:0/885529205 conn(0x7fe07006bed0 msgr2=0x7fe07006c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:21.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.007+0000 7fe0761a7640 1 -- 172.21.15.12:0/885529205 shutdown_connections 2023-10-30T12:56:21.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.007+0000 7fe0761a7640 1 -- 172.21.15.12:0/885529205 wait complete. 2023-10-30T12:56:21.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0761a7640 1 Processor -- start 2023-10-30T12:56:21.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0761a7640 1 -- start start 2023-10-30T12:56:21.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0761a7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0700c5550 0x7fe0701f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0761a7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0700cab00 0x7fe070151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0761a7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0700cb050 0x7fe070151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0761a7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0701500a0 con 0x7fe0700cab00 2023-10-30T12:56:21.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0761a7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0700cf4a0 con 0x7fe0700c5550 2023-10-30T12:56:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0761a7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0700c4990 con 0x7fe0700cb050 2023-10-30T12:56:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0749a4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0700cab00 0x7fe070151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0749a4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0700cab00 0x7fe070151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50644/0 (socket says 172.21.15.12:50644) 2023-10-30T12:56:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0749a4640 1 -- 172.21.15.12:0/3701198159 learned_addr learned my addr 172.21.15.12:0/3701198159 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0751a5640 1 --2- 172.21.15.12:0/3701198159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0700c5550 0x7fe0701f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0759a6640 1 --2- 172.21.15.12:0/3701198159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0700cb050 0x7fe070151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0749a4640 1 -- 172.21.15.12:0/3701198159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0700cb050 msgr2=0x7fe070151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0749a4640 1 --2- 172.21.15.12:0/3701198159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0700cb050 0x7fe070151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0749a4640 1 -- 172.21.15.12:0/3701198159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0700c5550 msgr2=0x7fe0701f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0749a4640 1 --2- 172.21.15.12:0/3701198159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0700c5550 0x7fe0701f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0749a4640 1 -- 172.21.15.12:0/3701198159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe060009480 con 0x7fe0700cab00 2023-10-30T12:56:21.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0749a4640 1 --2- 172.21.15.12:0/3701198159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0700cab00 0x7fe070151620 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe06001a150 tx=0x7fe0600192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:21.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe065ffb640 1 -- 172.21.15.12:0/3701198159 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060005e70 con 0x7fe0700cab00 2023-10-30T12:56:21.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe065ffb640 1 -- 172.21.15.12:0/3701198159 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0600037c0 con 0x7fe0700cab00 2023-10-30T12:56:21.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe065ffb640 1 -- 172.21.15.12:0/3701198159 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060003b70 con 0x7fe0700cab00 2023-10-30T12:56:21.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0761a7640 1 -- 172.21.15.12:0/3701198159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0700cfe30 con 0x7fe0700cab00 2023-10-30T12:56:21.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.011+0000 7fe0761a7640 1 -- 172.21.15.12:0/3701198159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0701f5f60 con 0x7fe0700cab00 2023-10-30T12:56:21.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.015+0000 7fe0761a7640 1 -- 172.21.15.12:0/3701198159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0700c8590 con 0x7fe0700cab00 2023-10-30T12:56:21.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.015+0000 7fe065ffb640 1 -- 172.21.15.12:0/3701198159 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe0600b2070 con 0x7fe0700cab00 2023-10-30T12:56:21.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.015+0000 7fe065ffb640 1 --2- 172.21.15.12:0/3701198159 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe0540664d0 0x7fe054068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.015+0000 7fe065ffb640 1 -- 172.21.15.12:0/3701198159 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fe06011d560 con 0x7fe0700cab00 2023-10-30T12:56:21.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.015+0000 7fe0751a5640 1 --2- 172.21.15.12:0/3701198159 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe0540664d0 0x7fe054068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.015+0000 7fe0751a5640 1 --2- 172.21.15.12:0/3701198159 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe0540664d0 0x7fe054068990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe06c000f70 tx=0x7fe06c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:21.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.019+0000 7fe065ffb640 1 -- 172.21.15.12:0/3701198159 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe0600ea600 con 0x7fe0700cab00 2023-10-30T12:56:21.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.175+0000 7fe0761a7640 1 -- 172.21.15.12:0/3701198159 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fe07015d570 con 0x7fe0540664d0 2023-10-30T12:56:21.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.179+0000 7fe065ffb640 1 -- 172.21.15.12:0/3701198159 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe06c001380 con 0x7fe0540664d0 2023-10-30T12:56:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 6 to deep-scrub 2023-10-30T12:56:21.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.179+0000 7fe0761a7640 1 -- 172.21.15.12:0/3701198159 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe0540664d0 msgr2=0x7fe054068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.179+0000 7fe0761a7640 1 --2- 172.21.15.12:0/3701198159 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe0540664d0 0x7fe054068990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe06c000f70 tx=0x7fe06c009040 comp rx=0 tx=0).stop 2023-10-30T12:56:21.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.179+0000 7fe0761a7640 1 -- 172.21.15.12:0/3701198159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0700cab00 msgr2=0x7fe070151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.179+0000 7fe0761a7640 1 --2- 172.21.15.12:0/3701198159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0700cab00 0x7fe070151620 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe06001a150 tx=0x7fe0600192c0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.179+0000 7fe0761a7640 1 -- 172.21.15.12:0/3701198159 shutdown_connections 2023-10-30T12:56:21.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.179+0000 7fe0761a7640 1 --2- 172.21.15.12:0/3701198159 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe0540664d0 0x7fe054068990 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.179+0000 7fe0761a7640 1 --2- 172.21.15.12:0/3701198159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0700cb050 0x7fe070151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.179+0000 7fe0761a7640 1 --2- 172.21.15.12:0/3701198159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0700cab00 0x7fe070151620 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.179+0000 7fe0761a7640 1 --2- 172.21.15.12:0/3701198159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0700c5550 0x7fe0701f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.179+0000 7fe0761a7640 1 -- 172.21.15.12:0/3701198159 >> 172.21.15.12:0/3701198159 conn(0x7fe07006bed0 msgr2=0x7fe070158ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:21.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.179+0000 7fe0761a7640 1 -- 172.21.15.12:0/3701198159 shutdown_connections 2023-10-30T12:56:21.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.179+0000 7fe0761a7640 1 -- 172.21.15.12:0/3701198159 wait complete. 2023-10-30T12:56:21.206 INFO:tasks.ceph:Scrubbing osd.7 2023-10-30T12:56:21.207 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:56:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.351+0000 7eff2c365640 1 Processor -- start 2023-10-30T12:56:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.351+0000 7eff2c365640 1 -- start start 2023-10-30T12:56:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff2c365640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7eff2414cf40 0x7eff24069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff2c365640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff2415a700 0x7eff24156490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff2c365640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff2415a270 con 0x7eff241569d0 2023-10-30T12:56:21.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff2c365640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff2406c7c0 con 0x7eff2414cf40 2023-10-30T12:56:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff2c365640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2406a3d0 con 0x7eff2415a700 2023-10-30T12:56:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff2a902640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7eff241569d0 0x7eff240c6920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44002/0 (socket says 172.21.15.12:44002) 2023-10-30T12:56:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff2a902640 1 -- 172.21.15.12:0/2695881424 learned_addr learned my addr 172.21.15.12:0/2695881424 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff2a101640 1 --2- 172.21.15.12:0/2695881424 >> v2:172.21.15.154:3300/0 conn(0x7eff2414cf40 0x7eff24069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff29900640 1 --2- 172.21.15.12:0/2695881424 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff2415a700 0x7eff24156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff290ff640 1 -- 172.21.15.12:0/2695881424 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578322310 0 0) 0x7eff2415a270 con 0x7eff241569d0 2023-10-30T12:56:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff290ff640 1 -- 172.21.15.12:0/2695881424 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff10003530 con 0x7eff241569d0 2023-10-30T12:56:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff290ff640 1 -- 172.21.15.12:0/2695881424 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3331372541 0 0) 0x7eff10003530 con 0x7eff241569d0 2023-10-30T12:56:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff29900640 1 -- 172.21.15.12:0/2695881424 >> v2:172.21.15.154:3300/0 conn(0x7eff2414cf40 msgr2=0x7eff24069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff29900640 1 --2- 172.21.15.12:0/2695881424 >> v2:172.21.15.154:3300/0 conn(0x7eff2414cf40 0x7eff24069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff29900640 1 -- 172.21.15.12:0/2695881424 >> v1:172.21.15.12:6789/0 conn(0x7eff241569d0 legacy=0x7eff240c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff29900640 1 -- 172.21.15.12:0/2695881424 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff24156ed0 con 0x7eff2415a700 2023-10-30T12:56:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.355+0000 7eff2a101640 1 --2- 172.21.15.12:0/2695881424 >> v2:172.21.15.154:3300/0 conn(0x7eff2414cf40 0x7eff24069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:56:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.359+0000 7eff29900640 1 --2- 172.21.15.12:0/2695881424 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff2415a700 0x7eff24156490 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7eff140097c0 tx=0x7eff1401c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7af81f4f03c04037 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.359+0000 7eff290ff640 1 -- 172.21.15.12:0/2695881424 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c001a00 con 0x7eff2415a700 2023-10-30T12:56:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.359+0000 7eff290ff640 1 -- 172.21.15.12:0/2695881424 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff140a7050 con 0x7eff2415a700 2023-10-30T12:56:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.359+0000 7eff290ff640 1 -- 172.21.15.12:0/2695881424 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff140ab600 con 0x7eff2415a700 2023-10-30T12:56:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.359+0000 7eff2c365640 1 -- 172.21.15.12:0/2695881424 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff2415a700 msgr2=0x7eff24156490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.359+0000 7eff2c365640 1 --2- 172.21.15.12:0/2695881424 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff2415a700 0x7eff24156490 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7eff140097c0 tx=0x7eff1401c3a0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.359+0000 7eff2c365640 1 -- 172.21.15.12:0/2695881424 shutdown_connections 2023-10-30T12:56:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.359+0000 7eff2c365640 1 --2- 172.21.15.12:0/2695881424 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff2415a700 0x7eff24156490 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.359+0000 7eff2c365640 1 --2- 172.21.15.12:0/2695881424 >> v2:172.21.15.154:3300/0 conn(0x7eff2414cf40 0x7eff24069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.359+0000 7eff2c365640 1 -- 172.21.15.12:0/2695881424 >> 172.21.15.12:0/2695881424 conn(0x7eff240c7910 msgr2=0x7eff240c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.359+0000 7eff2c365640 1 -- 172.21.15.12:0/2695881424 shutdown_connections 2023-10-30T12:56:21.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.359+0000 7eff2c365640 1 -- 172.21.15.12:0/2695881424 wait complete. 2023-10-30T12:56:21.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2c365640 1 Processor -- start 2023-10-30T12:56:21.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2c365640 1 -- start start 2023-10-30T12:56:21.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2c365640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff2414cf40 0x7eff24147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2c365640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff241569d0 0x7eff24140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2c365640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff2415a700 0x7eff24140ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2c365640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2414a3c0 con 0x7eff2414cf40 2023-10-30T12:56:21.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2c365640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff241429f0 con 0x7eff2415a700 2023-10-30T12:56:21.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2c365640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff24142cd0 con 0x7eff241569d0 2023-10-30T12:56:21.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2a101640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff2414cf40 0x7eff24147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2a101640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff2414cf40 0x7eff24147970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50652/0 (socket says 172.21.15.12:50652) 2023-10-30T12:56:21.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2a101640 1 -- 172.21.15.12:0/3585151885 learned_addr learned my addr 172.21.15.12:0/3585151885 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:21.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2a902640 1 --2- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff2415a700 0x7eff24140ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff29900640 1 --2- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff241569d0 0x7eff24140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2a101640 1 -- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff241569d0 msgr2=0x7eff24140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2a101640 1 --2- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff241569d0 0x7eff24140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2a101640 1 -- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff2415a700 msgr2=0x7eff24140ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2a101640 1 --2- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff2415a700 0x7eff24140ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2a101640 1 -- 172.21.15.12:0/3585151885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff20009420 con 0x7eff2414cf40 2023-10-30T12:56:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2a101640 1 --2- 172.21.15.12:0/3585151885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff2414cf40 0x7eff24147970 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7eff2000ebe0 tx=0x7eff2000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff1affd640 1 -- 172.21.15.12:0/3585151885 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2000eee0 con 0x7eff2414cf40 2023-10-30T12:56:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff1affd640 1 -- 172.21.15.12:0/3585151885 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2000cc30 con 0x7eff2414cf40 2023-10-30T12:56:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff1affd640 1 -- 172.21.15.12:0/3585151885 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20013470 con 0x7eff2414cf40 2023-10-30T12:56:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2c365640 1 -- 172.21.15.12:0/3585151885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff14009420 con 0x7eff2414cf40 2023-10-30T12:56:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2c365640 1 -- 172.21.15.12:0/3585151885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff240ca800 con 0x7eff2414cf40 2023-10-30T12:56:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.363+0000 7eff2c365640 1 -- 172.21.15.12:0/3585151885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7eff2414d320 con 0x7eff2414cf40 2023-10-30T12:56:21.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.367+0000 7eff1affd640 1 -- 172.21.15.12:0/3585151885 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7eff200135d0 con 0x7eff2414cf40 2023-10-30T12:56:21.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.367+0000 7eff1affd640 1 --2- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7eff080664a0 0x7eff08068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.367+0000 7eff1affd640 1 -- 172.21.15.12:0/3585151885 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7eff20084900 con 0x7eff2414cf40 2023-10-30T12:56:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.367+0000 7eff1affd640 1 --2- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:6815/219299884,v1:172.21.15.154:6818/219299884] conn(0x7eff0806ef60 0x7eff080713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.367+0000 7eff1affd640 1 -- 172.21.15.12:0/3585151885 --> [v2:172.21.15.154:6815/219299884,v1:172.21.15.154:6818/219299884] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff08071ab0 con 0x7eff0806ef60 2023-10-30T12:56:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.367+0000 7eff1affd640 1 -- 172.21.15.12:0/3585151885 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff200892a0 con 0x7eff2414cf40 2023-10-30T12:56:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.367+0000 7eff29900640 1 --2- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7eff080664a0 0x7eff08068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.367+0000 7eff2a902640 1 --2- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:6815/219299884,v1:172.21.15.154:6818/219299884] conn(0x7eff0806ef60 0x7eff080713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.367+0000 7eff29900640 1 --2- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7eff080664a0 0x7eff08068960 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7eff140ad000 tx=0x7eff140a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.367+0000 7eff2a902640 1 --2- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:6815/219299884,v1:172.21.15.154:6818/219299884] conn(0x7eff0806ef60 0x7eff080713e0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.371+0000 7eff1affd640 1 -- 172.21.15.12:0/3585151885 <== osd.7 v2:172.21.15.154:6815/219299884 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7eff08071ab0 con 0x7eff0806ef60 2023-10-30T12:56:21.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.383+0000 7eff2c365640 1 -- 172.21.15.12:0/3585151885 --> [v2:172.21.15.154:6815/219299884,v1:172.21.15.154:6818/219299884] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7eff24159f00 con 0x7eff0806ef60 2023-10-30T12:56:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.383+0000 7eff1affd640 1 -- 172.21.15.12:0/3585151885 <== osd.7 v2:172.21.15.154:6815/219299884 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7eff1c0070f0 con 0x7eff0806ef60 2023-10-30T12:56:21.387 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-30T12:56:21.388 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:56:21.388 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-30T12:56:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.383+0000 7eff2c365640 1 -- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:6815/219299884,v1:172.21.15.154:6818/219299884] conn(0x7eff0806ef60 msgr2=0x7eff080713e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.383+0000 7eff2c365640 1 --2- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:6815/219299884,v1:172.21.15.154:6818/219299884] conn(0x7eff0806ef60 0x7eff080713e0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.383+0000 7eff2c365640 1 -- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7eff080664a0 msgr2=0x7eff08068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.383+0000 7eff2c365640 1 --2- 172.21.15.12:0/3585151885 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7eff080664a0 0x7eff08068960 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7eff140ad000 tx=0x7eff140a6000 comp rx=0 tx=0).stop 2023-10-30T12:56:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.383+0000 7eff2c365640 1 -- 172.21.15.12:0/3585151885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff2414cf40 msgr2=0x7eff24147970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.383+0000 7eff2c365640 1 --2- 172.21.15.12:0/3585151885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff2414cf40 0x7eff24147970 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7eff2000ebe0 tx=0x7eff2000c520 comp rx=0 tx=0).stop 2023-10-30T12:56:21.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.383+0000 7eff2a902640 1 -- 172.21.15.12:0/3585151885 reap_dead start 2023-10-30T12:56:21.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.383+0000 7eff2c365640 1 -- 172.21.15.12:0/3585151885 shutdown_connections 2023-10-30T12:56:21.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.383+0000 7eff2c365640 1 -- 172.21.15.12:0/3585151885 >> 172.21.15.12:0/3585151885 conn(0x7eff240c7910 msgr2=0x7eff240c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:21.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.383+0000 7eff2c365640 1 -- 172.21.15.12:0/3585151885 shutdown_connections 2023-10-30T12:56:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.383+0000 7eff2c365640 1 -- 172.21.15.12:0/3585151885 wait complete. 2023-10-30T12:56:21.406 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-30T12:56:21.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.547+0000 7fd7d8f2a640 1 Processor -- start 2023-10-30T12:56:21.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.547+0000 7fd7d8f2a640 1 -- start start 2023-10-30T12:56:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.547+0000 7fd7d8f2a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7d40c7830 0x7fd7d406ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.547+0000 7fd7d8f2a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd7d406e430 0x7fd7d40cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.547+0000 7fd7d8f2a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d406d710 con 0x7fd7d40cce00 2023-10-30T12:56:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.547+0000 7fd7d8f2a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d406dd30 con 0x7fd7d406e430 2023-10-30T12:56:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.547+0000 7fd7d8f2a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d406beb0 con 0x7fd7d40c7830 2023-10-30T12:56:21.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.547+0000 7fd7d2d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd7d406e430 0x7fd7d40cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.547+0000 7fd7d259c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd7d40cce00 0x7fd7d4159850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44016/0 (socket says 172.21.15.12:44016) 2023-10-30T12:56:21.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.547+0000 7fd7d259c640 1 -- 172.21.15.12:0/369776493 learned_addr learned my addr 172.21.15.12:0/369776493 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.551+0000 7fd7cbfff640 1 --2- 172.21.15.12:0/369776493 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7d40c7830 0x7fd7d406ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.551+0000 7fd7d2d9d640 1 -- 172.21.15.12:0/369776493 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7d40c7830 msgr2=0x7fd7d406ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.551+0000 7fd7d2d9d640 1 --2- 172.21.15.12:0/369776493 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7d40c7830 0x7fd7d406ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.551+0000 7fd7d2d9d640 1 -- 172.21.15.12:0/369776493 >> v1:172.21.15.12:6789/0 conn(0x7fd7d40cce00 legacy=0x7fd7d4159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.551+0000 7fd7d2d9d640 1 -- 172.21.15.12:0/369776493 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d406b560 con 0x7fd7d406e430 2023-10-30T12:56:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.551+0000 7fd7d1d9b640 1 -- 172.21.15.12:0/369776493 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551368230 0 0) 0x7fd7d406d710 con 0x7fd7d40cce00 2023-10-30T12:56:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.551+0000 7fd7d2d9d640 1 --2- 172.21.15.12:0/369776493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d406e430 0x7fd7d40cd1e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009d90 tx=0x7fd7cc01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2eeea955b58c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.551+0000 7fd7d1d9b640 1 -- 172.21.15.12:0/369776493 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d415a770 con 0x7fd7d406e430 2023-10-30T12:56:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.551+0000 7fd7d1d9b640 1 -- 172.21.15.12:0/369776493 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7cc0a5810 con 0x7fd7d406e430 2023-10-30T12:56:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.551+0000 7fd7d1d9b640 1 -- 172.21.15.12:0/369776493 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0a5ba0 con 0x7fd7d406e430 2023-10-30T12:56:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.551+0000 7fd7d8f2a640 1 -- 172.21.15.12:0/369776493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d406e430 msgr2=0x7fd7d40cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.551+0000 7fd7d8f2a640 1 --2- 172.21.15.12:0/369776493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d406e430 0x7fd7d40cd1e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009d90 tx=0x7fd7cc01c950 comp rx=0 tx=0).stop 2023-10-30T12:56:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d8f2a640 1 -- 172.21.15.12:0/369776493 shutdown_connections 2023-10-30T12:56:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d8f2a640 1 --2- 172.21.15.12:0/369776493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d406e430 0x7fd7d40cd1e0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d8f2a640 1 --2- 172.21.15.12:0/369776493 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7d40c7830 0x7fd7d406ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d8f2a640 1 -- 172.21.15.12:0/369776493 >> 172.21.15.12:0/369776493 conn(0x7fd7d4071ea0 msgr2=0x7fd7d4065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d8f2a640 1 -- 172.21.15.12:0/369776493 shutdown_connections 2023-10-30T12:56:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d8f2a640 1 -- 172.21.15.12:0/369776493 wait complete. 2023-10-30T12:56:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d8f2a640 1 Processor -- start 2023-10-30T12:56:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d8f2a640 1 -- start start 2023-10-30T12:56:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d8f2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d406e430 0x7fd7d40c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d8f2a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7d40c7830 0x7fd7d41596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d8f2a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7d40cce00 0x7fd7d4152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d8f2a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d4158ff0 con 0x7fd7d406e430 2023-10-30T12:56:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d8f2a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d4158840 con 0x7fd7d40cce00 2023-10-30T12:56:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d8f2a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d40c6f00 con 0x7fd7d40c7830 2023-10-30T12:56:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d406e430 0x7fd7d40c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d406e430 0x7fd7d40c67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50672/0 (socket says 172.21.15.12:50672) 2023-10-30T12:56:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d259c640 1 -- 172.21.15.12:0/2086783192 learned_addr learned my addr 172.21.15.12:0/2086783192 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d2d9d640 1 --2- 172.21.15.12:0/2086783192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7d40cce00 0x7fd7d4152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7cbfff640 1 --2- 172.21.15.12:0/2086783192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7d40c7830 0x7fd7d41596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d259c640 1 -- 172.21.15.12:0/2086783192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7d40c7830 msgr2=0x7fd7d41596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d259c640 1 --2- 172.21.15.12:0/2086783192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7d40c7830 0x7fd7d41596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d259c640 1 -- 172.21.15.12:0/2086783192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7d40cce00 msgr2=0x7fd7d4152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d259c640 1 --2- 172.21.15.12:0/2086783192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7d40cce00 0x7fd7d4152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d259c640 1 -- 172.21.15.12:0/2086783192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc009a70 con 0x7fd7d406e430 2023-10-30T12:56:21.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.555+0000 7fd7d259c640 1 --2- 172.21.15.12:0/2086783192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d406e430 0x7fd7d40c67d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40011f0 tx=0x7fd7c400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:21.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.559+0000 7fd7caffd640 1 -- 172.21.15.12:0/2086783192 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40074e0 con 0x7fd7d406e430 2023-10-30T12:56:21.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.559+0000 7fd7caffd640 1 -- 172.21.15.12:0/2086783192 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c4007640 con 0x7fd7d406e430 2023-10-30T12:56:21.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.559+0000 7fd7caffd640 1 -- 172.21.15.12:0/2086783192 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4004a10 con 0x7fd7d406e430 2023-10-30T12:56:21.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.559+0000 7fd7d8f2a640 1 -- 172.21.15.12:0/2086783192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d40cb430 con 0x7fd7d406e430 2023-10-30T12:56:21.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.559+0000 7fd7d8f2a640 1 -- 172.21.15.12:0/2086783192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d41f5ce0 con 0x7fd7d406e430 2023-10-30T12:56:21.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.559+0000 7fd7d8f2a640 1 -- 172.21.15.12:0/2086783192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d404e170 con 0x7fd7d406e430 2023-10-30T12:56:21.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.563+0000 7fd7caffd640 1 -- 172.21.15.12:0/2086783192 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd7c4004b70 con 0x7fd7d406e430 2023-10-30T12:56:21.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.563+0000 7fd7caffd640 1 --2- 172.21.15.12:0/2086783192 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd7b40664a0 0x7fd7b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.563+0000 7fd7caffd640 1 -- 172.21.15.12:0/2086783192 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fd7c40835b0 con 0x7fd7d406e430 2023-10-30T12:56:21.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.563+0000 7fd7cbfff640 1 --2- 172.21.15.12:0/2086783192 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd7b40664a0 0x7fd7b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.563+0000 7fd7cbfff640 1 --2- 172.21.15.12:0/2086783192 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd7b40664a0 0x7fd7b4068960 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd7d415ce80 tx=0x7fd7bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:21.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.567+0000 7fd7caffd640 1 -- 172.21.15.12:0/2086783192 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd7c40505c0 con 0x7fd7d406e430 2023-10-30T12:56:21.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.743+0000 7fd7d8f2a640 1 -- 172.21.15.12:0/2086783192 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7d406b0d0 con 0x7fd7b40664a0 2023-10-30T12:56:21.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.743+0000 7fd7caffd640 1 -- 172.21.15.12:0/2086783192 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd7bc00b670 con 0x7fd7b40664a0 2023-10-30T12:56:21.747 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 7 to deep-scrub 2023-10-30T12:56:21.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.747+0000 7fd7d8f2a640 1 -- 172.21.15.12:0/2086783192 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd7b40664a0 msgr2=0x7fd7b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.747+0000 7fd7d8f2a640 1 --2- 172.21.15.12:0/2086783192 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd7b40664a0 0x7fd7b4068960 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd7d415ce80 tx=0x7fd7bc008040 comp rx=0 tx=0).stop 2023-10-30T12:56:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.747+0000 7fd7d8f2a640 1 -- 172.21.15.12:0/2086783192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d406e430 msgr2=0x7fd7d40c67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.747+0000 7fd7d8f2a640 1 --2- 172.21.15.12:0/2086783192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d406e430 0x7fd7d40c67d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40011f0 tx=0x7fd7c400c9b0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.747+0000 7fd7d8f2a640 1 -- 172.21.15.12:0/2086783192 shutdown_connections 2023-10-30T12:56:21.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.747+0000 7fd7d8f2a640 1 --2- 172.21.15.12:0/2086783192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7d40cce00 0x7fd7d4152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.747+0000 7fd7d8f2a640 1 --2- 172.21.15.12:0/2086783192 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fd7b40664a0 0x7fd7b4068960 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.747+0000 7fd7d8f2a640 1 --2- 172.21.15.12:0/2086783192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7d40c7830 0x7fd7d41596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.747+0000 7fd7d8f2a640 1 --2- 172.21.15.12:0/2086783192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d406e430 0x7fd7d40c67d0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.747+0000 7fd7d8f2a640 1 -- 172.21.15.12:0/2086783192 >> 172.21.15.12:0/2086783192 conn(0x7fd7d4071ea0 msgr2=0x7fd7d40c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:21.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.747+0000 7fd7d8f2a640 1 -- 172.21.15.12:0/2086783192 shutdown_connections 2023-10-30T12:56:21.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.747+0000 7fd7d8f2a640 1 -- 172.21.15.12:0/2086783192 wait complete. 2023-10-30T12:56:21.770 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:56:21.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.911+0000 7fb0255d3640 1 Processor -- start 2023-10-30T12:56:21.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.911+0000 7fb0255d3640 1 -- start start 2023-10-30T12:56:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.911+0000 7fb0255d3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb0201538a0 0x7fb02015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.911+0000 7fb0255d3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0200cb800 con 0x7fb0200ca0e0 2023-10-30T12:56:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.911+0000 7fb0255d3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb02015dc30 con 0x7fb0200cf160 2023-10-30T12:56:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.911+0000 7fb0255d3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0200cb330 con 0x7fb0201538a0 2023-10-30T12:56:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.915+0000 7fb01effd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb0201538a0 0x7fb02015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.915+0000 7fb01effd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb0201538a0 0x7fb02015d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50688/0 (socket says 172.21.15.12:50688) 2023-10-30T12:56:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.915+0000 7fb017fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb0200ca0e0 0x7fb02015a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44022/0 (socket says 172.21.15.12:44022) 2023-10-30T12:56:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.915+0000 7fb01effd640 1 -- 172.21.15.12:0/1391881928 learned_addr learned my addr 172.21.15.12:0/1391881928 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.915+0000 7fb01effd640 1 -- 172.21.15.12:0/1391881928 >> v1:172.21.15.154:6789/0 conn(0x7fb0200cf160 legacy=0x7fb020164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.915+0000 7fb01effd640 1 -- 172.21.15.12:0/1391881928 >> v1:172.21.15.12:6789/0 conn(0x7fb0200ca0e0 legacy=0x7fb02015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.915+0000 7fb01e7fc640 1 -- 172.21.15.12:0/1391881928 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249338459 0 0) 0x7fb0200cb800 con 0x7fb0200ca0e0 2023-10-30T12:56:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.915+0000 7fb01effd640 1 -- 172.21.15.12:0/1391881928 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0200cad70 con 0x7fb0201538a0 2023-10-30T12:56:21.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.915+0000 7fb01effd640 1 --2- 172.21.15.12:0/1391881928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0201538a0 0x7fb02015d6f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb010009870 tx=0x7fb01001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1465969fe1a7dcb server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:21.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.915+0000 7fb01e7fc640 1 -- 172.21.15.12:0/1391881928 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0100a8a30 con 0x7fb0201538a0 2023-10-30T12:56:21.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.915+0000 7fb01e7fc640 1 -- 172.21.15.12:0/1391881928 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0100a8b90 con 0x7fb0201538a0 2023-10-30T12:56:21.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.915+0000 7fb01e7fc640 1 -- 172.21.15.12:0/1391881928 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0100b0440 con 0x7fb0201538a0 2023-10-30T12:56:21.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb0255d3640 1 -- 172.21.15.12:0/1391881928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0201538a0 msgr2=0x7fb02015d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb0255d3640 1 --2- 172.21.15.12:0/1391881928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0201538a0 0x7fb02015d6f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb010009870 tx=0x7fb01001ed30 comp rx=0 tx=0).stop 2023-10-30T12:56:21.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb0255d3640 1 -- 172.21.15.12:0/1391881928 shutdown_connections 2023-10-30T12:56:21.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb0255d3640 1 --2- 172.21.15.12:0/1391881928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0201538a0 0x7fb02015d6f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb0255d3640 1 -- 172.21.15.12:0/1391881928 >> 172.21.15.12:0/1391881928 conn(0x7fb02006bfe0 msgr2=0x7fb02006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:21.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb0255d3640 1 -- 172.21.15.12:0/1391881928 shutdown_connections 2023-10-30T12:56:21.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb0255d3640 1 -- 172.21.15.12:0/1391881928 wait complete. 2023-10-30T12:56:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb0255d3640 1 Processor -- start 2023-10-30T12:56:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb0255d3640 1 -- start start 2023-10-30T12:56:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb0255d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0200ca0e0 0x7fb0200b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb0255d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200cf160 0x7fb020148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb0255d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0201538a0 0x7fb020145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb0255d3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0200b3bc0 con 0x7fb0200cf160 2023-10-30T12:56:21.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb0255d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0201583c0 con 0x7fb0200ca0e0 2023-10-30T12:56:21.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb0255d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0200b3df0 con 0x7fb0201538a0 2023-10-30T12:56:21.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb017fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200cf160 0x7fb020148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb017fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200cf160 0x7fb020148ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50694/0 (socket says 172.21.15.12:50694) 2023-10-30T12:56:21.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb017fff640 1 -- 172.21.15.12:0/183415570 learned_addr learned my addr 172.21.15.12:0/183415570 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:21.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb01effd640 1 --2- 172.21.15.12:0/183415570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0200ca0e0 0x7fb0200b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb01f7fe640 1 --2- 172.21.15.12:0/183415570 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0201538a0 0x7fb020145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb017fff640 1 -- 172.21.15.12:0/183415570 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0201538a0 msgr2=0x7fb020145c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb017fff640 1 --2- 172.21.15.12:0/183415570 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0201538a0 0x7fb020145c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb017fff640 1 -- 172.21.15.12:0/183415570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0200ca0e0 msgr2=0x7fb0200b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:21.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb017fff640 1 --2- 172.21.15.12:0/183415570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0200ca0e0 0x7fb0200b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.919+0000 7fb017fff640 1 -- 172.21.15.12:0/183415570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0100094d0 con 0x7fb0200cf160 2023-10-30T12:56:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.923+0000 7fb017fff640 1 --2- 172.21.15.12:0/183415570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200cf160 0x7fb020148ef0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c410 tx=0x7fb00800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.923+0000 7fb0177fe640 1 -- 172.21.15.12:0/183415570 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0080074e0 con 0x7fb0200cf160 2023-10-30T12:56:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.923+0000 7fb0177fe640 1 -- 172.21.15.12:0/183415570 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb008007640 con 0x7fb0200cf160 2023-10-30T12:56:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.923+0000 7fb0255d3640 1 -- 172.21.15.12:0/183415570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb020159290 con 0x7fb0200cf160 2023-10-30T12:56:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.923+0000 7fb0177fe640 1 -- 172.21.15.12:0/183415570 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb008004a30 con 0x7fb0200cf160 2023-10-30T12:56:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.923+0000 7fb0255d3640 1 -- 172.21.15.12:0/183415570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0200cec70 con 0x7fb0200cf160 2023-10-30T12:56:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.923+0000 7fb0255d3640 1 -- 172.21.15.12:0/183415570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0200cf630 con 0x7fb0200cf160 2023-10-30T12:56:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.923+0000 7fb0177fe640 1 -- 172.21.15.12:0/183415570 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb008004b90 con 0x7fb0200cf160 2023-10-30T12:56:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.923+0000 7fb0177fe640 1 --2- 172.21.15.12:0/183415570 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb0000664d0 0x7fb000068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:21.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.923+0000 7fb0177fe640 1 -- 172.21.15.12:0/183415570 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fb008083ff0 con 0x7fb0200cf160 2023-10-30T12:56:21.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.927+0000 7fb01effd640 1 --2- 172.21.15.12:0/183415570 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb0000664d0 0x7fb000068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:21.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.927+0000 7fb01effd640 1 --2- 172.21.15.12:0/183415570 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb0000664d0 0x7fb000068990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb0100b2000 tx=0x7fb0100aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:21.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:21.927+0000 7fb0177fe640 1 -- 172.21.15.12:0/183415570 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb008051090 con 0x7fb0200cf160 2023-10-30T12:56:22.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:22.083+0000 7fb0255d3640 1 -- 172.21.15.12:0/183415570 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb020159890 con 0x7fb0000664d0 2023-10-30T12:56:22.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:22.107+0000 7fb0177fe640 1 -- 172.21.15.12:0/183415570 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292607 (secure 0 0 0) 0x7fb0100b04c0 con 0x7fb0000664d0 2023-10-30T12:56:22.113 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:56:22.114 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-10-30T12:56:22.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:22.111+0000 7fb0255d3640 1 -- 172.21.15.12:0/183415570 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb0000664d0 msgr2=0x7fb000068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:22.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:22.111+0000 7fb0255d3640 1 --2- 172.21.15.12:0/183415570 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb0000664d0 0x7fb000068990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb0100b2000 tx=0x7fb0100aa070 comp rx=0 tx=0).stop 2023-10-30T12:56:22.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:22.111+0000 7fb0255d3640 1 -- 172.21.15.12:0/183415570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200cf160 msgr2=0x7fb020148ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:22.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:22.111+0000 7fb0255d3640 1 --2- 172.21.15.12:0/183415570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200cf160 0x7fb020148ef0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c410 tx=0x7fb00800c8d0 comp rx=0 tx=0).stop 2023-10-30T12:56:22.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:22.111+0000 7fb0255d3640 1 -- 172.21.15.12:0/183415570 shutdown_connections 2023-10-30T12:56:22.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:22.111+0000 7fb0255d3640 1 --2- 172.21.15.12:0/183415570 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fb0000664d0 0x7fb000068990 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:22.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:22.111+0000 7fb0255d3640 1 --2- 172.21.15.12:0/183415570 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0201538a0 0x7fb020145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:22.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:22.111+0000 7fb0255d3640 1 --2- 172.21.15.12:0/183415570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200cf160 0x7fb020148ef0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:22.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:22.111+0000 7fb0255d3640 1 --2- 172.21.15.12:0/183415570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0200ca0e0 0x7fb0200b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:22.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:22.111+0000 7fb0255d3640 1 -- 172.21.15.12:0/183415570 >> 172.21.15.12:0/183415570 conn(0x7fb02006bfe0 msgr2=0x7fb02006c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:22.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:22.111+0000 7fb0255d3640 1 -- 172.21.15.12:0/183415570 shutdown_connections 2023-10-30T12:56:22.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:22.111+0000 7fb0255d3640 1 -- 172.21.15.12:0/183415570 wait complete. 2023-10-30T12:56:22.133 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":693,"stamp":"2023-10-30T12:56:21.411880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1284244645,"num_objects":10514,"num_object_clones":0,"num_object_copies":21028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10514,"num_whiteouts":0,"num_read":5687,"num_read_kb":1040706,"num_write":53212,"num_write_kb":2313421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57778,"ondisk_log_size":57778,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3230032,"kb_used_data":2564304,"kb_used_omap":12,"kb_used_meta":665651,"kb_avail":751744688,"statfs":{"total":773094113280,"available":769786560512,"internally_reserved":0,"allocated":2625847296,"data_stored":2569419578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12729,"internal_metadata":681627207},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":130,"apply_latency_ms":130,"commit_latency_ns":130000000,"apply_latency_ns":130000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32474072,"num_objects":1187,"num_object_clones":0,"num_object_copies":2374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1321,"num_write_kb":32431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004398"},"pg_stats":[{"pgid":"3.3f","version":"55'737","reported_seq":2902,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.185011+0000","last_change":"2023-10-30T12:42:02.472363+0000","last_active":"2023-10-30T12:56:14.185011+0000","last_peered":"2023-10-30T12:56:14.185011+0000","last_clean":"2023-10-30T12:56:14.185011+0000","last_became_active":"2023-10-30T12:35:06.222724+0000","last_became_peered":"2023-10-30T12:35:06.222724+0000","last_unstale":"2023-10-30T12:56:14.185011+0000","last_undegraded":"2023-10-30T12:56:14.185011+0000","last_fullsized":"2023-10-30T12:56:14.185011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:27:39.311531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059524979999999996,"stat_sum":{"num_bytes":14421277,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":73,"num_read_kb":8207,"num_write":692,"num_write_kb":25708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"55'711","reported_seq":2873,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.163164+0000","last_change":"2023-10-30T12:42:02.711331+0000","last_active":"2023-10-30T12:56:14.163164+0000","last_peered":"2023-10-30T12:56:14.163164+0000","last_clean":"2023-10-30T12:56:14.163164+0000","last_became_active":"2023-10-30T12:35:06.182441+0000","last_became_peered":"2023-10-30T12:35:06.182441+0000","last_unstale":"2023-10-30T12:56:14.163164+0000","last_undegraded":"2023-10-30T12:56:14.163164+0000","last_fullsized":"2023-10-30T12:56:14.163164+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:00:23.194349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026942979999999999,"stat_sum":{"num_bytes":14617239,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":70,"num_read_kb":10,"num_write":675,"num_write_kb":36983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"55'697","reported_seq":2869,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.157961+0000","last_change":"2023-10-30T12:42:02.724489+0000","last_active":"2023-10-30T12:56:14.157961+0000","last_peered":"2023-10-30T12:56:14.157961+0000","last_clean":"2023-10-30T12:56:14.157961+0000","last_became_active":"2023-10-30T12:35:06.180132+0000","last_became_peered":"2023-10-30T12:35:06.180132+0000","last_unstale":"2023-10-30T12:56:14.157961+0000","last_undegraded":"2023-10-30T12:56:14.157961+0000","last_fullsized":"2023-10-30T12:56:14.157961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:12.922855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003537278,"stat_sum":{"num_bytes":22250243,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":80,"num_read_kb":12305,"num_write":653,"num_write_kb":42611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"55'682","reported_seq":2831,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.195228+0000","last_change":"2023-10-30T12:42:02.742917+0000","last_active":"2023-10-30T12:56:14.195228+0000","last_peered":"2023-10-30T12:56:14.195228+0000","last_clean":"2023-10-30T12:56:14.195228+0000","last_became_active":"2023-10-30T12:35:06.213987+0000","last_became_peered":"2023-10-30T12:35:06.213987+0000","last_unstale":"2023-10-30T12:56:14.195228+0000","last_undegraded":"2023-10-30T12:56:14.195228+0000","last_fullsized":"2023-10-30T12:56:14.195228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:39:55.675219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013528057,"stat_sum":{"num_bytes":18553875,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":57,"num_read_kb":8205,"num_write":627,"num_write_kb":28534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"55'670","reported_seq":2811,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.183935+0000","last_change":"2023-10-30T12:42:02.470142+0000","last_active":"2023-10-30T12:56:14.183935+0000","last_peered":"2023-10-30T12:56:14.183935+0000","last_clean":"2023-10-30T12:56:14.183935+0000","last_became_active":"2023-10-30T12:35:06.214964+0000","last_became_peered":"2023-10-30T12:35:06.214964+0000","last_unstale":"2023-10-30T12:56:14.183935+0000","last_undegraded":"2023-10-30T12:56:14.183935+0000","last_fullsized":"2023-10-30T12:56:14.183935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:56:51.594442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046834600000000004,"stat_sum":{"num_bytes":8686005,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":51,"num_read_kb":4112,"num_write":616,"num_write_kb":25845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"55'687","reported_seq":2838,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.194746+0000","last_change":"2023-10-30T12:42:02.747177+0000","last_active":"2023-10-30T12:56:14.194746+0000","last_peered":"2023-10-30T12:56:14.194746+0000","last_clean":"2023-10-30T12:56:14.194746+0000","last_became_active":"2023-10-30T12:35:06.214219+0000","last_became_peered":"2023-10-30T12:35:06.214219+0000","last_unstale":"2023-10-30T12:56:14.194746+0000","last_undegraded":"2023-10-30T12:56:14.194746+0000","last_fullsized":"2023-10-30T12:56:14.194746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:36:56.070585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015862434000000002,"stat_sum":{"num_bytes":14811547,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":61,"num_read_kb":16,"num_write":643,"num_write_kb":33253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"55'697","reported_seq":2838,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.171468+0000","last_change":"2023-10-30T12:42:02.467959+0000","last_active":"2023-10-30T12:56:14.171468+0000","last_peered":"2023-10-30T12:56:14.171468+0000","last_clean":"2023-10-30T12:56:14.171468+0000","last_became_active":"2023-10-30T12:35:06.216468+0000","last_became_peered":"2023-10-30T12:35:06.216468+0000","last_unstale":"2023-10-30T12:56:14.171468+0000","last_undegraded":"2023-10-30T12:56:14.171468+0000","last_fullsized":"2023-10-30T12:56:14.171468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:19:43.632233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031309480000000002,"stat_sum":{"num_bytes":9595467,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":52,"num_read_kb":12216,"num_write":649,"num_write_kb":36101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"55'625","reported_seq":2771,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.176232+0000","last_change":"2023-10-30T12:42:02.241155+0000","last_active":"2023-10-30T12:56:14.176232+0000","last_peered":"2023-10-30T12:56:14.176232+0000","last_clean":"2023-10-30T12:56:14.176232+0000","last_became_active":"2023-10-30T12:35:06.214384+0000","last_became_peered":"2023-10-30T12:35:06.214384+0000","last_unstale":"2023-10-30T12:56:14.176232+0000","last_undegraded":"2023-10-30T12:56:14.176232+0000","last_fullsized":"2023-10-30T12:56:14.176232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:44:07.845577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045552559999999997,"stat_sum":{"num_bytes":9825651,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":56,"num_read_kb":15,"num_write":593,"num_write_kb":23392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"55'695","reported_seq":2857,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.185210+0000","last_change":"2023-10-30T12:42:02.473886+0000","last_active":"2023-10-30T12:56:14.185210+0000","last_peered":"2023-10-30T12:56:14.185210+0000","last_clean":"2023-10-30T12:56:14.185210+0000","last_became_active":"2023-10-30T12:35:06.222692+0000","last_became_peered":"2023-10-30T12:35:06.222692+0000","last_unstale":"2023-10-30T12:56:14.185210+0000","last_undegraded":"2023-10-30T12:56:14.185210+0000","last_fullsized":"2023-10-30T12:56:14.185210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:12:07.552579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074617599999999996,"stat_sum":{"num_bytes":5524945,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":72,"num_read_kb":4115,"num_write":631,"num_write_kb":10887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"55'686","reported_seq":2840,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.176137+0000","last_change":"2023-10-30T12:42:02.739661+0000","last_active":"2023-10-30T12:56:14.176137+0000","last_peered":"2023-10-30T12:56:14.176137+0000","last_clean":"2023-10-30T12:56:14.176137+0000","last_became_active":"2023-10-30T12:35:06.215384+0000","last_became_peered":"2023-10-30T12:35:06.215384+0000","last_unstale":"2023-10-30T12:56:14.176137+0000","last_undegraded":"2023-10-30T12:56:14.176137+0000","last_fullsized":"2023-10-30T12:56:14.176137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:02:59.748923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080059940000000007,"stat_sum":{"num_bytes":17915594,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":64,"num_read_kb":28,"num_write":636,"num_write_kb":19744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"55'698","reported_seq":2855,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.180657+0000","last_change":"2023-10-30T12:42:02.739462+0000","last_active":"2023-10-30T12:56:14.180657+0000","last_peered":"2023-10-30T12:56:14.180657+0000","last_clean":"2023-10-30T12:56:14.180657+0000","last_became_active":"2023-10-30T12:35:06.215355+0000","last_became_peered":"2023-10-30T12:35:06.215355+0000","last_unstale":"2023-10-30T12:56:14.180657+0000","last_undegraded":"2023-10-30T12:56:14.180657+0000","last_fullsized":"2023-10-30T12:56:14.180657+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:04:51.314044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011778769,"stat_sum":{"num_bytes":10475791,"num_objects":135,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":67,"num_read_kb":8209,"num_write":651,"num_write_kb":25832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"55'680","reported_seq":2835,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.190803+0000","last_change":"2023-10-30T12:42:02.253484+0000","last_active":"2023-10-30T12:56:14.190803+0000","last_peered":"2023-10-30T12:56:14.190803+0000","last_clean":"2023-10-30T12:56:14.190803+0000","last_became_active":"2023-10-30T12:35:06.196503+0000","last_became_peered":"2023-10-30T12:35:06.196503+0000","last_unstale":"2023-10-30T12:56:14.190803+0000","last_undegraded":"2023-10-30T12:56:14.190803+0000","last_fullsized":"2023-10-30T12:56:14.190803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:33:54.826649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030385149999999999,"stat_sum":{"num_bytes":16276064,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":65,"num_read_kb":7648,"num_write":632,"num_write_kb":36085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"55'724","reported_seq":2891,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.145378+0000","last_change":"2023-10-30T12:42:02.241621+0000","last_active":"2023-10-30T12:56:14.145378+0000","last_peered":"2023-10-30T12:56:14.145378+0000","last_clean":"2023-10-30T12:56:14.145378+0000","last_became_active":"2023-10-30T12:35:06.195408+0000","last_became_peered":"2023-10-30T12:35:06.195408+0000","last_unstale":"2023-10-30T12:56:14.145378+0000","last_undegraded":"2023-10-30T12:56:14.145378+0000","last_fullsized":"2023-10-30T12:56:14.145378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:06:33.968324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004560812,"stat_sum":{"num_bytes":30579039,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":77,"num_read_kb":9808,"num_write":665,"num_write_kb":49630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"55'687","reported_seq":2849,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.162474+0000","last_change":"2023-10-30T12:42:02.722894+0000","last_active":"2023-10-30T12:56:14.162474+0000","last_peered":"2023-10-30T12:56:14.162474+0000","last_clean":"2023-10-30T12:56:14.162474+0000","last_became_active":"2023-10-30T12:35:06.182573+0000","last_became_peered":"2023-10-30T12:35:06.182573+0000","last_unstale":"2023-10-30T12:56:14.162474+0000","last_undegraded":"2023-10-30T12:56:14.162474+0000","last_fullsized":"2023-10-30T12:56:14.162474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:42:24.313362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003020062,"stat_sum":{"num_bytes":14557524,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":71,"num_read_kb":4113,"num_write":642,"num_write_kb":31601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"55'700","reported_seq":2854,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.169529+0000","last_change":"2023-10-30T12:42:02.710474+0000","last_active":"2023-10-30T12:56:14.169529+0000","last_peered":"2023-10-30T12:56:14.169529+0000","last_clean":"2023-10-30T12:56:14.169529+0000","last_became_active":"2023-10-30T12:35:06.185290+0000","last_became_peered":"2023-10-30T12:35:06.185290+0000","last_unstale":"2023-10-30T12:56:14.169529+0000","last_undegraded":"2023-10-30T12:56:14.169529+0000","last_fullsized":"2023-10-30T12:56:14.169529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:18:23.116561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002282292,"stat_sum":{"num_bytes":14763187,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":63,"num_read_kb":11730,"num_write":643,"num_write_kb":27692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"55'718","reported_seq":2868,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.187519+0000","last_change":"2023-10-30T12:42:02.748073+0000","last_active":"2023-10-30T12:56:14.187519+0000","last_peered":"2023-10-30T12:56:14.187519+0000","last_clean":"2023-10-30T12:56:14.187519+0000","last_became_active":"2023-10-30T12:35:06.213627+0000","last_became_peered":"2023-10-30T12:35:06.213627+0000","last_unstale":"2023-10-30T12:56:14.187519+0000","last_undegraded":"2023-10-30T12:56:14.187519+0000","last_fullsized":"2023-10-30T12:56:14.187519+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:56:05.792085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014095760000000001,"stat_sum":{"num_bytes":9839966,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":59,"num_read_kb":4114,"num_write":651,"num_write_kb":31705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"55'739","reported_seq":2892,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.185459+0000","last_change":"2023-10-30T12:42:02.745783+0000","last_active":"2023-10-30T12:56:14.185459+0000","last_peered":"2023-10-30T12:56:14.185459+0000","last_clean":"2023-10-30T12:56:14.185459+0000","last_became_active":"2023-10-30T12:35:06.221696+0000","last_became_peered":"2023-10-30T12:35:06.221696+0000","last_unstale":"2023-10-30T12:56:14.185459+0000","last_undegraded":"2023-10-30T12:56:14.185459+0000","last_fullsized":"2023-10-30T12:56:14.185459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:26:40.837242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015743672,"stat_sum":{"num_bytes":28343169,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":62,"num_read_kb":16401,"num_write":691,"num_write_kb":59410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"55'690","reported_seq":2863,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.146431+0000","last_change":"2023-10-30T12:42:02.242356+0000","last_active":"2023-10-30T12:56:14.146431+0000","last_peered":"2023-10-30T12:56:14.146431+0000","last_clean":"2023-10-30T12:56:14.146431+0000","last_became_active":"2023-10-30T12:35:06.198074+0000","last_became_peered":"2023-10-30T12:35:06.198074+0000","last_unstale":"2023-10-30T12:56:14.146431+0000","last_undegraded":"2023-10-30T12:56:14.146431+0000","last_fullsized":"2023-10-30T12:56:14.146431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:55:38.021093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023389270000000002,"stat_sum":{"num_bytes":28976684,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":82,"num_read_kb":27,"num_write":642,"num_write_kb":39937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"55'706","reported_seq":2857,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.183869+0000","last_change":"2023-10-30T12:42:02.473536+0000","last_active":"2023-10-30T12:56:14.183869+0000","last_peered":"2023-10-30T12:56:14.183869+0000","last_clean":"2023-10-30T12:56:14.183869+0000","last_became_active":"2023-10-30T12:35:06.215935+0000","last_became_peered":"2023-10-30T12:35:06.215935+0000","last_unstale":"2023-10-30T12:56:14.183869+0000","last_undegraded":"2023-10-30T12:56:14.183869+0000","last_fullsized":"2023-10-30T12:56:14.183869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:39:23.851788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062537060000000004,"stat_sum":{"num_bytes":21535467,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":60,"num_read_kb":12305,"num_write":656,"num_write_kb":39478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"55'709","reported_seq":2875,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.188294+0000","last_change":"2023-10-30T12:42:02.739007+0000","last_active":"2023-10-30T12:56:14.188294+0000","last_peered":"2023-10-30T12:56:14.188294+0000","last_clean":"2023-10-30T12:56:14.188294+0000","last_became_active":"2023-10-30T12:35:06.199812+0000","last_became_peered":"2023-10-30T12:35:06.199812+0000","last_unstale":"2023-10-30T12:56:14.188294+0000","last_undegraded":"2023-10-30T12:56:14.188294+0000","last_fullsized":"2023-10-30T12:56:14.188294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:07:11.762372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011375481999999999,"stat_sum":{"num_bytes":6303882,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":75,"num_read_kb":20150,"num_write":647,"num_write_kb":36292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"55'719","reported_seq":2888,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.183190+0000","last_change":"2023-10-30T12:42:02.725299+0000","last_active":"2023-10-30T12:56:14.183190+0000","last_peered":"2023-10-30T12:56:14.183190+0000","last_clean":"2023-10-30T12:56:14.183190+0000","last_became_active":"2023-10-30T12:35:06.216861+0000","last_became_peered":"2023-10-30T12:35:06.216861+0000","last_unstale":"2023-10-30T12:56:14.183190+0000","last_undegraded":"2023-10-30T12:56:14.183190+0000","last_fullsized":"2023-10-30T12:56:14.183190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:58:16.568819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040841849999999997,"stat_sum":{"num_bytes":12370338,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":78,"num_read_kb":8213,"num_write":666,"num_write_kb":26377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"55'736","reported_seq":2885,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.164729+0000","last_change":"2023-10-30T12:42:02.241419+0000","last_active":"2023-10-30T12:56:14.164729+0000","last_peered":"2023-10-30T12:56:14.164729+0000","last_clean":"2023-10-30T12:56:14.164729+0000","last_became_active":"2023-10-30T12:35:46.122517+0000","last_became_peered":"2023-10-30T12:35:46.122517+0000","last_unstale":"2023-10-30T12:56:14.164729+0000","last_undegraded":"2023-10-30T12:56:14.164729+0000","last_fullsized":"2023-10-30T12:56:14.164729+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:10:07.253977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020435240000000001,"stat_sum":{"num_bytes":19531616,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":61,"num_read_kb":21,"num_write":688,"num_write_kb":30111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"55'701","reported_seq":2872,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.162902+0000","last_change":"2023-10-30T12:42:02.241859+0000","last_active":"2023-10-30T12:56:14.162902+0000","last_peered":"2023-10-30T12:56:14.162902+0000","last_clean":"2023-10-30T12:56:14.162902+0000","last_became_active":"2023-10-30T12:35:06.182314+0000","last_became_peered":"2023-10-30T12:35:06.182314+0000","last_unstale":"2023-10-30T12:56:14.162902+0000","last_undegraded":"2023-10-30T12:56:14.162902+0000","last_fullsized":"2023-10-30T12:56:14.162902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:20:19.153859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051111200000000002,"stat_sum":{"num_bytes":5467082,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":80,"num_read_kb":15696,"num_write":658,"num_write_kb":25502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"55'714","reported_seq":2872,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.188179+0000","last_change":"2023-10-30T12:42:02.748079+0000","last_active":"2023-10-30T12:56:14.188179+0000","last_peered":"2023-10-30T12:56:14.188179+0000","last_clean":"2023-10-30T12:56:14.188179+0000","last_became_active":"2023-10-30T12:35:06.213402+0000","last_became_peered":"2023-10-30T12:35:06.213402+0000","last_unstale":"2023-10-30T12:56:14.188179+0000","last_undegraded":"2023-10-30T12:56:14.188179+0000","last_fullsized":"2023-10-30T12:56:14.188179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:11.732115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014055767,"stat_sum":{"num_bytes":10241279,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":67,"num_read_kb":4112,"num_write":657,"num_write_kb":20530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"55'689","reported_seq":2839,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.147222+0000","last_change":"2023-10-30T12:42:02.242578+0000","last_active":"2023-10-30T12:56:14.147222+0000","last_peered":"2023-10-30T12:56:14.147222+0000","last_clean":"2023-10-30T12:56:14.147222+0000","last_became_active":"2023-10-30T12:35:06.188298+0000","last_became_peered":"2023-10-30T12:35:06.188298+0000","last_unstale":"2023-10-30T12:56:14.147222+0000","last_undegraded":"2023-10-30T12:56:14.147222+0000","last_fullsized":"2023-10-30T12:56:14.147222+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:56:11.006974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053580889999999999,"stat_sum":{"num_bytes":5087264,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":59,"num_read_kb":8204,"num_write":621,"num_write_kb":44181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"55'763","reported_seq":2941,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.185263+0000","last_change":"2023-10-30T12:42:02.740157+0000","last_active":"2023-10-30T12:56:14.185263+0000","last_peered":"2023-10-30T12:56:14.185263+0000","last_clean":"2023-10-30T12:56:14.185263+0000","last_became_active":"2023-10-30T12:35:06.202507+0000","last_became_peered":"2023-10-30T12:35:06.202507+0000","last_unstale":"2023-10-30T12:56:14.185263+0000","last_undegraded":"2023-10-30T12:56:14.185263+0000","last_fullsized":"2023-10-30T12:56:14.185263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:55.774699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082500649999999991,"stat_sum":{"num_bytes":6587197,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":87,"num_read_kb":1892,"num_write":715,"num_write_kb":28794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"55'695","reported_seq":2849,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.154548+0000","last_change":"2023-10-30T12:42:02.710762+0000","last_active":"2023-10-30T12:56:14.154548+0000","last_peered":"2023-10-30T12:56:14.154548+0000","last_clean":"2023-10-30T12:56:14.154548+0000","last_became_active":"2023-10-30T12:35:06.170030+0000","last_became_peered":"2023-10-30T12:35:06.170030+0000","last_unstale":"2023-10-30T12:56:14.154548+0000","last_undegraded":"2023-10-30T12:56:14.154548+0000","last_fullsized":"2023-10-30T12:56:14.154548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:04.189974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024132049999999999,"stat_sum":{"num_bytes":15296106,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":63,"num_read_kb":15,"num_write":636,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"55'652","reported_seq":2821,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.180083+0000","last_change":"2023-10-30T12:42:02.242975+0000","last_active":"2023-10-30T12:56:14.180083+0000","last_peered":"2023-10-30T12:56:14.180083+0000","last_clean":"2023-10-30T12:56:14.180083+0000","last_became_active":"2023-10-30T12:35:06.189171+0000","last_became_peered":"2023-10-30T12:35:06.189171+0000","last_unstale":"2023-10-30T12:56:14.180083+0000","last_undegraded":"2023-10-30T12:56:14.180083+0000","last_fullsized":"2023-10-30T12:56:14.180083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:00:28.317415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056904870000000001,"stat_sum":{"num_bytes":14055782,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":78,"num_read_kb":12372,"num_write":607,"num_write_kb":31259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"55'672","reported_seq":2824,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.182217+0000","last_change":"2023-10-30T12:42:02.725392+0000","last_active":"2023-10-30T12:56:14.182217+0000","last_peered":"2023-10-30T12:56:14.182217+0000","last_clean":"2023-10-30T12:56:14.182217+0000","last_became_active":"2023-10-30T12:35:06.183348+0000","last_became_peered":"2023-10-30T12:35:06.183348+0000","last_unstale":"2023-10-30T12:56:14.182217+0000","last_undegraded":"2023-10-30T12:56:14.182217+0000","last_fullsized":"2023-10-30T12:56:14.182217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:15:11.322585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041317979999999999,"stat_sum":{"num_bytes":14615922,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":60,"num_read_kb":7678,"num_write":625,"num_write_kb":36364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"55'742","reported_seq":2909,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.191938+0000","last_change":"2023-10-30T12:42:02.747343+0000","last_active":"2023-10-30T12:56:14.191938+0000","last_peered":"2023-10-30T12:56:14.191938+0000","last_clean":"2023-10-30T12:56:14.191938+0000","last_became_active":"2023-10-30T12:35:06.196368+0000","last_became_peered":"2023-10-30T12:35:06.196368+0000","last_unstale":"2023-10-30T12:56:14.191938+0000","last_undegraded":"2023-10-30T12:56:14.191938+0000","last_fullsized":"2023-10-30T12:56:14.191938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:53:23.269916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015569382999999999,"stat_sum":{"num_bytes":14491906,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":75,"num_read_kb":9352,"num_write":694,"num_write_kb":32749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"55'759","reported_seq":2923,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.151187+0000","last_change":"2023-10-30T12:42:02.241047+0000","last_active":"2023-10-30T12:56:14.151187+0000","last_peered":"2023-10-30T12:56:14.151187+0000","last_clean":"2023-10-30T12:56:14.151187+0000","last_became_active":"2023-10-30T12:35:06.213994+0000","last_became_peered":"2023-10-30T12:35:06.213994+0000","last_unstale":"2023-10-30T12:56:14.151187+0000","last_undegraded":"2023-10-30T12:56:14.151187+0000","last_fullsized":"2023-10-30T12:56:14.151187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:49:43.203436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001873502,"stat_sum":{"num_bytes":23363209,"num_objects":188,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":188,"num_whiteouts":0,"num_read":72,"num_read_kb":4118,"num_write":712,"num_write_kb":32388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"55'701","reported_seq":2863,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.177300+0000","last_change":"2023-10-30T12:42:02.738216+0000","last_active":"2023-10-30T12:56:14.177300+0000","last_peered":"2023-10-30T12:56:14.177300+0000","last_clean":"2023-10-30T12:56:14.177300+0000","last_became_active":"2023-10-30T12:35:06.197647+0000","last_became_peered":"2023-10-30T12:35:06.197647+0000","last_unstale":"2023-10-30T12:56:14.177300+0000","last_undegraded":"2023-10-30T12:56:14.177300+0000","last_fullsized":"2023-10-30T12:56:14.177300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:21:00.884446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011378932,"stat_sum":{"num_bytes":18364745,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":70,"num_read_kb":8212,"num_write":651,"num_write_kb":28257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"55'666","reported_seq":2818,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.158934+0000","last_change":"2023-10-30T12:42:02.251618+0000","last_active":"2023-10-30T12:56:14.158934+0000","last_peered":"2023-10-30T12:56:14.158934+0000","last_clean":"2023-10-30T12:56:14.158934+0000","last_became_active":"2023-10-30T12:35:06.167558+0000","last_became_peered":"2023-10-30T12:35:06.167558+0000","last_unstale":"2023-10-30T12:56:14.158934+0000","last_undegraded":"2023-10-30T12:56:14.158934+0000","last_fullsized":"2023-10-30T12:56:14.158934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:11:30.490780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048907120000000002,"stat_sum":{"num_bytes":9710360,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":62,"num_read_kb":17,"num_write":609,"num_write_kb":15663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"55'233","reported_seq":298,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:07.513824+0000","last_change":"2023-10-30T12:35:05.200844+0000","last_active":"2023-10-30T12:55:07.513824+0000","last_peered":"2023-10-30T12:55:07.513824+0000","last_clean":"2023-10-30T12:55:07.513824+0000","last_became_active":"2023-10-30T12:35:05.200169+0000","last_became_peered":"2023-10-30T12:35:05.200169+0000","last_unstale":"2023-10-30T12:55:07.513824+0000","last_undegraded":"2023-10-30T12:55:07.513824+0000","last_fullsized":"2023-10-30T12:55:07.513824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:07:19.571357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":36867,"num_write":183,"num_write_kb":12445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"55'675","reported_seq":2826,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.182272+0000","last_change":"2023-10-30T12:42:02.725641+0000","last_active":"2023-10-30T12:56:14.182272+0000","last_peered":"2023-10-30T12:56:14.182272+0000","last_clean":"2023-10-30T12:56:14.182272+0000","last_became_active":"2023-10-30T12:35:06.194689+0000","last_became_peered":"2023-10-30T12:35:06.194689+0000","last_unstale":"2023-10-30T12:56:14.182272+0000","last_undegraded":"2023-10-30T12:56:14.182272+0000","last_fullsized":"2023-10-30T12:56:14.182272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:38:22.949407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004291034,"stat_sum":{"num_bytes":10922380,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":61,"num_read_kb":18,"num_write":626,"num_write_kb":24711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"55'975","reported_seq":1166,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:35.771201+0000","last_change":"2023-10-30T12:35:05.183569+0000","last_active":"2023-10-30T12:55:35.771201+0000","last_peered":"2023-10-30T12:55:35.771201+0000","last_clean":"2023-10-30T12:55:35.771201+0000","last_became_active":"2023-10-30T12:35:05.182761+0000","last_became_peered":"2023-10-30T12:35:05.182761+0000","last_unstale":"2023-10-30T12:55:35.771201+0000","last_undegraded":"2023-10-30T12:55:35.771201+0000","last_fullsized":"2023-10-30T12:55:35.771201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:49:44.311814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25625406,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":150,"num_read_kb":45229,"num_write":895,"num_write_kb":25503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"55'453","reported_seq":658,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.700473+0000","last_change":"2023-10-30T12:35:05.181278+0000","last_active":"2023-10-30T12:56:14.700473+0000","last_peered":"2023-10-30T12:56:14.700473+0000","last_clean":"2023-10-30T12:56:14.700473+0000","last_became_active":"2023-10-30T12:35:05.180730+0000","last_became_peered":"2023-10-30T12:35:05.180730+0000","last_unstale":"2023-10-30T12:56:14.700473+0000","last_undegraded":"2023-10-30T12:56:14.700473+0000","last_fullsized":"2023-10-30T12:56:14.700473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:07:10.828707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8777431,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":162,"num_read_kb":8759,"num_write":388,"num_write_kb":8822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"55'785","reported_seq":2954,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.169536+0000","last_change":"2023-10-30T12:42:02.240966+0000","last_active":"2023-10-30T12:56:14.169536+0000","last_peered":"2023-10-30T12:56:14.169536+0000","last_clean":"2023-10-30T12:56:14.169536+0000","last_became_active":"2023-10-30T12:35:46.051057+0000","last_became_peered":"2023-10-30T12:35:46.051057+0000","last_unstale":"2023-10-30T12:56:14.169536+0000","last_undegraded":"2023-10-30T12:56:14.169536+0000","last_fullsized":"2023-10-30T12:56:14.169536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:39:59.687314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018351520000000001,"stat_sum":{"num_bytes":19060908,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":82,"num_read_kb":16408,"num_write":727,"num_write_kb":53009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"55'983","reported_seq":1049,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:04.638735+0000","last_change":"2023-10-30T12:35:05.202539+0000","last_active":"2023-10-30T12:55:04.638735+0000","last_peered":"2023-10-30T12:55:04.638735+0000","last_clean":"2023-10-30T12:55:04.638735+0000","last_became_active":"2023-10-30T12:35:05.201853+0000","last_became_peered":"2023-10-30T12:35:05.201853+0000","last_unstale":"2023-10-30T12:55:04.638735+0000","last_undegraded":"2023-10-30T12:55:04.638735+0000","last_fullsized":"2023-10-30T12:55:04.638735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:33:33.151433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":38325,"num_write":914,"num_write_kb":16753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"55'734","reported_seq":2897,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.158300+0000","last_change":"2023-10-30T12:42:02.710378+0000","last_active":"2023-10-30T12:56:14.158300+0000","last_peered":"2023-10-30T12:56:14.158300+0000","last_clean":"2023-10-30T12:56:14.158300+0000","last_became_active":"2023-10-30T12:35:06.170179+0000","last_became_peered":"2023-10-30T12:35:06.170179+0000","last_unstale":"2023-10-30T12:56:14.158300+0000","last_undegraded":"2023-10-30T12:56:14.158300+0000","last_fullsized":"2023-10-30T12:56:14.158300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:21:27.326814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021794980000000002,"stat_sum":{"num_bytes":14292082,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":73,"num_read_kb":12307,"num_write":687,"num_write_kb":28727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"55'145","reported_seq":204,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:54.873264+0000","last_change":"2023-10-30T12:35:05.198546+0000","last_active":"2023-10-30T12:55:54.873264+0000","last_peered":"2023-10-30T12:55:54.873264+0000","last_clean":"2023-10-30T12:55:54.873264+0000","last_became_active":"2023-10-30T12:35:05.197606+0000","last_became_peered":"2023-10-30T12:35:05.197606+0000","last_unstale":"2023-10-30T12:55:54.873264+0000","last_undegraded":"2023-10-30T12:55:54.873264+0000","last_fullsized":"2023-10-30T12:55:54.873264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:26:57.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":90,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"55'698","reported_seq":2865,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.185693+0000","last_change":"2023-10-30T12:42:02.745501+0000","last_active":"2023-10-30T12:56:14.185693+0000","last_peered":"2023-10-30T12:56:14.185693+0000","last_clean":"2023-10-30T12:56:14.185693+0000","last_became_active":"2023-10-30T12:35:06.225392+0000","last_became_peered":"2023-10-30T12:35:06.225392+0000","last_unstale":"2023-10-30T12:56:14.185693+0000","last_undegraded":"2023-10-30T12:56:14.185693+0000","last_fullsized":"2023-10-30T12:56:14.185693+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:06:30.011008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01552166,"stat_sum":{"num_bytes":9754816,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":77,"num_read_kb":18,"num_write":651,"num_write_kb":19117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"55'525","reported_seq":586,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.879474+0000","last_change":"2023-10-30T12:35:05.197614+0000","last_active":"2023-10-30T12:56:14.879474+0000","last_peered":"2023-10-30T12:56:14.879474+0000","last_clean":"2023-10-30T12:56:14.879474+0000","last_became_active":"2023-10-30T12:35:05.196899+0000","last_became_peered":"2023-10-30T12:35:05.196899+0000","last_unstale":"2023-10-30T12:56:14.879474+0000","last_undegraded":"2023-10-30T12:56:14.879474+0000","last_fullsized":"2023-10-30T12:56:14.879474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:21:00.152426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17406311,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":14173,"num_write":457,"num_write_kb":17255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"55'706","reported_seq":2859,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.175188+0000","last_change":"2023-10-30T12:42:02.724824+0000","last_active":"2023-10-30T12:56:14.175188+0000","last_peered":"2023-10-30T12:56:14.175188+0000","last_clean":"2023-10-30T12:56:14.175188+0000","last_became_active":"2023-10-30T12:35:06.222057+0000","last_became_peered":"2023-10-30T12:35:06.222057+0000","last_unstale":"2023-10-30T12:56:14.175188+0000","last_undegraded":"2023-10-30T12:56:14.175188+0000","last_fullsized":"2023-10-30T12:56:14.175188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:13:07.261307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004210915,"stat_sum":{"num_bytes":31400359,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":63,"num_read_kb":12308,"num_write":657,"num_write_kb":50495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"55'312","reported_seq":375,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:08.116088+0000","last_change":"2023-10-30T12:35:05.197501+0000","last_active":"2023-10-30T12:56:08.116088+0000","last_peered":"2023-10-30T12:56:08.116088+0000","last_clean":"2023-10-30T12:56:08.116088+0000","last_became_active":"2023-10-30T12:35:05.197190+0000","last_became_peered":"2023-10-30T12:35:05.197190+0000","last_unstale":"2023-10-30T12:56:08.116088+0000","last_undegraded":"2023-10-30T12:56:08.116088+0000","last_fullsized":"2023-10-30T12:56:08.116088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:53:31.121721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":4111,"num_write":258,"num_write_kb":8429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"55'679","reported_seq":2830,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.190107+0000","last_change":"2023-10-30T12:42:02.253699+0000","last_active":"2023-10-30T12:56:14.190107+0000","last_peered":"2023-10-30T12:56:14.190107+0000","last_clean":"2023-10-30T12:56:14.190107+0000","last_became_active":"2023-10-30T12:35:06.204122+0000","last_became_peered":"2023-10-30T12:35:06.204122+0000","last_unstale":"2023-10-30T12:56:14.190107+0000","last_undegraded":"2023-10-30T12:56:14.190107+0000","last_fullsized":"2023-10-30T12:56:14.190107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:59:50.717088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030324050000000002,"stat_sum":{"num_bytes":15588017,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":60,"num_read_kb":8209,"num_write":632,"num_write_kb":41942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"55'359","reported_seq":431,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.533756+0000","last_change":"2023-10-30T12:35:05.180005+0000","last_active":"2023-10-30T12:56:09.533756+0000","last_peered":"2023-10-30T12:56:09.533756+0000","last_clean":"2023-10-30T12:56:09.533756+0000","last_became_active":"2023-10-30T12:35:05.179547+0000","last_became_peered":"2023-10-30T12:35:05.179547+0000","last_unstale":"2023-10-30T12:56:09.533756+0000","last_undegraded":"2023-10-30T12:56:09.533756+0000","last_fullsized":"2023-10-30T12:56:09.533756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:13:03.137433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":24600,"num_write":299,"num_write_kb":12510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"55'657","reported_seq":2808,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.184035+0000","last_change":"2023-10-30T12:42:02.473160+0000","last_active":"2023-10-30T12:56:14.184035+0000","last_peered":"2023-10-30T12:56:14.184035+0000","last_clean":"2023-10-30T12:56:14.184035+0000","last_became_active":"2023-10-30T12:35:06.215128+0000","last_became_peered":"2023-10-30T12:35:06.215128+0000","last_unstale":"2023-10-30T12:56:14.184035+0000","last_undegraded":"2023-10-30T12:56:14.184035+0000","last_fullsized":"2023-10-30T12:56:14.184035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:47:38.300886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064662909999999999,"stat_sum":{"num_bytes":22774544,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":61,"num_read_kb":11,"num_write":605,"num_write_kb":27691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"55'209","reported_seq":283,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.553645+0000","last_change":"2023-10-30T12:35:05.190041+0000","last_active":"2023-10-30T12:56:04.553645+0000","last_peered":"2023-10-30T12:56:04.553645+0000","last_clean":"2023-10-30T12:56:04.553645+0000","last_became_active":"2023-10-30T12:35:05.189573+0000","last_became_peered":"2023-10-30T12:35:05.189573+0000","last_unstale":"2023-10-30T12:56:04.553645+0000","last_undegraded":"2023-10-30T12:56:04.553645+0000","last_fullsized":"2023-10-30T12:56:04.553645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:02:57.642790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":24624,"num_write":154,"num_write_kb":8340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"55'738","reported_seq":2906,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.180434+0000","last_change":"2023-10-30T12:42:02.724827+0000","last_active":"2023-10-30T12:56:14.180434+0000","last_peered":"2023-10-30T12:56:14.180434+0000","last_clean":"2023-10-30T12:56:14.180434+0000","last_became_active":"2023-10-30T12:35:06.216885+0000","last_became_peered":"2023-10-30T12:35:06.216885+0000","last_unstale":"2023-10-30T12:56:14.180434+0000","last_undegraded":"2023-10-30T12:56:14.180434+0000","last_fullsized":"2023-10-30T12:56:14.180434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:50:48.181232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042006370000000001,"stat_sum":{"num_bytes":19251186,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":77,"num_read_kb":16402,"num_write":677,"num_write_kb":45337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"55'232","reported_seq":293,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.565577+0000","last_change":"2023-10-30T12:35:05.203198+0000","last_active":"2023-10-30T12:56:04.565577+0000","last_peered":"2023-10-30T12:56:04.565577+0000","last_clean":"2023-10-30T12:56:04.565577+0000","last_became_active":"2023-10-30T12:35:05.202612+0000","last_became_peered":"2023-10-30T12:35:05.202612+0000","last_unstale":"2023-10-30T12:56:04.565577+0000","last_undegraded":"2023-10-30T12:56:04.565577+0000","last_fullsized":"2023-10-30T12:56:04.565577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:55.986236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":181,"num_write_kb":8338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"55'710","reported_seq":2876,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.184300+0000","last_change":"2023-10-30T12:42:02.471602+0000","last_active":"2023-10-30T12:56:14.184300+0000","last_peered":"2023-10-30T12:56:14.184300+0000","last_clean":"2023-10-30T12:56:14.184300+0000","last_became_active":"2023-10-30T12:35:06.215553+0000","last_became_peered":"2023-10-30T12:35:06.215553+0000","last_unstale":"2023-10-30T12:56:14.184300+0000","last_undegraded":"2023-10-30T12:56:14.184300+0000","last_fullsized":"2023-10-30T12:56:14.184300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:58:00.184959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060418609999999999,"stat_sum":{"num_bytes":19055642,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":76,"num_read_kb":33,"num_write":654,"num_write_kb":32706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"55'289","reported_seq":360,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:27.641002+0000","last_change":"2023-10-30T12:35:05.181882+0000","last_active":"2023-10-30T12:55:27.641002+0000","last_peered":"2023-10-30T12:55:27.641002+0000","last_clean":"2023-10-30T12:55:27.641002+0000","last_became_active":"2023-10-30T12:35:05.181115+0000","last_became_peered":"2023-10-30T12:35:05.181115+0000","last_unstale":"2023-10-30T12:55:27.641002+0000","last_undegraded":"2023-10-30T12:55:27.641002+0000","last_fullsized":"2023-10-30T12:55:27.641002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:18:55.496940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":43239,"num_write":227,"num_write_kb":16591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"55'678","reported_seq":2829,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.178673+0000","last_change":"2023-10-30T12:42:02.738621+0000","last_active":"2023-10-30T12:56:14.178673+0000","last_peered":"2023-10-30T12:56:14.178673+0000","last_clean":"2023-10-30T12:56:14.178673+0000","last_became_active":"2023-10-30T12:35:06.197722+0000","last_became_peered":"2023-10-30T12:35:06.197722+0000","last_unstale":"2023-10-30T12:56:14.178673+0000","last_undegraded":"2023-10-30T12:56:14.178673+0000","last_fullsized":"2023-10-30T12:56:14.178673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:56:33.480519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011777173,"stat_sum":{"num_bytes":11179157,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":60,"num_read_kb":16,"num_write":620,"num_write_kb":21517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"55'481","reported_seq":572,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:04.552101+0000","last_change":"2023-10-30T12:35:05.169455+0000","last_active":"2023-10-30T12:56:04.552101+0000","last_peered":"2023-10-30T12:56:04.552101+0000","last_clean":"2023-10-30T12:56:04.552101+0000","last_became_active":"2023-10-30T12:35:05.169130+0000","last_became_peered":"2023-10-30T12:35:05.169130+0000","last_unstale":"2023-10-30T12:56:04.552101+0000","last_undegraded":"2023-10-30T12:56:04.552101+0000","last_fullsized":"2023-10-30T12:56:04.552101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:59:14.096315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583024,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":18341,"num_write":415,"num_write_kb":12551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"55'720","reported_seq":2902,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.171394+0000","last_change":"2023-10-30T12:42:02.470189+0000","last_active":"2023-10-30T12:56:14.171394+0000","last_peered":"2023-10-30T12:56:14.171394+0000","last_clean":"2023-10-30T12:56:14.171394+0000","last_became_active":"2023-10-30T12:35:06.214839+0000","last_became_peered":"2023-10-30T12:35:06.214839+0000","last_unstale":"2023-10-30T12:56:14.171394+0000","last_undegraded":"2023-10-30T12:56:14.171394+0000","last_fullsized":"2023-10-30T12:56:14.171394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:09:56.796250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049147749999999997,"stat_sum":{"num_bytes":12184897,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":91,"num_read_kb":16402,"num_write":664,"num_write_kb":29797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"55'313","reported_seq":378,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.749127+0000","last_change":"2023-10-30T12:35:05.189361+0000","last_active":"2023-10-30T12:56:09.749127+0000","last_peered":"2023-10-30T12:56:09.749127+0000","last_clean":"2023-10-30T12:56:09.749127+0000","last_became_active":"2023-10-30T12:35:05.188862+0000","last_became_peered":"2023-10-30T12:35:05.188862+0000","last_unstale":"2023-10-30T12:56:09.749127+0000","last_undegraded":"2023-10-30T12:56:09.749127+0000","last_fullsized":"2023-10-30T12:56:09.749127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:13.319732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":12343,"num_write":388,"num_write_kb":4750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"55'673","reported_seq":2829,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.170403+0000","last_change":"2023-10-30T12:42:02.472755+0000","last_active":"2023-10-30T12:56:14.170403+0000","last_peered":"2023-10-30T12:56:14.170403+0000","last_clean":"2023-10-30T12:56:14.170403+0000","last_became_active":"2023-10-30T12:35:06.216533+0000","last_became_peered":"2023-10-30T12:35:06.216533+0000","last_unstale":"2023-10-30T12:56:14.170403+0000","last_undegraded":"2023-10-30T12:56:14.170403+0000","last_fullsized":"2023-10-30T12:56:14.170403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:13:01.401936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060962430000000003,"stat_sum":{"num_bytes":10771712,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":65,"num_read_kb":4109,"num_write":629,"num_write_kb":24152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"55'234","reported_seq":295,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:04.569141+0000","last_change":"2023-10-30T12:35:05.177609+0000","last_active":"2023-10-30T12:55:04.569141+0000","last_peered":"2023-10-30T12:55:04.569141+0000","last_clean":"2023-10-30T12:55:04.569141+0000","last_became_active":"2023-10-30T12:35:05.177316+0000","last_became_peered":"2023-10-30T12:35:05.177316+0000","last_unstale":"2023-10-30T12:55:04.569141+0000","last_undegraded":"2023-10-30T12:55:04.569141+0000","last_fullsized":"2023-10-30T12:55:04.569141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:46.820925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":180,"num_write_kb":8325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"55'702","reported_seq":2862,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.160344+0000","last_change":"2023-10-30T12:42:02.241911+0000","last_active":"2023-10-30T12:56:14.160344+0000","last_peered":"2023-10-30T12:56:14.160344+0000","last_clean":"2023-10-30T12:56:14.160344+0000","last_became_active":"2023-10-30T12:35:46.049464+0000","last_became_peered":"2023-10-30T12:35:46.049464+0000","last_unstale":"2023-10-30T12:56:14.160344+0000","last_undegraded":"2023-10-30T12:56:14.160344+0000","last_fullsized":"2023-10-30T12:56:14.160344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:06:39.487053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023101760000000002,"stat_sum":{"num_bytes":7006288,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":663,"num_write_kb":12867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"55'211","reported_seq":272,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:28.291856+0000","last_change":"2023-10-30T12:35:05.171787+0000","last_active":"2023-10-30T12:55:28.291856+0000","last_peered":"2023-10-30T12:55:28.291856+0000","last_clean":"2023-10-30T12:55:28.291856+0000","last_became_active":"2023-10-30T12:35:05.171491+0000","last_became_peered":"2023-10-30T12:35:05.171491+0000","last_unstale":"2023-10-30T12:55:28.291856+0000","last_undegraded":"2023-10-30T12:55:28.291856+0000","last_fullsized":"2023-10-30T12:55:28.291856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:43:15.331175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":16387,"num_write":154,"num_write_kb":4208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"55'771","reported_seq":2932,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.197803+0000","last_change":"2023-10-30T12:42:02.746323+0000","last_active":"2023-10-30T12:56:14.197803+0000","last_peered":"2023-10-30T12:56:14.197803+0000","last_clean":"2023-10-30T12:56:14.197803+0000","last_became_active":"2023-10-30T12:35:06.213493+0000","last_became_peered":"2023-10-30T12:35:06.213493+0000","last_unstale":"2023-10-30T12:56:14.197803+0000","last_undegraded":"2023-10-30T12:56:14.197803+0000","last_fullsized":"2023-10-30T12:56:14.197803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:38:05.007816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015847370999999999,"stat_sum":{"num_bytes":18244405,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":70,"num_read_kb":4114,"num_write":718,"num_write_kb":33517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"55'305","reported_seq":368,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:04.570822+0000","last_change":"2023-10-30T12:35:05.197811+0000","last_active":"2023-10-30T12:55:04.570822+0000","last_peered":"2023-10-30T12:55:04.570822+0000","last_clean":"2023-10-30T12:55:04.570822+0000","last_became_active":"2023-10-30T12:35:05.197453+0000","last_became_peered":"2023-10-30T12:35:05.197453+0000","last_unstale":"2023-10-30T12:55:04.570822+0000","last_undegraded":"2023-10-30T12:55:04.570822+0000","last_fullsized":"2023-10-30T12:55:04.570822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:16:14.550887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":240,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"55'691","reported_seq":2843,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.179027+0000","last_change":"2023-10-30T12:42:02.472032+0000","last_active":"2023-10-30T12:56:14.179027+0000","last_peered":"2023-10-30T12:56:14.179027+0000","last_clean":"2023-10-30T12:56:14.179027+0000","last_became_active":"2023-10-30T12:35:06.219526+0000","last_became_peered":"2023-10-30T12:35:06.219526+0000","last_unstale":"2023-10-30T12:56:14.179027+0000","last_undegraded":"2023-10-30T12:56:14.179027+0000","last_fullsized":"2023-10-30T12:56:14.179027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:01:04.257663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060180570000000003,"stat_sum":{"num_bytes":14352444,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":61,"num_read_kb":19722,"num_write":644,"num_write_kb":44993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":208,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:54:25.564818+0000","last_change":"2023-10-30T12:34:55.206976+0000","last_active":"2023-10-30T12:54:25.564818+0000","last_peered":"2023-10-30T12:54:25.564818+0000","last_clean":"2023-10-30T12:54:25.564818+0000","last_became_active":"2023-10-30T12:34:55.205923+0000","last_became_peered":"2023-10-30T12:34:55.205923+0000","last_unstale":"2023-10-30T12:54:25.564818+0000","last_undegraded":"2023-10-30T12:54:25.564818+0000","last_fullsized":"2023-10-30T12:54:25.564818+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:34:53.444020+0000","last_clean_scrub_stamp":"2023-10-30T12:34:53.444020+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:25:28.782111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"53'311","reported_seq":380,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:54:54.680160+0000","last_change":"2023-10-30T12:35:05.197974+0000","last_active":"2023-10-30T12:54:54.680160+0000","last_peered":"2023-10-30T12:54:54.680160+0000","last_clean":"2023-10-30T12:54:54.680160+0000","last_became_active":"2023-10-30T12:35:05.197201+0000","last_became_peered":"2023-10-30T12:35:05.197201+0000","last_unstale":"2023-10-30T12:54:54.680160+0000","last_undegraded":"2023-10-30T12:54:54.680160+0000","last_fullsized":"2023-10-30T12:54:54.680160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:39:47.119091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":23,"num_read_kb":16415,"num_write":260,"num_write_kb":8374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"55'645","reported_seq":2788,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.165966+0000","last_change":"2023-10-30T12:42:02.243413+0000","last_active":"2023-10-30T12:56:14.165966+0000","last_peered":"2023-10-30T12:56:14.165966+0000","last_clean":"2023-10-30T12:56:14.165966+0000","last_became_active":"2023-10-30T12:35:06.217687+0000","last_became_peered":"2023-10-30T12:35:06.217687+0000","last_unstale":"2023-10-30T12:56:14.165966+0000","last_undegraded":"2023-10-30T12:56:14.165966+0000","last_fullsized":"2023-10-30T12:56:14.165966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:47:42.301163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059801699999999999,"stat_sum":{"num_bytes":18791963,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":51,"num_read_kb":8203,"num_write":599,"num_write_kb":32827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"55'386","reported_seq":458,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:31.185445+0000","last_change":"2023-10-30T12:35:05.208339+0000","last_active":"2023-10-30T12:55:31.185445+0000","last_peered":"2023-10-30T12:55:31.185445+0000","last_clean":"2023-10-30T12:55:31.185445+0000","last_became_active":"2023-10-30T12:35:05.207828+0000","last_became_peered":"2023-10-30T12:35:05.207828+0000","last_unstale":"2023-10-30T12:55:31.185445+0000","last_undegraded":"2023-10-30T12:55:31.185445+0000","last_fullsized":"2023-10-30T12:55:31.185445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:00.649048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":45074,"num_write":317,"num_write_kb":20666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"55'717","reported_seq":2867,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.172166+0000","last_change":"2023-10-30T12:42:02.724463+0000","last_active":"2023-10-30T12:56:14.172166+0000","last_peered":"2023-10-30T12:56:14.172166+0000","last_clean":"2023-10-30T12:56:14.172166+0000","last_became_active":"2023-10-30T12:35:06.194791+0000","last_became_peered":"2023-10-30T12:35:06.194791+0000","last_unstale":"2023-10-30T12:56:14.172166+0000","last_undegraded":"2023-10-30T12:56:14.172166+0000","last_fullsized":"2023-10-30T12:56:14.172166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:13:52.749170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043414500000000002,"stat_sum":{"num_bytes":10614378,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":58,"num_read_kb":3682,"num_write":656,"num_write_kb":34629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"55'607","reported_seq":965,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:57.501324+0000","last_change":"2023-10-30T12:35:05.176296+0000","last_active":"2023-10-30T12:55:57.501324+0000","last_peered":"2023-10-30T12:55:57.501324+0000","last_clean":"2023-10-30T12:55:57.501324+0000","last_became_active":"2023-10-30T12:35:05.175959+0000","last_became_peered":"2023-10-30T12:35:05.175959+0000","last_unstale":"2023-10-30T12:55:57.501324+0000","last_undegraded":"2023-10-30T12:55:57.501324+0000","last_fullsized":"2023-10-30T12:55:57.501324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:53.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9398249,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":313,"num_read_kb":20888,"num_write":566,"num_write_kb":9626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"55'706","reported_seq":2866,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.160213+0000","last_change":"2023-10-30T12:42:02.711348+0000","last_active":"2023-10-30T12:56:14.160213+0000","last_peered":"2023-10-30T12:56:14.160213+0000","last_clean":"2023-10-30T12:56:14.160213+0000","last_became_active":"2023-10-30T12:35:06.164711+0000","last_became_peered":"2023-10-30T12:35:06.164711+0000","last_unstale":"2023-10-30T12:56:14.160213+0000","last_undegraded":"2023-10-30T12:56:14.160213+0000","last_fullsized":"2023-10-30T12:56:14.160213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:21:46.319142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002714419,"stat_sum":{"num_bytes":18668779,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":68,"num_read_kb":8213,"num_write":645,"num_write_kb":33825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"55'497","reported_seq":561,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:56.727539+0000","last_change":"2023-10-30T12:35:05.198964+0000","last_active":"2023-10-30T12:55:56.727539+0000","last_peered":"2023-10-30T12:55:56.727539+0000","last_clean":"2023-10-30T12:55:56.727539+0000","last_became_active":"2023-10-30T12:35:05.196873+0000","last_became_peered":"2023-10-30T12:35:05.196873+0000","last_unstale":"2023-10-30T12:55:56.727539+0000","last_undegraded":"2023-10-30T12:55:56.727539+0000","last_fullsized":"2023-10-30T12:55:56.727539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:09:47.793585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":431,"num_write_kb":16687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"55'708","reported_seq":2872,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.182230+0000","last_change":"2023-10-30T12:42:02.250101+0000","last_active":"2023-10-30T12:56:14.182230+0000","last_peered":"2023-10-30T12:56:14.182230+0000","last_clean":"2023-10-30T12:56:14.182230+0000","last_became_active":"2023-10-30T12:35:06.183225+0000","last_became_peered":"2023-10-30T12:35:06.183225+0000","last_unstale":"2023-10-30T12:56:14.182230+0000","last_undegraded":"2023-10-30T12:56:14.182230+0000","last_fullsized":"2023-10-30T12:56:14.182230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:02:51.357971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032989500000000001,"stat_sum":{"num_bytes":18054478,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":72,"num_read_kb":12,"num_write":656,"num_write_kb":36307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"55'183","reported_seq":239,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:27.599652+0000","last_change":"2023-10-30T12:35:05.177571+0000","last_active":"2023-10-30T12:55:27.599652+0000","last_peered":"2023-10-30T12:55:27.599652+0000","last_clean":"2023-10-30T12:55:27.599652+0000","last_became_active":"2023-10-30T12:35:05.177266+0000","last_became_peered":"2023-10-30T12:35:05.177266+0000","last_unstale":"2023-10-30T12:55:27.599652+0000","last_undegraded":"2023-10-30T12:55:27.599652+0000","last_fullsized":"2023-10-30T12:55:27.599652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:14.601934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":129,"num_write_kb":8259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"55'688","reported_seq":2849,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.198180+0000","last_change":"2023-10-30T12:42:02.745017+0000","last_active":"2023-10-30T12:56:14.198180+0000","last_peered":"2023-10-30T12:56:14.198180+0000","last_clean":"2023-10-30T12:56:14.198180+0000","last_became_active":"2023-10-30T12:35:06.196307+0000","last_became_peered":"2023-10-30T12:35:06.196307+0000","last_unstale":"2023-10-30T12:56:14.198180+0000","last_undegraded":"2023-10-30T12:56:14.198180+0000","last_fullsized":"2023-10-30T12:56:14.198180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:09:56.936899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015197106,"stat_sum":{"num_bytes":5885946,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":69,"num_read_kb":13,"num_write":633,"num_write_kb":24590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"55'493","reported_seq":556,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:13.935701+0000","last_change":"2023-10-30T12:35:05.207594+0000","last_active":"2023-10-30T12:56:13.935701+0000","last_peered":"2023-10-30T12:56:13.935701+0000","last_clean":"2023-10-30T12:56:13.935701+0000","last_became_active":"2023-10-30T12:35:05.206930+0000","last_became_peered":"2023-10-30T12:35:05.206930+0000","last_unstale":"2023-10-30T12:56:13.935701+0000","last_undegraded":"2023-10-30T12:56:13.935701+0000","last_fullsized":"2023-10-30T12:56:13.935701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:23:46.519077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16162750,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":8,"num_read_kb":9517,"num_write":433,"num_write_kb":16052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"55'701","reported_seq":2865,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.145518+0000","last_change":"2023-10-30T12:42:02.710925+0000","last_active":"2023-10-30T12:56:14.145518+0000","last_peered":"2023-10-30T12:56:14.145518+0000","last_clean":"2023-10-30T12:56:14.145518+0000","last_became_active":"2023-10-30T12:35:06.164733+0000","last_became_peered":"2023-10-30T12:35:06.164733+0000","last_unstale":"2023-10-30T12:56:14.145518+0000","last_undegraded":"2023-10-30T12:56:14.145518+0000","last_fullsized":"2023-10-30T12:56:14.145518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:04:21.927512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024278020000000002,"stat_sum":{"num_bytes":19206810,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":652,"num_write_kb":32920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"55'375","reported_seq":442,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.590832+0000","last_change":"2023-10-30T12:35:05.185034+0000","last_active":"2023-10-30T12:56:09.590832+0000","last_peered":"2023-10-30T12:56:09.590832+0000","last_clean":"2023-10-30T12:56:09.590832+0000","last_became_active":"2023-10-30T12:35:05.184695+0000","last_became_peered":"2023-10-30T12:35:05.184695+0000","last_unstale":"2023-10-30T12:56:09.590832+0000","last_undegraded":"2023-10-30T12:56:09.590832+0000","last_fullsized":"2023-10-30T12:56:09.590832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:01:42.019128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":8489,"num_write":574,"num_write_kb":5859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"55'682","reported_seq":2835,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.176081+0000","last_change":"2023-10-30T12:42:02.242088+0000","last_active":"2023-10-30T12:56:14.176081+0000","last_peered":"2023-10-30T12:56:14.176081+0000","last_clean":"2023-10-30T12:56:14.176081+0000","last_became_active":"2023-10-30T12:35:06.204422+0000","last_became_peered":"2023-10-30T12:35:06.204422+0000","last_unstale":"2023-10-30T12:56:14.176081+0000","last_undegraded":"2023-10-30T12:56:14.176081+0000","last_fullsized":"2023-10-30T12:56:14.176081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:38:09.854923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048535030000000003,"stat_sum":{"num_bytes":22995855,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":62,"num_read_kb":4110,"num_write":640,"num_write_kb":33018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"55'230","reported_seq":292,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:28.060181+0000","last_change":"2023-10-30T12:35:05.200871+0000","last_active":"2023-10-30T12:55:28.060181+0000","last_peered":"2023-10-30T12:55:28.060181+0000","last_clean":"2023-10-30T12:55:28.060181+0000","last_became_active":"2023-10-30T12:35:05.200449+0000","last_became_peered":"2023-10-30T12:35:05.200449+0000","last_unstale":"2023-10-30T12:55:28.060181+0000","last_undegraded":"2023-10-30T12:55:28.060181+0000","last_fullsized":"2023-10-30T12:55:28.060181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:39:20.867587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":4117,"num_write":187,"num_write_kb":4299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"55'665","reported_seq":2825,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.158220+0000","last_change":"2023-10-30T12:42:02.724106+0000","last_active":"2023-10-30T12:56:14.158220+0000","last_peered":"2023-10-30T12:56:14.158220+0000","last_clean":"2023-10-30T12:56:14.158220+0000","last_became_active":"2023-10-30T12:35:06.179412+0000","last_became_peered":"2023-10-30T12:35:06.179412+0000","last_unstale":"2023-10-30T12:56:14.158220+0000","last_undegraded":"2023-10-30T12:56:14.158220+0000","last_fullsized":"2023-10-30T12:56:14.158220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:15:21.046958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003661568,"stat_sum":{"num_bytes":1664351,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":69,"num_read_kb":22,"num_write":614,"num_write_kb":12460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"55'552","reported_seq":617,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:26.056739+0000","last_change":"2023-10-30T12:35:05.207392+0000","last_active":"2023-10-30T12:55:26.056739+0000","last_peered":"2023-10-30T12:55:26.056739+0000","last_clean":"2023-10-30T12:55:26.056739+0000","last_became_active":"2023-10-30T12:35:05.206607+0000","last_became_peered":"2023-10-30T12:35:05.206607+0000","last_unstale":"2023-10-30T12:55:26.056739+0000","last_undegraded":"2023-10-30T12:55:26.056739+0000","last_fullsized":"2023-10-30T12:55:26.056739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:13:36.758740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":17,"num_read_kb":23179,"num_write":497,"num_write_kb":12662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"55'701","reported_seq":2862,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.179404+0000","last_change":"2023-10-30T12:42:02.251623+0000","last_active":"2023-10-30T12:56:14.179404+0000","last_peered":"2023-10-30T12:56:14.179404+0000","last_clean":"2023-10-30T12:56:14.179404+0000","last_became_active":"2023-10-30T12:35:06.181256+0000","last_became_peered":"2023-10-30T12:35:06.181256+0000","last_unstale":"2023-10-30T12:56:14.179404+0000","last_undegraded":"2023-10-30T12:56:14.179404+0000","last_fullsized":"2023-10-30T12:56:14.179404+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:30:13.806672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030387220000000002,"stat_sum":{"num_bytes":10464290,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":70,"num_read_kb":4103,"num_write":644,"num_write_kb":29233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"55'369","reported_seq":432,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:02.291363+0000","last_change":"2023-10-30T12:35:05.201024+0000","last_active":"2023-10-30T12:56:02.291363+0000","last_peered":"2023-10-30T12:56:02.291363+0000","last_clean":"2023-10-30T12:56:02.291363+0000","last_became_active":"2023-10-30T12:35:05.200724+0000","last_became_peered":"2023-10-30T12:35:05.200724+0000","last_unstale":"2023-10-30T12:56:02.291363+0000","last_undegraded":"2023-10-30T12:56:02.291363+0000","last_fullsized":"2023-10-30T12:56:02.291363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:24.746469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388698,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":12300,"num_write":322,"num_write_kb":8548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"55'711","reported_seq":2860,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.152617+0000","last_change":"2023-10-30T12:42:02.241265+0000","last_active":"2023-10-30T12:56:14.152617+0000","last_peered":"2023-10-30T12:56:14.152617+0000","last_clean":"2023-10-30T12:56:14.152617+0000","last_became_active":"2023-10-30T12:35:06.173996+0000","last_became_peered":"2023-10-30T12:35:06.173996+0000","last_unstale":"2023-10-30T12:56:14.152617+0000","last_undegraded":"2023-10-30T12:56:14.152617+0000","last_fullsized":"2023-10-30T12:56:14.152617+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:34:09.185233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045562010000000002,"stat_sum":{"num_bytes":5693555,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":58,"num_read_kb":18,"num_write":647,"num_write_kb":16270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"55'874","reported_seq":935,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:09.677198+0000","last_change":"2023-10-30T12:35:05.164079+0000","last_active":"2023-10-30T12:56:09.677198+0000","last_peered":"2023-10-30T12:56:09.677198+0000","last_clean":"2023-10-30T12:56:09.677198+0000","last_became_active":"2023-10-30T12:35:05.163794+0000","last_became_peered":"2023-10-30T12:35:05.163794+0000","last_unstale":"2023-10-30T12:56:09.677198+0000","last_undegraded":"2023-10-30T12:56:09.677198+0000","last_fullsized":"2023-10-30T12:56:09.677198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:19.795656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":10,"num_read_kb":8454,"num_write":954,"num_write_kb":13780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"55'668","reported_seq":2828,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.177104+0000","last_change":"2023-10-30T12:42:02.242567+0000","last_active":"2023-10-30T12:56:14.177104+0000","last_peered":"2023-10-30T12:56:14.177104+0000","last_clean":"2023-10-30T12:56:14.177104+0000","last_became_active":"2023-10-30T12:35:06.183226+0000","last_became_peered":"2023-10-30T12:35:06.183226+0000","last_unstale":"2023-10-30T12:56:14.177104+0000","last_undegraded":"2023-10-30T12:56:14.177104+0000","last_fullsized":"2023-10-30T12:56:14.177104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:30:48.627756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052521379999999999,"stat_sum":{"num_bytes":5799195,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":69,"num_read_kb":4112,"num_write":618,"num_write_kb":16472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"55'336","reported_seq":402,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:36.391405+0000","last_change":"2023-10-30T12:35:05.206995+0000","last_active":"2023-10-30T12:55:36.391405+0000","last_peered":"2023-10-30T12:55:36.391405+0000","last_clean":"2023-10-30T12:55:36.391405+0000","last_became_active":"2023-10-30T12:35:05.206518+0000","last_became_peered":"2023-10-30T12:35:05.206518+0000","last_unstale":"2023-10-30T12:55:36.391405+0000","last_undegraded":"2023-10-30T12:55:36.391405+0000","last_fullsized":"2023-10-30T12:55:36.391405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:41:33.703261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":268,"num_write_kb":12461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"55'662","reported_seq":2821,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.145115+0000","last_change":"2023-10-30T12:42:02.737782+0000","last_active":"2023-10-30T12:56:14.145115+0000","last_peered":"2023-10-30T12:56:14.145115+0000","last_clean":"2023-10-30T12:56:14.145115+0000","last_became_active":"2023-10-30T12:35:06.197861+0000","last_became_peered":"2023-10-30T12:35:06.197861+0000","last_unstale":"2023-10-30T12:56:14.145115+0000","last_undegraded":"2023-10-30T12:56:14.145115+0000","last_fullsized":"2023-10-30T12:56:14.145115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:38:17.960141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011118161,"stat_sum":{"num_bytes":14763185,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":68,"num_read_kb":15,"num_write":609,"num_write_kb":24605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"55'685","reported_seq":2856,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.196774+0000","last_change":"2023-10-30T12:42:02.744356+0000","last_active":"2023-10-30T12:56:14.196774+0000","last_peered":"2023-10-30T12:56:14.196774+0000","last_clean":"2023-10-30T12:56:14.196774+0000","last_became_active":"2023-10-30T12:35:06.212548+0000","last_became_peered":"2023-10-30T12:35:06.212548+0000","last_unstale":"2023-10-30T12:56:14.196774+0000","last_undegraded":"2023-10-30T12:56:14.196774+0000","last_fullsized":"2023-10-30T12:56:14.196774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:58:19.323075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015114931999999999,"stat_sum":{"num_bytes":19366418,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":81,"num_read_kb":7349,"num_write":643,"num_write_kb":32740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"55'363","reported_seq":423,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:07.528197+0000","last_change":"2023-10-30T12:35:05.206980+0000","last_active":"2023-10-30T12:55:07.528197+0000","last_peered":"2023-10-30T12:55:07.528197+0000","last_clean":"2023-10-30T12:55:07.528197+0000","last_became_active":"2023-10-30T12:35:05.206559+0000","last_became_peered":"2023-10-30T12:35:05.206559+0000","last_unstale":"2023-10-30T12:55:07.528197+0000","last_undegraded":"2023-10-30T12:55:07.528197+0000","last_fullsized":"2023-10-30T12:55:07.528197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:40.259622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":312,"num_write_kb":8389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"55'687","reported_seq":2847,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.190990+0000","last_change":"2023-10-30T12:42:02.744472+0000","last_active":"2023-10-30T12:56:14.190990+0000","last_peered":"2023-10-30T12:56:14.190990+0000","last_clean":"2023-10-30T12:56:14.190990+0000","last_became_active":"2023-10-30T12:35:06.212505+0000","last_became_peered":"2023-10-30T12:35:06.212505+0000","last_unstale":"2023-10-30T12:56:14.190990+0000","last_undegraded":"2023-10-30T12:56:14.190990+0000","last_fullsized":"2023-10-30T12:56:14.190990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:28:56.182227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014553393,"stat_sum":{"num_bytes":9800415,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":70,"num_read_kb":12304,"num_write":642,"num_write_kb":36520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"55'377","reported_seq":448,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:59.997488+0000","last_change":"2023-10-30T12:35:05.207851+0000","last_active":"2023-10-30T12:55:59.997488+0000","last_peered":"2023-10-30T12:55:59.997488+0000","last_clean":"2023-10-30T12:55:59.997488+0000","last_became_active":"2023-10-30T12:35:05.206879+0000","last_became_peered":"2023-10-30T12:35:05.206879+0000","last_unstale":"2023-10-30T12:55:59.997488+0000","last_undegraded":"2023-10-30T12:55:59.997488+0000","last_fullsized":"2023-10-30T12:55:59.997488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:43:15.717507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":32,"num_read_kb":45093,"num_write":322,"num_write_kb":20730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"55'643","reported_seq":2805,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.164381+0000","last_change":"2023-10-30T12:42:02.723663+0000","last_active":"2023-10-30T12:56:14.164381+0000","last_peered":"2023-10-30T12:56:14.164381+0000","last_clean":"2023-10-30T12:56:14.164381+0000","last_became_active":"2023-10-30T12:35:46.275077+0000","last_became_peered":"2023-10-30T12:35:46.275077+0000","last_unstale":"2023-10-30T12:56:14.164381+0000","last_undegraded":"2023-10-30T12:56:14.164381+0000","last_fullsized":"2023-10-30T12:56:14.164381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:31:10.982688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032416139999999999,"stat_sum":{"num_bytes":6205387,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":63,"num_read_kb":4113,"num_write":608,"num_write_kb":9154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"55'508","reported_seq":569,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:35.795056+0000","last_change":"2023-10-30T12:35:05.174319+0000","last_active":"2023-10-30T12:55:35.795056+0000","last_peered":"2023-10-30T12:55:35.795056+0000","last_clean":"2023-10-30T12:55:35.795056+0000","last_became_active":"2023-10-30T12:35:05.173984+0000","last_became_peered":"2023-10-30T12:35:05.173984+0000","last_unstale":"2023-10-30T12:55:35.795056+0000","last_undegraded":"2023-10-30T12:55:35.795056+0000","last_fullsized":"2023-10-30T12:55:35.795056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:11.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":430,"num_write_kb":12596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"55'700","reported_seq":2853,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.142391+0000","last_change":"2023-10-30T12:42:02.243078+0000","last_active":"2023-10-30T12:56:14.142391+0000","last_peered":"2023-10-30T12:56:14.142391+0000","last_clean":"2023-10-30T12:56:14.142391+0000","last_became_active":"2023-10-30T12:35:06.198504+0000","last_became_peered":"2023-10-30T12:35:06.198504+0000","last_unstale":"2023-10-30T12:56:14.142391+0000","last_undegraded":"2023-10-30T12:56:14.142391+0000","last_fullsized":"2023-10-30T12:56:14.142391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:44:32.447527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055292249999999996,"stat_sum":{"num_bytes":14413504,"num_objects":182,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":61,"num_read_kb":1995,"num_write":647,"num_write_kb":19115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"55'208","reported_seq":290,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:55:28.291837+0000","last_change":"2023-10-30T12:35:05.200926+0000","last_active":"2023-10-30T12:55:28.291837+0000","last_peered":"2023-10-30T12:55:28.291837+0000","last_clean":"2023-10-30T12:55:28.291837+0000","last_became_active":"2023-10-30T12:35:05.200623+0000","last_became_peered":"2023-10-30T12:35:05.200623+0000","last_unstale":"2023-10-30T12:55:28.291837+0000","last_undegraded":"2023-10-30T12:55:28.291837+0000","last_fullsized":"2023-10-30T12:55:28.291837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:38.942922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":63,"num_read_kb":24633,"num_write":147,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":925262768,"num_objects":10302,"num_object_clones":0,"num_object_copies":20604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10302,"num_whiteouts":0,"num_read":4339,"num_read_kb":400205,"num_write":41481,"num_write_kb":1950238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1902673920,"data_stored":1848985440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44733,"ondisk_log_size":44733,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":358391509,"num_objects":210,"num_object_clones":0,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":0,"num_read":1232,"num_read_kb":640401,"num_write":11572,"num_write_kb":360767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":716939264,"data_stored":716783018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12928,"ondisk_log_size":12928,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542406,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491392,"kb_used_data":401912,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":93880448,"statfs":{"total":96636764160,"available":96133578752,"internally_reserved":0,"allocated":411557888,"data_stored":403485968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":91617741},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.48599999999999999,"5min":0.438,"15min":0.42899999999999999},"max":{"1min":1.2330000000000001,"5min":1.371,"15min":1.371},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.82099999999999995,"15min":0.82399999999999995},"min":{"1min":0.54200000000000004,"5min":0.437,"15min":0.41699999999999998},"max":{"1min":1.137,"5min":1.45,"15min":1.45},"last":1.155}]},{"osd":1,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82699999999999996,"15min":0.85299999999999998},"min":{"1min":0.52700000000000002,"5min":0.46700000000000003,"15min":0.44600000000000001},"max":{"1min":1.222,"5min":1.222,"15min":1.496},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.86599999999999999,"15min":0.89600000000000002},"min":{"1min":0.55700000000000005,"5min":0.53500000000000003,"15min":0.51800000000000002},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":1.099}]},{"osd":2,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.873,"15min":0.82799999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.1659999999999999,"5min":1.242,"15min":1.9059999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.84199999999999997,"15min":0.89700000000000002},"min":{"1min":0.60299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.93100000000000005}]},{"osd":3,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":0.92600000000000005,"15min":0.91100000000000003},"min":{"1min":0.76300000000000001,"5min":0.54200000000000004,"15min":0.54000000000000004},"max":{"1min":1.474,"5min":1.607,"15min":1.9810000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.004,"15min":1},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.81499999999999995}]},{"osd":4,"last update":"Mon Oct 30 12:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90100000000000002,"15min":0.92800000000000005},"min":{"1min":0.71599999999999997,"5min":0.52500000000000002,"15min":0.46899999999999997},"max":{"1min":1.329,"5min":1.4059999999999999,"15min":2.7309999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.016,"15min":0.99399999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.41999999999999998},"max":{"1min":1.3600000000000001,"5min":1.694,"15min":1.8859999999999999},"last":0.75700000000000001}]},{"osd":5,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.048,"15min":1.0700000000000001},"min":{"1min":0.72099999999999997,"5min":0.59399999999999997,"15min":0.49099999999999999},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":2.2120000000000002},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.218,"15min":1.143},"min":{"1min":1.0089999999999999,"5min":0.72599999999999998,"15min":0.69899999999999995},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.25}]},{"osd":6,"last update":"Mon Oct 30 12:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0229999999999999,"15min":1.087},"min":{"1min":0.79400000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.456,"5min":1.478,"15min":1.802},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.206,"5min":1.1359999999999999,"15min":1.1930000000000001},"min":{"1min":0.92800000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.4870000000000001,"5min":1.6639999999999999,"15min":1.881},"last":0.99399999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542405,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384296,"kb_used_data":299040,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":93987544,"statfs":{"total":96636764160,"available":96243245056,"internally_reserved":0,"allocated":306216960,"data_stored":300882091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":87292361},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.83199999999999996,"15min":0.82399999999999995},"min":{"1min":0.496,"5min":0.45600000000000002,"15min":0.38600000000000001},"max":{"1min":1.056,"5min":5.8760000000000003,"15min":5.8760000000000003},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.87,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.60899999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.161,"5min":1.508,"15min":1.673},"last":0.65500000000000003}]},{"osd":1,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.85699999999999998,"15min":0.92100000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.113,"5min":2.2679999999999998,"15min":5.0949999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.91100000000000003,"15min":0.92700000000000005},"min":{"1min":0.65000000000000002,"5min":0.54800000000000004,"15min":0.45500000000000002},"max":{"1min":1.282,"5min":1.423,"15min":5.0519999999999996},"last":0.83299999999999996}]},{"osd":2,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.86499999999999999,"15min":0.84699999999999998},"min":{"1min":0.621,"5min":0.52900000000000003,"15min":0.51800000000000002},"max":{"1min":1.403,"5min":2.649,"15min":2.649},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.069,"5min":0.97099999999999997,"15min":0.92500000000000004},"min":{"1min":0.74099999999999999,"5min":0.56899999999999995,"15min":0.44700000000000001},"max":{"1min":1.528,"5min":2.282,"15min":2.282},"last":0.91300000000000003}]},{"osd":3,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.88500000000000001,"15min":0.94599999999999995},"min":{"1min":0.62,"5min":0.51700000000000002,"15min":0.40200000000000002},"max":{"1min":1.1120000000000001,"5min":1.929,"15min":1.929},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.97499999999999998,"15min":1},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.42699999999999999},"max":{"1min":1.246,"5min":1.5640000000000001,"15min":1.7589999999999999},"last":0.73799999999999999}]},{"osd":4,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.03,"15min":1.0009999999999999},"min":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.27600000000000002},"max":{"1min":1.617,"5min":1.619,"15min":1.9219999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.05,"15min":1.0329999999999999},"min":{"1min":0.65600000000000003,"5min":0.64200000000000002,"15min":0.32400000000000001},"max":{"1min":1.3160000000000001,"5min":1.5700000000000001,"15min":1.625},"last":0.88300000000000001}]},{"osd":5,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":1.0109999999999999,"15min":1.0349999999999999},"min":{"1min":0.66600000000000004,"5min":0.66300000000000003,"15min":0.29599999999999999},"max":{"1min":1.194,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.1040000000000001,"15min":1.133},"min":{"1min":0.78500000000000003,"5min":0.65300000000000002,"15min":0.34399999999999997},"max":{"1min":1.3089999999999999,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":0.97499999999999998}]},{"osd":7,"last update":"Mon Oct 30 12:55:31 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.179,"15min":1.149},"min":{"1min":0.82999999999999996,"5min":0.79300000000000004,"15min":0.37},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":2.0350000000000001},"last":1.028},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2030000000000001,"15min":1.1950000000000001},"min":{"1min":0.90200000000000002,"5min":0.82099999999999995,"15min":0.46800000000000003},"max":{"1min":1.754,"5min":2.274,"15min":2.274},"last":1.0029999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413036,"kb_used_data":319396,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":93958804,"statfs":{"total":96636764160,"available":96213815296,"internally_reserved":0,"allocated":327061504,"data_stored":318875235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79500000000000004,"15min":0.80900000000000005},"min":{"1min":0.53200000000000003,"5min":0.47199999999999998,"15min":0.437},"max":{"1min":1.1819999999999999,"5min":1.3700000000000001,"15min":1.5649999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.83299999999999996,"15min":0.84099999999999997},"min":{"1min":0.58899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2390000000000001,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.53300000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.034,"5min":1.3069999999999999,"15min":2.4910000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.86299999999999999,"15min":0.84999999999999998},"min":{"1min":0.63,"5min":0.433,"15min":0.433},"max":{"1min":1.2030000000000001,"5min":1.54,"15min":1.9730000000000001},"last":0.86399999999999999}]},{"osd":2,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.67300000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1950000000000001,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":1.0600000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.55600000000000005},"max":{"1min":1.2170000000000001,"5min":4.6879999999999997,"15min":4.6879999999999997},"last":1.0700000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.92500000000000004,"15min":0.94399999999999995},"min":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.284,"5min":2.266,"15min":2.5819999999999999},"last":1.032},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.94999999999999996,"15min":0.90700000000000003},"min":{"1min":0.78200000000000003,"5min":0.54300000000000004,"15min":0.53500000000000003},"max":{"1min":1.411,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.1899999999999999}]},{"osd":5,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.97099999999999997,"15min":1.0029999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.129,"5min":2.1779999999999999,"15min":2.3180000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.0880000000000001,"15min":1.0620000000000001},"min":{"1min":1.0660000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3959999999999999,"5min":3.173,"15min":3.173},"last":1.248}]},{"osd":6,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":1.0509999999999999,"15min":1.0880000000000001},"min":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.59299999999999997},"max":{"1min":1.131,"5min":2.2709999999999999,"15min":2.73},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.093,"5min":1.1080000000000001,"15min":1.0640000000000001},"min":{"1min":0.83999999999999997,"5min":0.748,"15min":0.56499999999999995},"max":{"1min":1.262,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":1.105}]},{"osd":7,"last update":"Mon Oct 30 12:55:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.087,"15min":1.0700000000000001},"min":{"1min":0.70699999999999996,"5min":0.68899999999999995,"15min":0.64900000000000002},"max":{"1min":1.196,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.157},{"interface":"front","average":{"1min":1.204,"5min":1.175,"15min":1.131},"min":{"1min":0.92500000000000004,"5min":0.70299999999999996,"15min":0.65300000000000002},"max":{"1min":1.4950000000000001,"5min":2.375,"15min":3.0649999999999999},"last":1.298}]}]},{"osd":2,"up_from":14,"seq":60129542405,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298704,"kb_used_data":241160,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94073136,"statfs":{"total":96636764160,"available":96330891264,"internally_reserved":0,"allocated":246947840,"data_stored":241832282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.48599999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.22700000000000001},"max":{"1min":1.6930000000000001,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.58199999999999996,"15min":0.53200000000000003},"min":{"1min":0.40000000000000002,"5min":0.33200000000000002,"15min":0.28699999999999998},"max":{"1min":2.0619999999999998,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":0.42099999999999999}]},{"osd":1,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.33000000000000002,"5min":0.317,"15min":0.27900000000000003},"max":{"1min":3.02,"5min":3.5059999999999998,"15min":4.5019999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.39100000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.0299999999999998,"5min":3.5430000000000001,"15min":3.5430000000000001},"last":0.38}]},{"osd":3,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.76000000000000001,"15min":0.60399999999999998},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26100000000000001},"max":{"1min":8.9550000000000001,"5min":18.747,"15min":18.747},"last":0.45700000000000002},{"interface":"front","average":{"1min":1.0389999999999999,"5min":0.85499999999999998,"15min":0.70099999999999996},"min":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38300000000000001},"max":{"1min":8.9900000000000002,"5min":18.863,"15min":18.863},"last":0.503}]},{"osd":4,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85999999999999999},"min":{"1min":0.626,"5min":0.56999999999999995,"15min":0.53600000000000003},"max":{"1min":3.0760000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.321,"5min":0.96299999999999997,"15min":0.89700000000000002},"min":{"1min":0.73299999999999998,"5min":0.51100000000000001,"15min":0.48299999999999998},"max":{"1min":9.6289999999999996,"5min":9.6289999999999996,"15min":9.6289999999999996},"last":0.84599999999999997}]},{"osd":5,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":0.876,"15min":0.85199999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":9.7289999999999992,"5min":9.7289999999999992,"15min":9.7289999999999992},"last":0.68400000000000005},{"interface":"front","average":{"1min":5.577,"5min":1.9490000000000001,"15min":1.2},"min":{"1min":0.626,"5min":0.60399999999999998,"15min":0.51900000000000002},"max":{"1min":91.072999999999993,"5min":91.072999999999993,"15min":91.072999999999993},"last":0.748}]},{"osd":6,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.048,"15min":0.91700000000000004},"min":{"1min":0.59299999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":5.4640000000000004,"5min":19.126000000000001,"15min":19.126000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.2509999999999999,"15min":1.006},"min":{"1min":0.72499999999999998,"5min":0.60299999999999998,"15min":0.59399999999999997},"max":{"1min":21.469999999999999,"5min":21.469999999999999,"15min":21.469999999999999},"last":0.86099999999999999}]},{"osd":7,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":0.94699999999999995,"15min":0.89600000000000002},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.47099999999999997},"max":{"1min":9.6869999999999994,"5min":9.6869999999999994,"15min":9.6869999999999994},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.1479999999999999,"15min":0.96799999999999997},"min":{"1min":0.80800000000000005,"5min":0.69799999999999995,"15min":0.53200000000000003},"max":{"1min":5.524,"5min":19.050999999999998,"15min":19.050999999999998},"last":0.93600000000000005}]}]},{"osd":0,"up_from":14,"seq":60129542406,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430360,"kb_used_data":340880,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":93941480,"statfs":{"total":96636764160,"available":96196075520,"internally_reserved":0,"allocated":349061120,"data_stored":340810124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":91617736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.46600000000000003,"15min":0.48399999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.248},"max":{"1min":0.69099999999999995,"5min":0.78700000000000003,"15min":8.8059999999999992},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.504,"15min":0.61099999999999999},"min":{"1min":0.30099999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69299999999999995,"5min":0.83899999999999997,"15min":35.084000000000003},"last":0.377}]},{"osd":2,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51500000000000001,"15min":0.63500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.28999999999999998},"max":{"1min":0.68000000000000005,"5min":0.93100000000000005,"15min":34.741},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58099999999999996,"15min":0.69299999999999995},"min":{"1min":0.42999999999999999,"5min":0.37,"15min":0.35299999999999998},"max":{"1min":0.72999999999999998,"5min":1.0049999999999999,"15min":35.058999999999997},"last":0.437}]},{"osd":3,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.35899999999999999,"5min":0.32000000000000001,"15min":0.29499999999999998},"max":{"1min":0.76000000000000001,"5min":0.877,"15min":8.8469999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.629},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.88800000000000001,"15min":11.916},"last":0.45600000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.85499999999999998,"15min":0.79700000000000004},"min":{"1min":0.66600000000000004,"5min":0.55500000000000005,"15min":0.373},"max":{"1min":1.0389999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.82499999999999996,"15min":0.92800000000000005},"min":{"1min":0.63900000000000001,"5min":0.57099999999999995,"15min":0.39900000000000002},"max":{"1min":1.1419999999999999,"5min":1.175,"15min":34.902999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82099999999999995,"15min":0.84799999999999998},"min":{"1min":0.59499999999999997,"5min":0.51800000000000002,"15min":0.5},"max":{"1min":1.0820000000000001,"5min":1.171,"15min":8.4900000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.90600000000000003,"15min":0.97499999999999998},"min":{"1min":0.78500000000000003,"5min":0.66900000000000004,"15min":0.48799999999999999},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":34.698},"last":0.85099999999999998}]},{"osd":6,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81000000000000005},"min":{"1min":0.64200000000000002,"5min":0.58099999999999996,"15min":0.47799999999999998},"max":{"1min":1.097,"5min":1.1439999999999999,"15min":1.2789999999999999},"last":0.749},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.878,"15min":0.88700000000000001},"min":{"1min":0.64100000000000001,"5min":0.629,"15min":0.50900000000000001},"max":{"1min":1.304,"5min":1.304,"15min":8.8800000000000008},"last":0.78600000000000003}]},{"osd":7,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82499999999999996,"15min":0.84099999999999997},"min":{"1min":0.65000000000000002,"5min":0.53000000000000003,"15min":0.437},"max":{"1min":1.042,"5min":1.1930000000000001,"15min":1.304},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.88900000000000001,"15min":0.92900000000000005},"min":{"1min":0.70699999999999996,"5min":0.60599999999999998,"15min":0.55900000000000005},"max":{"1min":1.1479999999999999,"5min":1.323,"15min":8.5549999999999997},"last":0.81200000000000006}]}]},{"osd":5,"up_from":14,"seq":60129542404,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468336,"kb_used_data":378856,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":93903504,"statfs":{"total":96636764160,"available":96157188096,"internally_reserved":0,"allocated":387948544,"data_stored":379823129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":91617737},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.88300000000000001,"15min":0.84399999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.41899999999999998},"max":{"1min":1.3129999999999999,"5min":1.627,"15min":1.627},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.87,"5min":0.88900000000000001,"15min":0.84999999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.495},"max":{"1min":1.2649999999999999,"5min":1.468,"15min":1.468},"last":0.872}]},{"osd":1,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.82899999999999996,"15min":0.91300000000000003},"min":{"1min":0.52800000000000002,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.3089999999999999,"5min":1.363,"15min":1.746},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.82399999999999995,"15min":0.88},"min":{"1min":0.52700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2969999999999999,"5min":1.3169999999999999,"15min":1.337},"last":0.71499999999999997}]},{"osd":2,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89700000000000002,"15min":0.872},"min":{"1min":0.60299999999999998,"5min":0.53800000000000003,"15min":0.501},"max":{"1min":1.4139999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.86199999999999999,"15min":0.83899999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.442},"max":{"1min":1.1699999999999999,"5min":1.29,"15min":1.4770000000000001},"last":0.83699999999999997}]},{"osd":3,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.9850000000000001,"15min":1.3180000000000001},"min":{"1min":0.63900000000000001,"5min":0.54400000000000004,"15min":0.504},"max":{"1min":1.3640000000000001,"5min":104.88500000000001,"15min":104.88500000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.034,"5min":2.0619999999999998,"15min":1.3700000000000001},"min":{"1min":0.71399999999999997,"5min":0.624,"15min":0.60699999999999998},"max":{"1min":1.405,"5min":105.042,"15min":105.042},"last":0.76100000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0349999999999999,"15min":0.94099999999999995},"min":{"1min":0.69899999999999995,"5min":0.58699999999999997,"15min":0.38500000000000001},"max":{"1min":1.262,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.077,"15min":1.1220000000000001},"min":{"1min":0.68100000000000005,"5min":0.59799999999999998,"15min":0.57399999999999995},"max":{"1min":1.5229999999999999,"5min":1.542,"15min":1.712},"last":1.0960000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.96199999999999997,"15min":0.95599999999999996},"min":{"1min":0.77800000000000002,"5min":0.627,"15min":0.35899999999999999},"max":{"1min":1.389,"5min":1.3899999999999999,"15min":1.9410000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.083,"5min":1.0649999999999999,"15min":1.056},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.48899999999999999},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":2.0299999999999998},"last":1.0669999999999999}]},{"osd":7,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.1080000000000001,"15min":1.095},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.40600000000000003},"max":{"1min":1.357,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.018},{"interface":"front","average":{"1min":1.111,"5min":1.113,"15min":1.1120000000000001},"min":{"1min":0.88,"5min":0.70499999999999996,"15min":0.53300000000000003},"max":{"1min":1.5149999999999999,"5min":1.5660000000000001,"15min":1.7689999999999999},"last":0.997}]}]},{"osd":3,"up_from":14,"seq":60129542406,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":445884,"kb_used_data":352244,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":93925956,"statfs":{"total":96636764160,"available":96180178944,"internally_reserved":0,"allocated":360697856,"data_stored":352338555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":95877578},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:29 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42999999999999999,"15min":0.436},"min":{"1min":0.26600000000000001,"5min":0.25600000000000001,"15min":0.224},"max":{"1min":0.68100000000000005,"5min":0.77100000000000002,"15min":2.1459999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.46700000000000003,"15min":0.54800000000000004},"min":{"1min":0.32100000000000001,"5min":0.28399999999999997,"15min":0.27200000000000002},"max":{"1min":0.59999999999999998,"5min":0.77900000000000003,"15min":21.495999999999999},"last":0.33300000000000002}]},{"osd":1,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.52100000000000002,"15min":0.53200000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72999999999999998,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48999999999999999,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.314,"15min":0.30399999999999999},"max":{"1min":0.68500000000000005,"5min":1.2450000000000001,"15min":15.102},"last":0.47799999999999998}]},{"osd":2,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53900000000000003,"15min":0.60899999999999999},"min":{"1min":0.33300000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.75800000000000001,"5min":1.044,"15min":21.321000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.495,"5min":0.39000000000000001,"15min":0.379},"max":{"1min":0.77000000000000002,"5min":0.97099999999999997,"15min":1.681},"last":0.55200000000000005}]},{"osd":4,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.80600000000000005,"15min":0.90000000000000002},"min":{"1min":0.57999999999999996,"5min":0.49199999999999999,"15min":0.48499999999999999},"max":{"1min":1.21,"5min":1.21,"15min":21.382999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.879,"15min":0.91900000000000004},"min":{"1min":0.73799999999999999,"5min":0.60399999999999998,"15min":0.46700000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":15.231999999999999},"last":0.85299999999999998}]},{"osd":5,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.84399999999999997,"15min":0.89500000000000002},"min":{"1min":0.58699999999999997,"5min":0.54200000000000004,"15min":0.47799999999999998},"max":{"1min":1.0600000000000001,"5min":1.1879999999999999,"15min":15.273999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.90500000000000003,"15min":0.95399999999999996},"min":{"1min":0.69499999999999995,"5min":0.67700000000000005,"15min":0.57499999999999996},"max":{"1min":1.1619999999999999,"5min":1.2170000000000001,"15min":15.359999999999999},"last":0.877}]},{"osd":6,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79400000000000004,"15min":0.81699999999999995},"min":{"1min":0.61099999999999999,"5min":0.53200000000000003,"15min":0.51100000000000001},"max":{"1min":1.099,"5min":1.296,"15min":1.5620000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81599999999999995,"15min":0.89100000000000001},"min":{"1min":0.66000000000000003,"5min":0.55700000000000005,"15min":0.498},"max":{"1min":1.04,"5min":1.2210000000000001,"15min":15.313000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Mon Oct 30 12:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.85199999999999998,"15min":0.92700000000000005},"min":{"1min":0.65000000000000002,"5min":0.58799999999999997,"15min":0.55200000000000005},"max":{"1min":1.0620000000000001,"5min":1.137,"15min":21.428999999999998},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.85299999999999998,"15min":0.879},"min":{"1min":0.81699999999999995,"5min":0.56899999999999995,"15min":0.56799999999999995},"max":{"1min":1.1180000000000001,"5min":1.28,"15min":1.3100000000000001},"last":0.71899999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542404,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298024,"kb_used_data":230816,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94073816,"statfs":{"total":96636764160,"available":96331587584,"internally_reserved":0,"allocated":236355584,"data_stored":231372194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.68500000000000005,"15min":0.54400000000000004},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.62,"5min":15.343999999999999,"15min":15.343999999999999},"last":0.434},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.73299999999999998,"15min":0.59999999999999998},"min":{"1min":0.34100000000000003,"5min":0.313,"15min":0.29899999999999999},"max":{"1min":0.66800000000000004,"5min":14.68,"15min":14.68},"last":0.45200000000000001}]},{"osd":2,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.66700000000000004,"15min":0.59799999999999998},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68999999999999995,"5min":8.1359999999999992,"15min":8.1359999999999992},"last":0.438},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.33900000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69899999999999995,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.505}]},{"osd":3,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.65100000000000002,"15min":0.59599999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.81200000000000006,"5min":8.0920000000000005,"15min":8.0920000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59399999999999997,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.35199999999999998,"15min":0.32000000000000001},"max":{"1min":0.79500000000000004,"5min":5.9199999999999999,"15min":5.9199999999999999},"last":0.40799999999999997}]},{"osd":4,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":1.133,"15min":0.94099999999999995},"min":{"1min":0.629,"5min":0.60499999999999998,"15min":0.41799999999999998},"max":{"1min":1.2010000000000001,"5min":21.27,"15min":21.27},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.93899999999999995,"15min":0.90700000000000003},"min":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.53600000000000003},"max":{"1min":1.3420000000000001,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.95899999999999996}]},{"osd":5,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.83099999999999996,"15min":0.82599999999999996},"min":{"1min":0.63900000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0089999999999999,"5min":1.355,"15min":1.355},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.872,"15min":0.85599999999999998},"min":{"1min":0.68300000000000005,"5min":0.499,"15min":0.46200000000000002},"max":{"1min":1.1599999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.78600000000000003}]},{"osd":6,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85599999999999998,"15min":0.83999999999999997},"min":{"1min":0.70999999999999996,"5min":0.61399999999999999,"15min":0.51300000000000001},"max":{"1min":1.111,"5min":1.29,"15min":1.4450000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.88400000000000001,"15min":0.88500000000000001},"min":{"1min":0.64600000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.246,"5min":1.323,"15min":1.468},"last":0.89500000000000002}]},{"osd":7,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.871,"15min":0.85299999999999998},"min":{"1min":0.67800000000000005,"5min":0.629,"15min":0.496},"max":{"1min":1.304,"5min":1.304,"15min":1.4299999999999999},"last":1.018},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.86399999999999999,"15min":0.874},"min":{"1min":0.76200000000000001,"5min":0.63100000000000001,"15min":0.443},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2410000000000001},"last":0.745}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":74899456,"data_stored":74883644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96493568,"data_stored":96469936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42975232,"data_stored":42953409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":85307392,"data_stored":85285250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":88555520,"data_stored":88540092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":88731648,"data_stored":88710857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":110551040,"data_stored":110530166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":129425408,"data_stored":129409664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272936960,"data_stored":265027314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139231232,"data_stored":134593460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":198570075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":274759680,"data_stored":266744507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":237875200,"data_stored":230026345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":298586112,"data_stored":290803474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":195035136,"data_stored":190043127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":280907776,"data_stored":273177138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:56:22.142 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.143 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.143 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.143 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.144 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.144 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.144 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.145 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.145 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.145 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.146 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.146 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.147 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.147 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.147 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.148 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.148 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.148 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.149 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.149 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.149 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.150 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.150 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.150 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.151 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.151 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.152 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.152 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.152 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.153 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.153 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.154 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.154 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.154 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.155 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.155 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.156 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.156 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.156 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.157 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.157 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.157 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.158 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.158 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.159 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.159 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.159 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.160 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.160 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.161 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.161 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.161 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.162 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.162 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.163 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.163 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.163 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.164 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.164 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.165 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.165 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.166 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.166 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.166 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.167 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-30T12:34:53.444020+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=34, tm_sec=53, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.167 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.167 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.168 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.168 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.169 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.169 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.169 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.170 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.170 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.170 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.171 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.171 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.171 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.172 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.172 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.173 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.173 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.173 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.174 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.174 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.174 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.175 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.175 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.176 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.176 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.176 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.177 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.177 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.177 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.178 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.178 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.179 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:22.179 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-30T12:56:38.779 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:56:38.783 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:56:42.180 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:56:42.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.327+0000 7ff53be6b640 1 Processor -- start 2023-10-30T12:56:42.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.327+0000 7ff53be6b640 1 -- start start 2023-10-30T12:56:42.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.331+0000 7ff53be6b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff5340cf160 0x7ff5340cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:42.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.331+0000 7ff53be6b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5340cb800 con 0x7ff5341538a0 2023-10-30T12:56:42.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.331+0000 7ff53be6b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5340cb330 con 0x7ff53415d6f0 2023-10-30T12:56:42.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.331+0000 7ff53be6b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5340cad70 con 0x7ff5340cf160 2023-10-30T12:56:42.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.331+0000 7ff539c07640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff5341538a0 0x7ff5340ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33230/0 (socket says 172.21.15.12:33230) 2023-10-30T12:56:42.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.331+0000 7ff539c07640 1 -- 172.21.15.12:0/94535532 learned_addr learned my addr 172.21.15.12:0/94535532 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:56:42.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.331+0000 7ff53a408640 1 --2- 172.21.15.12:0/94535532 >> v2:172.21.15.154:3300/0 conn(0x7ff5340cf160 0x7ff5340cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:42.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.331+0000 7ff538c05640 1 -- 172.21.15.12:0/94535532 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487018333 0 0) 0x7ff5340cb800 con 0x7ff5341538a0 2023-10-30T12:56:42.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.331+0000 7ff538c05640 1 -- 172.21.15.12:0/94535532 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff520003530 con 0x7ff5341538a0 2023-10-30T12:56:42.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.331+0000 7ff53a408640 1 -- 172.21.15.12:0/94535532 >> v1:172.21.15.154:6789/0 conn(0x7ff53415d6f0 legacy=0x7ff53415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:42.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.331+0000 7ff53a408640 1 -- 172.21.15.12:0/94535532 >> v1:172.21.15.12:6789/0 conn(0x7ff5341538a0 legacy=0x7ff5340ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:42.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.331+0000 7ff53a408640 1 -- 172.21.15.12:0/94535532 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5340cb050 con 0x7ff5340cf160 2023-10-30T12:56:42.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53a408640 1 --2- 172.21.15.12:0/94535532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5340cf160 0x7ff5340cf590 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009d90 tx=0x7ff52c01f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0e490f3f84239b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:42.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff538c05640 1 -- 172.21.15.12:0/94535532 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0af600 con 0x7ff5340cf160 2023-10-30T12:56:42.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff538c05640 1 -- 172.21.15.12:0/94535532 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c0ab050 con 0x7ff5340cf160 2023-10-30T12:56:42.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff538c05640 1 -- 172.21.15.12:0/94535532 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0af960 con 0x7ff5340cf160 2023-10-30T12:56:42.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53be6b640 1 -- 172.21.15.12:0/94535532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5340cf160 msgr2=0x7ff5340cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:42.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53be6b640 1 --2- 172.21.15.12:0/94535532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5340cf160 0x7ff5340cf590 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009d90 tx=0x7ff52c01f130 comp rx=0 tx=0).stop 2023-10-30T12:56:42.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53be6b640 1 -- 172.21.15.12:0/94535532 shutdown_connections 2023-10-30T12:56:42.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53be6b640 1 --2- 172.21.15.12:0/94535532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5340cf160 0x7ff5340cf590 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:42.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53be6b640 1 -- 172.21.15.12:0/94535532 >> 172.21.15.12:0/94535532 conn(0x7ff53406bfe0 msgr2=0x7ff53406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:42.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53be6b640 1 -- 172.21.15.12:0/94535532 shutdown_connections 2023-10-30T12:56:42.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53be6b640 1 -- 172.21.15.12:0/94535532 wait complete. 2023-10-30T12:56:42.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53be6b640 1 Processor -- start 2023-10-30T12:56:42.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53be6b640 1 -- start start 2023-10-30T12:56:42.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53be6b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5340cf160 0x7ff5340ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:42.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53be6b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5341538a0 0x7ff534157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:42.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53be6b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53415d6f0 0x7ff53415b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:42.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53be6b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5341590f0 con 0x7ff5340cf160 2023-10-30T12:56:42.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53be6b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340ce3d0 con 0x7ff5341538a0 2023-10-30T12:56:42.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.335+0000 7ff53be6b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340b3530 con 0x7ff53415d6f0 2023-10-30T12:56:42.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff539c07640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5340cf160 0x7ff5340ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:42.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff539c07640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5340cf160 0x7ff5340ce790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56788/0 (socket says 172.21.15.12:56788) 2023-10-30T12:56:42.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff539c07640 1 -- 172.21.15.12:0/1298214502 learned_addr learned my addr 172.21.15.12:0/1298214502 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:56:42.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff539406640 1 --2- 172.21.15.12:0/1298214502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5341538a0 0x7ff534157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:42.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff53a408640 1 --2- 172.21.15.12:0/1298214502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53415d6f0 0x7ff53415b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:42.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff539c07640 1 -- 172.21.15.12:0/1298214502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53415d6f0 msgr2=0x7ff53415b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:42.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff539c07640 1 --2- 172.21.15.12:0/1298214502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53415d6f0 0x7ff53415b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:42.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff539c07640 1 -- 172.21.15.12:0/1298214502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5341538a0 msgr2=0x7ff534157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:42.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff539c07640 1 --2- 172.21.15.12:0/1298214502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5341538a0 0x7ff534157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff539c07640 1 -- 172.21.15.12:0/1298214502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c009a70 con 0x7ff5340cf160 2023-10-30T12:56:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff539c07640 1 --2- 172.21.15.12:0/1298214502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5340cf160 0x7ff5340ce790 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff53000c4d0 tx=0x7ff53000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff52a7fc640 1 -- 172.21.15.12:0/1298214502 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530005cf0 con 0x7ff5340cf160 2023-10-30T12:56:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff52a7fc640 1 -- 172.21.15.12:0/1298214502 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff530005e50 con 0x7ff5340cf160 2023-10-30T12:56:42.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff53be6b640 1 -- 172.21.15.12:0/1298214502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff534159320 con 0x7ff5340cf160 2023-10-30T12:56:42.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff52a7fc640 1 -- 172.21.15.12:0/1298214502 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530007270 con 0x7ff5340cf160 2023-10-30T12:56:42.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff53be6b640 1 -- 172.21.15.12:0/1298214502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5340ce060 con 0x7ff5340cf160 2023-10-30T12:56:42.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff53be6b640 1 -- 172.21.15.12:0/1298214502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff534157b60 con 0x7ff5340cf160 2023-10-30T12:56:42.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.339+0000 7ff52a7fc640 1 -- 172.21.15.12:0/1298214502 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff5300073d0 con 0x7ff5340cf160 2023-10-30T12:56:42.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.343+0000 7ff52a7fc640 1 --2- 172.21.15.12:0/1298214502 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5180664a0 0x7ff518068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:42.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.343+0000 7ff52a7fc640 1 -- 172.21.15.12:0/1298214502 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7ff5300837a0 con 0x7ff5340cf160 2023-10-30T12:56:42.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.343+0000 7ff539406640 1 --2- 172.21.15.12:0/1298214502 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5180664a0 0x7ff518068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:42.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.343+0000 7ff539406640 1 --2- 172.21.15.12:0/1298214502 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5180664a0 0x7ff518068960 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff53414d660 tx=0x7ff524009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.347+0000 7ff52a7fc640 1 -- 172.21.15.12:0/1298214502 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff530050860 con 0x7ff5340cf160 2023-10-30T12:56:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.523+0000 7ff53be6b640 1 -- 172.21.15.12:0/1298214502 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff534159b70 con 0x7ff5180664a0 2023-10-30T12:56:42.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.543+0000 7ff52a7fc640 1 -- 172.21.15.12:0/1298214502 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+291978 (secure 0 0 0) 0x7ff5240013b0 con 0x7ff5180664a0 2023-10-30T12:56:42.551 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:56:42.553 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-10-30T12:56:42.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.547+0000 7ff53be6b640 1 -- 172.21.15.12:0/1298214502 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5180664a0 msgr2=0x7ff518068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:42.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.547+0000 7ff53be6b640 1 --2- 172.21.15.12:0/1298214502 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5180664a0 0x7ff518068960 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff53414d660 tx=0x7ff524009040 comp rx=0 tx=0).stop 2023-10-30T12:56:42.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.547+0000 7ff53be6b640 1 -- 172.21.15.12:0/1298214502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5340cf160 msgr2=0x7ff5340ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:42.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.547+0000 7ff53be6b640 1 --2- 172.21.15.12:0/1298214502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5340cf160 0x7ff5340ce790 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff53000c4d0 tx=0x7ff53000c990 comp rx=0 tx=0).stop 2023-10-30T12:56:42.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.547+0000 7ff53be6b640 1 -- 172.21.15.12:0/1298214502 shutdown_connections 2023-10-30T12:56:42.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.547+0000 7ff53be6b640 1 --2- 172.21.15.12:0/1298214502 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5180664a0 0x7ff518068960 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.547+0000 7ff53be6b640 1 --2- 172.21.15.12:0/1298214502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53415d6f0 0x7ff53415b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.547+0000 7ff53be6b640 1 --2- 172.21.15.12:0/1298214502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5341538a0 0x7ff534157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.547+0000 7ff53be6b640 1 --2- 172.21.15.12:0/1298214502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5340cf160 0x7ff5340ce790 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.547+0000 7ff53be6b640 1 -- 172.21.15.12:0/1298214502 >> 172.21.15.12:0/1298214502 conn(0x7ff53406bfe0 msgr2=0x7ff53415aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.547+0000 7ff53be6b640 1 -- 172.21.15.12:0/1298214502 shutdown_connections 2023-10-30T12:56:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:56:42.547+0000 7ff53be6b640 1 -- 172.21.15.12:0/1298214502 wait complete. 2023-10-30T12:56:42.580 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":703,"stamp":"2023-10-30T12:56:41.419838+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1286445879,"num_objects":10515,"num_object_clones":0,"num_object_copies":21030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10515,"num_whiteouts":0,"num_read":5687,"num_read_kb":1040706,"num_write":53343,"num_write_kb":2315641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":548331,"num_omap_keys":1024,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60167,"ondisk_log_size":60167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3242656,"kb_used_data":2568608,"kb_used_omap":12,"kb_used_meta":673971,"kb_avail":751732064,"statfs":{"total":773094113280,"available":769773633536,"internally_reserved":0,"allocated":2630254592,"data_stored":2573822046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12729,"internal_metadata":690146887},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":138422,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":229197,"num_omap_keys":428,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004927"},"pg_stats":[{"pgid":"3.3f","version":"55'737","reported_seq":2903,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.185374+0000","last_change":"2023-10-30T12:42:02.472363+0000","last_active":"2023-10-30T12:56:14.185374+0000","last_peered":"2023-10-30T12:56:14.185374+0000","last_clean":"2023-10-30T12:56:14.185374+0000","last_became_active":"2023-10-30T12:35:06.222724+0000","last_became_peered":"2023-10-30T12:35:06.222724+0000","last_unstale":"2023-10-30T12:56:14.185374+0000","last_undegraded":"2023-10-30T12:56:14.185374+0000","last_fullsized":"2023-10-30T12:56:14.185374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059524979999999996,"stat_sum":{"num_bytes":14421277,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":73,"num_read_kb":8207,"num_write":692,"num_write_kb":25708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"55'711","reported_seq":2874,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.163338+0000","last_change":"2023-10-30T12:42:02.711331+0000","last_active":"2023-10-30T12:56:14.163338+0000","last_peered":"2023-10-30T12:56:14.163338+0000","last_clean":"2023-10-30T12:56:14.163338+0000","last_became_active":"2023-10-30T12:35:06.182441+0000","last_became_peered":"2023-10-30T12:35:06.182441+0000","last_unstale":"2023-10-30T12:56:14.163338+0000","last_undegraded":"2023-10-30T12:56:14.163338+0000","last_fullsized":"2023-10-30T12:56:14.163338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026942979999999999,"stat_sum":{"num_bytes":14617239,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":70,"num_read_kb":10,"num_write":675,"num_write_kb":36983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"55'697","reported_seq":2870,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.158119+0000","last_change":"2023-10-30T12:42:02.724489+0000","last_active":"2023-10-30T12:56:14.158119+0000","last_peered":"2023-10-30T12:56:14.158119+0000","last_clean":"2023-10-30T12:56:14.158119+0000","last_became_active":"2023-10-30T12:35:06.180132+0000","last_became_peered":"2023-10-30T12:35:06.180132+0000","last_unstale":"2023-10-30T12:56:14.158119+0000","last_undegraded":"2023-10-30T12:56:14.158119+0000","last_fullsized":"2023-10-30T12:56:14.158119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003537278,"stat_sum":{"num_bytes":22250243,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":80,"num_read_kb":12305,"num_write":653,"num_write_kb":42611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"55'682","reported_seq":2832,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.197902+0000","last_change":"2023-10-30T12:42:02.742917+0000","last_active":"2023-10-30T12:56:14.197902+0000","last_peered":"2023-10-30T12:56:14.197902+0000","last_clean":"2023-10-30T12:56:14.197902+0000","last_became_active":"2023-10-30T12:35:06.213987+0000","last_became_peered":"2023-10-30T12:35:06.213987+0000","last_unstale":"2023-10-30T12:56:14.197902+0000","last_undegraded":"2023-10-30T12:56:14.197902+0000","last_fullsized":"2023-10-30T12:56:14.197902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013528057,"stat_sum":{"num_bytes":18553875,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":57,"num_read_kb":8205,"num_write":627,"num_write_kb":28534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"55'670","reported_seq":2812,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.184369+0000","last_change":"2023-10-30T12:42:02.470142+0000","last_active":"2023-10-30T12:56:14.184369+0000","last_peered":"2023-10-30T12:56:14.184369+0000","last_clean":"2023-10-30T12:56:14.184369+0000","last_became_active":"2023-10-30T12:35:06.214964+0000","last_became_peered":"2023-10-30T12:35:06.214964+0000","last_unstale":"2023-10-30T12:56:14.184369+0000","last_undegraded":"2023-10-30T12:56:14.184369+0000","last_fullsized":"2023-10-30T12:56:14.184369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046834600000000004,"stat_sum":{"num_bytes":8686005,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":51,"num_read_kb":4112,"num_write":616,"num_write_kb":25845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"55'687","reported_seq":2839,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.194822+0000","last_change":"2023-10-30T12:42:02.747177+0000","last_active":"2023-10-30T12:56:14.194822+0000","last_peered":"2023-10-30T12:56:14.194822+0000","last_clean":"2023-10-30T12:56:14.194822+0000","last_became_active":"2023-10-30T12:35:06.214219+0000","last_became_peered":"2023-10-30T12:35:06.214219+0000","last_unstale":"2023-10-30T12:56:14.194822+0000","last_undegraded":"2023-10-30T12:56:14.194822+0000","last_fullsized":"2023-10-30T12:56:14.194822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015862434000000002,"stat_sum":{"num_bytes":14811547,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":61,"num_read_kb":16,"num_write":643,"num_write_kb":33253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"55'697","reported_seq":2839,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.171845+0000","last_change":"2023-10-30T12:42:02.467959+0000","last_active":"2023-10-30T12:56:14.171845+0000","last_peered":"2023-10-30T12:56:14.171845+0000","last_clean":"2023-10-30T12:56:14.171845+0000","last_became_active":"2023-10-30T12:35:06.216468+0000","last_became_peered":"2023-10-30T12:35:06.216468+0000","last_unstale":"2023-10-30T12:56:14.171845+0000","last_undegraded":"2023-10-30T12:56:14.171845+0000","last_fullsized":"2023-10-30T12:56:14.171845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031309480000000002,"stat_sum":{"num_bytes":9595467,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":52,"num_read_kb":12216,"num_write":649,"num_write_kb":36101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"55'625","reported_seq":2772,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.176378+0000","last_change":"2023-10-30T12:42:02.241155+0000","last_active":"2023-10-30T12:56:14.176378+0000","last_peered":"2023-10-30T12:56:14.176378+0000","last_clean":"2023-10-30T12:56:14.176378+0000","last_became_active":"2023-10-30T12:35:06.214384+0000","last_became_peered":"2023-10-30T12:35:06.214384+0000","last_unstale":"2023-10-30T12:56:14.176378+0000","last_undegraded":"2023-10-30T12:56:14.176378+0000","last_fullsized":"2023-10-30T12:56:14.176378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045552559999999997,"stat_sum":{"num_bytes":9825651,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":56,"num_read_kb":15,"num_write":593,"num_write_kb":23392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"55'695","reported_seq":2858,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.185282+0000","last_change":"2023-10-30T12:42:02.473886+0000","last_active":"2023-10-30T12:56:14.185282+0000","last_peered":"2023-10-30T12:56:14.185282+0000","last_clean":"2023-10-30T12:56:14.185282+0000","last_became_active":"2023-10-30T12:35:06.222692+0000","last_became_peered":"2023-10-30T12:35:06.222692+0000","last_unstale":"2023-10-30T12:56:14.185282+0000","last_undegraded":"2023-10-30T12:56:14.185282+0000","last_fullsized":"2023-10-30T12:56:14.185282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074617599999999996,"stat_sum":{"num_bytes":5524945,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":72,"num_read_kb":4115,"num_write":631,"num_write_kb":10887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"55'686","reported_seq":2841,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.176360+0000","last_change":"2023-10-30T12:42:02.739661+0000","last_active":"2023-10-30T12:56:14.176360+0000","last_peered":"2023-10-30T12:56:14.176360+0000","last_clean":"2023-10-30T12:56:14.176360+0000","last_became_active":"2023-10-30T12:35:06.215384+0000","last_became_peered":"2023-10-30T12:35:06.215384+0000","last_unstale":"2023-10-30T12:56:14.176360+0000","last_undegraded":"2023-10-30T12:56:14.176360+0000","last_fullsized":"2023-10-30T12:56:14.176360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080059940000000007,"stat_sum":{"num_bytes":17915594,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":64,"num_read_kb":28,"num_write":636,"num_write_kb":19744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"55'698","reported_seq":2856,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.180780+0000","last_change":"2023-10-30T12:42:02.739462+0000","last_active":"2023-10-30T12:56:14.180780+0000","last_peered":"2023-10-30T12:56:14.180780+0000","last_clean":"2023-10-30T12:56:14.180780+0000","last_became_active":"2023-10-30T12:35:06.215355+0000","last_became_peered":"2023-10-30T12:35:06.215355+0000","last_unstale":"2023-10-30T12:56:14.180780+0000","last_undegraded":"2023-10-30T12:56:14.180780+0000","last_fullsized":"2023-10-30T12:56:14.180780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011778769,"stat_sum":{"num_bytes":10475791,"num_objects":135,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":67,"num_read_kb":8209,"num_write":651,"num_write_kb":25832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"55'780","reported_seq":2950,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:27.866591+0000","last_change":"2023-10-30T12:56:27.866741+0000","last_active":"2023-10-30T12:56:27.866591+0000","last_peered":"2023-10-30T12:56:27.866591+0000","last_clean":"2023-10-30T12:56:27.866591+0000","last_became_active":"2023-10-30T12:35:06.196503+0000","last_became_peered":"2023-10-30T12:35:06.196503+0000","last_unstale":"2023-10-30T12:56:27.866591+0000","last_undegraded":"2023-10-30T12:56:27.866591+0000","last_fullsized":"2023-10-30T12:56:27.866591+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'780","last_scrub_stamp":"2023-10-30T12:56:27.866427+0000","last_deep_scrub":"55'780","last_deep_scrub_stamp":"2023-10-30T12:56:27.866427+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.866427+0000","objects_scrubbed":163,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:24:58.673270+0000","scrub_duration":0.27317086899999998,"objects_trimmed":0,"snaptrim_duration":0.0030385149999999999,"stat_sum":{"num_bytes":16276064,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":65,"num_read_kb":7648,"num_write":632,"num_write_kb":36085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"55'724","reported_seq":2892,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.145453+0000","last_change":"2023-10-30T12:42:02.241621+0000","last_active":"2023-10-30T12:56:14.145453+0000","last_peered":"2023-10-30T12:56:14.145453+0000","last_clean":"2023-10-30T12:56:14.145453+0000","last_became_active":"2023-10-30T12:35:06.195408+0000","last_became_peered":"2023-10-30T12:35:06.195408+0000","last_unstale":"2023-10-30T12:56:14.145453+0000","last_undegraded":"2023-10-30T12:56:14.145453+0000","last_fullsized":"2023-10-30T12:56:14.145453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004560812,"stat_sum":{"num_bytes":30579039,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":77,"num_read_kb":9808,"num_write":665,"num_write_kb":49630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"55'687","reported_seq":2850,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.162607+0000","last_change":"2023-10-30T12:42:02.722894+0000","last_active":"2023-10-30T12:56:14.162607+0000","last_peered":"2023-10-30T12:56:14.162607+0000","last_clean":"2023-10-30T12:56:14.162607+0000","last_became_active":"2023-10-30T12:35:06.182573+0000","last_became_peered":"2023-10-30T12:35:06.182573+0000","last_unstale":"2023-10-30T12:56:14.162607+0000","last_undegraded":"2023-10-30T12:56:14.162607+0000","last_fullsized":"2023-10-30T12:56:14.162607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003020062,"stat_sum":{"num_bytes":14557524,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":71,"num_read_kb":4113,"num_write":642,"num_write_kb":31601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"55'700","reported_seq":2855,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.169649+0000","last_change":"2023-10-30T12:42:02.710474+0000","last_active":"2023-10-30T12:56:14.169649+0000","last_peered":"2023-10-30T12:56:14.169649+0000","last_clean":"2023-10-30T12:56:14.169649+0000","last_became_active":"2023-10-30T12:35:06.185290+0000","last_became_peered":"2023-10-30T12:35:06.185290+0000","last_unstale":"2023-10-30T12:56:14.169649+0000","last_undegraded":"2023-10-30T12:56:14.169649+0000","last_fullsized":"2023-10-30T12:56:14.169649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002282292,"stat_sum":{"num_bytes":14763187,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":63,"num_read_kb":11730,"num_write":643,"num_write_kb":27692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"55'718","reported_seq":2869,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.187624+0000","last_change":"2023-10-30T12:42:02.748073+0000","last_active":"2023-10-30T12:56:14.187624+0000","last_peered":"2023-10-30T12:56:14.187624+0000","last_clean":"2023-10-30T12:56:14.187624+0000","last_became_active":"2023-10-30T12:35:06.213627+0000","last_became_peered":"2023-10-30T12:35:06.213627+0000","last_unstale":"2023-10-30T12:56:14.187624+0000","last_undegraded":"2023-10-30T12:56:14.187624+0000","last_fullsized":"2023-10-30T12:56:14.187624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014095760000000001,"stat_sum":{"num_bytes":9839966,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":59,"num_read_kb":4114,"num_write":651,"num_write_kb":31705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"55'739","reported_seq":2893,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.185634+0000","last_change":"2023-10-30T12:42:02.745783+0000","last_active":"2023-10-30T12:56:14.185634+0000","last_peered":"2023-10-30T12:56:14.185634+0000","last_clean":"2023-10-30T12:56:14.185634+0000","last_became_active":"2023-10-30T12:35:06.221696+0000","last_became_peered":"2023-10-30T12:35:06.221696+0000","last_unstale":"2023-10-30T12:56:14.185634+0000","last_undegraded":"2023-10-30T12:56:14.185634+0000","last_fullsized":"2023-10-30T12:56:14.185634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015743672,"stat_sum":{"num_bytes":28343169,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":62,"num_read_kb":16401,"num_write":691,"num_write_kb":59410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"55'804","reported_seq":2992,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.435565+0000","last_change":"2023-10-30T12:56:24.435609+0000","last_active":"2023-10-30T12:56:24.435565+0000","last_peered":"2023-10-30T12:56:24.435565+0000","last_clean":"2023-10-30T12:56:24.435565+0000","last_became_active":"2023-10-30T12:35:06.198074+0000","last_became_peered":"2023-10-30T12:35:06.198074+0000","last_unstale":"2023-10-30T12:56:24.435565+0000","last_undegraded":"2023-10-30T12:56:24.435565+0000","last_fullsized":"2023-10-30T12:56:24.435565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'804","last_scrub_stamp":"2023-10-30T12:56:24.435488+0000","last_deep_scrub":"55'804","last_deep_scrub_stamp":"2023-10-30T12:56:24.435488+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.435488+0000","objects_scrubbed":156,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:44:11.442590+0000","scrub_duration":0.24729864700000001,"objects_trimmed":0,"snaptrim_duration":0.0023389270000000002,"stat_sum":{"num_bytes":28976684,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":82,"num_read_kb":27,"num_write":642,"num_write_kb":39937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"55'706","reported_seq":2858,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.837938+0000","last_change":"2023-10-30T12:42:02.473536+0000","last_active":"2023-10-30T12:56:14.837938+0000","last_peered":"2023-10-30T12:56:14.837938+0000","last_clean":"2023-10-30T12:56:14.837938+0000","last_became_active":"2023-10-30T12:35:06.215935+0000","last_became_peered":"2023-10-30T12:35:06.215935+0000","last_unstale":"2023-10-30T12:56:14.837938+0000","last_undegraded":"2023-10-30T12:56:14.837938+0000","last_fullsized":"2023-10-30T12:56:14.837938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062537060000000004,"stat_sum":{"num_bytes":21535467,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":60,"num_read_kb":12305,"num_write":656,"num_write_kb":39478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"55'805","reported_seq":2985,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:30.019053+0000","last_change":"2023-10-30T12:56:30.019160+0000","last_active":"2023-10-30T12:56:30.019053+0000","last_peered":"2023-10-30T12:56:30.019053+0000","last_clean":"2023-10-30T12:56:30.019053+0000","last_became_active":"2023-10-30T12:35:06.199812+0000","last_became_peered":"2023-10-30T12:35:06.199812+0000","last_unstale":"2023-10-30T12:56:30.019053+0000","last_undegraded":"2023-10-30T12:56:30.019053+0000","last_fullsized":"2023-10-30T12:56:30.019053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'805","last_scrub_stamp":"2023-10-30T12:56:30.018936+0000","last_deep_scrub":"55'805","last_deep_scrub_stamp":"2023-10-30T12:56:30.018936+0000","last_clean_scrub_stamp":"2023-10-30T12:56:30.018936+0000","objects_scrubbed":143,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:47:14.318405+0000","scrub_duration":0.21736029700000001,"objects_trimmed":0,"snaptrim_duration":0.011375481999999999,"stat_sum":{"num_bytes":6303882,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":75,"num_read_kb":20150,"num_write":647,"num_write_kb":36292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"55'719","reported_seq":2889,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.185051+0000","last_change":"2023-10-30T12:42:02.725299+0000","last_active":"2023-10-30T12:56:14.185051+0000","last_peered":"2023-10-30T12:56:14.185051+0000","last_clean":"2023-10-30T12:56:14.185051+0000","last_became_active":"2023-10-30T12:35:06.216861+0000","last_became_peered":"2023-10-30T12:35:06.216861+0000","last_unstale":"2023-10-30T12:56:14.185051+0000","last_undegraded":"2023-10-30T12:56:14.185051+0000","last_fullsized":"2023-10-30T12:56:14.185051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040841849999999997,"stat_sum":{"num_bytes":12370338,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":78,"num_read_kb":8213,"num_write":666,"num_write_kb":26377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"55'868","reported_seq":3032,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:33.615651+0000","last_change":"2023-10-30T12:56:33.615740+0000","last_active":"2023-10-30T12:56:33.615651+0000","last_peered":"2023-10-30T12:56:33.615651+0000","last_clean":"2023-10-30T12:56:33.615651+0000","last_became_active":"2023-10-30T12:35:46.122517+0000","last_became_peered":"2023-10-30T12:35:46.122517+0000","last_unstale":"2023-10-30T12:56:33.615651+0000","last_undegraded":"2023-10-30T12:56:33.615651+0000","last_fullsized":"2023-10-30T12:56:33.615651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'868","last_scrub_stamp":"2023-10-30T12:56:33.615559+0000","last_deep_scrub":"55'868","last_deep_scrub_stamp":"2023-10-30T12:56:33.615559+0000","last_clean_scrub_stamp":"2023-10-30T12:56:33.615559+0000","objects_scrubbed":170,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:51:10.318228+0000","scrub_duration":0.26875042100000002,"objects_trimmed":0,"snaptrim_duration":0.0020435240000000001,"stat_sum":{"num_bytes":19531616,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":61,"num_read_kb":21,"num_write":688,"num_write_kb":30111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"55'701","reported_seq":2873,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.177073+0000","last_change":"2023-10-30T12:42:02.241859+0000","last_active":"2023-10-30T12:56:14.177073+0000","last_peered":"2023-10-30T12:56:14.177073+0000","last_clean":"2023-10-30T12:56:14.177073+0000","last_became_active":"2023-10-30T12:35:06.182314+0000","last_became_peered":"2023-10-30T12:35:06.182314+0000","last_unstale":"2023-10-30T12:56:14.177073+0000","last_undegraded":"2023-10-30T12:56:14.177073+0000","last_fullsized":"2023-10-30T12:56:14.177073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051111200000000002,"stat_sum":{"num_bytes":5467082,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":80,"num_read_kb":15696,"num_write":658,"num_write_kb":25502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"55'829","reported_seq":3002,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:32.815631+0000","last_change":"2023-10-30T12:56:32.815869+0000","last_active":"2023-10-30T12:56:32.815631+0000","last_peered":"2023-10-30T12:56:32.815631+0000","last_clean":"2023-10-30T12:56:32.815631+0000","last_became_active":"2023-10-30T12:35:06.213402+0000","last_became_peered":"2023-10-30T12:35:06.213402+0000","last_unstale":"2023-10-30T12:56:32.815631+0000","last_undegraded":"2023-10-30T12:56:32.815631+0000","last_fullsized":"2023-10-30T12:56:32.815631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'829","last_scrub_stamp":"2023-10-30T12:56:32.815282+0000","last_deep_scrub":"55'829","last_deep_scrub_stamp":"2023-10-30T12:56:32.815282+0000","last_clean_scrub_stamp":"2023-10-30T12:56:32.815282+0000","objects_scrubbed":159,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:42:54.304491+0000","scrub_duration":0.307345284,"objects_trimmed":0,"snaptrim_duration":0.014055767,"stat_sum":{"num_bytes":10241279,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":67,"num_read_kb":4112,"num_write":657,"num_write_kb":20530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"55'689","reported_seq":2840,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.160317+0000","last_change":"2023-10-30T12:42:02.242578+0000","last_active":"2023-10-30T12:56:14.160317+0000","last_peered":"2023-10-30T12:56:14.160317+0000","last_clean":"2023-10-30T12:56:14.160317+0000","last_became_active":"2023-10-30T12:35:06.188298+0000","last_became_peered":"2023-10-30T12:35:06.188298+0000","last_unstale":"2023-10-30T12:56:14.160317+0000","last_undegraded":"2023-10-30T12:56:14.160317+0000","last_fullsized":"2023-10-30T12:56:14.160317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053580889999999999,"stat_sum":{"num_bytes":5087264,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":59,"num_read_kb":8204,"num_write":621,"num_write_kb":44181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"55'896","reported_seq":3089,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.080001+0000","last_change":"2023-10-30T12:56:34.080071+0000","last_active":"2023-10-30T12:56:34.080001+0000","last_peered":"2023-10-30T12:56:34.080001+0000","last_clean":"2023-10-30T12:56:34.080001+0000","last_became_active":"2023-10-30T12:35:06.202507+0000","last_became_peered":"2023-10-30T12:35:06.202507+0000","last_unstale":"2023-10-30T12:56:34.080001+0000","last_undegraded":"2023-10-30T12:56:34.080001+0000","last_fullsized":"2023-10-30T12:56:34.080001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'896","last_scrub_stamp":"2023-10-30T12:56:34.079910+0000","last_deep_scrub":"55'896","last_deep_scrub_stamp":"2023-10-30T12:56:34.079910+0000","last_clean_scrub_stamp":"2023-10-30T12:56:34.079910+0000","objects_scrubbed":173,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:15:45.705065+0000","scrub_duration":0.25185000000000002,"objects_trimmed":0,"snaptrim_duration":0.0082500649999999991,"stat_sum":{"num_bytes":6587197,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":87,"num_read_kb":1892,"num_write":715,"num_write_kb":28794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"55'695","reported_seq":2850,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.161824+0000","last_change":"2023-10-30T12:42:02.710762+0000","last_active":"2023-10-30T12:56:14.161824+0000","last_peered":"2023-10-30T12:56:14.161824+0000","last_clean":"2023-10-30T12:56:14.161824+0000","last_became_active":"2023-10-30T12:35:06.170030+0000","last_became_peered":"2023-10-30T12:35:06.170030+0000","last_unstale":"2023-10-30T12:56:14.161824+0000","last_undegraded":"2023-10-30T12:56:14.161824+0000","last_fullsized":"2023-10-30T12:56:14.161824+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024132049999999999,"stat_sum":{"num_bytes":15296106,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":63,"num_read_kb":15,"num_write":636,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"55'652","reported_seq":2822,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.180291+0000","last_change":"2023-10-30T12:42:02.242975+0000","last_active":"2023-10-30T12:56:14.180291+0000","last_peered":"2023-10-30T12:56:14.180291+0000","last_clean":"2023-10-30T12:56:14.180291+0000","last_became_active":"2023-10-30T12:35:06.189171+0000","last_became_peered":"2023-10-30T12:35:06.189171+0000","last_unstale":"2023-10-30T12:56:14.180291+0000","last_undegraded":"2023-10-30T12:56:14.180291+0000","last_fullsized":"2023-10-30T12:56:14.180291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056904870000000001,"stat_sum":{"num_bytes":14055782,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":78,"num_read_kb":12372,"num_write":607,"num_write_kb":31259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"55'672","reported_seq":2825,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.182308+0000","last_change":"2023-10-30T12:42:02.725392+0000","last_active":"2023-10-30T12:56:14.182308+0000","last_peered":"2023-10-30T12:56:14.182308+0000","last_clean":"2023-10-30T12:56:14.182308+0000","last_became_active":"2023-10-30T12:35:06.183348+0000","last_became_peered":"2023-10-30T12:35:06.183348+0000","last_unstale":"2023-10-30T12:56:14.182308+0000","last_undegraded":"2023-10-30T12:56:14.182308+0000","last_fullsized":"2023-10-30T12:56:14.182308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041317979999999999,"stat_sum":{"num_bytes":14615922,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":60,"num_read_kb":7678,"num_write":625,"num_write_kb":36364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"55'742","reported_seq":2910,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.192076+0000","last_change":"2023-10-30T12:42:02.747343+0000","last_active":"2023-10-30T12:56:14.192076+0000","last_peered":"2023-10-30T12:56:14.192076+0000","last_clean":"2023-10-30T12:56:14.192076+0000","last_became_active":"2023-10-30T12:35:06.196368+0000","last_became_peered":"2023-10-30T12:35:06.196368+0000","last_unstale":"2023-10-30T12:56:14.192076+0000","last_undegraded":"2023-10-30T12:56:14.192076+0000","last_fullsized":"2023-10-30T12:56:14.192076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015569382999999999,"stat_sum":{"num_bytes":14491906,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":75,"num_read_kb":9352,"num_write":694,"num_write_kb":32749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"55'888","reported_seq":3068,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:25.513128+0000","last_change":"2023-10-30T12:56:25.513186+0000","last_active":"2023-10-30T12:56:25.513128+0000","last_peered":"2023-10-30T12:56:25.513128+0000","last_clean":"2023-10-30T12:56:25.513128+0000","last_became_active":"2023-10-30T12:35:06.213994+0000","last_became_peered":"2023-10-30T12:35:06.213994+0000","last_unstale":"2023-10-30T12:56:25.513128+0000","last_undegraded":"2023-10-30T12:56:25.513128+0000","last_fullsized":"2023-10-30T12:56:25.513128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'888","last_scrub_stamp":"2023-10-30T12:56:25.513052+0000","last_deep_scrub":"55'888","last_deep_scrub_stamp":"2023-10-30T12:56:25.513052+0000","last_clean_scrub_stamp":"2023-10-30T12:56:25.513052+0000","objects_scrubbed":188,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:49:47.291732+0000","scrub_duration":0.28763551900000001,"objects_trimmed":0,"snaptrim_duration":0.001873502,"stat_sum":{"num_bytes":23363209,"num_objects":188,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":188,"num_whiteouts":0,"num_read":72,"num_read_kb":4118,"num_write":712,"num_write_kb":32388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"55'701","reported_seq":2864,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.177515+0000","last_change":"2023-10-30T12:42:02.738216+0000","last_active":"2023-10-30T12:56:14.177515+0000","last_peered":"2023-10-30T12:56:14.177515+0000","last_clean":"2023-10-30T12:56:14.177515+0000","last_became_active":"2023-10-30T12:35:06.197647+0000","last_became_peered":"2023-10-30T12:35:06.197647+0000","last_unstale":"2023-10-30T12:56:14.177515+0000","last_undegraded":"2023-10-30T12:56:14.177515+0000","last_fullsized":"2023-10-30T12:56:14.177515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011378932,"stat_sum":{"num_bytes":18364745,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":70,"num_read_kb":8212,"num_write":651,"num_write_kb":28257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"55'773","reported_seq":2939,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:21.810651+0000","last_change":"2023-10-30T12:56:21.810710+0000","last_active":"2023-10-30T12:56:21.810651+0000","last_peered":"2023-10-30T12:56:21.810651+0000","last_clean":"2023-10-30T12:56:21.810651+0000","last_became_active":"2023-10-30T12:35:06.167558+0000","last_became_peered":"2023-10-30T12:35:06.167558+0000","last_unstale":"2023-10-30T12:56:21.810651+0000","last_undegraded":"2023-10-30T12:56:21.810651+0000","last_fullsized":"2023-10-30T12:56:21.810651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'773","last_scrub_stamp":"2023-10-30T12:56:21.810556+0000","last_deep_scrub":"55'773","last_deep_scrub_stamp":"2023-10-30T12:56:21.810556+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.810556+0000","objects_scrubbed":143,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:13:19.546302+0000","scrub_duration":0.23224787799999999,"objects_trimmed":0,"snaptrim_duration":0.0048907120000000002,"stat_sum":{"num_bytes":9710360,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":62,"num_read_kb":17,"num_write":609,"num_write_kb":15663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"55'239","reported_seq":313,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:27.077441+0000","last_change":"2023-10-30T12:56:27.077499+0000","last_active":"2023-10-30T12:56:27.077441+0000","last_peered":"2023-10-30T12:56:27.077441+0000","last_clean":"2023-10-30T12:56:27.077441+0000","last_became_active":"2023-10-30T12:35:05.200169+0000","last_became_peered":"2023-10-30T12:35:05.200169+0000","last_unstale":"2023-10-30T12:56:27.077441+0000","last_undegraded":"2023-10-30T12:56:27.077441+0000","last_fullsized":"2023-10-30T12:56:27.077441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'239","last_scrub_stamp":"2023-10-30T12:56:27.077343+0000","last_deep_scrub":"55'239","last_deep_scrub_stamp":"2023-10-30T12:56:27.077343+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.077343+0000","objects_scrubbed":6,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:21:31.745900+0000","scrub_duration":0.044083678000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":36867,"num_write":183,"num_write_kb":12445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16490,"num_omap_keys":31,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"55'675","reported_seq":2827,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.186194+0000","last_change":"2023-10-30T12:42:02.725641+0000","last_active":"2023-10-30T12:56:14.186194+0000","last_peered":"2023-10-30T12:56:14.186194+0000","last_clean":"2023-10-30T12:56:14.186194+0000","last_became_active":"2023-10-30T12:35:06.194689+0000","last_became_peered":"2023-10-30T12:35:06.194689+0000","last_unstale":"2023-10-30T12:56:14.186194+0000","last_undegraded":"2023-10-30T12:56:14.186194+0000","last_fullsized":"2023-10-30T12:56:14.186194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004291034,"stat_sum":{"num_bytes":10922380,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":61,"num_read_kb":18,"num_write":626,"num_write_kb":24711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"55'983","reported_seq":1183,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.860910+0000","last_change":"2023-10-30T12:56:24.861007+0000","last_active":"2023-10-30T12:56:24.860910+0000","last_peered":"2023-10-30T12:56:24.860910+0000","last_clean":"2023-10-30T12:56:24.860910+0000","last_became_active":"2023-10-30T12:35:05.182761+0000","last_became_peered":"2023-10-30T12:35:05.182761+0000","last_unstale":"2023-10-30T12:56:24.860910+0000","last_undegraded":"2023-10-30T12:56:24.860910+0000","last_fullsized":"2023-10-30T12:56:24.860910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'983","last_scrub_stamp":"2023-10-30T12:56:24.860748+0000","last_deep_scrub":"55'983","last_deep_scrub_stamp":"2023-10-30T12:56:24.860748+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.860748+0000","objects_scrubbed":8,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:51:38.567709+0000","scrub_duration":0.11984236299999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25625406,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":150,"num_read_kb":45229,"num_write":895,"num_write_kb":25503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"55'458","reported_seq":672,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:21.989466+0000","last_change":"2023-10-30T12:56:21.989512+0000","last_active":"2023-10-30T12:56:21.989466+0000","last_peered":"2023-10-30T12:56:21.989466+0000","last_clean":"2023-10-30T12:56:21.989466+0000","last_became_active":"2023-10-30T12:35:05.180730+0000","last_became_peered":"2023-10-30T12:35:05.180730+0000","last_unstale":"2023-10-30T12:56:21.989466+0000","last_undegraded":"2023-10-30T12:56:21.989466+0000","last_fullsized":"2023-10-30T12:56:21.989466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'458","last_scrub_stamp":"2023-10-30T12:56:21.989395+0000","last_deep_scrub":"55'458","last_deep_scrub_stamp":"2023-10-30T12:56:21.989395+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.989395+0000","objects_scrubbed":4,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:10:50.980457+0000","scrub_duration":0.030827730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8855749,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":162,"num_read_kb":8759,"num_write":390,"num_write_kb":8900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"55'906","reported_seq":3091,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:29.424480+0000","last_change":"2023-10-30T12:56:29.424522+0000","last_active":"2023-10-30T12:56:29.424480+0000","last_peered":"2023-10-30T12:56:29.424480+0000","last_clean":"2023-10-30T12:56:29.424480+0000","last_became_active":"2023-10-30T12:35:46.051057+0000","last_became_peered":"2023-10-30T12:35:46.051057+0000","last_unstale":"2023-10-30T12:56:29.424480+0000","last_undegraded":"2023-10-30T12:56:29.424480+0000","last_fullsized":"2023-10-30T12:56:29.424480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'906","last_scrub_stamp":"2023-10-30T12:56:29.424403+0000","last_deep_scrub":"55'906","last_deep_scrub_stamp":"2023-10-30T12:56:29.424403+0000","last_clean_scrub_stamp":"2023-10-30T12:56:29.424403+0000","objects_scrubbed":175,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:57:06.583434+0000","scrub_duration":0.16198085700000001,"objects_trimmed":0,"snaptrim_duration":0.0018351520000000001,"stat_sum":{"num_bytes":19060908,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":82,"num_read_kb":16408,"num_write":727,"num_write_kb":53009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"55'988","reported_seq":1063,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:30.499189+0000","last_change":"2023-10-30T12:56:30.499309+0000","last_active":"2023-10-30T12:56:30.499189+0000","last_peered":"2023-10-30T12:56:30.499189+0000","last_clean":"2023-10-30T12:56:30.499189+0000","last_became_active":"2023-10-30T12:35:05.201853+0000","last_became_peered":"2023-10-30T12:35:05.201853+0000","last_unstale":"2023-10-30T12:56:30.499189+0000","last_undegraded":"2023-10-30T12:56:30.499189+0000","last_fullsized":"2023-10-30T12:56:30.499189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'988","last_scrub_stamp":"2023-10-30T12:56:30.498957+0000","last_deep_scrub":"55'988","last_deep_scrub_stamp":"2023-10-30T12:56:30.498957+0000","last_clean_scrub_stamp":"2023-10-30T12:56:30.498957+0000","objects_scrubbed":5,"log_size":988,"log_dups_size":0,"ondisk_log_size":988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:34.744919+0000","scrub_duration":0.073408293999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":38325,"num_write":914,"num_write_kb":16753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"55'734","reported_seq":2898,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.162746+0000","last_change":"2023-10-30T12:42:02.710378+0000","last_active":"2023-10-30T12:56:14.162746+0000","last_peered":"2023-10-30T12:56:14.162746+0000","last_clean":"2023-10-30T12:56:14.162746+0000","last_became_active":"2023-10-30T12:35:06.170179+0000","last_became_peered":"2023-10-30T12:35:06.170179+0000","last_unstale":"2023-10-30T12:56:14.162746+0000","last_undegraded":"2023-10-30T12:56:14.162746+0000","last_fullsized":"2023-10-30T12:56:14.162746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021794980000000002,"stat_sum":{"num_bytes":14292082,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":73,"num_read_kb":12307,"num_write":687,"num_write_kb":28727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"55'150","reported_seq":218,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:27.831941+0000","last_change":"2023-10-30T12:56:27.831941+0000","last_active":"2023-10-30T12:56:27.831941+0000","last_peered":"2023-10-30T12:56:27.831941+0000","last_clean":"2023-10-30T12:56:27.831941+0000","last_became_active":"2023-10-30T12:35:05.197606+0000","last_became_peered":"2023-10-30T12:35:05.197606+0000","last_unstale":"2023-10-30T12:56:27.831941+0000","last_undegraded":"2023-10-30T12:56:27.831941+0000","last_fullsized":"2023-10-30T12:56:27.831941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'149","last_scrub_stamp":"2023-10-30T12:56:27.831485+0000","last_deep_scrub":"55'149","last_deep_scrub_stamp":"2023-10-30T12:56:27.831485+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.831485+0000","objects_scrubbed":5,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:57:34.144550+0000","scrub_duration":0.012229985000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":91,"num_write_kb":200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52416,"num_omap_keys":96,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"55'814","reported_seq":2996,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.786143+0000","last_change":"2023-10-30T12:56:34.786207+0000","last_active":"2023-10-30T12:56:34.786143+0000","last_peered":"2023-10-30T12:56:34.786143+0000","last_clean":"2023-10-30T12:56:34.786143+0000","last_became_active":"2023-10-30T12:35:06.225392+0000","last_became_peered":"2023-10-30T12:35:06.225392+0000","last_unstale":"2023-10-30T12:56:34.786143+0000","last_undegraded":"2023-10-30T12:56:34.786143+0000","last_fullsized":"2023-10-30T12:56:34.786143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'814","last_scrub_stamp":"2023-10-30T12:56:34.786044+0000","last_deep_scrub":"55'814","last_deep_scrub_stamp":"2023-10-30T12:56:34.786044+0000","last_clean_scrub_stamp":"2023-10-30T12:56:34.786044+0000","objects_scrubbed":159,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:26:14.139459+0000","scrub_duration":0.24530717799999999,"objects_trimmed":0,"snaptrim_duration":0.01552166,"stat_sum":{"num_bytes":9754816,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":77,"num_read_kb":18,"num_write":651,"num_write_kb":19117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"55'593","reported_seq":663,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.535430+0000","last_change":"2023-10-30T12:56:23.447764+0000","last_active":"2023-10-30T12:56:34.535430+0000","last_peered":"2023-10-30T12:56:34.535430+0000","last_clean":"2023-10-30T12:56:34.535430+0000","last_became_active":"2023-10-30T12:35:05.196899+0000","last_became_peered":"2023-10-30T12:35:05.196899+0000","last_unstale":"2023-10-30T12:56:34.535430+0000","last_undegraded":"2023-10-30T12:56:34.535430+0000","last_fullsized":"2023-10-30T12:56:34.535430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'579","last_scrub_stamp":"2023-10-30T12:56:23.447316+0000","last_deep_scrub":"55'579","last_deep_scrub_stamp":"2023-10-30T12:56:23.447316+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.447316+0000","objects_scrubbed":6,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:59:20.096502+0000","scrub_duration":0.067786201000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18152535,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":14173,"num_write":522,"num_write_kb":18015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"55'706","reported_seq":2860,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.175295+0000","last_change":"2023-10-30T12:42:02.724824+0000","last_active":"2023-10-30T12:56:14.175295+0000","last_peered":"2023-10-30T12:56:14.175295+0000","last_clean":"2023-10-30T12:56:14.175295+0000","last_became_active":"2023-10-30T12:35:06.222057+0000","last_became_peered":"2023-10-30T12:35:06.222057+0000","last_unstale":"2023-10-30T12:56:14.175295+0000","last_undegraded":"2023-10-30T12:56:14.175295+0000","last_fullsized":"2023-10-30T12:56:14.175295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004210915,"stat_sum":{"num_bytes":31400359,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":63,"num_read_kb":12308,"num_write":657,"num_write_kb":50495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"55'317","reported_seq":389,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:18.453350+0000","last_change":"2023-10-30T12:56:18.453440+0000","last_active":"2023-10-30T12:56:18.453350+0000","last_peered":"2023-10-30T12:56:18.453350+0000","last_clean":"2023-10-30T12:56:18.453350+0000","last_became_active":"2023-10-30T12:35:05.197190+0000","last_became_peered":"2023-10-30T12:35:05.197190+0000","last_unstale":"2023-10-30T12:56:18.453350+0000","last_undegraded":"2023-10-30T12:56:18.453350+0000","last_fullsized":"2023-10-30T12:56:18.453350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'317","last_scrub_stamp":"2023-10-30T12:56:18.453257+0000","last_deep_scrub":"55'317","last_deep_scrub_stamp":"2023-10-30T12:56:18.453257+0000","last_clean_scrub_stamp":"2023-10-30T12:56:18.453257+0000","objects_scrubbed":7,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:09:36.655470+0000","scrub_duration":0.034206814000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":4111,"num_write":258,"num_write_kb":8429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28930,"num_omap_keys":53,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"55'791","reported_seq":2957,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:25.811201+0000","last_change":"2023-10-30T12:56:25.811267+0000","last_active":"2023-10-30T12:56:25.811201+0000","last_peered":"2023-10-30T12:56:25.811201+0000","last_clean":"2023-10-30T12:56:25.811201+0000","last_became_active":"2023-10-30T12:35:06.204122+0000","last_became_peered":"2023-10-30T12:35:06.204122+0000","last_unstale":"2023-10-30T12:56:25.811201+0000","last_undegraded":"2023-10-30T12:56:25.811201+0000","last_fullsized":"2023-10-30T12:56:25.811201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'791","last_scrub_stamp":"2023-10-30T12:56:25.811099+0000","last_deep_scrub":"55'791","last_deep_scrub_stamp":"2023-10-30T12:56:25.811099+0000","last_clean_scrub_stamp":"2023-10-30T12:56:25.811099+0000","objects_scrubbed":158,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:21:46.064934+0000","scrub_duration":0.24490572399999999,"objects_trimmed":0,"snaptrim_duration":0.0030324050000000002,"stat_sum":{"num_bytes":15588017,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":60,"num_read_kb":8209,"num_write":632,"num_write_kb":41942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"55'368","reported_seq":449,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:25.809608+0000","last_change":"2023-10-30T12:56:25.809737+0000","last_active":"2023-10-30T12:56:25.809608+0000","last_peered":"2023-10-30T12:56:25.809608+0000","last_clean":"2023-10-30T12:56:25.809608+0000","last_became_active":"2023-10-30T12:35:05.179547+0000","last_became_peered":"2023-10-30T12:35:05.179547+0000","last_unstale":"2023-10-30T12:56:25.809608+0000","last_undegraded":"2023-10-30T12:56:25.809608+0000","last_fullsized":"2023-10-30T12:56:25.809608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'368","last_scrub_stamp":"2023-10-30T12:56:25.809432+0000","last_deep_scrub":"55'368","last_deep_scrub_stamp":"2023-10-30T12:56:25.809432+0000","last_clean_scrub_stamp":"2023-10-30T12:56:25.809432+0000","objects_scrubbed":9,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:55:48.660238+0000","scrub_duration":0.040587917000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":24600,"num_write":300,"num_write_kb":12511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17926,"num_omap_keys":33,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"55'784","reported_seq":2950,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.106113+0000","last_change":"2023-10-30T12:56:24.106255+0000","last_active":"2023-10-30T12:56:24.106113+0000","last_peered":"2023-10-30T12:56:24.106113+0000","last_clean":"2023-10-30T12:56:24.106113+0000","last_became_active":"2023-10-30T12:35:06.215128+0000","last_became_peered":"2023-10-30T12:35:06.215128+0000","last_unstale":"2023-10-30T12:56:24.106113+0000","last_undegraded":"2023-10-30T12:56:24.106113+0000","last_fullsized":"2023-10-30T12:56:24.106113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'784","last_scrub_stamp":"2023-10-30T12:56:24.105935+0000","last_deep_scrub":"55'784","last_deep_scrub_stamp":"2023-10-30T12:56:24.105935+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.105935+0000","objects_scrubbed":171,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:40:02.361124+0000","scrub_duration":0.30957912999999998,"objects_trimmed":0,"snaptrim_duration":0.0064662909999999999,"stat_sum":{"num_bytes":22774544,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":61,"num_read_kb":11,"num_write":605,"num_write_kb":27691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"55'215","reported_seq":298,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:18.560689+0000","last_change":"2023-10-30T12:56:18.560737+0000","last_active":"2023-10-30T12:56:18.560689+0000","last_peered":"2023-10-30T12:56:18.560689+0000","last_clean":"2023-10-30T12:56:18.560689+0000","last_became_active":"2023-10-30T12:35:05.189573+0000","last_became_peered":"2023-10-30T12:35:05.189573+0000","last_unstale":"2023-10-30T12:56:18.560689+0000","last_undegraded":"2023-10-30T12:56:18.560689+0000","last_fullsized":"2023-10-30T12:56:18.560689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'215","last_scrub_stamp":"2023-10-30T12:56:18.560612+0000","last_deep_scrub":"55'215","last_deep_scrub_stamp":"2023-10-30T12:56:18.560612+0000","last_clean_scrub_stamp":"2023-10-30T12:56:18.560612+0000","objects_scrubbed":7,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:40:38.364129+0000","scrub_duration":0.037472814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":24624,"num_write":154,"num_write_kb":8340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15144,"num_omap_keys":29,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"55'738","reported_seq":2907,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.180630+0000","last_change":"2023-10-30T12:42:02.724827+0000","last_active":"2023-10-30T12:56:14.180630+0000","last_peered":"2023-10-30T12:56:14.180630+0000","last_clean":"2023-10-30T12:56:14.180630+0000","last_became_active":"2023-10-30T12:35:06.216885+0000","last_became_peered":"2023-10-30T12:35:06.216885+0000","last_unstale":"2023-10-30T12:56:14.180630+0000","last_undegraded":"2023-10-30T12:56:14.180630+0000","last_fullsized":"2023-10-30T12:56:14.180630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042006370000000001,"stat_sum":{"num_bytes":19251186,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":77,"num_read_kb":16402,"num_write":677,"num_write_kb":45337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"55'237","reported_seq":307,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:31.515367+0000","last_change":"2023-10-30T12:56:31.515500+0000","last_active":"2023-10-30T12:56:31.515367+0000","last_peered":"2023-10-30T12:56:31.515367+0000","last_clean":"2023-10-30T12:56:31.515367+0000","last_became_active":"2023-10-30T12:35:05.202612+0000","last_became_peered":"2023-10-30T12:35:05.202612+0000","last_unstale":"2023-10-30T12:56:31.515367+0000","last_undegraded":"2023-10-30T12:56:31.515367+0000","last_fullsized":"2023-10-30T12:56:31.515367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'237","last_scrub_stamp":"2023-10-30T12:56:31.515170+0000","last_deep_scrub":"55'237","last_deep_scrub_stamp":"2023-10-30T12:56:31.515170+0000","last_clean_scrub_stamp":"2023-10-30T12:56:31.515170+0000","objects_scrubbed":7,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:51:38.652440+0000","scrub_duration":0.042566652000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":181,"num_write_kb":8338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45273,"num_omap_keys":85,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"55'710","reported_seq":2877,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.184438+0000","last_change":"2023-10-30T12:42:02.471602+0000","last_active":"2023-10-30T12:56:14.184438+0000","last_peered":"2023-10-30T12:56:14.184438+0000","last_clean":"2023-10-30T12:56:14.184438+0000","last_became_active":"2023-10-30T12:35:06.215553+0000","last_became_peered":"2023-10-30T12:35:06.215553+0000","last_unstale":"2023-10-30T12:56:14.184438+0000","last_undegraded":"2023-10-30T12:56:14.184438+0000","last_fullsized":"2023-10-30T12:56:14.184438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060418609999999999,"stat_sum":{"num_bytes":19055642,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":76,"num_read_kb":33,"num_write":654,"num_write_kb":32706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"55'297","reported_seq":377,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:21.765254+0000","last_change":"2023-10-30T12:56:21.765381+0000","last_active":"2023-10-30T12:56:21.765254+0000","last_peered":"2023-10-30T12:56:21.765254+0000","last_clean":"2023-10-30T12:56:21.765254+0000","last_became_active":"2023-10-30T12:35:05.181115+0000","last_became_peered":"2023-10-30T12:35:05.181115+0000","last_unstale":"2023-10-30T12:56:21.765254+0000","last_undegraded":"2023-10-30T12:56:21.765254+0000","last_fullsized":"2023-10-30T12:56:21.765254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'297","last_scrub_stamp":"2023-10-30T12:56:21.765065+0000","last_deep_scrub":"55'297","last_deep_scrub_stamp":"2023-10-30T12:56:21.765065+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.765065+0000","objects_scrubbed":8,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:20:05.549971+0000","scrub_duration":0.074545823999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":43239,"num_write":227,"num_write_kb":16591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":22378,"num_omap_keys":41,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"55'794","reported_seq":2960,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:31.034978+0000","last_change":"2023-10-30T12:56:31.035128+0000","last_active":"2023-10-30T12:56:31.034978+0000","last_peered":"2023-10-30T12:56:31.034978+0000","last_clean":"2023-10-30T12:56:31.034978+0000","last_became_active":"2023-10-30T12:35:06.197722+0000","last_became_peered":"2023-10-30T12:35:06.197722+0000","last_unstale":"2023-10-30T12:56:31.034978+0000","last_undegraded":"2023-10-30T12:56:31.034978+0000","last_fullsized":"2023-10-30T12:56:31.034978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'794","last_scrub_stamp":"2023-10-30T12:56:31.034831+0000","last_deep_scrub":"55'794","last_deep_scrub_stamp":"2023-10-30T12:56:31.034831+0000","last_clean_scrub_stamp":"2023-10-30T12:56:31.034831+0000","objects_scrubbed":168,"log_size":794,"log_dups_size":0,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:35:46.015437+0000","scrub_duration":0.24651738600000001,"objects_trimmed":0,"snaptrim_duration":0.011777173,"stat_sum":{"num_bytes":11179157,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":60,"num_read_kb":16,"num_write":620,"num_write_kb":21517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"55'492","reported_seq":592,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:19.245439+0000","last_change":"2023-10-30T12:56:19.245517+0000","last_active":"2023-10-30T12:56:19.245439+0000","last_peered":"2023-10-30T12:56:19.245439+0000","last_clean":"2023-10-30T12:56:19.245439+0000","last_became_active":"2023-10-30T12:35:05.169130+0000","last_became_peered":"2023-10-30T12:35:05.169130+0000","last_unstale":"2023-10-30T12:56:19.245439+0000","last_undegraded":"2023-10-30T12:56:19.245439+0000","last_fullsized":"2023-10-30T12:56:19.245439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'492","last_scrub_stamp":"2023-10-30T12:56:19.245361+0000","last_deep_scrub":"55'492","last_deep_scrub_stamp":"2023-10-30T12:56:19.245361+0000","last_clean_scrub_stamp":"2023-10-30T12:56:19.245361+0000","objects_scrubbed":12,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:27:01.796605+0000","scrub_duration":0.039470346000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583024,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":18341,"num_write":415,"num_write_kb":12551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10636,"num_omap_keys":20,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"55'720","reported_seq":2903,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.171633+0000","last_change":"2023-10-30T12:42:02.470189+0000","last_active":"2023-10-30T12:56:14.171633+0000","last_peered":"2023-10-30T12:56:14.171633+0000","last_clean":"2023-10-30T12:56:14.171633+0000","last_became_active":"2023-10-30T12:35:06.214839+0000","last_became_peered":"2023-10-30T12:35:06.214839+0000","last_unstale":"2023-10-30T12:56:14.171633+0000","last_undegraded":"2023-10-30T12:56:14.171633+0000","last_fullsized":"2023-10-30T12:56:14.171633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049147749999999997,"stat_sum":{"num_bytes":12184897,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":91,"num_read_kb":16402,"num_write":664,"num_write_kb":29797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"55'315","reported_seq":389,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:20.671077+0000","last_change":"2023-10-30T12:56:20.671275+0000","last_active":"2023-10-30T12:56:20.671077+0000","last_peered":"2023-10-30T12:56:20.671077+0000","last_clean":"2023-10-30T12:56:20.671077+0000","last_became_active":"2023-10-30T12:35:05.188862+0000","last_became_peered":"2023-10-30T12:35:05.188862+0000","last_unstale":"2023-10-30T12:56:20.671077+0000","last_undegraded":"2023-10-30T12:56:20.671077+0000","last_fullsized":"2023-10-30T12:56:20.671077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'315","last_scrub_stamp":"2023-10-30T12:56:20.670866+0000","last_deep_scrub":"55'315","last_deep_scrub_stamp":"2023-10-30T12:56:20.670866+0000","last_clean_scrub_stamp":"2023-10-30T12:56:20.670866+0000","objects_scrubbed":4,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:07:40.146504+0000","scrub_duration":0.058071512999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":12343,"num_write":388,"num_write_kb":4750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9690,"num_omap_keys":18,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"55'673","reported_seq":2830,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.172053+0000","last_change":"2023-10-30T12:42:02.472755+0000","last_active":"2023-10-30T12:56:14.172053+0000","last_peered":"2023-10-30T12:56:14.172053+0000","last_clean":"2023-10-30T12:56:14.172053+0000","last_became_active":"2023-10-30T12:35:06.216533+0000","last_became_peered":"2023-10-30T12:35:06.216533+0000","last_unstale":"2023-10-30T12:56:14.172053+0000","last_undegraded":"2023-10-30T12:56:14.172053+0000","last_fullsized":"2023-10-30T12:56:14.172053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060962430000000003,"stat_sum":{"num_bytes":10771712,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":65,"num_read_kb":4109,"num_write":629,"num_write_kb":24152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"55'238","reported_seq":308,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:23.246213+0000","last_change":"2023-10-30T12:56:23.246263+0000","last_active":"2023-10-30T12:56:23.246213+0000","last_peered":"2023-10-30T12:56:23.246213+0000","last_clean":"2023-10-30T12:56:23.246213+0000","last_became_active":"2023-10-30T12:35:05.177316+0000","last_became_peered":"2023-10-30T12:35:05.177316+0000","last_unstale":"2023-10-30T12:56:23.246213+0000","last_undegraded":"2023-10-30T12:56:23.246213+0000","last_fullsized":"2023-10-30T12:56:23.246213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'238","last_scrub_stamp":"2023-10-30T12:56:23.246132+0000","last_deep_scrub":"55'238","last_deep_scrub_stamp":"2023-10-30T12:56:23.246132+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.246132+0000","objects_scrubbed":4,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:22:00.496493+0000","scrub_duration":0.023404655999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":180,"num_write_kb":8325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12558,"num_omap_keys":23,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"55'702","reported_seq":2863,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.160430+0000","last_change":"2023-10-30T12:42:02.241911+0000","last_active":"2023-10-30T12:56:14.160430+0000","last_peered":"2023-10-30T12:56:14.160430+0000","last_clean":"2023-10-30T12:56:14.160430+0000","last_became_active":"2023-10-30T12:35:46.049464+0000","last_became_peered":"2023-10-30T12:35:46.049464+0000","last_unstale":"2023-10-30T12:56:14.160430+0000","last_undegraded":"2023-10-30T12:56:14.160430+0000","last_fullsized":"2023-10-30T12:56:14.160430+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023101760000000002,"stat_sum":{"num_bytes":7006288,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":663,"num_write_kb":12867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"55'215","reported_seq":285,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:32.554259+0000","last_change":"2023-10-30T12:56:32.554326+0000","last_active":"2023-10-30T12:56:32.554259+0000","last_peered":"2023-10-30T12:56:32.554259+0000","last_clean":"2023-10-30T12:56:32.554259+0000","last_became_active":"2023-10-30T12:35:05.171491+0000","last_became_peered":"2023-10-30T12:35:05.171491+0000","last_unstale":"2023-10-30T12:56:32.554259+0000","last_undegraded":"2023-10-30T12:56:32.554259+0000","last_fullsized":"2023-10-30T12:56:32.554259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'215","last_scrub_stamp":"2023-10-30T12:56:32.554161+0000","last_deep_scrub":"55'215","last_deep_scrub_stamp":"2023-10-30T12:56:32.554161+0000","last_clean_scrub_stamp":"2023-10-30T12:56:32.554161+0000","objects_scrubbed":4,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:25:32.692345+0000","scrub_duration":0.026739378000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":16387,"num_write":154,"num_write_kb":4208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11738,"num_omap_keys":23,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"55'771","reported_seq":2933,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.197946+0000","last_change":"2023-10-30T12:42:02.746323+0000","last_active":"2023-10-30T12:56:14.197946+0000","last_peered":"2023-10-30T12:56:14.197946+0000","last_clean":"2023-10-30T12:56:14.197946+0000","last_became_active":"2023-10-30T12:35:06.213493+0000","last_became_peered":"2023-10-30T12:35:06.213493+0000","last_unstale":"2023-10-30T12:56:14.197946+0000","last_undegraded":"2023-10-30T12:56:14.197946+0000","last_fullsized":"2023-10-30T12:56:14.197946+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015847370999999999,"stat_sum":{"num_bytes":18244405,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":70,"num_read_kb":4114,"num_write":718,"num_write_kb":33517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"55'310","reported_seq":382,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:20.198207+0000","last_change":"2023-10-30T12:56:20.198258+0000","last_active":"2023-10-30T12:56:20.198207+0000","last_peered":"2023-10-30T12:56:20.198207+0000","last_clean":"2023-10-30T12:56:20.198207+0000","last_became_active":"2023-10-30T12:35:05.197453+0000","last_became_peered":"2023-10-30T12:35:05.197453+0000","last_unstale":"2023-10-30T12:56:20.198207+0000","last_undegraded":"2023-10-30T12:56:20.198207+0000","last_fullsized":"2023-10-30T12:56:20.198207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'310","last_scrub_stamp":"2023-10-30T12:56:20.198131+0000","last_deep_scrub":"55'310","last_deep_scrub_stamp":"2023-10-30T12:56:20.198131+0000","last_clean_scrub_stamp":"2023-10-30T12:56:20.198131+0000","objects_scrubbed":5,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:00:10.556698+0000","scrub_duration":0.022980041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":240,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17972,"num_omap_keys":33,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"55'691","reported_seq":2844,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.183546+0000","last_change":"2023-10-30T12:42:02.472032+0000","last_active":"2023-10-30T12:56:14.183546+0000","last_peered":"2023-10-30T12:56:14.183546+0000","last_clean":"2023-10-30T12:56:14.183546+0000","last_became_active":"2023-10-30T12:35:06.219526+0000","last_became_peered":"2023-10-30T12:35:06.219526+0000","last_unstale":"2023-10-30T12:56:14.183546+0000","last_undegraded":"2023-10-30T12:56:14.183546+0000","last_fullsized":"2023-10-30T12:56:14.183546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060180570000000003,"stat_sum":{"num_bytes":14352444,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":61,"num_read_kb":19722,"num_write":644,"num_write_kb":44993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"55'119","reported_seq":219,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:27.867971+0000","last_change":"2023-10-30T12:56:27.868066+0000","last_active":"2023-10-30T12:56:27.867971+0000","last_peered":"2023-10-30T12:56:27.867971+0000","last_clean":"2023-10-30T12:56:27.867971+0000","last_became_active":"2023-10-30T12:34:55.205923+0000","last_became_peered":"2023-10-30T12:34:55.205923+0000","last_unstale":"2023-10-30T12:56:27.867971+0000","last_undegraded":"2023-10-30T12:56:27.867971+0000","last_fullsized":"2023-10-30T12:56:27.867971+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'119","last_scrub_stamp":"2023-10-30T12:56:27.867851+0000","last_deep_scrub":"55'119","last_deep_scrub_stamp":"2023-10-30T12:56:27.867851+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.867851+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:43:27.714386+0000","scrub_duration":0.0099090500000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"55'316","reported_seq":394,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:28.481897+0000","last_change":"2023-10-30T12:56:28.482017+0000","last_active":"2023-10-30T12:56:28.481897+0000","last_peered":"2023-10-30T12:56:28.481897+0000","last_clean":"2023-10-30T12:56:28.481897+0000","last_became_active":"2023-10-30T12:35:05.197201+0000","last_became_peered":"2023-10-30T12:35:05.197201+0000","last_unstale":"2023-10-30T12:56:28.481897+0000","last_undegraded":"2023-10-30T12:56:28.481897+0000","last_fullsized":"2023-10-30T12:56:28.481897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'316","last_scrub_stamp":"2023-10-30T12:56:28.481711+0000","last_deep_scrub":"55'316","last_deep_scrub_stamp":"2023-10-30T12:56:28.481711+0000","last_clean_scrub_stamp":"2023-10-30T12:56:28.481711+0000","objects_scrubbed":5,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:52.589549+0000","scrub_duration":0.039869544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":23,"num_read_kb":16415,"num_write":260,"num_write_kb":8374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"55'645","reported_seq":2789,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.166088+0000","last_change":"2023-10-30T12:42:02.243413+0000","last_active":"2023-10-30T12:56:14.166088+0000","last_peered":"2023-10-30T12:56:14.166088+0000","last_clean":"2023-10-30T12:56:14.166088+0000","last_became_active":"2023-10-30T12:35:06.217687+0000","last_became_peered":"2023-10-30T12:35:06.217687+0000","last_unstale":"2023-10-30T12:56:14.166088+0000","last_undegraded":"2023-10-30T12:56:14.166088+0000","last_fullsized":"2023-10-30T12:56:14.166088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059801699999999999,"stat_sum":{"num_bytes":18791963,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":51,"num_read_kb":8203,"num_write":599,"num_write_kb":32827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"55'413","reported_seq":486,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:29.691679+0000","last_change":"2023-10-30T12:35:05.208339+0000","last_active":"2023-10-30T12:56:29.691679+0000","last_peered":"2023-10-30T12:56:29.691679+0000","last_clean":"2023-10-30T12:56:29.691679+0000","last_became_active":"2023-10-30T12:35:05.207828+0000","last_became_peered":"2023-10-30T12:35:05.207828+0000","last_unstale":"2023-10-30T12:56:29.691679+0000","last_undegraded":"2023-10-30T12:56:29.691679+0000","last_fullsized":"2023-10-30T12:56:29.691679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22018716,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":29,"num_read_kb":45074,"num_write":344,"num_write_kb":21704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"55'837","reported_seq":3002,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:28.741030+0000","last_change":"2023-10-30T12:56:28.741094+0000","last_active":"2023-10-30T12:56:28.741030+0000","last_peered":"2023-10-30T12:56:28.741030+0000","last_clean":"2023-10-30T12:56:28.741030+0000","last_became_active":"2023-10-30T12:35:06.194791+0000","last_became_peered":"2023-10-30T12:35:06.194791+0000","last_unstale":"2023-10-30T12:56:28.741030+0000","last_undegraded":"2023-10-30T12:56:28.741030+0000","last_fullsized":"2023-10-30T12:56:28.741030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'837","last_scrub_stamp":"2023-10-30T12:56:28.740945+0000","last_deep_scrub":"55'837","last_deep_scrub_stamp":"2023-10-30T12:56:28.740945+0000","last_clean_scrub_stamp":"2023-10-30T12:56:28.740945+0000","objects_scrubbed":162,"log_size":837,"log_dups_size":0,"ondisk_log_size":837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:55:53.265534+0000","scrub_duration":0.21527656000000001,"objects_trimmed":0,"snaptrim_duration":0.0043414500000000002,"stat_sum":{"num_bytes":10614378,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":58,"num_read_kb":3682,"num_write":656,"num_write_kb":34629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"55'616","reported_seq":983,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:21.219063+0000","last_change":"2023-10-30T12:56:21.219063+0000","last_active":"2023-10-30T12:56:21.219063+0000","last_peered":"2023-10-30T12:56:21.219063+0000","last_clean":"2023-10-30T12:56:21.219063+0000","last_became_active":"2023-10-30T12:35:05.175959+0000","last_became_peered":"2023-10-30T12:35:05.175959+0000","last_unstale":"2023-10-30T12:56:21.219063+0000","last_undegraded":"2023-10-30T12:56:21.219063+0000","last_fullsized":"2023-10-30T12:56:21.219063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'615","last_scrub_stamp":"2023-10-30T12:56:21.218930+0000","last_deep_scrub":"55'615","last_deep_scrub_stamp":"2023-10-30T12:56:21.218930+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.218930+0000","objects_scrubbed":9,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:53:08.332507+0000","scrub_duration":0.031017952000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9398249,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":313,"num_read_kb":20888,"num_write":567,"num_write_kb":9627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34344,"num_omap_keys":63,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"55'706","reported_seq":2867,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.160969+0000","last_change":"2023-10-30T12:42:02.711348+0000","last_active":"2023-10-30T12:56:14.160969+0000","last_peered":"2023-10-30T12:56:14.160969+0000","last_clean":"2023-10-30T12:56:14.160969+0000","last_became_active":"2023-10-30T12:35:06.164711+0000","last_became_peered":"2023-10-30T12:35:06.164711+0000","last_unstale":"2023-10-30T12:56:14.160969+0000","last_undegraded":"2023-10-30T12:56:14.160969+0000","last_fullsized":"2023-10-30T12:56:14.160969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002714419,"stat_sum":{"num_bytes":18668779,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":68,"num_read_kb":8213,"num_write":645,"num_write_kb":33825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"55'501","reported_seq":574,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:22.428808+0000","last_change":"2023-10-30T12:56:22.428932+0000","last_active":"2023-10-30T12:56:22.428808+0000","last_peered":"2023-10-30T12:56:22.428808+0000","last_clean":"2023-10-30T12:56:22.428808+0000","last_became_active":"2023-10-30T12:35:05.196873+0000","last_became_peered":"2023-10-30T12:35:05.196873+0000","last_unstale":"2023-10-30T12:56:22.428808+0000","last_undegraded":"2023-10-30T12:56:22.428808+0000","last_fullsized":"2023-10-30T12:56:22.428808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'501","last_scrub_stamp":"2023-10-30T12:56:22.428591+0000","last_deep_scrub":"55'501","last_deep_scrub_stamp":"2023-10-30T12:56:22.428591+0000","last_clean_scrub_stamp":"2023-10-30T12:56:22.428591+0000","objects_scrubbed":5,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:31.946150+0000","scrub_duration":0.065339878000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":431,"num_write_kb":16687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8414,"num_omap_keys":16,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"55'813","reported_seq":2992,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:22.854721+0000","last_change":"2023-10-30T12:56:22.854850+0000","last_active":"2023-10-30T12:56:22.854721+0000","last_peered":"2023-10-30T12:56:22.854721+0000","last_clean":"2023-10-30T12:56:22.854721+0000","last_became_active":"2023-10-30T12:35:06.183225+0000","last_became_peered":"2023-10-30T12:35:06.183225+0000","last_unstale":"2023-10-30T12:56:22.854721+0000","last_undegraded":"2023-10-30T12:56:22.854721+0000","last_fullsized":"2023-10-30T12:56:22.854721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'813","last_scrub_stamp":"2023-10-30T12:56:22.854526+0000","last_deep_scrub":"55'813","last_deep_scrub_stamp":"2023-10-30T12:56:22.854526+0000","last_clean_scrub_stamp":"2023-10-30T12:56:22.854526+0000","objects_scrubbed":154,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:19:37.784115+0000","scrub_duration":0.30563644699999998,"objects_trimmed":0,"snaptrim_duration":0.0032989500000000001,"stat_sum":{"num_bytes":18054478,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":72,"num_read_kb":12,"num_write":656,"num_write_kb":36307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"55'185","reported_seq":250,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:36.447235+0000","last_change":"2023-10-30T12:56:36.447291+0000","last_active":"2023-10-30T12:56:36.447235+0000","last_peered":"2023-10-30T12:56:36.447235+0000","last_clean":"2023-10-30T12:56:36.447235+0000","last_became_active":"2023-10-30T12:35:05.177266+0000","last_became_peered":"2023-10-30T12:35:05.177266+0000","last_unstale":"2023-10-30T12:56:36.447235+0000","last_undegraded":"2023-10-30T12:56:36.447235+0000","last_fullsized":"2023-10-30T12:56:36.447235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'185","last_scrub_stamp":"2023-10-30T12:56:36.447125+0000","last_deep_scrub":"55'185","last_deep_scrub_stamp":"2023-10-30T12:56:36.447125+0000","last_clean_scrub_stamp":"2023-10-30T12:56:36.447125+0000","objects_scrubbed":2,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:52:35.695395+0000","scrub_duration":0.022317884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":129,"num_write_kb":8259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"55'688","reported_seq":2850,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.198507+0000","last_change":"2023-10-30T12:42:02.745017+0000","last_active":"2023-10-30T12:56:14.198507+0000","last_peered":"2023-10-30T12:56:14.198507+0000","last_clean":"2023-10-30T12:56:14.198507+0000","last_became_active":"2023-10-30T12:35:06.196307+0000","last_became_peered":"2023-10-30T12:35:06.196307+0000","last_unstale":"2023-10-30T12:56:14.198507+0000","last_undegraded":"2023-10-30T12:56:14.198507+0000","last_fullsized":"2023-10-30T12:56:14.198507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015197106,"stat_sum":{"num_bytes":5885946,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":69,"num_read_kb":13,"num_write":633,"num_write_kb":24590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"55'526","reported_seq":590,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:29.634122+0000","last_change":"2023-10-30T12:35:05.207594+0000","last_active":"2023-10-30T12:56:29.634122+0000","last_peered":"2023-10-30T12:56:29.634122+0000","last_clean":"2023-10-30T12:56:29.634122+0000","last_became_active":"2023-10-30T12:35:05.206930+0000","last_became_peered":"2023-10-30T12:35:05.206930+0000","last_unstale":"2023-10-30T12:56:29.634122+0000","last_undegraded":"2023-10-30T12:56:29.634122+0000","last_fullsized":"2023-10-30T12:56:29.634122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16492246,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":8,"num_read_kb":9517,"num_write":466,"num_write_kb":16393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"55'701","reported_seq":2866,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.145630+0000","last_change":"2023-10-30T12:42:02.710925+0000","last_active":"2023-10-30T12:56:14.145630+0000","last_peered":"2023-10-30T12:56:14.145630+0000","last_clean":"2023-10-30T12:56:14.145630+0000","last_became_active":"2023-10-30T12:35:06.164733+0000","last_became_peered":"2023-10-30T12:35:06.164733+0000","last_unstale":"2023-10-30T12:56:14.145630+0000","last_undegraded":"2023-10-30T12:56:14.145630+0000","last_fullsized":"2023-10-30T12:56:14.145630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024278020000000002,"stat_sum":{"num_bytes":19206810,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":652,"num_write_kb":32920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"55'381","reported_seq":457,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:26.587550+0000","last_change":"2023-10-30T12:56:26.587606+0000","last_active":"2023-10-30T12:56:26.587550+0000","last_peered":"2023-10-30T12:56:26.587550+0000","last_clean":"2023-10-30T12:56:26.587550+0000","last_became_active":"2023-10-30T12:35:05.184695+0000","last_became_peered":"2023-10-30T12:35:05.184695+0000","last_unstale":"2023-10-30T12:56:26.587550+0000","last_undegraded":"2023-10-30T12:56:26.587550+0000","last_fullsized":"2023-10-30T12:56:26.587550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'381","last_scrub_stamp":"2023-10-30T12:56:26.587467+0000","last_deep_scrub":"55'381","last_deep_scrub_stamp":"2023-10-30T12:56:26.587467+0000","last_clean_scrub_stamp":"2023-10-30T12:56:26.587467+0000","objects_scrubbed":7,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:27:14.820811+0000","scrub_duration":0.083109266000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":8489,"num_write":574,"num_write_kb":5859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54698,"num_omap_keys":106,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"55'682","reported_seq":2836,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.176192+0000","last_change":"2023-10-30T12:42:02.242088+0000","last_active":"2023-10-30T12:56:14.176192+0000","last_peered":"2023-10-30T12:56:14.176192+0000","last_clean":"2023-10-30T12:56:14.176192+0000","last_became_active":"2023-10-30T12:35:06.204422+0000","last_became_peered":"2023-10-30T12:35:06.204422+0000","last_unstale":"2023-10-30T12:56:14.176192+0000","last_undegraded":"2023-10-30T12:56:14.176192+0000","last_fullsized":"2023-10-30T12:56:14.176192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048535030000000003,"stat_sum":{"num_bytes":22995855,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":62,"num_read_kb":4110,"num_write":640,"num_write_kb":33018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"55'236","reported_seq":307,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:23.771307+0000","last_change":"2023-10-30T12:56:23.771446+0000","last_active":"2023-10-30T12:56:23.771307+0000","last_peered":"2023-10-30T12:56:23.771307+0000","last_clean":"2023-10-30T12:56:23.771307+0000","last_became_active":"2023-10-30T12:35:05.200449+0000","last_became_peered":"2023-10-30T12:35:05.200449+0000","last_unstale":"2023-10-30T12:56:23.771307+0000","last_undegraded":"2023-10-30T12:56:23.771307+0000","last_fullsized":"2023-10-30T12:56:23.771307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'236","last_scrub_stamp":"2023-10-30T12:56:23.771024+0000","last_deep_scrub":"55'236","last_deep_scrub_stamp":"2023-10-30T12:56:23.771024+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.771024+0000","objects_scrubbed":6,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:55:13.133447+0000","scrub_duration":0.033750645000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":4117,"num_write":187,"num_write_kb":4299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18018,"num_omap_keys":33,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"55'665","reported_seq":2826,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.158339+0000","last_change":"2023-10-30T12:42:02.724106+0000","last_active":"2023-10-30T12:56:14.158339+0000","last_peered":"2023-10-30T12:56:14.158339+0000","last_clean":"2023-10-30T12:56:14.158339+0000","last_became_active":"2023-10-30T12:35:06.179412+0000","last_became_peered":"2023-10-30T12:35:06.179412+0000","last_unstale":"2023-10-30T12:56:14.158339+0000","last_undegraded":"2023-10-30T12:56:14.158339+0000","last_fullsized":"2023-10-30T12:56:14.158339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003661568,"stat_sum":{"num_bytes":1664351,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":69,"num_read_kb":22,"num_write":614,"num_write_kb":12460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"55'560","reported_seq":634,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:32.834922+0000","last_change":"2023-10-30T12:56:32.835075+0000","last_active":"2023-10-30T12:56:32.834922+0000","last_peered":"2023-10-30T12:56:32.834922+0000","last_clean":"2023-10-30T12:56:32.834922+0000","last_became_active":"2023-10-30T12:35:05.206607+0000","last_became_peered":"2023-10-30T12:35:05.206607+0000","last_unstale":"2023-10-30T12:56:32.834922+0000","last_undegraded":"2023-10-30T12:56:32.834922+0000","last_fullsized":"2023-10-30T12:56:32.834922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'560","last_scrub_stamp":"2023-10-30T12:56:32.834680+0000","last_deep_scrub":"55'560","last_deep_scrub_stamp":"2023-10-30T12:56:32.834680+0000","last_clean_scrub_stamp":"2023-10-30T12:56:32.834680+0000","objects_scrubbed":8,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:19:51.459668+0000","scrub_duration":0.066587212000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":17,"num_read_kb":23179,"num_write":497,"num_write_kb":12662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34806,"num_omap_keys":64,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"55'809","reported_seq":2985,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.829260+0000","last_change":"2023-10-30T12:56:24.829353+0000","last_active":"2023-10-30T12:56:24.829260+0000","last_peered":"2023-10-30T12:56:24.829260+0000","last_clean":"2023-10-30T12:56:24.829260+0000","last_became_active":"2023-10-30T12:35:06.181256+0000","last_became_peered":"2023-10-30T12:35:06.181256+0000","last_unstale":"2023-10-30T12:56:24.829260+0000","last_undegraded":"2023-10-30T12:56:24.829260+0000","last_fullsized":"2023-10-30T12:56:24.829260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'809","last_scrub_stamp":"2023-10-30T12:56:24.829117+0000","last_deep_scrub":"55'809","last_deep_scrub_stamp":"2023-10-30T12:56:24.829117+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.829117+0000","objects_scrubbed":153,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:42:27.862243+0000","scrub_duration":0.27743204100000002,"objects_trimmed":0,"snaptrim_duration":0.0030387220000000002,"stat_sum":{"num_bytes":10464290,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":70,"num_read_kb":4103,"num_write":644,"num_write_kb":29233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"55'374","reported_seq":446,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.513421+0000","last_change":"2023-10-30T12:56:24.513468+0000","last_active":"2023-10-30T12:56:24.513421+0000","last_peered":"2023-10-30T12:56:24.513421+0000","last_clean":"2023-10-30T12:56:24.513421+0000","last_became_active":"2023-10-30T12:35:05.200724+0000","last_became_peered":"2023-10-30T12:35:05.200724+0000","last_unstale":"2023-10-30T12:56:24.513421+0000","last_undegraded":"2023-10-30T12:56:24.513421+0000","last_fullsized":"2023-10-30T12:56:24.513421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'374","last_scrub_stamp":"2023-10-30T12:56:24.513345+0000","last_deep_scrub":"55'374","last_deep_scrub_stamp":"2023-10-30T12:56:24.513345+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.513345+0000","objects_scrubbed":7,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:34:47.132641+0000","scrub_duration":0.034143153000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388698,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":12300,"num_write":322,"num_write_kb":8548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25586,"num_omap_keys":47,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"55'835","reported_seq":2999,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:35.315142+0000","last_change":"2023-10-30T12:56:35.315188+0000","last_active":"2023-10-30T12:56:35.315142+0000","last_peered":"2023-10-30T12:56:35.315142+0000","last_clean":"2023-10-30T12:56:35.315142+0000","last_became_active":"2023-10-30T12:35:06.173996+0000","last_became_peered":"2023-10-30T12:35:06.173996+0000","last_unstale":"2023-10-30T12:56:35.315142+0000","last_undegraded":"2023-10-30T12:56:35.315142+0000","last_fullsized":"2023-10-30T12:56:35.315142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'835","last_scrub_stamp":"2023-10-30T12:56:35.315063+0000","last_deep_scrub":"55'835","last_deep_scrub_stamp":"2023-10-30T12:56:35.315063+0000","last_clean_scrub_stamp":"2023-10-30T12:56:35.315063+0000","objects_scrubbed":159,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:30:07.209945+0000","scrub_duration":0.14615183500000001,"objects_trimmed":0,"snaptrim_duration":0.0045562010000000002,"stat_sum":{"num_bytes":5693555,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":58,"num_read_kb":18,"num_write":647,"num_write_kb":16270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"55'881","reported_seq":951,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:30.588970+0000","last_change":"2023-10-30T12:56:30.589026+0000","last_active":"2023-10-30T12:56:30.588970+0000","last_peered":"2023-10-30T12:56:30.588970+0000","last_clean":"2023-10-30T12:56:30.588970+0000","last_became_active":"2023-10-30T12:35:05.163794+0000","last_became_peered":"2023-10-30T12:35:05.163794+0000","last_unstale":"2023-10-30T12:56:30.588970+0000","last_undegraded":"2023-10-30T12:56:30.588970+0000","last_fullsized":"2023-10-30T12:56:30.588970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'881","last_scrub_stamp":"2023-10-30T12:56:30.588883+0000","last_deep_scrub":"55'881","last_deep_scrub_stamp":"2023-10-30T12:56:30.588883+0000","last_clean_scrub_stamp":"2023-10-30T12:56:30.588883+0000","objects_scrubbed":7,"log_size":881,"log_dups_size":0,"ondisk_log_size":881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:50:39.690725+0000","scrub_duration":0.062172023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":10,"num_read_kb":8454,"num_write":955,"num_write_kb":13780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23914,"num_omap_keys":46,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"55'668","reported_seq":2829,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.177236+0000","last_change":"2023-10-30T12:42:02.242567+0000","last_active":"2023-10-30T12:56:14.177236+0000","last_peered":"2023-10-30T12:56:14.177236+0000","last_clean":"2023-10-30T12:56:14.177236+0000","last_became_active":"2023-10-30T12:35:06.183226+0000","last_became_peered":"2023-10-30T12:35:06.183226+0000","last_unstale":"2023-10-30T12:56:14.177236+0000","last_undegraded":"2023-10-30T12:56:14.177236+0000","last_fullsized":"2023-10-30T12:56:14.177236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052521379999999999,"stat_sum":{"num_bytes":5799195,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":69,"num_read_kb":4112,"num_write":618,"num_write_kb":16472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"55'344","reported_seq":419,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:33.883120+0000","last_change":"2023-10-30T12:56:33.883350+0000","last_active":"2023-10-30T12:56:33.883120+0000","last_peered":"2023-10-30T12:56:33.883120+0000","last_clean":"2023-10-30T12:56:33.883120+0000","last_became_active":"2023-10-30T12:35:05.206518+0000","last_became_peered":"2023-10-30T12:35:05.206518+0000","last_unstale":"2023-10-30T12:56:33.883120+0000","last_undegraded":"2023-10-30T12:56:33.883120+0000","last_fullsized":"2023-10-30T12:56:33.883120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'344","last_scrub_stamp":"2023-10-30T12:56:33.882814+0000","last_deep_scrub":"55'344","last_deep_scrub_stamp":"2023-10-30T12:56:33.882814+0000","last_clean_scrub_stamp":"2023-10-30T12:56:33.882814+0000","objects_scrubbed":8,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:13:20.348463+0000","scrub_duration":0.065105804000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":268,"num_write_kb":12461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":36978,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"55'781","reported_seq":2955,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:29.111283+0000","last_change":"2023-10-30T12:56:29.111348+0000","last_active":"2023-10-30T12:56:29.111283+0000","last_peered":"2023-10-30T12:56:29.111283+0000","last_clean":"2023-10-30T12:56:29.111283+0000","last_became_active":"2023-10-30T12:35:06.197861+0000","last_became_peered":"2023-10-30T12:35:06.197861+0000","last_unstale":"2023-10-30T12:56:29.111283+0000","last_undegraded":"2023-10-30T12:56:29.111283+0000","last_fullsized":"2023-10-30T12:56:29.111283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'781","last_scrub_stamp":"2023-10-30T12:56:29.111192+0000","last_deep_scrub":"55'781","last_deep_scrub_stamp":"2023-10-30T12:56:29.111192+0000","last_clean_scrub_stamp":"2023-10-30T12:56:29.111192+0000","objects_scrubbed":157,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:53:54.048846+0000","scrub_duration":0.27439538299999999,"objects_trimmed":0,"snaptrim_duration":0.011118161,"stat_sum":{"num_bytes":14763185,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":68,"num_read_kb":15,"num_write":609,"num_write_kb":24605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"55'685","reported_seq":2857,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.197283+0000","last_change":"2023-10-30T12:42:02.744356+0000","last_active":"2023-10-30T12:56:14.197283+0000","last_peered":"2023-10-30T12:56:14.197283+0000","last_clean":"2023-10-30T12:56:14.197283+0000","last_became_active":"2023-10-30T12:35:06.212548+0000","last_became_peered":"2023-10-30T12:35:06.212548+0000","last_unstale":"2023-10-30T12:56:14.197283+0000","last_undegraded":"2023-10-30T12:56:14.197283+0000","last_fullsized":"2023-10-30T12:56:14.197283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015114931999999999,"stat_sum":{"num_bytes":19366418,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":81,"num_read_kb":7349,"num_write":643,"num_write_kb":32740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"55'368","reported_seq":437,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.907303+0000","last_change":"2023-10-30T12:56:34.907429+0000","last_active":"2023-10-30T12:56:34.907303+0000","last_peered":"2023-10-30T12:56:34.907303+0000","last_clean":"2023-10-30T12:56:34.907303+0000","last_became_active":"2023-10-30T12:35:05.206559+0000","last_became_peered":"2023-10-30T12:35:05.206559+0000","last_unstale":"2023-10-30T12:56:34.907303+0000","last_undegraded":"2023-10-30T12:56:34.907303+0000","last_fullsized":"2023-10-30T12:56:34.907303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'368","last_scrub_stamp":"2023-10-30T12:56:34.907076+0000","last_deep_scrub":"55'368","last_deep_scrub_stamp":"2023-10-30T12:56:34.907076+0000","last_clean_scrub_stamp":"2023-10-30T12:56:34.907076+0000","objects_scrubbed":5,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:12:13.284723+0000","scrub_duration":0.045168409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":312,"num_write_kb":8389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19072,"num_omap_keys":35,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"55'687","reported_seq":2848,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.357325+0000","last_change":"2023-10-30T12:42:02.744472+0000","last_active":"2023-10-30T12:56:14.357325+0000","last_peered":"2023-10-30T12:56:14.357325+0000","last_clean":"2023-10-30T12:56:14.357325+0000","last_became_active":"2023-10-30T12:35:06.212505+0000","last_became_peered":"2023-10-30T12:35:06.212505+0000","last_unstale":"2023-10-30T12:56:14.357325+0000","last_undegraded":"2023-10-30T12:56:14.357325+0000","last_fullsized":"2023-10-30T12:56:14.357325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014553393,"stat_sum":{"num_bytes":9800415,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":70,"num_read_kb":12304,"num_write":642,"num_write_kb":36520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"55'377","reported_seq":449,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:01.925479+0000","last_change":"2023-10-30T12:35:05.207851+0000","last_active":"2023-10-30T12:56:01.925479+0000","last_peered":"2023-10-30T12:56:01.925479+0000","last_clean":"2023-10-30T12:56:01.925479+0000","last_became_active":"2023-10-30T12:35:05.206879+0000","last_became_peered":"2023-10-30T12:35:05.206879+0000","last_unstale":"2023-10-30T12:56:01.925479+0000","last_undegraded":"2023-10-30T12:56:01.925479+0000","last_fullsized":"2023-10-30T12:56:01.925479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":32,"num_read_kb":45093,"num_write":322,"num_write_kb":20730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"55'643","reported_seq":2806,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.164580+0000","last_change":"2023-10-30T12:42:02.723663+0000","last_active":"2023-10-30T12:56:14.164580+0000","last_peered":"2023-10-30T12:56:14.164580+0000","last_clean":"2023-10-30T12:56:14.164580+0000","last_became_active":"2023-10-30T12:35:46.275077+0000","last_became_peered":"2023-10-30T12:35:46.275077+0000","last_unstale":"2023-10-30T12:56:14.164580+0000","last_undegraded":"2023-10-30T12:56:14.164580+0000","last_fullsized":"2023-10-30T12:56:14.164580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032416139999999999,"stat_sum":{"num_bytes":6205387,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":63,"num_read_kb":4113,"num_write":608,"num_write_kb":9154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"55'512","reported_seq":582,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:23.020730+0000","last_change":"2023-10-30T12:56:23.020777+0000","last_active":"2023-10-30T12:56:23.020730+0000","last_peered":"2023-10-30T12:56:23.020730+0000","last_clean":"2023-10-30T12:56:23.020730+0000","last_became_active":"2023-10-30T12:35:05.173984+0000","last_became_peered":"2023-10-30T12:35:05.173984+0000","last_unstale":"2023-10-30T12:56:23.020730+0000","last_undegraded":"2023-10-30T12:56:23.020730+0000","last_fullsized":"2023-10-30T12:56:23.020730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'512","last_scrub_stamp":"2023-10-30T12:56:23.020659+0000","last_deep_scrub":"55'512","last_deep_scrub_stamp":"2023-10-30T12:56:23.020659+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.020659+0000","objects_scrubbed":5,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:43:31.108759+0000","scrub_duration":0.032786733999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":430,"num_write_kb":12596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19196,"num_omap_keys":36,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"55'700","reported_seq":2854,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.148426+0000","last_change":"2023-10-30T12:42:02.243078+0000","last_active":"2023-10-30T12:56:14.148426+0000","last_peered":"2023-10-30T12:56:14.148426+0000","last_clean":"2023-10-30T12:56:14.148426+0000","last_became_active":"2023-10-30T12:35:06.198504+0000","last_became_peered":"2023-10-30T12:35:06.198504+0000","last_unstale":"2023-10-30T12:56:14.148426+0000","last_undegraded":"2023-10-30T12:56:14.148426+0000","last_fullsized":"2023-10-30T12:56:14.148426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055292249999999996,"stat_sum":{"num_bytes":14413504,"num_objects":182,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":61,"num_read_kb":1995,"num_write":647,"num_write_kb":19115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"55'216","reported_seq":307,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:20.534250+0000","last_change":"2023-10-30T12:56:20.534300+0000","last_active":"2023-10-30T12:56:20.534250+0000","last_peered":"2023-10-30T12:56:20.534250+0000","last_clean":"2023-10-30T12:56:20.534250+0000","last_became_active":"2023-10-30T12:35:05.200623+0000","last_became_peered":"2023-10-30T12:35:05.200623+0000","last_unstale":"2023-10-30T12:56:20.534250+0000","last_undegraded":"2023-10-30T12:56:20.534250+0000","last_fullsized":"2023-10-30T12:56:20.534250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'216","last_scrub_stamp":"2023-10-30T12:56:20.534185+0000","last_deep_scrub":"55'216","last_deep_scrub_stamp":"2023-10-30T12:56:20.534185+0000","last_clean_scrub_stamp":"2023-10-30T12:56:20.534185+0000","objects_scrubbed":8,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:19:00.729850+0000","scrub_duration":0.044608693999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":63,"num_read_kb":24633,"num_write":147,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7154,"num_omap_keys":14,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":925262768,"num_objects":10302,"num_object_clones":0,"num_object_copies":20604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10302,"num_whiteouts":0,"num_read":4339,"num_read_kb":400205,"num_write":41481,"num_write_kb":1950238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1902673920,"data_stored":1848985440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46827,"ondisk_log_size":46827,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":360592743,"num_objects":211,"num_object_clones":0,"num_object_copies":422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":211,"num_whiteouts":0,"num_read":1232,"num_read_kb":640401,"num_write":11703,"num_write_kb":362987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":548331,"num_omap_keys":1024,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":721346560,"data_stored":721185486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13221,"ondisk_log_size":13221,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542410,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496900,"kb_used_data":403260,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":93874940,"statfs":{"total":96636764160,"available":96127938560,"internally_reserved":0,"allocated":412938240,"data_stored":404862660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":95877581},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.82299999999999995,"15min":0.81399999999999995},"min":{"1min":0.47699999999999998,"5min":0.438,"15min":0.42899999999999999},"max":{"1min":1.1659999999999999,"5min":1.371,"15min":1.371},"last":1.145},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.82899999999999996,"15min":0.82099999999999995},"min":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.41699999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.45},"last":0.81000000000000005}]},{"osd":1,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83599999999999997,"15min":0.85199999999999998},"min":{"1min":0.55200000000000005,"5min":0.52700000000000002,"15min":0.44600000000000001},"max":{"1min":1.095,"5min":1.222,"15min":1.496},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.88,"15min":0.89300000000000002},"min":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.51800000000000002},"max":{"1min":1.282,"5min":1.494,"15min":1.494},"last":0.94999999999999996}]},{"osd":2,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.84399999999999997,"15min":0.82699999999999996},"min":{"1min":0.64800000000000002,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.1479999999999999,"5min":1.242,"15min":1.9059999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.008,"5min":0.872,"15min":0.89700000000000002},"min":{"1min":0.71499999999999997,"5min":0.58299999999999996,"15min":0.48999999999999999},"max":{"1min":1.5609999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.3160000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":0.96899999999999997,"15min":0.92500000000000004},"min":{"1min":0.77000000000000002,"5min":0.56999999999999995,"15min":0.54000000000000004},"max":{"1min":1.306,"5min":1.607,"15min":1.9810000000000001},"last":1.2310000000000001},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.98099999999999998,"15min":0.996},"min":{"1min":0.59299999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2749999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90100000000000002,"15min":0.92800000000000005},"min":{"1min":0.71599999999999997,"5min":0.52500000000000002,"15min":0.46899999999999997},"max":{"1min":1.329,"5min":1.4059999999999999,"15min":2.7309999999999999},"last":1.024},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.016,"15min":0.99399999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.41999999999999998},"max":{"1min":1.3600000000000001,"5min":1.694,"15min":1.8859999999999999},"last":1.2749999999999999}]},{"osd":5,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0449999999999999,"15min":1.0780000000000001},"min":{"1min":0.64500000000000002,"5min":0.59399999999999997,"15min":0.49099999999999999},"max":{"1min":1.375,"5min":1.6379999999999999,"15min":2.2120000000000002},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.228,"15min":1.1539999999999999},"min":{"1min":1.0900000000000001,"5min":0.89900000000000002,"15min":0.69899999999999995},"max":{"1min":1.4330000000000001,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.504}]},{"osd":6,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.03,"15min":1.0900000000000001},"min":{"1min":0.72999999999999998,"5min":0.65100000000000002,"15min":0.63600000000000001},"max":{"1min":1.4350000000000001,"5min":1.478,"15min":1.802},"last":1.375},{"interface":"front","average":{"1min":1.127,"5min":1.1419999999999999,"15min":1.1919999999999999},"min":{"1min":0.91200000000000003,"5min":0.879,"15min":0.83799999999999997},"max":{"1min":1.48,"5min":1.6639999999999999,"15min":1.881},"last":1.462}]}]},{"osd":6,"up_from":14,"seq":60129542409,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389260,"kb_used_data":299844,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":93982580,"statfs":{"total":96636764160,"available":96238161920,"internally_reserved":0,"allocated":307040256,"data_stored":301706633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":91552201},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.38600000000000001},"max":{"1min":1.2569999999999999,"5min":5.8760000000000003,"15min":5.8760000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.85699999999999998,"15min":0.84499999999999997},"min":{"1min":0.55200000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.194,"5min":1.3919999999999999,"15min":1.673},"last":0.71699999999999997}]},{"osd":1,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.85099999999999998,"15min":0.91200000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2290000000000001,"5min":1.6379999999999999,"15min":5.0949999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.94999999999999996,"15min":0.92500000000000004},"min":{"1min":0.75600000000000001,"5min":0.56899999999999995,"15min":0.45500000000000002},"max":{"1min":1.321,"5min":1.423,"15min":5.0519999999999996},"last":0.83899999999999997}]},{"osd":2,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.90600000000000003,"15min":0.85199999999999998},"min":{"1min":0.66600000000000004,"5min":0.54400000000000004,"15min":0.51800000000000002},"max":{"1min":1.306,"5min":2.649,"15min":2.649},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.01,"15min":0.93600000000000005},"min":{"1min":0.82499999999999996,"5min":0.59699999999999998,"15min":0.44700000000000001},"max":{"1min":1.4299999999999999,"5min":2.282,"15min":2.282},"last":1.2629999999999999}]},{"osd":3,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.87,"15min":0.93899999999999995},"min":{"1min":0.58899999999999997,"5min":0.51700000000000002,"15min":0.40200000000000002},"max":{"1min":1.1739999999999999,"5min":1.427,"15min":1.929},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.878,"5min":0.96399999999999997,"15min":0.99099999999999999},"min":{"1min":0.64300000000000002,"5min":0.58499999999999996,"15min":0.42699999999999999},"max":{"1min":1.2470000000000001,"5min":1.444,"15min":1.7589999999999999},"last":0.84599999999999997}]},{"osd":4,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0269999999999999,"15min":0.999},"min":{"1min":0.63600000000000001,"5min":0.63100000000000001,"15min":0.27600000000000002},"max":{"1min":1.5309999999999999,"5min":1.617,"15min":1.9219999999999999},"last":1.5169999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0489999999999999,"15min":1.0269999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.32400000000000001},"max":{"1min":1.2809999999999999,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.087}]},{"osd":5,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.97199999999999998,"15min":1.022},"min":{"1min":0.67300000000000004,"5min":0.66300000000000003,"15min":0.29599999999999999},"max":{"1min":1.232,"5min":1.619,"15min":2.9529999999999998},"last":1.04},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0780000000000001,"15min":1.1220000000000001},"min":{"1min":0.85999999999999999,"5min":0.65300000000000002,"15min":0.34399999999999997},"max":{"1min":1.278,"5min":1.764,"15min":2.5680000000000001},"last":1.1160000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.167,"15min":1.145},"min":{"1min":0.80100000000000005,"5min":0.79300000000000004,"15min":0.37},"max":{"1min":1.482,"5min":1.7689999999999999,"15min":2.0350000000000001},"last":1.234},{"interface":"front","average":{"1min":1.256,"5min":1.216,"15min":1.194},"min":{"1min":0.95299999999999996,"5min":0.82099999999999995,"15min":0.46800000000000003},"max":{"1min":1.5940000000000001,"5min":1.7829999999999999,"15min":2.274},"last":1.427}]}]},{"osd":4,"up_from":14,"seq":60129542408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400556,"kb_used_data":319396,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":93971284,"statfs":{"total":96636764160,"available":96226594816,"internally_reserved":0,"allocated":327061504,"data_stored":318875235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":83098054},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78100000000000003,"15min":0.80700000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.437},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.5649999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.49299999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.165,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.81200000000000006}]},{"osd":1,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.874,"15min":0.91500000000000004},"min":{"1min":0.55800000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.28,"5min":1.3069999999999999,"15min":2.4910000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85699999999999998,"15min":0.84899999999999998},"min":{"1min":0.58899999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.2130000000000001,"5min":1.2390000000000001,"15min":1.9730000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.93999999999999995,"15min":0.95399999999999996},"min":{"1min":0.59599999999999997,"5min":0.55900000000000005,"15min":0.53400000000000003},"max":{"1min":1.2669999999999999,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.88},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.0509999999999999,"15min":1.052},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.55600000000000005},"max":{"1min":1.1859999999999999,"5min":4.6879999999999997,"15min":4.6879999999999997},"last":1.032}]},{"osd":3,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.91600000000000004,"15min":0.93300000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.27,"5min":2.266,"15min":2.5819999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.96299999999999997,"15min":0.91400000000000003},"min":{"1min":0.68500000000000005,"5min":0.54300000000000004,"15min":0.53500000000000003},"max":{"1min":1.282,"5min":1.411,"15min":1.6890000000000001},"last":1.103}]},{"osd":5,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.93899999999999995,"15min":1.002},"min":{"1min":0.67700000000000005,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.3520000000000001,"5min":2.1779999999999999,"15min":2.3180000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1080000000000001,"15min":1.0760000000000001},"min":{"1min":1.012,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.478,"5min":3.173,"15min":3.173},"last":1.2849999999999999}]},{"osd":6,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.012,"15min":1.087},"min":{"1min":0.73099999999999998,"5min":0.66600000000000004,"15min":0.59299999999999997},"max":{"1min":1.2649999999999999,"5min":1.641,"15min":2.73},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.073,"15min":1.0800000000000001},"min":{"1min":0.93899999999999995,"5min":0.748,"15min":0.56499999999999995},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":4.7130000000000001},"last":1.206}]},{"osd":7,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.86799999999999999,"5min":0.68899999999999995,"15min":0.64900000000000002},"max":{"1min":1.3440000000000001,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":0.999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.161,"15min":1.129},"min":{"1min":0.95399999999999996,"5min":0.70299999999999996,"15min":0.65300000000000002},"max":{"1min":1.4370000000000001,"5min":1.766,"15min":3.0649999999999999},"last":1.3580000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542409,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298704,"kb_used_data":241160,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94073136,"statfs":{"total":96636764160,"available":96330891264,"internally_reserved":0,"allocated":246947840,"data_stored":241832282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.48599999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.22700000000000001},"max":{"1min":1.6930000000000001,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.5},{"interface":"front","average":{"1min":0.622,"5min":0.58199999999999996,"15min":0.53200000000000003},"min":{"1min":0.40000000000000002,"5min":0.33200000000000002,"15min":0.28699999999999998},"max":{"1min":2.0619999999999998,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":0.42199999999999999}]},{"osd":1,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.33000000000000002,"5min":0.317,"15min":0.27900000000000003},"max":{"1min":3.02,"5min":3.5059999999999998,"15min":4.5019999999999998},"last":0.442},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.39100000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.0299999999999998,"5min":3.5430000000000001,"15min":3.5430000000000001},"last":0.53400000000000003}]},{"osd":3,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.76000000000000001,"15min":0.60399999999999998},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26100000000000001},"max":{"1min":8.9550000000000001,"5min":18.747,"15min":18.747},"last":0.38400000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":0.85499999999999998,"15min":0.70099999999999996},"min":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38300000000000001},"max":{"1min":8.9900000000000002,"5min":18.863,"15min":18.863},"last":0.48399999999999999}]},{"osd":4,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85999999999999999},"min":{"1min":0.626,"5min":0.56999999999999995,"15min":0.53600000000000003},"max":{"1min":3.0760000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.321,"5min":0.96299999999999997,"15min":0.89700000000000002},"min":{"1min":0.73299999999999998,"5min":0.51100000000000001,"15min":0.48299999999999998},"max":{"1min":9.6289999999999996,"5min":9.6289999999999996,"15min":9.6289999999999996},"last":0.89700000000000002}]},{"osd":5,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":0.876,"15min":0.85199999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":9.7289999999999992,"5min":9.7289999999999992,"15min":9.7289999999999992},"last":0.621},{"interface":"front","average":{"1min":5.577,"5min":1.9490000000000001,"15min":1.2},"min":{"1min":0.626,"5min":0.60399999999999998,"15min":0.51900000000000002},"max":{"1min":91.072999999999993,"5min":91.072999999999993,"15min":91.072999999999993},"last":0.77600000000000002}]},{"osd":6,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.048,"15min":0.91700000000000004},"min":{"1min":0.59299999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":5.4640000000000004,"5min":19.126000000000001,"15min":19.126000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.2509999999999999,"15min":1.006},"min":{"1min":0.72499999999999998,"5min":0.60299999999999998,"15min":0.59399999999999997},"max":{"1min":21.469999999999999,"5min":21.469999999999999,"15min":21.469999999999999},"last":0.752}]},{"osd":7,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":0.94699999999999995,"15min":0.89600000000000002},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.47099999999999997},"max":{"1min":9.6869999999999994,"5min":9.6869999999999994,"15min":9.6869999999999994},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.1479999999999999,"15min":0.96799999999999997},"min":{"1min":0.80800000000000005,"5min":0.69799999999999995,"15min":0.53200000000000003},"max":{"1min":5.524,"5min":19.050999999999998,"15min":19.050999999999998},"last":0.84299999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434844,"kb_used_data":341204,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":93936996,"statfs":{"total":96636764160,"available":96191483904,"internally_reserved":0,"allocated":349392896,"data_stored":341139620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":95877576},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.46600000000000003,"15min":0.48399999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.248},"max":{"1min":0.69099999999999995,"5min":0.78700000000000003,"15min":8.8059999999999992},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.504,"15min":0.61099999999999999},"min":{"1min":0.30099999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69299999999999995,"5min":0.83899999999999997,"15min":35.084000000000003},"last":0.33000000000000002}]},{"osd":2,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51500000000000001,"15min":0.63500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.28999999999999998},"max":{"1min":0.68000000000000005,"5min":0.93100000000000005,"15min":34.741},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58099999999999996,"15min":0.69299999999999995},"min":{"1min":0.42999999999999999,"5min":0.37,"15min":0.35299999999999998},"max":{"1min":0.72999999999999998,"5min":1.0049999999999999,"15min":35.058999999999997},"last":0.55300000000000005}]},{"osd":3,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.35899999999999999,"5min":0.32000000000000001,"15min":0.29499999999999998},"max":{"1min":0.76000000000000001,"5min":0.877,"15min":8.8469999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.629},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.88800000000000001,"15min":11.916},"last":0.52100000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.85499999999999998,"15min":0.79700000000000004},"min":{"1min":0.66600000000000004,"5min":0.55500000000000005,"15min":0.373},"max":{"1min":1.0389999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.82499999999999996,"15min":0.92800000000000005},"min":{"1min":0.63900000000000001,"5min":0.57099999999999995,"15min":0.39900000000000002},"max":{"1min":1.1419999999999999,"5min":1.175,"15min":34.902999999999999},"last":0.93500000000000005}]},{"osd":5,"last update":"Mon Oct 30 12:55:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82099999999999995,"15min":0.84799999999999998},"min":{"1min":0.59499999999999997,"5min":0.51800000000000002,"15min":0.5},"max":{"1min":1.0820000000000001,"5min":1.171,"15min":8.4900000000000002},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.90600000000000003,"15min":0.97499999999999998},"min":{"1min":0.78500000000000003,"5min":0.66900000000000004,"15min":0.48799999999999999},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":34.698},"last":1.0369999999999999}]},{"osd":6,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81000000000000005},"min":{"1min":0.64200000000000002,"5min":0.58099999999999996,"15min":0.47799999999999998},"max":{"1min":1.097,"5min":1.1439999999999999,"15min":1.2789999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.878,"15min":0.88700000000000001},"min":{"1min":0.64100000000000001,"5min":0.629,"15min":0.50900000000000001},"max":{"1min":1.304,"5min":1.304,"15min":8.8800000000000008},"last":0.88800000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82499999999999996,"15min":0.84099999999999997},"min":{"1min":0.65000000000000002,"5min":0.53000000000000003,"15min":0.437},"max":{"1min":1.042,"5min":1.1930000000000001,"15min":1.304},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.88900000000000001,"15min":0.92900000000000005},"min":{"1min":0.70699999999999996,"5min":0.60599999999999998,"15min":0.55900000000000005},"max":{"1min":1.1479999999999999,"5min":1.323,"15min":8.5549999999999997},"last":0.87}]}]},{"osd":5,"up_from":14,"seq":60129542408,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473224,"kb_used_data":379584,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":93898616,"statfs":{"total":96636764160,"available":96152182784,"internally_reserved":0,"allocated":388694016,"data_stored":380569353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":95877577},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.88300000000000001,"15min":0.84399999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.41899999999999998},"max":{"1min":1.3129999999999999,"5min":1.627,"15min":1.627},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.88900000000000001,"15min":0.84999999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.495},"max":{"1min":1.2649999999999999,"5min":1.468,"15min":1.468},"last":0.77700000000000002}]},{"osd":1,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.82899999999999996,"15min":0.91300000000000003},"min":{"1min":0.52800000000000002,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.3089999999999999,"5min":1.363,"15min":1.746},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.82399999999999995,"15min":0.88},"min":{"1min":0.52700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2969999999999999,"5min":1.3169999999999999,"15min":1.337},"last":0.92000000000000004}]},{"osd":2,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89700000000000002,"15min":0.872},"min":{"1min":0.60299999999999998,"5min":0.53800000000000003,"15min":0.501},"max":{"1min":1.4139999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.86199999999999999,"15min":0.83899999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.442},"max":{"1min":1.1699999999999999,"5min":1.29,"15min":1.4770000000000001},"last":1.0349999999999999}]},{"osd":3,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.9850000000000001,"15min":1.3180000000000001},"min":{"1min":0.63900000000000001,"5min":0.54400000000000004,"15min":0.504},"max":{"1min":1.3640000000000001,"5min":104.88500000000001,"15min":104.88500000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.034,"5min":2.0619999999999998,"15min":1.3700000000000001},"min":{"1min":0.71399999999999997,"5min":0.624,"15min":0.60699999999999998},"max":{"1min":1.405,"5min":105.042,"15min":105.042},"last":0.98899999999999999}]},{"osd":4,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0349999999999999,"15min":0.94099999999999995},"min":{"1min":0.69899999999999995,"5min":0.58699999999999997,"15min":0.38500000000000001},"max":{"1min":1.262,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.091},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.077,"15min":1.1220000000000001},"min":{"1min":0.68100000000000005,"5min":0.59799999999999998,"15min":0.57399999999999995},"max":{"1min":1.5229999999999999,"5min":1.542,"15min":1.712},"last":1.22}]},{"osd":6,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.96199999999999997,"15min":0.95599999999999996},"min":{"1min":0.77800000000000002,"5min":0.627,"15min":0.35899999999999999},"max":{"1min":1.389,"5min":1.3899999999999999,"15min":1.9410000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.083,"5min":1.0649999999999999,"15min":1.056},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.48899999999999999},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":2.0299999999999998},"last":0.89300000000000002}]},{"osd":7,"last update":"Mon Oct 30 12:55:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.1080000000000001,"15min":1.095},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.40600000000000003},"max":{"1min":1.357,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.125},{"interface":"front","average":{"1min":1.111,"5min":1.113,"15min":1.1120000000000001},"min":{"1min":0.88,"5min":0.70499999999999996,"15min":0.53300000000000003},"max":{"1min":1.5149999999999999,"5min":1.5660000000000001,"15min":1.7689999999999999},"last":1.0649999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450120,"kb_used_data":352320,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":93921720,"statfs":{"total":96636764160,"available":96175841280,"internally_reserved":0,"allocated":360775680,"data_stored":352416873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":100137418},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.434,"15min":0.435},"min":{"1min":0.27900000000000003,"5min":0.25600000000000001,"15min":0.224},"max":{"1min":0.64600000000000002,"5min":0.77100000000000002,"15min":2.1459999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.46999999999999997,"15min":0.54700000000000004},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.27200000000000002},"max":{"1min":0.66300000000000003,"5min":0.77900000000000003,"15min":21.495999999999999},"last":0.59199999999999997}]},{"osd":1,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.497,"15min":0.53200000000000003},"min":{"1min":0.32700000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.66100000000000003,"5min":1.0700000000000001,"15min":1.1459999999999999},"last":0.621},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.49199999999999999,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.32600000000000001,"15min":0.30399999999999999},"max":{"1min":0.72899999999999998,"5min":0.83299999999999996,"15min":15.102},"last":0.67300000000000004}]},{"osd":2,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53200000000000003,"15min":0.60499999999999998},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.28299999999999997},"max":{"1min":0.68500000000000005,"5min":0.86399999999999999,"15min":21.321000000000002},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.41599999999999998,"5min":0.39300000000000002,"15min":0.379},"max":{"1min":0.74299999999999999,"5min":0.97099999999999997,"15min":1.681},"last":0.73799999999999999}]},{"osd":4,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.81399999999999995,"15min":0.89800000000000002},"min":{"1min":0.69399999999999995,"5min":0.49199999999999999,"15min":0.48499999999999999},"max":{"1min":1.0309999999999999,"5min":1.21,"15min":21.382999999999999},"last":1.006},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.872,"15min":0.92000000000000004},"min":{"1min":0.71299999999999997,"5min":0.60399999999999998,"15min":0.46700000000000003},"max":{"1min":1.0580000000000001,"5min":1.1910000000000001,"15min":15.231999999999999},"last":1.0469999999999999}]},{"osd":5,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85599999999999998,"15min":0.90200000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.47799999999999998},"max":{"1min":1.282,"5min":1.282,"15min":15.273999999999999},"last":1.028},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91100000000000003,"15min":0.95899999999999996},"min":{"1min":0.71399999999999997,"5min":0.67700000000000005,"15min":0.57499999999999996},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":15.359999999999999},"last":1.079}]},{"osd":6,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.59299999999999997,"5min":0.53200000000000003,"15min":0.51100000000000001},"max":{"1min":1.091,"5min":1.296,"15min":1.5620000000000001},"last":0.877},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82599999999999996,"15min":0.88600000000000001},"min":{"1min":0.66400000000000003,"5min":0.59799999999999998,"15min":0.498},"max":{"1min":1.1279999999999999,"5min":1.198,"15min":15.313000000000001},"last":0.91400000000000003}]},{"osd":7,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.85799999999999998,"15min":0.93000000000000005},"min":{"1min":0.67500000000000004,"5min":0.58799999999999997,"15min":0.55200000000000005},"max":{"1min":1.177,"5min":1.177,"15min":21.428999999999998},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.88,"15min":0.88200000000000001},"min":{"1min":0.71899999999999997,"5min":0.56899999999999995,"15min":0.56799999999999995},"max":{"1min":1.1439999999999999,"5min":1.28,"15min":1.3100000000000001},"last":0.98199999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542408,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299048,"kb_used_data":231840,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94072792,"statfs":{"total":96636764160,"available":96330539008,"internally_reserved":0,"allocated":237404160,"data_stored":232419390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.68500000000000005,"15min":0.54400000000000004},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.62,"5min":15.343999999999999,"15min":15.343999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.73299999999999998,"15min":0.59999999999999998},"min":{"1min":0.34100000000000003,"5min":0.313,"15min":0.29899999999999999},"max":{"1min":0.66800000000000004,"5min":14.68,"15min":14.68},"last":0.45900000000000002}]},{"osd":2,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.66700000000000004,"15min":0.59799999999999998},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68999999999999995,"5min":8.1359999999999992,"15min":8.1359999999999992},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.33900000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69899999999999995,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.64200000000000002}]},{"osd":3,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.65100000000000002,"15min":0.59599999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.81200000000000006,"5min":8.0920000000000005,"15min":8.0920000000000005},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59399999999999997,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.35199999999999998,"15min":0.32000000000000001},"max":{"1min":0.79500000000000004,"5min":5.9199999999999999,"15min":5.9199999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":1.133,"15min":0.94099999999999995},"min":{"1min":0.629,"5min":0.60499999999999998,"15min":0.41799999999999998},"max":{"1min":1.2010000000000001,"5min":21.27,"15min":21.27},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.93899999999999995,"15min":0.90700000000000003},"min":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.53600000000000003},"max":{"1min":1.3420000000000001,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.69899999999999995}]},{"osd":5,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.83099999999999996,"15min":0.82599999999999996},"min":{"1min":0.63900000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0089999999999999,"5min":1.355,"15min":1.355},"last":0.751},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.872,"15min":0.85599999999999998},"min":{"1min":0.68300000000000005,"5min":0.499,"15min":0.46200000000000002},"max":{"1min":1.1599999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85599999999999998,"15min":0.83999999999999997},"min":{"1min":0.70999999999999996,"5min":0.61399999999999999,"15min":0.51300000000000001},"max":{"1min":1.111,"5min":1.29,"15min":1.4450000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.88400000000000001,"15min":0.88500000000000001},"min":{"1min":0.64600000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.246,"5min":1.323,"15min":1.468},"last":0.82499999999999996}]},{"osd":7,"last update":"Mon Oct 30 12:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.871,"15min":0.85299999999999998},"min":{"1min":0.67800000000000005,"5min":0.629,"15min":0.496},"max":{"1min":1.304,"5min":1.304,"15min":1.4299999999999999},"last":1.05},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.86399999999999999,"15min":0.874},"min":{"1min":0.76200000000000001,"5min":0.63100000000000001,"15min":0.443},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2410000000000001},"last":0.94899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":75231232,"data_stored":75213140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":97542144,"data_stored":97517132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42975232,"data_stored":42953409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":85385216,"data_stored":85363568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":88555520,"data_stored":88540092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":89477120,"data_stored":89457081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":111374336,"data_stored":111354708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":130805760,"data_stored":130786356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272936960,"data_stored":265027314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139231232,"data_stored":134593460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":198570075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":274759680,"data_stored":266744507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":237875200,"data_stored":230026345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":298586112,"data_stored":290803474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":195035136,"data_stored":190043127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":280907776,"data_stored":273177138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:56:42.586 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.587 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.587 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.588 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.588 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.588 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.589 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.589 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.589 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.590 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.590 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.591 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.591 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.591 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.592 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.592 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.593 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.593 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.593 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.594 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.594 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.594 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.595 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.595 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.596 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.596 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.596 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.597 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.597 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.598 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.598 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.598 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.599 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.599 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.599 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.600 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.600 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.601 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.601 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.601 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.602 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.602 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.603 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.603 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.603 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.604 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.604 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.605 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.605 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:56:42.605 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-30T12:57:02.606 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:57:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4c00b0f640 1 Processor -- start 2023-10-30T12:57:02.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4c00b0f640 1 -- start start 2023-10-30T12:57:02.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4c00b0f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4bfc0c6bd0 0x7f4bfc06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:02.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4c00b0f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4bfc071110 0x7f4bfc158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:02.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4c00b0f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bfc066cf0 con 0x7f4bfc06f7a0 2023-10-30T12:57:02.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4c00b0f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bfc0700c0 con 0x7f4bfc071110 2023-10-30T12:57:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4c00b0f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc0c70d0 con 0x7f4bfc0c6bd0 2023-10-30T12:57:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4bfad9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4bfc071110 0x7f4bfc158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4bfa59c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4bfc06f7a0 0x7f4bfc069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36180/0 (socket says 172.21.15.12:36180) 2023-10-30T12:57:02.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4bfa59c640 1 -- 172.21.15.12:0/1072422578 learned_addr learned my addr 172.21.15.12:0/1072422578 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:57:02.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4bf3fff640 1 --2- 172.21.15.12:0/1072422578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4bfc0c6bd0 0x7f4bfc06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:02.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4bfad9d640 1 -- 172.21.15.12:0/1072422578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4bfc0c6bd0 msgr2=0x7f4bfc06fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4bfad9d640 1 --2- 172.21.15.12:0/1072422578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4bfc0c6bd0 0x7f4bfc06fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4bfad9d640 1 -- 172.21.15.12:0/1072422578 >> v1:172.21.15.12:6789/0 conn(0x7f4bfc06f7a0 legacy=0x7f4bfc069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4bfad9d640 1 -- 172.21.15.12:0/1072422578 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc0c7390 con 0x7f4bfc071110 2023-10-30T12:57:02.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.759+0000 7f4bf9d9b640 1 -- 172.21.15.12:0/1072422578 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124572782 0 0) 0x7f4bfc066cf0 con 0x7f4bfc06f7a0 2023-10-30T12:57:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.763+0000 7f4bfad9d640 1 --2- 172.21.15.12:0/1072422578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bfc071110 0x7f4bfc158110 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009d90 tx=0x7f4bf401f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0b56dcb4bb43c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.763+0000 7f4bf9d9b640 1 -- 172.21.15.12:0/1072422578 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf40af600 con 0x7f4bfc071110 2023-10-30T12:57:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.763+0000 7f4bf9d9b640 1 -- 172.21.15.12:0/1072422578 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf40ab050 con 0x7f4bfc071110 2023-10-30T12:57:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.763+0000 7f4bf9d9b640 1 -- 172.21.15.12:0/1072422578 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf40af980 con 0x7f4bfc071110 2023-10-30T12:57:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.763+0000 7f4c00b0f640 1 -- 172.21.15.12:0/1072422578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bfc071110 msgr2=0x7f4bfc158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.763+0000 7f4c00b0f640 1 --2- 172.21.15.12:0/1072422578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bfc071110 0x7f4bfc158110 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009d90 tx=0x7f4bf401f130 comp rx=0 tx=0).stop 2023-10-30T12:57:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.763+0000 7f4c00b0f640 1 -- 172.21.15.12:0/1072422578 shutdown_connections 2023-10-30T12:57:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.763+0000 7f4c00b0f640 1 --2- 172.21.15.12:0/1072422578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bfc071110 0x7f4bfc158110 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:02.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.763+0000 7f4c00b0f640 1 --2- 172.21.15.12:0/1072422578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4bfc0c6bd0 0x7f4bfc06fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:02.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.763+0000 7f4c00b0f640 1 -- 172.21.15.12:0/1072422578 >> 172.21.15.12:0/1072422578 conn(0x7f4bfc0c7910 msgr2=0x7f4bfc0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:02.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.763+0000 7f4c00b0f640 1 -- 172.21.15.12:0/1072422578 shutdown_connections 2023-10-30T12:57:02.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.763+0000 7f4c00b0f640 1 -- 172.21.15.12:0/1072422578 wait complete. 2023-10-30T12:57:02.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4c00b0f640 1 Processor -- start 2023-10-30T12:57:02.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4c00b0f640 1 -- start start 2023-10-30T12:57:02.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4c00b0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bfc06f7a0 0x7f4bfc15ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:02.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4c00b0f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4bfc071110 0x7f4bfc0c10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:02.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4c00b0f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4bfc0c6bd0 0x7f4bfc1f0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:02.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4c00b0f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc157d50 con 0x7f4bfc06f7a0 2023-10-30T12:57:02.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4c00b0f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc159cc0 con 0x7f4bfc071110 2023-10-30T12:57:02.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4c00b0f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc158940 con 0x7f4bfc0c6bd0 2023-10-30T12:57:02.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4bfa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bfc06f7a0 0x7f4bfc15ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:02.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4bfa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bfc06f7a0 0x7f4bfc15ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48180/0 (socket says 172.21.15.12:48180) 2023-10-30T12:57:02.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4bfa59c640 1 -- 172.21.15.12:0/97251429 learned_addr learned my addr 172.21.15.12:0/97251429 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:57:02.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4bfad9d640 1 --2- 172.21.15.12:0/97251429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4bfc0c6bd0 0x7f4bfc1f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:02.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4bf3fff640 1 --2- 172.21.15.12:0/97251429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4bfc071110 0x7f4bfc0c10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:02.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4bfa59c640 1 -- 172.21.15.12:0/97251429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4bfc0c6bd0 msgr2=0x7f4bfc1f0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:02.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4bfa59c640 1 --2- 172.21.15.12:0/97251429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4bfc0c6bd0 0x7f4bfc1f0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:02.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4bfa59c640 1 -- 172.21.15.12:0/97251429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4bfc071110 msgr2=0x7f4bfc0c10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:02.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4bfa59c640 1 --2- 172.21.15.12:0/97251429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4bfc071110 0x7f4bfc0c10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:02.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4bfa59c640 1 -- 172.21.15.12:0/97251429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf4009a70 con 0x7f4bfc06f7a0 2023-10-30T12:57:02.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4bfa59c640 1 --2- 172.21.15.12:0/97251429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bfc06f7a0 0x7f4bfc15ad70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00c4b0 tx=0x7f4bec00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:02.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4bf2ffd640 1 -- 172.21.15.12:0/97251429 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec0074e0 con 0x7f4bfc06f7a0 2023-10-30T12:57:02.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4bf2ffd640 1 -- 172.21.15.12:0/97251429 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec007660 con 0x7f4bfc06f7a0 2023-10-30T12:57:02.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4bf2ffd640 1 -- 172.21.15.12:0/97251429 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec004a30 con 0x7f4bfc06f7a0 2023-10-30T12:57:02.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4c00b0f640 1 -- 172.21.15.12:0/97251429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bfc158c20 con 0x7f4bfc06f7a0 2023-10-30T12:57:02.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.767+0000 7f4c00b0f640 1 -- 172.21.15.12:0/97251429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bfc1591e0 con 0x7f4bfc06f7a0 2023-10-30T12:57:02.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.771+0000 7f4c00b0f640 1 -- 172.21.15.12:0/97251429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bfc04e170 con 0x7f4bfc06f7a0 2023-10-30T12:57:02.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.771+0000 7f4bf2ffd640 1 -- 172.21.15.12:0/97251429 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4bec004b90 con 0x7f4bfc06f7a0 2023-10-30T12:57:02.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.771+0000 7f4bf2ffd640 1 --2- 172.21.15.12:0/97251429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4bdc0664d0 0x7f4bdc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:02.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.771+0000 7f4bf2ffd640 1 -- 172.21.15.12:0/97251429 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f4bec083490 con 0x7f4bfc06f7a0 2023-10-30T12:57:02.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.771+0000 7f4bf3fff640 1 --2- 172.21.15.12:0/97251429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4bdc0664d0 0x7f4bdc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:02.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.771+0000 7f4bf3fff640 1 --2- 172.21.15.12:0/97251429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4bdc0664d0 0x7f4bdc068990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0c6a70 tx=0x7f4be4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:02.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.775+0000 7f4bf2ffd640 1 -- 172.21.15.12:0/97251429 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4bec050410 con 0x7f4bfc06f7a0 2023-10-30T12:57:02.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.931+0000 7f4c00b0f640 1 -- 172.21.15.12:0/97251429 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4bfc070350 con 0x7f4bdc0664d0 2023-10-30T12:57:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.951+0000 7f4bf2ffd640 1 -- 172.21.15.12:0/97251429 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+293793 (secure 0 0 0) 0x7f4be400b670 con 0x7f4bdc0664d0 2023-10-30T12:57:02.959 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:57:02.961 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-10-30T12:57:02.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.955+0000 7f4c00b0f640 1 -- 172.21.15.12:0/97251429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4bdc0664d0 msgr2=0x7f4bdc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:02.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.955+0000 7f4c00b0f640 1 --2- 172.21.15.12:0/97251429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4bdc0664d0 0x7f4bdc068990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0c6a70 tx=0x7f4be4008040 comp rx=0 tx=0).stop 2023-10-30T12:57:02.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.955+0000 7f4c00b0f640 1 -- 172.21.15.12:0/97251429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bfc06f7a0 msgr2=0x7f4bfc15ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:02.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.955+0000 7f4c00b0f640 1 --2- 172.21.15.12:0/97251429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bfc06f7a0 0x7f4bfc15ad70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00c4b0 tx=0x7f4bec00c970 comp rx=0 tx=0).stop 2023-10-30T12:57:02.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.955+0000 7f4c00b0f640 1 -- 172.21.15.12:0/97251429 shutdown_connections 2023-10-30T12:57:02.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.955+0000 7f4c00b0f640 1 --2- 172.21.15.12:0/97251429 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f4bdc0664d0 0x7f4bdc068990 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:02.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.955+0000 7f4c00b0f640 1 --2- 172.21.15.12:0/97251429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4bfc0c6bd0 0x7f4bfc1f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:02.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.955+0000 7f4c00b0f640 1 --2- 172.21.15.12:0/97251429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4bfc071110 0x7f4bfc0c10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:02.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.955+0000 7f4c00b0f640 1 --2- 172.21.15.12:0/97251429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bfc06f7a0 0x7f4bfc15ad70 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:02.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.955+0000 7f4c00b0f640 1 -- 172.21.15.12:0/97251429 >> 172.21.15.12:0/97251429 conn(0x7f4bfc0c7910 msgr2=0x7f4bfc0c92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:02.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.955+0000 7f4c00b0f640 1 -- 172.21.15.12:0/97251429 shutdown_connections 2023-10-30T12:57:02.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:02.955+0000 7f4c00b0f640 1 -- 172.21.15.12:0/97251429 wait complete. 2023-10-30T12:57:02.979 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":713,"stamp":"2023-10-30T12:57:01.427367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1286646922,"num_objects":10515,"num_object_clones":0,"num_object_copies":21030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10515,"num_whiteouts":0,"num_read":5687,"num_read_kb":1040706,"num_write":53361,"num_write_kb":2315849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":579427,"num_omap_keys":1083,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64867,"ondisk_log_size":64867,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3222256,"kb_used_data":2569008,"kb_used_omap":12,"kb_used_meta":653171,"kb_avail":751752464,"statfs":{"total":773094113280,"available":769794523136,"internally_reserved":0,"allocated":2630664192,"data_stored":2574224132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12729,"internal_metadata":668847687},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":56708,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004671"},"pg_stats":[{"pgid":"3.3f","version":"55'737","reported_seq":2903,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.185374+0000","last_change":"2023-10-30T12:42:02.472363+0000","last_active":"2023-10-30T12:56:14.185374+0000","last_peered":"2023-10-30T12:56:14.185374+0000","last_clean":"2023-10-30T12:56:14.185374+0000","last_became_active":"2023-10-30T12:35:06.222724+0000","last_became_peered":"2023-10-30T12:35:06.222724+0000","last_unstale":"2023-10-30T12:56:14.185374+0000","last_undegraded":"2023-10-30T12:56:14.185374+0000","last_fullsized":"2023-10-30T12:56:14.185374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059524979999999996,"stat_sum":{"num_bytes":14421277,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":73,"num_read_kb":8207,"num_write":692,"num_write_kb":25708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"55'855","reported_seq":3032,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:38.668823+0000","last_change":"2023-10-30T12:56:38.668869+0000","last_active":"2023-10-30T12:56:38.668823+0000","last_peered":"2023-10-30T12:56:38.668823+0000","last_clean":"2023-10-30T12:56:38.668823+0000","last_became_active":"2023-10-30T12:35:06.182441+0000","last_became_peered":"2023-10-30T12:35:06.182441+0000","last_unstale":"2023-10-30T12:56:38.668823+0000","last_undegraded":"2023-10-30T12:56:38.668823+0000","last_fullsized":"2023-10-30T12:56:38.668823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'855","last_scrub_stamp":"2023-10-30T12:56:38.668744+0000","last_deep_scrub":"55'855","last_deep_scrub_stamp":"2023-10-30T12:56:38.668744+0000","last_clean_scrub_stamp":"2023-10-30T12:56:38.668744+0000","objects_scrubbed":154,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:43:43.147201+0000","scrub_duration":0.17361679199999999,"objects_trimmed":0,"snaptrim_duration":0.0026942979999999999,"stat_sum":{"num_bytes":14617239,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":70,"num_read_kb":10,"num_write":675,"num_write_kb":36983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"55'856","reported_seq":3043,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:55.585283+0000","last_change":"2023-10-30T12:56:55.585332+0000","last_active":"2023-10-30T12:56:55.585283+0000","last_peered":"2023-10-30T12:56:55.585283+0000","last_clean":"2023-10-30T12:56:55.585283+0000","last_became_active":"2023-10-30T12:35:06.180132+0000","last_became_peered":"2023-10-30T12:35:06.180132+0000","last_unstale":"2023-10-30T12:56:55.585283+0000","last_undegraded":"2023-10-30T12:56:55.585283+0000","last_fullsized":"2023-10-30T12:56:55.585283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'856","last_scrub_stamp":"2023-10-30T12:56:55.585201+0000","last_deep_scrub":"55'856","last_deep_scrub_stamp":"2023-10-30T12:56:55.585201+0000","last_clean_scrub_stamp":"2023-10-30T12:56:55.585201+0000","objects_scrubbed":159,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:09:47.246202+0000","scrub_duration":0.184201582,"objects_trimmed":0,"snaptrim_duration":0.003537278,"stat_sum":{"num_bytes":22250243,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":80,"num_read_kb":12305,"num_write":653,"num_write_kb":42611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"55'682","reported_seq":2832,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.197902+0000","last_change":"2023-10-30T12:42:02.742917+0000","last_active":"2023-10-30T12:56:14.197902+0000","last_peered":"2023-10-30T12:56:14.197902+0000","last_clean":"2023-10-30T12:56:14.197902+0000","last_became_active":"2023-10-30T12:35:06.213987+0000","last_became_peered":"2023-10-30T12:35:06.213987+0000","last_unstale":"2023-10-30T12:56:14.197902+0000","last_undegraded":"2023-10-30T12:56:14.197902+0000","last_fullsized":"2023-10-30T12:56:14.197902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013528057,"stat_sum":{"num_bytes":18553875,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":57,"num_read_kb":8205,"num_write":627,"num_write_kb":28534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"55'670","reported_seq":2812,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.184369+0000","last_change":"2023-10-30T12:42:02.470142+0000","last_active":"2023-10-30T12:56:14.184369+0000","last_peered":"2023-10-30T12:56:14.184369+0000","last_clean":"2023-10-30T12:56:14.184369+0000","last_became_active":"2023-10-30T12:35:06.214964+0000","last_became_peered":"2023-10-30T12:35:06.214964+0000","last_unstale":"2023-10-30T12:56:14.184369+0000","last_undegraded":"2023-10-30T12:56:14.184369+0000","last_fullsized":"2023-10-30T12:56:14.184369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046834600000000004,"stat_sum":{"num_bytes":8686005,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":51,"num_read_kb":4112,"num_write":616,"num_write_kb":25845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"55'857","reported_seq":3023,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:52.695822+0000","last_change":"2023-10-30T12:56:52.695935+0000","last_active":"2023-10-30T12:56:52.695822+0000","last_peered":"2023-10-30T12:56:52.695822+0000","last_clean":"2023-10-30T12:56:52.695822+0000","last_became_active":"2023-10-30T12:35:06.214219+0000","last_became_peered":"2023-10-30T12:35:06.214219+0000","last_unstale":"2023-10-30T12:56:52.695822+0000","last_undegraded":"2023-10-30T12:56:52.695822+0000","last_fullsized":"2023-10-30T12:56:52.695822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'857","last_scrub_stamp":"2023-10-30T12:56:52.695649+0000","last_deep_scrub":"55'857","last_deep_scrub_stamp":"2023-10-30T12:56:52.695649+0000","last_clean_scrub_stamp":"2023-10-30T12:56:52.695649+0000","objects_scrubbed":170,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:36:31.854780+0000","scrub_duration":0.32795837700000002,"objects_trimmed":0,"snaptrim_duration":0.015862434000000002,"stat_sum":{"num_bytes":14811547,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":61,"num_read_kb":16,"num_write":643,"num_write_kb":33253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"55'697","reported_seq":2839,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.171845+0000","last_change":"2023-10-30T12:42:02.467959+0000","last_active":"2023-10-30T12:56:14.171845+0000","last_peered":"2023-10-30T12:56:14.171845+0000","last_clean":"2023-10-30T12:56:14.171845+0000","last_became_active":"2023-10-30T12:35:06.216468+0000","last_became_peered":"2023-10-30T12:35:06.216468+0000","last_unstale":"2023-10-30T12:56:14.171845+0000","last_undegraded":"2023-10-30T12:56:14.171845+0000","last_fullsized":"2023-10-30T12:56:14.171845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031309480000000002,"stat_sum":{"num_bytes":9595467,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":52,"num_read_kb":12216,"num_write":649,"num_write_kb":36101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"55'625","reported_seq":2772,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.176378+0000","last_change":"2023-10-30T12:42:02.241155+0000","last_active":"2023-10-30T12:56:14.176378+0000","last_peered":"2023-10-30T12:56:14.176378+0000","last_clean":"2023-10-30T12:56:14.176378+0000","last_became_active":"2023-10-30T12:35:06.214384+0000","last_became_peered":"2023-10-30T12:35:06.214384+0000","last_unstale":"2023-10-30T12:56:14.176378+0000","last_undegraded":"2023-10-30T12:56:14.176378+0000","last_fullsized":"2023-10-30T12:56:14.176378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045552559999999997,"stat_sum":{"num_bytes":9825651,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":56,"num_read_kb":15,"num_write":593,"num_write_kb":23392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"55'695","reported_seq":2859,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:21.757142+0000","last_change":"2023-10-30T12:42:02.473886+0000","last_active":"2023-10-30T12:56:21.757142+0000","last_peered":"2023-10-30T12:56:21.757142+0000","last_clean":"2023-10-30T12:56:21.757142+0000","last_became_active":"2023-10-30T12:35:06.222692+0000","last_became_peered":"2023-10-30T12:35:06.222692+0000","last_unstale":"2023-10-30T12:56:21.757142+0000","last_undegraded":"2023-10-30T12:56:21.757142+0000","last_fullsized":"2023-10-30T12:56:21.757142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074617599999999996,"stat_sum":{"num_bytes":5524945,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":72,"num_read_kb":4115,"num_write":631,"num_write_kb":10887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"55'822","reported_seq":2991,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:38.051395+0000","last_change":"2023-10-30T12:56:38.051395+0000","last_active":"2023-10-30T12:56:38.051395+0000","last_peered":"2023-10-30T12:56:38.051395+0000","last_clean":"2023-10-30T12:56:38.051395+0000","last_became_active":"2023-10-30T12:35:06.215384+0000","last_became_peered":"2023-10-30T12:35:06.215384+0000","last_unstale":"2023-10-30T12:56:38.051395+0000","last_undegraded":"2023-10-30T12:56:38.051395+0000","last_fullsized":"2023-10-30T12:56:38.051395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'822","last_scrub_stamp":"2023-10-30T12:56:38.051202+0000","last_deep_scrub":"55'822","last_deep_scrub_stamp":"2023-10-30T12:56:38.051202+0000","last_clean_scrub_stamp":"2023-10-30T12:56:38.051202+0000","objects_scrubbed":148,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:06:45.015408+0000","scrub_duration":0.26303434999999997,"objects_trimmed":0,"snaptrim_duration":0.0080059940000000007,"stat_sum":{"num_bytes":17915594,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":64,"num_read_kb":28,"num_write":636,"num_write_kb":19744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"55'821","reported_seq":2993,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:38.940436+0000","last_change":"2023-10-30T12:56:38.940436+0000","last_active":"2023-10-30T12:56:38.940436+0000","last_peered":"2023-10-30T12:56:38.940436+0000","last_clean":"2023-10-30T12:56:38.940436+0000","last_became_active":"2023-10-30T12:35:06.215355+0000","last_became_peered":"2023-10-30T12:35:06.215355+0000","last_unstale":"2023-10-30T12:56:38.940436+0000","last_undegraded":"2023-10-30T12:56:38.940436+0000","last_fullsized":"2023-10-30T12:56:38.940436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'821","last_scrub_stamp":"2023-10-30T12:56:38.940294+0000","last_deep_scrub":"55'821","last_deep_scrub_stamp":"2023-10-30T12:56:38.940294+0000","last_clean_scrub_stamp":"2023-10-30T12:56:38.940294+0000","objects_scrubbed":135,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:00:12.683787+0000","scrub_duration":0.18969223099999999,"objects_trimmed":0,"snaptrim_duration":0.011778769,"stat_sum":{"num_bytes":10475791,"num_objects":135,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":67,"num_read_kb":8209,"num_write":651,"num_write_kb":25832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"55'780","reported_seq":2951,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:27.866741+0000","last_change":"2023-10-30T12:56:27.866741+0000","last_active":"2023-10-30T12:56:27.866741+0000","last_peered":"2023-10-30T12:56:27.866741+0000","last_clean":"2023-10-30T12:56:27.866741+0000","last_became_active":"2023-10-30T12:35:06.196503+0000","last_became_peered":"2023-10-30T12:35:06.196503+0000","last_unstale":"2023-10-30T12:56:27.866741+0000","last_undegraded":"2023-10-30T12:56:27.866741+0000","last_fullsized":"2023-10-30T12:56:27.866741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'780","last_scrub_stamp":"2023-10-30T12:56:27.866427+0000","last_deep_scrub":"55'780","last_deep_scrub_stamp":"2023-10-30T12:56:27.866427+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.866427+0000","objects_scrubbed":163,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:24:58.673270+0000","scrub_duration":0.27317086899999998,"objects_trimmed":0,"snaptrim_duration":0.0030385149999999999,"stat_sum":{"num_bytes":16276064,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":65,"num_read_kb":7648,"num_write":632,"num_write_kb":36085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"55'724","reported_seq":2893,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:19.488388+0000","last_change":"2023-10-30T12:42:02.241621+0000","last_active":"2023-10-30T12:56:19.488388+0000","last_peered":"2023-10-30T12:56:19.488388+0000","last_clean":"2023-10-30T12:56:19.488388+0000","last_became_active":"2023-10-30T12:35:06.195408+0000","last_became_peered":"2023-10-30T12:35:06.195408+0000","last_unstale":"2023-10-30T12:56:19.488388+0000","last_undegraded":"2023-10-30T12:56:19.488388+0000","last_fullsized":"2023-10-30T12:56:19.488388+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004560812,"stat_sum":{"num_bytes":30579039,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":77,"num_read_kb":9808,"num_write":665,"num_write_kb":49630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"55'828","reported_seq":3005,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:41.613067+0000","last_change":"2023-10-30T12:56:41.613105+0000","last_active":"2023-10-30T12:56:41.613067+0000","last_peered":"2023-10-30T12:56:41.613067+0000","last_clean":"2023-10-30T12:56:41.613067+0000","last_became_active":"2023-10-30T12:35:06.182573+0000","last_became_peered":"2023-10-30T12:35:06.182573+0000","last_unstale":"2023-10-30T12:56:41.613067+0000","last_undegraded":"2023-10-30T12:56:41.613067+0000","last_fullsized":"2023-10-30T12:56:41.613067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'828","last_scrub_stamp":"2023-10-30T12:56:41.612983+0000","last_deep_scrub":"55'828","last_deep_scrub_stamp":"2023-10-30T12:56:41.612983+0000","last_clean_scrub_stamp":"2023-10-30T12:56:41.612983+0000","objects_scrubbed":150,"log_size":828,"log_dups_size":0,"ondisk_log_size":828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:01:59.566492+0000","scrub_duration":0.154696469,"objects_trimmed":0,"snaptrim_duration":0.003020062,"stat_sum":{"num_bytes":14557524,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":71,"num_read_kb":4113,"num_write":642,"num_write_kb":31601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"55'865","reported_seq":3034,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:42.698759+0000","last_change":"2023-10-30T12:56:42.698825+0000","last_active":"2023-10-30T12:56:42.698759+0000","last_peered":"2023-10-30T12:56:42.698759+0000","last_clean":"2023-10-30T12:56:42.698759+0000","last_became_active":"2023-10-30T12:35:06.185290+0000","last_became_peered":"2023-10-30T12:35:06.185290+0000","last_unstale":"2023-10-30T12:56:42.698759+0000","last_undegraded":"2023-10-30T12:56:42.698759+0000","last_fullsized":"2023-10-30T12:56:42.698759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'865","last_scrub_stamp":"2023-10-30T12:56:42.698683+0000","last_deep_scrub":"55'865","last_deep_scrub_stamp":"2023-10-30T12:56:42.698683+0000","last_clean_scrub_stamp":"2023-10-30T12:56:42.698683+0000","objects_scrubbed":171,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:56.915839+0000","scrub_duration":0.293370403,"objects_trimmed":0,"snaptrim_duration":0.002282292,"stat_sum":{"num_bytes":14763187,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":63,"num_read_kb":11730,"num_write":643,"num_write_kb":27692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"55'718","reported_seq":2869,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.187624+0000","last_change":"2023-10-30T12:42:02.748073+0000","last_active":"2023-10-30T12:56:14.187624+0000","last_peered":"2023-10-30T12:56:14.187624+0000","last_clean":"2023-10-30T12:56:14.187624+0000","last_became_active":"2023-10-30T12:35:06.213627+0000","last_became_peered":"2023-10-30T12:35:06.213627+0000","last_unstale":"2023-10-30T12:56:14.187624+0000","last_undegraded":"2023-10-30T12:56:14.187624+0000","last_fullsized":"2023-10-30T12:56:14.187624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014095760000000001,"stat_sum":{"num_bytes":9839966,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":59,"num_read_kb":4114,"num_write":651,"num_write_kb":31705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"55'914","reported_seq":3083,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:45.835829+0000","last_change":"2023-10-30T12:56:45.835874+0000","last_active":"2023-10-30T12:56:45.835829+0000","last_peered":"2023-10-30T12:56:45.835829+0000","last_clean":"2023-10-30T12:56:45.835829+0000","last_became_active":"2023-10-30T12:35:06.221696+0000","last_became_peered":"2023-10-30T12:35:06.221696+0000","last_unstale":"2023-10-30T12:56:45.835829+0000","last_undegraded":"2023-10-30T12:56:45.835829+0000","last_fullsized":"2023-10-30T12:56:45.835829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'914","last_scrub_stamp":"2023-10-30T12:56:45.835737+0000","last_deep_scrub":"55'914","last_deep_scrub_stamp":"2023-10-30T12:56:45.835737+0000","last_clean_scrub_stamp":"2023-10-30T12:56:45.835737+0000","objects_scrubbed":175,"log_size":914,"log_dups_size":0,"ondisk_log_size":914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:37:21.337050+0000","scrub_duration":0.41149325399999997,"objects_trimmed":0,"snaptrim_duration":0.015743672,"stat_sum":{"num_bytes":28343169,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":62,"num_read_kb":16401,"num_write":691,"num_write_kb":59410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"55'804","reported_seq":2992,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.435565+0000","last_change":"2023-10-30T12:56:24.435609+0000","last_active":"2023-10-30T12:56:24.435565+0000","last_peered":"2023-10-30T12:56:24.435565+0000","last_clean":"2023-10-30T12:56:24.435565+0000","last_became_active":"2023-10-30T12:35:06.198074+0000","last_became_peered":"2023-10-30T12:35:06.198074+0000","last_unstale":"2023-10-30T12:56:24.435565+0000","last_undegraded":"2023-10-30T12:56:24.435565+0000","last_fullsized":"2023-10-30T12:56:24.435565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'804","last_scrub_stamp":"2023-10-30T12:56:24.435488+0000","last_deep_scrub":"55'804","last_deep_scrub_stamp":"2023-10-30T12:56:24.435488+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.435488+0000","objects_scrubbed":156,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:44:11.442590+0000","scrub_duration":0.24729864700000001,"objects_trimmed":0,"snaptrim_duration":0.0023389270000000002,"stat_sum":{"num_bytes":28976684,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":82,"num_read_kb":27,"num_write":642,"num_write_kb":39937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"55'706","reported_seq":2858,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.837938+0000","last_change":"2023-10-30T12:42:02.473536+0000","last_active":"2023-10-30T12:56:14.837938+0000","last_peered":"2023-10-30T12:56:14.837938+0000","last_clean":"2023-10-30T12:56:14.837938+0000","last_became_active":"2023-10-30T12:35:06.215935+0000","last_became_peered":"2023-10-30T12:35:06.215935+0000","last_unstale":"2023-10-30T12:56:14.837938+0000","last_undegraded":"2023-10-30T12:56:14.837938+0000","last_fullsized":"2023-10-30T12:56:14.837938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062537060000000004,"stat_sum":{"num_bytes":21535467,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":60,"num_read_kb":12305,"num_write":656,"num_write_kb":39478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"55'805","reported_seq":2985,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:30.019053+0000","last_change":"2023-10-30T12:56:30.019160+0000","last_active":"2023-10-30T12:56:30.019053+0000","last_peered":"2023-10-30T12:56:30.019053+0000","last_clean":"2023-10-30T12:56:30.019053+0000","last_became_active":"2023-10-30T12:35:06.199812+0000","last_became_peered":"2023-10-30T12:35:06.199812+0000","last_unstale":"2023-10-30T12:56:30.019053+0000","last_undegraded":"2023-10-30T12:56:30.019053+0000","last_fullsized":"2023-10-30T12:56:30.019053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'805","last_scrub_stamp":"2023-10-30T12:56:30.018936+0000","last_deep_scrub":"55'805","last_deep_scrub_stamp":"2023-10-30T12:56:30.018936+0000","last_clean_scrub_stamp":"2023-10-30T12:56:30.018936+0000","objects_scrubbed":143,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:47:14.318405+0000","scrub_duration":0.21736029700000001,"objects_trimmed":0,"snaptrim_duration":0.011375481999999999,"stat_sum":{"num_bytes":6303882,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":75,"num_read_kb":20150,"num_write":647,"num_write_kb":36292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"55'899","reported_seq":3084,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:50.693229+0000","last_change":"2023-10-30T12:56:50.693274+0000","last_active":"2023-10-30T12:56:50.693229+0000","last_peered":"2023-10-30T12:56:50.693229+0000","last_clean":"2023-10-30T12:56:50.693229+0000","last_became_active":"2023-10-30T12:35:06.216861+0000","last_became_peered":"2023-10-30T12:35:06.216861+0000","last_unstale":"2023-10-30T12:56:50.693229+0000","last_undegraded":"2023-10-30T12:56:50.693229+0000","last_fullsized":"2023-10-30T12:56:50.693229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'899","last_scrub_stamp":"2023-10-30T12:56:50.693152+0000","last_deep_scrub":"55'899","last_deep_scrub_stamp":"2023-10-30T12:56:50.693152+0000","last_clean_scrub_stamp":"2023-10-30T12:56:50.693152+0000","objects_scrubbed":180,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:56:05.148326+0000","scrub_duration":0.23889443799999999,"objects_trimmed":0,"snaptrim_duration":0.0040841849999999997,"stat_sum":{"num_bytes":12370338,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":78,"num_read_kb":8213,"num_write":666,"num_write_kb":26377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"55'868","reported_seq":3032,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:33.615651+0000","last_change":"2023-10-30T12:56:33.615740+0000","last_active":"2023-10-30T12:56:33.615651+0000","last_peered":"2023-10-30T12:56:33.615651+0000","last_clean":"2023-10-30T12:56:33.615651+0000","last_became_active":"2023-10-30T12:35:46.122517+0000","last_became_peered":"2023-10-30T12:35:46.122517+0000","last_unstale":"2023-10-30T12:56:33.615651+0000","last_undegraded":"2023-10-30T12:56:33.615651+0000","last_fullsized":"2023-10-30T12:56:33.615651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'868","last_scrub_stamp":"2023-10-30T12:56:33.615559+0000","last_deep_scrub":"55'868","last_deep_scrub_stamp":"2023-10-30T12:56:33.615559+0000","last_clean_scrub_stamp":"2023-10-30T12:56:33.615559+0000","objects_scrubbed":170,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:51:10.318228+0000","scrub_duration":0.26875042100000002,"objects_trimmed":0,"snaptrim_duration":0.0020435240000000001,"stat_sum":{"num_bytes":19531616,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":61,"num_read_kb":21,"num_write":688,"num_write_kb":30111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"55'848","reported_seq":3035,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:42.567324+0000","last_change":"2023-10-30T12:56:42.567417+0000","last_active":"2023-10-30T12:56:42.567324+0000","last_peered":"2023-10-30T12:56:42.567324+0000","last_clean":"2023-10-30T12:56:42.567324+0000","last_became_active":"2023-10-30T12:35:06.182314+0000","last_became_peered":"2023-10-30T12:35:06.182314+0000","last_unstale":"2023-10-30T12:56:42.567324+0000","last_undegraded":"2023-10-30T12:56:42.567324+0000","last_fullsized":"2023-10-30T12:56:42.567324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'848","last_scrub_stamp":"2023-10-30T12:56:42.567236+0000","last_deep_scrub":"55'848","last_deep_scrub_stamp":"2023-10-30T12:56:42.567236+0000","last_clean_scrub_stamp":"2023-10-30T12:56:42.567236+0000","objects_scrubbed":161,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:44:52.946140+0000","scrub_duration":0.27460055700000002,"objects_trimmed":0,"snaptrim_duration":0.0051111200000000002,"stat_sum":{"num_bytes":5467082,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":80,"num_read_kb":15696,"num_write":658,"num_write_kb":25502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"55'829","reported_seq":3003,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:32.815869+0000","last_change":"2023-10-30T12:56:32.815869+0000","last_active":"2023-10-30T12:56:32.815869+0000","last_peered":"2023-10-30T12:56:32.815869+0000","last_clean":"2023-10-30T12:56:32.815869+0000","last_became_active":"2023-10-30T12:35:06.213402+0000","last_became_peered":"2023-10-30T12:35:06.213402+0000","last_unstale":"2023-10-30T12:56:32.815869+0000","last_undegraded":"2023-10-30T12:56:32.815869+0000","last_fullsized":"2023-10-30T12:56:32.815869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'829","last_scrub_stamp":"2023-10-30T12:56:32.815282+0000","last_deep_scrub":"55'829","last_deep_scrub_stamp":"2023-10-30T12:56:32.815282+0000","last_clean_scrub_stamp":"2023-10-30T12:56:32.815282+0000","objects_scrubbed":159,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:42:54.304491+0000","scrub_duration":0.307345284,"objects_trimmed":0,"snaptrim_duration":0.014055767,"stat_sum":{"num_bytes":10241279,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":67,"num_read_kb":4112,"num_write":657,"num_write_kb":20530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"55'713","reported_seq":2864,"reported_epoch":55,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-30T12:56:57.534368+0000","last_change":"2023-10-30T12:56:57.491192+0000","last_active":"2023-10-30T12:56:57.534368+0000","last_peered":"2023-10-30T12:56:57.534368+0000","last_clean":"2023-10-30T12:56:57.534368+0000","last_became_active":"2023-10-30T12:35:06.188298+0000","last_became_peered":"2023-10-30T12:35:06.188298+0000","last_unstale":"2023-10-30T12:56:57.534368+0000","last_undegraded":"2023-10-30T12:56:57.534368+0000","last_fullsized":"2023-10-30T12:56:57.534368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":24,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053580889999999999,"stat_sum":{"num_bytes":5087264,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":59,"num_read_kb":8204,"num_write":621,"num_write_kb":44181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"55'896","reported_seq":3090,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.080071+0000","last_change":"2023-10-30T12:56:34.080071+0000","last_active":"2023-10-30T12:56:34.080071+0000","last_peered":"2023-10-30T12:56:34.080071+0000","last_clean":"2023-10-30T12:56:34.080071+0000","last_became_active":"2023-10-30T12:35:06.202507+0000","last_became_peered":"2023-10-30T12:35:06.202507+0000","last_unstale":"2023-10-30T12:56:34.080071+0000","last_undegraded":"2023-10-30T12:56:34.080071+0000","last_fullsized":"2023-10-30T12:56:34.080071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'896","last_scrub_stamp":"2023-10-30T12:56:34.079910+0000","last_deep_scrub":"55'896","last_deep_scrub_stamp":"2023-10-30T12:56:34.079910+0000","last_clean_scrub_stamp":"2023-10-30T12:56:34.079910+0000","objects_scrubbed":173,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:15:45.705065+0000","scrub_duration":0.25185000000000002,"objects_trimmed":0,"snaptrim_duration":0.0082500649999999991,"stat_sum":{"num_bytes":6587197,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":87,"num_read_kb":1892,"num_write":715,"num_write_kb":28794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"55'861","reported_seq":3031,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:46.673561+0000","last_change":"2023-10-30T12:56:46.673623+0000","last_active":"2023-10-30T12:56:46.673561+0000","last_peered":"2023-10-30T12:56:46.673561+0000","last_clean":"2023-10-30T12:56:46.673561+0000","last_became_active":"2023-10-30T12:35:06.170030+0000","last_became_peered":"2023-10-30T12:35:06.170030+0000","last_unstale":"2023-10-30T12:56:46.673561+0000","last_undegraded":"2023-10-30T12:56:46.673561+0000","last_fullsized":"2023-10-30T12:56:46.673561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'861","last_scrub_stamp":"2023-10-30T12:56:46.673491+0000","last_deep_scrub":"55'861","last_deep_scrub_stamp":"2023-10-30T12:56:46.673491+0000","last_clean_scrub_stamp":"2023-10-30T12:56:46.673491+0000","objects_scrubbed":166,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:55:04.341832+0000","scrub_duration":0.18274528900000001,"objects_trimmed":0,"snaptrim_duration":0.0024132049999999999,"stat_sum":{"num_bytes":15296106,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":63,"num_read_kb":15,"num_write":636,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"55'652","reported_seq":2823,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:19.488484+0000","last_change":"2023-10-30T12:42:02.242975+0000","last_active":"2023-10-30T12:56:19.488484+0000","last_peered":"2023-10-30T12:56:19.488484+0000","last_clean":"2023-10-30T12:56:19.488484+0000","last_became_active":"2023-10-30T12:35:06.189171+0000","last_became_peered":"2023-10-30T12:35:06.189171+0000","last_unstale":"2023-10-30T12:56:19.488484+0000","last_undegraded":"2023-10-30T12:56:19.488484+0000","last_fullsized":"2023-10-30T12:56:19.488484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056904870000000001,"stat_sum":{"num_bytes":14055782,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":78,"num_read_kb":12372,"num_write":607,"num_write_kb":31259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"55'824","reported_seq":2992,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:47.631354+0000","last_change":"2023-10-30T12:56:47.631397+0000","last_active":"2023-10-30T12:56:47.631354+0000","last_peered":"2023-10-30T12:56:47.631354+0000","last_clean":"2023-10-30T12:56:47.631354+0000","last_became_active":"2023-10-30T12:35:06.183348+0000","last_became_peered":"2023-10-30T12:35:06.183348+0000","last_unstale":"2023-10-30T12:56:47.631354+0000","last_undegraded":"2023-10-30T12:56:47.631354+0000","last_fullsized":"2023-10-30T12:56:47.631354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'824","last_scrub_stamp":"2023-10-30T12:56:47.631281+0000","last_deep_scrub":"55'824","last_deep_scrub_stamp":"2023-10-30T12:56:47.631281+0000","last_clean_scrub_stamp":"2023-10-30T12:56:47.631281+0000","objects_scrubbed":152,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:43:06.067352+0000","scrub_duration":0.17130867799999999,"objects_trimmed":0,"snaptrim_duration":0.0041317979999999999,"stat_sum":{"num_bytes":14615922,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":60,"num_read_kb":7678,"num_write":625,"num_write_kb":36364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"55'882","reported_seq":3063,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:39.686942+0000","last_change":"2023-10-30T12:56:39.687006+0000","last_active":"2023-10-30T12:56:39.686942+0000","last_peered":"2023-10-30T12:56:39.686942+0000","last_clean":"2023-10-30T12:56:39.686942+0000","last_became_active":"2023-10-30T12:35:06.196368+0000","last_became_peered":"2023-10-30T12:35:06.196368+0000","last_unstale":"2023-10-30T12:56:39.686942+0000","last_undegraded":"2023-10-30T12:56:39.686942+0000","last_fullsized":"2023-10-30T12:56:39.686942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'882","last_scrub_stamp":"2023-10-30T12:56:39.686847+0000","last_deep_scrub":"55'882","last_deep_scrub_stamp":"2023-10-30T12:56:39.686847+0000","last_clean_scrub_stamp":"2023-10-30T12:56:39.686847+0000","objects_scrubbed":148,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:44:17.768223+0000","scrub_duration":0.25242409199999999,"objects_trimmed":0,"snaptrim_duration":0.015569382999999999,"stat_sum":{"num_bytes":14491906,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":75,"num_read_kb":9352,"num_write":694,"num_write_kb":32749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"55'888","reported_seq":3068,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:25.513128+0000","last_change":"2023-10-30T12:56:25.513186+0000","last_active":"2023-10-30T12:56:25.513128+0000","last_peered":"2023-10-30T12:56:25.513128+0000","last_clean":"2023-10-30T12:56:25.513128+0000","last_became_active":"2023-10-30T12:35:06.213994+0000","last_became_peered":"2023-10-30T12:35:06.213994+0000","last_unstale":"2023-10-30T12:56:25.513128+0000","last_undegraded":"2023-10-30T12:56:25.513128+0000","last_fullsized":"2023-10-30T12:56:25.513128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'888","last_scrub_stamp":"2023-10-30T12:56:25.513052+0000","last_deep_scrub":"55'888","last_deep_scrub_stamp":"2023-10-30T12:56:25.513052+0000","last_clean_scrub_stamp":"2023-10-30T12:56:25.513052+0000","objects_scrubbed":188,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:49:47.291732+0000","scrub_duration":0.28763551900000001,"objects_trimmed":0,"snaptrim_duration":0.001873502,"stat_sum":{"num_bytes":23363209,"num_objects":188,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":188,"num_whiteouts":0,"num_read":72,"num_read_kb":4118,"num_write":712,"num_write_kb":32388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"55'855","reported_seq":3032,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:42.099804+0000","last_change":"2023-10-30T12:56:42.099940+0000","last_active":"2023-10-30T12:56:42.099804+0000","last_peered":"2023-10-30T12:56:42.099804+0000","last_clean":"2023-10-30T12:56:42.099804+0000","last_became_active":"2023-10-30T12:35:06.197647+0000","last_became_peered":"2023-10-30T12:35:06.197647+0000","last_unstale":"2023-10-30T12:56:42.099804+0000","last_undegraded":"2023-10-30T12:56:42.099804+0000","last_fullsized":"2023-10-30T12:56:42.099804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'855","last_scrub_stamp":"2023-10-30T12:56:42.099593+0000","last_deep_scrub":"55'855","last_deep_scrub_stamp":"2023-10-30T12:56:42.099593+0000","last_clean_scrub_stamp":"2023-10-30T12:56:42.099593+0000","objects_scrubbed":167,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:51:11.019062+0000","scrub_duration":0.343353621,"objects_trimmed":0,"snaptrim_duration":0.011378932,"stat_sum":{"num_bytes":18364745,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":70,"num_read_kb":8212,"num_write":651,"num_write_kb":28257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"55'773","reported_seq":2939,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:21.810651+0000","last_change":"2023-10-30T12:56:21.810710+0000","last_active":"2023-10-30T12:56:21.810651+0000","last_peered":"2023-10-30T12:56:21.810651+0000","last_clean":"2023-10-30T12:56:21.810651+0000","last_became_active":"2023-10-30T12:35:06.167558+0000","last_became_peered":"2023-10-30T12:35:06.167558+0000","last_unstale":"2023-10-30T12:56:21.810651+0000","last_undegraded":"2023-10-30T12:56:21.810651+0000","last_fullsized":"2023-10-30T12:56:21.810651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'773","last_scrub_stamp":"2023-10-30T12:56:21.810556+0000","last_deep_scrub":"55'773","last_deep_scrub_stamp":"2023-10-30T12:56:21.810556+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.810556+0000","objects_scrubbed":143,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:13:19.546302+0000","scrub_duration":0.23224787799999999,"objects_trimmed":0,"snaptrim_duration":0.0048907120000000002,"stat_sum":{"num_bytes":9710360,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":62,"num_read_kb":17,"num_write":609,"num_write_kb":15663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"55'239","reported_seq":313,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:27.077441+0000","last_change":"2023-10-30T12:56:27.077499+0000","last_active":"2023-10-30T12:56:27.077441+0000","last_peered":"2023-10-30T12:56:27.077441+0000","last_clean":"2023-10-30T12:56:27.077441+0000","last_became_active":"2023-10-30T12:35:05.200169+0000","last_became_peered":"2023-10-30T12:35:05.200169+0000","last_unstale":"2023-10-30T12:56:27.077441+0000","last_undegraded":"2023-10-30T12:56:27.077441+0000","last_fullsized":"2023-10-30T12:56:27.077441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'239","last_scrub_stamp":"2023-10-30T12:56:27.077343+0000","last_deep_scrub":"55'239","last_deep_scrub_stamp":"2023-10-30T12:56:27.077343+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.077343+0000","objects_scrubbed":6,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:21:31.745900+0000","scrub_duration":0.044083678000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":36867,"num_write":183,"num_write_kb":12445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16490,"num_omap_keys":31,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"55'812","reported_seq":2977,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:38.697479+0000","last_change":"2023-10-30T12:56:38.697551+0000","last_active":"2023-10-30T12:56:38.697479+0000","last_peered":"2023-10-30T12:56:38.697479+0000","last_clean":"2023-10-30T12:56:38.697479+0000","last_became_active":"2023-10-30T12:35:06.194689+0000","last_became_peered":"2023-10-30T12:35:06.194689+0000","last_unstale":"2023-10-30T12:56:38.697479+0000","last_undegraded":"2023-10-30T12:56:38.697479+0000","last_fullsized":"2023-10-30T12:56:38.697479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'812","last_scrub_stamp":"2023-10-30T12:56:38.697404+0000","last_deep_scrub":"55'812","last_deep_scrub_stamp":"2023-10-30T12:56:38.697404+0000","last_clean_scrub_stamp":"2023-10-30T12:56:38.697404+0000","objects_scrubbed":144,"log_size":812,"log_dups_size":0,"ondisk_log_size":812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:44:05.310987+0000","scrub_duration":0.23620231899999999,"objects_trimmed":0,"snaptrim_duration":0.004291034,"stat_sum":{"num_bytes":10922380,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":61,"num_read_kb":18,"num_write":626,"num_write_kb":24711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"55'983","reported_seq":1183,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.860910+0000","last_change":"2023-10-30T12:56:24.861007+0000","last_active":"2023-10-30T12:56:24.860910+0000","last_peered":"2023-10-30T12:56:24.860910+0000","last_clean":"2023-10-30T12:56:24.860910+0000","last_became_active":"2023-10-30T12:35:05.182761+0000","last_became_peered":"2023-10-30T12:35:05.182761+0000","last_unstale":"2023-10-30T12:56:24.860910+0000","last_undegraded":"2023-10-30T12:56:24.860910+0000","last_fullsized":"2023-10-30T12:56:24.860910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'983","last_scrub_stamp":"2023-10-30T12:56:24.860748+0000","last_deep_scrub":"55'983","last_deep_scrub_stamp":"2023-10-30T12:56:24.860748+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.860748+0000","objects_scrubbed":8,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:51:38.567709+0000","scrub_duration":0.11984236299999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25625406,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":150,"num_read_kb":45229,"num_write":895,"num_write_kb":25503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"55'458","reported_seq":673,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:21.989512+0000","last_change":"2023-10-30T12:56:21.989512+0000","last_active":"2023-10-30T12:56:21.989512+0000","last_peered":"2023-10-30T12:56:21.989512+0000","last_clean":"2023-10-30T12:56:21.989512+0000","last_became_active":"2023-10-30T12:35:05.180730+0000","last_became_peered":"2023-10-30T12:35:05.180730+0000","last_unstale":"2023-10-30T12:56:21.989512+0000","last_undegraded":"2023-10-30T12:56:21.989512+0000","last_fullsized":"2023-10-30T12:56:21.989512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'458","last_scrub_stamp":"2023-10-30T12:56:21.989395+0000","last_deep_scrub":"55'458","last_deep_scrub_stamp":"2023-10-30T12:56:21.989395+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.989395+0000","objects_scrubbed":4,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:10:50.980457+0000","scrub_duration":0.030827730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8855749,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":162,"num_read_kb":8759,"num_write":390,"num_write_kb":8900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"55'906","reported_seq":3091,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:29.424480+0000","last_change":"2023-10-30T12:56:29.424522+0000","last_active":"2023-10-30T12:56:29.424480+0000","last_peered":"2023-10-30T12:56:29.424480+0000","last_clean":"2023-10-30T12:56:29.424480+0000","last_became_active":"2023-10-30T12:35:46.051057+0000","last_became_peered":"2023-10-30T12:35:46.051057+0000","last_unstale":"2023-10-30T12:56:29.424480+0000","last_undegraded":"2023-10-30T12:56:29.424480+0000","last_fullsized":"2023-10-30T12:56:29.424480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'906","last_scrub_stamp":"2023-10-30T12:56:29.424403+0000","last_deep_scrub":"55'906","last_deep_scrub_stamp":"2023-10-30T12:56:29.424403+0000","last_clean_scrub_stamp":"2023-10-30T12:56:29.424403+0000","objects_scrubbed":175,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:57:06.583434+0000","scrub_duration":0.16198085700000001,"objects_trimmed":0,"snaptrim_duration":0.0018351520000000001,"stat_sum":{"num_bytes":19060908,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":82,"num_read_kb":16408,"num_write":727,"num_write_kb":53009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"55'988","reported_seq":1064,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:30.499309+0000","last_change":"2023-10-30T12:56:30.499309+0000","last_active":"2023-10-30T12:56:30.499309+0000","last_peered":"2023-10-30T12:56:30.499309+0000","last_clean":"2023-10-30T12:56:30.499309+0000","last_became_active":"2023-10-30T12:35:05.201853+0000","last_became_peered":"2023-10-30T12:35:05.201853+0000","last_unstale":"2023-10-30T12:56:30.499309+0000","last_undegraded":"2023-10-30T12:56:30.499309+0000","last_fullsized":"2023-10-30T12:56:30.499309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'988","last_scrub_stamp":"2023-10-30T12:56:30.498957+0000","last_deep_scrub":"55'988","last_deep_scrub_stamp":"2023-10-30T12:56:30.498957+0000","last_clean_scrub_stamp":"2023-10-30T12:56:30.498957+0000","objects_scrubbed":5,"log_size":988,"log_dups_size":0,"ondisk_log_size":988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:34.744919+0000","scrub_duration":0.073408293999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":38325,"num_write":914,"num_write_kb":16753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"55'900","reported_seq":3080,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:39.676436+0000","last_change":"2023-10-30T12:56:39.676436+0000","last_active":"2023-10-30T12:56:39.676436+0000","last_peered":"2023-10-30T12:56:39.676436+0000","last_clean":"2023-10-30T12:56:39.676436+0000","last_became_active":"2023-10-30T12:35:06.170179+0000","last_became_peered":"2023-10-30T12:35:06.170179+0000","last_unstale":"2023-10-30T12:56:39.676436+0000","last_undegraded":"2023-10-30T12:56:39.676436+0000","last_fullsized":"2023-10-30T12:56:39.676436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'900","last_scrub_stamp":"2023-10-30T12:56:39.676328+0000","last_deep_scrub":"55'900","last_deep_scrub_stamp":"2023-10-30T12:56:39.676328+0000","last_clean_scrub_stamp":"2023-10-30T12:56:39.676328+0000","objects_scrubbed":175,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:24:31.809070+0000","scrub_duration":0.198132067,"objects_trimmed":0,"snaptrim_duration":0.0021794980000000002,"stat_sum":{"num_bytes":14292082,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":73,"num_read_kb":12307,"num_write":687,"num_write_kb":28727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"55'151","reported_seq":220,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:49.683601+0000","last_change":"2023-10-30T12:56:27.831941+0000","last_active":"2023-10-30T12:56:49.683601+0000","last_peered":"2023-10-30T12:56:49.683601+0000","last_clean":"2023-10-30T12:56:49.683601+0000","last_became_active":"2023-10-30T12:35:05.197606+0000","last_became_peered":"2023-10-30T12:35:05.197606+0000","last_unstale":"2023-10-30T12:56:49.683601+0000","last_undegraded":"2023-10-30T12:56:49.683601+0000","last_fullsized":"2023-10-30T12:56:49.683601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'149","last_scrub_stamp":"2023-10-30T12:56:27.831485+0000","last_deep_scrub":"55'149","last_deep_scrub_stamp":"2023-10-30T12:56:27.831485+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.831485+0000","objects_scrubbed":5,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:57:34.144550+0000","scrub_duration":0.012229985000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":92,"num_write_kb":201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52416,"num_omap_keys":96,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"55'814","reported_seq":2997,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.786207+0000","last_change":"2023-10-30T12:56:34.786207+0000","last_active":"2023-10-30T12:56:34.786207+0000","last_peered":"2023-10-30T12:56:34.786207+0000","last_clean":"2023-10-30T12:56:34.786207+0000","last_became_active":"2023-10-30T12:35:06.225392+0000","last_became_peered":"2023-10-30T12:35:06.225392+0000","last_unstale":"2023-10-30T12:56:34.786207+0000","last_undegraded":"2023-10-30T12:56:34.786207+0000","last_fullsized":"2023-10-30T12:56:34.786207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'814","last_scrub_stamp":"2023-10-30T12:56:34.786044+0000","last_deep_scrub":"55'814","last_deep_scrub_stamp":"2023-10-30T12:56:34.786044+0000","last_clean_scrub_stamp":"2023-10-30T12:56:34.786044+0000","objects_scrubbed":159,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:26:14.139459+0000","scrub_duration":0.24530717799999999,"objects_trimmed":0,"snaptrim_duration":0.01552166,"stat_sum":{"num_bytes":9754816,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":77,"num_read_kb":18,"num_write":651,"num_write_kb":19117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"55'598","reported_seq":669,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:49.535900+0000","last_change":"2023-10-30T12:56:23.447764+0000","last_active":"2023-10-30T12:56:49.535900+0000","last_peered":"2023-10-30T12:56:49.535900+0000","last_clean":"2023-10-30T12:56:49.535900+0000","last_became_active":"2023-10-30T12:35:05.196899+0000","last_became_peered":"2023-10-30T12:35:05.196899+0000","last_unstale":"2023-10-30T12:56:49.535900+0000","last_undegraded":"2023-10-30T12:56:49.535900+0000","last_fullsized":"2023-10-30T12:56:49.535900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'579","last_scrub_stamp":"2023-10-30T12:56:23.447316+0000","last_deep_scrub":"55'579","last_deep_scrub_stamp":"2023-10-30T12:56:23.447316+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.447316+0000","objects_scrubbed":6,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:59:20.096502+0000","scrub_duration":0.067786201000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18332441,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":14173,"num_write":527,"num_write_kb":18193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"55'865","reported_seq":3034,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:45.787426+0000","last_change":"2023-10-30T12:56:45.787426+0000","last_active":"2023-10-30T12:56:45.787426+0000","last_peered":"2023-10-30T12:56:45.787426+0000","last_clean":"2023-10-30T12:56:45.787426+0000","last_became_active":"2023-10-30T12:35:06.222057+0000","last_became_peered":"2023-10-30T12:35:06.222057+0000","last_unstale":"2023-10-30T12:56:45.787426+0000","last_undegraded":"2023-10-30T12:56:45.787426+0000","last_fullsized":"2023-10-30T12:56:45.787426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'865","last_scrub_stamp":"2023-10-30T12:56:45.787252+0000","last_deep_scrub":"55'865","last_deep_scrub_stamp":"2023-10-30T12:56:45.787252+0000","last_clean_scrub_stamp":"2023-10-30T12:56:45.787252+0000","objects_scrubbed":159,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:23:24.865367+0000","scrub_duration":0.40644253200000002,"objects_trimmed":0,"snaptrim_duration":0.004210915,"stat_sum":{"num_bytes":31400359,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":63,"num_read_kb":12308,"num_write":657,"num_write_kb":50495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"55'317","reported_seq":390,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:18.453440+0000","last_change":"2023-10-30T12:56:18.453440+0000","last_active":"2023-10-30T12:56:18.453440+0000","last_peered":"2023-10-30T12:56:18.453440+0000","last_clean":"2023-10-30T12:56:18.453440+0000","last_became_active":"2023-10-30T12:35:05.197190+0000","last_became_peered":"2023-10-30T12:35:05.197190+0000","last_unstale":"2023-10-30T12:56:18.453440+0000","last_undegraded":"2023-10-30T12:56:18.453440+0000","last_fullsized":"2023-10-30T12:56:18.453440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'317","last_scrub_stamp":"2023-10-30T12:56:18.453257+0000","last_deep_scrub":"55'317","last_deep_scrub_stamp":"2023-10-30T12:56:18.453257+0000","last_clean_scrub_stamp":"2023-10-30T12:56:18.453257+0000","objects_scrubbed":7,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:09:36.655470+0000","scrub_duration":0.034206814000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":4111,"num_write":258,"num_write_kb":8429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28930,"num_omap_keys":53,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"55'791","reported_seq":2957,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:25.811201+0000","last_change":"2023-10-30T12:56:25.811267+0000","last_active":"2023-10-30T12:56:25.811201+0000","last_peered":"2023-10-30T12:56:25.811201+0000","last_clean":"2023-10-30T12:56:25.811201+0000","last_became_active":"2023-10-30T12:35:06.204122+0000","last_became_peered":"2023-10-30T12:35:06.204122+0000","last_unstale":"2023-10-30T12:56:25.811201+0000","last_undegraded":"2023-10-30T12:56:25.811201+0000","last_fullsized":"2023-10-30T12:56:25.811201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'791","last_scrub_stamp":"2023-10-30T12:56:25.811099+0000","last_deep_scrub":"55'791","last_deep_scrub_stamp":"2023-10-30T12:56:25.811099+0000","last_clean_scrub_stamp":"2023-10-30T12:56:25.811099+0000","objects_scrubbed":158,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:21:46.064934+0000","scrub_duration":0.24490572399999999,"objects_trimmed":0,"snaptrim_duration":0.0030324050000000002,"stat_sum":{"num_bytes":15588017,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":60,"num_read_kb":8209,"num_write":632,"num_write_kb":41942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"55'369","reported_seq":450,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:25.809737+0000","last_change":"2023-10-30T12:56:25.809737+0000","last_active":"2023-10-30T12:56:25.809737+0000","last_peered":"2023-10-30T12:56:25.809737+0000","last_clean":"2023-10-30T12:56:25.809737+0000","last_became_active":"2023-10-30T12:35:05.179547+0000","last_became_peered":"2023-10-30T12:35:05.179547+0000","last_unstale":"2023-10-30T12:56:25.809737+0000","last_undegraded":"2023-10-30T12:56:25.809737+0000","last_fullsized":"2023-10-30T12:56:25.809737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'368","last_scrub_stamp":"2023-10-30T12:56:25.809432+0000","last_deep_scrub":"55'368","last_deep_scrub_stamp":"2023-10-30T12:56:25.809432+0000","last_clean_scrub_stamp":"2023-10-30T12:56:25.809432+0000","objects_scrubbed":9,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:55:48.660238+0000","scrub_duration":0.040587917000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":24600,"num_write":301,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17926,"num_omap_keys":33,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"55'784","reported_seq":2951,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.106255+0000","last_change":"2023-10-30T12:56:24.106255+0000","last_active":"2023-10-30T12:56:24.106255+0000","last_peered":"2023-10-30T12:56:24.106255+0000","last_clean":"2023-10-30T12:56:24.106255+0000","last_became_active":"2023-10-30T12:35:06.215128+0000","last_became_peered":"2023-10-30T12:35:06.215128+0000","last_unstale":"2023-10-30T12:56:24.106255+0000","last_undegraded":"2023-10-30T12:56:24.106255+0000","last_fullsized":"2023-10-30T12:56:24.106255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'784","last_scrub_stamp":"2023-10-30T12:56:24.105935+0000","last_deep_scrub":"55'784","last_deep_scrub_stamp":"2023-10-30T12:56:24.105935+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.105935+0000","objects_scrubbed":171,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:40:02.361124+0000","scrub_duration":0.30957912999999998,"objects_trimmed":0,"snaptrim_duration":0.0064662909999999999,"stat_sum":{"num_bytes":22774544,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":61,"num_read_kb":11,"num_write":605,"num_write_kb":27691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"55'215","reported_seq":298,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:18.560689+0000","last_change":"2023-10-30T12:56:18.560737+0000","last_active":"2023-10-30T12:56:18.560689+0000","last_peered":"2023-10-30T12:56:18.560689+0000","last_clean":"2023-10-30T12:56:18.560689+0000","last_became_active":"2023-10-30T12:35:05.189573+0000","last_became_peered":"2023-10-30T12:35:05.189573+0000","last_unstale":"2023-10-30T12:56:18.560689+0000","last_undegraded":"2023-10-30T12:56:18.560689+0000","last_fullsized":"2023-10-30T12:56:18.560689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'215","last_scrub_stamp":"2023-10-30T12:56:18.560612+0000","last_deep_scrub":"55'215","last_deep_scrub_stamp":"2023-10-30T12:56:18.560612+0000","last_clean_scrub_stamp":"2023-10-30T12:56:18.560612+0000","objects_scrubbed":7,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:40:38.364129+0000","scrub_duration":0.037472814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":24624,"num_write":154,"num_write_kb":8340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15144,"num_omap_keys":29,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"55'901","reported_seq":3085,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:40.810421+0000","last_change":"2023-10-30T12:56:40.810463+0000","last_active":"2023-10-30T12:56:40.810421+0000","last_peered":"2023-10-30T12:56:40.810421+0000","last_clean":"2023-10-30T12:56:40.810421+0000","last_became_active":"2023-10-30T12:35:06.216885+0000","last_became_peered":"2023-10-30T12:35:06.216885+0000","last_unstale":"2023-10-30T12:56:40.810421+0000","last_undegraded":"2023-10-30T12:56:40.810421+0000","last_fullsized":"2023-10-30T12:56:40.810421+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'901","last_scrub_stamp":"2023-10-30T12:56:40.810347+0000","last_deep_scrub":"55'901","last_deep_scrub_stamp":"2023-10-30T12:56:40.810347+0000","last_clean_scrub_stamp":"2023-10-30T12:56:40.810347+0000","objects_scrubbed":175,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:17:37.604756+0000","scrub_duration":0.31851338200000001,"objects_trimmed":0,"snaptrim_duration":0.0042006370000000001,"stat_sum":{"num_bytes":19251186,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":77,"num_read_kb":16402,"num_write":677,"num_write_kb":45337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"55'237","reported_seq":307,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:31.515367+0000","last_change":"2023-10-30T12:56:31.515500+0000","last_active":"2023-10-30T12:56:31.515367+0000","last_peered":"2023-10-30T12:56:31.515367+0000","last_clean":"2023-10-30T12:56:31.515367+0000","last_became_active":"2023-10-30T12:35:05.202612+0000","last_became_peered":"2023-10-30T12:35:05.202612+0000","last_unstale":"2023-10-30T12:56:31.515367+0000","last_undegraded":"2023-10-30T12:56:31.515367+0000","last_fullsized":"2023-10-30T12:56:31.515367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'237","last_scrub_stamp":"2023-10-30T12:56:31.515170+0000","last_deep_scrub":"55'237","last_deep_scrub_stamp":"2023-10-30T12:56:31.515170+0000","last_clean_scrub_stamp":"2023-10-30T12:56:31.515170+0000","objects_scrubbed":7,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:51:38.652440+0000","scrub_duration":0.042566652000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":181,"num_write_kb":8338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45273,"num_omap_keys":85,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"55'710","reported_seq":2877,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.184438+0000","last_change":"2023-10-30T12:42:02.471602+0000","last_active":"2023-10-30T12:56:14.184438+0000","last_peered":"2023-10-30T12:56:14.184438+0000","last_clean":"2023-10-30T12:56:14.184438+0000","last_became_active":"2023-10-30T12:35:06.215553+0000","last_became_peered":"2023-10-30T12:35:06.215553+0000","last_unstale":"2023-10-30T12:56:14.184438+0000","last_undegraded":"2023-10-30T12:56:14.184438+0000","last_fullsized":"2023-10-30T12:56:14.184438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060418609999999999,"stat_sum":{"num_bytes":19055642,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":76,"num_read_kb":33,"num_write":654,"num_write_kb":32706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"55'297","reported_seq":377,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:21.765254+0000","last_change":"2023-10-30T12:56:21.765381+0000","last_active":"2023-10-30T12:56:21.765254+0000","last_peered":"2023-10-30T12:56:21.765254+0000","last_clean":"2023-10-30T12:56:21.765254+0000","last_became_active":"2023-10-30T12:35:05.181115+0000","last_became_peered":"2023-10-30T12:35:05.181115+0000","last_unstale":"2023-10-30T12:56:21.765254+0000","last_undegraded":"2023-10-30T12:56:21.765254+0000","last_fullsized":"2023-10-30T12:56:21.765254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'297","last_scrub_stamp":"2023-10-30T12:56:21.765065+0000","last_deep_scrub":"55'297","last_deep_scrub_stamp":"2023-10-30T12:56:21.765065+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.765065+0000","objects_scrubbed":8,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:20:05.549971+0000","scrub_duration":0.074545823999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":43239,"num_write":227,"num_write_kb":16591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":22378,"num_omap_keys":41,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"55'794","reported_seq":2960,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:31.034978+0000","last_change":"2023-10-30T12:56:31.035128+0000","last_active":"2023-10-30T12:56:31.034978+0000","last_peered":"2023-10-30T12:56:31.034978+0000","last_clean":"2023-10-30T12:56:31.034978+0000","last_became_active":"2023-10-30T12:35:06.197722+0000","last_became_peered":"2023-10-30T12:35:06.197722+0000","last_unstale":"2023-10-30T12:56:31.034978+0000","last_undegraded":"2023-10-30T12:56:31.034978+0000","last_fullsized":"2023-10-30T12:56:31.034978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'794","last_scrub_stamp":"2023-10-30T12:56:31.034831+0000","last_deep_scrub":"55'794","last_deep_scrub_stamp":"2023-10-30T12:56:31.034831+0000","last_clean_scrub_stamp":"2023-10-30T12:56:31.034831+0000","objects_scrubbed":168,"log_size":794,"log_dups_size":0,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:35:46.015437+0000","scrub_duration":0.24651738600000001,"objects_trimmed":0,"snaptrim_duration":0.011777173,"stat_sum":{"num_bytes":11179157,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":60,"num_read_kb":16,"num_write":620,"num_write_kb":21517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"55'492","reported_seq":592,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:19.245439+0000","last_change":"2023-10-30T12:56:19.245517+0000","last_active":"2023-10-30T12:56:19.245439+0000","last_peered":"2023-10-30T12:56:19.245439+0000","last_clean":"2023-10-30T12:56:19.245439+0000","last_became_active":"2023-10-30T12:35:05.169130+0000","last_became_peered":"2023-10-30T12:35:05.169130+0000","last_unstale":"2023-10-30T12:56:19.245439+0000","last_undegraded":"2023-10-30T12:56:19.245439+0000","last_fullsized":"2023-10-30T12:56:19.245439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'492","last_scrub_stamp":"2023-10-30T12:56:19.245361+0000","last_deep_scrub":"55'492","last_deep_scrub_stamp":"2023-10-30T12:56:19.245361+0000","last_clean_scrub_stamp":"2023-10-30T12:56:19.245361+0000","objects_scrubbed":12,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:27:01.796605+0000","scrub_duration":0.039470346000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583024,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":18341,"num_write":415,"num_write_kb":12551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10636,"num_omap_keys":20,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"55'720","reported_seq":2903,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.171633+0000","last_change":"2023-10-30T12:42:02.470189+0000","last_active":"2023-10-30T12:56:14.171633+0000","last_peered":"2023-10-30T12:56:14.171633+0000","last_clean":"2023-10-30T12:56:14.171633+0000","last_became_active":"2023-10-30T12:35:06.214839+0000","last_became_peered":"2023-10-30T12:35:06.214839+0000","last_unstale":"2023-10-30T12:56:14.171633+0000","last_undegraded":"2023-10-30T12:56:14.171633+0000","last_fullsized":"2023-10-30T12:56:14.171633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049147749999999997,"stat_sum":{"num_bytes":12184897,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":91,"num_read_kb":16402,"num_write":664,"num_write_kb":29797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"55'315","reported_seq":389,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:20.671077+0000","last_change":"2023-10-30T12:56:20.671275+0000","last_active":"2023-10-30T12:56:20.671077+0000","last_peered":"2023-10-30T12:56:20.671077+0000","last_clean":"2023-10-30T12:56:20.671077+0000","last_became_active":"2023-10-30T12:35:05.188862+0000","last_became_peered":"2023-10-30T12:35:05.188862+0000","last_unstale":"2023-10-30T12:56:20.671077+0000","last_undegraded":"2023-10-30T12:56:20.671077+0000","last_fullsized":"2023-10-30T12:56:20.671077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'315","last_scrub_stamp":"2023-10-30T12:56:20.670866+0000","last_deep_scrub":"55'315","last_deep_scrub_stamp":"2023-10-30T12:56:20.670866+0000","last_clean_scrub_stamp":"2023-10-30T12:56:20.670866+0000","objects_scrubbed":4,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:07:40.146504+0000","scrub_duration":0.058071512999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":12343,"num_write":388,"num_write_kb":4750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9690,"num_omap_keys":18,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"55'673","reported_seq":2830,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.172053+0000","last_change":"2023-10-30T12:42:02.472755+0000","last_active":"2023-10-30T12:56:14.172053+0000","last_peered":"2023-10-30T12:56:14.172053+0000","last_clean":"2023-10-30T12:56:14.172053+0000","last_became_active":"2023-10-30T12:35:06.216533+0000","last_became_peered":"2023-10-30T12:35:06.216533+0000","last_unstale":"2023-10-30T12:56:14.172053+0000","last_undegraded":"2023-10-30T12:56:14.172053+0000","last_fullsized":"2023-10-30T12:56:14.172053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060962430000000003,"stat_sum":{"num_bytes":10771712,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":65,"num_read_kb":4109,"num_write":629,"num_write_kb":24152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"55'238","reported_seq":308,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:23.246213+0000","last_change":"2023-10-30T12:56:23.246263+0000","last_active":"2023-10-30T12:56:23.246213+0000","last_peered":"2023-10-30T12:56:23.246213+0000","last_clean":"2023-10-30T12:56:23.246213+0000","last_became_active":"2023-10-30T12:35:05.177316+0000","last_became_peered":"2023-10-30T12:35:05.177316+0000","last_unstale":"2023-10-30T12:56:23.246213+0000","last_undegraded":"2023-10-30T12:56:23.246213+0000","last_fullsized":"2023-10-30T12:56:23.246213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'238","last_scrub_stamp":"2023-10-30T12:56:23.246132+0000","last_deep_scrub":"55'238","last_deep_scrub_stamp":"2023-10-30T12:56:23.246132+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.246132+0000","objects_scrubbed":4,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:22:00.496493+0000","scrub_duration":0.023404655999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":180,"num_write_kb":8325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12558,"num_omap_keys":23,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"55'846","reported_seq":3021,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:39.581411+0000","last_change":"2023-10-30T12:56:39.581493+0000","last_active":"2023-10-30T12:56:39.581411+0000","last_peered":"2023-10-30T12:56:39.581411+0000","last_clean":"2023-10-30T12:56:39.581411+0000","last_became_active":"2023-10-30T12:35:46.049464+0000","last_became_peered":"2023-10-30T12:35:46.049464+0000","last_unstale":"2023-10-30T12:56:39.581411+0000","last_undegraded":"2023-10-30T12:56:39.581411+0000","last_fullsized":"2023-10-30T12:56:39.581411+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'846","last_scrub_stamp":"2023-10-30T12:56:39.581325+0000","last_deep_scrub":"55'846","last_deep_scrub_stamp":"2023-10-30T12:56:39.581325+0000","last_clean_scrub_stamp":"2023-10-30T12:56:39.581325+0000","objects_scrubbed":163,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:25:02.063225+0000","scrub_duration":0.248567017,"objects_trimmed":0,"snaptrim_duration":0.0023101760000000002,"stat_sum":{"num_bytes":7006288,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":663,"num_write_kb":12867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"55'215","reported_seq":285,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:32.554259+0000","last_change":"2023-10-30T12:56:32.554326+0000","last_active":"2023-10-30T12:56:32.554259+0000","last_peered":"2023-10-30T12:56:32.554259+0000","last_clean":"2023-10-30T12:56:32.554259+0000","last_became_active":"2023-10-30T12:35:05.171491+0000","last_became_peered":"2023-10-30T12:35:05.171491+0000","last_unstale":"2023-10-30T12:56:32.554259+0000","last_undegraded":"2023-10-30T12:56:32.554259+0000","last_fullsized":"2023-10-30T12:56:32.554259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'215","last_scrub_stamp":"2023-10-30T12:56:32.554161+0000","last_deep_scrub":"55'215","last_deep_scrub_stamp":"2023-10-30T12:56:32.554161+0000","last_clean_scrub_stamp":"2023-10-30T12:56:32.554161+0000","objects_scrubbed":4,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:25:32.692345+0000","scrub_duration":0.026739378000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":16387,"num_write":154,"num_write_kb":4208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11738,"num_omap_keys":23,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"55'895","reported_seq":3046,"reported_epoch":55,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-30T12:56:56.801053+0000","last_change":"2023-10-30T12:56:56.344250+0000","last_active":"2023-10-30T12:56:56.801053+0000","last_peered":"2023-10-30T12:56:56.801053+0000","last_clean":"2023-10-30T12:56:56.801053+0000","last_became_active":"2023-10-30T12:35:06.213493+0000","last_became_peered":"2023-10-30T12:35:06.213493+0000","last_unstale":"2023-10-30T12:56:56.801053+0000","last_undegraded":"2023-10-30T12:56:56.801053+0000","last_fullsized":"2023-10-30T12:56:56.801053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":124,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015847370999999999,"stat_sum":{"num_bytes":18244405,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":70,"num_read_kb":4114,"num_write":718,"num_write_kb":33517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"55'310","reported_seq":382,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:20.198207+0000","last_change":"2023-10-30T12:56:20.198258+0000","last_active":"2023-10-30T12:56:20.198207+0000","last_peered":"2023-10-30T12:56:20.198207+0000","last_clean":"2023-10-30T12:56:20.198207+0000","last_became_active":"2023-10-30T12:35:05.197453+0000","last_became_peered":"2023-10-30T12:35:05.197453+0000","last_unstale":"2023-10-30T12:56:20.198207+0000","last_undegraded":"2023-10-30T12:56:20.198207+0000","last_fullsized":"2023-10-30T12:56:20.198207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'310","last_scrub_stamp":"2023-10-30T12:56:20.198131+0000","last_deep_scrub":"55'310","last_deep_scrub_stamp":"2023-10-30T12:56:20.198131+0000","last_clean_scrub_stamp":"2023-10-30T12:56:20.198131+0000","objects_scrubbed":5,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:00:10.556698+0000","scrub_duration":0.022980041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":240,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17972,"num_omap_keys":33,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"55'691","reported_seq":2844,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.183546+0000","last_change":"2023-10-30T12:42:02.472032+0000","last_active":"2023-10-30T12:56:14.183546+0000","last_peered":"2023-10-30T12:56:14.183546+0000","last_clean":"2023-10-30T12:56:14.183546+0000","last_became_active":"2023-10-30T12:35:06.219526+0000","last_became_peered":"2023-10-30T12:35:06.219526+0000","last_unstale":"2023-10-30T12:56:14.183546+0000","last_undegraded":"2023-10-30T12:56:14.183546+0000","last_fullsized":"2023-10-30T12:56:14.183546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060180570000000003,"stat_sum":{"num_bytes":14352444,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":61,"num_read_kb":19722,"num_write":644,"num_write_kb":44993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"55'119","reported_seq":219,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:27.867971+0000","last_change":"2023-10-30T12:56:27.868066+0000","last_active":"2023-10-30T12:56:27.867971+0000","last_peered":"2023-10-30T12:56:27.867971+0000","last_clean":"2023-10-30T12:56:27.867971+0000","last_became_active":"2023-10-30T12:34:55.205923+0000","last_became_peered":"2023-10-30T12:34:55.205923+0000","last_unstale":"2023-10-30T12:56:27.867971+0000","last_undegraded":"2023-10-30T12:56:27.867971+0000","last_fullsized":"2023-10-30T12:56:27.867971+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'119","last_scrub_stamp":"2023-10-30T12:56:27.867851+0000","last_deep_scrub":"55'119","last_deep_scrub_stamp":"2023-10-30T12:56:27.867851+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.867851+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:43:27.714386+0000","scrub_duration":0.0099090500000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"55'316","reported_seq":394,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:28.481897+0000","last_change":"2023-10-30T12:56:28.482017+0000","last_active":"2023-10-30T12:56:28.481897+0000","last_peered":"2023-10-30T12:56:28.481897+0000","last_clean":"2023-10-30T12:56:28.481897+0000","last_became_active":"2023-10-30T12:35:05.197201+0000","last_became_peered":"2023-10-30T12:35:05.197201+0000","last_unstale":"2023-10-30T12:56:28.481897+0000","last_undegraded":"2023-10-30T12:56:28.481897+0000","last_fullsized":"2023-10-30T12:56:28.481897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'316","last_scrub_stamp":"2023-10-30T12:56:28.481711+0000","last_deep_scrub":"55'316","last_deep_scrub_stamp":"2023-10-30T12:56:28.481711+0000","last_clean_scrub_stamp":"2023-10-30T12:56:28.481711+0000","objects_scrubbed":5,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:52.589549+0000","scrub_duration":0.039869544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":23,"num_read_kb":16415,"num_write":260,"num_write_kb":8374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"55'803","reported_seq":2961,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:43.609349+0000","last_change":"2023-10-30T12:56:43.609412+0000","last_active":"2023-10-30T12:56:43.609349+0000","last_peered":"2023-10-30T12:56:43.609349+0000","last_clean":"2023-10-30T12:56:43.609349+0000","last_became_active":"2023-10-30T12:35:06.217687+0000","last_became_peered":"2023-10-30T12:35:06.217687+0000","last_unstale":"2023-10-30T12:56:43.609349+0000","last_undegraded":"2023-10-30T12:56:43.609349+0000","last_fullsized":"2023-10-30T12:56:43.609349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'803","last_scrub_stamp":"2023-10-30T12:56:43.609269+0000","last_deep_scrub":"55'803","last_deep_scrub_stamp":"2023-10-30T12:56:43.609269+0000","last_clean_scrub_stamp":"2023-10-30T12:56:43.609269+0000","objects_scrubbed":158,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:35:10.362189+0000","scrub_duration":0.29605744099999998,"objects_trimmed":0,"snaptrim_duration":0.0059801699999999999,"stat_sum":{"num_bytes":18791963,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":51,"num_read_kb":8203,"num_write":599,"num_write_kb":32827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"55'426","reported_seq":507,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:53.936338+0000","last_change":"2023-10-30T12:56:53.936455+0000","last_active":"2023-10-30T12:56:53.936338+0000","last_peered":"2023-10-30T12:56:53.936338+0000","last_clean":"2023-10-30T12:56:53.936338+0000","last_became_active":"2023-10-30T12:35:05.207828+0000","last_became_peered":"2023-10-30T12:35:05.207828+0000","last_unstale":"2023-10-30T12:56:53.936338+0000","last_undegraded":"2023-10-30T12:56:53.936338+0000","last_fullsized":"2023-10-30T12:56:53.936338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'426","last_scrub_stamp":"2023-10-30T12:56:53.936122+0000","last_deep_scrub":"55'426","last_deep_scrub_stamp":"2023-10-30T12:56:53.936122+0000","last_clean_scrub_stamp":"2023-10-30T12:56:53.936122+0000","objects_scrubbed":7,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:37:43.112167+0000","scrub_duration":0.083282911000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22036983,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":29,"num_read_kb":45074,"num_write":351,"num_write_kb":21726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"55'837","reported_seq":3002,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:28.741030+0000","last_change":"2023-10-30T12:56:28.741094+0000","last_active":"2023-10-30T12:56:28.741030+0000","last_peered":"2023-10-30T12:56:28.741030+0000","last_clean":"2023-10-30T12:56:28.741030+0000","last_became_active":"2023-10-30T12:35:06.194791+0000","last_became_peered":"2023-10-30T12:35:06.194791+0000","last_unstale":"2023-10-30T12:56:28.741030+0000","last_undegraded":"2023-10-30T12:56:28.741030+0000","last_fullsized":"2023-10-30T12:56:28.741030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'837","last_scrub_stamp":"2023-10-30T12:56:28.740945+0000","last_deep_scrub":"55'837","last_deep_scrub_stamp":"2023-10-30T12:56:28.740945+0000","last_clean_scrub_stamp":"2023-10-30T12:56:28.740945+0000","objects_scrubbed":162,"log_size":837,"log_dups_size":0,"ondisk_log_size":837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:55:53.265534+0000","scrub_duration":0.21527656000000001,"objects_trimmed":0,"snaptrim_duration":0.0043414500000000002,"stat_sum":{"num_bytes":10614378,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":58,"num_read_kb":3682,"num_write":656,"num_write_kb":34629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"55'617","reported_seq":984,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:29.620362+0000","last_change":"2023-10-30T12:56:21.219063+0000","last_active":"2023-10-30T12:56:29.620362+0000","last_peered":"2023-10-30T12:56:29.620362+0000","last_clean":"2023-10-30T12:56:29.620362+0000","last_became_active":"2023-10-30T12:35:05.175959+0000","last_became_peered":"2023-10-30T12:35:05.175959+0000","last_unstale":"2023-10-30T12:56:29.620362+0000","last_undegraded":"2023-10-30T12:56:29.620362+0000","last_fullsized":"2023-10-30T12:56:29.620362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'615","last_scrub_stamp":"2023-10-30T12:56:21.218930+0000","last_deep_scrub":"55'615","last_deep_scrub_stamp":"2023-10-30T12:56:21.218930+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.218930+0000","objects_scrubbed":9,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:53:08.332507+0000","scrub_duration":0.031017952000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9398249,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":313,"num_read_kb":20888,"num_write":568,"num_write_kb":9628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34344,"num_omap_keys":63,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"55'863","reported_seq":3038,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:40.689061+0000","last_change":"2023-10-30T12:56:40.689114+0000","last_active":"2023-10-30T12:56:40.689061+0000","last_peered":"2023-10-30T12:56:40.689061+0000","last_clean":"2023-10-30T12:56:40.689061+0000","last_became_active":"2023-10-30T12:35:06.164711+0000","last_became_peered":"2023-10-30T12:35:06.164711+0000","last_unstale":"2023-10-30T12:56:40.689061+0000","last_undegraded":"2023-10-30T12:56:40.689061+0000","last_fullsized":"2023-10-30T12:56:40.689061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'863","last_scrub_stamp":"2023-10-30T12:56:40.688985+0000","last_deep_scrub":"55'863","last_deep_scrub_stamp":"2023-10-30T12:56:40.688985+0000","last_clean_scrub_stamp":"2023-10-30T12:56:40.688985+0000","objects_scrubbed":162,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:58:54.635220+0000","scrub_duration":0.20007371800000001,"objects_trimmed":0,"snaptrim_duration":0.002714419,"stat_sum":{"num_bytes":18668779,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":68,"num_read_kb":8213,"num_write":645,"num_write_kb":33825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"55'501","reported_seq":574,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:22.428808+0000","last_change":"2023-10-30T12:56:22.428932+0000","last_active":"2023-10-30T12:56:22.428808+0000","last_peered":"2023-10-30T12:56:22.428808+0000","last_clean":"2023-10-30T12:56:22.428808+0000","last_became_active":"2023-10-30T12:35:05.196873+0000","last_became_peered":"2023-10-30T12:35:05.196873+0000","last_unstale":"2023-10-30T12:56:22.428808+0000","last_undegraded":"2023-10-30T12:56:22.428808+0000","last_fullsized":"2023-10-30T12:56:22.428808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'501","last_scrub_stamp":"2023-10-30T12:56:22.428591+0000","last_deep_scrub":"55'501","last_deep_scrub_stamp":"2023-10-30T12:56:22.428591+0000","last_clean_scrub_stamp":"2023-10-30T12:56:22.428591+0000","objects_scrubbed":5,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:31.946150+0000","scrub_duration":0.065339878000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":431,"num_write_kb":16687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8414,"num_omap_keys":16,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"55'813","reported_seq":2992,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:22.854721+0000","last_change":"2023-10-30T12:56:22.854850+0000","last_active":"2023-10-30T12:56:22.854721+0000","last_peered":"2023-10-30T12:56:22.854721+0000","last_clean":"2023-10-30T12:56:22.854721+0000","last_became_active":"2023-10-30T12:35:06.183225+0000","last_became_peered":"2023-10-30T12:35:06.183225+0000","last_unstale":"2023-10-30T12:56:22.854721+0000","last_undegraded":"2023-10-30T12:56:22.854721+0000","last_fullsized":"2023-10-30T12:56:22.854721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'813","last_scrub_stamp":"2023-10-30T12:56:22.854526+0000","last_deep_scrub":"55'813","last_deep_scrub_stamp":"2023-10-30T12:56:22.854526+0000","last_clean_scrub_stamp":"2023-10-30T12:56:22.854526+0000","objects_scrubbed":154,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:19:37.784115+0000","scrub_duration":0.30563644699999998,"objects_trimmed":0,"snaptrim_duration":0.0032989500000000001,"stat_sum":{"num_bytes":18054478,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":72,"num_read_kb":12,"num_write":656,"num_write_kb":36307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"55'185","reported_seq":250,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:36.447235+0000","last_change":"2023-10-30T12:56:36.447291+0000","last_active":"2023-10-30T12:56:36.447235+0000","last_peered":"2023-10-30T12:56:36.447235+0000","last_clean":"2023-10-30T12:56:36.447235+0000","last_became_active":"2023-10-30T12:35:05.177266+0000","last_became_peered":"2023-10-30T12:35:05.177266+0000","last_unstale":"2023-10-30T12:56:36.447235+0000","last_undegraded":"2023-10-30T12:56:36.447235+0000","last_fullsized":"2023-10-30T12:56:36.447235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'185","last_scrub_stamp":"2023-10-30T12:56:36.447125+0000","last_deep_scrub":"55'185","last_deep_scrub_stamp":"2023-10-30T12:56:36.447125+0000","last_clean_scrub_stamp":"2023-10-30T12:56:36.447125+0000","objects_scrubbed":2,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:52:35.695395+0000","scrub_duration":0.022317884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":129,"num_write_kb":8259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"55'846","reported_seq":3022,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:43.791966+0000","last_change":"2023-10-30T12:56:43.792110+0000","last_active":"2023-10-30T12:56:43.791966+0000","last_peered":"2023-10-30T12:56:43.791966+0000","last_clean":"2023-10-30T12:56:43.791966+0000","last_became_active":"2023-10-30T12:35:06.196307+0000","last_became_peered":"2023-10-30T12:35:06.196307+0000","last_unstale":"2023-10-30T12:56:43.791966+0000","last_undegraded":"2023-10-30T12:56:43.791966+0000","last_fullsized":"2023-10-30T12:56:43.791966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'846","last_scrub_stamp":"2023-10-30T12:56:43.791757+0000","last_deep_scrub":"55'846","last_deep_scrub_stamp":"2023-10-30T12:56:43.791757+0000","last_clean_scrub_stamp":"2023-10-30T12:56:43.791757+0000","objects_scrubbed":158,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:38:44.933357+0000","scrub_duration":0.30352262899999999,"objects_trimmed":0,"snaptrim_duration":0.015197106,"stat_sum":{"num_bytes":5885946,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":69,"num_read_kb":13,"num_write":633,"num_write_kb":24590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"55'529","reported_seq":593,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:44.631261+0000","last_change":"2023-10-30T12:35:05.207594+0000","last_active":"2023-10-30T12:56:44.631261+0000","last_peered":"2023-10-30T12:56:44.631261+0000","last_clean":"2023-10-30T12:56:44.631261+0000","last_became_active":"2023-10-30T12:35:05.206930+0000","last_became_peered":"2023-10-30T12:35:05.206930+0000","last_unstale":"2023-10-30T12:56:44.631261+0000","last_undegraded":"2023-10-30T12:56:44.631261+0000","last_fullsized":"2023-10-30T12:56:44.631261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:04.132399+0000","last_clean_scrub_stamp":"2023-10-30T12:35:04.132399+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16495116,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":8,"num_read_kb":9517,"num_write":469,"num_write_kb":16398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"55'886","reported_seq":3066,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:44.620900+0000","last_change":"2023-10-30T12:56:44.620946+0000","last_active":"2023-10-30T12:56:44.620900+0000","last_peered":"2023-10-30T12:56:44.620900+0000","last_clean":"2023-10-30T12:56:44.620900+0000","last_became_active":"2023-10-30T12:35:06.164733+0000","last_became_peered":"2023-10-30T12:35:06.164733+0000","last_unstale":"2023-10-30T12:56:44.620900+0000","last_undegraded":"2023-10-30T12:56:44.620900+0000","last_fullsized":"2023-10-30T12:56:44.620900+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'886","last_scrub_stamp":"2023-10-30T12:56:44.620823+0000","last_deep_scrub":"55'886","last_deep_scrub_stamp":"2023-10-30T12:56:44.620823+0000","last_clean_scrub_stamp":"2023-10-30T12:56:44.620823+0000","objects_scrubbed":185,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:12.989394+0000","scrub_duration":0.199244536,"objects_trimmed":0,"snaptrim_duration":0.0024278020000000002,"stat_sum":{"num_bytes":19206810,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":652,"num_write_kb":32920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"55'381","reported_seq":458,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:26.587606+0000","last_change":"2023-10-30T12:56:26.587606+0000","last_active":"2023-10-30T12:56:26.587606+0000","last_peered":"2023-10-30T12:56:26.587606+0000","last_clean":"2023-10-30T12:56:26.587606+0000","last_became_active":"2023-10-30T12:35:05.184695+0000","last_became_peered":"2023-10-30T12:35:05.184695+0000","last_unstale":"2023-10-30T12:56:26.587606+0000","last_undegraded":"2023-10-30T12:56:26.587606+0000","last_fullsized":"2023-10-30T12:56:26.587606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'381","last_scrub_stamp":"2023-10-30T12:56:26.587467+0000","last_deep_scrub":"55'381","last_deep_scrub_stamp":"2023-10-30T12:56:26.587467+0000","last_clean_scrub_stamp":"2023-10-30T12:56:26.587467+0000","objects_scrubbed":7,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:27:14.820811+0000","scrub_duration":0.083109266000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":8489,"num_write":574,"num_write_kb":5859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54698,"num_omap_keys":106,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"55'682","reported_seq":2837,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:19.488439+0000","last_change":"2023-10-30T12:42:02.242088+0000","last_active":"2023-10-30T12:56:19.488439+0000","last_peered":"2023-10-30T12:56:19.488439+0000","last_clean":"2023-10-30T12:56:19.488439+0000","last_became_active":"2023-10-30T12:35:06.204422+0000","last_became_peered":"2023-10-30T12:35:06.204422+0000","last_unstale":"2023-10-30T12:56:19.488439+0000","last_undegraded":"2023-10-30T12:56:19.488439+0000","last_fullsized":"2023-10-30T12:56:19.488439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048535030000000003,"stat_sum":{"num_bytes":22995855,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":62,"num_read_kb":4110,"num_write":640,"num_write_kb":33018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"55'236","reported_seq":308,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:23.771446+0000","last_change":"2023-10-30T12:56:23.771446+0000","last_active":"2023-10-30T12:56:23.771446+0000","last_peered":"2023-10-30T12:56:23.771446+0000","last_clean":"2023-10-30T12:56:23.771446+0000","last_became_active":"2023-10-30T12:35:05.200449+0000","last_became_peered":"2023-10-30T12:35:05.200449+0000","last_unstale":"2023-10-30T12:56:23.771446+0000","last_undegraded":"2023-10-30T12:56:23.771446+0000","last_fullsized":"2023-10-30T12:56:23.771446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'236","last_scrub_stamp":"2023-10-30T12:56:23.771024+0000","last_deep_scrub":"55'236","last_deep_scrub_stamp":"2023-10-30T12:56:23.771024+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.771024+0000","objects_scrubbed":6,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:55:13.133447+0000","scrub_duration":0.033750645000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":4117,"num_write":187,"num_write_kb":4299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18018,"num_omap_keys":33,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"55'831","reported_seq":3007,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:48.796437+0000","last_change":"2023-10-30T12:56:48.796491+0000","last_active":"2023-10-30T12:56:48.796437+0000","last_peered":"2023-10-30T12:56:48.796437+0000","last_clean":"2023-10-30T12:56:48.796437+0000","last_became_active":"2023-10-30T12:35:06.179412+0000","last_became_peered":"2023-10-30T12:35:06.179412+0000","last_unstale":"2023-10-30T12:56:48.796437+0000","last_undegraded":"2023-10-30T12:56:48.796437+0000","last_fullsized":"2023-10-30T12:56:48.796437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'831","last_scrub_stamp":"2023-10-30T12:56:48.796353+0000","last_deep_scrub":"55'831","last_deep_scrub_stamp":"2023-10-30T12:56:48.796353+0000","last_clean_scrub_stamp":"2023-10-30T12:56:48.796353+0000","objects_scrubbed":166,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:11:50.029607+0000","scrub_duration":0.362753192,"objects_trimmed":0,"snaptrim_duration":0.003661568,"stat_sum":{"num_bytes":1664351,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":69,"num_read_kb":22,"num_write":614,"num_write_kb":12460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"55'560","reported_seq":635,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:32.835075+0000","last_change":"2023-10-30T12:56:32.835075+0000","last_active":"2023-10-30T12:56:32.835075+0000","last_peered":"2023-10-30T12:56:32.835075+0000","last_clean":"2023-10-30T12:56:32.835075+0000","last_became_active":"2023-10-30T12:35:05.206607+0000","last_became_peered":"2023-10-30T12:35:05.206607+0000","last_unstale":"2023-10-30T12:56:32.835075+0000","last_undegraded":"2023-10-30T12:56:32.835075+0000","last_fullsized":"2023-10-30T12:56:32.835075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'560","last_scrub_stamp":"2023-10-30T12:56:32.834680+0000","last_deep_scrub":"55'560","last_deep_scrub_stamp":"2023-10-30T12:56:32.834680+0000","last_clean_scrub_stamp":"2023-10-30T12:56:32.834680+0000","objects_scrubbed":8,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:19:51.459668+0000","scrub_duration":0.066587212000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":17,"num_read_kb":23179,"num_write":497,"num_write_kb":12662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34806,"num_omap_keys":64,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"55'809","reported_seq":2986,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.829353+0000","last_change":"2023-10-30T12:56:24.829353+0000","last_active":"2023-10-30T12:56:24.829353+0000","last_peered":"2023-10-30T12:56:24.829353+0000","last_clean":"2023-10-30T12:56:24.829353+0000","last_became_active":"2023-10-30T12:35:06.181256+0000","last_became_peered":"2023-10-30T12:35:06.181256+0000","last_unstale":"2023-10-30T12:56:24.829353+0000","last_undegraded":"2023-10-30T12:56:24.829353+0000","last_fullsized":"2023-10-30T12:56:24.829353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'809","last_scrub_stamp":"2023-10-30T12:56:24.829117+0000","last_deep_scrub":"55'809","last_deep_scrub_stamp":"2023-10-30T12:56:24.829117+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.829117+0000","objects_scrubbed":153,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:42:27.862243+0000","scrub_duration":0.27743204100000002,"objects_trimmed":0,"snaptrim_duration":0.0030387220000000002,"stat_sum":{"num_bytes":10464290,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":70,"num_read_kb":4103,"num_write":644,"num_write_kb":29233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"55'374","reported_seq":447,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.513468+0000","last_change":"2023-10-30T12:56:24.513468+0000","last_active":"2023-10-30T12:56:24.513468+0000","last_peered":"2023-10-30T12:56:24.513468+0000","last_clean":"2023-10-30T12:56:24.513468+0000","last_became_active":"2023-10-30T12:35:05.200724+0000","last_became_peered":"2023-10-30T12:35:05.200724+0000","last_unstale":"2023-10-30T12:56:24.513468+0000","last_undegraded":"2023-10-30T12:56:24.513468+0000","last_fullsized":"2023-10-30T12:56:24.513468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'374","last_scrub_stamp":"2023-10-30T12:56:24.513345+0000","last_deep_scrub":"55'374","last_deep_scrub_stamp":"2023-10-30T12:56:24.513345+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.513345+0000","objects_scrubbed":7,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:34:47.132641+0000","scrub_duration":0.034143153000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388698,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":12300,"num_write":322,"num_write_kb":8548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25586,"num_omap_keys":47,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"55'835","reported_seq":3000,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:35.315188+0000","last_change":"2023-10-30T12:56:35.315188+0000","last_active":"2023-10-30T12:56:35.315188+0000","last_peered":"2023-10-30T12:56:35.315188+0000","last_clean":"2023-10-30T12:56:35.315188+0000","last_became_active":"2023-10-30T12:35:06.173996+0000","last_became_peered":"2023-10-30T12:35:06.173996+0000","last_unstale":"2023-10-30T12:56:35.315188+0000","last_undegraded":"2023-10-30T12:56:35.315188+0000","last_fullsized":"2023-10-30T12:56:35.315188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'835","last_scrub_stamp":"2023-10-30T12:56:35.315063+0000","last_deep_scrub":"55'835","last_deep_scrub_stamp":"2023-10-30T12:56:35.315063+0000","last_clean_scrub_stamp":"2023-10-30T12:56:35.315063+0000","objects_scrubbed":159,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:30:07.209945+0000","scrub_duration":0.14615183500000001,"objects_trimmed":0,"snaptrim_duration":0.0045562010000000002,"stat_sum":{"num_bytes":5693555,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":58,"num_read_kb":18,"num_write":647,"num_write_kb":16270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"55'881","reported_seq":952,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:30.589026+0000","last_change":"2023-10-30T12:56:30.589026+0000","last_active":"2023-10-30T12:56:30.589026+0000","last_peered":"2023-10-30T12:56:30.589026+0000","last_clean":"2023-10-30T12:56:30.589026+0000","last_became_active":"2023-10-30T12:35:05.163794+0000","last_became_peered":"2023-10-30T12:35:05.163794+0000","last_unstale":"2023-10-30T12:56:30.589026+0000","last_undegraded":"2023-10-30T12:56:30.589026+0000","last_fullsized":"2023-10-30T12:56:30.589026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'881","last_scrub_stamp":"2023-10-30T12:56:30.588883+0000","last_deep_scrub":"55'881","last_deep_scrub_stamp":"2023-10-30T12:56:30.588883+0000","last_clean_scrub_stamp":"2023-10-30T12:56:30.588883+0000","objects_scrubbed":7,"log_size":881,"log_dups_size":0,"ondisk_log_size":881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:50:39.690725+0000","scrub_duration":0.062172023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":10,"num_read_kb":8454,"num_write":955,"num_write_kb":13780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23914,"num_omap_keys":46,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"55'808","reported_seq":2982,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:51.694094+0000","last_change":"2023-10-30T12:56:51.694150+0000","last_active":"2023-10-30T12:56:51.694094+0000","last_peered":"2023-10-30T12:56:51.694094+0000","last_clean":"2023-10-30T12:56:51.694094+0000","last_became_active":"2023-10-30T12:35:06.183226+0000","last_became_peered":"2023-10-30T12:35:06.183226+0000","last_unstale":"2023-10-30T12:56:51.694094+0000","last_undegraded":"2023-10-30T12:56:51.694094+0000","last_fullsized":"2023-10-30T12:56:51.694094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'808","last_scrub_stamp":"2023-10-30T12:56:51.694014+0000","last_deep_scrub":"55'808","last_deep_scrub_stamp":"2023-10-30T12:56:51.694014+0000","last_clean_scrub_stamp":"2023-10-30T12:56:51.694014+0000","objects_scrubbed":140,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:16:30.542141+0000","scrub_duration":0.15061528399999999,"objects_trimmed":0,"snaptrim_duration":0.0052521379999999999,"stat_sum":{"num_bytes":5799195,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":69,"num_read_kb":4112,"num_write":618,"num_write_kb":16472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"55'344","reported_seq":419,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:33.883120+0000","last_change":"2023-10-30T12:56:33.883350+0000","last_active":"2023-10-30T12:56:33.883120+0000","last_peered":"2023-10-30T12:56:33.883120+0000","last_clean":"2023-10-30T12:56:33.883120+0000","last_became_active":"2023-10-30T12:35:05.206518+0000","last_became_peered":"2023-10-30T12:35:05.206518+0000","last_unstale":"2023-10-30T12:56:33.883120+0000","last_undegraded":"2023-10-30T12:56:33.883120+0000","last_fullsized":"2023-10-30T12:56:33.883120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'344","last_scrub_stamp":"2023-10-30T12:56:33.882814+0000","last_deep_scrub":"55'344","last_deep_scrub_stamp":"2023-10-30T12:56:33.882814+0000","last_clean_scrub_stamp":"2023-10-30T12:56:33.882814+0000","objects_scrubbed":8,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:13:20.348463+0000","scrub_duration":0.065105804000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":268,"num_write_kb":12461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":36978,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"55'781","reported_seq":2956,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:29.111348+0000","last_change":"2023-10-30T12:56:29.111348+0000","last_active":"2023-10-30T12:56:29.111348+0000","last_peered":"2023-10-30T12:56:29.111348+0000","last_clean":"2023-10-30T12:56:29.111348+0000","last_became_active":"2023-10-30T12:35:06.197861+0000","last_became_peered":"2023-10-30T12:35:06.197861+0000","last_unstale":"2023-10-30T12:56:29.111348+0000","last_undegraded":"2023-10-30T12:56:29.111348+0000","last_fullsized":"2023-10-30T12:56:29.111348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'781","last_scrub_stamp":"2023-10-30T12:56:29.111192+0000","last_deep_scrub":"55'781","last_deep_scrub_stamp":"2023-10-30T12:56:29.111192+0000","last_clean_scrub_stamp":"2023-10-30T12:56:29.111192+0000","objects_scrubbed":157,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:53:54.048846+0000","scrub_duration":0.27439538299999999,"objects_trimmed":0,"snaptrim_duration":0.011118161,"stat_sum":{"num_bytes":14763185,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":68,"num_read_kb":15,"num_write":609,"num_write_kb":24605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"55'844","reported_seq":3030,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:49.658569+0000","last_change":"2023-10-30T12:56:49.658623+0000","last_active":"2023-10-30T12:56:49.658569+0000","last_peered":"2023-10-30T12:56:49.658569+0000","last_clean":"2023-10-30T12:56:49.658569+0000","last_became_active":"2023-10-30T12:35:06.212548+0000","last_became_peered":"2023-10-30T12:35:06.212548+0000","last_unstale":"2023-10-30T12:56:49.658569+0000","last_undegraded":"2023-10-30T12:56:49.658569+0000","last_fullsized":"2023-10-30T12:56:49.658569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'844","last_scrub_stamp":"2023-10-30T12:56:49.658439+0000","last_deep_scrub":"55'844","last_deep_scrub_stamp":"2023-10-30T12:56:49.658439+0000","last_clean_scrub_stamp":"2023-10-30T12:56:49.658439+0000","objects_scrubbed":159,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:08:37.480683+0000","scrub_duration":0.29739869600000002,"objects_trimmed":0,"snaptrim_duration":0.015114931999999999,"stat_sum":{"num_bytes":19366418,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":81,"num_read_kb":7349,"num_write":643,"num_write_kb":32740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"55'368","reported_seq":437,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.907303+0000","last_change":"2023-10-30T12:56:34.907429+0000","last_active":"2023-10-30T12:56:34.907303+0000","last_peered":"2023-10-30T12:56:34.907303+0000","last_clean":"2023-10-30T12:56:34.907303+0000","last_became_active":"2023-10-30T12:35:05.206559+0000","last_became_peered":"2023-10-30T12:35:05.206559+0000","last_unstale":"2023-10-30T12:56:34.907303+0000","last_undegraded":"2023-10-30T12:56:34.907303+0000","last_fullsized":"2023-10-30T12:56:34.907303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'368","last_scrub_stamp":"2023-10-30T12:56:34.907076+0000","last_deep_scrub":"55'368","last_deep_scrub_stamp":"2023-10-30T12:56:34.907076+0000","last_clean_scrub_stamp":"2023-10-30T12:56:34.907076+0000","objects_scrubbed":5,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:12:13.284723+0000","scrub_duration":0.045168409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":312,"num_write_kb":8389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19072,"num_omap_keys":35,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"55'843","reported_seq":3018,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:50.637357+0000","last_change":"2023-10-30T12:56:50.637414+0000","last_active":"2023-10-30T12:56:50.637357+0000","last_peered":"2023-10-30T12:56:50.637357+0000","last_clean":"2023-10-30T12:56:50.637357+0000","last_became_active":"2023-10-30T12:35:06.212505+0000","last_became_peered":"2023-10-30T12:35:06.212505+0000","last_unstale":"2023-10-30T12:56:50.637357+0000","last_undegraded":"2023-10-30T12:56:50.637357+0000","last_fullsized":"2023-10-30T12:56:50.637357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'843","last_scrub_stamp":"2023-10-30T12:56:50.637270+0000","last_deep_scrub":"55'843","last_deep_scrub_stamp":"2023-10-30T12:56:50.637270+0000","last_clean_scrub_stamp":"2023-10-30T12:56:50.637270+0000","objects_scrubbed":156,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:28:10.699275+0000","scrub_duration":0.27534876899999999,"objects_trimmed":0,"snaptrim_duration":0.014553393,"stat_sum":{"num_bytes":9800415,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":70,"num_read_kb":12304,"num_write":642,"num_write_kb":36520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"55'387","reported_seq":467,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:37.980220+0000","last_change":"2023-10-30T12:56:37.980331+0000","last_active":"2023-10-30T12:56:37.980220+0000","last_peered":"2023-10-30T12:56:37.980220+0000","last_clean":"2023-10-30T12:56:37.980220+0000","last_became_active":"2023-10-30T12:35:05.206879+0000","last_became_peered":"2023-10-30T12:35:05.206879+0000","last_unstale":"2023-10-30T12:56:37.980220+0000","last_undegraded":"2023-10-30T12:56:37.980220+0000","last_fullsized":"2023-10-30T12:56:37.980220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'387","last_scrub_stamp":"2023-10-30T12:56:37.980050+0000","last_deep_scrub":"55'387","last_deep_scrub_stamp":"2023-10-30T12:56:37.980050+0000","last_clean_scrub_stamp":"2023-10-30T12:56:37.980050+0000","objects_scrubbed":10,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:37:17.148411+0000","scrub_duration":0.082641178999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":32,"num_read_kb":45093,"num_write":322,"num_write_kb":20730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31096,"num_omap_keys":59,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"55'779","reported_seq":2955,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:56.563086+0000","last_change":"2023-10-30T12:56:56.563165+0000","last_active":"2023-10-30T12:56:56.563086+0000","last_peered":"2023-10-30T12:56:56.563086+0000","last_clean":"2023-10-30T12:56:56.563086+0000","last_became_active":"2023-10-30T12:35:46.275077+0000","last_became_peered":"2023-10-30T12:35:46.275077+0000","last_unstale":"2023-10-30T12:56:56.563086+0000","last_undegraded":"2023-10-30T12:56:56.563086+0000","last_fullsized":"2023-10-30T12:56:56.563086+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'779","last_scrub_stamp":"2023-10-30T12:56:56.562988+0000","last_deep_scrub":"55'779","last_deep_scrub_stamp":"2023-10-30T12:56:56.562988+0000","last_clean_scrub_stamp":"2023-10-30T12:56:56.562988+0000","objects_scrubbed":136,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:59:27.981737+0000","scrub_duration":0.14057847700000001,"objects_trimmed":0,"snaptrim_duration":0.0032416139999999999,"stat_sum":{"num_bytes":6205387,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":63,"num_read_kb":4113,"num_write":608,"num_write_kb":9154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"55'512","reported_seq":582,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:23.020730+0000","last_change":"2023-10-30T12:56:23.020777+0000","last_active":"2023-10-30T12:56:23.020730+0000","last_peered":"2023-10-30T12:56:23.020730+0000","last_clean":"2023-10-30T12:56:23.020730+0000","last_became_active":"2023-10-30T12:35:05.173984+0000","last_became_peered":"2023-10-30T12:35:05.173984+0000","last_unstale":"2023-10-30T12:56:23.020730+0000","last_undegraded":"2023-10-30T12:56:23.020730+0000","last_fullsized":"2023-10-30T12:56:23.020730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'512","last_scrub_stamp":"2023-10-30T12:56:23.020659+0000","last_deep_scrub":"55'512","last_deep_scrub_stamp":"2023-10-30T12:56:23.020659+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.020659+0000","objects_scrubbed":5,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:43:31.108759+0000","scrub_duration":0.032786733999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":430,"num_write_kb":12596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19196,"num_omap_keys":36,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"55'882","reported_seq":3051,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:49.685690+0000","last_change":"2023-10-30T12:56:49.685736+0000","last_active":"2023-10-30T12:56:49.685690+0000","last_peered":"2023-10-30T12:56:49.685690+0000","last_clean":"2023-10-30T12:56:49.685690+0000","last_became_active":"2023-10-30T12:35:06.198504+0000","last_became_peered":"2023-10-30T12:35:06.198504+0000","last_unstale":"2023-10-30T12:56:49.685690+0000","last_undegraded":"2023-10-30T12:56:49.685690+0000","last_fullsized":"2023-10-30T12:56:49.685690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'882","last_scrub_stamp":"2023-10-30T12:56:49.685619+0000","last_deep_scrub":"55'882","last_deep_scrub_stamp":"2023-10-30T12:56:49.685619+0000","last_clean_scrub_stamp":"2023-10-30T12:56:49.685619+0000","objects_scrubbed":182,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:10:12.105803+0000","scrub_duration":0.23854751900000001,"objects_trimmed":0,"snaptrim_duration":0.0055292249999999996,"stat_sum":{"num_bytes":14413504,"num_objects":182,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":61,"num_read_kb":1995,"num_write":647,"num_write_kb":19115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"55'216","reported_seq":307,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:20.534250+0000","last_change":"2023-10-30T12:56:20.534300+0000","last_active":"2023-10-30T12:56:20.534250+0000","last_peered":"2023-10-30T12:56:20.534250+0000","last_clean":"2023-10-30T12:56:20.534250+0000","last_became_active":"2023-10-30T12:35:05.200623+0000","last_became_peered":"2023-10-30T12:35:05.200623+0000","last_unstale":"2023-10-30T12:56:20.534250+0000","last_undegraded":"2023-10-30T12:56:20.534250+0000","last_fullsized":"2023-10-30T12:56:20.534250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'216","last_scrub_stamp":"2023-10-30T12:56:20.534185+0000","last_deep_scrub":"55'216","last_deep_scrub_stamp":"2023-10-30T12:56:20.534185+0000","last_clean_scrub_stamp":"2023-10-30T12:56:20.534185+0000","objects_scrubbed":8,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:19:00.729850+0000","scrub_duration":0.044608693999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":63,"num_read_kb":24633,"num_write":147,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7154,"num_omap_keys":14,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":925262768,"num_objects":10302,"num_object_clones":0,"num_object_copies":20604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10302,"num_whiteouts":0,"num_read":4339,"num_read_kb":400205,"num_write":41481,"num_write_kb":1950238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1902673920,"data_stored":1848985440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51493,"ondisk_log_size":51493,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":360793786,"num_objects":211,"num_object_clones":0,"num_object_copies":422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":211,"num_whiteouts":0,"num_read":1232,"num_read_kb":640401,"num_write":11721,"num_write_kb":363195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":579427,"num_omap_keys":1083,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":721756160,"data_stored":721587572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13255,"ondisk_log_size":13255,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542414,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496924,"kb_used_data":403284,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":93874916,"statfs":{"total":96636764160,"available":96127913984,"internally_reserved":0,"allocated":412962816,"data_stored":404883797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":95877581},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.82299999999999995,"15min":0.81399999999999995},"min":{"1min":0.47699999999999998,"5min":0.438,"15min":0.42899999999999999},"max":{"1min":1.1659999999999999,"5min":1.371,"15min":1.371},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.82899999999999996,"15min":0.82099999999999995},"min":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.41699999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.45},"last":0.89600000000000002}]},{"osd":1,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83599999999999997,"15min":0.85199999999999998},"min":{"1min":0.55200000000000005,"5min":0.52700000000000002,"15min":0.44600000000000001},"max":{"1min":1.095,"5min":1.222,"15min":1.496},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.88,"15min":0.89300000000000002},"min":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.51800000000000002},"max":{"1min":1.282,"5min":1.494,"15min":1.494},"last":1.1319999999999999}]},{"osd":2,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.84399999999999997,"15min":0.82699999999999996},"min":{"1min":0.64800000000000002,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.1479999999999999,"5min":1.242,"15min":1.9059999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.008,"5min":0.872,"15min":0.89700000000000002},"min":{"1min":0.71499999999999997,"5min":0.58299999999999996,"15min":0.48999999999999999},"max":{"1min":1.5609999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.22}]},{"osd":3,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":0.96899999999999997,"15min":0.92500000000000004},"min":{"1min":0.77000000000000002,"5min":0.56999999999999995,"15min":0.54000000000000004},"max":{"1min":1.306,"5min":1.607,"15min":1.9810000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.98099999999999998,"15min":0.996},"min":{"1min":0.59299999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2749999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.94699999999999995}]},{"osd":4,"last update":"Mon Oct 30 12:56:42 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.91400000000000003,"15min":0.93200000000000005},"min":{"1min":0.622,"5min":0.53300000000000003,"15min":0.46899999999999997},"max":{"1min":1.1259999999999999,"5min":1.4059999999999999,"15min":2.7309999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1,"5min":1.0129999999999999,"15min":0.998},"min":{"1min":0.75700000000000001,"5min":0.64800000000000002,"15min":0.41999999999999998},"max":{"1min":1.3680000000000001,"5min":1.694,"15min":1.8859999999999999},"last":1.363}]},{"osd":5,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0449999999999999,"15min":1.0780000000000001},"min":{"1min":0.64500000000000002,"5min":0.59399999999999997,"15min":0.49099999999999999},"max":{"1min":1.375,"5min":1.6379999999999999,"15min":2.2120000000000002},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.228,"15min":1.1539999999999999},"min":{"1min":1.0900000000000001,"5min":0.89900000000000002,"15min":0.69899999999999995},"max":{"1min":1.4330000000000001,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.5549999999999999}]},{"osd":6,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.03,"15min":1.0900000000000001},"min":{"1min":0.72999999999999998,"5min":0.65100000000000002,"15min":0.63600000000000001},"max":{"1min":1.4350000000000001,"5min":1.478,"15min":1.802},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.1419999999999999,"15min":1.1919999999999999},"min":{"1min":0.91200000000000003,"5min":0.879,"15min":0.83799999999999997},"max":{"1min":1.48,"5min":1.6639999999999999,"15min":1.881},"last":1.605}]}]},{"osd":6,"up_from":14,"seq":60129542413,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389436,"kb_used_data":300020,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":93982404,"statfs":{"total":96636764160,"available":96237981696,"internally_reserved":0,"allocated":307220480,"data_stored":301886539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":91552201},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.38600000000000001},"max":{"1min":1.2569999999999999,"5min":5.8760000000000003,"15min":5.8760000000000003},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.85699999999999998,"15min":0.84499999999999997},"min":{"1min":0.55200000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.194,"5min":1.3919999999999999,"15min":1.673},"last":0.68400000000000005}]},{"osd":1,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.85099999999999998,"15min":0.91200000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2290000000000001,"5min":1.6379999999999999,"15min":5.0949999999999998},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.94999999999999996,"15min":0.92500000000000004},"min":{"1min":0.75600000000000001,"5min":0.56899999999999995,"15min":0.45500000000000002},"max":{"1min":1.321,"5min":1.423,"15min":5.0519999999999996},"last":1.056}]},{"osd":2,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.90600000000000003,"15min":0.85199999999999998},"min":{"1min":0.66600000000000004,"5min":0.54400000000000004,"15min":0.51800000000000002},"max":{"1min":1.306,"5min":2.649,"15min":2.649},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.01,"15min":0.93600000000000005},"min":{"1min":0.82499999999999996,"5min":0.59699999999999998,"15min":0.44700000000000001},"max":{"1min":1.4299999999999999,"5min":2.282,"15min":2.282},"last":1.1299999999999999}]},{"osd":3,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.87,"15min":0.93899999999999995},"min":{"1min":0.58899999999999997,"5min":0.51700000000000002,"15min":0.40200000000000002},"max":{"1min":1.1739999999999999,"5min":1.427,"15min":1.929},"last":0.748},{"interface":"front","average":{"1min":0.878,"5min":0.96399999999999997,"15min":0.99099999999999999},"min":{"1min":0.64300000000000002,"5min":0.58499999999999996,"15min":0.42699999999999999},"max":{"1min":1.2470000000000001,"5min":1.444,"15min":1.7589999999999999},"last":0.97799999999999998}]},{"osd":4,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0269999999999999,"15min":0.999},"min":{"1min":0.63600000000000001,"5min":0.63100000000000001,"15min":0.27600000000000002},"max":{"1min":1.5309999999999999,"5min":1.617,"15min":1.9219999999999999},"last":1.177},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0489999999999999,"15min":1.0269999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.32400000000000001},"max":{"1min":1.2809999999999999,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.89000000000000001}]},{"osd":5,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.97199999999999998,"15min":1.022},"min":{"1min":0.67300000000000004,"5min":0.66300000000000003,"15min":0.29599999999999999},"max":{"1min":1.232,"5min":1.619,"15min":2.9529999999999998},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0780000000000001,"15min":1.1220000000000001},"min":{"1min":0.85999999999999999,"5min":0.65300000000000002,"15min":0.34399999999999997},"max":{"1min":1.278,"5min":1.764,"15min":2.5680000000000001},"last":0.84999999999999998}]},{"osd":7,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.167,"15min":1.145},"min":{"1min":0.80100000000000005,"5min":0.79300000000000004,"15min":0.37},"max":{"1min":1.482,"5min":1.7689999999999999,"15min":2.0350000000000001},"last":1.028},{"interface":"front","average":{"1min":1.256,"5min":1.216,"15min":1.194},"min":{"1min":0.95299999999999996,"5min":0.82099999999999995,"15min":0.46800000000000003},"max":{"1min":1.5940000000000001,"5min":1.7829999999999999,"15min":2.274},"last":1.222}]}]},{"osd":4,"up_from":14,"seq":60129542412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404716,"kb_used_data":319396,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":93967124,"statfs":{"total":96636764160,"available":96222334976,"internally_reserved":0,"allocated":327061504,"data_stored":318875235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":87357894},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78100000000000003,"15min":0.80700000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.437},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.5649999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.49299999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.165,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.84999999999999998}]},{"osd":1,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.874,"15min":0.91500000000000004},"min":{"1min":0.55800000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.28,"5min":1.3069999999999999,"15min":2.4910000000000001},"last":0.629},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85699999999999998,"15min":0.84899999999999998},"min":{"1min":0.58899999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.2130000000000001,"5min":1.2390000000000001,"15min":1.9730000000000001},"last":0.90100000000000002}]},{"osd":2,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.93999999999999995,"15min":0.95399999999999996},"min":{"1min":0.59599999999999997,"5min":0.55900000000000005,"15min":0.53400000000000003},"max":{"1min":1.2669999999999999,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.0509999999999999,"15min":1.052},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.55600000000000005},"max":{"1min":1.1859999999999999,"5min":4.6879999999999997,"15min":4.6879999999999997},"last":0.80500000000000005}]},{"osd":3,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.91600000000000004,"15min":0.93300000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.27,"5min":2.266,"15min":2.5819999999999999},"last":1.03},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.96299999999999997,"15min":0.91400000000000003},"min":{"1min":0.68500000000000005,"5min":0.54300000000000004,"15min":0.53500000000000003},"max":{"1min":1.282,"5min":1.411,"15min":1.6890000000000001},"last":0.77700000000000002}]},{"osd":5,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.93899999999999995,"15min":1.002},"min":{"1min":0.67700000000000005,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.3520000000000001,"5min":2.1779999999999999,"15min":2.3180000000000001},"last":1.115},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1080000000000001,"15min":1.0760000000000001},"min":{"1min":1.012,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.478,"5min":3.173,"15min":3.173},"last":1.1659999999999999}]},{"osd":6,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.012,"15min":1.087},"min":{"1min":0.73099999999999998,"5min":0.66600000000000004,"15min":0.59299999999999997},"max":{"1min":1.2649999999999999,"5min":1.641,"15min":2.73},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.073,"15min":1.0800000000000001},"min":{"1min":0.93899999999999995,"5min":0.748,"15min":0.56499999999999995},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":4.7130000000000001},"last":0.97999999999999998}]},{"osd":7,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.86799999999999999,"5min":0.68899999999999995,"15min":0.64900000000000002},"max":{"1min":1.3440000000000001,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.095},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.161,"15min":1.129},"min":{"1min":0.95399999999999996,"5min":0.70299999999999996,"15min":0.65300000000000002},"max":{"1min":1.4370000000000001,"5min":1.766,"15min":3.0649999999999999},"last":1.0589999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542413,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":286224,"kb_used_data":241160,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94085616,"statfs":{"total":96636764160,"available":96343670784,"internally_reserved":0,"allocated":246947840,"data_stored":241832282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.51200000000000001,"15min":0.48499999999999999},"min":{"1min":0.23999999999999999,"5min":0.23200000000000001,"15min":0.22700000000000001},"max":{"1min":0.55000000000000004,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.58099999999999996,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.28699999999999998},"max":{"1min":0.64700000000000002,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":0.61199999999999999,"5min":3.5059999999999998,"15min":4.5019999999999998},"last":0.316},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.35299999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.53400000000000003,"5min":3.5430000000000001,"15min":3.5430000000000001},"last":0.41499999999999998}]},{"osd":3,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.76400000000000001,"15min":0.60399999999999998},"min":{"1min":0.309,"5min":0.26800000000000002,"15min":0.26100000000000001},"max":{"1min":0.71499999999999997,"5min":18.747,"15min":18.747},"last":0.374},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.86199999999999999,"15min":0.69999999999999996},"min":{"1min":0.42599999999999999,"5min":0.38800000000000001,"15min":0.38300000000000001},"max":{"1min":0.81000000000000005,"5min":18.863,"15min":18.863},"last":0.45600000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.85899999999999999,"15min":0.85999999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0620000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.95199999999999996,"15min":0.89900000000000002},"min":{"1min":0.59899999999999998,"5min":0.57599999999999996,"15min":0.48299999999999998},"max":{"1min":1.0029999999999999,"5min":9.6289999999999996,"15min":9.6289999999999996},"last":0.77300000000000002}]},{"osd":5,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.88,"15min":0.84799999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98099999999999998,"5min":9.7289999999999992,"15min":9.7289999999999992},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.9430000000000001,"15min":1.2},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.51900000000000002},"max":{"1min":1.0049999999999999,"5min":91.072999999999993,"15min":91.072999999999993},"last":0.64100000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":1.046,"15min":0.91500000000000004},"min":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.52800000000000002},"max":{"1min":1.137,"5min":19.126000000000001,"15min":19.126000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.86699999999999999,"5min":1.26,"15min":1.008},"min":{"1min":0.67600000000000005,"5min":0.60299999999999998,"15min":0.59399999999999997},"max":{"1min":1.21,"5min":21.469999999999999,"15min":21.469999999999999},"last":0.69299999999999995}]},{"osd":7,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.94499999999999995,"15min":0.89800000000000002},"min":{"1min":0.628,"5min":0.61299999999999999,"15min":0.52100000000000002},"max":{"1min":1.159,"5min":9.6869999999999994,"15min":9.6869999999999994},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.91600000000000004,"5min":1.147,"15min":0.97299999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.53200000000000003},"max":{"1min":1.129,"5min":19.050999999999998,"15min":19.050999999999998},"last":0.82999999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542414,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422368,"kb_used_data":341208,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":93949472,"statfs":{"total":96636764160,"available":96204259328,"internally_reserved":0,"allocated":349396992,"data_stored":341142490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":83098056},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 12:56:40 2023","interfaces":[{"interface":"back","average":{"1min":2.2690000000000001,"5min":0.82999999999999996,"15min":0.60699999999999998},"min":{"1min":0.317,"5min":0.25900000000000001,"15min":0.248},"max":{"1min":38.555999999999997,"5min":38.555999999999997,"15min":38.555999999999997},"last":0.36099999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.60799999999999998,"15min":0.64600000000000002},"min":{"1min":0.32500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":11.653,"5min":11.653,"15min":35.084000000000003},"last":0.51700000000000002}]},{"osd":2,"last update":"Mon Oct 30 12:56:40 2023","interfaces":[{"interface":"back","average":{"1min":2.3090000000000002,"5min":0.878,"15min":0.755},"min":{"1min":0.36199999999999999,"5min":0.312,"15min":0.29799999999999999},"max":{"1min":38.755000000000003,"5min":38.755000000000003,"15min":38.755000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":2.3620000000000001,"5min":0.93799999999999994,"15min":0.81200000000000006},"min":{"1min":0.434,"5min":0.37,"15min":0.35299999999999998},"max":{"1min":38.506999999999998,"5min":38.506999999999998,"15min":38.506999999999998},"last":0.48099999999999998}]},{"osd":3,"last update":"Mon Oct 30 12:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.9079999999999999,"5min":0.81100000000000005,"15min":0.64600000000000002},"min":{"1min":0.38600000000000001,"5min":0.32000000000000001,"15min":0.29499999999999998},"max":{"1min":29.565000000000001,"5min":29.565000000000001,"15min":29.565000000000001},"last":0.63},{"interface":"front","average":{"1min":1.0880000000000001,"5min":0.68300000000000005,"15min":0.66300000000000003},"min":{"1min":0.44900000000000001,"5min":0.38200000000000001,"15min":0.33400000000000002},"max":{"1min":11.609,"5min":11.609,"15min":11.916},"last":0.66500000000000004}]},{"osd":4,"last update":"Mon Oct 30 12:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.05,"15min":0.86799999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.373},"max":{"1min":21.550999999999998,"5min":21.550999999999998,"15min":21.550999999999998},"last":0.92500000000000004},{"interface":"front","average":{"1min":2.7440000000000002,"5min":1.2110000000000001,"15min":1.056},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.39900000000000002},"max":{"1min":38.798000000000002,"5min":38.798000000000002,"15min":38.798000000000002},"last":1.056}]},{"osd":5,"last update":"Mon Oct 30 12:56:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":0.92800000000000005,"15min":0.88300000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.5},"max":{"1min":11.683999999999999,"5min":11.683999999999999,"15min":11.683999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.48799999999999999},"max":{"1min":11.714,"5min":11.714,"15min":34.698},"last":1.012}]},{"osd":6,"last update":"Mon Oct 30 12:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":0.95799999999999996,"15min":0.85999999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.47799999999999998},"max":{"1min":16.495999999999999,"5min":16.495999999999999,"15min":16.495999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.044,"5min":0.92000000000000004,"15min":0.90000000000000002},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.50900000000000001},"max":{"1min":5.6479999999999997,"5min":5.6479999999999997,"15min":8.8800000000000008},"last":1.083}]},{"osd":7,"last update":"Mon Oct 30 12:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.853,"5min":1.032,"15min":0.90700000000000003},"min":{"1min":0.65600000000000003,"5min":0.53000000000000003,"15min":0.437},"max":{"1min":21.462,"5min":21.462,"15min":21.462},"last":0.83099999999999996},{"interface":"front","average":{"1min":2.2930000000000001,"5min":1.173,"15min":1.0269999999999999},"min":{"1min":0.74199999999999999,"5min":0.60599999999999998,"15min":0.55900000000000005},"max":{"1min":28.943999999999999,"5min":28.943999999999999,"15min":28.943999999999999},"last":0.98599999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542412,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477560,"kb_used_data":379760,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":93894280,"statfs":{"total":96636764160,"available":96147742720,"internally_reserved":0,"allocated":388874240,"data_stored":380749259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":100137417},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":0.95099999999999996,"15min":0.86399999999999999},"min":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.41899999999999998},"max":{"1min":7.9409999999999998,"5min":7.9409999999999998,"15min":7.9409999999999998},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.871,"15min":0.85099999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.3169999999999999,"5min":1.341,"15min":1.468},"last":0.81999999999999995}]},{"osd":1,"last update":"Mon Oct 30 12:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":0.91100000000000003,"15min":0.93100000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":12.224,"5min":12.224,"15min":12.224},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.84499999999999997,"15min":0.878},"min":{"1min":0.54000000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":0.877}]},{"osd":2,"last update":"Mon Oct 30 12:56:41 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.90000000000000002,"15min":0.878},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.501},"max":{"1min":2.2770000000000001,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.90200000000000002,"15min":0.85499999999999998},"min":{"1min":0.60399999999999998,"5min":0.54600000000000004,"15min":0.442},"max":{"1min":5.0650000000000004,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":1.1040000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:56:41 2023","interfaces":[{"interface":"back","average":{"1min":2.2879999999999998,"5min":1.2090000000000001,"15min":1.407},"min":{"1min":0.66700000000000004,"5min":0.54400000000000004,"15min":0.504},"max":{"1min":31.815000000000001,"5min":31.815000000000001,"15min":104.88500000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.089,"15min":1.395},"min":{"1min":0.73399999999999999,"5min":0.69099999999999995,"15min":0.60699999999999998},"max":{"1min":8.9019999999999992,"5min":8.9019999999999992,"15min":105.042},"last":0.97499999999999998}]},{"osd":4,"last update":"Mon Oct 30 12:56:41 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.05,"15min":0.95199999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.38500000000000001},"max":{"1min":1.4159999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.069,"5min":1.0629999999999999,"15min":1.125},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.57399999999999995},"max":{"1min":1.3280000000000001,"5min":1.5229999999999999,"15min":1.712},"last":1.226}]},{"osd":6,"last update":"Mon Oct 30 12:56:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.97499999999999998,"15min":0.95899999999999996},"min":{"1min":0.65100000000000002,"5min":0.627,"15min":0.35899999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.9410000000000001},"last":1.135},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.089,"15min":1.0640000000000001},"min":{"1min":0.83799999999999997,"5min":0.72199999999999998,"15min":0.48899999999999999},"max":{"1min":2.9049999999999998,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.3129999999999999}]},{"osd":7,"last update":"Mon Oct 30 12:56:41 2023","interfaces":[{"interface":"back","average":{"1min":2.1030000000000002,"5min":1.3160000000000001,"15min":1.1699999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.40600000000000003},"max":{"1min":24.173999999999999,"5min":24.173999999999999,"15min":24.173999999999999},"last":1.276},{"interface":"front","average":{"1min":1.579,"5min":1.2130000000000001,"15min":1.1459999999999999},"min":{"1min":0.72899999999999998,"5min":0.70699999999999996,"15min":0.53300000000000003},"max":{"1min":12.055,"5min":12.055,"15min":12.055},"last":1.256}]}]},{"osd":3,"up_from":14,"seq":60129542414,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437640,"kb_used_data":352320,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":93934200,"statfs":{"total":96636764160,"available":96188620800,"internally_reserved":0,"allocated":360775680,"data_stored":352416873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":87357898},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.434,"15min":0.435},"min":{"1min":0.27900000000000003,"5min":0.25600000000000001,"15min":0.224},"max":{"1min":0.64600000000000002,"5min":0.77100000000000002,"15min":2.1459999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.46999999999999997,"15min":0.54700000000000004},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.27200000000000002},"max":{"1min":0.66300000000000003,"5min":0.77900000000000003,"15min":21.495999999999999},"last":0.53300000000000003}]},{"osd":1,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.497,"15min":0.53200000000000003},"min":{"1min":0.32700000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.66100000000000003,"5min":1.0700000000000001,"15min":1.1459999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.49199999999999999,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.32600000000000001,"15min":0.30399999999999999},"max":{"1min":0.72899999999999998,"5min":0.83299999999999996,"15min":15.102},"last":0.61299999999999999}]},{"osd":2,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53200000000000003,"15min":0.60499999999999998},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.28299999999999997},"max":{"1min":0.68500000000000005,"5min":0.86399999999999999,"15min":21.321000000000002},"last":0.503},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.41599999999999998,"5min":0.39300000000000002,"15min":0.379},"max":{"1min":0.74299999999999999,"5min":0.97099999999999997,"15min":1.681},"last":0.57699999999999996}]},{"osd":4,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.81399999999999995,"15min":0.89800000000000002},"min":{"1min":0.69399999999999995,"5min":0.49199999999999999,"15min":0.48499999999999999},"max":{"1min":1.0309999999999999,"5min":1.21,"15min":21.382999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.872,"15min":0.92000000000000004},"min":{"1min":0.71299999999999997,"5min":0.60399999999999998,"15min":0.46700000000000003},"max":{"1min":1.0580000000000001,"5min":1.1910000000000001,"15min":15.231999999999999},"last":0.749}]},{"osd":5,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85599999999999998,"15min":0.90200000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.47799999999999998},"max":{"1min":1.282,"5min":1.282,"15min":15.273999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91100000000000003,"15min":0.95899999999999996},"min":{"1min":0.71399999999999997,"5min":0.67700000000000005,"15min":0.57499999999999996},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":15.359999999999999},"last":0.89000000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.59299999999999997,"5min":0.53200000000000003,"15min":0.51100000000000001},"max":{"1min":1.091,"5min":1.296,"15min":1.5620000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82599999999999996,"15min":0.88600000000000001},"min":{"1min":0.66400000000000003,"5min":0.59799999999999998,"15min":0.498},"max":{"1min":1.1279999999999999,"5min":1.198,"15min":15.313000000000001},"last":0.91700000000000004}]},{"osd":7,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.85799999999999998,"15min":0.93000000000000005},"min":{"1min":0.67500000000000004,"5min":0.58799999999999997,"15min":0.55200000000000005},"max":{"1min":1.177,"5min":1.177,"15min":21.428999999999998},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.88,"15min":0.88200000000000001},"min":{"1min":0.71899999999999997,"5min":0.56899999999999995,"15min":0.56799999999999995},"max":{"1min":1.1439999999999999,"5min":1.28,"15min":1.3100000000000001},"last":0.94699999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542412,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307388,"kb_used_data":231860,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94064452,"statfs":{"total":96636764160,"available":96321998848,"internally_reserved":0,"allocated":237424640,"data_stored":232437657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77330888},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.68899999999999995,"15min":0.54500000000000004},"min":{"1min":0.28899999999999998,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.68799999999999994,"5min":15.343999999999999,"15min":15.343999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.73799999999999999,"15min":0.60299999999999998},"min":{"1min":0.39600000000000002,"5min":0.313,"15min":0.29899999999999999},"max":{"1min":1.091,"5min":14.68,"15min":14.68},"last":0.65900000000000003}]},{"osd":2,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.67900000000000005,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.308,"15min":0.29599999999999999},"max":{"1min":0.79900000000000004,"5min":8.1359999999999992,"15min":8.1359999999999992},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.27300000000000002},"max":{"1min":1.3560000000000001,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.57899999999999996}]},{"osd":3,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.64400000000000002,"15min":0.59299999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.044,"5min":8.0920000000000005,"15min":8.0920000000000005},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.32000000000000001},"max":{"1min":0.73699999999999999,"5min":5.9199999999999999,"15min":5.9199999999999999},"last":0.71199999999999997}]},{"osd":4,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":1.131,"15min":0.93999999999999995},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.41799999999999998},"max":{"1min":1.1579999999999999,"5min":21.27,"15min":21.27},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.875,"5min":0.94899999999999995,"15min":0.90600000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.53600000000000003},"max":{"1min":1.145,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":1.083}]},{"osd":5,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83499999999999996,"15min":0.82599999999999996},"min":{"1min":0.63100000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1990000000000001,"5min":1.355,"15min":1.355},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.85899999999999999,"15min":0.85599999999999998},"min":{"1min":0.63700000000000001,"5min":0.499,"15min":0.46200000000000002},"max":{"1min":1.1299999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.88900000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.84599999999999997,"15min":0.84099999999999997},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.51300000000000001},"max":{"1min":1.167,"5min":1.29,"15min":1.4450000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.88100000000000001,"15min":0.88400000000000001},"min":{"1min":0.71199999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2350000000000001,"5min":1.323,"15min":1.468},"last":1.095}]},{"osd":7,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.85499999999999998},"min":{"1min":0.68300000000000005,"5min":0.629,"15min":0.496},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.4299999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.87},"min":{"1min":0.623,"5min":0.623,"15min":0.443},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.86799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":75235328,"data_stored":75216010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":97562624,"data_stored":97535399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42975232,"data_stored":42953409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":85385216,"data_stored":85363568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":88555520,"data_stored":88540092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":89657344,"data_stored":89636987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":111554560,"data_stored":111534614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":130830336,"data_stored":130807493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272936960,"data_stored":265027314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139231232,"data_stored":134593460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":198570075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":274759680,"data_stored":266744507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":237875200,"data_stored":230026345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":298586112,"data_stored":290803474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":195035136,"data_stored":190043127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":280907776,"data_stored":273177138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:57:03.022 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.022 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.023 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.023 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.023 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.024 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.024 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.025 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.025 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.025 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.026 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.027 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.027 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.027 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.028 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.028 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.029 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-30T12:35:04.132399+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=4, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.029 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:03.030 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-30T12:57:08.838 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:57:08.842 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:57:23.031 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:57:23.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.182+0000 7ff6f151d640 1 Processor -- start 2023-10-30T12:57:23.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.182+0000 7ff6f151d640 1 -- start start 2023-10-30T12:57:23.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.182+0000 7ff6f151d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff6ec0c6bd0 0x7ff6ec06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:23.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.182+0000 7ff6f151d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6ec071110 0x7ff6ec158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:23.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.182+0000 7ff6f151d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ec066cf0 con 0x7ff6ec06f7a0 2023-10-30T12:57:23.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.182+0000 7ff6f151d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec0700c0 con 0x7ff6ec0c6bd0 2023-10-30T12:57:23.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.182+0000 7ff6f151d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec0c70d0 con 0x7ff6ec071110 2023-10-30T12:57:23.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.182+0000 7ff6e3fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff6ec0c6bd0 0x7ff6ec06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:23.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.182+0000 7ff6e3fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff6ec0c6bd0 0x7ff6ec06fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49356/0 (socket says 172.21.15.12:49356) 2023-10-30T12:57:23.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.182+0000 7ff6e3fff640 1 -- 172.21.15.12:0/2605208792 learned_addr learned my addr 172.21.15.12:0/2605208792 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:57:23.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.186+0000 7ff6eb7fe640 1 --2- 172.21.15.12:0/2605208792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6ec071110 0x7ff6ec158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:23.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.186+0000 7ff6e3fff640 1 -- 172.21.15.12:0/2605208792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6ec071110 msgr2=0x7ff6ec158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:23.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.186+0000 7ff6e3fff640 1 --2- 172.21.15.12:0/2605208792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6ec071110 0x7ff6ec158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.186+0000 7ff6e3fff640 1 -- 172.21.15.12:0/2605208792 >> v1:172.21.15.154:6789/0 conn(0x7ff6ec06f7a0 legacy=0x7ff6ec069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:23.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.186+0000 7ff6e3fff640 1 -- 172.21.15.12:0/2605208792 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0c7390 con 0x7ff6ec0c6bd0 2023-10-30T12:57:23.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.186+0000 7ff6e3fff640 1 --2- 172.21.15.12:0/2605208792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6ec0c6bd0 0x7ff6ec06fb80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009870 tx=0x7ff6d401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9126fc72ea331f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:23.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.186+0000 7ff6ea7fc640 1 -- 172.21.15.12:0/2605208792 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40a8a30 con 0x7ff6ec0c6bd0 2023-10-30T12:57:23.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.186+0000 7ff6ea7fc640 1 -- 172.21.15.12:0/2605208792 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d40a8b90 con 0x7ff6ec0c6bd0 2023-10-30T12:57:23.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.186+0000 7ff6ea7fc640 1 -- 172.21.15.12:0/2605208792 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40b0440 con 0x7ff6ec0c6bd0 2023-10-30T12:57:23.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.186+0000 7ff6f151d640 1 -- 172.21.15.12:0/2605208792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6ec0c6bd0 msgr2=0x7ff6ec06fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:23.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.186+0000 7ff6f151d640 1 --2- 172.21.15.12:0/2605208792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6ec0c6bd0 0x7ff6ec06fb80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009870 tx=0x7ff6d401ed30 comp rx=0 tx=0).stop 2023-10-30T12:57:23.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6f151d640 1 -- 172.21.15.12:0/2605208792 shutdown_connections 2023-10-30T12:57:23.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6f151d640 1 --2- 172.21.15.12:0/2605208792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6ec071110 0x7ff6ec158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6f151d640 1 --2- 172.21.15.12:0/2605208792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6ec0c6bd0 0x7ff6ec06fb80 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6f151d640 1 -- 172.21.15.12:0/2605208792 >> 172.21.15.12:0/2605208792 conn(0x7ff6ec0c7910 msgr2=0x7ff6ec0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:23.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6f151d640 1 -- 172.21.15.12:0/2605208792 shutdown_connections 2023-10-30T12:57:23.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6f151d640 1 -- 172.21.15.12:0/2605208792 wait complete. 2023-10-30T12:57:23.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6f151d640 1 Processor -- start 2023-10-30T12:57:23.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6f151d640 1 -- start start 2023-10-30T12:57:23.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6f151d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6ec06f7a0 0x7ff6ec1e4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:23.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6f151d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6ec071110 0x7ff6ec14fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:23.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6f151d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6ec0c6bd0 0x7ff6ec153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:23.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6f151d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec1e30c0 con 0x7ff6ec071110 2023-10-30T12:57:23.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6f151d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec1e2810 con 0x7ff6ec0c6bd0 2023-10-30T12:57:23.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6f151d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec0c6880 con 0x7ff6ec06f7a0 2023-10-30T12:57:23.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6e3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6ec071110 0x7ff6ec14fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:23.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6e3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6ec071110 0x7ff6ec14fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49366/0 (socket says 172.21.15.12:49366) 2023-10-30T12:57:23.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6e3fff640 1 -- 172.21.15.12:0/232873300 learned_addr learned my addr 172.21.15.12:0/232873300 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:57:23.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6eb7fe640 1 --2- 172.21.15.12:0/232873300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6ec0c6bd0 0x7ff6ec153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:23.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6eaffd640 1 --2- 172.21.15.12:0/232873300 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6ec06f7a0 0x7ff6ec1e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:23.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6e3fff640 1 -- 172.21.15.12:0/232873300 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6ec06f7a0 msgr2=0x7ff6ec1e4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:23.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6e3fff640 1 --2- 172.21.15.12:0/232873300 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6ec06f7a0 0x7ff6ec1e4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6e3fff640 1 -- 172.21.15.12:0/232873300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6ec0c6bd0 msgr2=0x7ff6ec153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:23.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6e3fff640 1 --2- 172.21.15.12:0/232873300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6ec0c6bd0 0x7ff6ec153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6e3fff640 1 -- 172.21.15.12:0/232873300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d40094d0 con 0x7ff6ec071110 2023-10-30T12:57:23.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.190+0000 7ff6e3fff640 1 --2- 172.21.15.12:0/232873300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6ec071110 0x7ff6ec14fcb0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40b3040 tx=0x7ff6d401bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:23.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.194+0000 7ff6e37fe640 1 -- 172.21.15.12:0/232873300 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40b9620 con 0x7ff6ec071110 2023-10-30T12:57:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.194+0000 7ff6e37fe640 1 -- 172.21.15.12:0/232873300 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d40b97c0 con 0x7ff6ec071110 2023-10-30T12:57:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.194+0000 7ff6e37fe640 1 -- 172.21.15.12:0/232873300 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40b9b70 con 0x7ff6ec071110 2023-10-30T12:57:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.194+0000 7ff6f151d640 1 -- 172.21.15.12:0/232873300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ec1e2ca0 con 0x7ff6ec071110 2023-10-30T12:57:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.194+0000 7ff6f151d640 1 -- 172.21.15.12:0/232873300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ec0b0320 con 0x7ff6ec071110 2023-10-30T12:57:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.194+0000 7ff6f151d640 1 -- 172.21.15.12:0/232873300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ec1e25e0 con 0x7ff6ec071110 2023-10-30T12:57:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.194+0000 7ff6e37fe640 1 -- 172.21.15.12:0/232873300 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff6d40b7070 con 0x7ff6ec071110 2023-10-30T12:57:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.194+0000 7ff6e37fe640 1 --2- 172.21.15.12:0/232873300 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff6cc0664d0 0x7ff6cc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:23.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.194+0000 7ff6e37fe640 1 -- 172.21.15.12:0/232873300 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7ff6d4122190 con 0x7ff6ec071110 2023-10-30T12:57:23.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.194+0000 7ff6eaffd640 1 --2- 172.21.15.12:0/232873300 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff6cc0664d0 0x7ff6cc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:23.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.198+0000 7ff6eaffd640 1 --2- 172.21.15.12:0/232873300 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff6cc0664d0 0x7ff6cc068990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc000f70 tx=0x7ff6dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:23.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.198+0000 7ff6e37fe640 1 -- 172.21.15.12:0/232873300 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff6d40ef220 con 0x7ff6ec071110 2023-10-30T12:57:23.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.350+0000 7ff6f151d640 1 -- 172.21.15.12:0/232873300 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff6ec070350 con 0x7ff6cc0664d0 2023-10-30T12:57:23.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.374+0000 7ff6e37fe640 1 -- 172.21.15.12:0/232873300 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294640 (secure 0 0 0) 0x7ff6dc001380 con 0x7ff6cc0664d0 2023-10-30T12:57:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:57:23.383 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-10-30T12:57:23.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.378+0000 7ff6f151d640 1 -- 172.21.15.12:0/232873300 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff6cc0664d0 msgr2=0x7ff6cc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:23.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.378+0000 7ff6f151d640 1 --2- 172.21.15.12:0/232873300 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff6cc0664d0 0x7ff6cc068990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc000f70 tx=0x7ff6dc009040 comp rx=0 tx=0).stop 2023-10-30T12:57:23.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.378+0000 7ff6f151d640 1 -- 172.21.15.12:0/232873300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6ec071110 msgr2=0x7ff6ec14fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:23.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.378+0000 7ff6f151d640 1 --2- 172.21.15.12:0/232873300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6ec071110 0x7ff6ec14fcb0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40b3040 tx=0x7ff6d401bad0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.378+0000 7ff6f151d640 1 -- 172.21.15.12:0/232873300 shutdown_connections 2023-10-30T12:57:23.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.378+0000 7ff6f151d640 1 --2- 172.21.15.12:0/232873300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6ec0c6bd0 0x7ff6ec153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.378+0000 7ff6f151d640 1 --2- 172.21.15.12:0/232873300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6ec071110 0x7ff6ec14fcb0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.378+0000 7ff6f151d640 1 --2- 172.21.15.12:0/232873300 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff6cc0664d0 0x7ff6cc068990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.378+0000 7ff6f151d640 1 --2- 172.21.15.12:0/232873300 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6ec06f7a0 0x7ff6ec1e4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.378+0000 7ff6f151d640 1 -- 172.21.15.12:0/232873300 >> 172.21.15.12:0/232873300 conn(0x7ff6ec0c7910 msgr2=0x7ff6ec0c3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:23.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.378+0000 7ff6f151d640 1 -- 172.21.15.12:0/232873300 shutdown_connections 2023-10-30T12:57:23.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:23.378+0000 7ff6f151d640 1 -- 172.21.15.12:0/232873300 wait complete. 2023-10-30T12:57:23.401 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":723,"stamp":"2023-10-30T12:57:21.435142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1286759173,"num_objects":10515,"num_object_clones":0,"num_object_copies":21030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10515,"num_whiteouts":0,"num_read":5687,"num_read_kb":1040706,"num_write":53371,"num_write_kb":2315965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":636231,"num_omap_keys":1188,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66984,"ondisk_log_size":66984,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3197512,"kb_used_data":2569224,"kb_used_omap":12,"kb_used_meta":628211,"kb_avail":751777208,"statfs":{"total":773094113280,"available":769819860992,"internally_reserved":0,"allocated":2630885376,"data_stored":2574448634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12729,"internal_metadata":643288647},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":54913,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004651"},"pg_stats":[{"pgid":"3.3f","version":"55'737","reported_seq":2903,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.185374+0000","last_change":"2023-10-30T12:42:02.472363+0000","last_active":"2023-10-30T12:56:14.185374+0000","last_peered":"2023-10-30T12:56:14.185374+0000","last_clean":"2023-10-30T12:56:14.185374+0000","last_became_active":"2023-10-30T12:35:06.222724+0000","last_became_peered":"2023-10-30T12:35:06.222724+0000","last_unstale":"2023-10-30T12:56:14.185374+0000","last_undegraded":"2023-10-30T12:56:14.185374+0000","last_fullsized":"2023-10-30T12:56:14.185374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059524979999999996,"stat_sum":{"num_bytes":14421277,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":73,"num_read_kb":8207,"num_write":692,"num_write_kb":25708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"55'855","reported_seq":3032,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:38.668823+0000","last_change":"2023-10-30T12:56:38.668869+0000","last_active":"2023-10-30T12:56:38.668823+0000","last_peered":"2023-10-30T12:56:38.668823+0000","last_clean":"2023-10-30T12:56:38.668823+0000","last_became_active":"2023-10-30T12:35:06.182441+0000","last_became_peered":"2023-10-30T12:35:06.182441+0000","last_unstale":"2023-10-30T12:56:38.668823+0000","last_undegraded":"2023-10-30T12:56:38.668823+0000","last_fullsized":"2023-10-30T12:56:38.668823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'855","last_scrub_stamp":"2023-10-30T12:56:38.668744+0000","last_deep_scrub":"55'855","last_deep_scrub_stamp":"2023-10-30T12:56:38.668744+0000","last_clean_scrub_stamp":"2023-10-30T12:56:38.668744+0000","objects_scrubbed":154,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:43:43.147201+0000","scrub_duration":0.17361679199999999,"objects_trimmed":0,"snaptrim_duration":0.0026942979999999999,"stat_sum":{"num_bytes":14617239,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":70,"num_read_kb":10,"num_write":675,"num_write_kb":36983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"55'856","reported_seq":3043,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:55.585283+0000","last_change":"2023-10-30T12:56:55.585332+0000","last_active":"2023-10-30T12:56:55.585283+0000","last_peered":"2023-10-30T12:56:55.585283+0000","last_clean":"2023-10-30T12:56:55.585283+0000","last_became_active":"2023-10-30T12:35:06.180132+0000","last_became_peered":"2023-10-30T12:35:06.180132+0000","last_unstale":"2023-10-30T12:56:55.585283+0000","last_undegraded":"2023-10-30T12:56:55.585283+0000","last_fullsized":"2023-10-30T12:56:55.585283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'856","last_scrub_stamp":"2023-10-30T12:56:55.585201+0000","last_deep_scrub":"55'856","last_deep_scrub_stamp":"2023-10-30T12:56:55.585201+0000","last_clean_scrub_stamp":"2023-10-30T12:56:55.585201+0000","objects_scrubbed":159,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:09:47.246202+0000","scrub_duration":0.184201582,"objects_trimmed":0,"snaptrim_duration":0.003537278,"stat_sum":{"num_bytes":22250243,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":80,"num_read_kb":12305,"num_write":653,"num_write_kb":42611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"55'857","reported_seq":3023,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:00.673573+0000","last_change":"2023-10-30T12:57:00.673573+0000","last_active":"2023-10-30T12:57:00.673573+0000","last_peered":"2023-10-30T12:57:00.673573+0000","last_clean":"2023-10-30T12:57:00.673573+0000","last_became_active":"2023-10-30T12:35:06.213987+0000","last_became_peered":"2023-10-30T12:35:06.213987+0000","last_unstale":"2023-10-30T12:57:00.673573+0000","last_undegraded":"2023-10-30T12:57:00.673573+0000","last_fullsized":"2023-10-30T12:57:00.673573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'857","last_scrub_stamp":"2023-10-30T12:57:00.673331+0000","last_deep_scrub":"55'857","last_deep_scrub_stamp":"2023-10-30T12:57:00.673331+0000","last_clean_scrub_stamp":"2023-10-30T12:57:00.673331+0000","objects_scrubbed":175,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:32:46.251054+0000","scrub_duration":0.34115118,"objects_trimmed":0,"snaptrim_duration":0.013528057,"stat_sum":{"num_bytes":18553875,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":57,"num_read_kb":8205,"num_write":627,"num_write_kb":28534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"55'821","reported_seq":2977,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:13.280816+0000","last_change":"2023-10-30T12:57:13.280928+0000","last_active":"2023-10-30T12:57:13.280816+0000","last_peered":"2023-10-30T12:57:13.280816+0000","last_clean":"2023-10-30T12:57:13.280816+0000","last_became_active":"2023-10-30T12:35:06.214964+0000","last_became_peered":"2023-10-30T12:35:06.214964+0000","last_unstale":"2023-10-30T12:57:13.280816+0000","last_undegraded":"2023-10-30T12:57:13.280816+0000","last_fullsized":"2023-10-30T12:57:13.280816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'821","last_scrub_stamp":"2023-10-30T12:57:13.280603+0000","last_deep_scrub":"55'821","last_deep_scrub_stamp":"2023-10-30T12:57:13.280603+0000","last_clean_scrub_stamp":"2023-10-30T12:57:13.280603+0000","objects_scrubbed":151,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:01:06.258406+0000","scrub_duration":0.33545699899999998,"objects_trimmed":0,"snaptrim_duration":0.0046834600000000004,"stat_sum":{"num_bytes":8686005,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":51,"num_read_kb":4112,"num_write":616,"num_write_kb":25845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"55'857","reported_seq":3023,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:52.695822+0000","last_change":"2023-10-30T12:56:52.695935+0000","last_active":"2023-10-30T12:56:52.695822+0000","last_peered":"2023-10-30T12:56:52.695822+0000","last_clean":"2023-10-30T12:56:52.695822+0000","last_became_active":"2023-10-30T12:35:06.214219+0000","last_became_peered":"2023-10-30T12:35:06.214219+0000","last_unstale":"2023-10-30T12:56:52.695822+0000","last_undegraded":"2023-10-30T12:56:52.695822+0000","last_fullsized":"2023-10-30T12:56:52.695822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'857","last_scrub_stamp":"2023-10-30T12:56:52.695649+0000","last_deep_scrub":"55'857","last_deep_scrub_stamp":"2023-10-30T12:56:52.695649+0000","last_clean_scrub_stamp":"2023-10-30T12:56:52.695649+0000","objects_scrubbed":170,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:36:31.854780+0000","scrub_duration":0.32795837700000002,"objects_trimmed":0,"snaptrim_duration":0.015862434000000002,"stat_sum":{"num_bytes":14811547,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":61,"num_read_kb":16,"num_write":643,"num_write_kb":33253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"55'853","reported_seq":3009,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:01.292575+0000","last_change":"2023-10-30T12:57:01.292711+0000","last_active":"2023-10-30T12:57:01.292575+0000","last_peered":"2023-10-30T12:57:01.292575+0000","last_clean":"2023-10-30T12:57:01.292575+0000","last_became_active":"2023-10-30T12:35:06.216468+0000","last_became_peered":"2023-10-30T12:35:06.216468+0000","last_unstale":"2023-10-30T12:57:01.292575+0000","last_undegraded":"2023-10-30T12:57:01.292575+0000","last_fullsized":"2023-10-30T12:57:01.292575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'853","last_scrub_stamp":"2023-10-30T12:57:01.292372+0000","last_deep_scrub":"55'853","last_deep_scrub_stamp":"2023-10-30T12:57:01.292372+0000","last_clean_scrub_stamp":"2023-10-30T12:57:01.292372+0000","objects_scrubbed":156,"log_size":853,"log_dups_size":0,"ondisk_log_size":853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:07:43.317255+0000","scrub_duration":0.33415243100000003,"objects_trimmed":0,"snaptrim_duration":0.0031309480000000002,"stat_sum":{"num_bytes":9595467,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":52,"num_read_kb":12216,"num_write":649,"num_write_kb":36101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"55'786","reported_seq":2947,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:09.776064+0000","last_change":"2023-10-30T12:57:09.776112+0000","last_active":"2023-10-30T12:57:09.776064+0000","last_peered":"2023-10-30T12:57:09.776064+0000","last_clean":"2023-10-30T12:57:09.776064+0000","last_became_active":"2023-10-30T12:35:06.214384+0000","last_became_peered":"2023-10-30T12:35:06.214384+0000","last_unstale":"2023-10-30T12:57:09.776064+0000","last_undegraded":"2023-10-30T12:57:09.776064+0000","last_fullsized":"2023-10-30T12:57:09.776064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'786","last_scrub_stamp":"2023-10-30T12:57:09.775984+0000","last_deep_scrub":"55'786","last_deep_scrub_stamp":"2023-10-30T12:57:09.775984+0000","last_clean_scrub_stamp":"2023-10-30T12:57:09.775984+0000","objects_scrubbed":161,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:50:00.794904+0000","scrub_duration":0.29424167899999998,"objects_trimmed":0,"snaptrim_duration":0.0045552559999999997,"stat_sum":{"num_bytes":9825651,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":56,"num_read_kb":15,"num_write":593,"num_write_kb":23392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"55'695","reported_seq":2859,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:21.757142+0000","last_change":"2023-10-30T12:42:02.473886+0000","last_active":"2023-10-30T12:56:21.757142+0000","last_peered":"2023-10-30T12:56:21.757142+0000","last_clean":"2023-10-30T12:56:21.757142+0000","last_became_active":"2023-10-30T12:35:06.222692+0000","last_became_peered":"2023-10-30T12:35:06.222692+0000","last_unstale":"2023-10-30T12:56:21.757142+0000","last_undegraded":"2023-10-30T12:56:21.757142+0000","last_fullsized":"2023-10-30T12:56:21.757142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074617599999999996,"stat_sum":{"num_bytes":5524945,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":72,"num_read_kb":4115,"num_write":631,"num_write_kb":10887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"55'822","reported_seq":2991,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:38.051395+0000","last_change":"2023-10-30T12:56:38.051395+0000","last_active":"2023-10-30T12:56:38.051395+0000","last_peered":"2023-10-30T12:56:38.051395+0000","last_clean":"2023-10-30T12:56:38.051395+0000","last_became_active":"2023-10-30T12:35:06.215384+0000","last_became_peered":"2023-10-30T12:35:06.215384+0000","last_unstale":"2023-10-30T12:56:38.051395+0000","last_undegraded":"2023-10-30T12:56:38.051395+0000","last_fullsized":"2023-10-30T12:56:38.051395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'822","last_scrub_stamp":"2023-10-30T12:56:38.051202+0000","last_deep_scrub":"55'822","last_deep_scrub_stamp":"2023-10-30T12:56:38.051202+0000","last_clean_scrub_stamp":"2023-10-30T12:56:38.051202+0000","objects_scrubbed":148,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:06:45.015408+0000","scrub_duration":0.26303434999999997,"objects_trimmed":0,"snaptrim_duration":0.0080059940000000007,"stat_sum":{"num_bytes":17915594,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":64,"num_read_kb":28,"num_write":636,"num_write_kb":19744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"55'821","reported_seq":2993,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:38.940436+0000","last_change":"2023-10-30T12:56:38.940436+0000","last_active":"2023-10-30T12:56:38.940436+0000","last_peered":"2023-10-30T12:56:38.940436+0000","last_clean":"2023-10-30T12:56:38.940436+0000","last_became_active":"2023-10-30T12:35:06.215355+0000","last_became_peered":"2023-10-30T12:35:06.215355+0000","last_unstale":"2023-10-30T12:56:38.940436+0000","last_undegraded":"2023-10-30T12:56:38.940436+0000","last_fullsized":"2023-10-30T12:56:38.940436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'821","last_scrub_stamp":"2023-10-30T12:56:38.940294+0000","last_deep_scrub":"55'821","last_deep_scrub_stamp":"2023-10-30T12:56:38.940294+0000","last_clean_scrub_stamp":"2023-10-30T12:56:38.940294+0000","objects_scrubbed":135,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:00:12.683787+0000","scrub_duration":0.18969223099999999,"objects_trimmed":0,"snaptrim_duration":0.011778769,"stat_sum":{"num_bytes":10475791,"num_objects":135,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":67,"num_read_kb":8209,"num_write":651,"num_write_kb":25832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"55'780","reported_seq":2951,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:27.866741+0000","last_change":"2023-10-30T12:56:27.866741+0000","last_active":"2023-10-30T12:56:27.866741+0000","last_peered":"2023-10-30T12:56:27.866741+0000","last_clean":"2023-10-30T12:56:27.866741+0000","last_became_active":"2023-10-30T12:35:06.196503+0000","last_became_peered":"2023-10-30T12:35:06.196503+0000","last_unstale":"2023-10-30T12:56:27.866741+0000","last_undegraded":"2023-10-30T12:56:27.866741+0000","last_fullsized":"2023-10-30T12:56:27.866741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'780","last_scrub_stamp":"2023-10-30T12:56:27.866427+0000","last_deep_scrub":"55'780","last_deep_scrub_stamp":"2023-10-30T12:56:27.866427+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.866427+0000","objects_scrubbed":163,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:24:58.673270+0000","scrub_duration":0.27317086899999998,"objects_trimmed":0,"snaptrim_duration":0.0030385149999999999,"stat_sum":{"num_bytes":16276064,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":65,"num_read_kb":7648,"num_write":632,"num_write_kb":36085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"55'885","reported_seq":3068,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:01.675556+0000","last_change":"2023-10-30T12:57:01.675608+0000","last_active":"2023-10-30T12:57:01.675556+0000","last_peered":"2023-10-30T12:57:01.675556+0000","last_clean":"2023-10-30T12:57:01.675556+0000","last_became_active":"2023-10-30T12:35:06.195408+0000","last_became_peered":"2023-10-30T12:35:06.195408+0000","last_unstale":"2023-10-30T12:57:01.675556+0000","last_undegraded":"2023-10-30T12:57:01.675556+0000","last_fullsized":"2023-10-30T12:57:01.675556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'885","last_scrub_stamp":"2023-10-30T12:57:01.675469+0000","last_deep_scrub":"55'885","last_deep_scrub_stamp":"2023-10-30T12:57:01.675469+0000","last_clean_scrub_stamp":"2023-10-30T12:57:01.675469+0000","objects_scrubbed":161,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:01:54.627925+0000","scrub_duration":0.27230626800000002,"objects_trimmed":0,"snaptrim_duration":0.004560812,"stat_sum":{"num_bytes":30579039,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":77,"num_read_kb":9808,"num_write":665,"num_write_kb":49630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"55'828","reported_seq":3005,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:41.613067+0000","last_change":"2023-10-30T12:56:41.613105+0000","last_active":"2023-10-30T12:56:41.613067+0000","last_peered":"2023-10-30T12:56:41.613067+0000","last_clean":"2023-10-30T12:56:41.613067+0000","last_became_active":"2023-10-30T12:35:06.182573+0000","last_became_peered":"2023-10-30T12:35:06.182573+0000","last_unstale":"2023-10-30T12:56:41.613067+0000","last_undegraded":"2023-10-30T12:56:41.613067+0000","last_fullsized":"2023-10-30T12:56:41.613067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'828","last_scrub_stamp":"2023-10-30T12:56:41.612983+0000","last_deep_scrub":"55'828","last_deep_scrub_stamp":"2023-10-30T12:56:41.612983+0000","last_clean_scrub_stamp":"2023-10-30T12:56:41.612983+0000","objects_scrubbed":150,"log_size":828,"log_dups_size":0,"ondisk_log_size":828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:01:59.566492+0000","scrub_duration":0.154696469,"objects_trimmed":0,"snaptrim_duration":0.003020062,"stat_sum":{"num_bytes":14557524,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":71,"num_read_kb":4113,"num_write":642,"num_write_kb":31601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"55'865","reported_seq":3034,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:42.698759+0000","last_change":"2023-10-30T12:56:42.698825+0000","last_active":"2023-10-30T12:56:42.698759+0000","last_peered":"2023-10-30T12:56:42.698759+0000","last_clean":"2023-10-30T12:56:42.698759+0000","last_became_active":"2023-10-30T12:35:06.185290+0000","last_became_peered":"2023-10-30T12:35:06.185290+0000","last_unstale":"2023-10-30T12:56:42.698759+0000","last_undegraded":"2023-10-30T12:56:42.698759+0000","last_fullsized":"2023-10-30T12:56:42.698759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'865","last_scrub_stamp":"2023-10-30T12:56:42.698683+0000","last_deep_scrub":"55'865","last_deep_scrub_stamp":"2023-10-30T12:56:42.698683+0000","last_clean_scrub_stamp":"2023-10-30T12:56:42.698683+0000","objects_scrubbed":171,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:56.915839+0000","scrub_duration":0.293370403,"objects_trimmed":0,"snaptrim_duration":0.002282292,"stat_sum":{"num_bytes":14763187,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":63,"num_read_kb":11730,"num_write":643,"num_write_kb":27692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"55'877","reported_seq":3042,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:01.636658+0000","last_change":"2023-10-30T12:57:01.636781+0000","last_active":"2023-10-30T12:57:01.636658+0000","last_peered":"2023-10-30T12:57:01.636658+0000","last_clean":"2023-10-30T12:57:01.636658+0000","last_became_active":"2023-10-30T12:35:06.213627+0000","last_became_peered":"2023-10-30T12:35:06.213627+0000","last_unstale":"2023-10-30T12:57:01.636658+0000","last_undegraded":"2023-10-30T12:57:01.636658+0000","last_fullsized":"2023-10-30T12:57:01.636658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'877","last_scrub_stamp":"2023-10-30T12:57:01.636491+0000","last_deep_scrub":"55'877","last_deep_scrub_stamp":"2023-10-30T12:57:01.636491+0000","last_clean_scrub_stamp":"2023-10-30T12:57:01.636491+0000","objects_scrubbed":159,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:42:41.484120+0000","scrub_duration":0.27376288700000001,"objects_trimmed":0,"snaptrim_duration":0.014095760000000001,"stat_sum":{"num_bytes":9839966,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":59,"num_read_kb":4114,"num_write":651,"num_write_kb":31705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"55'914","reported_seq":3083,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:45.835829+0000","last_change":"2023-10-30T12:56:45.835874+0000","last_active":"2023-10-30T12:56:45.835829+0000","last_peered":"2023-10-30T12:56:45.835829+0000","last_clean":"2023-10-30T12:56:45.835829+0000","last_became_active":"2023-10-30T12:35:06.221696+0000","last_became_peered":"2023-10-30T12:35:06.221696+0000","last_unstale":"2023-10-30T12:56:45.835829+0000","last_undegraded":"2023-10-30T12:56:45.835829+0000","last_fullsized":"2023-10-30T12:56:45.835829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'914","last_scrub_stamp":"2023-10-30T12:56:45.835737+0000","last_deep_scrub":"55'914","last_deep_scrub_stamp":"2023-10-30T12:56:45.835737+0000","last_clean_scrub_stamp":"2023-10-30T12:56:45.835737+0000","objects_scrubbed":175,"log_size":914,"log_dups_size":0,"ondisk_log_size":914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:37:21.337050+0000","scrub_duration":0.41149325399999997,"objects_trimmed":0,"snaptrim_duration":0.015743672,"stat_sum":{"num_bytes":28343169,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":62,"num_read_kb":16401,"num_write":691,"num_write_kb":59410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"55'804","reported_seq":2992,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.435565+0000","last_change":"2023-10-30T12:56:24.435609+0000","last_active":"2023-10-30T12:56:24.435565+0000","last_peered":"2023-10-30T12:56:24.435565+0000","last_clean":"2023-10-30T12:56:24.435565+0000","last_became_active":"2023-10-30T12:35:06.198074+0000","last_became_peered":"2023-10-30T12:35:06.198074+0000","last_unstale":"2023-10-30T12:56:24.435565+0000","last_undegraded":"2023-10-30T12:56:24.435565+0000","last_fullsized":"2023-10-30T12:56:24.435565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'804","last_scrub_stamp":"2023-10-30T12:56:24.435488+0000","last_deep_scrub":"55'804","last_deep_scrub_stamp":"2023-10-30T12:56:24.435488+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.435488+0000","objects_scrubbed":156,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:44:11.442590+0000","scrub_duration":0.24729864700000001,"objects_trimmed":0,"snaptrim_duration":0.0023389270000000002,"stat_sum":{"num_bytes":28976684,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":82,"num_read_kb":27,"num_write":642,"num_write_kb":39937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"55'872","reported_seq":3038,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.322037+0000","last_change":"2023-10-30T12:57:12.322160+0000","last_active":"2023-10-30T12:57:12.322037+0000","last_peered":"2023-10-30T12:57:12.322037+0000","last_clean":"2023-10-30T12:57:12.322037+0000","last_became_active":"2023-10-30T12:35:06.215935+0000","last_became_peered":"2023-10-30T12:35:06.215935+0000","last_unstale":"2023-10-30T12:57:12.322037+0000","last_undegraded":"2023-10-30T12:57:12.322037+0000","last_fullsized":"2023-10-30T12:57:12.322037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'872","last_scrub_stamp":"2023-10-30T12:57:12.321838+0000","last_deep_scrub":"55'872","last_deep_scrub_stamp":"2023-10-30T12:57:12.321838+0000","last_clean_scrub_stamp":"2023-10-30T12:57:12.321838+0000","objects_scrubbed":166,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:50:28.308528+0000","scrub_duration":0.34909467599999999,"objects_trimmed":0,"snaptrim_duration":0.0062537060000000004,"stat_sum":{"num_bytes":21535467,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":60,"num_read_kb":12305,"num_write":656,"num_write_kb":39478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"55'805","reported_seq":2985,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:30.019053+0000","last_change":"2023-10-30T12:56:30.019160+0000","last_active":"2023-10-30T12:56:30.019053+0000","last_peered":"2023-10-30T12:56:30.019053+0000","last_clean":"2023-10-30T12:56:30.019053+0000","last_became_active":"2023-10-30T12:35:06.199812+0000","last_became_peered":"2023-10-30T12:35:06.199812+0000","last_unstale":"2023-10-30T12:56:30.019053+0000","last_undegraded":"2023-10-30T12:56:30.019053+0000","last_fullsized":"2023-10-30T12:56:30.019053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'805","last_scrub_stamp":"2023-10-30T12:56:30.018936+0000","last_deep_scrub":"55'805","last_deep_scrub_stamp":"2023-10-30T12:56:30.018936+0000","last_clean_scrub_stamp":"2023-10-30T12:56:30.018936+0000","objects_scrubbed":143,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:47:14.318405+0000","scrub_duration":0.21736029700000001,"objects_trimmed":0,"snaptrim_duration":0.011375481999999999,"stat_sum":{"num_bytes":6303882,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":75,"num_read_kb":20150,"num_write":647,"num_write_kb":36292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"55'899","reported_seq":3084,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:50.693229+0000","last_change":"2023-10-30T12:56:50.693274+0000","last_active":"2023-10-30T12:56:50.693229+0000","last_peered":"2023-10-30T12:56:50.693229+0000","last_clean":"2023-10-30T12:56:50.693229+0000","last_became_active":"2023-10-30T12:35:06.216861+0000","last_became_peered":"2023-10-30T12:35:06.216861+0000","last_unstale":"2023-10-30T12:56:50.693229+0000","last_undegraded":"2023-10-30T12:56:50.693229+0000","last_fullsized":"2023-10-30T12:56:50.693229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'899","last_scrub_stamp":"2023-10-30T12:56:50.693152+0000","last_deep_scrub":"55'899","last_deep_scrub_stamp":"2023-10-30T12:56:50.693152+0000","last_clean_scrub_stamp":"2023-10-30T12:56:50.693152+0000","objects_scrubbed":180,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:56:05.148326+0000","scrub_duration":0.23889443799999999,"objects_trimmed":0,"snaptrim_duration":0.0040841849999999997,"stat_sum":{"num_bytes":12370338,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":78,"num_read_kb":8213,"num_write":666,"num_write_kb":26377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"55'868","reported_seq":3032,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:33.615651+0000","last_change":"2023-10-30T12:56:33.615740+0000","last_active":"2023-10-30T12:56:33.615651+0000","last_peered":"2023-10-30T12:56:33.615651+0000","last_clean":"2023-10-30T12:56:33.615651+0000","last_became_active":"2023-10-30T12:35:46.122517+0000","last_became_peered":"2023-10-30T12:35:46.122517+0000","last_unstale":"2023-10-30T12:56:33.615651+0000","last_undegraded":"2023-10-30T12:56:33.615651+0000","last_fullsized":"2023-10-30T12:56:33.615651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'868","last_scrub_stamp":"2023-10-30T12:56:33.615559+0000","last_deep_scrub":"55'868","last_deep_scrub_stamp":"2023-10-30T12:56:33.615559+0000","last_clean_scrub_stamp":"2023-10-30T12:56:33.615559+0000","objects_scrubbed":170,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:51:10.318228+0000","scrub_duration":0.26875042100000002,"objects_trimmed":0,"snaptrim_duration":0.0020435240000000001,"stat_sum":{"num_bytes":19531616,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":61,"num_read_kb":21,"num_write":688,"num_write_kb":30111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"55'848","reported_seq":3035,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:42.567324+0000","last_change":"2023-10-30T12:56:42.567417+0000","last_active":"2023-10-30T12:56:42.567324+0000","last_peered":"2023-10-30T12:56:42.567324+0000","last_clean":"2023-10-30T12:56:42.567324+0000","last_became_active":"2023-10-30T12:35:06.182314+0000","last_became_peered":"2023-10-30T12:35:06.182314+0000","last_unstale":"2023-10-30T12:56:42.567324+0000","last_undegraded":"2023-10-30T12:56:42.567324+0000","last_fullsized":"2023-10-30T12:56:42.567324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'848","last_scrub_stamp":"2023-10-30T12:56:42.567236+0000","last_deep_scrub":"55'848","last_deep_scrub_stamp":"2023-10-30T12:56:42.567236+0000","last_clean_scrub_stamp":"2023-10-30T12:56:42.567236+0000","objects_scrubbed":161,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:44:52.946140+0000","scrub_duration":0.27460055700000002,"objects_trimmed":0,"snaptrim_duration":0.0051111200000000002,"stat_sum":{"num_bytes":5467082,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":80,"num_read_kb":15696,"num_write":658,"num_write_kb":25502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"55'829","reported_seq":3003,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:32.815869+0000","last_change":"2023-10-30T12:56:32.815869+0000","last_active":"2023-10-30T12:56:32.815869+0000","last_peered":"2023-10-30T12:56:32.815869+0000","last_clean":"2023-10-30T12:56:32.815869+0000","last_became_active":"2023-10-30T12:35:06.213402+0000","last_became_peered":"2023-10-30T12:35:06.213402+0000","last_unstale":"2023-10-30T12:56:32.815869+0000","last_undegraded":"2023-10-30T12:56:32.815869+0000","last_fullsized":"2023-10-30T12:56:32.815869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'829","last_scrub_stamp":"2023-10-30T12:56:32.815282+0000","last_deep_scrub":"55'829","last_deep_scrub_stamp":"2023-10-30T12:56:32.815282+0000","last_clean_scrub_stamp":"2023-10-30T12:56:32.815282+0000","objects_scrubbed":159,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:42:54.304491+0000","scrub_duration":0.307345284,"objects_trimmed":0,"snaptrim_duration":0.014055767,"stat_sum":{"num_bytes":10241279,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":67,"num_read_kb":4112,"num_write":657,"num_write_kb":20530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"55'838","reported_seq":3003,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:57.716386+0000","last_change":"2023-10-30T12:56:57.716444+0000","last_active":"2023-10-30T12:56:57.716386+0000","last_peered":"2023-10-30T12:56:57.716386+0000","last_clean":"2023-10-30T12:56:57.716386+0000","last_became_active":"2023-10-30T12:35:06.188298+0000","last_became_peered":"2023-10-30T12:35:06.188298+0000","last_unstale":"2023-10-30T12:56:57.716386+0000","last_undegraded":"2023-10-30T12:56:57.716386+0000","last_fullsized":"2023-10-30T12:56:57.716386+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'838","last_scrub_stamp":"2023-10-30T12:56:57.716307+0000","last_deep_scrub":"55'838","last_deep_scrub_stamp":"2023-10-30T12:56:57.716307+0000","last_clean_scrub_stamp":"2023-10-30T12:56:57.716307+0000","objects_scrubbed":149,"log_size":838,"log_dups_size":0,"ondisk_log_size":838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:19:18.533282+0000","scrub_duration":0.22493126999999999,"objects_trimmed":0,"snaptrim_duration":0.0053580889999999999,"stat_sum":{"num_bytes":5087264,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":59,"num_read_kb":8204,"num_write":621,"num_write_kb":44181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"55'896","reported_seq":3090,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.080071+0000","last_change":"2023-10-30T12:56:34.080071+0000","last_active":"2023-10-30T12:56:34.080071+0000","last_peered":"2023-10-30T12:56:34.080071+0000","last_clean":"2023-10-30T12:56:34.080071+0000","last_became_active":"2023-10-30T12:35:06.202507+0000","last_became_peered":"2023-10-30T12:35:06.202507+0000","last_unstale":"2023-10-30T12:56:34.080071+0000","last_undegraded":"2023-10-30T12:56:34.080071+0000","last_fullsized":"2023-10-30T12:56:34.080071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'896","last_scrub_stamp":"2023-10-30T12:56:34.079910+0000","last_deep_scrub":"55'896","last_deep_scrub_stamp":"2023-10-30T12:56:34.079910+0000","last_clean_scrub_stamp":"2023-10-30T12:56:34.079910+0000","objects_scrubbed":173,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:15:45.705065+0000","scrub_duration":0.25185000000000002,"objects_trimmed":0,"snaptrim_duration":0.0082500649999999991,"stat_sum":{"num_bytes":6587197,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":87,"num_read_kb":1892,"num_write":715,"num_write_kb":28794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"55'861","reported_seq":3031,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:46.673561+0000","last_change":"2023-10-30T12:56:46.673623+0000","last_active":"2023-10-30T12:56:46.673561+0000","last_peered":"2023-10-30T12:56:46.673561+0000","last_clean":"2023-10-30T12:56:46.673561+0000","last_became_active":"2023-10-30T12:35:06.170030+0000","last_became_peered":"2023-10-30T12:35:06.170030+0000","last_unstale":"2023-10-30T12:56:46.673561+0000","last_undegraded":"2023-10-30T12:56:46.673561+0000","last_fullsized":"2023-10-30T12:56:46.673561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'861","last_scrub_stamp":"2023-10-30T12:56:46.673491+0000","last_deep_scrub":"55'861","last_deep_scrub_stamp":"2023-10-30T12:56:46.673491+0000","last_clean_scrub_stamp":"2023-10-30T12:56:46.673491+0000","objects_scrubbed":166,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:55:04.341832+0000","scrub_duration":0.18274528900000001,"objects_trimmed":0,"snaptrim_duration":0.0024132049999999999,"stat_sum":{"num_bytes":15296106,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":63,"num_read_kb":15,"num_write":636,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"55'823","reported_seq":3009,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.671124+0000","last_change":"2023-10-30T12:57:07.671175+0000","last_active":"2023-10-30T12:57:07.671124+0000","last_peered":"2023-10-30T12:57:07.671124+0000","last_clean":"2023-10-30T12:57:07.671124+0000","last_became_active":"2023-10-30T12:35:06.189171+0000","last_became_peered":"2023-10-30T12:35:06.189171+0000","last_unstale":"2023-10-30T12:57:07.671124+0000","last_undegraded":"2023-10-30T12:57:07.671124+0000","last_fullsized":"2023-10-30T12:57:07.671124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'823","last_scrub_stamp":"2023-10-30T12:57:07.671050+0000","last_deep_scrub":"55'823","last_deep_scrub_stamp":"2023-10-30T12:57:07.671050+0000","last_clean_scrub_stamp":"2023-10-30T12:57:07.671050+0000","objects_scrubbed":171,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:41:38.114367+0000","scrub_duration":0.27815337899999998,"objects_trimmed":0,"snaptrim_duration":0.0056904870000000001,"stat_sum":{"num_bytes":14055782,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":78,"num_read_kb":12372,"num_write":607,"num_write_kb":31259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"55'824","reported_seq":2992,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:47.631354+0000","last_change":"2023-10-30T12:56:47.631397+0000","last_active":"2023-10-30T12:56:47.631354+0000","last_peered":"2023-10-30T12:56:47.631354+0000","last_clean":"2023-10-30T12:56:47.631354+0000","last_became_active":"2023-10-30T12:35:06.183348+0000","last_became_peered":"2023-10-30T12:35:06.183348+0000","last_unstale":"2023-10-30T12:56:47.631354+0000","last_undegraded":"2023-10-30T12:56:47.631354+0000","last_fullsized":"2023-10-30T12:56:47.631354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'824","last_scrub_stamp":"2023-10-30T12:56:47.631281+0000","last_deep_scrub":"55'824","last_deep_scrub_stamp":"2023-10-30T12:56:47.631281+0000","last_clean_scrub_stamp":"2023-10-30T12:56:47.631281+0000","objects_scrubbed":152,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:43:06.067352+0000","scrub_duration":0.17130867799999999,"objects_trimmed":0,"snaptrim_duration":0.0041317979999999999,"stat_sum":{"num_bytes":14615922,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":60,"num_read_kb":7678,"num_write":625,"num_write_kb":36364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"55'882","reported_seq":3063,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:39.686942+0000","last_change":"2023-10-30T12:56:39.687006+0000","last_active":"2023-10-30T12:56:39.686942+0000","last_peered":"2023-10-30T12:56:39.686942+0000","last_clean":"2023-10-30T12:56:39.686942+0000","last_became_active":"2023-10-30T12:35:06.196368+0000","last_became_peered":"2023-10-30T12:35:06.196368+0000","last_unstale":"2023-10-30T12:56:39.686942+0000","last_undegraded":"2023-10-30T12:56:39.686942+0000","last_fullsized":"2023-10-30T12:56:39.686942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'882","last_scrub_stamp":"2023-10-30T12:56:39.686847+0000","last_deep_scrub":"55'882","last_deep_scrub_stamp":"2023-10-30T12:56:39.686847+0000","last_clean_scrub_stamp":"2023-10-30T12:56:39.686847+0000","objects_scrubbed":148,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:44:17.768223+0000","scrub_duration":0.25242409199999999,"objects_trimmed":0,"snaptrim_duration":0.015569382999999999,"stat_sum":{"num_bytes":14491906,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":75,"num_read_kb":9352,"num_write":694,"num_write_kb":32749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"55'888","reported_seq":3068,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:25.513128+0000","last_change":"2023-10-30T12:56:25.513186+0000","last_active":"2023-10-30T12:56:25.513128+0000","last_peered":"2023-10-30T12:56:25.513128+0000","last_clean":"2023-10-30T12:56:25.513128+0000","last_became_active":"2023-10-30T12:35:06.213994+0000","last_became_peered":"2023-10-30T12:35:06.213994+0000","last_unstale":"2023-10-30T12:56:25.513128+0000","last_undegraded":"2023-10-30T12:56:25.513128+0000","last_fullsized":"2023-10-30T12:56:25.513128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'888","last_scrub_stamp":"2023-10-30T12:56:25.513052+0000","last_deep_scrub":"55'888","last_deep_scrub_stamp":"2023-10-30T12:56:25.513052+0000","last_clean_scrub_stamp":"2023-10-30T12:56:25.513052+0000","objects_scrubbed":188,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:49:47.291732+0000","scrub_duration":0.28763551900000001,"objects_trimmed":0,"snaptrim_duration":0.001873502,"stat_sum":{"num_bytes":23363209,"num_objects":188,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":188,"num_whiteouts":0,"num_read":72,"num_read_kb":4118,"num_write":712,"num_write_kb":32388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"55'855","reported_seq":3032,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:42.099804+0000","last_change":"2023-10-30T12:56:42.099940+0000","last_active":"2023-10-30T12:56:42.099804+0000","last_peered":"2023-10-30T12:56:42.099804+0000","last_clean":"2023-10-30T12:56:42.099804+0000","last_became_active":"2023-10-30T12:35:06.197647+0000","last_became_peered":"2023-10-30T12:35:06.197647+0000","last_unstale":"2023-10-30T12:56:42.099804+0000","last_undegraded":"2023-10-30T12:56:42.099804+0000","last_fullsized":"2023-10-30T12:56:42.099804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'855","last_scrub_stamp":"2023-10-30T12:56:42.099593+0000","last_deep_scrub":"55'855","last_deep_scrub_stamp":"2023-10-30T12:56:42.099593+0000","last_clean_scrub_stamp":"2023-10-30T12:56:42.099593+0000","objects_scrubbed":167,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:51:11.019062+0000","scrub_duration":0.343353621,"objects_trimmed":0,"snaptrim_duration":0.011378932,"stat_sum":{"num_bytes":18364745,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":70,"num_read_kb":8212,"num_write":651,"num_write_kb":28257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"55'773","reported_seq":2939,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:21.810651+0000","last_change":"2023-10-30T12:56:21.810710+0000","last_active":"2023-10-30T12:56:21.810651+0000","last_peered":"2023-10-30T12:56:21.810651+0000","last_clean":"2023-10-30T12:56:21.810651+0000","last_became_active":"2023-10-30T12:35:06.167558+0000","last_became_peered":"2023-10-30T12:35:06.167558+0000","last_unstale":"2023-10-30T12:56:21.810651+0000","last_undegraded":"2023-10-30T12:56:21.810651+0000","last_fullsized":"2023-10-30T12:56:21.810651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'773","last_scrub_stamp":"2023-10-30T12:56:21.810556+0000","last_deep_scrub":"55'773","last_deep_scrub_stamp":"2023-10-30T12:56:21.810556+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.810556+0000","objects_scrubbed":143,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:13:19.546302+0000","scrub_duration":0.23224787799999999,"objects_trimmed":0,"snaptrim_duration":0.0048907120000000002,"stat_sum":{"num_bytes":9710360,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":62,"num_read_kb":17,"num_write":609,"num_write_kb":15663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"55'239","reported_seq":313,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:27.077441+0000","last_change":"2023-10-30T12:56:27.077499+0000","last_active":"2023-10-30T12:56:27.077441+0000","last_peered":"2023-10-30T12:56:27.077441+0000","last_clean":"2023-10-30T12:56:27.077441+0000","last_became_active":"2023-10-30T12:35:05.200169+0000","last_became_peered":"2023-10-30T12:35:05.200169+0000","last_unstale":"2023-10-30T12:56:27.077441+0000","last_undegraded":"2023-10-30T12:56:27.077441+0000","last_fullsized":"2023-10-30T12:56:27.077441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'239","last_scrub_stamp":"2023-10-30T12:56:27.077343+0000","last_deep_scrub":"55'239","last_deep_scrub_stamp":"2023-10-30T12:56:27.077343+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.077343+0000","objects_scrubbed":6,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:21:31.745900+0000","scrub_duration":0.044083678000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":36867,"num_write":183,"num_write_kb":12445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16490,"num_omap_keys":31,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"55'812","reported_seq":2977,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:38.697479+0000","last_change":"2023-10-30T12:56:38.697551+0000","last_active":"2023-10-30T12:56:38.697479+0000","last_peered":"2023-10-30T12:56:38.697479+0000","last_clean":"2023-10-30T12:56:38.697479+0000","last_became_active":"2023-10-30T12:35:06.194689+0000","last_became_peered":"2023-10-30T12:35:06.194689+0000","last_unstale":"2023-10-30T12:56:38.697479+0000","last_undegraded":"2023-10-30T12:56:38.697479+0000","last_fullsized":"2023-10-30T12:56:38.697479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'812","last_scrub_stamp":"2023-10-30T12:56:38.697404+0000","last_deep_scrub":"55'812","last_deep_scrub_stamp":"2023-10-30T12:56:38.697404+0000","last_clean_scrub_stamp":"2023-10-30T12:56:38.697404+0000","objects_scrubbed":144,"log_size":812,"log_dups_size":0,"ondisk_log_size":812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:44:05.310987+0000","scrub_duration":0.23620231899999999,"objects_trimmed":0,"snaptrim_duration":0.004291034,"stat_sum":{"num_bytes":10922380,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":61,"num_read_kb":18,"num_write":626,"num_write_kb":24711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"55'983","reported_seq":1183,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.860910+0000","last_change":"2023-10-30T12:56:24.861007+0000","last_active":"2023-10-30T12:56:24.860910+0000","last_peered":"2023-10-30T12:56:24.860910+0000","last_clean":"2023-10-30T12:56:24.860910+0000","last_became_active":"2023-10-30T12:35:05.182761+0000","last_became_peered":"2023-10-30T12:35:05.182761+0000","last_unstale":"2023-10-30T12:56:24.860910+0000","last_undegraded":"2023-10-30T12:56:24.860910+0000","last_fullsized":"2023-10-30T12:56:24.860910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'983","last_scrub_stamp":"2023-10-30T12:56:24.860748+0000","last_deep_scrub":"55'983","last_deep_scrub_stamp":"2023-10-30T12:56:24.860748+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.860748+0000","objects_scrubbed":8,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:51:38.567709+0000","scrub_duration":0.11984236299999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25625406,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":150,"num_read_kb":45229,"num_write":895,"num_write_kb":25503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"55'458","reported_seq":673,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:21.989512+0000","last_change":"2023-10-30T12:56:21.989512+0000","last_active":"2023-10-30T12:56:21.989512+0000","last_peered":"2023-10-30T12:56:21.989512+0000","last_clean":"2023-10-30T12:56:21.989512+0000","last_became_active":"2023-10-30T12:35:05.180730+0000","last_became_peered":"2023-10-30T12:35:05.180730+0000","last_unstale":"2023-10-30T12:56:21.989512+0000","last_undegraded":"2023-10-30T12:56:21.989512+0000","last_fullsized":"2023-10-30T12:56:21.989512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'458","last_scrub_stamp":"2023-10-30T12:56:21.989395+0000","last_deep_scrub":"55'458","last_deep_scrub_stamp":"2023-10-30T12:56:21.989395+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.989395+0000","objects_scrubbed":4,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:10:50.980457+0000","scrub_duration":0.030827730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8855749,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":162,"num_read_kb":8759,"num_write":390,"num_write_kb":8900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"55'906","reported_seq":3091,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:29.424480+0000","last_change":"2023-10-30T12:56:29.424522+0000","last_active":"2023-10-30T12:56:29.424480+0000","last_peered":"2023-10-30T12:56:29.424480+0000","last_clean":"2023-10-30T12:56:29.424480+0000","last_became_active":"2023-10-30T12:35:46.051057+0000","last_became_peered":"2023-10-30T12:35:46.051057+0000","last_unstale":"2023-10-30T12:56:29.424480+0000","last_undegraded":"2023-10-30T12:56:29.424480+0000","last_fullsized":"2023-10-30T12:56:29.424480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'906","last_scrub_stamp":"2023-10-30T12:56:29.424403+0000","last_deep_scrub":"55'906","last_deep_scrub_stamp":"2023-10-30T12:56:29.424403+0000","last_clean_scrub_stamp":"2023-10-30T12:56:29.424403+0000","objects_scrubbed":175,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:57:06.583434+0000","scrub_duration":0.16198085700000001,"objects_trimmed":0,"snaptrim_duration":0.0018351520000000001,"stat_sum":{"num_bytes":19060908,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":82,"num_read_kb":16408,"num_write":727,"num_write_kb":53009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"55'988","reported_seq":1064,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:30.499309+0000","last_change":"2023-10-30T12:56:30.499309+0000","last_active":"2023-10-30T12:56:30.499309+0000","last_peered":"2023-10-30T12:56:30.499309+0000","last_clean":"2023-10-30T12:56:30.499309+0000","last_became_active":"2023-10-30T12:35:05.201853+0000","last_became_peered":"2023-10-30T12:35:05.201853+0000","last_unstale":"2023-10-30T12:56:30.499309+0000","last_undegraded":"2023-10-30T12:56:30.499309+0000","last_fullsized":"2023-10-30T12:56:30.499309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'988","last_scrub_stamp":"2023-10-30T12:56:30.498957+0000","last_deep_scrub":"55'988","last_deep_scrub_stamp":"2023-10-30T12:56:30.498957+0000","last_clean_scrub_stamp":"2023-10-30T12:56:30.498957+0000","objects_scrubbed":5,"log_size":988,"log_dups_size":0,"ondisk_log_size":988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:34.744919+0000","scrub_duration":0.073408293999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":38325,"num_write":914,"num_write_kb":16753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"55'900","reported_seq":3080,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:39.676436+0000","last_change":"2023-10-30T12:56:39.676436+0000","last_active":"2023-10-30T12:56:39.676436+0000","last_peered":"2023-10-30T12:56:39.676436+0000","last_clean":"2023-10-30T12:56:39.676436+0000","last_became_active":"2023-10-30T12:35:06.170179+0000","last_became_peered":"2023-10-30T12:35:06.170179+0000","last_unstale":"2023-10-30T12:56:39.676436+0000","last_undegraded":"2023-10-30T12:56:39.676436+0000","last_fullsized":"2023-10-30T12:56:39.676436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'900","last_scrub_stamp":"2023-10-30T12:56:39.676328+0000","last_deep_scrub":"55'900","last_deep_scrub_stamp":"2023-10-30T12:56:39.676328+0000","last_clean_scrub_stamp":"2023-10-30T12:56:39.676328+0000","objects_scrubbed":175,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:24:31.809070+0000","scrub_duration":0.198132067,"objects_trimmed":0,"snaptrim_duration":0.0021794980000000002,"stat_sum":{"num_bytes":14292082,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":73,"num_read_kb":12307,"num_write":687,"num_write_kb":28727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"55'152","reported_seq":221,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:52.550152+0000","last_change":"2023-10-30T12:56:27.831941+0000","last_active":"2023-10-30T12:56:52.550152+0000","last_peered":"2023-10-30T12:56:52.550152+0000","last_clean":"2023-10-30T12:56:52.550152+0000","last_became_active":"2023-10-30T12:35:05.197606+0000","last_became_peered":"2023-10-30T12:35:05.197606+0000","last_unstale":"2023-10-30T12:56:52.550152+0000","last_undegraded":"2023-10-30T12:56:52.550152+0000","last_fullsized":"2023-10-30T12:56:52.550152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'149","last_scrub_stamp":"2023-10-30T12:56:27.831485+0000","last_deep_scrub":"55'149","last_deep_scrub_stamp":"2023-10-30T12:56:27.831485+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.831485+0000","objects_scrubbed":5,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:57:34.144550+0000","scrub_duration":0.012229985000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":93,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52416,"num_omap_keys":96,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"55'814","reported_seq":2997,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.786207+0000","last_change":"2023-10-30T12:56:34.786207+0000","last_active":"2023-10-30T12:56:34.786207+0000","last_peered":"2023-10-30T12:56:34.786207+0000","last_clean":"2023-10-30T12:56:34.786207+0000","last_became_active":"2023-10-30T12:35:06.225392+0000","last_became_peered":"2023-10-30T12:35:06.225392+0000","last_unstale":"2023-10-30T12:56:34.786207+0000","last_undegraded":"2023-10-30T12:56:34.786207+0000","last_fullsized":"2023-10-30T12:56:34.786207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'814","last_scrub_stamp":"2023-10-30T12:56:34.786044+0000","last_deep_scrub":"55'814","last_deep_scrub_stamp":"2023-10-30T12:56:34.786044+0000","last_clean_scrub_stamp":"2023-10-30T12:56:34.786044+0000","objects_scrubbed":159,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:26:14.139459+0000","scrub_duration":0.24530717799999999,"objects_trimmed":0,"snaptrim_duration":0.01552166,"stat_sum":{"num_bytes":9754816,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":77,"num_read_kb":18,"num_write":651,"num_write_kb":19117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"55'600","reported_seq":671,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:59.690048+0000","last_change":"2023-10-30T12:56:23.447764+0000","last_active":"2023-10-30T12:56:59.690048+0000","last_peered":"2023-10-30T12:56:59.690048+0000","last_clean":"2023-10-30T12:56:59.690048+0000","last_became_active":"2023-10-30T12:35:05.196899+0000","last_became_peered":"2023-10-30T12:35:05.196899+0000","last_unstale":"2023-10-30T12:56:59.690048+0000","last_undegraded":"2023-10-30T12:56:59.690048+0000","last_fullsized":"2023-10-30T12:56:59.690048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'579","last_scrub_stamp":"2023-10-30T12:56:23.447316+0000","last_deep_scrub":"55'579","last_deep_scrub_stamp":"2023-10-30T12:56:23.447316+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.447316+0000","objects_scrubbed":6,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:59:20.096502+0000","scrub_duration":0.067786201000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18436527,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":14173,"num_write":529,"num_write_kb":18295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"55'865","reported_seq":3034,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:45.787426+0000","last_change":"2023-10-30T12:56:45.787426+0000","last_active":"2023-10-30T12:56:45.787426+0000","last_peered":"2023-10-30T12:56:45.787426+0000","last_clean":"2023-10-30T12:56:45.787426+0000","last_became_active":"2023-10-30T12:35:06.222057+0000","last_became_peered":"2023-10-30T12:35:06.222057+0000","last_unstale":"2023-10-30T12:56:45.787426+0000","last_undegraded":"2023-10-30T12:56:45.787426+0000","last_fullsized":"2023-10-30T12:56:45.787426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'865","last_scrub_stamp":"2023-10-30T12:56:45.787252+0000","last_deep_scrub":"55'865","last_deep_scrub_stamp":"2023-10-30T12:56:45.787252+0000","last_clean_scrub_stamp":"2023-10-30T12:56:45.787252+0000","objects_scrubbed":159,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:23:24.865367+0000","scrub_duration":0.40644253200000002,"objects_trimmed":0,"snaptrim_duration":0.004210915,"stat_sum":{"num_bytes":31400359,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":63,"num_read_kb":12308,"num_write":657,"num_write_kb":50495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"55'317","reported_seq":390,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:18.453440+0000","last_change":"2023-10-30T12:56:18.453440+0000","last_active":"2023-10-30T12:56:18.453440+0000","last_peered":"2023-10-30T12:56:18.453440+0000","last_clean":"2023-10-30T12:56:18.453440+0000","last_became_active":"2023-10-30T12:35:05.197190+0000","last_became_peered":"2023-10-30T12:35:05.197190+0000","last_unstale":"2023-10-30T12:56:18.453440+0000","last_undegraded":"2023-10-30T12:56:18.453440+0000","last_fullsized":"2023-10-30T12:56:18.453440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'317","last_scrub_stamp":"2023-10-30T12:56:18.453257+0000","last_deep_scrub":"55'317","last_deep_scrub_stamp":"2023-10-30T12:56:18.453257+0000","last_clean_scrub_stamp":"2023-10-30T12:56:18.453257+0000","objects_scrubbed":7,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:09:36.655470+0000","scrub_duration":0.034206814000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":4111,"num_write":258,"num_write_kb":8429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28930,"num_omap_keys":53,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"55'791","reported_seq":2957,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:25.811201+0000","last_change":"2023-10-30T12:56:25.811267+0000","last_active":"2023-10-30T12:56:25.811201+0000","last_peered":"2023-10-30T12:56:25.811201+0000","last_clean":"2023-10-30T12:56:25.811201+0000","last_became_active":"2023-10-30T12:35:06.204122+0000","last_became_peered":"2023-10-30T12:35:06.204122+0000","last_unstale":"2023-10-30T12:56:25.811201+0000","last_undegraded":"2023-10-30T12:56:25.811201+0000","last_fullsized":"2023-10-30T12:56:25.811201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'791","last_scrub_stamp":"2023-10-30T12:56:25.811099+0000","last_deep_scrub":"55'791","last_deep_scrub_stamp":"2023-10-30T12:56:25.811099+0000","last_clean_scrub_stamp":"2023-10-30T12:56:25.811099+0000","objects_scrubbed":158,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:21:46.064934+0000","scrub_duration":0.24490572399999999,"objects_trimmed":0,"snaptrim_duration":0.0030324050000000002,"stat_sum":{"num_bytes":15588017,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":60,"num_read_kb":8209,"num_write":632,"num_write_kb":41942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"55'370","reported_seq":451,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:39.535926+0000","last_change":"2023-10-30T12:56:25.809737+0000","last_active":"2023-10-30T12:56:39.535926+0000","last_peered":"2023-10-30T12:56:39.535926+0000","last_clean":"2023-10-30T12:56:39.535926+0000","last_became_active":"2023-10-30T12:35:05.179547+0000","last_became_peered":"2023-10-30T12:35:05.179547+0000","last_unstale":"2023-10-30T12:56:39.535926+0000","last_undegraded":"2023-10-30T12:56:39.535926+0000","last_fullsized":"2023-10-30T12:56:39.535926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'368","last_scrub_stamp":"2023-10-30T12:56:25.809432+0000","last_deep_scrub":"55'368","last_deep_scrub_stamp":"2023-10-30T12:56:25.809432+0000","last_clean_scrub_stamp":"2023-10-30T12:56:25.809432+0000","objects_scrubbed":9,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:55:48.660238+0000","scrub_duration":0.040587917000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":24600,"num_write":302,"num_write_kb":12513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17926,"num_omap_keys":33,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"55'784","reported_seq":2951,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.106255+0000","last_change":"2023-10-30T12:56:24.106255+0000","last_active":"2023-10-30T12:56:24.106255+0000","last_peered":"2023-10-30T12:56:24.106255+0000","last_clean":"2023-10-30T12:56:24.106255+0000","last_became_active":"2023-10-30T12:35:06.215128+0000","last_became_peered":"2023-10-30T12:35:06.215128+0000","last_unstale":"2023-10-30T12:56:24.106255+0000","last_undegraded":"2023-10-30T12:56:24.106255+0000","last_fullsized":"2023-10-30T12:56:24.106255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'784","last_scrub_stamp":"2023-10-30T12:56:24.105935+0000","last_deep_scrub":"55'784","last_deep_scrub_stamp":"2023-10-30T12:56:24.105935+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.105935+0000","objects_scrubbed":171,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:40:02.361124+0000","scrub_duration":0.30957912999999998,"objects_trimmed":0,"snaptrim_duration":0.0064662909999999999,"stat_sum":{"num_bytes":22774544,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":61,"num_read_kb":11,"num_write":605,"num_write_kb":27691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"55'215","reported_seq":298,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:18.560689+0000","last_change":"2023-10-30T12:56:18.560737+0000","last_active":"2023-10-30T12:56:18.560689+0000","last_peered":"2023-10-30T12:56:18.560689+0000","last_clean":"2023-10-30T12:56:18.560689+0000","last_became_active":"2023-10-30T12:35:05.189573+0000","last_became_peered":"2023-10-30T12:35:05.189573+0000","last_unstale":"2023-10-30T12:56:18.560689+0000","last_undegraded":"2023-10-30T12:56:18.560689+0000","last_fullsized":"2023-10-30T12:56:18.560689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'215","last_scrub_stamp":"2023-10-30T12:56:18.560612+0000","last_deep_scrub":"55'215","last_deep_scrub_stamp":"2023-10-30T12:56:18.560612+0000","last_clean_scrub_stamp":"2023-10-30T12:56:18.560612+0000","objects_scrubbed":7,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:40:38.364129+0000","scrub_duration":0.037472814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":24624,"num_write":154,"num_write_kb":8340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15144,"num_omap_keys":29,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"55'901","reported_seq":3085,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:40.810421+0000","last_change":"2023-10-30T12:56:40.810463+0000","last_active":"2023-10-30T12:56:40.810421+0000","last_peered":"2023-10-30T12:56:40.810421+0000","last_clean":"2023-10-30T12:56:40.810421+0000","last_became_active":"2023-10-30T12:35:06.216885+0000","last_became_peered":"2023-10-30T12:35:06.216885+0000","last_unstale":"2023-10-30T12:56:40.810421+0000","last_undegraded":"2023-10-30T12:56:40.810421+0000","last_fullsized":"2023-10-30T12:56:40.810421+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'901","last_scrub_stamp":"2023-10-30T12:56:40.810347+0000","last_deep_scrub":"55'901","last_deep_scrub_stamp":"2023-10-30T12:56:40.810347+0000","last_clean_scrub_stamp":"2023-10-30T12:56:40.810347+0000","objects_scrubbed":175,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:17:37.604756+0000","scrub_duration":0.31851338200000001,"objects_trimmed":0,"snaptrim_duration":0.0042006370000000001,"stat_sum":{"num_bytes":19251186,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":77,"num_read_kb":16402,"num_write":677,"num_write_kb":45337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"55'237","reported_seq":307,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:31.515367+0000","last_change":"2023-10-30T12:56:31.515500+0000","last_active":"2023-10-30T12:56:31.515367+0000","last_peered":"2023-10-30T12:56:31.515367+0000","last_clean":"2023-10-30T12:56:31.515367+0000","last_became_active":"2023-10-30T12:35:05.202612+0000","last_became_peered":"2023-10-30T12:35:05.202612+0000","last_unstale":"2023-10-30T12:56:31.515367+0000","last_undegraded":"2023-10-30T12:56:31.515367+0000","last_fullsized":"2023-10-30T12:56:31.515367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'237","last_scrub_stamp":"2023-10-30T12:56:31.515170+0000","last_deep_scrub":"55'237","last_deep_scrub_stamp":"2023-10-30T12:56:31.515170+0000","last_clean_scrub_stamp":"2023-10-30T12:56:31.515170+0000","objects_scrubbed":7,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:51:38.652440+0000","scrub_duration":0.042566652000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":181,"num_write_kb":8338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45273,"num_omap_keys":85,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"55'854","reported_seq":3034,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:17.119819+0000","last_change":"2023-10-30T12:57:17.119888+0000","last_active":"2023-10-30T12:57:17.119819+0000","last_peered":"2023-10-30T12:57:17.119819+0000","last_clean":"2023-10-30T12:57:17.119819+0000","last_became_active":"2023-10-30T12:35:06.215553+0000","last_became_peered":"2023-10-30T12:35:06.215553+0000","last_unstale":"2023-10-30T12:57:17.119819+0000","last_undegraded":"2023-10-30T12:57:17.119819+0000","last_fullsized":"2023-10-30T12:57:17.119819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'854","last_scrub_stamp":"2023-10-30T12:57:17.119722+0000","last_deep_scrub":"55'854","last_deep_scrub_stamp":"2023-10-30T12:57:17.119722+0000","last_clean_scrub_stamp":"2023-10-30T12:57:17.119722+0000","objects_scrubbed":144,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:50:50.978198+0000","scrub_duration":0.28060836500000003,"objects_trimmed":0,"snaptrim_duration":0.0060418609999999999,"stat_sum":{"num_bytes":19055642,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":76,"num_read_kb":33,"num_write":654,"num_write_kb":32706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"55'297","reported_seq":377,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:21.765254+0000","last_change":"2023-10-30T12:56:21.765381+0000","last_active":"2023-10-30T12:56:21.765254+0000","last_peered":"2023-10-30T12:56:21.765254+0000","last_clean":"2023-10-30T12:56:21.765254+0000","last_became_active":"2023-10-30T12:35:05.181115+0000","last_became_peered":"2023-10-30T12:35:05.181115+0000","last_unstale":"2023-10-30T12:56:21.765254+0000","last_undegraded":"2023-10-30T12:56:21.765254+0000","last_fullsized":"2023-10-30T12:56:21.765254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'297","last_scrub_stamp":"2023-10-30T12:56:21.765065+0000","last_deep_scrub":"55'297","last_deep_scrub_stamp":"2023-10-30T12:56:21.765065+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.765065+0000","objects_scrubbed":8,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:20:05.549971+0000","scrub_duration":0.074545823999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":43239,"num_write":227,"num_write_kb":16591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":22378,"num_omap_keys":41,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"55'794","reported_seq":2960,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:31.034978+0000","last_change":"2023-10-30T12:56:31.035128+0000","last_active":"2023-10-30T12:56:31.034978+0000","last_peered":"2023-10-30T12:56:31.034978+0000","last_clean":"2023-10-30T12:56:31.034978+0000","last_became_active":"2023-10-30T12:35:06.197722+0000","last_became_peered":"2023-10-30T12:35:06.197722+0000","last_unstale":"2023-10-30T12:56:31.034978+0000","last_undegraded":"2023-10-30T12:56:31.034978+0000","last_fullsized":"2023-10-30T12:56:31.034978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'794","last_scrub_stamp":"2023-10-30T12:56:31.034831+0000","last_deep_scrub":"55'794","last_deep_scrub_stamp":"2023-10-30T12:56:31.034831+0000","last_clean_scrub_stamp":"2023-10-30T12:56:31.034831+0000","objects_scrubbed":168,"log_size":794,"log_dups_size":0,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:35:46.015437+0000","scrub_duration":0.24651738600000001,"objects_trimmed":0,"snaptrim_duration":0.011777173,"stat_sum":{"num_bytes":11179157,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":60,"num_read_kb":16,"num_write":620,"num_write_kb":21517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"55'492","reported_seq":592,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:19.245439+0000","last_change":"2023-10-30T12:56:19.245517+0000","last_active":"2023-10-30T12:56:19.245439+0000","last_peered":"2023-10-30T12:56:19.245439+0000","last_clean":"2023-10-30T12:56:19.245439+0000","last_became_active":"2023-10-30T12:35:05.169130+0000","last_became_peered":"2023-10-30T12:35:05.169130+0000","last_unstale":"2023-10-30T12:56:19.245439+0000","last_undegraded":"2023-10-30T12:56:19.245439+0000","last_fullsized":"2023-10-30T12:56:19.245439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'492","last_scrub_stamp":"2023-10-30T12:56:19.245361+0000","last_deep_scrub":"55'492","last_deep_scrub_stamp":"2023-10-30T12:56:19.245361+0000","last_clean_scrub_stamp":"2023-10-30T12:56:19.245361+0000","objects_scrubbed":12,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:27:01.796605+0000","scrub_duration":0.039470346000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583024,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":18341,"num_write":415,"num_write_kb":12551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10636,"num_omap_keys":20,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"55'888","reported_seq":3085,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:00.210068+0000","last_change":"2023-10-30T12:57:00.210138+0000","last_active":"2023-10-30T12:57:00.210068+0000","last_peered":"2023-10-30T12:57:00.210068+0000","last_clean":"2023-10-30T12:57:00.210068+0000","last_became_active":"2023-10-30T12:35:06.214839+0000","last_became_peered":"2023-10-30T12:35:06.214839+0000","last_unstale":"2023-10-30T12:57:00.210068+0000","last_undegraded":"2023-10-30T12:57:00.210068+0000","last_fullsized":"2023-10-30T12:57:00.210068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'888","last_scrub_stamp":"2023-10-30T12:57:00.209983+0000","last_deep_scrub":"55'888","last_deep_scrub_stamp":"2023-10-30T12:57:00.209983+0000","last_clean_scrub_stamp":"2023-10-30T12:57:00.209983+0000","objects_scrubbed":168,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:49:53.540651+0000","scrub_duration":0.24779405400000001,"objects_trimmed":0,"snaptrim_duration":0.0049147749999999997,"stat_sum":{"num_bytes":12184897,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":91,"num_read_kb":16402,"num_write":664,"num_write_kb":29797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"55'315","reported_seq":389,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:20.671077+0000","last_change":"2023-10-30T12:56:20.671275+0000","last_active":"2023-10-30T12:56:20.671077+0000","last_peered":"2023-10-30T12:56:20.671077+0000","last_clean":"2023-10-30T12:56:20.671077+0000","last_became_active":"2023-10-30T12:35:05.188862+0000","last_became_peered":"2023-10-30T12:35:05.188862+0000","last_unstale":"2023-10-30T12:56:20.671077+0000","last_undegraded":"2023-10-30T12:56:20.671077+0000","last_fullsized":"2023-10-30T12:56:20.671077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'315","last_scrub_stamp":"2023-10-30T12:56:20.670866+0000","last_deep_scrub":"55'315","last_deep_scrub_stamp":"2023-10-30T12:56:20.670866+0000","last_clean_scrub_stamp":"2023-10-30T12:56:20.670866+0000","objects_scrubbed":4,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:07:40.146504+0000","scrub_duration":0.058071512999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":12343,"num_write":388,"num_write_kb":4750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9690,"num_omap_keys":18,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"55'673","reported_seq":2830,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.172053+0000","last_change":"2023-10-30T12:42:02.472755+0000","last_active":"2023-10-30T12:56:14.172053+0000","last_peered":"2023-10-30T12:56:14.172053+0000","last_clean":"2023-10-30T12:56:14.172053+0000","last_became_active":"2023-10-30T12:35:06.216533+0000","last_became_peered":"2023-10-30T12:35:06.216533+0000","last_unstale":"2023-10-30T12:56:14.172053+0000","last_undegraded":"2023-10-30T12:56:14.172053+0000","last_fullsized":"2023-10-30T12:56:14.172053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:35:05.142552+0000","last_clean_scrub_stamp":"2023-10-30T12:35:05.142552+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060962430000000003,"stat_sum":{"num_bytes":10771712,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":65,"num_read_kb":4109,"num_write":629,"num_write_kb":24152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"55'238","reported_seq":308,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:23.246213+0000","last_change":"2023-10-30T12:56:23.246263+0000","last_active":"2023-10-30T12:56:23.246213+0000","last_peered":"2023-10-30T12:56:23.246213+0000","last_clean":"2023-10-30T12:56:23.246213+0000","last_became_active":"2023-10-30T12:35:05.177316+0000","last_became_peered":"2023-10-30T12:35:05.177316+0000","last_unstale":"2023-10-30T12:56:23.246213+0000","last_undegraded":"2023-10-30T12:56:23.246213+0000","last_fullsized":"2023-10-30T12:56:23.246213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'238","last_scrub_stamp":"2023-10-30T12:56:23.246132+0000","last_deep_scrub":"55'238","last_deep_scrub_stamp":"2023-10-30T12:56:23.246132+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.246132+0000","objects_scrubbed":4,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:22:00.496493+0000","scrub_duration":0.023404655999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":180,"num_write_kb":8325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12558,"num_omap_keys":23,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"55'846","reported_seq":3021,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:39.581411+0000","last_change":"2023-10-30T12:56:39.581493+0000","last_active":"2023-10-30T12:56:39.581411+0000","last_peered":"2023-10-30T12:56:39.581411+0000","last_clean":"2023-10-30T12:56:39.581411+0000","last_became_active":"2023-10-30T12:35:46.049464+0000","last_became_peered":"2023-10-30T12:35:46.049464+0000","last_unstale":"2023-10-30T12:56:39.581411+0000","last_undegraded":"2023-10-30T12:56:39.581411+0000","last_fullsized":"2023-10-30T12:56:39.581411+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'846","last_scrub_stamp":"2023-10-30T12:56:39.581325+0000","last_deep_scrub":"55'846","last_deep_scrub_stamp":"2023-10-30T12:56:39.581325+0000","last_clean_scrub_stamp":"2023-10-30T12:56:39.581325+0000","objects_scrubbed":163,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:25:02.063225+0000","scrub_duration":0.248567017,"objects_trimmed":0,"snaptrim_duration":0.0023101760000000002,"stat_sum":{"num_bytes":7006288,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":663,"num_write_kb":12867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"55'215","reported_seq":285,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:32.554259+0000","last_change":"2023-10-30T12:56:32.554326+0000","last_active":"2023-10-30T12:56:32.554259+0000","last_peered":"2023-10-30T12:56:32.554259+0000","last_clean":"2023-10-30T12:56:32.554259+0000","last_became_active":"2023-10-30T12:35:05.171491+0000","last_became_peered":"2023-10-30T12:35:05.171491+0000","last_unstale":"2023-10-30T12:56:32.554259+0000","last_undegraded":"2023-10-30T12:56:32.554259+0000","last_fullsized":"2023-10-30T12:56:32.554259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'215","last_scrub_stamp":"2023-10-30T12:56:32.554161+0000","last_deep_scrub":"55'215","last_deep_scrub_stamp":"2023-10-30T12:56:32.554161+0000","last_clean_scrub_stamp":"2023-10-30T12:56:32.554161+0000","objects_scrubbed":4,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:25:32.692345+0000","scrub_duration":0.026739378000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":16387,"num_write":154,"num_write_kb":4208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11738,"num_omap_keys":23,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"55'931","reported_seq":3108,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:56.900412+0000","last_change":"2023-10-30T12:56:56.900531+0000","last_active":"2023-10-30T12:56:56.900412+0000","last_peered":"2023-10-30T12:56:56.900412+0000","last_clean":"2023-10-30T12:56:56.900412+0000","last_became_active":"2023-10-30T12:35:06.213493+0000","last_became_peered":"2023-10-30T12:35:06.213493+0000","last_unstale":"2023-10-30T12:56:56.900412+0000","last_undegraded":"2023-10-30T12:56:56.900412+0000","last_fullsized":"2023-10-30T12:56:56.900412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'931","last_scrub_stamp":"2023-10-30T12:56:56.900213+0000","last_deep_scrub":"55'931","last_deep_scrub_stamp":"2023-10-30T12:56:56.900213+0000","last_clean_scrub_stamp":"2023-10-30T12:56:56.900213+0000","objects_scrubbed":160,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:31:30.596032+0000","scrub_duration":0.55565498499999999,"objects_trimmed":0,"snaptrim_duration":0.015847370999999999,"stat_sum":{"num_bytes":18244405,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":70,"num_read_kb":4114,"num_write":718,"num_write_kb":33517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"55'310","reported_seq":382,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:20.198207+0000","last_change":"2023-10-30T12:56:20.198258+0000","last_active":"2023-10-30T12:56:20.198207+0000","last_peered":"2023-10-30T12:56:20.198207+0000","last_clean":"2023-10-30T12:56:20.198207+0000","last_became_active":"2023-10-30T12:35:05.197453+0000","last_became_peered":"2023-10-30T12:35:05.197453+0000","last_unstale":"2023-10-30T12:56:20.198207+0000","last_undegraded":"2023-10-30T12:56:20.198207+0000","last_fullsized":"2023-10-30T12:56:20.198207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'310","last_scrub_stamp":"2023-10-30T12:56:20.198131+0000","last_deep_scrub":"55'310","last_deep_scrub_stamp":"2023-10-30T12:56:20.198131+0000","last_clean_scrub_stamp":"2023-10-30T12:56:20.198131+0000","objects_scrubbed":5,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:00:10.556698+0000","scrub_duration":0.022980041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":240,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17972,"num_omap_keys":33,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"55'841","reported_seq":3008,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:02.283724+0000","last_change":"2023-10-30T12:57:02.283816+0000","last_active":"2023-10-30T12:57:02.283724+0000","last_peered":"2023-10-30T12:57:02.283724+0000","last_clean":"2023-10-30T12:57:02.283724+0000","last_became_active":"2023-10-30T12:35:06.219526+0000","last_became_peered":"2023-10-30T12:35:06.219526+0000","last_unstale":"2023-10-30T12:57:02.283724+0000","last_undegraded":"2023-10-30T12:57:02.283724+0000","last_fullsized":"2023-10-30T12:57:02.283724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'841","last_scrub_stamp":"2023-10-30T12:57:02.283577+0000","last_deep_scrub":"55'841","last_deep_scrub_stamp":"2023-10-30T12:57:02.283577+0000","last_clean_scrub_stamp":"2023-10-30T12:57:02.283577+0000","objects_scrubbed":150,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:20:08.888700+0000","scrub_duration":0.308269513,"objects_trimmed":0,"snaptrim_duration":0.0060180570000000003,"stat_sum":{"num_bytes":14352444,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":61,"num_read_kb":19722,"num_write":644,"num_write_kb":44993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"55'119","reported_seq":220,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:27.868066+0000","last_change":"2023-10-30T12:56:27.868066+0000","last_active":"2023-10-30T12:56:27.868066+0000","last_peered":"2023-10-30T12:56:27.868066+0000","last_clean":"2023-10-30T12:56:27.868066+0000","last_became_active":"2023-10-30T12:34:55.205923+0000","last_became_peered":"2023-10-30T12:34:55.205923+0000","last_unstale":"2023-10-30T12:56:27.868066+0000","last_undegraded":"2023-10-30T12:56:27.868066+0000","last_fullsized":"2023-10-30T12:56:27.868066+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'119","last_scrub_stamp":"2023-10-30T12:56:27.867851+0000","last_deep_scrub":"55'119","last_deep_scrub_stamp":"2023-10-30T12:56:27.867851+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.867851+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:43:27.714386+0000","scrub_duration":0.0099090500000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"55'316","reported_seq":394,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:28.481897+0000","last_change":"2023-10-30T12:56:28.482017+0000","last_active":"2023-10-30T12:56:28.481897+0000","last_peered":"2023-10-30T12:56:28.481897+0000","last_clean":"2023-10-30T12:56:28.481897+0000","last_became_active":"2023-10-30T12:35:05.197201+0000","last_became_peered":"2023-10-30T12:35:05.197201+0000","last_unstale":"2023-10-30T12:56:28.481897+0000","last_undegraded":"2023-10-30T12:56:28.481897+0000","last_fullsized":"2023-10-30T12:56:28.481897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'316","last_scrub_stamp":"2023-10-30T12:56:28.481711+0000","last_deep_scrub":"55'316","last_deep_scrub_stamp":"2023-10-30T12:56:28.481711+0000","last_clean_scrub_stamp":"2023-10-30T12:56:28.481711+0000","objects_scrubbed":5,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:52.589549+0000","scrub_duration":0.039869544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":23,"num_read_kb":16415,"num_write":260,"num_write_kb":8374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"55'803","reported_seq":2961,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:43.609349+0000","last_change":"2023-10-30T12:56:43.609412+0000","last_active":"2023-10-30T12:56:43.609349+0000","last_peered":"2023-10-30T12:56:43.609349+0000","last_clean":"2023-10-30T12:56:43.609349+0000","last_became_active":"2023-10-30T12:35:06.217687+0000","last_became_peered":"2023-10-30T12:35:06.217687+0000","last_unstale":"2023-10-30T12:56:43.609349+0000","last_undegraded":"2023-10-30T12:56:43.609349+0000","last_fullsized":"2023-10-30T12:56:43.609349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'803","last_scrub_stamp":"2023-10-30T12:56:43.609269+0000","last_deep_scrub":"55'803","last_deep_scrub_stamp":"2023-10-30T12:56:43.609269+0000","last_clean_scrub_stamp":"2023-10-30T12:56:43.609269+0000","objects_scrubbed":158,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:35:10.362189+0000","scrub_duration":0.29605744099999998,"objects_trimmed":0,"snaptrim_duration":0.0059801699999999999,"stat_sum":{"num_bytes":18791963,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":51,"num_read_kb":8203,"num_write":599,"num_write_kb":32827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"55'429","reported_seq":511,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.556052+0000","last_change":"2023-10-30T12:56:53.936455+0000","last_active":"2023-10-30T12:57:12.556052+0000","last_peered":"2023-10-30T12:57:12.556052+0000","last_clean":"2023-10-30T12:57:12.556052+0000","last_became_active":"2023-10-30T12:35:05.207828+0000","last_became_peered":"2023-10-30T12:35:05.207828+0000","last_unstale":"2023-10-30T12:57:12.556052+0000","last_undegraded":"2023-10-30T12:57:12.556052+0000","last_fullsized":"2023-10-30T12:57:12.556052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'426","last_scrub_stamp":"2023-10-30T12:56:53.936122+0000","last_deep_scrub":"55'426","last_deep_scrub_stamp":"2023-10-30T12:56:53.936122+0000","last_clean_scrub_stamp":"2023-10-30T12:56:53.936122+0000","objects_scrubbed":7,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:37:43.112167+0000","scrub_duration":0.083282911000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22042852,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":29,"num_read_kb":45074,"num_write":354,"num_write_kb":21733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"55'837","reported_seq":3002,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:28.741030+0000","last_change":"2023-10-30T12:56:28.741094+0000","last_active":"2023-10-30T12:56:28.741030+0000","last_peered":"2023-10-30T12:56:28.741030+0000","last_clean":"2023-10-30T12:56:28.741030+0000","last_became_active":"2023-10-30T12:35:06.194791+0000","last_became_peered":"2023-10-30T12:35:06.194791+0000","last_unstale":"2023-10-30T12:56:28.741030+0000","last_undegraded":"2023-10-30T12:56:28.741030+0000","last_fullsized":"2023-10-30T12:56:28.741030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'837","last_scrub_stamp":"2023-10-30T12:56:28.740945+0000","last_deep_scrub":"55'837","last_deep_scrub_stamp":"2023-10-30T12:56:28.740945+0000","last_clean_scrub_stamp":"2023-10-30T12:56:28.740945+0000","objects_scrubbed":162,"log_size":837,"log_dups_size":0,"ondisk_log_size":837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:55:53.265534+0000","scrub_duration":0.21527656000000001,"objects_trimmed":0,"snaptrim_duration":0.0043414500000000002,"stat_sum":{"num_bytes":10614378,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":58,"num_read_kb":3682,"num_write":656,"num_write_kb":34629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"55'618","reported_seq":985,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:49.619643+0000","last_change":"2023-10-30T12:56:21.219063+0000","last_active":"2023-10-30T12:56:49.619643+0000","last_peered":"2023-10-30T12:56:49.619643+0000","last_clean":"2023-10-30T12:56:49.619643+0000","last_became_active":"2023-10-30T12:35:05.175959+0000","last_became_peered":"2023-10-30T12:35:05.175959+0000","last_unstale":"2023-10-30T12:56:49.619643+0000","last_undegraded":"2023-10-30T12:56:49.619643+0000","last_fullsized":"2023-10-30T12:56:49.619643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'615","last_scrub_stamp":"2023-10-30T12:56:21.218930+0000","last_deep_scrub":"55'615","last_deep_scrub_stamp":"2023-10-30T12:56:21.218930+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.218930+0000","objects_scrubbed":9,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:53:08.332507+0000","scrub_duration":0.031017952000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9398249,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":313,"num_read_kb":20888,"num_write":569,"num_write_kb":9629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34344,"num_omap_keys":63,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"55'863","reported_seq":3039,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:40.689114+0000","last_change":"2023-10-30T12:56:40.689114+0000","last_active":"2023-10-30T12:56:40.689114+0000","last_peered":"2023-10-30T12:56:40.689114+0000","last_clean":"2023-10-30T12:56:40.689114+0000","last_became_active":"2023-10-30T12:35:06.164711+0000","last_became_peered":"2023-10-30T12:35:06.164711+0000","last_unstale":"2023-10-30T12:56:40.689114+0000","last_undegraded":"2023-10-30T12:56:40.689114+0000","last_fullsized":"2023-10-30T12:56:40.689114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'863","last_scrub_stamp":"2023-10-30T12:56:40.688985+0000","last_deep_scrub":"55'863","last_deep_scrub_stamp":"2023-10-30T12:56:40.688985+0000","last_clean_scrub_stamp":"2023-10-30T12:56:40.688985+0000","objects_scrubbed":162,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:58:54.635220+0000","scrub_duration":0.20007371800000001,"objects_trimmed":0,"snaptrim_duration":0.002714419,"stat_sum":{"num_bytes":18668779,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":68,"num_read_kb":8213,"num_write":645,"num_write_kb":33825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"55'501","reported_seq":574,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:22.428808+0000","last_change":"2023-10-30T12:56:22.428932+0000","last_active":"2023-10-30T12:56:22.428808+0000","last_peered":"2023-10-30T12:56:22.428808+0000","last_clean":"2023-10-30T12:56:22.428808+0000","last_became_active":"2023-10-30T12:35:05.196873+0000","last_became_peered":"2023-10-30T12:35:05.196873+0000","last_unstale":"2023-10-30T12:56:22.428808+0000","last_undegraded":"2023-10-30T12:56:22.428808+0000","last_fullsized":"2023-10-30T12:56:22.428808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'501","last_scrub_stamp":"2023-10-30T12:56:22.428591+0000","last_deep_scrub":"55'501","last_deep_scrub_stamp":"2023-10-30T12:56:22.428591+0000","last_clean_scrub_stamp":"2023-10-30T12:56:22.428591+0000","objects_scrubbed":5,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:31.946150+0000","scrub_duration":0.065339878000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":431,"num_write_kb":16687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8414,"num_omap_keys":16,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"55'813","reported_seq":2992,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:22.854721+0000","last_change":"2023-10-30T12:56:22.854850+0000","last_active":"2023-10-30T12:56:22.854721+0000","last_peered":"2023-10-30T12:56:22.854721+0000","last_clean":"2023-10-30T12:56:22.854721+0000","last_became_active":"2023-10-30T12:35:06.183225+0000","last_became_peered":"2023-10-30T12:35:06.183225+0000","last_unstale":"2023-10-30T12:56:22.854721+0000","last_undegraded":"2023-10-30T12:56:22.854721+0000","last_fullsized":"2023-10-30T12:56:22.854721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'813","last_scrub_stamp":"2023-10-30T12:56:22.854526+0000","last_deep_scrub":"55'813","last_deep_scrub_stamp":"2023-10-30T12:56:22.854526+0000","last_clean_scrub_stamp":"2023-10-30T12:56:22.854526+0000","objects_scrubbed":154,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:19:37.784115+0000","scrub_duration":0.30563644699999998,"objects_trimmed":0,"snaptrim_duration":0.0032989500000000001,"stat_sum":{"num_bytes":18054478,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":72,"num_read_kb":12,"num_write":656,"num_write_kb":36307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"55'185","reported_seq":250,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:36.447235+0000","last_change":"2023-10-30T12:56:36.447291+0000","last_active":"2023-10-30T12:56:36.447235+0000","last_peered":"2023-10-30T12:56:36.447235+0000","last_clean":"2023-10-30T12:56:36.447235+0000","last_became_active":"2023-10-30T12:35:05.177266+0000","last_became_peered":"2023-10-30T12:35:05.177266+0000","last_unstale":"2023-10-30T12:56:36.447235+0000","last_undegraded":"2023-10-30T12:56:36.447235+0000","last_fullsized":"2023-10-30T12:56:36.447235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'185","last_scrub_stamp":"2023-10-30T12:56:36.447125+0000","last_deep_scrub":"55'185","last_deep_scrub_stamp":"2023-10-30T12:56:36.447125+0000","last_clean_scrub_stamp":"2023-10-30T12:56:36.447125+0000","objects_scrubbed":2,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:52:35.695395+0000","scrub_duration":0.022317884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":129,"num_write_kb":8259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"55'846","reported_seq":3022,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:43.791966+0000","last_change":"2023-10-30T12:56:43.792110+0000","last_active":"2023-10-30T12:56:43.791966+0000","last_peered":"2023-10-30T12:56:43.791966+0000","last_clean":"2023-10-30T12:56:43.791966+0000","last_became_active":"2023-10-30T12:35:06.196307+0000","last_became_peered":"2023-10-30T12:35:06.196307+0000","last_unstale":"2023-10-30T12:56:43.791966+0000","last_undegraded":"2023-10-30T12:56:43.791966+0000","last_fullsized":"2023-10-30T12:56:43.791966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'846","last_scrub_stamp":"2023-10-30T12:56:43.791757+0000","last_deep_scrub":"55'846","last_deep_scrub_stamp":"2023-10-30T12:56:43.791757+0000","last_clean_scrub_stamp":"2023-10-30T12:56:43.791757+0000","objects_scrubbed":158,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:38:44.933357+0000","scrub_duration":0.30352262899999999,"objects_trimmed":0,"snaptrim_duration":0.015197106,"stat_sum":{"num_bytes":5885946,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":69,"num_read_kb":13,"num_write":633,"num_write_kb":24590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"55'541","reported_seq":613,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:04.631724+0000","last_change":"2023-10-30T12:56:58.010026+0000","last_active":"2023-10-30T12:57:04.631724+0000","last_peered":"2023-10-30T12:57:04.631724+0000","last_clean":"2023-10-30T12:57:04.631724+0000","last_became_active":"2023-10-30T12:35:05.206930+0000","last_became_peered":"2023-10-30T12:35:05.206930+0000","last_unstale":"2023-10-30T12:57:04.631724+0000","last_undegraded":"2023-10-30T12:57:04.631724+0000","last_fullsized":"2023-10-30T12:57:04.631724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'539","last_scrub_stamp":"2023-10-30T12:56:58.009697+0000","last_deep_scrub":"55'539","last_deep_scrub_stamp":"2023-10-30T12:56:58.009697+0000","last_clean_scrub_stamp":"2023-10-30T12:56:58.009697+0000","objects_scrubbed":11,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:24:47.680526+0000","scrub_duration":0.068773216999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16497412,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":8,"num_read_kb":9517,"num_write":471,"num_write_kb":16402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":56804,"num_omap_keys":105,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"55'886","reported_seq":3066,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:44.620900+0000","last_change":"2023-10-30T12:56:44.620946+0000","last_active":"2023-10-30T12:56:44.620900+0000","last_peered":"2023-10-30T12:56:44.620900+0000","last_clean":"2023-10-30T12:56:44.620900+0000","last_became_active":"2023-10-30T12:35:06.164733+0000","last_became_peered":"2023-10-30T12:35:06.164733+0000","last_unstale":"2023-10-30T12:56:44.620900+0000","last_undegraded":"2023-10-30T12:56:44.620900+0000","last_fullsized":"2023-10-30T12:56:44.620900+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'886","last_scrub_stamp":"2023-10-30T12:56:44.620823+0000","last_deep_scrub":"55'886","last_deep_scrub_stamp":"2023-10-30T12:56:44.620823+0000","last_clean_scrub_stamp":"2023-10-30T12:56:44.620823+0000","objects_scrubbed":185,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:12.989394+0000","scrub_duration":0.199244536,"objects_trimmed":0,"snaptrim_duration":0.0024278020000000002,"stat_sum":{"num_bytes":19206810,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":652,"num_write_kb":32920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"55'381","reported_seq":458,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:26.587606+0000","last_change":"2023-10-30T12:56:26.587606+0000","last_active":"2023-10-30T12:56:26.587606+0000","last_peered":"2023-10-30T12:56:26.587606+0000","last_clean":"2023-10-30T12:56:26.587606+0000","last_became_active":"2023-10-30T12:35:05.184695+0000","last_became_peered":"2023-10-30T12:35:05.184695+0000","last_unstale":"2023-10-30T12:56:26.587606+0000","last_undegraded":"2023-10-30T12:56:26.587606+0000","last_fullsized":"2023-10-30T12:56:26.587606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'381","last_scrub_stamp":"2023-10-30T12:56:26.587467+0000","last_deep_scrub":"55'381","last_deep_scrub_stamp":"2023-10-30T12:56:26.587467+0000","last_clean_scrub_stamp":"2023-10-30T12:56:26.587467+0000","objects_scrubbed":7,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:27:14.820811+0000","scrub_duration":0.083109266000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":8489,"num_write":574,"num_write_kb":5859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54698,"num_omap_keys":106,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"55'856","reported_seq":3025,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:10.816652+0000","last_change":"2023-10-30T12:57:10.816704+0000","last_active":"2023-10-30T12:57:10.816652+0000","last_peered":"2023-10-30T12:57:10.816652+0000","last_clean":"2023-10-30T12:57:10.816652+0000","last_became_active":"2023-10-30T12:35:06.204422+0000","last_became_peered":"2023-10-30T12:35:06.204422+0000","last_unstale":"2023-10-30T12:57:10.816652+0000","last_undegraded":"2023-10-30T12:57:10.816652+0000","last_fullsized":"2023-10-30T12:57:10.816652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'856","last_scrub_stamp":"2023-10-30T12:57:10.816574+0000","last_deep_scrub":"55'856","last_deep_scrub_stamp":"2023-10-30T12:57:10.816574+0000","last_clean_scrub_stamp":"2023-10-30T12:57:10.816574+0000","objects_scrubbed":174,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:27:05.996137+0000","scrub_duration":0.332536045,"objects_trimmed":0,"snaptrim_duration":0.0048535030000000003,"stat_sum":{"num_bytes":22995855,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":62,"num_read_kb":4110,"num_write":640,"num_write_kb":33018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"55'236","reported_seq":308,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:23.771446+0000","last_change":"2023-10-30T12:56:23.771446+0000","last_active":"2023-10-30T12:56:23.771446+0000","last_peered":"2023-10-30T12:56:23.771446+0000","last_clean":"2023-10-30T12:56:23.771446+0000","last_became_active":"2023-10-30T12:35:05.200449+0000","last_became_peered":"2023-10-30T12:35:05.200449+0000","last_unstale":"2023-10-30T12:56:23.771446+0000","last_undegraded":"2023-10-30T12:56:23.771446+0000","last_fullsized":"2023-10-30T12:56:23.771446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'236","last_scrub_stamp":"2023-10-30T12:56:23.771024+0000","last_deep_scrub":"55'236","last_deep_scrub_stamp":"2023-10-30T12:56:23.771024+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.771024+0000","objects_scrubbed":6,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:55:13.133447+0000","scrub_duration":0.033750645000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":4117,"num_write":187,"num_write_kb":4299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18018,"num_omap_keys":33,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"55'831","reported_seq":3007,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:48.796437+0000","last_change":"2023-10-30T12:56:48.796491+0000","last_active":"2023-10-30T12:56:48.796437+0000","last_peered":"2023-10-30T12:56:48.796437+0000","last_clean":"2023-10-30T12:56:48.796437+0000","last_became_active":"2023-10-30T12:35:06.179412+0000","last_became_peered":"2023-10-30T12:35:06.179412+0000","last_unstale":"2023-10-30T12:56:48.796437+0000","last_undegraded":"2023-10-30T12:56:48.796437+0000","last_fullsized":"2023-10-30T12:56:48.796437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'831","last_scrub_stamp":"2023-10-30T12:56:48.796353+0000","last_deep_scrub":"55'831","last_deep_scrub_stamp":"2023-10-30T12:56:48.796353+0000","last_clean_scrub_stamp":"2023-10-30T12:56:48.796353+0000","objects_scrubbed":166,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:11:50.029607+0000","scrub_duration":0.362753192,"objects_trimmed":0,"snaptrim_duration":0.003661568,"stat_sum":{"num_bytes":1664351,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":69,"num_read_kb":22,"num_write":614,"num_write_kb":12460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"55'560","reported_seq":635,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:32.835075+0000","last_change":"2023-10-30T12:56:32.835075+0000","last_active":"2023-10-30T12:56:32.835075+0000","last_peered":"2023-10-30T12:56:32.835075+0000","last_clean":"2023-10-30T12:56:32.835075+0000","last_became_active":"2023-10-30T12:35:05.206607+0000","last_became_peered":"2023-10-30T12:35:05.206607+0000","last_unstale":"2023-10-30T12:56:32.835075+0000","last_undegraded":"2023-10-30T12:56:32.835075+0000","last_fullsized":"2023-10-30T12:56:32.835075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'560","last_scrub_stamp":"2023-10-30T12:56:32.834680+0000","last_deep_scrub":"55'560","last_deep_scrub_stamp":"2023-10-30T12:56:32.834680+0000","last_clean_scrub_stamp":"2023-10-30T12:56:32.834680+0000","objects_scrubbed":8,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:19:51.459668+0000","scrub_duration":0.066587212000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":17,"num_read_kb":23179,"num_write":497,"num_write_kb":12662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34806,"num_omap_keys":64,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"55'809","reported_seq":2986,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.829353+0000","last_change":"2023-10-30T12:56:24.829353+0000","last_active":"2023-10-30T12:56:24.829353+0000","last_peered":"2023-10-30T12:56:24.829353+0000","last_clean":"2023-10-30T12:56:24.829353+0000","last_became_active":"2023-10-30T12:35:06.181256+0000","last_became_peered":"2023-10-30T12:35:06.181256+0000","last_unstale":"2023-10-30T12:56:24.829353+0000","last_undegraded":"2023-10-30T12:56:24.829353+0000","last_fullsized":"2023-10-30T12:56:24.829353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'809","last_scrub_stamp":"2023-10-30T12:56:24.829117+0000","last_deep_scrub":"55'809","last_deep_scrub_stamp":"2023-10-30T12:56:24.829117+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.829117+0000","objects_scrubbed":153,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:42:27.862243+0000","scrub_duration":0.27743204100000002,"objects_trimmed":0,"snaptrim_duration":0.0030387220000000002,"stat_sum":{"num_bytes":10464290,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":70,"num_read_kb":4103,"num_write":644,"num_write_kb":29233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"55'374","reported_seq":447,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.513468+0000","last_change":"2023-10-30T12:56:24.513468+0000","last_active":"2023-10-30T12:56:24.513468+0000","last_peered":"2023-10-30T12:56:24.513468+0000","last_clean":"2023-10-30T12:56:24.513468+0000","last_became_active":"2023-10-30T12:35:05.200724+0000","last_became_peered":"2023-10-30T12:35:05.200724+0000","last_unstale":"2023-10-30T12:56:24.513468+0000","last_undegraded":"2023-10-30T12:56:24.513468+0000","last_fullsized":"2023-10-30T12:56:24.513468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'374","last_scrub_stamp":"2023-10-30T12:56:24.513345+0000","last_deep_scrub":"55'374","last_deep_scrub_stamp":"2023-10-30T12:56:24.513345+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.513345+0000","objects_scrubbed":7,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:34:47.132641+0000","scrub_duration":0.034143153000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388698,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":12300,"num_write":322,"num_write_kb":8548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25586,"num_omap_keys":47,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"55'835","reported_seq":3000,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:35.315188+0000","last_change":"2023-10-30T12:56:35.315188+0000","last_active":"2023-10-30T12:56:35.315188+0000","last_peered":"2023-10-30T12:56:35.315188+0000","last_clean":"2023-10-30T12:56:35.315188+0000","last_became_active":"2023-10-30T12:35:06.173996+0000","last_became_peered":"2023-10-30T12:35:06.173996+0000","last_unstale":"2023-10-30T12:56:35.315188+0000","last_undegraded":"2023-10-30T12:56:35.315188+0000","last_fullsized":"2023-10-30T12:56:35.315188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'835","last_scrub_stamp":"2023-10-30T12:56:35.315063+0000","last_deep_scrub":"55'835","last_deep_scrub_stamp":"2023-10-30T12:56:35.315063+0000","last_clean_scrub_stamp":"2023-10-30T12:56:35.315063+0000","objects_scrubbed":159,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:30:07.209945+0000","scrub_duration":0.14615183500000001,"objects_trimmed":0,"snaptrim_duration":0.0045562010000000002,"stat_sum":{"num_bytes":5693555,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":58,"num_read_kb":18,"num_write":647,"num_write_kb":16270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"55'881","reported_seq":952,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:30.589026+0000","last_change":"2023-10-30T12:56:30.589026+0000","last_active":"2023-10-30T12:56:30.589026+0000","last_peered":"2023-10-30T12:56:30.589026+0000","last_clean":"2023-10-30T12:56:30.589026+0000","last_became_active":"2023-10-30T12:35:05.163794+0000","last_became_peered":"2023-10-30T12:35:05.163794+0000","last_unstale":"2023-10-30T12:56:30.589026+0000","last_undegraded":"2023-10-30T12:56:30.589026+0000","last_fullsized":"2023-10-30T12:56:30.589026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'881","last_scrub_stamp":"2023-10-30T12:56:30.588883+0000","last_deep_scrub":"55'881","last_deep_scrub_stamp":"2023-10-30T12:56:30.588883+0000","last_clean_scrub_stamp":"2023-10-30T12:56:30.588883+0000","objects_scrubbed":7,"log_size":881,"log_dups_size":0,"ondisk_log_size":881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:50:39.690725+0000","scrub_duration":0.062172023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":10,"num_read_kb":8454,"num_write":955,"num_write_kb":13780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23914,"num_omap_keys":46,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"55'808","reported_seq":2982,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:51.694094+0000","last_change":"2023-10-30T12:56:51.694150+0000","last_active":"2023-10-30T12:56:51.694094+0000","last_peered":"2023-10-30T12:56:51.694094+0000","last_clean":"2023-10-30T12:56:51.694094+0000","last_became_active":"2023-10-30T12:35:06.183226+0000","last_became_peered":"2023-10-30T12:35:06.183226+0000","last_unstale":"2023-10-30T12:56:51.694094+0000","last_undegraded":"2023-10-30T12:56:51.694094+0000","last_fullsized":"2023-10-30T12:56:51.694094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'808","last_scrub_stamp":"2023-10-30T12:56:51.694014+0000","last_deep_scrub":"55'808","last_deep_scrub_stamp":"2023-10-30T12:56:51.694014+0000","last_clean_scrub_stamp":"2023-10-30T12:56:51.694014+0000","objects_scrubbed":140,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:16:30.542141+0000","scrub_duration":0.15061528399999999,"objects_trimmed":0,"snaptrim_duration":0.0052521379999999999,"stat_sum":{"num_bytes":5799195,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":69,"num_read_kb":4112,"num_write":618,"num_write_kb":16472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"55'344","reported_seq":419,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:33.883120+0000","last_change":"2023-10-30T12:56:33.883350+0000","last_active":"2023-10-30T12:56:33.883120+0000","last_peered":"2023-10-30T12:56:33.883120+0000","last_clean":"2023-10-30T12:56:33.883120+0000","last_became_active":"2023-10-30T12:35:05.206518+0000","last_became_peered":"2023-10-30T12:35:05.206518+0000","last_unstale":"2023-10-30T12:56:33.883120+0000","last_undegraded":"2023-10-30T12:56:33.883120+0000","last_fullsized":"2023-10-30T12:56:33.883120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'344","last_scrub_stamp":"2023-10-30T12:56:33.882814+0000","last_deep_scrub":"55'344","last_deep_scrub_stamp":"2023-10-30T12:56:33.882814+0000","last_clean_scrub_stamp":"2023-10-30T12:56:33.882814+0000","objects_scrubbed":8,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:13:20.348463+0000","scrub_duration":0.065105804000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":268,"num_write_kb":12461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":36978,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"55'781","reported_seq":2956,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:29.111348+0000","last_change":"2023-10-30T12:56:29.111348+0000","last_active":"2023-10-30T12:56:29.111348+0000","last_peered":"2023-10-30T12:56:29.111348+0000","last_clean":"2023-10-30T12:56:29.111348+0000","last_became_active":"2023-10-30T12:35:06.197861+0000","last_became_peered":"2023-10-30T12:35:06.197861+0000","last_unstale":"2023-10-30T12:56:29.111348+0000","last_undegraded":"2023-10-30T12:56:29.111348+0000","last_fullsized":"2023-10-30T12:56:29.111348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'781","last_scrub_stamp":"2023-10-30T12:56:29.111192+0000","last_deep_scrub":"55'781","last_deep_scrub_stamp":"2023-10-30T12:56:29.111192+0000","last_clean_scrub_stamp":"2023-10-30T12:56:29.111192+0000","objects_scrubbed":157,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:53:54.048846+0000","scrub_duration":0.27439538299999999,"objects_trimmed":0,"snaptrim_duration":0.011118161,"stat_sum":{"num_bytes":14763185,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":68,"num_read_kb":15,"num_write":609,"num_write_kb":24605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"55'844","reported_seq":3030,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:49.658569+0000","last_change":"2023-10-30T12:56:49.658623+0000","last_active":"2023-10-30T12:56:49.658569+0000","last_peered":"2023-10-30T12:56:49.658569+0000","last_clean":"2023-10-30T12:56:49.658569+0000","last_became_active":"2023-10-30T12:35:06.212548+0000","last_became_peered":"2023-10-30T12:35:06.212548+0000","last_unstale":"2023-10-30T12:56:49.658569+0000","last_undegraded":"2023-10-30T12:56:49.658569+0000","last_fullsized":"2023-10-30T12:56:49.658569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'844","last_scrub_stamp":"2023-10-30T12:56:49.658439+0000","last_deep_scrub":"55'844","last_deep_scrub_stamp":"2023-10-30T12:56:49.658439+0000","last_clean_scrub_stamp":"2023-10-30T12:56:49.658439+0000","objects_scrubbed":159,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:08:37.480683+0000","scrub_duration":0.29739869600000002,"objects_trimmed":0,"snaptrim_duration":0.015114931999999999,"stat_sum":{"num_bytes":19366418,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":81,"num_read_kb":7349,"num_write":643,"num_write_kb":32740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"55'368","reported_seq":437,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.907303+0000","last_change":"2023-10-30T12:56:34.907429+0000","last_active":"2023-10-30T12:56:34.907303+0000","last_peered":"2023-10-30T12:56:34.907303+0000","last_clean":"2023-10-30T12:56:34.907303+0000","last_became_active":"2023-10-30T12:35:05.206559+0000","last_became_peered":"2023-10-30T12:35:05.206559+0000","last_unstale":"2023-10-30T12:56:34.907303+0000","last_undegraded":"2023-10-30T12:56:34.907303+0000","last_fullsized":"2023-10-30T12:56:34.907303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'368","last_scrub_stamp":"2023-10-30T12:56:34.907076+0000","last_deep_scrub":"55'368","last_deep_scrub_stamp":"2023-10-30T12:56:34.907076+0000","last_clean_scrub_stamp":"2023-10-30T12:56:34.907076+0000","objects_scrubbed":5,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:12:13.284723+0000","scrub_duration":0.045168409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":312,"num_write_kb":8389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19072,"num_omap_keys":35,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"55'843","reported_seq":3018,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:50.637357+0000","last_change":"2023-10-30T12:56:50.637414+0000","last_active":"2023-10-30T12:56:50.637357+0000","last_peered":"2023-10-30T12:56:50.637357+0000","last_clean":"2023-10-30T12:56:50.637357+0000","last_became_active":"2023-10-30T12:35:06.212505+0000","last_became_peered":"2023-10-30T12:35:06.212505+0000","last_unstale":"2023-10-30T12:56:50.637357+0000","last_undegraded":"2023-10-30T12:56:50.637357+0000","last_fullsized":"2023-10-30T12:56:50.637357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'843","last_scrub_stamp":"2023-10-30T12:56:50.637270+0000","last_deep_scrub":"55'843","last_deep_scrub_stamp":"2023-10-30T12:56:50.637270+0000","last_clean_scrub_stamp":"2023-10-30T12:56:50.637270+0000","objects_scrubbed":156,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:28:10.699275+0000","scrub_duration":0.27534876899999999,"objects_trimmed":0,"snaptrim_duration":0.014553393,"stat_sum":{"num_bytes":9800415,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":70,"num_read_kb":12304,"num_write":642,"num_write_kb":36520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"55'387","reported_seq":467,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:37.980220+0000","last_change":"2023-10-30T12:56:37.980331+0000","last_active":"2023-10-30T12:56:37.980220+0000","last_peered":"2023-10-30T12:56:37.980220+0000","last_clean":"2023-10-30T12:56:37.980220+0000","last_became_active":"2023-10-30T12:35:05.206879+0000","last_became_peered":"2023-10-30T12:35:05.206879+0000","last_unstale":"2023-10-30T12:56:37.980220+0000","last_undegraded":"2023-10-30T12:56:37.980220+0000","last_fullsized":"2023-10-30T12:56:37.980220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'387","last_scrub_stamp":"2023-10-30T12:56:37.980050+0000","last_deep_scrub":"55'387","last_deep_scrub_stamp":"2023-10-30T12:56:37.980050+0000","last_clean_scrub_stamp":"2023-10-30T12:56:37.980050+0000","objects_scrubbed":10,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:37:17.148411+0000","scrub_duration":0.082641178999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":32,"num_read_kb":45093,"num_write":322,"num_write_kb":20730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31096,"num_omap_keys":59,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"55'779","reported_seq":2955,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:56.563086+0000","last_change":"2023-10-30T12:56:56.563165+0000","last_active":"2023-10-30T12:56:56.563086+0000","last_peered":"2023-10-30T12:56:56.563086+0000","last_clean":"2023-10-30T12:56:56.563086+0000","last_became_active":"2023-10-30T12:35:46.275077+0000","last_became_peered":"2023-10-30T12:35:46.275077+0000","last_unstale":"2023-10-30T12:56:56.563086+0000","last_undegraded":"2023-10-30T12:56:56.563086+0000","last_fullsized":"2023-10-30T12:56:56.563086+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'779","last_scrub_stamp":"2023-10-30T12:56:56.562988+0000","last_deep_scrub":"55'779","last_deep_scrub_stamp":"2023-10-30T12:56:56.562988+0000","last_clean_scrub_stamp":"2023-10-30T12:56:56.562988+0000","objects_scrubbed":136,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:59:27.981737+0000","scrub_duration":0.14057847700000001,"objects_trimmed":0,"snaptrim_duration":0.0032416139999999999,"stat_sum":{"num_bytes":6205387,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":63,"num_read_kb":4113,"num_write":608,"num_write_kb":9154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"55'512","reported_seq":583,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:23.020777+0000","last_change":"2023-10-30T12:56:23.020777+0000","last_active":"2023-10-30T12:56:23.020777+0000","last_peered":"2023-10-30T12:56:23.020777+0000","last_clean":"2023-10-30T12:56:23.020777+0000","last_became_active":"2023-10-30T12:35:05.173984+0000","last_became_peered":"2023-10-30T12:35:05.173984+0000","last_unstale":"2023-10-30T12:56:23.020777+0000","last_undegraded":"2023-10-30T12:56:23.020777+0000","last_fullsized":"2023-10-30T12:56:23.020777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'512","last_scrub_stamp":"2023-10-30T12:56:23.020659+0000","last_deep_scrub":"55'512","last_deep_scrub_stamp":"2023-10-30T12:56:23.020659+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.020659+0000","objects_scrubbed":5,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:43:31.108759+0000","scrub_duration":0.032786733999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":430,"num_write_kb":12596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19196,"num_omap_keys":36,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"55'882","reported_seq":3052,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:49.685736+0000","last_change":"2023-10-30T12:56:49.685736+0000","last_active":"2023-10-30T12:56:49.685736+0000","last_peered":"2023-10-30T12:56:49.685736+0000","last_clean":"2023-10-30T12:56:49.685736+0000","last_became_active":"2023-10-30T12:35:06.198504+0000","last_became_peered":"2023-10-30T12:35:06.198504+0000","last_unstale":"2023-10-30T12:56:49.685736+0000","last_undegraded":"2023-10-30T12:56:49.685736+0000","last_fullsized":"2023-10-30T12:56:49.685736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'882","last_scrub_stamp":"2023-10-30T12:56:49.685619+0000","last_deep_scrub":"55'882","last_deep_scrub_stamp":"2023-10-30T12:56:49.685619+0000","last_clean_scrub_stamp":"2023-10-30T12:56:49.685619+0000","objects_scrubbed":182,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:10:12.105803+0000","scrub_duration":0.23854751900000001,"objects_trimmed":0,"snaptrim_duration":0.0055292249999999996,"stat_sum":{"num_bytes":14413504,"num_objects":182,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":61,"num_read_kb":1995,"num_write":647,"num_write_kb":19115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"55'216","reported_seq":307,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:20.534250+0000","last_change":"2023-10-30T12:56:20.534300+0000","last_active":"2023-10-30T12:56:20.534250+0000","last_peered":"2023-10-30T12:56:20.534250+0000","last_clean":"2023-10-30T12:56:20.534250+0000","last_became_active":"2023-10-30T12:35:05.200623+0000","last_became_peered":"2023-10-30T12:35:05.200623+0000","last_unstale":"2023-10-30T12:56:20.534250+0000","last_undegraded":"2023-10-30T12:56:20.534250+0000","last_fullsized":"2023-10-30T12:56:20.534250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'216","last_scrub_stamp":"2023-10-30T12:56:20.534185+0000","last_deep_scrub":"55'216","last_deep_scrub_stamp":"2023-10-30T12:56:20.534185+0000","last_clean_scrub_stamp":"2023-10-30T12:56:20.534185+0000","objects_scrubbed":8,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:19:00.729850+0000","scrub_duration":0.044608693999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":63,"num_read_kb":24633,"num_write":147,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7154,"num_omap_keys":14,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":925262768,"num_objects":10302,"num_object_clones":0,"num_object_copies":20604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10302,"num_whiteouts":0,"num_read":4339,"num_read_kb":400205,"num_write":41481,"num_write_kb":1950238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1902673920,"data_stored":1848985440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53590,"ondisk_log_size":53590,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":360906037,"num_objects":211,"num_object_clones":0,"num_object_copies":422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":211,"num_whiteouts":0,"num_read":1232,"num_read_kb":640401,"num_write":11731,"num_write_kb":363311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":636231,"num_omap_keys":1188,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":721977344,"data_stored":721812074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13275,"ondisk_log_size":13275,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542418,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484448,"kb_used_data":403288,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":93887392,"statfs":{"total":96636764160,"available":96140689408,"internally_reserved":0,"allocated":412966912,"data_stored":404891962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":83098061},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.82299999999999995,"15min":0.81399999999999995},"min":{"1min":0.47699999999999998,"5min":0.438,"15min":0.42899999999999999},"max":{"1min":1.1659999999999999,"5min":1.371,"15min":1.371},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.82899999999999996,"15min":0.82099999999999995},"min":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.41699999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.45},"last":1.216}]},{"osd":1,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83599999999999997,"15min":0.85199999999999998},"min":{"1min":0.55200000000000005,"5min":0.52700000000000002,"15min":0.44600000000000001},"max":{"1min":1.095,"5min":1.222,"15min":1.496},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.88,"15min":0.89300000000000002},"min":{"1min":0.58999999999999997,"5min":0.54800000000000004,"15min":0.51800000000000002},"max":{"1min":1.282,"5min":1.494,"15min":1.494},"last":1.0409999999999999}]},{"osd":2,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.84399999999999997,"15min":0.82699999999999996},"min":{"1min":0.64800000000000002,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.1479999999999999,"5min":1.242,"15min":1.9059999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.008,"5min":0.872,"15min":0.89700000000000002},"min":{"1min":0.71499999999999997,"5min":0.58299999999999996,"15min":0.48999999999999999},"max":{"1min":1.5609999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":0.96899999999999997,"15min":0.92500000000000004},"min":{"1min":0.77000000000000002,"5min":0.56999999999999995,"15min":0.54000000000000004},"max":{"1min":1.306,"5min":1.607,"15min":1.9810000000000001},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.98099999999999998,"15min":0.996},"min":{"1min":0.59299999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2749999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.97399999999999998}]},{"osd":4,"last update":"Mon Oct 30 12:56:42 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.91400000000000003,"15min":0.93200000000000005},"min":{"1min":0.622,"5min":0.53300000000000003,"15min":0.46899999999999997},"max":{"1min":1.1259999999999999,"5min":1.4059999999999999,"15min":2.7309999999999999},"last":1.085},{"interface":"front","average":{"1min":1,"5min":1.0129999999999999,"15min":0.998},"min":{"1min":0.75700000000000001,"5min":0.64800000000000002,"15min":0.41999999999999998},"max":{"1min":1.3680000000000001,"5min":1.694,"15min":1.8859999999999999},"last":0.81499999999999995}]},{"osd":5,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0449999999999999,"15min":1.0780000000000001},"min":{"1min":0.64500000000000002,"5min":0.59399999999999997,"15min":0.49099999999999999},"max":{"1min":1.375,"5min":1.6379999999999999,"15min":2.2120000000000002},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.228,"15min":1.1539999999999999},"min":{"1min":1.0900000000000001,"5min":0.89900000000000002,"15min":0.69899999999999995},"max":{"1min":1.4330000000000001,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.262}]},{"osd":6,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.03,"15min":1.0900000000000001},"min":{"1min":0.72999999999999998,"5min":0.65100000000000002,"15min":0.63600000000000001},"max":{"1min":1.4350000000000001,"5min":1.478,"15min":1.802},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.127,"5min":1.1419999999999999,"15min":1.1919999999999999},"min":{"1min":0.91200000000000003,"5min":0.879,"15min":0.83799999999999997},"max":{"1min":1.48,"5min":1.6639999999999999,"15min":1.881},"last":1.117}]}]},{"osd":6,"up_from":14,"seq":60129542417,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389540,"kb_used_data":300124,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":93982300,"statfs":{"total":96636764160,"available":96237875200,"internally_reserved":0,"allocated":307326976,"data_stored":301990625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":91552201},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.38600000000000001},"max":{"1min":1.2569999999999999,"5min":5.8760000000000003,"15min":5.8760000000000003},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.85699999999999998,"15min":0.84499999999999997},"min":{"1min":0.55200000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.194,"5min":1.3919999999999999,"15min":1.673},"last":0.91900000000000004}]},{"osd":1,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.85099999999999998,"15min":0.91200000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2290000000000001,"5min":1.6379999999999999,"15min":5.0949999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.94999999999999996,"15min":0.92500000000000004},"min":{"1min":0.75600000000000001,"5min":0.56899999999999995,"15min":0.45500000000000002},"max":{"1min":1.321,"5min":1.423,"15min":5.0519999999999996},"last":0.79000000000000004}]},{"osd":2,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.90600000000000003,"15min":0.85199999999999998},"min":{"1min":0.66600000000000004,"5min":0.54400000000000004,"15min":0.51800000000000002},"max":{"1min":1.306,"5min":2.649,"15min":2.649},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.01,"15min":0.93600000000000005},"min":{"1min":0.82499999999999996,"5min":0.59699999999999998,"15min":0.44700000000000001},"max":{"1min":1.4299999999999999,"5min":2.282,"15min":2.282},"last":1.121}]},{"osd":3,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.87,"15min":0.93899999999999995},"min":{"1min":0.58899999999999997,"5min":0.51700000000000002,"15min":0.40200000000000002},"max":{"1min":1.1739999999999999,"5min":1.427,"15min":1.929},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.878,"5min":0.96399999999999997,"15min":0.99099999999999999},"min":{"1min":0.64300000000000002,"5min":0.58499999999999996,"15min":0.42699999999999999},"max":{"1min":1.2470000000000001,"5min":1.444,"15min":1.7589999999999999},"last":0.84599999999999997}]},{"osd":4,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0269999999999999,"15min":0.999},"min":{"1min":0.63600000000000001,"5min":0.63100000000000001,"15min":0.27600000000000002},"max":{"1min":1.5309999999999999,"5min":1.617,"15min":1.9219999999999999},"last":1.206},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0489999999999999,"15min":1.0269999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.32400000000000001},"max":{"1min":1.2809999999999999,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.056}]},{"osd":5,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.97199999999999998,"15min":1.022},"min":{"1min":0.67300000000000004,"5min":0.66300000000000003,"15min":0.29599999999999999},"max":{"1min":1.232,"5min":1.619,"15min":2.9529999999999998},"last":1.016},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0780000000000001,"15min":1.1220000000000001},"min":{"1min":0.85999999999999999,"5min":0.65300000000000002,"15min":0.34399999999999997},"max":{"1min":1.278,"5min":1.764,"15min":2.5680000000000001},"last":1.0920000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.167,"15min":1.145},"min":{"1min":0.80100000000000005,"5min":0.79300000000000004,"15min":0.37},"max":{"1min":1.482,"5min":1.7689999999999999,"15min":2.0350000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.216,"15min":1.194},"min":{"1min":0.95299999999999996,"5min":0.82099999999999995,"15min":0.46800000000000003},"max":{"1min":1.5940000000000001,"5min":1.7829999999999999,"15min":2.274},"last":1.292}]}]},{"osd":4,"up_from":14,"seq":60129542416,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408876,"kb_used_data":319396,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":93962964,"statfs":{"total":96636764160,"available":96218075136,"internally_reserved":0,"allocated":327061504,"data_stored":318875235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":91617734},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78100000000000003,"15min":0.80700000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.437},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.5649999999999999},"last":0.871},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.49299999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.165,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.028}]},{"osd":1,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.874,"15min":0.91500000000000004},"min":{"1min":0.55800000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.28,"5min":1.3069999999999999,"15min":2.4910000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85699999999999998,"15min":0.84899999999999998},"min":{"1min":0.58899999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.2130000000000001,"5min":1.2390000000000001,"15min":1.9730000000000001},"last":0.82899999999999996}]},{"osd":2,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.93999999999999995,"15min":0.95399999999999996},"min":{"1min":0.59599999999999997,"5min":0.55900000000000005,"15min":0.53400000000000003},"max":{"1min":1.2669999999999999,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.0509999999999999,"15min":1.052},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.55600000000000005},"max":{"1min":1.1859999999999999,"5min":4.6879999999999997,"15min":4.6879999999999997},"last":0.94999999999999996}]},{"osd":3,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.91600000000000004,"15min":0.93300000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.27,"5min":2.266,"15min":2.5819999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.96299999999999997,"15min":0.91400000000000003},"min":{"1min":0.68500000000000005,"5min":0.54300000000000004,"15min":0.53500000000000003},"max":{"1min":1.282,"5min":1.411,"15min":1.6890000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.93899999999999995,"15min":1.002},"min":{"1min":0.67700000000000005,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.3520000000000001,"5min":2.1779999999999999,"15min":2.3180000000000001},"last":1.125},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1080000000000001,"15min":1.0760000000000001},"min":{"1min":1.012,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.478,"5min":3.173,"15min":3.173},"last":1.204}]},{"osd":6,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.012,"15min":1.087},"min":{"1min":0.73099999999999998,"5min":0.66600000000000004,"15min":0.59299999999999997},"max":{"1min":1.2649999999999999,"5min":1.641,"15min":2.73},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.073,"15min":1.0800000000000001},"min":{"1min":0.93899999999999995,"5min":0.748,"15min":0.56499999999999995},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":4.7130000000000001},"last":1.081}]},{"osd":7,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.86799999999999999,"5min":0.68899999999999995,"15min":0.64900000000000002},"max":{"1min":1.3440000000000001,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.161,"15min":1.129},"min":{"1min":0.95399999999999996,"5min":0.70299999999999996,"15min":0.65300000000000002},"max":{"1min":1.4370000000000001,"5min":1.766,"15min":3.0649999999999999},"last":1.1639999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542417,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":286224,"kb_used_data":241160,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94085616,"statfs":{"total":96636764160,"available":96343670784,"internally_reserved":0,"allocated":246947840,"data_stored":241832282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.51200000000000001,"15min":0.48499999999999999},"min":{"1min":0.23999999999999999,"5min":0.23200000000000001,"15min":0.22700000000000001},"max":{"1min":0.55000000000000004,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.58099999999999996,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.28699999999999998},"max":{"1min":0.64700000000000002,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":0.79500000000000004}]},{"osd":1,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":0.61199999999999999,"5min":3.5059999999999998,"15min":4.5019999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.35299999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.53400000000000003,"5min":3.5430000000000001,"15min":3.5430000000000001},"last":0.72199999999999998}]},{"osd":3,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.76400000000000001,"15min":0.60399999999999998},"min":{"1min":0.309,"5min":0.26800000000000002,"15min":0.26100000000000001},"max":{"1min":0.71499999999999997,"5min":18.747,"15min":18.747},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.86199999999999999,"15min":0.69999999999999996},"min":{"1min":0.42599999999999999,"5min":0.38800000000000001,"15min":0.38300000000000001},"max":{"1min":0.81000000000000005,"5min":18.863,"15min":18.863},"last":0.68999999999999995}]},{"osd":4,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.85899999999999999,"15min":0.85999999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0620000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.95199999999999996,"15min":0.89900000000000002},"min":{"1min":0.59899999999999998,"5min":0.57599999999999996,"15min":0.48299999999999998},"max":{"1min":1.0029999999999999,"5min":9.6289999999999996,"15min":9.6289999999999996},"last":0.753}]},{"osd":5,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.88,"15min":0.84799999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98099999999999998,"5min":9.7289999999999992,"15min":9.7289999999999992},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.9430000000000001,"15min":1.2},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.51900000000000002},"max":{"1min":1.0049999999999999,"5min":91.072999999999993,"15min":91.072999999999993},"last":0.91600000000000004}]},{"osd":6,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":1.046,"15min":0.91500000000000004},"min":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.52800000000000002},"max":{"1min":1.137,"5min":19.126000000000001,"15min":19.126000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.86699999999999999,"5min":1.26,"15min":1.008},"min":{"1min":0.67600000000000005,"5min":0.60299999999999998,"15min":0.59399999999999997},"max":{"1min":1.21,"5min":21.469999999999999,"15min":21.469999999999999},"last":0.97299999999999998}]},{"osd":7,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.94499999999999995,"15min":0.89800000000000002},"min":{"1min":0.628,"5min":0.61299999999999999,"15min":0.52100000000000002},"max":{"1min":1.159,"5min":9.6869999999999994,"15min":9.6869999999999994},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.91600000000000004,"5min":1.147,"15min":0.97299999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.53200000000000003},"max":{"1min":1.129,"5min":19.050999999999998,"15min":19.050999999999998},"last":1.014}]}]},{"osd":0,"up_from":14,"seq":60129542418,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422368,"kb_used_data":341208,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":93949472,"statfs":{"total":96636764160,"available":96204259328,"internally_reserved":0,"allocated":349396992,"data_stored":341144786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":83098056},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 12:56:40 2023","interfaces":[{"interface":"back","average":{"1min":2.2690000000000001,"5min":0.82999999999999996,"15min":0.60699999999999998},"min":{"1min":0.317,"5min":0.25900000000000001,"15min":0.248},"max":{"1min":38.555999999999997,"5min":38.555999999999997,"15min":38.555999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.60799999999999998,"15min":0.64600000000000002},"min":{"1min":0.32500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":11.653,"5min":11.653,"15min":35.084000000000003},"last":0.52900000000000003}]},{"osd":2,"last update":"Mon Oct 30 12:56:40 2023","interfaces":[{"interface":"back","average":{"1min":2.3090000000000002,"5min":0.878,"15min":0.755},"min":{"1min":0.36199999999999999,"5min":0.312,"15min":0.29799999999999999},"max":{"1min":38.755000000000003,"5min":38.755000000000003,"15min":38.755000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":2.3620000000000001,"5min":0.93799999999999994,"15min":0.81200000000000006},"min":{"1min":0.434,"5min":0.37,"15min":0.35299999999999998},"max":{"1min":38.506999999999998,"5min":38.506999999999998,"15min":38.506999999999998},"last":0.54700000000000004}]},{"osd":3,"last update":"Mon Oct 30 12:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.9079999999999999,"5min":0.81100000000000005,"15min":0.64600000000000002},"min":{"1min":0.38600000000000001,"5min":0.32000000000000001,"15min":0.29499999999999998},"max":{"1min":29.565000000000001,"5min":29.565000000000001,"15min":29.565000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":1.0880000000000001,"5min":0.68300000000000005,"15min":0.66300000000000003},"min":{"1min":0.44900000000000001,"5min":0.38200000000000001,"15min":0.33400000000000002},"max":{"1min":11.609,"5min":11.609,"15min":11.916},"last":0.60099999999999998}]},{"osd":4,"last update":"Mon Oct 30 12:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.05,"15min":0.86799999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.373},"max":{"1min":21.550999999999998,"5min":21.550999999999998,"15min":21.550999999999998},"last":0.92600000000000005},{"interface":"front","average":{"1min":2.7440000000000002,"5min":1.2110000000000001,"15min":1.056},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.39900000000000002},"max":{"1min":38.798000000000002,"5min":38.798000000000002,"15min":38.798000000000002},"last":1.036}]},{"osd":5,"last update":"Mon Oct 30 12:56:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":0.92800000000000005,"15min":0.88300000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.5},"max":{"1min":11.683999999999999,"5min":11.683999999999999,"15min":11.683999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.48799999999999999},"max":{"1min":11.714,"5min":11.714,"15min":34.698},"last":1.008}]},{"osd":6,"last update":"Mon Oct 30 12:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":0.95799999999999996,"15min":0.85999999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.47799999999999998},"max":{"1min":16.495999999999999,"5min":16.495999999999999,"15min":16.495999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.044,"5min":0.92000000000000004,"15min":0.90000000000000002},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.50900000000000001},"max":{"1min":5.6479999999999997,"5min":5.6479999999999997,"15min":8.8800000000000008},"last":1.0600000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.853,"5min":1.032,"15min":0.90700000000000003},"min":{"1min":0.65600000000000003,"5min":0.53000000000000003,"15min":0.437},"max":{"1min":21.462,"5min":21.462,"15min":21.462},"last":1.0229999999999999},{"interface":"front","average":{"1min":2.2930000000000001,"5min":1.173,"15min":1.0269999999999999},"min":{"1min":0.74199999999999999,"5min":0.60599999999999998,"15min":0.55900000000000005},"max":{"1min":28.943999999999999,"5min":28.943999999999999,"15min":28.943999999999999},"last":0.872}]}]},{"osd":5,"up_from":14,"seq":60129542416,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461024,"kb_used_data":379864,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":93910816,"statfs":{"total":96636764160,"available":96164675584,"internally_reserved":0,"allocated":388980736,"data_stored":380853345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":83098057},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":0.95099999999999996,"15min":0.86399999999999999},"min":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.41899999999999998},"max":{"1min":7.9409999999999998,"5min":7.9409999999999998,"15min":7.9409999999999998},"last":1.097},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.871,"15min":0.85099999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.3169999999999999,"5min":1.341,"15min":1.468},"last":1.1930000000000001}]},{"osd":1,"last update":"Mon Oct 30 12:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":0.91100000000000003,"15min":0.93100000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":12.224,"5min":12.224,"15min":12.224},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.84499999999999997,"15min":0.878},"min":{"1min":0.54000000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.0629999999999999}]},{"osd":2,"last update":"Mon Oct 30 12:56:41 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.90000000000000002,"15min":0.878},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.501},"max":{"1min":2.2770000000000001,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.90200000000000002,"15min":0.85499999999999998},"min":{"1min":0.60399999999999998,"5min":0.54600000000000004,"15min":0.442},"max":{"1min":5.0650000000000004,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":0.91200000000000003}]},{"osd":3,"last update":"Mon Oct 30 12:56:41 2023","interfaces":[{"interface":"back","average":{"1min":2.2879999999999998,"5min":1.2090000000000001,"15min":1.407},"min":{"1min":0.66700000000000004,"5min":0.54400000000000004,"15min":0.504},"max":{"1min":31.815000000000001,"5min":31.815000000000001,"15min":104.88500000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.089,"15min":1.395},"min":{"1min":0.73399999999999999,"5min":0.69099999999999995,"15min":0.60699999999999998},"max":{"1min":8.9019999999999992,"5min":8.9019999999999992,"15min":105.042},"last":1.238}]},{"osd":4,"last update":"Mon Oct 30 12:56:41 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.05,"15min":0.95199999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.38500000000000001},"max":{"1min":1.4159999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.069,"5min":1.0629999999999999,"15min":1.125},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.57399999999999995},"max":{"1min":1.3280000000000001,"5min":1.5229999999999999,"15min":1.712},"last":0.97199999999999998}]},{"osd":6,"last update":"Mon Oct 30 12:56:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.97499999999999998,"15min":0.95899999999999996},"min":{"1min":0.65100000000000002,"5min":0.627,"15min":0.35899999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.9410000000000001},"last":0.752},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.089,"15min":1.0640000000000001},"min":{"1min":0.83799999999999997,"5min":0.72199999999999998,"15min":0.48899999999999999},"max":{"1min":2.9049999999999998,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.99399999999999999}]},{"osd":7,"last update":"Mon Oct 30 12:56:41 2023","interfaces":[{"interface":"back","average":{"1min":2.1030000000000002,"5min":1.3160000000000001,"15min":1.1699999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.40600000000000003},"max":{"1min":24.173999999999999,"5min":24.173999999999999,"15min":24.173999999999999},"last":1.018},{"interface":"front","average":{"1min":1.579,"5min":1.2130000000000001,"15min":1.1459999999999999},"min":{"1min":0.72899999999999998,"5min":0.70699999999999996,"15min":0.53300000000000003},"max":{"1min":12.055,"5min":12.055,"15min":12.055},"last":0.88600000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542418,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437640,"kb_used_data":352320,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":93934200,"statfs":{"total":96636764160,"available":96188620800,"internally_reserved":0,"allocated":360775680,"data_stored":352416873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":87357898},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.434,"15min":0.435},"min":{"1min":0.27900000000000003,"5min":0.25600000000000001,"15min":0.224},"max":{"1min":0.64600000000000002,"5min":0.77100000000000002,"15min":2.1459999999999999},"last":0.374},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.46999999999999997,"15min":0.54700000000000004},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.27200000000000002},"max":{"1min":0.66300000000000003,"5min":0.77900000000000003,"15min":21.495999999999999},"last":0.45900000000000002}]},{"osd":1,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.497,"15min":0.53200000000000003},"min":{"1min":0.32700000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.66100000000000003,"5min":1.0700000000000001,"15min":1.1459999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.49199999999999999,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.32600000000000001,"15min":0.30399999999999999},"max":{"1min":0.72899999999999998,"5min":0.83299999999999996,"15min":15.102},"last":0.40200000000000002}]},{"osd":2,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53200000000000003,"15min":0.60499999999999998},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.28299999999999997},"max":{"1min":0.68500000000000005,"5min":0.86399999999999999,"15min":21.321000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.41599999999999998,"5min":0.39300000000000002,"15min":0.379},"max":{"1min":0.74299999999999999,"5min":0.97099999999999997,"15min":1.681},"last":0.54700000000000004}]},{"osd":4,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.81399999999999995,"15min":0.89800000000000002},"min":{"1min":0.69399999999999995,"5min":0.49199999999999999,"15min":0.48499999999999999},"max":{"1min":1.0309999999999999,"5min":1.21,"15min":21.382999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.872,"15min":0.92000000000000004},"min":{"1min":0.71299999999999997,"5min":0.60399999999999998,"15min":0.46700000000000003},"max":{"1min":1.0580000000000001,"5min":1.1910000000000001,"15min":15.231999999999999},"last":0.76800000000000002}]},{"osd":5,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85599999999999998,"15min":0.90200000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.47799999999999998},"max":{"1min":1.282,"5min":1.282,"15min":15.273999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91100000000000003,"15min":0.95899999999999996},"min":{"1min":0.71399999999999997,"5min":0.67700000000000005,"15min":0.57499999999999996},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":15.359999999999999},"last":0.78800000000000003}]},{"osd":6,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.59299999999999997,"5min":0.53200000000000003,"15min":0.51100000000000001},"max":{"1min":1.091,"5min":1.296,"15min":1.5620000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82599999999999996,"15min":0.88600000000000001},"min":{"1min":0.66400000000000003,"5min":0.59799999999999998,"15min":0.498},"max":{"1min":1.1279999999999999,"5min":1.198,"15min":15.313000000000001},"last":0.82599999999999996}]},{"osd":7,"last update":"Mon Oct 30 12:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.85799999999999998,"15min":0.93000000000000005},"min":{"1min":0.67500000000000004,"5min":0.58799999999999997,"15min":0.55200000000000005},"max":{"1min":1.177,"5min":1.177,"15min":21.428999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.88,"15min":0.88200000000000001},"min":{"1min":0.71899999999999997,"5min":0.56899999999999995,"15min":0.56799999999999995},"max":{"1min":1.1439999999999999,"5min":1.28,"15min":1.3100000000000001},"last":0.80200000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542416,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":231864,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":237428736,"data_stored":232443526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77330888},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.68899999999999995,"15min":0.54500000000000004},"min":{"1min":0.28899999999999998,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.68799999999999994,"5min":15.343999999999999,"15min":15.343999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.73799999999999999,"15min":0.60299999999999998},"min":{"1min":0.39600000000000002,"5min":0.313,"15min":0.29899999999999999},"max":{"1min":1.091,"5min":14.68,"15min":14.68},"last":0.41599999999999998}]},{"osd":2,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.67900000000000005,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.308,"15min":0.29599999999999999},"max":{"1min":0.79900000000000004,"5min":8.1359999999999992,"15min":8.1359999999999992},"last":0.439},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.27300000000000002},"max":{"1min":1.3560000000000001,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.33900000000000002}]},{"osd":3,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.64400000000000002,"15min":0.59299999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.044,"5min":8.0920000000000005,"15min":8.0920000000000005},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.32000000000000001},"max":{"1min":0.73699999999999999,"5min":5.9199999999999999,"15min":5.9199999999999999},"last":0.39000000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":1.131,"15min":0.93999999999999995},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.41799999999999998},"max":{"1min":1.1579999999999999,"5min":21.27,"15min":21.27},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.875,"5min":0.94899999999999995,"15min":0.90600000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.53600000000000003},"max":{"1min":1.145,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.91800000000000004}]},{"osd":5,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83499999999999996,"15min":0.82599999999999996},"min":{"1min":0.63100000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1990000000000001,"5min":1.355,"15min":1.355},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.85899999999999999,"15min":0.85599999999999998},"min":{"1min":0.63700000000000001,"5min":0.499,"15min":0.46200000000000002},"max":{"1min":1.1299999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.0740000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.84599999999999997,"15min":0.84099999999999997},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.51300000000000001},"max":{"1min":1.167,"5min":1.29,"15min":1.4450000000000001},"last":1.05},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.88100000000000001,"15min":0.88400000000000001},"min":{"1min":0.71199999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2350000000000001,"5min":1.323,"15min":1.468},"last":0.95699999999999996}]},{"osd":7,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.85499999999999998},"min":{"1min":0.68300000000000005,"5min":0.629,"15min":0.496},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.4299999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.87},"min":{"1min":0.623,"5min":0.623,"15min":0.443},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":1.002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":75235328,"data_stored":75218306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":97566720,"data_stored":97541268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42975232,"data_stored":42953409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":85385216,"data_stored":85363568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":88555520,"data_stored":88540092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":89763840,"data_stored":89741073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":111661056,"data_stored":111638700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":130834432,"data_stored":130815658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272936960,"data_stored":265027314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139231232,"data_stored":134593460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":198570075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":274759680,"data_stored":266744507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":237875200,"data_stored":230026345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":298586112,"data_stored":290803474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":195035136,"data_stored":190043127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":280907776,"data_stored":273177138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:57:23.409 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:23.409 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:23.410 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-10-30T12:35:05.142552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=35, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=56, tm_sec=16, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:23.411 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-30T12:57:39.034 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:57:39.039 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:57:43.412 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:57:43.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.562+0000 7f85fdf94640 1 Processor -- start 2023-10-30T12:57:43.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.562+0000 7f85fdf94640 1 -- start start 2023-10-30T12:57:43.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.562+0000 7f85fdf94640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f85f80bfec0 0x7f85f80c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.562+0000 7f85fdf94640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85f806a720 0x7f85f806abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.562+0000 7f85fdf94640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f8071290 con 0x7f85f8159e60 2023-10-30T12:57:43.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.562+0000 7f85fdf94640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f8071570 con 0x7f85f80bfec0 2023-10-30T12:57:43.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.562+0000 7f85fdf94640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f8071850 con 0x7f85f806a720 2023-10-30T12:57:43.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.562+0000 7f85f77fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f85f8159e60 0x7f85f806f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42454/0 (socket says 172.21.15.12:42454) 2023-10-30T12:57:43.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.562+0000 7f85f77fe640 1 -- 172.21.15.12:0/2967267645 learned_addr learned my addr 172.21.15.12:0/2967267645 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:57:43.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.562+0000 7f85f7fff640 1 --2- 172.21.15.12:0/2967267645 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85f806a720 0x7f85f806abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f6ffd640 1 --2- 172.21.15.12:0/2967267645 >> v2:172.21.15.154:3300/0 conn(0x7f85f80bfec0 0x7f85f80c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 -- 172.21.15.12:0/2967267645 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2095271582 0 0) 0x7f85f8071290 con 0x7f85f8159e60 2023-10-30T12:57:43.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 -- 172.21.15.12:0/2967267645 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85e0003530 con 0x7f85f8159e60 2023-10-30T12:57:43.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 -- 172.21.15.12:0/2967267645 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2090927067 0 0) 0x7f85e0003530 con 0x7f85f8159e60 2023-10-30T12:57:43.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 -- 172.21.15.12:0/2967267645 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85f806a720 msgr2=0x7f85f806abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:43.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 --2- 172.21.15.12:0/2967267645 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85f806a720 0x7f85f806abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 -- 172.21.15.12:0/2967267645 >> v2:172.21.15.154:3300/0 conn(0x7f85f80bfec0 msgr2=0x7f85f80c1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:43.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 --2- 172.21.15.12:0/2967267645 >> v2:172.21.15.154:3300/0 conn(0x7f85f80bfec0 0x7f85f80c1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 -- 172.21.15.12:0/2967267645 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f8071b30 con 0x7f85f8159e60 2023-10-30T12:57:43.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 -- 172.21.15.12:0/2967267645 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f85e8001680 con 0x7f85f8159e60 2023-10-30T12:57:43.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 -- 172.21.15.12:0/2967267645 >> v1:172.21.15.12:6789/0 conn(0x7f85f8159e60 legacy=0x7f85f806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 --2- 172.21.15.12:0/2967267645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e0003980 0x7f85e0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 --2- 172.21.15.12:0/2967267645 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85f806a720 0x7f85e00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 --2- 172.21.15.12:0/2967267645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85e0004d10 0x7f85e0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 -- 172.21.15.12:0/2967267645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f8071570 con 0x7f85e0004d10 2023-10-30T12:57:43.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 -- 172.21.15.12:0/2967267645 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f8071850 con 0x7f85e0003980 2023-10-30T12:57:43.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f67fc640 1 -- 172.21.15.12:0/2967267645 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e000b990 con 0x7f85f806a720 2023-10-30T12:57:43.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f7fff640 1 --2- 172.21.15.12:0/2967267645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85e0004d10 0x7f85e0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f6ffd640 1 --2- 172.21.15.12:0/2967267645 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85f806a720 0x7f85e00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f77fe640 1 --2- 172.21.15.12:0/2967267645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e0003980 0x7f85e0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f7fff640 1 -- 172.21.15.12:0/2967267645 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85f806a720 msgr2=0x7f85e00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:43.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f7fff640 1 --2- 172.21.15.12:0/2967267645 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85f806a720 0x7f85e00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f7fff640 1 -- 172.21.15.12:0/2967267645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e0003980 msgr2=0x7f85e0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:43.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f7fff640 1 --2- 172.21.15.12:0/2967267645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e0003980 0x7f85e0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f7fff640 1 -- 172.21.15.12:0/2967267645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e0003530 con 0x7f85e0004d10 2023-10-30T12:57:43.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f6ffd640 1 --2- 172.21.15.12:0/2967267645 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85f806a720 0x7f85e00047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:43.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.566+0000 7f85f77fe640 1 --2- 172.21.15.12:0/2967267645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e0003980 0x7f85e0003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:43.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85f7fff640 1 --2- 172.21.15.12:0/2967267645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85e0004d10 0x7f85e0005190 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f85f0009fc0 tx=0x7f85f001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ad0f3346cecddc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85f67fc640 1 -- 172.21.15.12:0/2967267645 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f00a5c70 con 0x7f85e0004d10 2023-10-30T12:57:43.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85f67fc640 1 -- 172.21.15.12:0/2967267645 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f000f040 con 0x7f85e0004d10 2023-10-30T12:57:43.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85f67fc640 1 -- 172.21.15.12:0/2967267645 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f00ad440 con 0x7f85e0004d10 2023-10-30T12:57:43.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85fdf94640 1 -- 172.21.15.12:0/2967267645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85e0004d10 msgr2=0x7f85e0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85fdf94640 1 --2- 172.21.15.12:0/2967267645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85e0004d10 0x7f85e0005190 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f85f0009fc0 tx=0x7f85f001cc60 comp rx=0 tx=0).stop 2023-10-30T12:57:43.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85f7fff640 1 -- 172.21.15.12:0/2967267645 reap_dead start 2023-10-30T12:57:43.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85fdf94640 1 -- 172.21.15.12:0/2967267645 shutdown_connections 2023-10-30T12:57:43.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85fdf94640 1 -- 172.21.15.12:0/2967267645 >> 172.21.15.12:0/2967267645 conn(0x7f85f80c7910 msgr2=0x7f85f80c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:43.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85fdf94640 1 -- 172.21.15.12:0/2967267645 shutdown_connections 2023-10-30T12:57:43.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85fdf94640 1 -- 172.21.15.12:0/2967267645 wait complete. 2023-10-30T12:57:43.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85fdf94640 1 Processor -- start 2023-10-30T12:57:43.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85fdf94640 1 -- start start 2023-10-30T12:57:43.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85fdf94640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85f80c6730 0x7f85f80af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85fdf94640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85f81dfab0 0x7f85f81e4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.570+0000 7f85fdf94640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f80c6b10 0x7f85f81e4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85fdf94640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f81e0490 con 0x7f85f80c6b10 2023-10-30T12:57:43.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85fdf94640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f81e5010 con 0x7f85f81dfab0 2023-10-30T12:57:43.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85fdf94640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f81ee9e0 con 0x7f85f80c6730 2023-10-30T12:57:43.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f80c6b10 0x7f85f81e4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f80c6b10 0x7f85f81e4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51080/0 (socket says 172.21.15.12:51080) 2023-10-30T12:57:43.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85f7fff640 1 -- 172.21.15.12:0/3543101543 learned_addr learned my addr 172.21.15.12:0/3543101543 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:57:43.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85f6ffd640 1 --2- 172.21.15.12:0/3543101543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85f81dfab0 0x7f85f81e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85f77fe640 1 --2- 172.21.15.12:0/3543101543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85f80c6730 0x7f85f80af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85f7fff640 1 -- 172.21.15.12:0/3543101543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85f80c6730 msgr2=0x7f85f80af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85f7fff640 1 --2- 172.21.15.12:0/3543101543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85f80c6730 0x7f85f80af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85f7fff640 1 -- 172.21.15.12:0/3543101543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85f81dfab0 msgr2=0x7f85f81e4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85f7fff640 1 --2- 172.21.15.12:0/3543101543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85f81dfab0 0x7f85f81e4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85f7fff640 1 -- 172.21.15.12:0/3543101543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f81ef9b0 con 0x7f85f80c6b10 2023-10-30T12:57:43.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85f7fff640 1 --2- 172.21.15.12:0/3543101543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f80c6b10 0x7f85f81e4ad0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f85f00b0040 tx=0x7f85f0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85dffff640 1 -- 172.21.15.12:0/3543101543 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f00bc070 con 0x7f85f80c6b10 2023-10-30T12:57:43.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85dffff640 1 -- 172.21.15.12:0/3543101543 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f00039a0 con 0x7f85f80c6b10 2023-10-30T12:57:43.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85dffff640 1 -- 172.21.15.12:0/3543101543 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f0003b30 con 0x7f85f80c6b10 2023-10-30T12:57:43.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85fdf94640 1 -- 172.21.15.12:0/3543101543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f8158220 con 0x7f85f80c6b10 2023-10-30T12:57:43.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85fdf94640 1 -- 172.21.15.12:0/3543101543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f81eeed0 con 0x7f85f80c6b10 2023-10-30T12:57:43.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85fdf94640 1 -- 172.21.15.12:0/3543101543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f804e170 con 0x7f85f80c6b10 2023-10-30T12:57:43.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.574+0000 7f85dffff640 1 -- 172.21.15.12:0/3543101543 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f85f00b4040 con 0x7f85f80c6b10 2023-10-30T12:57:43.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.578+0000 7f85dffff640 1 --2- 172.21.15.12:0/3543101543 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f85e40664a0 0x7f85e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.578+0000 7f85dffff640 1 -- 172.21.15.12:0/3543101543 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f85f011e3f0 con 0x7f85f80c6b10 2023-10-30T12:57:43.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.578+0000 7f85f77fe640 1 --2- 172.21.15.12:0/3543101543 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f85e40664a0 0x7f85e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.578+0000 7f85f77fe640 1 --2- 172.21.15.12:0/3543101543 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f85e40664a0 0x7f85e4068960 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f85e8000c10 tx=0x7f85e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.582+0000 7f85dffff640 1 -- 172.21.15.12:0/3543101543 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f85f00eb430 con 0x7f85f80c6b10 2023-10-30T12:57:43.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.734+0000 7f85fdf94640 1 -- 172.21.15.12:0/3543101543 --> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f85f80c1520 con 0x7f85e40664a0 2023-10-30T12:57:43.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.758+0000 7f85dffff640 1 -- 172.21.15.12:0/3543101543 <== mgr.4105 v2:172.21.15.154:6832/1474604059 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294678 (secure 0 0 0) 0x7f85e8010920 con 0x7f85e40664a0 2023-10-30T12:57:43.764 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:57:43.766 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-10-30T12:57:43.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.758+0000 7f85fdf94640 1 -- 172.21.15.12:0/3543101543 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f85e40664a0 msgr2=0x7f85e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.758+0000 7f85fdf94640 1 --2- 172.21.15.12:0/3543101543 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f85e40664a0 0x7f85e4068960 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f85e8000c10 tx=0x7f85e8009040 comp rx=0 tx=0).stop 2023-10-30T12:57:43.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.758+0000 7f85fdf94640 1 -- 172.21.15.12:0/3543101543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f80c6b10 msgr2=0x7f85f81e4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.758+0000 7f85fdf94640 1 --2- 172.21.15.12:0/3543101543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f80c6b10 0x7f85f81e4ad0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f85f00b0040 tx=0x7f85f0009750 comp rx=0 tx=0).stop 2023-10-30T12:57:43.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.758+0000 7f85fdf94640 1 -- 172.21.15.12:0/3543101543 shutdown_connections 2023-10-30T12:57:43.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.758+0000 7f85fdf94640 1 --2- 172.21.15.12:0/3543101543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f80c6b10 0x7f85f81e4ad0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.758+0000 7f85fdf94640 1 --2- 172.21.15.12:0/3543101543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85f81dfab0 0x7f85f81e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.758+0000 7f85fdf94640 1 --2- 172.21.15.12:0/3543101543 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f85e40664a0 0x7f85e4068960 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.758+0000 7f85fdf94640 1 --2- 172.21.15.12:0/3543101543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f85f80c6730 0x7f85f80af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.758+0000 7f85fdf94640 1 -- 172.21.15.12:0/3543101543 >> 172.21.15.12:0/3543101543 conn(0x7f85f80c7910 msgr2=0x7f85f8158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:43.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.758+0000 7f85fdf94640 1 -- 172.21.15.12:0/3543101543 shutdown_connections 2023-10-30T12:57:43.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.758+0000 7f85fdf94640 1 -- 172.21.15.12:0/3543101543 wait complete. 2023-10-30T12:57:43.784 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":734,"stamp":"2023-10-30T12:57:43.442829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1286759173,"num_objects":10515,"num_object_clones":0,"num_object_copies":21030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10515,"num_whiteouts":0,"num_read":5687,"num_read_kb":1040706,"num_write":53371,"num_write_kb":2315965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":636231,"num_omap_keys":1188,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67476,"ondisk_log_size":67476,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3201672,"kb_used_data":2569224,"kb_used_omap":12,"kb_used_meta":632371,"kb_avail":751773048,"statfs":{"total":773094113280,"available":769815601152,"internally_reserved":0,"allocated":2630885376,"data_stored":2574448634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12729,"internal_metadata":647548487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004099"},"pg_stats":[{"pgid":"3.3f","version":"55'905","reported_seq":3085,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:31.315245+0000","last_change":"2023-10-30T12:57:31.315316+0000","last_active":"2023-10-30T12:57:31.315245+0000","last_peered":"2023-10-30T12:57:31.315245+0000","last_clean":"2023-10-30T12:57:31.315245+0000","last_became_active":"2023-10-30T12:35:06.222724+0000","last_became_peered":"2023-10-30T12:35:06.222724+0000","last_unstale":"2023-10-30T12:57:31.315245+0000","last_undegraded":"2023-10-30T12:57:31.315245+0000","last_fullsized":"2023-10-30T12:57:31.315245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'905","last_scrub_stamp":"2023-10-30T12:57:31.315148+0000","last_deep_scrub":"55'905","last_deep_scrub_stamp":"2023-10-30T12:57:31.315148+0000","last_clean_scrub_stamp":"2023-10-30T12:57:31.315148+0000","objects_scrubbed":168,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:00:58.889518+0000","scrub_duration":0.33412740600000002,"objects_trimmed":0,"snaptrim_duration":0.0059524979999999996,"stat_sum":{"num_bytes":14421277,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":73,"num_read_kb":8207,"num_write":692,"num_write_kb":25708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"55'855","reported_seq":3032,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:38.668823+0000","last_change":"2023-10-30T12:56:38.668869+0000","last_active":"2023-10-30T12:56:38.668823+0000","last_peered":"2023-10-30T12:56:38.668823+0000","last_clean":"2023-10-30T12:56:38.668823+0000","last_became_active":"2023-10-30T12:35:06.182441+0000","last_became_peered":"2023-10-30T12:35:06.182441+0000","last_unstale":"2023-10-30T12:56:38.668823+0000","last_undegraded":"2023-10-30T12:56:38.668823+0000","last_fullsized":"2023-10-30T12:56:38.668823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'855","last_scrub_stamp":"2023-10-30T12:56:38.668744+0000","last_deep_scrub":"55'855","last_deep_scrub_stamp":"2023-10-30T12:56:38.668744+0000","last_clean_scrub_stamp":"2023-10-30T12:56:38.668744+0000","objects_scrubbed":154,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:43:43.147201+0000","scrub_duration":0.17361679199999999,"objects_trimmed":0,"snaptrim_duration":0.0026942979999999999,"stat_sum":{"num_bytes":14617239,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":70,"num_read_kb":10,"num_write":675,"num_write_kb":36983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"55'856","reported_seq":3043,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:55.585283+0000","last_change":"2023-10-30T12:56:55.585332+0000","last_active":"2023-10-30T12:56:55.585283+0000","last_peered":"2023-10-30T12:56:55.585283+0000","last_clean":"2023-10-30T12:56:55.585283+0000","last_became_active":"2023-10-30T12:35:06.180132+0000","last_became_peered":"2023-10-30T12:35:06.180132+0000","last_unstale":"2023-10-30T12:56:55.585283+0000","last_undegraded":"2023-10-30T12:56:55.585283+0000","last_fullsized":"2023-10-30T12:56:55.585283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'856","last_scrub_stamp":"2023-10-30T12:56:55.585201+0000","last_deep_scrub":"55'856","last_deep_scrub_stamp":"2023-10-30T12:56:55.585201+0000","last_clean_scrub_stamp":"2023-10-30T12:56:55.585201+0000","objects_scrubbed":159,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:09:47.246202+0000","scrub_duration":0.184201582,"objects_trimmed":0,"snaptrim_duration":0.003537278,"stat_sum":{"num_bytes":22250243,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":80,"num_read_kb":12305,"num_write":653,"num_write_kb":42611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"55'857","reported_seq":3023,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:00.673573+0000","last_change":"2023-10-30T12:57:00.673573+0000","last_active":"2023-10-30T12:57:00.673573+0000","last_peered":"2023-10-30T12:57:00.673573+0000","last_clean":"2023-10-30T12:57:00.673573+0000","last_became_active":"2023-10-30T12:35:06.213987+0000","last_became_peered":"2023-10-30T12:35:06.213987+0000","last_unstale":"2023-10-30T12:57:00.673573+0000","last_undegraded":"2023-10-30T12:57:00.673573+0000","last_fullsized":"2023-10-30T12:57:00.673573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'857","last_scrub_stamp":"2023-10-30T12:57:00.673331+0000","last_deep_scrub":"55'857","last_deep_scrub_stamp":"2023-10-30T12:57:00.673331+0000","last_clean_scrub_stamp":"2023-10-30T12:57:00.673331+0000","objects_scrubbed":175,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:32:46.251054+0000","scrub_duration":0.34115118,"objects_trimmed":0,"snaptrim_duration":0.013528057,"stat_sum":{"num_bytes":18553875,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":57,"num_read_kb":8205,"num_write":627,"num_write_kb":28534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"55'821","reported_seq":2978,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:13.280928+0000","last_change":"2023-10-30T12:57:13.280928+0000","last_active":"2023-10-30T12:57:13.280928+0000","last_peered":"2023-10-30T12:57:13.280928+0000","last_clean":"2023-10-30T12:57:13.280928+0000","last_became_active":"2023-10-30T12:35:06.214964+0000","last_became_peered":"2023-10-30T12:35:06.214964+0000","last_unstale":"2023-10-30T12:57:13.280928+0000","last_undegraded":"2023-10-30T12:57:13.280928+0000","last_fullsized":"2023-10-30T12:57:13.280928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'821","last_scrub_stamp":"2023-10-30T12:57:13.280603+0000","last_deep_scrub":"55'821","last_deep_scrub_stamp":"2023-10-30T12:57:13.280603+0000","last_clean_scrub_stamp":"2023-10-30T12:57:13.280603+0000","objects_scrubbed":151,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:01:06.258406+0000","scrub_duration":0.33545699899999998,"objects_trimmed":0,"snaptrim_duration":0.0046834600000000004,"stat_sum":{"num_bytes":8686005,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":51,"num_read_kb":4112,"num_write":616,"num_write_kb":25845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"55'857","reported_seq":3024,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:52.695935+0000","last_change":"2023-10-30T12:56:52.695935+0000","last_active":"2023-10-30T12:56:52.695935+0000","last_peered":"2023-10-30T12:56:52.695935+0000","last_clean":"2023-10-30T12:56:52.695935+0000","last_became_active":"2023-10-30T12:35:06.214219+0000","last_became_peered":"2023-10-30T12:35:06.214219+0000","last_unstale":"2023-10-30T12:56:52.695935+0000","last_undegraded":"2023-10-30T12:56:52.695935+0000","last_fullsized":"2023-10-30T12:56:52.695935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'857","last_scrub_stamp":"2023-10-30T12:56:52.695649+0000","last_deep_scrub":"55'857","last_deep_scrub_stamp":"2023-10-30T12:56:52.695649+0000","last_clean_scrub_stamp":"2023-10-30T12:56:52.695649+0000","objects_scrubbed":170,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:36:31.854780+0000","scrub_duration":0.32795837700000002,"objects_trimmed":0,"snaptrim_duration":0.015862434000000002,"stat_sum":{"num_bytes":14811547,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":61,"num_read_kb":16,"num_write":643,"num_write_kb":33253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"55'853","reported_seq":3010,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:01.292711+0000","last_change":"2023-10-30T12:57:01.292711+0000","last_active":"2023-10-30T12:57:01.292711+0000","last_peered":"2023-10-30T12:57:01.292711+0000","last_clean":"2023-10-30T12:57:01.292711+0000","last_became_active":"2023-10-30T12:35:06.216468+0000","last_became_peered":"2023-10-30T12:35:06.216468+0000","last_unstale":"2023-10-30T12:57:01.292711+0000","last_undegraded":"2023-10-30T12:57:01.292711+0000","last_fullsized":"2023-10-30T12:57:01.292711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'853","last_scrub_stamp":"2023-10-30T12:57:01.292372+0000","last_deep_scrub":"55'853","last_deep_scrub_stamp":"2023-10-30T12:57:01.292372+0000","last_clean_scrub_stamp":"2023-10-30T12:57:01.292372+0000","objects_scrubbed":156,"log_size":853,"log_dups_size":0,"ondisk_log_size":853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:07:43.317255+0000","scrub_duration":0.33415243100000003,"objects_trimmed":0,"snaptrim_duration":0.0031309480000000002,"stat_sum":{"num_bytes":9595467,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":52,"num_read_kb":12216,"num_write":649,"num_write_kb":36101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"55'786","reported_seq":2948,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:09.776112+0000","last_change":"2023-10-30T12:57:09.776112+0000","last_active":"2023-10-30T12:57:09.776112+0000","last_peered":"2023-10-30T12:57:09.776112+0000","last_clean":"2023-10-30T12:57:09.776112+0000","last_became_active":"2023-10-30T12:35:06.214384+0000","last_became_peered":"2023-10-30T12:35:06.214384+0000","last_unstale":"2023-10-30T12:57:09.776112+0000","last_undegraded":"2023-10-30T12:57:09.776112+0000","last_fullsized":"2023-10-30T12:57:09.776112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'786","last_scrub_stamp":"2023-10-30T12:57:09.775984+0000","last_deep_scrub":"55'786","last_deep_scrub_stamp":"2023-10-30T12:57:09.775984+0000","last_clean_scrub_stamp":"2023-10-30T12:57:09.775984+0000","objects_scrubbed":161,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:50:00.794904+0000","scrub_duration":0.29424167899999998,"objects_trimmed":0,"snaptrim_duration":0.0045552559999999997,"stat_sum":{"num_bytes":9825651,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":56,"num_read_kb":15,"num_write":593,"num_write_kb":23392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"55'836","reported_seq":3013,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.158188+0000","last_change":"2023-10-30T12:57:21.158288+0000","last_active":"2023-10-30T12:57:21.158188+0000","last_peered":"2023-10-30T12:57:21.158188+0000","last_clean":"2023-10-30T12:57:21.158188+0000","last_became_active":"2023-10-30T12:35:06.222692+0000","last_became_peered":"2023-10-30T12:35:06.222692+0000","last_unstale":"2023-10-30T12:57:21.158188+0000","last_undegraded":"2023-10-30T12:57:21.158188+0000","last_fullsized":"2023-10-30T12:57:21.158188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'836","last_scrub_stamp":"2023-10-30T12:57:21.158063+0000","last_deep_scrub":"55'836","last_deep_scrub_stamp":"2023-10-30T12:57:21.158063+0000","last_clean_scrub_stamp":"2023-10-30T12:57:21.158063+0000","objects_scrubbed":141,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:52:43.758949+0000","scrub_duration":0.25395279799999998,"objects_trimmed":0,"snaptrim_duration":0.0074617599999999996,"stat_sum":{"num_bytes":5524945,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":72,"num_read_kb":4115,"num_write":631,"num_write_kb":10887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"55'822","reported_seq":2991,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:38.051395+0000","last_change":"2023-10-30T12:56:38.051395+0000","last_active":"2023-10-30T12:56:38.051395+0000","last_peered":"2023-10-30T12:56:38.051395+0000","last_clean":"2023-10-30T12:56:38.051395+0000","last_became_active":"2023-10-30T12:35:06.215384+0000","last_became_peered":"2023-10-30T12:35:06.215384+0000","last_unstale":"2023-10-30T12:56:38.051395+0000","last_undegraded":"2023-10-30T12:56:38.051395+0000","last_fullsized":"2023-10-30T12:56:38.051395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'822","last_scrub_stamp":"2023-10-30T12:56:38.051202+0000","last_deep_scrub":"55'822","last_deep_scrub_stamp":"2023-10-30T12:56:38.051202+0000","last_clean_scrub_stamp":"2023-10-30T12:56:38.051202+0000","objects_scrubbed":148,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:06:45.015408+0000","scrub_duration":0.26303434999999997,"objects_trimmed":0,"snaptrim_duration":0.0080059940000000007,"stat_sum":{"num_bytes":17915594,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":64,"num_read_kb":28,"num_write":636,"num_write_kb":19744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"55'821","reported_seq":2993,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:38.940436+0000","last_change":"2023-10-30T12:56:38.940436+0000","last_active":"2023-10-30T12:56:38.940436+0000","last_peered":"2023-10-30T12:56:38.940436+0000","last_clean":"2023-10-30T12:56:38.940436+0000","last_became_active":"2023-10-30T12:35:06.215355+0000","last_became_peered":"2023-10-30T12:35:06.215355+0000","last_unstale":"2023-10-30T12:56:38.940436+0000","last_undegraded":"2023-10-30T12:56:38.940436+0000","last_fullsized":"2023-10-30T12:56:38.940436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'821","last_scrub_stamp":"2023-10-30T12:56:38.940294+0000","last_deep_scrub":"55'821","last_deep_scrub_stamp":"2023-10-30T12:56:38.940294+0000","last_clean_scrub_stamp":"2023-10-30T12:56:38.940294+0000","objects_scrubbed":135,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:00:12.683787+0000","scrub_duration":0.18969223099999999,"objects_trimmed":0,"snaptrim_duration":0.011778769,"stat_sum":{"num_bytes":10475791,"num_objects":135,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":67,"num_read_kb":8209,"num_write":651,"num_write_kb":25832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"55'780","reported_seq":2951,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:27.866741+0000","last_change":"2023-10-30T12:56:27.866741+0000","last_active":"2023-10-30T12:56:27.866741+0000","last_peered":"2023-10-30T12:56:27.866741+0000","last_clean":"2023-10-30T12:56:27.866741+0000","last_became_active":"2023-10-30T12:35:06.196503+0000","last_became_peered":"2023-10-30T12:35:06.196503+0000","last_unstale":"2023-10-30T12:56:27.866741+0000","last_undegraded":"2023-10-30T12:56:27.866741+0000","last_fullsized":"2023-10-30T12:56:27.866741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'780","last_scrub_stamp":"2023-10-30T12:56:27.866427+0000","last_deep_scrub":"55'780","last_deep_scrub_stamp":"2023-10-30T12:56:27.866427+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.866427+0000","objects_scrubbed":163,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:24:58.673270+0000","scrub_duration":0.27317086899999998,"objects_trimmed":0,"snaptrim_duration":0.0030385149999999999,"stat_sum":{"num_bytes":16276064,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":65,"num_read_kb":7648,"num_write":632,"num_write_kb":36085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"55'885","reported_seq":3068,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:01.675556+0000","last_change":"2023-10-30T12:57:01.675608+0000","last_active":"2023-10-30T12:57:01.675556+0000","last_peered":"2023-10-30T12:57:01.675556+0000","last_clean":"2023-10-30T12:57:01.675556+0000","last_became_active":"2023-10-30T12:35:06.195408+0000","last_became_peered":"2023-10-30T12:35:06.195408+0000","last_unstale":"2023-10-30T12:57:01.675556+0000","last_undegraded":"2023-10-30T12:57:01.675556+0000","last_fullsized":"2023-10-30T12:57:01.675556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'885","last_scrub_stamp":"2023-10-30T12:57:01.675469+0000","last_deep_scrub":"55'885","last_deep_scrub_stamp":"2023-10-30T12:57:01.675469+0000","last_clean_scrub_stamp":"2023-10-30T12:57:01.675469+0000","objects_scrubbed":161,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:01:54.627925+0000","scrub_duration":0.27230626800000002,"objects_trimmed":0,"snaptrim_duration":0.004560812,"stat_sum":{"num_bytes":30579039,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":77,"num_read_kb":9808,"num_write":665,"num_write_kb":49630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"55'828","reported_seq":3005,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:41.613067+0000","last_change":"2023-10-30T12:56:41.613105+0000","last_active":"2023-10-30T12:56:41.613067+0000","last_peered":"2023-10-30T12:56:41.613067+0000","last_clean":"2023-10-30T12:56:41.613067+0000","last_became_active":"2023-10-30T12:35:06.182573+0000","last_became_peered":"2023-10-30T12:35:06.182573+0000","last_unstale":"2023-10-30T12:56:41.613067+0000","last_undegraded":"2023-10-30T12:56:41.613067+0000","last_fullsized":"2023-10-30T12:56:41.613067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'828","last_scrub_stamp":"2023-10-30T12:56:41.612983+0000","last_deep_scrub":"55'828","last_deep_scrub_stamp":"2023-10-30T12:56:41.612983+0000","last_clean_scrub_stamp":"2023-10-30T12:56:41.612983+0000","objects_scrubbed":150,"log_size":828,"log_dups_size":0,"ondisk_log_size":828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:01:59.566492+0000","scrub_duration":0.154696469,"objects_trimmed":0,"snaptrim_duration":0.003020062,"stat_sum":{"num_bytes":14557524,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":71,"num_read_kb":4113,"num_write":642,"num_write_kb":31601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"55'865","reported_seq":3034,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:42.698759+0000","last_change":"2023-10-30T12:56:42.698825+0000","last_active":"2023-10-30T12:56:42.698759+0000","last_peered":"2023-10-30T12:56:42.698759+0000","last_clean":"2023-10-30T12:56:42.698759+0000","last_became_active":"2023-10-30T12:35:06.185290+0000","last_became_peered":"2023-10-30T12:35:06.185290+0000","last_unstale":"2023-10-30T12:56:42.698759+0000","last_undegraded":"2023-10-30T12:56:42.698759+0000","last_fullsized":"2023-10-30T12:56:42.698759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'865","last_scrub_stamp":"2023-10-30T12:56:42.698683+0000","last_deep_scrub":"55'865","last_deep_scrub_stamp":"2023-10-30T12:56:42.698683+0000","last_clean_scrub_stamp":"2023-10-30T12:56:42.698683+0000","objects_scrubbed":171,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:56.915839+0000","scrub_duration":0.293370403,"objects_trimmed":0,"snaptrim_duration":0.002282292,"stat_sum":{"num_bytes":14763187,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":63,"num_read_kb":11730,"num_write":643,"num_write_kb":27692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"55'877","reported_seq":3042,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:01.636658+0000","last_change":"2023-10-30T12:57:01.636781+0000","last_active":"2023-10-30T12:57:01.636658+0000","last_peered":"2023-10-30T12:57:01.636658+0000","last_clean":"2023-10-30T12:57:01.636658+0000","last_became_active":"2023-10-30T12:35:06.213627+0000","last_became_peered":"2023-10-30T12:35:06.213627+0000","last_unstale":"2023-10-30T12:57:01.636658+0000","last_undegraded":"2023-10-30T12:57:01.636658+0000","last_fullsized":"2023-10-30T12:57:01.636658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'877","last_scrub_stamp":"2023-10-30T12:57:01.636491+0000","last_deep_scrub":"55'877","last_deep_scrub_stamp":"2023-10-30T12:57:01.636491+0000","last_clean_scrub_stamp":"2023-10-30T12:57:01.636491+0000","objects_scrubbed":159,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:42:41.484120+0000","scrub_duration":0.27376288700000001,"objects_trimmed":0,"snaptrim_duration":0.014095760000000001,"stat_sum":{"num_bytes":9839966,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":59,"num_read_kb":4114,"num_write":651,"num_write_kb":31705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"55'914","reported_seq":3083,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:45.835829+0000","last_change":"2023-10-30T12:56:45.835874+0000","last_active":"2023-10-30T12:56:45.835829+0000","last_peered":"2023-10-30T12:56:45.835829+0000","last_clean":"2023-10-30T12:56:45.835829+0000","last_became_active":"2023-10-30T12:35:06.221696+0000","last_became_peered":"2023-10-30T12:35:06.221696+0000","last_unstale":"2023-10-30T12:56:45.835829+0000","last_undegraded":"2023-10-30T12:56:45.835829+0000","last_fullsized":"2023-10-30T12:56:45.835829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'914","last_scrub_stamp":"2023-10-30T12:56:45.835737+0000","last_deep_scrub":"55'914","last_deep_scrub_stamp":"2023-10-30T12:56:45.835737+0000","last_clean_scrub_stamp":"2023-10-30T12:56:45.835737+0000","objects_scrubbed":175,"log_size":914,"log_dups_size":0,"ondisk_log_size":914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:37:21.337050+0000","scrub_duration":0.41149325399999997,"objects_trimmed":0,"snaptrim_duration":0.015743672,"stat_sum":{"num_bytes":28343169,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":62,"num_read_kb":16401,"num_write":691,"num_write_kb":59410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"55'804","reported_seq":2992,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.435565+0000","last_change":"2023-10-30T12:56:24.435609+0000","last_active":"2023-10-30T12:56:24.435565+0000","last_peered":"2023-10-30T12:56:24.435565+0000","last_clean":"2023-10-30T12:56:24.435565+0000","last_became_active":"2023-10-30T12:35:06.198074+0000","last_became_peered":"2023-10-30T12:35:06.198074+0000","last_unstale":"2023-10-30T12:56:24.435565+0000","last_undegraded":"2023-10-30T12:56:24.435565+0000","last_fullsized":"2023-10-30T12:56:24.435565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'804","last_scrub_stamp":"2023-10-30T12:56:24.435488+0000","last_deep_scrub":"55'804","last_deep_scrub_stamp":"2023-10-30T12:56:24.435488+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.435488+0000","objects_scrubbed":156,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:44:11.442590+0000","scrub_duration":0.24729864700000001,"objects_trimmed":0,"snaptrim_duration":0.0023389270000000002,"stat_sum":{"num_bytes":28976684,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":82,"num_read_kb":27,"num_write":642,"num_write_kb":39937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"55'872","reported_seq":3039,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.322160+0000","last_change":"2023-10-30T12:57:12.322160+0000","last_active":"2023-10-30T12:57:12.322160+0000","last_peered":"2023-10-30T12:57:12.322160+0000","last_clean":"2023-10-30T12:57:12.322160+0000","last_became_active":"2023-10-30T12:35:06.215935+0000","last_became_peered":"2023-10-30T12:35:06.215935+0000","last_unstale":"2023-10-30T12:57:12.322160+0000","last_undegraded":"2023-10-30T12:57:12.322160+0000","last_fullsized":"2023-10-30T12:57:12.322160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'872","last_scrub_stamp":"2023-10-30T12:57:12.321838+0000","last_deep_scrub":"55'872","last_deep_scrub_stamp":"2023-10-30T12:57:12.321838+0000","last_clean_scrub_stamp":"2023-10-30T12:57:12.321838+0000","objects_scrubbed":166,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:50:28.308528+0000","scrub_duration":0.34909467599999999,"objects_trimmed":0,"snaptrim_duration":0.0062537060000000004,"stat_sum":{"num_bytes":21535467,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":60,"num_read_kb":12305,"num_write":656,"num_write_kb":39478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"55'805","reported_seq":2986,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:30.019160+0000","last_change":"2023-10-30T12:56:30.019160+0000","last_active":"2023-10-30T12:56:30.019160+0000","last_peered":"2023-10-30T12:56:30.019160+0000","last_clean":"2023-10-30T12:56:30.019160+0000","last_became_active":"2023-10-30T12:35:06.199812+0000","last_became_peered":"2023-10-30T12:35:06.199812+0000","last_unstale":"2023-10-30T12:56:30.019160+0000","last_undegraded":"2023-10-30T12:56:30.019160+0000","last_fullsized":"2023-10-30T12:56:30.019160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'805","last_scrub_stamp":"2023-10-30T12:56:30.018936+0000","last_deep_scrub":"55'805","last_deep_scrub_stamp":"2023-10-30T12:56:30.018936+0000","last_clean_scrub_stamp":"2023-10-30T12:56:30.018936+0000","objects_scrubbed":143,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:47:14.318405+0000","scrub_duration":0.21736029700000001,"objects_trimmed":0,"snaptrim_duration":0.011375481999999999,"stat_sum":{"num_bytes":6303882,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":75,"num_read_kb":20150,"num_write":647,"num_write_kb":36292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"55'899","reported_seq":3085,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:50.693274+0000","last_change":"2023-10-30T12:56:50.693274+0000","last_active":"2023-10-30T12:56:50.693274+0000","last_peered":"2023-10-30T12:56:50.693274+0000","last_clean":"2023-10-30T12:56:50.693274+0000","last_became_active":"2023-10-30T12:35:06.216861+0000","last_became_peered":"2023-10-30T12:35:06.216861+0000","last_unstale":"2023-10-30T12:56:50.693274+0000","last_undegraded":"2023-10-30T12:56:50.693274+0000","last_fullsized":"2023-10-30T12:56:50.693274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'899","last_scrub_stamp":"2023-10-30T12:56:50.693152+0000","last_deep_scrub":"55'899","last_deep_scrub_stamp":"2023-10-30T12:56:50.693152+0000","last_clean_scrub_stamp":"2023-10-30T12:56:50.693152+0000","objects_scrubbed":180,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:56:05.148326+0000","scrub_duration":0.23889443799999999,"objects_trimmed":0,"snaptrim_duration":0.0040841849999999997,"stat_sum":{"num_bytes":12370338,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":78,"num_read_kb":8213,"num_write":666,"num_write_kb":26377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"55'868","reported_seq":3032,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:33.615651+0000","last_change":"2023-10-30T12:56:33.615740+0000","last_active":"2023-10-30T12:56:33.615651+0000","last_peered":"2023-10-30T12:56:33.615651+0000","last_clean":"2023-10-30T12:56:33.615651+0000","last_became_active":"2023-10-30T12:35:46.122517+0000","last_became_peered":"2023-10-30T12:35:46.122517+0000","last_unstale":"2023-10-30T12:56:33.615651+0000","last_undegraded":"2023-10-30T12:56:33.615651+0000","last_fullsized":"2023-10-30T12:56:33.615651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'868","last_scrub_stamp":"2023-10-30T12:56:33.615559+0000","last_deep_scrub":"55'868","last_deep_scrub_stamp":"2023-10-30T12:56:33.615559+0000","last_clean_scrub_stamp":"2023-10-30T12:56:33.615559+0000","objects_scrubbed":170,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:51:10.318228+0000","scrub_duration":0.26875042100000002,"objects_trimmed":0,"snaptrim_duration":0.0020435240000000001,"stat_sum":{"num_bytes":19531616,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":61,"num_read_kb":21,"num_write":688,"num_write_kb":30111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"55'848","reported_seq":3035,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:42.567324+0000","last_change":"2023-10-30T12:56:42.567417+0000","last_active":"2023-10-30T12:56:42.567324+0000","last_peered":"2023-10-30T12:56:42.567324+0000","last_clean":"2023-10-30T12:56:42.567324+0000","last_became_active":"2023-10-30T12:35:06.182314+0000","last_became_peered":"2023-10-30T12:35:06.182314+0000","last_unstale":"2023-10-30T12:56:42.567324+0000","last_undegraded":"2023-10-30T12:56:42.567324+0000","last_fullsized":"2023-10-30T12:56:42.567324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'848","last_scrub_stamp":"2023-10-30T12:56:42.567236+0000","last_deep_scrub":"55'848","last_deep_scrub_stamp":"2023-10-30T12:56:42.567236+0000","last_clean_scrub_stamp":"2023-10-30T12:56:42.567236+0000","objects_scrubbed":161,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:44:52.946140+0000","scrub_duration":0.27460055700000002,"objects_trimmed":0,"snaptrim_duration":0.0051111200000000002,"stat_sum":{"num_bytes":5467082,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":80,"num_read_kb":15696,"num_write":658,"num_write_kb":25502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"55'829","reported_seq":3003,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:32.815869+0000","last_change":"2023-10-30T12:56:32.815869+0000","last_active":"2023-10-30T12:56:32.815869+0000","last_peered":"2023-10-30T12:56:32.815869+0000","last_clean":"2023-10-30T12:56:32.815869+0000","last_became_active":"2023-10-30T12:35:06.213402+0000","last_became_peered":"2023-10-30T12:35:06.213402+0000","last_unstale":"2023-10-30T12:56:32.815869+0000","last_undegraded":"2023-10-30T12:56:32.815869+0000","last_fullsized":"2023-10-30T12:56:32.815869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'829","last_scrub_stamp":"2023-10-30T12:56:32.815282+0000","last_deep_scrub":"55'829","last_deep_scrub_stamp":"2023-10-30T12:56:32.815282+0000","last_clean_scrub_stamp":"2023-10-30T12:56:32.815282+0000","objects_scrubbed":159,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:42:54.304491+0000","scrub_duration":0.307345284,"objects_trimmed":0,"snaptrim_duration":0.014055767,"stat_sum":{"num_bytes":10241279,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":67,"num_read_kb":4112,"num_write":657,"num_write_kb":20530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"55'838","reported_seq":3003,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:57.716386+0000","last_change":"2023-10-30T12:56:57.716444+0000","last_active":"2023-10-30T12:56:57.716386+0000","last_peered":"2023-10-30T12:56:57.716386+0000","last_clean":"2023-10-30T12:56:57.716386+0000","last_became_active":"2023-10-30T12:35:06.188298+0000","last_became_peered":"2023-10-30T12:35:06.188298+0000","last_unstale":"2023-10-30T12:56:57.716386+0000","last_undegraded":"2023-10-30T12:56:57.716386+0000","last_fullsized":"2023-10-30T12:56:57.716386+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'838","last_scrub_stamp":"2023-10-30T12:56:57.716307+0000","last_deep_scrub":"55'838","last_deep_scrub_stamp":"2023-10-30T12:56:57.716307+0000","last_clean_scrub_stamp":"2023-10-30T12:56:57.716307+0000","objects_scrubbed":149,"log_size":838,"log_dups_size":0,"ondisk_log_size":838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:19:18.533282+0000","scrub_duration":0.22493126999999999,"objects_trimmed":0,"snaptrim_duration":0.0053580889999999999,"stat_sum":{"num_bytes":5087264,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":59,"num_read_kb":8204,"num_write":621,"num_write_kb":44181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"55'896","reported_seq":3090,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.080071+0000","last_change":"2023-10-30T12:56:34.080071+0000","last_active":"2023-10-30T12:56:34.080071+0000","last_peered":"2023-10-30T12:56:34.080071+0000","last_clean":"2023-10-30T12:56:34.080071+0000","last_became_active":"2023-10-30T12:35:06.202507+0000","last_became_peered":"2023-10-30T12:35:06.202507+0000","last_unstale":"2023-10-30T12:56:34.080071+0000","last_undegraded":"2023-10-30T12:56:34.080071+0000","last_fullsized":"2023-10-30T12:56:34.080071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'896","last_scrub_stamp":"2023-10-30T12:56:34.079910+0000","last_deep_scrub":"55'896","last_deep_scrub_stamp":"2023-10-30T12:56:34.079910+0000","last_clean_scrub_stamp":"2023-10-30T12:56:34.079910+0000","objects_scrubbed":173,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:15:45.705065+0000","scrub_duration":0.25185000000000002,"objects_trimmed":0,"snaptrim_duration":0.0082500649999999991,"stat_sum":{"num_bytes":6587197,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":87,"num_read_kb":1892,"num_write":715,"num_write_kb":28794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"55'861","reported_seq":3031,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:46.673561+0000","last_change":"2023-10-30T12:56:46.673623+0000","last_active":"2023-10-30T12:56:46.673561+0000","last_peered":"2023-10-30T12:56:46.673561+0000","last_clean":"2023-10-30T12:56:46.673561+0000","last_became_active":"2023-10-30T12:35:06.170030+0000","last_became_peered":"2023-10-30T12:35:06.170030+0000","last_unstale":"2023-10-30T12:56:46.673561+0000","last_undegraded":"2023-10-30T12:56:46.673561+0000","last_fullsized":"2023-10-30T12:56:46.673561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'861","last_scrub_stamp":"2023-10-30T12:56:46.673491+0000","last_deep_scrub":"55'861","last_deep_scrub_stamp":"2023-10-30T12:56:46.673491+0000","last_clean_scrub_stamp":"2023-10-30T12:56:46.673491+0000","objects_scrubbed":166,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:55:04.341832+0000","scrub_duration":0.18274528900000001,"objects_trimmed":0,"snaptrim_duration":0.0024132049999999999,"stat_sum":{"num_bytes":15296106,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":63,"num_read_kb":15,"num_write":636,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"55'823","reported_seq":3009,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.671124+0000","last_change":"2023-10-30T12:57:07.671175+0000","last_active":"2023-10-30T12:57:07.671124+0000","last_peered":"2023-10-30T12:57:07.671124+0000","last_clean":"2023-10-30T12:57:07.671124+0000","last_became_active":"2023-10-30T12:35:06.189171+0000","last_became_peered":"2023-10-30T12:35:06.189171+0000","last_unstale":"2023-10-30T12:57:07.671124+0000","last_undegraded":"2023-10-30T12:57:07.671124+0000","last_fullsized":"2023-10-30T12:57:07.671124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'823","last_scrub_stamp":"2023-10-30T12:57:07.671050+0000","last_deep_scrub":"55'823","last_deep_scrub_stamp":"2023-10-30T12:57:07.671050+0000","last_clean_scrub_stamp":"2023-10-30T12:57:07.671050+0000","objects_scrubbed":171,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:41:38.114367+0000","scrub_duration":0.27815337899999998,"objects_trimmed":0,"snaptrim_duration":0.0056904870000000001,"stat_sum":{"num_bytes":14055782,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":78,"num_read_kb":12372,"num_write":607,"num_write_kb":31259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"55'824","reported_seq":2992,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:47.631354+0000","last_change":"2023-10-30T12:56:47.631397+0000","last_active":"2023-10-30T12:56:47.631354+0000","last_peered":"2023-10-30T12:56:47.631354+0000","last_clean":"2023-10-30T12:56:47.631354+0000","last_became_active":"2023-10-30T12:35:06.183348+0000","last_became_peered":"2023-10-30T12:35:06.183348+0000","last_unstale":"2023-10-30T12:56:47.631354+0000","last_undegraded":"2023-10-30T12:56:47.631354+0000","last_fullsized":"2023-10-30T12:56:47.631354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'824","last_scrub_stamp":"2023-10-30T12:56:47.631281+0000","last_deep_scrub":"55'824","last_deep_scrub_stamp":"2023-10-30T12:56:47.631281+0000","last_clean_scrub_stamp":"2023-10-30T12:56:47.631281+0000","objects_scrubbed":152,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:43:06.067352+0000","scrub_duration":0.17130867799999999,"objects_trimmed":0,"snaptrim_duration":0.0041317979999999999,"stat_sum":{"num_bytes":14615922,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":60,"num_read_kb":7678,"num_write":625,"num_write_kb":36364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"55'882","reported_seq":3063,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:39.686942+0000","last_change":"2023-10-30T12:56:39.687006+0000","last_active":"2023-10-30T12:56:39.686942+0000","last_peered":"2023-10-30T12:56:39.686942+0000","last_clean":"2023-10-30T12:56:39.686942+0000","last_became_active":"2023-10-30T12:35:06.196368+0000","last_became_peered":"2023-10-30T12:35:06.196368+0000","last_unstale":"2023-10-30T12:56:39.686942+0000","last_undegraded":"2023-10-30T12:56:39.686942+0000","last_fullsized":"2023-10-30T12:56:39.686942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'882","last_scrub_stamp":"2023-10-30T12:56:39.686847+0000","last_deep_scrub":"55'882","last_deep_scrub_stamp":"2023-10-30T12:56:39.686847+0000","last_clean_scrub_stamp":"2023-10-30T12:56:39.686847+0000","objects_scrubbed":148,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:44:17.768223+0000","scrub_duration":0.25242409199999999,"objects_trimmed":0,"snaptrim_duration":0.015569382999999999,"stat_sum":{"num_bytes":14491906,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":75,"num_read_kb":9352,"num_write":694,"num_write_kb":32749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"55'888","reported_seq":3068,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:25.513128+0000","last_change":"2023-10-30T12:56:25.513186+0000","last_active":"2023-10-30T12:56:25.513128+0000","last_peered":"2023-10-30T12:56:25.513128+0000","last_clean":"2023-10-30T12:56:25.513128+0000","last_became_active":"2023-10-30T12:35:06.213994+0000","last_became_peered":"2023-10-30T12:35:06.213994+0000","last_unstale":"2023-10-30T12:56:25.513128+0000","last_undegraded":"2023-10-30T12:56:25.513128+0000","last_fullsized":"2023-10-30T12:56:25.513128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'888","last_scrub_stamp":"2023-10-30T12:56:25.513052+0000","last_deep_scrub":"55'888","last_deep_scrub_stamp":"2023-10-30T12:56:25.513052+0000","last_clean_scrub_stamp":"2023-10-30T12:56:25.513052+0000","objects_scrubbed":188,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:49:47.291732+0000","scrub_duration":0.28763551900000001,"objects_trimmed":0,"snaptrim_duration":0.001873502,"stat_sum":{"num_bytes":23363209,"num_objects":188,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":188,"num_whiteouts":0,"num_read":72,"num_read_kb":4118,"num_write":712,"num_write_kb":32388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"55'855","reported_seq":3032,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:42.099804+0000","last_change":"2023-10-30T12:56:42.099940+0000","last_active":"2023-10-30T12:56:42.099804+0000","last_peered":"2023-10-30T12:56:42.099804+0000","last_clean":"2023-10-30T12:56:42.099804+0000","last_became_active":"2023-10-30T12:35:06.197647+0000","last_became_peered":"2023-10-30T12:35:06.197647+0000","last_unstale":"2023-10-30T12:56:42.099804+0000","last_undegraded":"2023-10-30T12:56:42.099804+0000","last_fullsized":"2023-10-30T12:56:42.099804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'855","last_scrub_stamp":"2023-10-30T12:56:42.099593+0000","last_deep_scrub":"55'855","last_deep_scrub_stamp":"2023-10-30T12:56:42.099593+0000","last_clean_scrub_stamp":"2023-10-30T12:56:42.099593+0000","objects_scrubbed":167,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:51:11.019062+0000","scrub_duration":0.343353621,"objects_trimmed":0,"snaptrim_duration":0.011378932,"stat_sum":{"num_bytes":18364745,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":70,"num_read_kb":8212,"num_write":651,"num_write_kb":28257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"55'773","reported_seq":2940,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:21.810710+0000","last_change":"2023-10-30T12:56:21.810710+0000","last_active":"2023-10-30T12:56:21.810710+0000","last_peered":"2023-10-30T12:56:21.810710+0000","last_clean":"2023-10-30T12:56:21.810710+0000","last_became_active":"2023-10-30T12:35:06.167558+0000","last_became_peered":"2023-10-30T12:35:06.167558+0000","last_unstale":"2023-10-30T12:56:21.810710+0000","last_undegraded":"2023-10-30T12:56:21.810710+0000","last_fullsized":"2023-10-30T12:56:21.810710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'773","last_scrub_stamp":"2023-10-30T12:56:21.810556+0000","last_deep_scrub":"55'773","last_deep_scrub_stamp":"2023-10-30T12:56:21.810556+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.810556+0000","objects_scrubbed":143,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:13:19.546302+0000","scrub_duration":0.23224787799999999,"objects_trimmed":0,"snaptrim_duration":0.0048907120000000002,"stat_sum":{"num_bytes":9710360,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":62,"num_read_kb":17,"num_write":609,"num_write_kb":15663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"55'239","reported_seq":314,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:27.077499+0000","last_change":"2023-10-30T12:56:27.077499+0000","last_active":"2023-10-30T12:56:27.077499+0000","last_peered":"2023-10-30T12:56:27.077499+0000","last_clean":"2023-10-30T12:56:27.077499+0000","last_became_active":"2023-10-30T12:35:05.200169+0000","last_became_peered":"2023-10-30T12:35:05.200169+0000","last_unstale":"2023-10-30T12:56:27.077499+0000","last_undegraded":"2023-10-30T12:56:27.077499+0000","last_fullsized":"2023-10-30T12:56:27.077499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'239","last_scrub_stamp":"2023-10-30T12:56:27.077343+0000","last_deep_scrub":"55'239","last_deep_scrub_stamp":"2023-10-30T12:56:27.077343+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.077343+0000","objects_scrubbed":6,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:21:31.745900+0000","scrub_duration":0.044083678000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":36867,"num_write":183,"num_write_kb":12445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16490,"num_omap_keys":31,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"55'812","reported_seq":2978,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:38.697551+0000","last_change":"2023-10-30T12:56:38.697551+0000","last_active":"2023-10-30T12:56:38.697551+0000","last_peered":"2023-10-30T12:56:38.697551+0000","last_clean":"2023-10-30T12:56:38.697551+0000","last_became_active":"2023-10-30T12:35:06.194689+0000","last_became_peered":"2023-10-30T12:35:06.194689+0000","last_unstale":"2023-10-30T12:56:38.697551+0000","last_undegraded":"2023-10-30T12:56:38.697551+0000","last_fullsized":"2023-10-30T12:56:38.697551+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'812","last_scrub_stamp":"2023-10-30T12:56:38.697404+0000","last_deep_scrub":"55'812","last_deep_scrub_stamp":"2023-10-30T12:56:38.697404+0000","last_clean_scrub_stamp":"2023-10-30T12:56:38.697404+0000","objects_scrubbed":144,"log_size":812,"log_dups_size":0,"ondisk_log_size":812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:44:05.310987+0000","scrub_duration":0.23620231899999999,"objects_trimmed":0,"snaptrim_duration":0.004291034,"stat_sum":{"num_bytes":10922380,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":61,"num_read_kb":18,"num_write":626,"num_write_kb":24711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"55'983","reported_seq":1184,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.861007+0000","last_change":"2023-10-30T12:56:24.861007+0000","last_active":"2023-10-30T12:56:24.861007+0000","last_peered":"2023-10-30T12:56:24.861007+0000","last_clean":"2023-10-30T12:56:24.861007+0000","last_became_active":"2023-10-30T12:35:05.182761+0000","last_became_peered":"2023-10-30T12:35:05.182761+0000","last_unstale":"2023-10-30T12:56:24.861007+0000","last_undegraded":"2023-10-30T12:56:24.861007+0000","last_fullsized":"2023-10-30T12:56:24.861007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'983","last_scrub_stamp":"2023-10-30T12:56:24.860748+0000","last_deep_scrub":"55'983","last_deep_scrub_stamp":"2023-10-30T12:56:24.860748+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.860748+0000","objects_scrubbed":8,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:51:38.567709+0000","scrub_duration":0.11984236299999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25625406,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":150,"num_read_kb":45229,"num_write":895,"num_write_kb":25503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"55'458","reported_seq":673,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:21.989512+0000","last_change":"2023-10-30T12:56:21.989512+0000","last_active":"2023-10-30T12:56:21.989512+0000","last_peered":"2023-10-30T12:56:21.989512+0000","last_clean":"2023-10-30T12:56:21.989512+0000","last_became_active":"2023-10-30T12:35:05.180730+0000","last_became_peered":"2023-10-30T12:35:05.180730+0000","last_unstale":"2023-10-30T12:56:21.989512+0000","last_undegraded":"2023-10-30T12:56:21.989512+0000","last_fullsized":"2023-10-30T12:56:21.989512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'458","last_scrub_stamp":"2023-10-30T12:56:21.989395+0000","last_deep_scrub":"55'458","last_deep_scrub_stamp":"2023-10-30T12:56:21.989395+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.989395+0000","objects_scrubbed":4,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:10:50.980457+0000","scrub_duration":0.030827730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8855749,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":162,"num_read_kb":8759,"num_write":390,"num_write_kb":8900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"55'906","reported_seq":3092,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:29.424522+0000","last_change":"2023-10-30T12:56:29.424522+0000","last_active":"2023-10-30T12:56:29.424522+0000","last_peered":"2023-10-30T12:56:29.424522+0000","last_clean":"2023-10-30T12:56:29.424522+0000","last_became_active":"2023-10-30T12:35:46.051057+0000","last_became_peered":"2023-10-30T12:35:46.051057+0000","last_unstale":"2023-10-30T12:56:29.424522+0000","last_undegraded":"2023-10-30T12:56:29.424522+0000","last_fullsized":"2023-10-30T12:56:29.424522+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'906","last_scrub_stamp":"2023-10-30T12:56:29.424403+0000","last_deep_scrub":"55'906","last_deep_scrub_stamp":"2023-10-30T12:56:29.424403+0000","last_clean_scrub_stamp":"2023-10-30T12:56:29.424403+0000","objects_scrubbed":175,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:57:06.583434+0000","scrub_duration":0.16198085700000001,"objects_trimmed":0,"snaptrim_duration":0.0018351520000000001,"stat_sum":{"num_bytes":19060908,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":82,"num_read_kb":16408,"num_write":727,"num_write_kb":53009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"55'988","reported_seq":1064,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:30.499309+0000","last_change":"2023-10-30T12:56:30.499309+0000","last_active":"2023-10-30T12:56:30.499309+0000","last_peered":"2023-10-30T12:56:30.499309+0000","last_clean":"2023-10-30T12:56:30.499309+0000","last_became_active":"2023-10-30T12:35:05.201853+0000","last_became_peered":"2023-10-30T12:35:05.201853+0000","last_unstale":"2023-10-30T12:56:30.499309+0000","last_undegraded":"2023-10-30T12:56:30.499309+0000","last_fullsized":"2023-10-30T12:56:30.499309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'988","last_scrub_stamp":"2023-10-30T12:56:30.498957+0000","last_deep_scrub":"55'988","last_deep_scrub_stamp":"2023-10-30T12:56:30.498957+0000","last_clean_scrub_stamp":"2023-10-30T12:56:30.498957+0000","objects_scrubbed":5,"log_size":988,"log_dups_size":0,"ondisk_log_size":988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:34.744919+0000","scrub_duration":0.073408293999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":38325,"num_write":914,"num_write_kb":16753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"55'900","reported_seq":3080,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:39.676436+0000","last_change":"2023-10-30T12:56:39.676436+0000","last_active":"2023-10-30T12:56:39.676436+0000","last_peered":"2023-10-30T12:56:39.676436+0000","last_clean":"2023-10-30T12:56:39.676436+0000","last_became_active":"2023-10-30T12:35:06.170179+0000","last_became_peered":"2023-10-30T12:35:06.170179+0000","last_unstale":"2023-10-30T12:56:39.676436+0000","last_undegraded":"2023-10-30T12:56:39.676436+0000","last_fullsized":"2023-10-30T12:56:39.676436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'900","last_scrub_stamp":"2023-10-30T12:56:39.676328+0000","last_deep_scrub":"55'900","last_deep_scrub_stamp":"2023-10-30T12:56:39.676328+0000","last_clean_scrub_stamp":"2023-10-30T12:56:39.676328+0000","objects_scrubbed":175,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:24:31.809070+0000","scrub_duration":0.198132067,"objects_trimmed":0,"snaptrim_duration":0.0021794980000000002,"stat_sum":{"num_bytes":14292082,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":73,"num_read_kb":12307,"num_write":687,"num_write_kb":28727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"55'152","reported_seq":221,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:52.550152+0000","last_change":"2023-10-30T12:56:27.831941+0000","last_active":"2023-10-30T12:56:52.550152+0000","last_peered":"2023-10-30T12:56:52.550152+0000","last_clean":"2023-10-30T12:56:52.550152+0000","last_became_active":"2023-10-30T12:35:05.197606+0000","last_became_peered":"2023-10-30T12:35:05.197606+0000","last_unstale":"2023-10-30T12:56:52.550152+0000","last_undegraded":"2023-10-30T12:56:52.550152+0000","last_fullsized":"2023-10-30T12:56:52.550152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'149","last_scrub_stamp":"2023-10-30T12:56:27.831485+0000","last_deep_scrub":"55'149","last_deep_scrub_stamp":"2023-10-30T12:56:27.831485+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.831485+0000","objects_scrubbed":5,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:57:34.144550+0000","scrub_duration":0.012229985000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":93,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52416,"num_omap_keys":96,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"55'814","reported_seq":2997,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.786207+0000","last_change":"2023-10-30T12:56:34.786207+0000","last_active":"2023-10-30T12:56:34.786207+0000","last_peered":"2023-10-30T12:56:34.786207+0000","last_clean":"2023-10-30T12:56:34.786207+0000","last_became_active":"2023-10-30T12:35:06.225392+0000","last_became_peered":"2023-10-30T12:35:06.225392+0000","last_unstale":"2023-10-30T12:56:34.786207+0000","last_undegraded":"2023-10-30T12:56:34.786207+0000","last_fullsized":"2023-10-30T12:56:34.786207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'814","last_scrub_stamp":"2023-10-30T12:56:34.786044+0000","last_deep_scrub":"55'814","last_deep_scrub_stamp":"2023-10-30T12:56:34.786044+0000","last_clean_scrub_stamp":"2023-10-30T12:56:34.786044+0000","objects_scrubbed":159,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:26:14.139459+0000","scrub_duration":0.24530717799999999,"objects_trimmed":0,"snaptrim_duration":0.01552166,"stat_sum":{"num_bytes":9754816,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":77,"num_read_kb":18,"num_write":651,"num_write_kb":19117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"55'600","reported_seq":671,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:59.690048+0000","last_change":"2023-10-30T12:56:23.447764+0000","last_active":"2023-10-30T12:56:59.690048+0000","last_peered":"2023-10-30T12:56:59.690048+0000","last_clean":"2023-10-30T12:56:59.690048+0000","last_became_active":"2023-10-30T12:35:05.196899+0000","last_became_peered":"2023-10-30T12:35:05.196899+0000","last_unstale":"2023-10-30T12:56:59.690048+0000","last_undegraded":"2023-10-30T12:56:59.690048+0000","last_fullsized":"2023-10-30T12:56:59.690048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'579","last_scrub_stamp":"2023-10-30T12:56:23.447316+0000","last_deep_scrub":"55'579","last_deep_scrub_stamp":"2023-10-30T12:56:23.447316+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.447316+0000","objects_scrubbed":6,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:59:20.096502+0000","scrub_duration":0.067786201000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18436527,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":14173,"num_write":529,"num_write_kb":18295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"55'865","reported_seq":3034,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:45.787426+0000","last_change":"2023-10-30T12:56:45.787426+0000","last_active":"2023-10-30T12:56:45.787426+0000","last_peered":"2023-10-30T12:56:45.787426+0000","last_clean":"2023-10-30T12:56:45.787426+0000","last_became_active":"2023-10-30T12:35:06.222057+0000","last_became_peered":"2023-10-30T12:35:06.222057+0000","last_unstale":"2023-10-30T12:56:45.787426+0000","last_undegraded":"2023-10-30T12:56:45.787426+0000","last_fullsized":"2023-10-30T12:56:45.787426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'865","last_scrub_stamp":"2023-10-30T12:56:45.787252+0000","last_deep_scrub":"55'865","last_deep_scrub_stamp":"2023-10-30T12:56:45.787252+0000","last_clean_scrub_stamp":"2023-10-30T12:56:45.787252+0000","objects_scrubbed":159,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:23:24.865367+0000","scrub_duration":0.40644253200000002,"objects_trimmed":0,"snaptrim_duration":0.004210915,"stat_sum":{"num_bytes":31400359,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":63,"num_read_kb":12308,"num_write":657,"num_write_kb":50495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"55'317","reported_seq":390,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:18.453440+0000","last_change":"2023-10-30T12:56:18.453440+0000","last_active":"2023-10-30T12:56:18.453440+0000","last_peered":"2023-10-30T12:56:18.453440+0000","last_clean":"2023-10-30T12:56:18.453440+0000","last_became_active":"2023-10-30T12:35:05.197190+0000","last_became_peered":"2023-10-30T12:35:05.197190+0000","last_unstale":"2023-10-30T12:56:18.453440+0000","last_undegraded":"2023-10-30T12:56:18.453440+0000","last_fullsized":"2023-10-30T12:56:18.453440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'317","last_scrub_stamp":"2023-10-30T12:56:18.453257+0000","last_deep_scrub":"55'317","last_deep_scrub_stamp":"2023-10-30T12:56:18.453257+0000","last_clean_scrub_stamp":"2023-10-30T12:56:18.453257+0000","objects_scrubbed":7,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:09:36.655470+0000","scrub_duration":0.034206814000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":4111,"num_write":258,"num_write_kb":8429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28930,"num_omap_keys":53,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"55'791","reported_seq":2957,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:25.811201+0000","last_change":"2023-10-30T12:56:25.811267+0000","last_active":"2023-10-30T12:56:25.811201+0000","last_peered":"2023-10-30T12:56:25.811201+0000","last_clean":"2023-10-30T12:56:25.811201+0000","last_became_active":"2023-10-30T12:35:06.204122+0000","last_became_peered":"2023-10-30T12:35:06.204122+0000","last_unstale":"2023-10-30T12:56:25.811201+0000","last_undegraded":"2023-10-30T12:56:25.811201+0000","last_fullsized":"2023-10-30T12:56:25.811201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'791","last_scrub_stamp":"2023-10-30T12:56:25.811099+0000","last_deep_scrub":"55'791","last_deep_scrub_stamp":"2023-10-30T12:56:25.811099+0000","last_clean_scrub_stamp":"2023-10-30T12:56:25.811099+0000","objects_scrubbed":158,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:21:46.064934+0000","scrub_duration":0.24490572399999999,"objects_trimmed":0,"snaptrim_duration":0.0030324050000000002,"stat_sum":{"num_bytes":15588017,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":60,"num_read_kb":8209,"num_write":632,"num_write_kb":41942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"55'370","reported_seq":451,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:39.535926+0000","last_change":"2023-10-30T12:56:25.809737+0000","last_active":"2023-10-30T12:56:39.535926+0000","last_peered":"2023-10-30T12:56:39.535926+0000","last_clean":"2023-10-30T12:56:39.535926+0000","last_became_active":"2023-10-30T12:35:05.179547+0000","last_became_peered":"2023-10-30T12:35:05.179547+0000","last_unstale":"2023-10-30T12:56:39.535926+0000","last_undegraded":"2023-10-30T12:56:39.535926+0000","last_fullsized":"2023-10-30T12:56:39.535926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'368","last_scrub_stamp":"2023-10-30T12:56:25.809432+0000","last_deep_scrub":"55'368","last_deep_scrub_stamp":"2023-10-30T12:56:25.809432+0000","last_clean_scrub_stamp":"2023-10-30T12:56:25.809432+0000","objects_scrubbed":9,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:55:48.660238+0000","scrub_duration":0.040587917000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":24600,"num_write":302,"num_write_kb":12513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17926,"num_omap_keys":33,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"55'784","reported_seq":2951,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.106255+0000","last_change":"2023-10-30T12:56:24.106255+0000","last_active":"2023-10-30T12:56:24.106255+0000","last_peered":"2023-10-30T12:56:24.106255+0000","last_clean":"2023-10-30T12:56:24.106255+0000","last_became_active":"2023-10-30T12:35:06.215128+0000","last_became_peered":"2023-10-30T12:35:06.215128+0000","last_unstale":"2023-10-30T12:56:24.106255+0000","last_undegraded":"2023-10-30T12:56:24.106255+0000","last_fullsized":"2023-10-30T12:56:24.106255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'784","last_scrub_stamp":"2023-10-30T12:56:24.105935+0000","last_deep_scrub":"55'784","last_deep_scrub_stamp":"2023-10-30T12:56:24.105935+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.105935+0000","objects_scrubbed":171,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:40:02.361124+0000","scrub_duration":0.30957912999999998,"objects_trimmed":0,"snaptrim_duration":0.0064662909999999999,"stat_sum":{"num_bytes":22774544,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":61,"num_read_kb":11,"num_write":605,"num_write_kb":27691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"55'215","reported_seq":298,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:18.560689+0000","last_change":"2023-10-30T12:56:18.560737+0000","last_active":"2023-10-30T12:56:18.560689+0000","last_peered":"2023-10-30T12:56:18.560689+0000","last_clean":"2023-10-30T12:56:18.560689+0000","last_became_active":"2023-10-30T12:35:05.189573+0000","last_became_peered":"2023-10-30T12:35:05.189573+0000","last_unstale":"2023-10-30T12:56:18.560689+0000","last_undegraded":"2023-10-30T12:56:18.560689+0000","last_fullsized":"2023-10-30T12:56:18.560689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'215","last_scrub_stamp":"2023-10-30T12:56:18.560612+0000","last_deep_scrub":"55'215","last_deep_scrub_stamp":"2023-10-30T12:56:18.560612+0000","last_clean_scrub_stamp":"2023-10-30T12:56:18.560612+0000","objects_scrubbed":7,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:40:38.364129+0000","scrub_duration":0.037472814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":24624,"num_write":154,"num_write_kb":8340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15144,"num_omap_keys":29,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"55'901","reported_seq":3085,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:40.810421+0000","last_change":"2023-10-30T12:56:40.810463+0000","last_active":"2023-10-30T12:56:40.810421+0000","last_peered":"2023-10-30T12:56:40.810421+0000","last_clean":"2023-10-30T12:56:40.810421+0000","last_became_active":"2023-10-30T12:35:06.216885+0000","last_became_peered":"2023-10-30T12:35:06.216885+0000","last_unstale":"2023-10-30T12:56:40.810421+0000","last_undegraded":"2023-10-30T12:56:40.810421+0000","last_fullsized":"2023-10-30T12:56:40.810421+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'901","last_scrub_stamp":"2023-10-30T12:56:40.810347+0000","last_deep_scrub":"55'901","last_deep_scrub_stamp":"2023-10-30T12:56:40.810347+0000","last_clean_scrub_stamp":"2023-10-30T12:56:40.810347+0000","objects_scrubbed":175,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:17:37.604756+0000","scrub_duration":0.31851338200000001,"objects_trimmed":0,"snaptrim_duration":0.0042006370000000001,"stat_sum":{"num_bytes":19251186,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":77,"num_read_kb":16402,"num_write":677,"num_write_kb":45337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"55'237","reported_seq":307,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:31.515367+0000","last_change":"2023-10-30T12:56:31.515500+0000","last_active":"2023-10-30T12:56:31.515367+0000","last_peered":"2023-10-30T12:56:31.515367+0000","last_clean":"2023-10-30T12:56:31.515367+0000","last_became_active":"2023-10-30T12:35:05.202612+0000","last_became_peered":"2023-10-30T12:35:05.202612+0000","last_unstale":"2023-10-30T12:56:31.515367+0000","last_undegraded":"2023-10-30T12:56:31.515367+0000","last_fullsized":"2023-10-30T12:56:31.515367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'237","last_scrub_stamp":"2023-10-30T12:56:31.515170+0000","last_deep_scrub":"55'237","last_deep_scrub_stamp":"2023-10-30T12:56:31.515170+0000","last_clean_scrub_stamp":"2023-10-30T12:56:31.515170+0000","objects_scrubbed":7,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:51:38.652440+0000","scrub_duration":0.042566652000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":181,"num_write_kb":8338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45273,"num_omap_keys":85,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"55'854","reported_seq":3034,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:17.119819+0000","last_change":"2023-10-30T12:57:17.119888+0000","last_active":"2023-10-30T12:57:17.119819+0000","last_peered":"2023-10-30T12:57:17.119819+0000","last_clean":"2023-10-30T12:57:17.119819+0000","last_became_active":"2023-10-30T12:35:06.215553+0000","last_became_peered":"2023-10-30T12:35:06.215553+0000","last_unstale":"2023-10-30T12:57:17.119819+0000","last_undegraded":"2023-10-30T12:57:17.119819+0000","last_fullsized":"2023-10-30T12:57:17.119819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'854","last_scrub_stamp":"2023-10-30T12:57:17.119722+0000","last_deep_scrub":"55'854","last_deep_scrub_stamp":"2023-10-30T12:57:17.119722+0000","last_clean_scrub_stamp":"2023-10-30T12:57:17.119722+0000","objects_scrubbed":144,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:50:50.978198+0000","scrub_duration":0.28060836500000003,"objects_trimmed":0,"snaptrim_duration":0.0060418609999999999,"stat_sum":{"num_bytes":19055642,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":76,"num_read_kb":33,"num_write":654,"num_write_kb":32706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"55'297","reported_seq":377,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:21.765254+0000","last_change":"2023-10-30T12:56:21.765381+0000","last_active":"2023-10-30T12:56:21.765254+0000","last_peered":"2023-10-30T12:56:21.765254+0000","last_clean":"2023-10-30T12:56:21.765254+0000","last_became_active":"2023-10-30T12:35:05.181115+0000","last_became_peered":"2023-10-30T12:35:05.181115+0000","last_unstale":"2023-10-30T12:56:21.765254+0000","last_undegraded":"2023-10-30T12:56:21.765254+0000","last_fullsized":"2023-10-30T12:56:21.765254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'297","last_scrub_stamp":"2023-10-30T12:56:21.765065+0000","last_deep_scrub":"55'297","last_deep_scrub_stamp":"2023-10-30T12:56:21.765065+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.765065+0000","objects_scrubbed":8,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:20:05.549971+0000","scrub_duration":0.074545823999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":43239,"num_write":227,"num_write_kb":16591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":22378,"num_omap_keys":41,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"55'794","reported_seq":2960,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:31.034978+0000","last_change":"2023-10-30T12:56:31.035128+0000","last_active":"2023-10-30T12:56:31.034978+0000","last_peered":"2023-10-30T12:56:31.034978+0000","last_clean":"2023-10-30T12:56:31.034978+0000","last_became_active":"2023-10-30T12:35:06.197722+0000","last_became_peered":"2023-10-30T12:35:06.197722+0000","last_unstale":"2023-10-30T12:56:31.034978+0000","last_undegraded":"2023-10-30T12:56:31.034978+0000","last_fullsized":"2023-10-30T12:56:31.034978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'794","last_scrub_stamp":"2023-10-30T12:56:31.034831+0000","last_deep_scrub":"55'794","last_deep_scrub_stamp":"2023-10-30T12:56:31.034831+0000","last_clean_scrub_stamp":"2023-10-30T12:56:31.034831+0000","objects_scrubbed":168,"log_size":794,"log_dups_size":0,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:35:46.015437+0000","scrub_duration":0.24651738600000001,"objects_trimmed":0,"snaptrim_duration":0.011777173,"stat_sum":{"num_bytes":11179157,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":60,"num_read_kb":16,"num_write":620,"num_write_kb":21517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"55'492","reported_seq":592,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:19.245439+0000","last_change":"2023-10-30T12:56:19.245517+0000","last_active":"2023-10-30T12:56:19.245439+0000","last_peered":"2023-10-30T12:56:19.245439+0000","last_clean":"2023-10-30T12:56:19.245439+0000","last_became_active":"2023-10-30T12:35:05.169130+0000","last_became_peered":"2023-10-30T12:35:05.169130+0000","last_unstale":"2023-10-30T12:56:19.245439+0000","last_undegraded":"2023-10-30T12:56:19.245439+0000","last_fullsized":"2023-10-30T12:56:19.245439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'492","last_scrub_stamp":"2023-10-30T12:56:19.245361+0000","last_deep_scrub":"55'492","last_deep_scrub_stamp":"2023-10-30T12:56:19.245361+0000","last_clean_scrub_stamp":"2023-10-30T12:56:19.245361+0000","objects_scrubbed":12,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:27:01.796605+0000","scrub_duration":0.039470346000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583024,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":18341,"num_write":415,"num_write_kb":12551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10636,"num_omap_keys":20,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"55'888","reported_seq":3085,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:00.210068+0000","last_change":"2023-10-30T12:57:00.210138+0000","last_active":"2023-10-30T12:57:00.210068+0000","last_peered":"2023-10-30T12:57:00.210068+0000","last_clean":"2023-10-30T12:57:00.210068+0000","last_became_active":"2023-10-30T12:35:06.214839+0000","last_became_peered":"2023-10-30T12:35:06.214839+0000","last_unstale":"2023-10-30T12:57:00.210068+0000","last_undegraded":"2023-10-30T12:57:00.210068+0000","last_fullsized":"2023-10-30T12:57:00.210068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'888","last_scrub_stamp":"2023-10-30T12:57:00.209983+0000","last_deep_scrub":"55'888","last_deep_scrub_stamp":"2023-10-30T12:57:00.209983+0000","last_clean_scrub_stamp":"2023-10-30T12:57:00.209983+0000","objects_scrubbed":168,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:49:53.540651+0000","scrub_duration":0.24779405400000001,"objects_trimmed":0,"snaptrim_duration":0.0049147749999999997,"stat_sum":{"num_bytes":12184897,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":91,"num_read_kb":16402,"num_write":664,"num_write_kb":29797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"55'315","reported_seq":390,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:20.671275+0000","last_change":"2023-10-30T12:56:20.671275+0000","last_active":"2023-10-30T12:56:20.671275+0000","last_peered":"2023-10-30T12:56:20.671275+0000","last_clean":"2023-10-30T12:56:20.671275+0000","last_became_active":"2023-10-30T12:35:05.188862+0000","last_became_peered":"2023-10-30T12:35:05.188862+0000","last_unstale":"2023-10-30T12:56:20.671275+0000","last_undegraded":"2023-10-30T12:56:20.671275+0000","last_fullsized":"2023-10-30T12:56:20.671275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'315","last_scrub_stamp":"2023-10-30T12:56:20.670866+0000","last_deep_scrub":"55'315","last_deep_scrub_stamp":"2023-10-30T12:56:20.670866+0000","last_clean_scrub_stamp":"2023-10-30T12:56:20.670866+0000","objects_scrubbed":4,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:07:40.146504+0000","scrub_duration":0.058071512999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":12343,"num_write":388,"num_write_kb":4750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9690,"num_omap_keys":18,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"55'856","reported_seq":3029,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:18.187781+0000","last_change":"2023-10-30T12:57:18.187781+0000","last_active":"2023-10-30T12:57:18.187781+0000","last_peered":"2023-10-30T12:57:18.187781+0000","last_clean":"2023-10-30T12:57:18.187781+0000","last_became_active":"2023-10-30T12:35:06.216533+0000","last_became_peered":"2023-10-30T12:35:06.216533+0000","last_unstale":"2023-10-30T12:57:18.187781+0000","last_undegraded":"2023-10-30T12:57:18.187781+0000","last_fullsized":"2023-10-30T12:57:18.187781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'856","last_scrub_stamp":"2023-10-30T12:57:18.187449+0000","last_deep_scrub":"55'856","last_deep_scrub_stamp":"2023-10-30T12:57:18.187449+0000","last_clean_scrub_stamp":"2023-10-30T12:57:18.187449+0000","objects_scrubbed":183,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:24:15.570290+0000","scrub_duration":0.30414718499999999,"objects_trimmed":0,"snaptrim_duration":0.0060962430000000003,"stat_sum":{"num_bytes":10771712,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":65,"num_read_kb":4109,"num_write":629,"num_write_kb":24152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"55'238","reported_seq":309,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:23.246263+0000","last_change":"2023-10-30T12:56:23.246263+0000","last_active":"2023-10-30T12:56:23.246263+0000","last_peered":"2023-10-30T12:56:23.246263+0000","last_clean":"2023-10-30T12:56:23.246263+0000","last_became_active":"2023-10-30T12:35:05.177316+0000","last_became_peered":"2023-10-30T12:35:05.177316+0000","last_unstale":"2023-10-30T12:56:23.246263+0000","last_undegraded":"2023-10-30T12:56:23.246263+0000","last_fullsized":"2023-10-30T12:56:23.246263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'238","last_scrub_stamp":"2023-10-30T12:56:23.246132+0000","last_deep_scrub":"55'238","last_deep_scrub_stamp":"2023-10-30T12:56:23.246132+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.246132+0000","objects_scrubbed":4,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:22:00.496493+0000","scrub_duration":0.023404655999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":180,"num_write_kb":8325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12558,"num_omap_keys":23,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"55'846","reported_seq":3021,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:39.581411+0000","last_change":"2023-10-30T12:56:39.581493+0000","last_active":"2023-10-30T12:56:39.581411+0000","last_peered":"2023-10-30T12:56:39.581411+0000","last_clean":"2023-10-30T12:56:39.581411+0000","last_became_active":"2023-10-30T12:35:46.049464+0000","last_became_peered":"2023-10-30T12:35:46.049464+0000","last_unstale":"2023-10-30T12:56:39.581411+0000","last_undegraded":"2023-10-30T12:56:39.581411+0000","last_fullsized":"2023-10-30T12:56:39.581411+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'846","last_scrub_stamp":"2023-10-30T12:56:39.581325+0000","last_deep_scrub":"55'846","last_deep_scrub_stamp":"2023-10-30T12:56:39.581325+0000","last_clean_scrub_stamp":"2023-10-30T12:56:39.581325+0000","objects_scrubbed":163,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:25:02.063225+0000","scrub_duration":0.248567017,"objects_trimmed":0,"snaptrim_duration":0.0023101760000000002,"stat_sum":{"num_bytes":7006288,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":663,"num_write_kb":12867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"55'215","reported_seq":286,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:32.554326+0000","last_change":"2023-10-30T12:56:32.554326+0000","last_active":"2023-10-30T12:56:32.554326+0000","last_peered":"2023-10-30T12:56:32.554326+0000","last_clean":"2023-10-30T12:56:32.554326+0000","last_became_active":"2023-10-30T12:35:05.171491+0000","last_became_peered":"2023-10-30T12:35:05.171491+0000","last_unstale":"2023-10-30T12:56:32.554326+0000","last_undegraded":"2023-10-30T12:56:32.554326+0000","last_fullsized":"2023-10-30T12:56:32.554326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'215","last_scrub_stamp":"2023-10-30T12:56:32.554161+0000","last_deep_scrub":"55'215","last_deep_scrub_stamp":"2023-10-30T12:56:32.554161+0000","last_clean_scrub_stamp":"2023-10-30T12:56:32.554161+0000","objects_scrubbed":4,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:25:32.692345+0000","scrub_duration":0.026739378000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":16387,"num_write":154,"num_write_kb":4208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11738,"num_omap_keys":23,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"55'931","reported_seq":3108,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:56.900412+0000","last_change":"2023-10-30T12:56:56.900531+0000","last_active":"2023-10-30T12:56:56.900412+0000","last_peered":"2023-10-30T12:56:56.900412+0000","last_clean":"2023-10-30T12:56:56.900412+0000","last_became_active":"2023-10-30T12:35:06.213493+0000","last_became_peered":"2023-10-30T12:35:06.213493+0000","last_unstale":"2023-10-30T12:56:56.900412+0000","last_undegraded":"2023-10-30T12:56:56.900412+0000","last_fullsized":"2023-10-30T12:56:56.900412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'931","last_scrub_stamp":"2023-10-30T12:56:56.900213+0000","last_deep_scrub":"55'931","last_deep_scrub_stamp":"2023-10-30T12:56:56.900213+0000","last_clean_scrub_stamp":"2023-10-30T12:56:56.900213+0000","objects_scrubbed":160,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:31:30.596032+0000","scrub_duration":0.55565498499999999,"objects_trimmed":0,"snaptrim_duration":0.015847370999999999,"stat_sum":{"num_bytes":18244405,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":70,"num_read_kb":4114,"num_write":718,"num_write_kb":33517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"55'310","reported_seq":383,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:20.198258+0000","last_change":"2023-10-30T12:56:20.198258+0000","last_active":"2023-10-30T12:56:20.198258+0000","last_peered":"2023-10-30T12:56:20.198258+0000","last_clean":"2023-10-30T12:56:20.198258+0000","last_became_active":"2023-10-30T12:35:05.197453+0000","last_became_peered":"2023-10-30T12:35:05.197453+0000","last_unstale":"2023-10-30T12:56:20.198258+0000","last_undegraded":"2023-10-30T12:56:20.198258+0000","last_fullsized":"2023-10-30T12:56:20.198258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'310","last_scrub_stamp":"2023-10-30T12:56:20.198131+0000","last_deep_scrub":"55'310","last_deep_scrub_stamp":"2023-10-30T12:56:20.198131+0000","last_clean_scrub_stamp":"2023-10-30T12:56:20.198131+0000","objects_scrubbed":5,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:00:10.556698+0000","scrub_duration":0.022980041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":240,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17972,"num_omap_keys":33,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"55'841","reported_seq":3009,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:02.283816+0000","last_change":"2023-10-30T12:57:02.283816+0000","last_active":"2023-10-30T12:57:02.283816+0000","last_peered":"2023-10-30T12:57:02.283816+0000","last_clean":"2023-10-30T12:57:02.283816+0000","last_became_active":"2023-10-30T12:35:06.219526+0000","last_became_peered":"2023-10-30T12:35:06.219526+0000","last_unstale":"2023-10-30T12:57:02.283816+0000","last_undegraded":"2023-10-30T12:57:02.283816+0000","last_fullsized":"2023-10-30T12:57:02.283816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'841","last_scrub_stamp":"2023-10-30T12:57:02.283577+0000","last_deep_scrub":"55'841","last_deep_scrub_stamp":"2023-10-30T12:57:02.283577+0000","last_clean_scrub_stamp":"2023-10-30T12:57:02.283577+0000","objects_scrubbed":150,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:20:08.888700+0000","scrub_duration":0.308269513,"objects_trimmed":0,"snaptrim_duration":0.0060180570000000003,"stat_sum":{"num_bytes":14352444,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":61,"num_read_kb":19722,"num_write":644,"num_write_kb":44993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"55'119","reported_seq":220,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:27.868066+0000","last_change":"2023-10-30T12:56:27.868066+0000","last_active":"2023-10-30T12:56:27.868066+0000","last_peered":"2023-10-30T12:56:27.868066+0000","last_clean":"2023-10-30T12:56:27.868066+0000","last_became_active":"2023-10-30T12:34:55.205923+0000","last_became_peered":"2023-10-30T12:34:55.205923+0000","last_unstale":"2023-10-30T12:56:27.868066+0000","last_undegraded":"2023-10-30T12:56:27.868066+0000","last_fullsized":"2023-10-30T12:56:27.868066+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'119","last_scrub_stamp":"2023-10-30T12:56:27.867851+0000","last_deep_scrub":"55'119","last_deep_scrub_stamp":"2023-10-30T12:56:27.867851+0000","last_clean_scrub_stamp":"2023-10-30T12:56:27.867851+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:43:27.714386+0000","scrub_duration":0.0099090500000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"55'316","reported_seq":394,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:28.481897+0000","last_change":"2023-10-30T12:56:28.482017+0000","last_active":"2023-10-30T12:56:28.481897+0000","last_peered":"2023-10-30T12:56:28.481897+0000","last_clean":"2023-10-30T12:56:28.481897+0000","last_became_active":"2023-10-30T12:35:05.197201+0000","last_became_peered":"2023-10-30T12:35:05.197201+0000","last_unstale":"2023-10-30T12:56:28.481897+0000","last_undegraded":"2023-10-30T12:56:28.481897+0000","last_fullsized":"2023-10-30T12:56:28.481897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'316","last_scrub_stamp":"2023-10-30T12:56:28.481711+0000","last_deep_scrub":"55'316","last_deep_scrub_stamp":"2023-10-30T12:56:28.481711+0000","last_clean_scrub_stamp":"2023-10-30T12:56:28.481711+0000","objects_scrubbed":5,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:52.589549+0000","scrub_duration":0.039869544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":23,"num_read_kb":16415,"num_write":260,"num_write_kb":8374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"55'803","reported_seq":2961,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:43.609349+0000","last_change":"2023-10-30T12:56:43.609412+0000","last_active":"2023-10-30T12:56:43.609349+0000","last_peered":"2023-10-30T12:56:43.609349+0000","last_clean":"2023-10-30T12:56:43.609349+0000","last_became_active":"2023-10-30T12:35:06.217687+0000","last_became_peered":"2023-10-30T12:35:06.217687+0000","last_unstale":"2023-10-30T12:56:43.609349+0000","last_undegraded":"2023-10-30T12:56:43.609349+0000","last_fullsized":"2023-10-30T12:56:43.609349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'803","last_scrub_stamp":"2023-10-30T12:56:43.609269+0000","last_deep_scrub":"55'803","last_deep_scrub_stamp":"2023-10-30T12:56:43.609269+0000","last_clean_scrub_stamp":"2023-10-30T12:56:43.609269+0000","objects_scrubbed":158,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:35:10.362189+0000","scrub_duration":0.29605744099999998,"objects_trimmed":0,"snaptrim_duration":0.0059801699999999999,"stat_sum":{"num_bytes":18791963,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":51,"num_read_kb":8203,"num_write":599,"num_write_kb":32827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"55'429","reported_seq":511,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.556052+0000","last_change":"2023-10-30T12:56:53.936455+0000","last_active":"2023-10-30T12:57:12.556052+0000","last_peered":"2023-10-30T12:57:12.556052+0000","last_clean":"2023-10-30T12:57:12.556052+0000","last_became_active":"2023-10-30T12:35:05.207828+0000","last_became_peered":"2023-10-30T12:35:05.207828+0000","last_unstale":"2023-10-30T12:57:12.556052+0000","last_undegraded":"2023-10-30T12:57:12.556052+0000","last_fullsized":"2023-10-30T12:57:12.556052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'426","last_scrub_stamp":"2023-10-30T12:56:53.936122+0000","last_deep_scrub":"55'426","last_deep_scrub_stamp":"2023-10-30T12:56:53.936122+0000","last_clean_scrub_stamp":"2023-10-30T12:56:53.936122+0000","objects_scrubbed":7,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:37:43.112167+0000","scrub_duration":0.083282911000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22042852,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":29,"num_read_kb":45074,"num_write":354,"num_write_kb":21733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"55'837","reported_seq":3002,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:28.741030+0000","last_change":"2023-10-30T12:56:28.741094+0000","last_active":"2023-10-30T12:56:28.741030+0000","last_peered":"2023-10-30T12:56:28.741030+0000","last_clean":"2023-10-30T12:56:28.741030+0000","last_became_active":"2023-10-30T12:35:06.194791+0000","last_became_peered":"2023-10-30T12:35:06.194791+0000","last_unstale":"2023-10-30T12:56:28.741030+0000","last_undegraded":"2023-10-30T12:56:28.741030+0000","last_fullsized":"2023-10-30T12:56:28.741030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'837","last_scrub_stamp":"2023-10-30T12:56:28.740945+0000","last_deep_scrub":"55'837","last_deep_scrub_stamp":"2023-10-30T12:56:28.740945+0000","last_clean_scrub_stamp":"2023-10-30T12:56:28.740945+0000","objects_scrubbed":162,"log_size":837,"log_dups_size":0,"ondisk_log_size":837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:55:53.265534+0000","scrub_duration":0.21527656000000001,"objects_trimmed":0,"snaptrim_duration":0.0043414500000000002,"stat_sum":{"num_bytes":10614378,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":58,"num_read_kb":3682,"num_write":656,"num_write_kb":34629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"55'618","reported_seq":985,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:49.619643+0000","last_change":"2023-10-30T12:56:21.219063+0000","last_active":"2023-10-30T12:56:49.619643+0000","last_peered":"2023-10-30T12:56:49.619643+0000","last_clean":"2023-10-30T12:56:49.619643+0000","last_became_active":"2023-10-30T12:35:05.175959+0000","last_became_peered":"2023-10-30T12:35:05.175959+0000","last_unstale":"2023-10-30T12:56:49.619643+0000","last_undegraded":"2023-10-30T12:56:49.619643+0000","last_fullsized":"2023-10-30T12:56:49.619643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'615","last_scrub_stamp":"2023-10-30T12:56:21.218930+0000","last_deep_scrub":"55'615","last_deep_scrub_stamp":"2023-10-30T12:56:21.218930+0000","last_clean_scrub_stamp":"2023-10-30T12:56:21.218930+0000","objects_scrubbed":9,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:53:08.332507+0000","scrub_duration":0.031017952000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9398249,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":313,"num_read_kb":20888,"num_write":569,"num_write_kb":9629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34344,"num_omap_keys":63,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"55'863","reported_seq":3039,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:40.689114+0000","last_change":"2023-10-30T12:56:40.689114+0000","last_active":"2023-10-30T12:56:40.689114+0000","last_peered":"2023-10-30T12:56:40.689114+0000","last_clean":"2023-10-30T12:56:40.689114+0000","last_became_active":"2023-10-30T12:35:06.164711+0000","last_became_peered":"2023-10-30T12:35:06.164711+0000","last_unstale":"2023-10-30T12:56:40.689114+0000","last_undegraded":"2023-10-30T12:56:40.689114+0000","last_fullsized":"2023-10-30T12:56:40.689114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'863","last_scrub_stamp":"2023-10-30T12:56:40.688985+0000","last_deep_scrub":"55'863","last_deep_scrub_stamp":"2023-10-30T12:56:40.688985+0000","last_clean_scrub_stamp":"2023-10-30T12:56:40.688985+0000","objects_scrubbed":162,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:58:54.635220+0000","scrub_duration":0.20007371800000001,"objects_trimmed":0,"snaptrim_duration":0.002714419,"stat_sum":{"num_bytes":18668779,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":68,"num_read_kb":8213,"num_write":645,"num_write_kb":33825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"55'501","reported_seq":574,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:22.428808+0000","last_change":"2023-10-30T12:56:22.428932+0000","last_active":"2023-10-30T12:56:22.428808+0000","last_peered":"2023-10-30T12:56:22.428808+0000","last_clean":"2023-10-30T12:56:22.428808+0000","last_became_active":"2023-10-30T12:35:05.196873+0000","last_became_peered":"2023-10-30T12:35:05.196873+0000","last_unstale":"2023-10-30T12:56:22.428808+0000","last_undegraded":"2023-10-30T12:56:22.428808+0000","last_fullsized":"2023-10-30T12:56:22.428808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'501","last_scrub_stamp":"2023-10-30T12:56:22.428591+0000","last_deep_scrub":"55'501","last_deep_scrub_stamp":"2023-10-30T12:56:22.428591+0000","last_clean_scrub_stamp":"2023-10-30T12:56:22.428591+0000","objects_scrubbed":5,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:31.946150+0000","scrub_duration":0.065339878000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":431,"num_write_kb":16687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8414,"num_omap_keys":16,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"55'813","reported_seq":2992,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:22.854721+0000","last_change":"2023-10-30T12:56:22.854850+0000","last_active":"2023-10-30T12:56:22.854721+0000","last_peered":"2023-10-30T12:56:22.854721+0000","last_clean":"2023-10-30T12:56:22.854721+0000","last_became_active":"2023-10-30T12:35:06.183225+0000","last_became_peered":"2023-10-30T12:35:06.183225+0000","last_unstale":"2023-10-30T12:56:22.854721+0000","last_undegraded":"2023-10-30T12:56:22.854721+0000","last_fullsized":"2023-10-30T12:56:22.854721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'813","last_scrub_stamp":"2023-10-30T12:56:22.854526+0000","last_deep_scrub":"55'813","last_deep_scrub_stamp":"2023-10-30T12:56:22.854526+0000","last_clean_scrub_stamp":"2023-10-30T12:56:22.854526+0000","objects_scrubbed":154,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:19:37.784115+0000","scrub_duration":0.30563644699999998,"objects_trimmed":0,"snaptrim_duration":0.0032989500000000001,"stat_sum":{"num_bytes":18054478,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":72,"num_read_kb":12,"num_write":656,"num_write_kb":36307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"55'185","reported_seq":250,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:36.447235+0000","last_change":"2023-10-30T12:56:36.447291+0000","last_active":"2023-10-30T12:56:36.447235+0000","last_peered":"2023-10-30T12:56:36.447235+0000","last_clean":"2023-10-30T12:56:36.447235+0000","last_became_active":"2023-10-30T12:35:05.177266+0000","last_became_peered":"2023-10-30T12:35:05.177266+0000","last_unstale":"2023-10-30T12:56:36.447235+0000","last_undegraded":"2023-10-30T12:56:36.447235+0000","last_fullsized":"2023-10-30T12:56:36.447235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'185","last_scrub_stamp":"2023-10-30T12:56:36.447125+0000","last_deep_scrub":"55'185","last_deep_scrub_stamp":"2023-10-30T12:56:36.447125+0000","last_clean_scrub_stamp":"2023-10-30T12:56:36.447125+0000","objects_scrubbed":2,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:52:35.695395+0000","scrub_duration":0.022317884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":129,"num_write_kb":8259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"55'846","reported_seq":3022,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:43.791966+0000","last_change":"2023-10-30T12:56:43.792110+0000","last_active":"2023-10-30T12:56:43.791966+0000","last_peered":"2023-10-30T12:56:43.791966+0000","last_clean":"2023-10-30T12:56:43.791966+0000","last_became_active":"2023-10-30T12:35:06.196307+0000","last_became_peered":"2023-10-30T12:35:06.196307+0000","last_unstale":"2023-10-30T12:56:43.791966+0000","last_undegraded":"2023-10-30T12:56:43.791966+0000","last_fullsized":"2023-10-30T12:56:43.791966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'846","last_scrub_stamp":"2023-10-30T12:56:43.791757+0000","last_deep_scrub":"55'846","last_deep_scrub_stamp":"2023-10-30T12:56:43.791757+0000","last_clean_scrub_stamp":"2023-10-30T12:56:43.791757+0000","objects_scrubbed":158,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:38:44.933357+0000","scrub_duration":0.30352262899999999,"objects_trimmed":0,"snaptrim_duration":0.015197106,"stat_sum":{"num_bytes":5885946,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":69,"num_read_kb":13,"num_write":633,"num_write_kb":24590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"55'541","reported_seq":613,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:04.631724+0000","last_change":"2023-10-30T12:56:58.010026+0000","last_active":"2023-10-30T12:57:04.631724+0000","last_peered":"2023-10-30T12:57:04.631724+0000","last_clean":"2023-10-30T12:57:04.631724+0000","last_became_active":"2023-10-30T12:35:05.206930+0000","last_became_peered":"2023-10-30T12:35:05.206930+0000","last_unstale":"2023-10-30T12:57:04.631724+0000","last_undegraded":"2023-10-30T12:57:04.631724+0000","last_fullsized":"2023-10-30T12:57:04.631724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'539","last_scrub_stamp":"2023-10-30T12:56:58.009697+0000","last_deep_scrub":"55'539","last_deep_scrub_stamp":"2023-10-30T12:56:58.009697+0000","last_clean_scrub_stamp":"2023-10-30T12:56:58.009697+0000","objects_scrubbed":11,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:24:47.680526+0000","scrub_duration":0.068773216999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16497412,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":8,"num_read_kb":9517,"num_write":471,"num_write_kb":16402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":56804,"num_omap_keys":105,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"55'886","reported_seq":3066,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:44.620900+0000","last_change":"2023-10-30T12:56:44.620946+0000","last_active":"2023-10-30T12:56:44.620900+0000","last_peered":"2023-10-30T12:56:44.620900+0000","last_clean":"2023-10-30T12:56:44.620900+0000","last_became_active":"2023-10-30T12:35:06.164733+0000","last_became_peered":"2023-10-30T12:35:06.164733+0000","last_unstale":"2023-10-30T12:56:44.620900+0000","last_undegraded":"2023-10-30T12:56:44.620900+0000","last_fullsized":"2023-10-30T12:56:44.620900+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'886","last_scrub_stamp":"2023-10-30T12:56:44.620823+0000","last_deep_scrub":"55'886","last_deep_scrub_stamp":"2023-10-30T12:56:44.620823+0000","last_clean_scrub_stamp":"2023-10-30T12:56:44.620823+0000","objects_scrubbed":185,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:12.989394+0000","scrub_duration":0.199244536,"objects_trimmed":0,"snaptrim_duration":0.0024278020000000002,"stat_sum":{"num_bytes":19206810,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":72,"num_read_kb":4114,"num_write":652,"num_write_kb":32920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"55'381","reported_seq":458,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:26.587606+0000","last_change":"2023-10-30T12:56:26.587606+0000","last_active":"2023-10-30T12:56:26.587606+0000","last_peered":"2023-10-30T12:56:26.587606+0000","last_clean":"2023-10-30T12:56:26.587606+0000","last_became_active":"2023-10-30T12:35:05.184695+0000","last_became_peered":"2023-10-30T12:35:05.184695+0000","last_unstale":"2023-10-30T12:56:26.587606+0000","last_undegraded":"2023-10-30T12:56:26.587606+0000","last_fullsized":"2023-10-30T12:56:26.587606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'381","last_scrub_stamp":"2023-10-30T12:56:26.587467+0000","last_deep_scrub":"55'381","last_deep_scrub_stamp":"2023-10-30T12:56:26.587467+0000","last_clean_scrub_stamp":"2023-10-30T12:56:26.587467+0000","objects_scrubbed":7,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:27:14.820811+0000","scrub_duration":0.083109266000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":8489,"num_write":574,"num_write_kb":5859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54698,"num_omap_keys":106,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"55'856","reported_seq":3025,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:57:10.816652+0000","last_change":"2023-10-30T12:57:10.816704+0000","last_active":"2023-10-30T12:57:10.816652+0000","last_peered":"2023-10-30T12:57:10.816652+0000","last_clean":"2023-10-30T12:57:10.816652+0000","last_became_active":"2023-10-30T12:35:06.204422+0000","last_became_peered":"2023-10-30T12:35:06.204422+0000","last_unstale":"2023-10-30T12:57:10.816652+0000","last_undegraded":"2023-10-30T12:57:10.816652+0000","last_fullsized":"2023-10-30T12:57:10.816652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'856","last_scrub_stamp":"2023-10-30T12:57:10.816574+0000","last_deep_scrub":"55'856","last_deep_scrub_stamp":"2023-10-30T12:57:10.816574+0000","last_clean_scrub_stamp":"2023-10-30T12:57:10.816574+0000","objects_scrubbed":174,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:27:05.996137+0000","scrub_duration":0.332536045,"objects_trimmed":0,"snaptrim_duration":0.0048535030000000003,"stat_sum":{"num_bytes":22995855,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":62,"num_read_kb":4110,"num_write":640,"num_write_kb":33018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"55'236","reported_seq":308,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:23.771446+0000","last_change":"2023-10-30T12:56:23.771446+0000","last_active":"2023-10-30T12:56:23.771446+0000","last_peered":"2023-10-30T12:56:23.771446+0000","last_clean":"2023-10-30T12:56:23.771446+0000","last_became_active":"2023-10-30T12:35:05.200449+0000","last_became_peered":"2023-10-30T12:35:05.200449+0000","last_unstale":"2023-10-30T12:56:23.771446+0000","last_undegraded":"2023-10-30T12:56:23.771446+0000","last_fullsized":"2023-10-30T12:56:23.771446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'236","last_scrub_stamp":"2023-10-30T12:56:23.771024+0000","last_deep_scrub":"55'236","last_deep_scrub_stamp":"2023-10-30T12:56:23.771024+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.771024+0000","objects_scrubbed":6,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:55:13.133447+0000","scrub_duration":0.033750645000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":4117,"num_write":187,"num_write_kb":4299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18018,"num_omap_keys":33,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"55'831","reported_seq":3007,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:48.796437+0000","last_change":"2023-10-30T12:56:48.796491+0000","last_active":"2023-10-30T12:56:48.796437+0000","last_peered":"2023-10-30T12:56:48.796437+0000","last_clean":"2023-10-30T12:56:48.796437+0000","last_became_active":"2023-10-30T12:35:06.179412+0000","last_became_peered":"2023-10-30T12:35:06.179412+0000","last_unstale":"2023-10-30T12:56:48.796437+0000","last_undegraded":"2023-10-30T12:56:48.796437+0000","last_fullsized":"2023-10-30T12:56:48.796437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'831","last_scrub_stamp":"2023-10-30T12:56:48.796353+0000","last_deep_scrub":"55'831","last_deep_scrub_stamp":"2023-10-30T12:56:48.796353+0000","last_clean_scrub_stamp":"2023-10-30T12:56:48.796353+0000","objects_scrubbed":166,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:11:50.029607+0000","scrub_duration":0.362753192,"objects_trimmed":0,"snaptrim_duration":0.003661568,"stat_sum":{"num_bytes":1664351,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":69,"num_read_kb":22,"num_write":614,"num_write_kb":12460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"55'560","reported_seq":635,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:32.835075+0000","last_change":"2023-10-30T12:56:32.835075+0000","last_active":"2023-10-30T12:56:32.835075+0000","last_peered":"2023-10-30T12:56:32.835075+0000","last_clean":"2023-10-30T12:56:32.835075+0000","last_became_active":"2023-10-30T12:35:05.206607+0000","last_became_peered":"2023-10-30T12:35:05.206607+0000","last_unstale":"2023-10-30T12:56:32.835075+0000","last_undegraded":"2023-10-30T12:56:32.835075+0000","last_fullsized":"2023-10-30T12:56:32.835075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'560","last_scrub_stamp":"2023-10-30T12:56:32.834680+0000","last_deep_scrub":"55'560","last_deep_scrub_stamp":"2023-10-30T12:56:32.834680+0000","last_clean_scrub_stamp":"2023-10-30T12:56:32.834680+0000","objects_scrubbed":8,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:19:51.459668+0000","scrub_duration":0.066587212000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":17,"num_read_kb":23179,"num_write":497,"num_write_kb":12662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34806,"num_omap_keys":64,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"55'809","reported_seq":2986,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.829353+0000","last_change":"2023-10-30T12:56:24.829353+0000","last_active":"2023-10-30T12:56:24.829353+0000","last_peered":"2023-10-30T12:56:24.829353+0000","last_clean":"2023-10-30T12:56:24.829353+0000","last_became_active":"2023-10-30T12:35:06.181256+0000","last_became_peered":"2023-10-30T12:35:06.181256+0000","last_unstale":"2023-10-30T12:56:24.829353+0000","last_undegraded":"2023-10-30T12:56:24.829353+0000","last_fullsized":"2023-10-30T12:56:24.829353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'809","last_scrub_stamp":"2023-10-30T12:56:24.829117+0000","last_deep_scrub":"55'809","last_deep_scrub_stamp":"2023-10-30T12:56:24.829117+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.829117+0000","objects_scrubbed":153,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:42:27.862243+0000","scrub_duration":0.27743204100000002,"objects_trimmed":0,"snaptrim_duration":0.0030387220000000002,"stat_sum":{"num_bytes":10464290,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":70,"num_read_kb":4103,"num_write":644,"num_write_kb":29233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"55'374","reported_seq":447,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:24.513468+0000","last_change":"2023-10-30T12:56:24.513468+0000","last_active":"2023-10-30T12:56:24.513468+0000","last_peered":"2023-10-30T12:56:24.513468+0000","last_clean":"2023-10-30T12:56:24.513468+0000","last_became_active":"2023-10-30T12:35:05.200724+0000","last_became_peered":"2023-10-30T12:35:05.200724+0000","last_unstale":"2023-10-30T12:56:24.513468+0000","last_undegraded":"2023-10-30T12:56:24.513468+0000","last_fullsized":"2023-10-30T12:56:24.513468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'374","last_scrub_stamp":"2023-10-30T12:56:24.513345+0000","last_deep_scrub":"55'374","last_deep_scrub_stamp":"2023-10-30T12:56:24.513345+0000","last_clean_scrub_stamp":"2023-10-30T12:56:24.513345+0000","objects_scrubbed":7,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:34:47.132641+0000","scrub_duration":0.034143153000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388698,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":12300,"num_write":322,"num_write_kb":8548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25586,"num_omap_keys":47,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"55'835","reported_seq":3000,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:35.315188+0000","last_change":"2023-10-30T12:56:35.315188+0000","last_active":"2023-10-30T12:56:35.315188+0000","last_peered":"2023-10-30T12:56:35.315188+0000","last_clean":"2023-10-30T12:56:35.315188+0000","last_became_active":"2023-10-30T12:35:06.173996+0000","last_became_peered":"2023-10-30T12:35:06.173996+0000","last_unstale":"2023-10-30T12:56:35.315188+0000","last_undegraded":"2023-10-30T12:56:35.315188+0000","last_fullsized":"2023-10-30T12:56:35.315188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'835","last_scrub_stamp":"2023-10-30T12:56:35.315063+0000","last_deep_scrub":"55'835","last_deep_scrub_stamp":"2023-10-30T12:56:35.315063+0000","last_clean_scrub_stamp":"2023-10-30T12:56:35.315063+0000","objects_scrubbed":159,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:30:07.209945+0000","scrub_duration":0.14615183500000001,"objects_trimmed":0,"snaptrim_duration":0.0045562010000000002,"stat_sum":{"num_bytes":5693555,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":58,"num_read_kb":18,"num_write":647,"num_write_kb":16270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"55'881","reported_seq":952,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:30.589026+0000","last_change":"2023-10-30T12:56:30.589026+0000","last_active":"2023-10-30T12:56:30.589026+0000","last_peered":"2023-10-30T12:56:30.589026+0000","last_clean":"2023-10-30T12:56:30.589026+0000","last_became_active":"2023-10-30T12:35:05.163794+0000","last_became_peered":"2023-10-30T12:35:05.163794+0000","last_unstale":"2023-10-30T12:56:30.589026+0000","last_undegraded":"2023-10-30T12:56:30.589026+0000","last_fullsized":"2023-10-30T12:56:30.589026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'881","last_scrub_stamp":"2023-10-30T12:56:30.588883+0000","last_deep_scrub":"55'881","last_deep_scrub_stamp":"2023-10-30T12:56:30.588883+0000","last_clean_scrub_stamp":"2023-10-30T12:56:30.588883+0000","objects_scrubbed":7,"log_size":881,"log_dups_size":0,"ondisk_log_size":881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:50:39.690725+0000","scrub_duration":0.062172023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":10,"num_read_kb":8454,"num_write":955,"num_write_kb":13780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23914,"num_omap_keys":46,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"55'808","reported_seq":2983,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:51.694150+0000","last_change":"2023-10-30T12:56:51.694150+0000","last_active":"2023-10-30T12:56:51.694150+0000","last_peered":"2023-10-30T12:56:51.694150+0000","last_clean":"2023-10-30T12:56:51.694150+0000","last_became_active":"2023-10-30T12:35:06.183226+0000","last_became_peered":"2023-10-30T12:35:06.183226+0000","last_unstale":"2023-10-30T12:56:51.694150+0000","last_undegraded":"2023-10-30T12:56:51.694150+0000","last_fullsized":"2023-10-30T12:56:51.694150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'808","last_scrub_stamp":"2023-10-30T12:56:51.694014+0000","last_deep_scrub":"55'808","last_deep_scrub_stamp":"2023-10-30T12:56:51.694014+0000","last_clean_scrub_stamp":"2023-10-30T12:56:51.694014+0000","objects_scrubbed":140,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:16:30.542141+0000","scrub_duration":0.15061528399999999,"objects_trimmed":0,"snaptrim_duration":0.0052521379999999999,"stat_sum":{"num_bytes":5799195,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":69,"num_read_kb":4112,"num_write":618,"num_write_kb":16472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"55'344","reported_seq":420,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:33.883350+0000","last_change":"2023-10-30T12:56:33.883350+0000","last_active":"2023-10-30T12:56:33.883350+0000","last_peered":"2023-10-30T12:56:33.883350+0000","last_clean":"2023-10-30T12:56:33.883350+0000","last_became_active":"2023-10-30T12:35:05.206518+0000","last_became_peered":"2023-10-30T12:35:05.206518+0000","last_unstale":"2023-10-30T12:56:33.883350+0000","last_undegraded":"2023-10-30T12:56:33.883350+0000","last_fullsized":"2023-10-30T12:56:33.883350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'344","last_scrub_stamp":"2023-10-30T12:56:33.882814+0000","last_deep_scrub":"55'344","last_deep_scrub_stamp":"2023-10-30T12:56:33.882814+0000","last_clean_scrub_stamp":"2023-10-30T12:56:33.882814+0000","objects_scrubbed":8,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:13:20.348463+0000","scrub_duration":0.065105804000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":268,"num_write_kb":12461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":36978,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"55'781","reported_seq":2956,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:29.111348+0000","last_change":"2023-10-30T12:56:29.111348+0000","last_active":"2023-10-30T12:56:29.111348+0000","last_peered":"2023-10-30T12:56:29.111348+0000","last_clean":"2023-10-30T12:56:29.111348+0000","last_became_active":"2023-10-30T12:35:06.197861+0000","last_became_peered":"2023-10-30T12:35:06.197861+0000","last_unstale":"2023-10-30T12:56:29.111348+0000","last_undegraded":"2023-10-30T12:56:29.111348+0000","last_fullsized":"2023-10-30T12:56:29.111348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'781","last_scrub_stamp":"2023-10-30T12:56:29.111192+0000","last_deep_scrub":"55'781","last_deep_scrub_stamp":"2023-10-30T12:56:29.111192+0000","last_clean_scrub_stamp":"2023-10-30T12:56:29.111192+0000","objects_scrubbed":157,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:53:54.048846+0000","scrub_duration":0.27439538299999999,"objects_trimmed":0,"snaptrim_duration":0.011118161,"stat_sum":{"num_bytes":14763185,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":68,"num_read_kb":15,"num_write":609,"num_write_kb":24605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"55'844","reported_seq":3031,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:49.658623+0000","last_change":"2023-10-30T12:56:49.658623+0000","last_active":"2023-10-30T12:56:49.658623+0000","last_peered":"2023-10-30T12:56:49.658623+0000","last_clean":"2023-10-30T12:56:49.658623+0000","last_became_active":"2023-10-30T12:35:06.212548+0000","last_became_peered":"2023-10-30T12:35:06.212548+0000","last_unstale":"2023-10-30T12:56:49.658623+0000","last_undegraded":"2023-10-30T12:56:49.658623+0000","last_fullsized":"2023-10-30T12:56:49.658623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'844","last_scrub_stamp":"2023-10-30T12:56:49.658439+0000","last_deep_scrub":"55'844","last_deep_scrub_stamp":"2023-10-30T12:56:49.658439+0000","last_clean_scrub_stamp":"2023-10-30T12:56:49.658439+0000","objects_scrubbed":159,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:08:37.480683+0000","scrub_duration":0.29739869600000002,"objects_trimmed":0,"snaptrim_duration":0.015114931999999999,"stat_sum":{"num_bytes":19366418,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":81,"num_read_kb":7349,"num_write":643,"num_write_kb":32740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"55'368","reported_seq":438,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.907429+0000","last_change":"2023-10-30T12:56:34.907429+0000","last_active":"2023-10-30T12:56:34.907429+0000","last_peered":"2023-10-30T12:56:34.907429+0000","last_clean":"2023-10-30T12:56:34.907429+0000","last_became_active":"2023-10-30T12:35:05.206559+0000","last_became_peered":"2023-10-30T12:35:05.206559+0000","last_unstale":"2023-10-30T12:56:34.907429+0000","last_undegraded":"2023-10-30T12:56:34.907429+0000","last_fullsized":"2023-10-30T12:56:34.907429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'368","last_scrub_stamp":"2023-10-30T12:56:34.907076+0000","last_deep_scrub":"55'368","last_deep_scrub_stamp":"2023-10-30T12:56:34.907076+0000","last_clean_scrub_stamp":"2023-10-30T12:56:34.907076+0000","objects_scrubbed":5,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:12:13.284723+0000","scrub_duration":0.045168409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":16393,"num_write":312,"num_write_kb":8389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19072,"num_omap_keys":35,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"55'843","reported_seq":3019,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:50.637414+0000","last_change":"2023-10-30T12:56:50.637414+0000","last_active":"2023-10-30T12:56:50.637414+0000","last_peered":"2023-10-30T12:56:50.637414+0000","last_clean":"2023-10-30T12:56:50.637414+0000","last_became_active":"2023-10-30T12:35:06.212505+0000","last_became_peered":"2023-10-30T12:35:06.212505+0000","last_unstale":"2023-10-30T12:56:50.637414+0000","last_undegraded":"2023-10-30T12:56:50.637414+0000","last_fullsized":"2023-10-30T12:56:50.637414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'843","last_scrub_stamp":"2023-10-30T12:56:50.637270+0000","last_deep_scrub":"55'843","last_deep_scrub_stamp":"2023-10-30T12:56:50.637270+0000","last_clean_scrub_stamp":"2023-10-30T12:56:50.637270+0000","objects_scrubbed":156,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:28:10.699275+0000","scrub_duration":0.27534876899999999,"objects_trimmed":0,"snaptrim_duration":0.014553393,"stat_sum":{"num_bytes":9800415,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":70,"num_read_kb":12304,"num_write":642,"num_write_kb":36520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"55'387","reported_seq":468,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:37.980331+0000","last_change":"2023-10-30T12:56:37.980331+0000","last_active":"2023-10-30T12:56:37.980331+0000","last_peered":"2023-10-30T12:56:37.980331+0000","last_clean":"2023-10-30T12:56:37.980331+0000","last_became_active":"2023-10-30T12:35:05.206879+0000","last_became_peered":"2023-10-30T12:35:05.206879+0000","last_unstale":"2023-10-30T12:56:37.980331+0000","last_undegraded":"2023-10-30T12:56:37.980331+0000","last_fullsized":"2023-10-30T12:56:37.980331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'387","last_scrub_stamp":"2023-10-30T12:56:37.980050+0000","last_deep_scrub":"55'387","last_deep_scrub_stamp":"2023-10-30T12:56:37.980050+0000","last_clean_scrub_stamp":"2023-10-30T12:56:37.980050+0000","objects_scrubbed":10,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:37:17.148411+0000","scrub_duration":0.082641178999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":32,"num_read_kb":45093,"num_write":322,"num_write_kb":20730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31096,"num_omap_keys":59,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"55'779","reported_seq":2955,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:56.563086+0000","last_change":"2023-10-30T12:56:56.563165+0000","last_active":"2023-10-30T12:56:56.563086+0000","last_peered":"2023-10-30T12:56:56.563086+0000","last_clean":"2023-10-30T12:56:56.563086+0000","last_became_active":"2023-10-30T12:35:46.275077+0000","last_became_peered":"2023-10-30T12:35:46.275077+0000","last_unstale":"2023-10-30T12:56:56.563086+0000","last_undegraded":"2023-10-30T12:56:56.563086+0000","last_fullsized":"2023-10-30T12:56:56.563086+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'779","last_scrub_stamp":"2023-10-30T12:56:56.562988+0000","last_deep_scrub":"55'779","last_deep_scrub_stamp":"2023-10-30T12:56:56.562988+0000","last_clean_scrub_stamp":"2023-10-30T12:56:56.562988+0000","objects_scrubbed":136,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:59:27.981737+0000","scrub_duration":0.14057847700000001,"objects_trimmed":0,"snaptrim_duration":0.0032416139999999999,"stat_sum":{"num_bytes":6205387,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":63,"num_read_kb":4113,"num_write":608,"num_write_kb":9154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"55'512","reported_seq":583,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:23.020777+0000","last_change":"2023-10-30T12:56:23.020777+0000","last_active":"2023-10-30T12:56:23.020777+0000","last_peered":"2023-10-30T12:56:23.020777+0000","last_clean":"2023-10-30T12:56:23.020777+0000","last_became_active":"2023-10-30T12:35:05.173984+0000","last_became_peered":"2023-10-30T12:35:05.173984+0000","last_unstale":"2023-10-30T12:56:23.020777+0000","last_undegraded":"2023-10-30T12:56:23.020777+0000","last_fullsized":"2023-10-30T12:56:23.020777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'512","last_scrub_stamp":"2023-10-30T12:56:23.020659+0000","last_deep_scrub":"55'512","last_deep_scrub_stamp":"2023-10-30T12:56:23.020659+0000","last_clean_scrub_stamp":"2023-10-30T12:56:23.020659+0000","objects_scrubbed":5,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:43:31.108759+0000","scrub_duration":0.032786733999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":430,"num_write_kb":12596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19196,"num_omap_keys":36,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"55'882","reported_seq":3052,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:49.685736+0000","last_change":"2023-10-30T12:56:49.685736+0000","last_active":"2023-10-30T12:56:49.685736+0000","last_peered":"2023-10-30T12:56:49.685736+0000","last_clean":"2023-10-30T12:56:49.685736+0000","last_became_active":"2023-10-30T12:35:06.198504+0000","last_became_peered":"2023-10-30T12:35:06.198504+0000","last_unstale":"2023-10-30T12:56:49.685736+0000","last_undegraded":"2023-10-30T12:56:49.685736+0000","last_fullsized":"2023-10-30T12:56:49.685736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'882","last_scrub_stamp":"2023-10-30T12:56:49.685619+0000","last_deep_scrub":"55'882","last_deep_scrub_stamp":"2023-10-30T12:56:49.685619+0000","last_clean_scrub_stamp":"2023-10-30T12:56:49.685619+0000","objects_scrubbed":182,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:10:12.105803+0000","scrub_duration":0.23854751900000001,"objects_trimmed":0,"snaptrim_duration":0.0055292249999999996,"stat_sum":{"num_bytes":14413504,"num_objects":182,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":61,"num_read_kb":1995,"num_write":647,"num_write_kb":19115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"55'216","reported_seq":307,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-30T12:56:20.534250+0000","last_change":"2023-10-30T12:56:20.534300+0000","last_active":"2023-10-30T12:56:20.534250+0000","last_peered":"2023-10-30T12:56:20.534250+0000","last_clean":"2023-10-30T12:56:20.534250+0000","last_became_active":"2023-10-30T12:35:05.200623+0000","last_became_peered":"2023-10-30T12:35:05.200623+0000","last_unstale":"2023-10-30T12:56:20.534250+0000","last_undegraded":"2023-10-30T12:56:20.534250+0000","last_fullsized":"2023-10-30T12:56:20.534250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'216","last_scrub_stamp":"2023-10-30T12:56:20.534185+0000","last_deep_scrub":"55'216","last_deep_scrub_stamp":"2023-10-30T12:56:20.534185+0000","last_clean_scrub_stamp":"2023-10-30T12:56:20.534185+0000","objects_scrubbed":8,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:19:00.729850+0000","scrub_duration":0.044608693999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":63,"num_read_kb":24633,"num_write":147,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7154,"num_omap_keys":14,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":925262768,"num_objects":10302,"num_object_clones":0,"num_object_copies":20604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10302,"num_whiteouts":0,"num_read":4339,"num_read_kb":400205,"num_write":41481,"num_write_kb":1950238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1902673920,"data_stored":1848985440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54082,"ondisk_log_size":54082,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":360906037,"num_objects":211,"num_object_clones":0,"num_object_copies":422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":211,"num_whiteouts":0,"num_read":1232,"num_read_kb":640401,"num_write":11731,"num_write_kb":363311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":636231,"num_omap_keys":1188,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":721977344,"data_stored":721812074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13275,"ondisk_log_size":13275,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542423,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488608,"kb_used_data":403288,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":93883232,"statfs":{"total":96636764160,"available":96136429568,"internally_reserved":0,"allocated":412966912,"data_stored":404891962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":87357901},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.85399999999999998,"15min":0.82599999999999996},"min":{"1min":0.60899999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.232,"5min":1.371,"15min":1.371},"last":0.871},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.86299999999999999,"15min":0.82699999999999996},"min":{"1min":0.55900000000000005,"5min":0.47199999999999998,"15min":0.41699999999999998},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.45},"last":0.80200000000000005}]},{"osd":1,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.82999999999999996,"15min":0.84899999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.44700000000000001},"max":{"1min":1.0049999999999999,"5min":1.222,"15min":1.496},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.91600000000000004,"15min":0.89700000000000002},"min":{"1min":0.56100000000000005,"5min":0.55700000000000005,"15min":0.51800000000000002},"max":{"1min":1.3009999999999999,"5min":1.494,"15min":1.494},"last":0.70799999999999996}]},{"osd":2,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.83099999999999996,"15min":0.82299999999999995},"min":{"1min":0.58699999999999997,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.139,"5min":1.2390000000000001,"15min":1.242},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.91200000000000003,"15min":0.90300000000000002},"min":{"1min":0.66100000000000003,"5min":0.60299999999999998,"15min":0.48999999999999999},"max":{"1min":1.3160000000000001,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.60899999999999999}]},{"osd":3,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0089999999999999,"15min":0.93500000000000005},"min":{"1min":0.77700000000000002,"5min":0.70799999999999996,"15min":0.54000000000000004},"max":{"1min":1.357,"5min":1.474,"15min":1.607},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.95799999999999996,"15min":0.99199999999999999},"min":{"1min":0.623,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1040000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.75600000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:56:42 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.91400000000000003,"15min":0.93200000000000005},"min":{"1min":0.622,"5min":0.53300000000000003,"15min":0.46899999999999997},"max":{"1min":1.1259999999999999,"5min":1.4059999999999999,"15min":2.7309999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1,"5min":1.0129999999999999,"15min":0.998},"min":{"1min":0.75700000000000001,"5min":0.64800000000000002,"15min":0.41999999999999998},"max":{"1min":1.3680000000000001,"5min":1.694,"15min":1.8859999999999999},"last":0.95599999999999996}]},{"osd":5,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.073,"15min":1.0840000000000001},"min":{"1min":0.71799999999999997,"5min":0.61499999999999999,"15min":0.59399999999999997},"max":{"1min":1.4770000000000001,"5min":1.6379999999999999,"15min":1.647},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.252,"15min":1.173},"min":{"1min":0.99299999999999999,"5min":0.89900000000000002,"15min":0.71899999999999997},"max":{"1min":1.5549999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.0980000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.03,"15min":1.0860000000000001},"min":{"1min":0.86199999999999999,"5min":0.65100000000000002,"15min":0.63600000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.802},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1479999999999999,"15min":1.1919999999999999},"min":{"1min":0.99199999999999999,"5min":0.879,"15min":0.83799999999999997},"max":{"1min":1.605,"5min":1.605,"15min":1.881},"last":1.0529999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542422,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389540,"kb_used_data":300124,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":93982300,"statfs":{"total":96636764160,"available":96237875200,"internally_reserved":0,"allocated":307326976,"data_stored":301990625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":91552201},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.83499999999999996,"15min":0.82299999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38600000000000001},"max":{"1min":2.0859999999999999,"5min":5.8760000000000003,"15min":5.8760000000000003},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.877,"15min":0.84599999999999997},"min":{"1min":0.51700000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.89700000000000002}]},{"osd":1,"last update":"Mon Oct 30 12:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82699999999999996,"15min":0.90100000000000002},"min":{"1min":0.52100000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3069999999999999,"5min":1.6379999999999999,"15min":5.0949999999999998},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.018,"5min":0.97799999999999998,"15min":0.92900000000000005},"min":{"1min":0.65000000000000002,"5min":0.58099999999999996,"15min":0.45500000000000002},"max":{"1min":1.2390000000000001,"5min":1.423,"15min":5.0519999999999996},"last":1.335}]},{"osd":2,"last update":"Mon Oct 30 12:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.93500000000000005,"15min":0.85899999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.421,"5min":2.649,"15min":2.649},"last":1.28},{"interface":"front","average":{"1min":1.028,"5min":1.04,"15min":0.94799999999999995},"min":{"1min":0.72999999999999998,"5min":0.63100000000000001,"15min":0.44700000000000001},"max":{"1min":1.375,"5min":2.282,"15min":2.282},"last":1.091}]},{"osd":3,"last update":"Mon Oct 30 12:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84099999999999997,"15min":0.92600000000000005},"min":{"1min":0.64800000000000002,"5min":0.51700000000000002,"15min":0.40200000000000002},"max":{"1min":1.081,"5min":1.351,"15min":1.929},"last":1.119},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95899999999999996,"15min":0.98399999999999999},"min":{"1min":0.73099999999999998,"5min":0.58499999999999996,"15min":0.42699999999999999},"max":{"1min":1.133,"5min":1.444,"15min":1.7589999999999999},"last":0.85199999999999998}]},{"osd":4,"last update":"Mon Oct 30 12:57:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0609999999999999,"15min":1.0069999999999999},"min":{"1min":0.70899999999999996,"5min":0.63600000000000001,"15min":0.27600000000000002},"max":{"1min":1.556,"5min":1.617,"15min":1.9219999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.998,"5min":1.0469999999999999,"15min":1.0189999999999999},"min":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.32400000000000001},"max":{"1min":2.605,"5min":2.605,"15min":2.605},"last":0.78200000000000003}]},{"osd":5,"last update":"Mon Oct 30 12:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.95299999999999996,"15min":1.0129999999999999},"min":{"1min":0.67200000000000004,"5min":0.66600000000000004,"15min":0.29599999999999999},"max":{"1min":1.127,"5min":1.4910000000000001,"15min":2.9529999999999998},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.0509999999999999,"15min":1.1120000000000001},"min":{"1min":0.83199999999999996,"5min":0.78300000000000003,"15min":0.34399999999999997},"max":{"1min":1.27,"5min":1.7010000000000001,"15min":2.5680000000000001},"last":1.232}]},{"osd":7,"last update":"Mon Oct 30 12:57:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.169,"15min":1.145},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.37},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.2330000000000001,"15min":1.1950000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.46800000000000003},"max":{"1min":1.486,"5min":1.7829999999999999,"15min":2.274},"last":1.391}]}]},{"osd":4,"up_from":14,"seq":60129542421,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408876,"kb_used_data":319396,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":93962964,"statfs":{"total":96636764160,"available":96218075136,"internally_reserved":0,"allocated":327061504,"data_stored":318875235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":91617734},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78000000000000003,"15min":0.80900000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.437},"max":{"1min":1.448,"5min":1.448,"15min":1.5649999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.82999999999999996,"15min":0.82799999999999996},"min":{"1min":0.56000000000000005,"5min":0.49299999999999999,"15min":0.39900000000000002},"max":{"1min":1.268,"5min":1.268,"15min":2.0129999999999999},"last":0.66500000000000004}]},{"osd":1,"last update":"Mon Oct 30 12:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.871,"15min":0.90500000000000003},"min":{"1min":0.51600000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1459999999999999,"5min":1.28,"15min":2.4910000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86099999999999999,"15min":0.85099999999999998},"min":{"1min":0.58999999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.1830000000000001,"5min":1.2130000000000001,"15min":1.9730000000000001},"last":0.748}]},{"osd":2,"last update":"Mon Oct 30 12:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.871,"15min":0.93899999999999995},"min":{"1min":0.58199999999999996,"5min":0.56899999999999995,"15min":0.53400000000000003},"max":{"1min":0.93400000000000005,"5min":1.488,"15min":4.5919999999999996},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.98999999999999999,"15min":1.0389999999999999},"min":{"1min":0.80500000000000005,"5min":0.68999999999999995,"15min":0.55600000000000005},"max":{"1min":1.2150000000000001,"5min":1.589,"15min":4.6879999999999997},"last":1.087}]},{"osd":3,"last update":"Mon Oct 30 12:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.88800000000000001,"15min":0.91300000000000003},"min":{"1min":0.54200000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2410000000000001,"5min":1.284,"15min":2.3940000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.97899999999999998,"15min":0.92200000000000004},"min":{"1min":0.64600000000000002,"5min":0.54300000000000004,"15min":0.53500000000000003},"max":{"1min":1.4059999999999999,"5min":1.411,"15min":1.6890000000000001},"last":1.0229999999999999}]},{"osd":5,"last update":"Mon Oct 30 12:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.91500000000000004,"15min":0.997},"min":{"1min":0.60499999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.3129999999999999,"5min":1.4570000000000001,"15min":2.3180000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.133,"15min":1.0820000000000001},"min":{"1min":0.97799999999999998,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3600000000000001,"5min":1.55,"15min":3.173},"last":1.1859999999999999}]},{"osd":6,"last update":"Mon Oct 30 12:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.98699999999999999,"15min":1.0649999999999999},"min":{"1min":0.69199999999999995,"5min":0.67400000000000004,"15min":0.59299999999999997},"max":{"1min":1.254,"5min":1.641,"15min":2.2709999999999999},"last":1},{"interface":"front","average":{"1min":1.175,"5min":1.0980000000000001,"15min":1.091},"min":{"1min":0.97299999999999998,"5min":0.748,"15min":0.56499999999999995},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":4.7130000000000001},"last":1.2310000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:57:35 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.0489999999999999,"15min":1.081},"min":{"1min":0.79400000000000004,"5min":0.70699999999999996,"15min":0.64900000000000002},"max":{"1min":1.329,"5min":1.395,"15min":2.3799999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.169,"15min":1.1279999999999999},"min":{"1min":0.81200000000000006,"5min":0.70299999999999996,"15min":0.65300000000000002},"max":{"1min":1.55,"5min":1.766,"15min":2.375},"last":1.1379999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542422,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":286224,"kb_used_data":241160,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94085616,"statfs":{"total":96636764160,"available":96343670784,"internally_reserved":0,"allocated":246947840,"data_stored":241832282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.51200000000000001,"15min":0.48499999999999999},"min":{"1min":0.23999999999999999,"5min":0.23200000000000001,"15min":0.22700000000000001},"max":{"1min":0.55000000000000004,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.58099999999999996,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.28699999999999998},"max":{"1min":0.64700000000000002,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":0.44800000000000001}]},{"osd":1,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":0.61199999999999999,"5min":3.5059999999999998,"15min":4.5019999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.35299999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.53400000000000003,"5min":3.5430000000000001,"15min":3.5430000000000001},"last":0.52100000000000002}]},{"osd":3,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.76400000000000001,"15min":0.60399999999999998},"min":{"1min":0.309,"5min":0.26800000000000002,"15min":0.26100000000000001},"max":{"1min":0.71499999999999997,"5min":18.747,"15min":18.747},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.86199999999999999,"15min":0.69999999999999996},"min":{"1min":0.42599999999999999,"5min":0.38800000000000001,"15min":0.38300000000000001},"max":{"1min":0.81000000000000005,"5min":18.863,"15min":18.863},"last":0.54500000000000004}]},{"osd":4,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.85899999999999999,"15min":0.85999999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0620000000000001,"5min":3.0760000000000001,"15min":3.0760000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.95199999999999996,"15min":0.89900000000000002},"min":{"1min":0.59899999999999998,"5min":0.57599999999999996,"15min":0.48299999999999998},"max":{"1min":1.0029999999999999,"5min":9.6289999999999996,"15min":9.6289999999999996},"last":0.83999999999999997}]},{"osd":5,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.88,"15min":0.84799999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98099999999999998,"5min":9.7289999999999992,"15min":9.7289999999999992},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.9430000000000001,"15min":1.2},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.51900000000000002},"max":{"1min":1.0049999999999999,"5min":91.072999999999993,"15min":91.072999999999993},"last":0.64100000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":1.046,"15min":0.91500000000000004},"min":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.52800000000000002},"max":{"1min":1.137,"5min":19.126000000000001,"15min":19.126000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":1.26,"15min":1.008},"min":{"1min":0.67600000000000005,"5min":0.60299999999999998,"15min":0.59399999999999997},"max":{"1min":1.21,"5min":21.469999999999999,"15min":21.469999999999999},"last":0.88200000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:56:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.94499999999999995,"15min":0.89800000000000002},"min":{"1min":0.628,"5min":0.61299999999999999,"15min":0.52100000000000002},"max":{"1min":1.159,"5min":9.6869999999999994,"15min":9.6869999999999994},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.91600000000000004,"5min":1.147,"15min":0.97299999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.53200000000000003},"max":{"1min":1.129,"5min":19.050999999999998,"15min":19.050999999999998},"last":0.92700000000000005}]}]},{"osd":0,"up_from":14,"seq":60129542423,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422368,"kb_used_data":341208,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":93949472,"statfs":{"total":96636764160,"available":96204259328,"internally_reserved":0,"allocated":349396992,"data_stored":341144786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":83098056},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 12:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.82599999999999996,"15min":0.60799999999999998},"min":{"1min":0.30399999999999999,"5min":0.25900000000000001,"15min":0.248},"max":{"1min":0.67400000000000004,"5min":38.555999999999997,"15min":38.555999999999997},"last":0.373},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.60699999999999998,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.24299999999999999},"max":{"1min":0.64100000000000001,"5min":11.653,"15min":35.084000000000003},"last":0.47599999999999998}]},{"osd":2,"last update":"Mon Oct 30 12:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.86899999999999999,"15min":0.75},"min":{"1min":0.36099999999999999,"5min":0.312,"15min":0.29799999999999999},"max":{"1min":0.63200000000000001,"5min":38.755000000000003,"15min":38.755000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.93400000000000005,"15min":0.81200000000000006},"min":{"1min":0.41399999999999998,"5min":0.37,"15min":0.35299999999999998},"max":{"1min":0.73999999999999999,"5min":38.506999999999998,"15min":38.506999999999998},"last":0.51000000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.81100000000000005,"15min":0.64600000000000002},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.29499999999999998},"max":{"1min":0.70399999999999996,"5min":29.565000000000001,"15min":29.565000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.68799999999999994,"15min":0.66200000000000003},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":11.609,"15min":11.916},"last":0.66300000000000003}]},{"osd":4,"last update":"Mon Oct 30 12:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":1.04,"15min":0.877},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.373},"max":{"1min":0.998,"5min":21.550999999999998,"15min":21.550999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.232,"15min":1.0660000000000001},"min":{"1min":0.76700000000000002,"5min":0.47999999999999998,"15min":0.39900000000000002},"max":{"1min":1.1180000000000001,"5min":38.798000000000002,"15min":38.798000000000002},"last":0.96699999999999997}]},{"osd":5,"last update":"Mon Oct 30 12:56:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":0.92800000000000005,"15min":0.88300000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.5},"max":{"1min":11.683999999999999,"5min":11.683999999999999,"15min":11.683999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.48799999999999999},"max":{"1min":11.714,"5min":11.714,"15min":34.698},"last":0.75600000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.95699999999999996,"15min":0.85699999999999998},"min":{"1min":0.68500000000000005,"5min":0.57699999999999996,"15min":0.47799999999999998},"max":{"1min":0.96799999999999997,"5min":16.495999999999999,"15min":16.495999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.94299999999999995,"15min":0.90600000000000003},"min":{"1min":0.70199999999999996,"5min":0.64000000000000001,"15min":0.50900000000000001},"max":{"1min":1.169,"5min":5.6479999999999997,"15min":8.8800000000000008},"last":1.024}]},{"osd":7,"last update":"Mon Oct 30 12:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":1.046,"15min":0.90300000000000002},"min":{"1min":0.66600000000000004,"5min":0.61799999999999999,"15min":0.437},"max":{"1min":1.093,"5min":21.462,"15min":21.462},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.879,"5min":1.1679999999999999,"15min":1.024},"min":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.55900000000000005},"max":{"1min":1.083,"5min":28.943999999999999,"15min":28.943999999999999},"last":0.80000000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542421,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461024,"kb_used_data":379864,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":93910816,"statfs":{"total":96636764160,"available":96164675584,"internally_reserved":0,"allocated":388980736,"data_stored":380853345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":83098057},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.95299999999999996,"15min":0.86599999999999999},"min":{"1min":0.504,"5min":0.42199999999999999,"15min":0.41899999999999998},"max":{"1min":1.204,"5min":7.9409999999999998,"15min":7.9409999999999998},"last":1.121},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.87,"15min":0.85099999999999998},"min":{"1min":0.55900000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1930000000000001,"5min":1.341,"15min":1.468},"last":0.748}]},{"osd":1,"last update":"Mon Oct 30 12:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.90900000000000003,"15min":0.91600000000000004},"min":{"1min":0.55000000000000004,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.1699999999999999,"5min":12.224,"15min":12.224},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.86899999999999999,"15min":0.874},"min":{"1min":0.60099999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.163,"5min":3.044,"15min":3.044},"last":1.016}]},{"osd":2,"last update":"Mon Oct 30 12:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.91000000000000003,"15min":0.876},"min":{"1min":0.60499999999999998,"5min":0.53500000000000003,"15min":0.501},"max":{"1min":1.173,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92300000000000004,"15min":0.85699999999999998},"min":{"1min":0.68799999999999994,"5min":0.54600000000000004,"15min":0.442},"max":{"1min":1.2110000000000001,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":1.2110000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.228,"15min":1.403},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.504},"max":{"1min":1.288,"5min":31.815000000000001,"15min":104.88500000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.101,"15min":1.3979999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.60699999999999998},"max":{"1min":1.2609999999999999,"5min":8.9019999999999992,"15min":105.042},"last":1.2609999999999999}]},{"osd":4,"last update":"Mon Oct 30 12:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.054,"15min":0.95099999999999996},"min":{"1min":0.65200000000000002,"5min":0.498,"15min":0.38500000000000001},"max":{"1min":1.3029999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0780000000000001,"15min":1.1180000000000001},"min":{"1min":0.751,"5min":0.59699999999999998,"15min":0.57399999999999995},"max":{"1min":1.3520000000000001,"5min":1.5229999999999999,"15min":1.6479999999999999},"last":1.3520000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:57:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.014,"15min":0.96199999999999997},"min":{"1min":0.747,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.996},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1220000000000001,"15min":1.0669999999999999},"min":{"1min":0.81100000000000005,"5min":0.72199999999999998,"15min":0.48899999999999999},"max":{"1min":1.5960000000000001,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.173}]},{"osd":7,"last update":"Mon Oct 30 12:57:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.341,"15min":1.175},"min":{"1min":0.97399999999999998,"5min":0.68899999999999995,"15min":0.40600000000000003},"max":{"1min":1.6160000000000001,"5min":24.173999999999999,"15min":24.173999999999999},"last":1.083},{"interface":"front","average":{"1min":1.157,"5min":1.2350000000000001,"15min":1.145},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.53300000000000003},"max":{"1min":1.4259999999999999,"5min":12.055,"15min":12.055},"last":1.3799999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542423,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437640,"kb_used_data":352320,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":93934200,"statfs":{"total":96636764160,"available":96188620800,"internally_reserved":0,"allocated":360775680,"data_stored":352416873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":87357898},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.437,"15min":0.436},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.224},"max":{"1min":0.65900000000000003,"5min":0.77100000000000002,"15min":2.1459999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.54600000000000004},"min":{"1min":0.29799999999999999,"5min":0.28999999999999998,"15min":0.27200000000000002},"max":{"1min":0.65200000000000002,"5min":0.77900000000000003,"15min":21.495999999999999},"last":0.86099999999999999}]},{"osd":1,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.496,"15min":0.53000000000000003},"min":{"1min":0.33200000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.63400000000000001,"5min":1.0700000000000001,"15min":1.1459999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.503,"15min":0.55500000000000005},"min":{"1min":0.40200000000000002,"5min":0.32600000000000001,"15min":0.30399999999999999},"max":{"1min":0.67300000000000004,"5min":0.83299999999999996,"15min":15.102},"last":0.81799999999999995}]},{"osd":2,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53700000000000003,"15min":0.60599999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.28299999999999997},"max":{"1min":0.85899999999999999,"5min":0.86399999999999999,"15min":21.321000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.379},"max":{"1min":0.82899999999999996,"5min":0.97099999999999997,"15min":1.681},"last":0.84099999999999997}]},{"osd":4,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82299999999999995,"15min":0.89800000000000002},"min":{"1min":0.56399999999999995,"5min":0.497,"15min":0.49199999999999999},"max":{"1min":1.323,"5min":1.323,"15min":21.382999999999999},"last":1.028},{"interface":"front","average":{"1min":0.874,"5min":0.876,"15min":0.92200000000000004},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.46700000000000003},"max":{"1min":1.1599999999999999,"5min":1.1910000000000001,"15min":15.231999999999999},"last":1.077}]},{"osd":5,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.85299999999999998,"15min":0.89800000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.47799999999999998},"max":{"1min":1.107,"5min":1.282,"15min":15.273999999999999},"last":1.01},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.91500000000000004,"15min":0.95499999999999996},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.57499999999999996},"max":{"1min":1.198,"5min":1.3120000000000001,"15min":15.359999999999999},"last":1.0609999999999999}]},{"osd":6,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.54600000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.276,"5min":1.296,"15min":1.5620000000000001},"last":1.042},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83499999999999996,"15min":0.879},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.498},"max":{"1min":1.0569999999999999,"5min":1.198,"15min":15.313000000000001},"last":0.95199999999999996}]},{"osd":7,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.85399999999999998,"15min":0.93000000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56200000000000006},"max":{"1min":1.1080000000000001,"5min":1.177,"15min":21.428999999999998},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.89700000000000002,"15min":0.88500000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.56799999999999995},"max":{"1min":1.2390000000000001,"5min":1.28,"15min":1.3100000000000001},"last":0.97199999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542421,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":231864,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":237428736,"data_stored":232443526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77330888},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.68899999999999995,"15min":0.54500000000000004},"min":{"1min":0.28899999999999998,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.68799999999999994,"5min":15.343999999999999,"15min":15.343999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.73799999999999999,"15min":0.60299999999999998},"min":{"1min":0.39600000000000002,"5min":0.313,"15min":0.29899999999999999},"max":{"1min":1.091,"5min":14.68,"15min":14.68},"last":0.433}]},{"osd":2,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.67900000000000005,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.308,"15min":0.29599999999999999},"max":{"1min":0.79900000000000004,"5min":8.1359999999999992,"15min":8.1359999999999992},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.27300000000000002},"max":{"1min":1.3560000000000001,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.40100000000000002}]},{"osd":3,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.64400000000000002,"15min":0.59299999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.044,"5min":8.0920000000000005,"15min":8.0920000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.32000000000000001},"max":{"1min":0.73699999999999999,"5min":5.9199999999999999,"15min":5.9199999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":1.131,"15min":0.93999999999999995},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.41799999999999998},"max":{"1min":1.1579999999999999,"5min":21.27,"15min":21.27},"last":0.755},{"interface":"front","average":{"1min":0.875,"5min":0.94899999999999995,"15min":0.90600000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.53600000000000003},"max":{"1min":1.145,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.84199999999999997}]},{"osd":5,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83499999999999996,"15min":0.82599999999999996},"min":{"1min":0.63100000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1990000000000001,"5min":1.355,"15min":1.355},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.85899999999999999,"15min":0.85599999999999998},"min":{"1min":0.63700000000000001,"5min":0.499,"15min":0.46200000000000002},"max":{"1min":1.1299999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.81399999999999995}]},{"osd":6,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.84599999999999997,"15min":0.84099999999999997},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.51300000000000001},"max":{"1min":1.167,"5min":1.29,"15min":1.4450000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.88100000000000001,"15min":0.88400000000000001},"min":{"1min":0.71199999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2350000000000001,"5min":1.323,"15min":1.468},"last":0.97899999999999998}]},{"osd":7,"last update":"Mon Oct 30 12:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.85499999999999998},"min":{"1min":0.68300000000000005,"5min":0.629,"15min":0.496},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.4299999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.87},"min":{"1min":0.623,"5min":0.623,"15min":0.443},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.93500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":75235328,"data_stored":75218306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":97566720,"data_stored":97541268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":42975232,"data_stored":42953409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":85385216,"data_stored":85363568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":88555520,"data_stored":88540092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":89763840,"data_stored":89741073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":111661056,"data_stored":111638700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":130834432,"data_stored":130815658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272936960,"data_stored":265027314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139231232,"data_stored":134593460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":198570075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":274759680,"data_stored":266744507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":237875200,"data_stored":230026345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":298586112,"data_stored":290803474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":195035136,"data_stored":190043127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":280907776,"data_stored":273177138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:57:43.792 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-30T12:57:43.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.958+0000 7f1c6689a640 1 Processor -- start 2023-10-30T12:57:43.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.962+0000 7f1c6689a640 1 -- start start 2023-10-30T12:57:43.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.962+0000 7f1c6689a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1c60159930 0x7f1c600cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.962+0000 7f1c6689a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1c600cf6e0 0x7f1c6015d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.962+0000 7f1c6689a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6015a920 con 0x7f1c60153980 2023-10-30T12:57:43.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.962+0000 7f1c6689a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c600cb930 con 0x7f1c60159930 2023-10-30T12:57:43.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.962+0000 7f1c6689a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c60159dc0 con 0x7f1c600cf6e0 2023-10-30T12:57:43.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.962+0000 7f1c5ffff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f1c60153980 0x7f1c600caca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42456/0 (socket says 172.21.15.12:42456) 2023-10-30T12:57:43.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.962+0000 7f1c5ffff640 1 -- 172.21.15.12:0/35784857 learned_addr learned my addr 172.21.15.12:0/35784857 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:57:43.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.962+0000 7f1c5f7fe640 1 --2- 172.21.15.12:0/35784857 >> v2:172.21.15.12:3300/0 conn(0x7f1c60159930 0x7f1c600cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.962+0000 7f1c64e37640 1 --2- 172.21.15.12:0/35784857 >> v2:172.21.15.154:3300/0 conn(0x7f1c600cf6e0 0x7f1c6015d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.962+0000 7f1c5f7fe640 1 -- 172.21.15.12:0/35784857 >> v2:172.21.15.154:3300/0 conn(0x7f1c600cf6e0 msgr2=0x7f1c6015d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:43.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.962+0000 7f1c5f7fe640 1 --2- 172.21.15.12:0/35784857 >> v2:172.21.15.154:3300/0 conn(0x7f1c600cf6e0 0x7f1c6015d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.962+0000 7f1c5f7fe640 1 -- 172.21.15.12:0/35784857 >> v1:172.21.15.12:6789/0 conn(0x7f1c60153980 legacy=0x7f1c600caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.962+0000 7f1c5f7fe640 1 -- 172.21.15.12:0/35784857 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6015a080 con 0x7f1c60159930 2023-10-30T12:57:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.962+0000 7f1c5effd640 1 -- 172.21.15.12:0/35784857 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148765451 0 0) 0x7f1c6015a920 con 0x7f1c60153980 2023-10-30T12:57:43.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.966+0000 7f1c5f7fe640 1 --2- 172.21.15.12:0/35784857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c60159930 0x7f1c600cf1a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1c600b1cf0 tx=0x7f1c5401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198987f7c2559189 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.966+0000 7f1c5effd640 1 -- 172.21.15.12:0/35784857 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540ab600 con 0x7f1c60159930 2023-10-30T12:57:43.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.966+0000 7f1c5effd640 1 -- 172.21.15.12:0/35784857 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c540a7050 con 0x7f1c60159930 2023-10-30T12:57:43.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.966+0000 7f1c5effd640 1 -- 172.21.15.12:0/35784857 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540ab980 con 0x7f1c60159930 2023-10-30T12:57:43.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.966+0000 7f1c6689a640 1 -- 172.21.15.12:0/35784857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c60159930 msgr2=0x7f1c600cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.966+0000 7f1c6689a640 1 --2- 172.21.15.12:0/35784857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c60159930 0x7f1c600cf1a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1c600b1cf0 tx=0x7f1c5401c430 comp rx=0 tx=0).stop 2023-10-30T12:57:43.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.966+0000 7f1c6689a640 1 -- 172.21.15.12:0/35784857 shutdown_connections 2023-10-30T12:57:43.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.966+0000 7f1c6689a640 1 --2- 172.21.15.12:0/35784857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c60159930 0x7f1c600cf1a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.966+0000 7f1c6689a640 1 --2- 172.21.15.12:0/35784857 >> v2:172.21.15.154:3300/0 conn(0x7f1c600cf6e0 0x7f1c6015d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.966+0000 7f1c6689a640 1 -- 172.21.15.12:0/35784857 >> 172.21.15.12:0/35784857 conn(0x7f1c6006c0f0 msgr2=0x7f1c6006c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:43.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.966+0000 7f1c6689a640 1 -- 172.21.15.12:0/35784857 shutdown_connections 2023-10-30T12:57:43.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.966+0000 7f1c6689a640 1 -- 172.21.15.12:0/35784857 wait complete. 2023-10-30T12:57:43.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.966+0000 7f1c6689a640 1 Processor -- start 2023-10-30T12:57:43.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.966+0000 7f1c6689a640 1 -- start start 2023-10-30T12:57:43.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c6689a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c600cf6e0 0x7f1c6013e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c6689a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c60153980 0x7f1c601457f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c6689a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c601483d0 0x7f1c601f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c6689a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60146030 con 0x7f1c600cf6e0 2023-10-30T12:57:43.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c6689a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60146310 con 0x7f1c601483d0 2023-10-30T12:57:43.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c6689a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60152440 con 0x7f1c60153980 2023-10-30T12:57:43.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c5ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c600cf6e0 0x7f1c6013e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c5ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c600cf6e0 0x7f1c6013e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51098/0 (socket says 172.21.15.12:51098) 2023-10-30T12:57:43.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c5ffff640 1 -- 172.21.15.12:0/1028871106 learned_addr learned my addr 172.21.15.12:0/1028871106 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:57:43.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c64e37640 1 --2- 172.21.15.12:0/1028871106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c601483d0 0x7f1c601f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c5f7fe640 1 --2- 172.21.15.12:0/1028871106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c60153980 0x7f1c601457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c5ffff640 1 -- 172.21.15.12:0/1028871106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c60153980 msgr2=0x7f1c601457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c5ffff640 1 --2- 172.21.15.12:0/1028871106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c60153980 0x7f1c601457f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c5ffff640 1 -- 172.21.15.12:0/1028871106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c601483d0 msgr2=0x7f1c601f94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c5ffff640 1 --2- 172.21.15.12:0/1028871106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c601483d0 0x7f1c601f94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c5ffff640 1 -- 172.21.15.12:0/1028871106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c54009420 con 0x7f1c600cf6e0 2023-10-30T12:57:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c5f7fe640 1 --2- 172.21.15.12:0/1028871106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c60153980 0x7f1c601457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c5ffff640 1 --2- 172.21.15.12:0/1028871106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c600cf6e0 0x7f1c6013e6b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1c500011f0 tx=0x7f1c5000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c5cff9640 1 -- 172.21.15.12:0/1028871106 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c500074e0 con 0x7f1c600cf6e0 2023-10-30T12:57:43.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c5cff9640 1 -- 172.21.15.12:0/1028871106 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c50007640 con 0x7f1c600cf6e0 2023-10-30T12:57:43.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c5cff9640 1 -- 172.21.15.12:0/1028871106 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50004a10 con 0x7f1c600cf6e0 2023-10-30T12:57:43.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c6689a640 1 -- 172.21.15.12:0/1028871106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c60152720 con 0x7f1c600cf6e0 2023-10-30T12:57:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c6689a640 1 -- 172.21.15.12:0/1028871106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c60145d30 con 0x7f1c600cf6e0 2023-10-30T12:57:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c6689a640 1 -- 172.21.15.12:0/1028871106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c600b51d0 con 0x7f1c600cf6e0 2023-10-30T12:57:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.970+0000 7f1c5cff9640 1 -- 172.21.15.12:0/1028871106 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1c50004b70 con 0x7f1c600cf6e0 2023-10-30T12:57:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.974+0000 7f1c5cff9640 1 --2- 172.21.15.12:0/1028871106 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1c4c0664a0 0x7f1c4c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.974+0000 7f1c5cff9640 1 -- 172.21.15.12:0/1028871106 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f1c500835d0 con 0x7f1c600cf6e0 2023-10-30T12:57:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.974+0000 7f1c5f7fe640 1 --2- 172.21.15.12:0/1028871106 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1c4c0664a0 0x7f1c4c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.974+0000 7f1c5f7fe640 1 --2- 172.21.15.12:0/1028871106 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1c4c0664a0 0x7f1c4c068960 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1c600cfd90 tx=0x7f1c540a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:43.978+0000 7f1c5cff9640 1 -- 172.21.15.12:0/1028871106 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1c50050600 con 0x7f1c600cf6e0 2023-10-30T12:57:44.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.138+0000 7f1c6689a640 1 -- 172.21.15.12:0/1028871106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f1c6004e170 con 0x7f1c600cf6e0 2023-10-30T12:57:44.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.142+0000 7f1c5cff9640 1 -- 172.21.15.12:0/1028871106 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f1c50018020 con 0x7f1c600cf6e0 2023-10-30T12:57:44.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.142+0000 7f1c5cff9640 1 -- 172.21.15.12:0/1028871106 <== mon.0 v2:172.21.15.12:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1c50050600 con 0x7f1c600cf6e0 2023-10-30T12:57:44.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.146+0000 7f1c6689a640 1 -- 172.21.15.12:0/1028871106 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1c4c0664a0 msgr2=0x7f1c4c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.146+0000 7f1c6689a640 1 --2- 172.21.15.12:0/1028871106 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1c4c0664a0 0x7f1c4c068960 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1c600cfd90 tx=0x7f1c540a6000 comp rx=0 tx=0).stop 2023-10-30T12:57:44.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.146+0000 7f1c6689a640 1 -- 172.21.15.12:0/1028871106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c600cf6e0 msgr2=0x7f1c6013e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.146+0000 7f1c6689a640 1 --2- 172.21.15.12:0/1028871106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c600cf6e0 0x7f1c6013e6b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1c500011f0 tx=0x7f1c5000c9b0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.146+0000 7f1c6689a640 1 -- 172.21.15.12:0/1028871106 shutdown_connections 2023-10-30T12:57:44.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.146+0000 7f1c6689a640 1 --2- 172.21.15.12:0/1028871106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c601483d0 0x7f1c601f94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.146+0000 7f1c6689a640 1 --2- 172.21.15.12:0/1028871106 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f1c4c0664a0 0x7f1c4c068960 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.146+0000 7f1c6689a640 1 --2- 172.21.15.12:0/1028871106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c60153980 0x7f1c601457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.146+0000 7f1c6689a640 1 --2- 172.21.15.12:0/1028871106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c600cf6e0 0x7f1c6013e6b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.146+0000 7f1c6689a640 1 -- 172.21.15.12:0/1028871106 >> 172.21.15.12:0/1028871106 conn(0x7f1c6006c0f0 msgr2=0x7f1c601577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:44.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.146+0000 7f1c6689a640 1 -- 172.21.15.12:0/1028871106 shutdown_connections 2023-10-30T12:57:44.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.146+0000 7f1c6689a640 1 -- 172.21.15.12:0/1028871106 wait complete. 2023-10-30T12:57:44.170 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-30T12:57:44.171 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:44.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46bf6c9640 1 Processor -- start 2023-10-30T12:57:44.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46bf6c9640 1 -- start start 2023-10-30T12:57:44.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46bf6c9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f46b81538a0 0x7f46b815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46bf6c9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f46b80ca0e0 0x7f46b815a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46bf6c9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b80cb800 con 0x7f46b80cf160 2023-10-30T12:57:44.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46bf6c9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b815dc30 con 0x7f46b80ca0e0 2023-10-30T12:57:44.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46bf6c9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b80cb330 con 0x7f46b81538a0 2023-10-30T12:57:44.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46bcc64640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f46b80ca0e0 0x7f46b815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46bcc64640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f46b80ca0e0 0x7f46b815a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51108/0 (socket says 172.21.15.12:51108) 2023-10-30T12:57:44.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46bcc64640 1 -- 172.21.15.12:0/2607781951 learned_addr learned my addr 172.21.15.12:0/2607781951 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:57:44.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46bd465640 1 --2- 172.21.15.12:0/2607781951 >> v2:172.21.15.154:3300/0 conn(0x7f46b81538a0 0x7f46b815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46bcc64640 1 -- 172.21.15.12:0/2607781951 >> v2:172.21.15.154:3300/0 conn(0x7f46b81538a0 msgr2=0x7f46b815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:44.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46bcc64640 1 --2- 172.21.15.12:0/2607781951 >> v2:172.21.15.154:3300/0 conn(0x7f46b81538a0 0x7f46b815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46bcc64640 1 -- 172.21.15.12:0/2607781951 >> v1:172.21.15.12:6789/0 conn(0x7f46b80cf160 legacy=0x7f46b80cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46affff640 1 -- 172.21.15.12:0/2607781951 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145594878 0 0) 0x7f46b80cb800 con 0x7f46b80cf160 2023-10-30T12:57:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.310+0000 7f46bcc64640 1 -- 172.21.15.12:0/2607781951 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b80cad70 con 0x7f46b80ca0e0 2023-10-30T12:57:44.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.314+0000 7f46bcc64640 1 --2- 172.21.15.12:0/2607781951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46b80ca0e0 0x7f46b815a090 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f46b80cfd10 tx=0x7f46a801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85eb171702c7c68f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.314+0000 7f46affff640 1 -- 172.21.15.12:0/2607781951 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a80a89f0 con 0x7f46b80ca0e0 2023-10-30T12:57:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.314+0000 7f46affff640 1 -- 172.21.15.12:0/2607781951 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f46a80a8b50 con 0x7f46b80ca0e0 2023-10-30T12:57:44.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.314+0000 7f46affff640 1 -- 172.21.15.12:0/2607781951 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a80b0440 con 0x7f46b80ca0e0 2023-10-30T12:57:44.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.314+0000 7f46bf6c9640 1 -- 172.21.15.12:0/2607781951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46b80ca0e0 msgr2=0x7f46b815a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.314+0000 7f46bf6c9640 1 --2- 172.21.15.12:0/2607781951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46b80ca0e0 0x7f46b815a090 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f46b80cfd10 tx=0x7f46a801ecf0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.314+0000 7f46bf6c9640 1 -- 172.21.15.12:0/2607781951 shutdown_connections 2023-10-30T12:57:44.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.314+0000 7f46bf6c9640 1 --2- 172.21.15.12:0/2607781951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46b80ca0e0 0x7f46b815a090 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.314+0000 7f46bf6c9640 1 --2- 172.21.15.12:0/2607781951 >> v2:172.21.15.154:3300/0 conn(0x7f46b81538a0 0x7f46b815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.314+0000 7f46bf6c9640 1 -- 172.21.15.12:0/2607781951 >> 172.21.15.12:0/2607781951 conn(0x7f46b806bfe0 msgr2=0x7f46b806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:44.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.314+0000 7f46bf6c9640 1 -- 172.21.15.12:0/2607781951 shutdown_connections 2023-10-30T12:57:44.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.314+0000 7f46bf6c9640 1 -- 172.21.15.12:0/2607781951 wait complete. 2023-10-30T12:57:44.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bf6c9640 1 Processor -- start 2023-10-30T12:57:44.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bf6c9640 1 -- start start 2023-10-30T12:57:44.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bf6c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46b80ca0e0 0x7f46b81e46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bf6c9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46b80cf160 0x7f46b80c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bf6c9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46b81538a0 0x7f46b814a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bf6c9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b81e4c30 con 0x7f46b80cf160 2023-10-30T12:57:44.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bf6c9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b81f2d50 con 0x7f46b80ca0e0 2023-10-30T12:57:44.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bf6c9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b80cee20 con 0x7f46b81538a0 2023-10-30T12:57:44.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bcc64640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46b80cf160 0x7f46b80c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bcc64640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46b80cf160 0x7f46b80c4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51120/0 (socket says 172.21.15.12:51120) 2023-10-30T12:57:44.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bcc64640 1 -- 172.21.15.12:0/3840853342 learned_addr learned my addr 172.21.15.12:0/3840853342 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:57:44.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bd465640 1 --2- 172.21.15.12:0/3840853342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46b80ca0e0 0x7f46b81e46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bdc66640 1 --2- 172.21.15.12:0/3840853342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46b81538a0 0x7f46b814a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bcc64640 1 -- 172.21.15.12:0/3840853342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46b81538a0 msgr2=0x7f46b814a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bcc64640 1 --2- 172.21.15.12:0/3840853342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46b81538a0 0x7f46b814a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bcc64640 1 -- 172.21.15.12:0/3840853342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46b80ca0e0 msgr2=0x7f46b81e46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bcc64640 1 --2- 172.21.15.12:0/3840853342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46b80ca0e0 0x7f46b81e46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bcc64640 1 -- 172.21.15.12:0/3840853342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a80094d0 con 0x7f46b80cf160 2023-10-30T12:57:44.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bcc64640 1 --2- 172.21.15.12:0/3840853342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46b80cf160 0x7f46b80c4be0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f46a801b900 tx=0x7f46a801ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46adffb640 1 -- 172.21.15.12:0/3840853342 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a80ba600 con 0x7f46b80cf160 2023-10-30T12:57:44.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46adffb640 1 -- 172.21.15.12:0/3840853342 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f46a80ba780 con 0x7f46b80cf160 2023-10-30T12:57:44.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46adffb640 1 -- 172.21.15.12:0/3840853342 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a80bab50 con 0x7f46b80cf160 2023-10-30T12:57:44.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bf6c9640 1 -- 172.21.15.12:0/3840853342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b81f2140 con 0x7f46b80cf160 2023-10-30T12:57:44.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.318+0000 7f46bf6c9640 1 -- 172.21.15.12:0/3840853342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b81f2700 con 0x7f46b80cf160 2023-10-30T12:57:44.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.322+0000 7f46bf6c9640 1 -- 172.21.15.12:0/3840853342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b80b2a50 con 0x7f46b80cf160 2023-10-30T12:57:44.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.322+0000 7f46adffb640 1 -- 172.21.15.12:0/3840853342 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f46a80bacb0 con 0x7f46b80cf160 2023-10-30T12:57:44.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.322+0000 7f46adffb640 1 --2- 172.21.15.12:0/3840853342 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f469c0664a0 0x7f469c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.322+0000 7f46adffb640 1 -- 172.21.15.12:0/3840853342 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f46a8122930 con 0x7f46b80cf160 2023-10-30T12:57:44.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.322+0000 7f46bd465640 1 --2- 172.21.15.12:0/3840853342 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f469c0664a0 0x7f469c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.322+0000 7f46bd465640 1 --2- 172.21.15.12:0/3840853342 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f469c0664a0 0x7f469c068960 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f46b400b910 tx=0x7f46b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.326+0000 7f46adffb640 1 -- 172.21.15.12:0/3840853342 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f46a80efa60 con 0x7f46b80cf160 2023-10-30T12:57:44.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.518+0000 7f46bf6c9640 1 -- 172.21.15.12:0/3840853342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46b804e170 con 0x7f46b80cf160 2023-10-30T12:57:44.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.518+0000 7f46adffb640 1 -- 172.21.15.12:0/3840853342 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+6541 (secure 0 0 0) 0x7f46a80b5090 con 0x7f46b80cf160 2023-10-30T12:57:44.524 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:57:44.524 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1182202818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1182202818},{"type":"v1","addr":"172.21.15.154:6839","nonce":1182202818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:55:34.679966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:44.526 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 107 2023-10-30T12:57:44.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.518+0000 7f46bf6c9640 1 -- 172.21.15.12:0/3840853342 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f469c0664a0 msgr2=0x7f469c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.518+0000 7f46bf6c9640 1 --2- 172.21.15.12:0/3840853342 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f469c0664a0 0x7f469c068960 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f46b400b910 tx=0x7f46b4008040 comp rx=0 tx=0).stop 2023-10-30T12:57:44.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.518+0000 7f46bf6c9640 1 -- 172.21.15.12:0/3840853342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46b80cf160 msgr2=0x7f46b80c4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.518+0000 7f46bf6c9640 1 --2- 172.21.15.12:0/3840853342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46b80cf160 0x7f46b80c4be0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f46a801b900 tx=0x7f46a801ba90 comp rx=0 tx=0).stop 2023-10-30T12:57:44.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.518+0000 7f46bf6c9640 1 -- 172.21.15.12:0/3840853342 shutdown_connections 2023-10-30T12:57:44.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.518+0000 7f46bf6c9640 1 --2- 172.21.15.12:0/3840853342 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f469c0664a0 0x7f469c068960 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.518+0000 7f46bf6c9640 1 --2- 172.21.15.12:0/3840853342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46b81538a0 0x7f46b814a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.522+0000 7f46bf6c9640 1 --2- 172.21.15.12:0/3840853342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46b80cf160 0x7f46b80c4be0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.522+0000 7f46bf6c9640 1 --2- 172.21.15.12:0/3840853342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46b80ca0e0 0x7f46b81e46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.522+0000 7f46bf6c9640 1 -- 172.21.15.12:0/3840853342 >> 172.21.15.12:0/3840853342 conn(0x7f46b806bfe0 msgr2=0x7f46b8152910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:44.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.522+0000 7f46bf6c9640 1 -- 172.21.15.12:0/3840853342 shutdown_connections 2023-10-30T12:57:44.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.522+0000 7f46bf6c9640 1 -- 172.21.15.12:0/3840853342 wait complete. 2023-10-30T12:57:44.544 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:44.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5e5fb8640 1 Processor -- start 2023-10-30T12:57:44.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5e5fb8640 1 -- start start 2023-10-30T12:57:44.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5e5fb8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5e00cf160 0x7ff5e00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5e5fb8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e00cb800 con 0x7ff5e01538a0 2023-10-30T12:57:44.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5e5fb8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e00cb330 con 0x7ff5e015d6f0 2023-10-30T12:57:44.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5e5fb8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e00cad70 con 0x7ff5e00cf160 2023-10-30T12:57:44.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5df7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff5e01538a0 0x7ff5e00ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42488/0 (socket says 172.21.15.12:42488) 2023-10-30T12:57:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5df7fe640 1 -- 172.21.15.12:0/1761267187 learned_addr learned my addr 172.21.15.12:0/1761267187 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:57:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5dffff640 1 --2- 172.21.15.12:0/1761267187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5e00cf160 0x7ff5e00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5de7fc640 1 -- 172.21.15.12:0/1761267187 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748216332 0 0) 0x7ff5e00cb800 con 0x7ff5e01538a0 2023-10-30T12:57:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5de7fc640 1 -- 172.21.15.12:0/1761267187 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5c8003530 con 0x7ff5e01538a0 2023-10-30T12:57:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5de7fc640 1 -- 172.21.15.12:0/1761267187 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3767957016 0 0) 0x7ff5c8003530 con 0x7ff5e01538a0 2023-10-30T12:57:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5de7fc640 1 -- 172.21.15.12:0/1761267187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5e00cf160 msgr2=0x7ff5e00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5de7fc640 1 --2- 172.21.15.12:0/1761267187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5e00cf160 0x7ff5e00cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5de7fc640 1 -- 172.21.15.12:0/1761267187 >> v1:172.21.15.154:6789/0 conn(0x7ff5e015d6f0 legacy=0x7ff5e015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5de7fc640 1 -- 172.21.15.12:0/1761267187 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e00cb050 con 0x7ff5e01538a0 2023-10-30T12:57:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5de7fc640 1 -- 172.21.15.12:0/1761267187 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7ff5d0001680 con 0x7ff5e01538a0 2023-10-30T12:57:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5de7fc640 1 -- 172.21.15.12:0/1761267187 >> v1:172.21.15.12:6789/0 conn(0x7ff5e01538a0 legacy=0x7ff5e00ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5de7fc640 1 --2- 172.21.15.12:0/1761267187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5c80039b0 0x7ff5c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5de7fc640 1 --2- 172.21.15.12:0/1761267187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5e00cf160 0x7ff5c80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5de7fc640 1 --2- 172.21.15.12:0/1761267187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8004d10 0x7ff5c8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5de7fc640 1 -- 172.21.15.12:0/1761267187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e00cad70 con 0x7ff5c8004d10 2023-10-30T12:57:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5de7fc640 1 -- 172.21.15.12:0/1761267187 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c800bad0 con 0x7ff5c80039b0 2023-10-30T12:57:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5de7fc640 1 -- 172.21.15.12:0/1761267187 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c800bc90 con 0x7ff5e00cf160 2023-10-30T12:57:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5dffff640 1 --2- 172.21.15.12:0/1761267187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8004d10 0x7ff5c8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5df7fe640 1 --2- 172.21.15.12:0/1761267187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5c80039b0 0x7ff5c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.686+0000 7ff5deffd640 1 --2- 172.21.15.12:0/1761267187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5e00cf160 0x7ff5c80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.690+0000 7ff5dffff640 1 -- 172.21.15.12:0/1761267187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5e00cf160 msgr2=0x7ff5c80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.690+0000 7ff5dffff640 1 --2- 172.21.15.12:0/1761267187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5e00cf160 0x7ff5c80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.690+0000 7ff5dffff640 1 -- 172.21.15.12:0/1761267187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5c80039b0 msgr2=0x7ff5c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:44.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.690+0000 7ff5dffff640 1 --2- 172.21.15.12:0/1761267187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5c80039b0 0x7ff5c8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.690+0000 7ff5dffff640 1 -- 172.21.15.12:0/1761267187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c8003530 con 0x7ff5c8004d10 2023-10-30T12:57:44.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.690+0000 7ff5deffd640 1 --2- 172.21.15.12:0/1761267187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5e00cf160 0x7ff5c80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.690+0000 7ff5dffff640 1 --2- 172.21.15.12:0/1761267187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8004d10 0x7ff5c8005190 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009f90 tx=0x7ff5d801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44ceaa73a52f52d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.690+0000 7ff5de7fc640 1 -- 172.21.15.12:0/1761267187 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d80a5c30 con 0x7ff5c8004d10 2023-10-30T12:57:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.690+0000 7ff5de7fc640 1 -- 172.21.15.12:0/1761267187 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff5d800f040 con 0x7ff5c8004d10 2023-10-30T12:57:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.690+0000 7ff5de7fc640 1 -- 172.21.15.12:0/1761267187 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d80ad440 con 0x7ff5c8004d10 2023-10-30T12:57:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.690+0000 7ff5e5fb8640 1 -- 172.21.15.12:0/1761267187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8004d10 msgr2=0x7ff5c8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.690+0000 7ff5e5fb8640 1 --2- 172.21.15.12:0/1761267187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8004d10 0x7ff5c8005190 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009f90 tx=0x7ff5d801cc20 comp rx=0 tx=0).stop 2023-10-30T12:57:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.690+0000 7ff5dffff640 1 -- 172.21.15.12:0/1761267187 reap_dead start 2023-10-30T12:57:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.690+0000 7ff5e5fb8640 1 -- 172.21.15.12:0/1761267187 shutdown_connections 2023-10-30T12:57:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.690+0000 7ff5e5fb8640 1 -- 172.21.15.12:0/1761267187 >> 172.21.15.12:0/1761267187 conn(0x7ff5e006bfe0 msgr2=0x7ff5e006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5e5fb8640 1 -- 172.21.15.12:0/1761267187 shutdown_connections 2023-10-30T12:57:44.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5e5fb8640 1 -- 172.21.15.12:0/1761267187 wait complete. 2023-10-30T12:57:44.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5e5fb8640 1 Processor -- start 2023-10-30T12:57:44.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5e5fb8640 1 -- start start 2023-10-30T12:57:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5e5fb8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8004d10 0x7ff5e01578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5e5fb8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5e0159430 0x7ff5e015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5e5fb8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5e00cd250 0x7ff5e00cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5e5fb8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e00cdc40 con 0x7ff5c8004d10 2023-10-30T12:57:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5e5fb8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e01581d0 con 0x7ff5e00cd250 2023-10-30T12:57:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5e5fb8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e01584b0 con 0x7ff5e0159430 2023-10-30T12:57:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5df7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8004d10 0x7ff5e01578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5df7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8004d10 0x7ff5e01578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51134/0 (socket says 172.21.15.12:51134) 2023-10-30T12:57:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5df7fe640 1 -- 172.21.15.12:0/1559503333 learned_addr learned my addr 172.21.15.12:0/1559503333 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:57:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5deffd640 1 --2- 172.21.15.12:0/1559503333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5e0159430 0x7ff5e015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5dffff640 1 --2- 172.21.15.12:0/1559503333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5e00cd250 0x7ff5e00cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5df7fe640 1 -- 172.21.15.12:0/1559503333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5e0159430 msgr2=0x7ff5e015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5df7fe640 1 --2- 172.21.15.12:0/1559503333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5e0159430 0x7ff5e015b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5df7fe640 1 -- 172.21.15.12:0/1559503333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5e00cd250 msgr2=0x7ff5e00cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5df7fe640 1 --2- 172.21.15.12:0/1559503333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5e00cd250 0x7ff5e00cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5df7fe640 1 -- 172.21.15.12:0/1559503333 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e0158790 con 0x7ff5c8004d10 2023-10-30T12:57:44.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.694+0000 7ff5df7fe640 1 --2- 172.21.15.12:0/1559503333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8004d10 0x7ff5e01578a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0007290 tx=0x7ff5d0007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.698+0000 7ff5c7fff640 1 -- 172.21.15.12:0/1559503333 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000f900 con 0x7ff5c8004d10 2023-10-30T12:57:44.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.698+0000 7ff5c7fff640 1 -- 172.21.15.12:0/1559503333 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff5d000fa60 con 0x7ff5c8004d10 2023-10-30T12:57:44.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.698+0000 7ff5c7fff640 1 -- 172.21.15.12:0/1559503333 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000fe30 con 0x7ff5c8004d10 2023-10-30T12:57:44.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.698+0000 7ff5e5fb8640 1 -- 172.21.15.12:0/1559503333 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e0158a70 con 0x7ff5c8004d10 2023-10-30T12:57:44.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.698+0000 7ff5e5fb8640 1 -- 172.21.15.12:0/1559503333 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e00cc990 con 0x7ff5c8004d10 2023-10-30T12:57:44.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.698+0000 7ff5e5fb8640 1 -- 172.21.15.12:0/1559503333 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e0159870 con 0x7ff5c8004d10 2023-10-30T12:57:44.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.698+0000 7ff5c7fff640 1 -- 172.21.15.12:0/1559503333 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff5d001d460 con 0x7ff5c8004d10 2023-10-30T12:57:44.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.698+0000 7ff5c7fff640 1 --2- 172.21.15.12:0/1559503333 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5cc0664a0 0x7ff5cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.698+0000 7ff5c7fff640 1 -- 172.21.15.12:0/1559503333 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7ff5d00846e0 con 0x7ff5c8004d10 2023-10-30T12:57:44.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.698+0000 7ff5deffd640 1 --2- 172.21.15.12:0/1559503333 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5cc0664a0 0x7ff5cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.702+0000 7ff5deffd640 1 --2- 172.21.15.12:0/1559503333 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5cc0664a0 0x7ff5cc068960 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff5e015d070 tx=0x7ff5d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.702+0000 7ff5c7fff640 1 -- 172.21.15.12:0/1559503333 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff5e0159870 con 0x7ff5c8004d10 2023-10-30T12:57:44.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.894+0000 7ff5e5fb8640 1 -- 172.21.15.12:0/1559503333 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5e0159870 con 0x7ff5c8004d10 2023-10-30T12:57:44.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.894+0000 7ff5c7fff640 1 -- 172.21.15.12:0/1559503333 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+6541 (secure 0 0 0) 0x7ff5e0159870 con 0x7ff5c8004d10 2023-10-30T12:57:44.899 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:57:44.900 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/665371837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":665371837},{"type":"v1","addr":"172.21.15.154:6837","nonce":665371837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6838/1718052620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1718052620},{"type":"v1","addr":"172.21.15.154:6838","nonce":1718052620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/1182202818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1182202818},{"type":"v1","addr":"172.21.15.154:6839","nonce":1182202818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:35:05.553867+0000","modified":"2023-10-30T12:55:34.679966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6021,"mds_1":6192,"mds_2":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":2,"incarnation":81,"state":"up:active","state_seq":22,"addr":"172.21.15.12:6835/2306258157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2306258157},{"type":"v1","addr":"172.21.15.12:6835","nonce":2306258157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6021":{"gid":6021,"name":"e","rank":0,"incarnation":89,"state":"up:active","state_seq":42,"addr":"172.21.15.12:6836/3877967042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3877967042},{"type":"v1","addr":"172.21.15.12:6836","nonce":3877967042}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6192":{"gid":6192,"name":"c","rank":1,"incarnation":99,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6837/2954662612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2954662612},{"type":"v1","addr":"172.21.15.12:6837","nonce":2954662612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:44.901 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 107 2023-10-30T12:57:44.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.894+0000 7ff5e5fb8640 1 -- 172.21.15.12:0/1559503333 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5cc0664a0 msgr2=0x7ff5cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.894+0000 7ff5e5fb8640 1 --2- 172.21.15.12:0/1559503333 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5cc0664a0 0x7ff5cc068960 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff5e015d070 tx=0x7ff5d4009040 comp rx=0 tx=0).stop 2023-10-30T12:57:44.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.894+0000 7ff5e5fb8640 1 -- 172.21.15.12:0/1559503333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8004d10 msgr2=0x7ff5e01578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.894+0000 7ff5e5fb8640 1 --2- 172.21.15.12:0/1559503333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8004d10 0x7ff5e01578a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0007290 tx=0x7ff5d0007750 comp rx=0 tx=0).stop 2023-10-30T12:57:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.894+0000 7ff5e5fb8640 1 -- 172.21.15.12:0/1559503333 shutdown_connections 2023-10-30T12:57:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.894+0000 7ff5e5fb8640 1 --2- 172.21.15.12:0/1559503333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5e00cd250 0x7ff5e00cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.894+0000 7ff5e5fb8640 1 --2- 172.21.15.12:0/1559503333 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7ff5cc0664a0 0x7ff5cc068960 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.894+0000 7ff5e5fb8640 1 --2- 172.21.15.12:0/1559503333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5e0159430 0x7ff5e015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.894+0000 7ff5e5fb8640 1 --2- 172.21.15.12:0/1559503333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8004d10 0x7ff5e01578a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.894+0000 7ff5e5fb8640 1 -- 172.21.15.12:0/1559503333 >> 172.21.15.12:0/1559503333 conn(0x7ff5e006bfe0 msgr2=0x7ff5e00b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:44.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.894+0000 7ff5e5fb8640 1 -- 172.21.15.12:0/1559503333 shutdown_connections 2023-10-30T12:57:44.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:44.894+0000 7ff5e5fb8640 1 -- 172.21.15.12:0/1559503333 wait complete. 2023-10-30T12:57:44.920 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:57:45.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.058+0000 7fe6b0046640 1 Processor -- start 2023-10-30T12:57:45.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.058+0000 7fe6b0046640 1 -- start start 2023-10-30T12:57:45.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.058+0000 7fe6b0046640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe6a80c2570 0x7fe6a80cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.058+0000 7fe6b0046640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a80c68b0 con 0x7fe6a8150920 2023-10-30T12:57:45.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.058+0000 7fe6b0046640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a8150d00 con 0x7fe6a80c29d0 2023-10-30T12:57:45.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.058+0000 7fe6b0046640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a80c2e90 con 0x7fe6a80c2570 2023-10-30T12:57:45.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.058+0000 7fe6ad5e1640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe6a80c2570 0x7fe6a80cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.058+0000 7fe6adde2640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe6a8150920 0x7fe6a806d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42492/0 (socket says 172.21.15.12:42492) 2023-10-30T12:57:45.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.058+0000 7fe6adde2640 1 -- 172.21.15.12:0/3191507514 learned_addr learned my addr 172.21.15.12:0/3191507514 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:57:45.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.058+0000 7fe6ad5e1640 1 -- 172.21.15.12:0/3191507514 >> v1:172.21.15.154:6789/0 conn(0x7fe6a80c29d0 legacy=0x7fe6a8156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.058+0000 7fe6ad5e1640 1 -- 172.21.15.12:0/3191507514 >> v1:172.21.15.12:6789/0 conn(0x7fe6a8150920 legacy=0x7fe6a806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.058+0000 7fe6ad5e1640 1 -- 172.21.15.12:0/3191507514 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a80c6b00 con 0x7fe6a80c2570 2023-10-30T12:57:45.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.062+0000 7fe6ad5e1640 1 --2- 172.21.15.12:0/3191507514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a80c2570 0x7fe6a80cb0d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe6980097c0 tx=0x7fe69801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6af04703de2a60 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.062+0000 7fe6acde0640 1 -- 172.21.15.12:0/3191507514 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6980ab600 con 0x7fe6a80c2570 2023-10-30T12:57:45.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.062+0000 7fe6acde0640 1 -- 172.21.15.12:0/3191507514 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe6980a7050 con 0x7fe6a80c2570 2023-10-30T12:57:45.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.062+0000 7fe6acde0640 1 -- 172.21.15.12:0/3191507514 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6980ab9a0 con 0x7fe6a80c2570 2023-10-30T12:57:45.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.062+0000 7fe6b0046640 1 -- 172.21.15.12:0/3191507514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a80c2570 msgr2=0x7fe6a80cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.062+0000 7fe6b0046640 1 --2- 172.21.15.12:0/3191507514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a80c2570 0x7fe6a80cb0d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe6980097c0 tx=0x7fe69801c480 comp rx=0 tx=0).stop 2023-10-30T12:57:45.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.062+0000 7fe6b0046640 1 -- 172.21.15.12:0/3191507514 shutdown_connections 2023-10-30T12:57:45.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.062+0000 7fe6b0046640 1 --2- 172.21.15.12:0/3191507514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a80c2570 0x7fe6a80cb0d0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.062+0000 7fe6b0046640 1 -- 172.21.15.12:0/3191507514 >> 172.21.15.12:0/3191507514 conn(0x7fe6a806dfd0 msgr2=0x7fe6a806e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:45.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.062+0000 7fe6b0046640 1 -- 172.21.15.12:0/3191507514 shutdown_connections 2023-10-30T12:57:45.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.062+0000 7fe6b0046640 1 -- 172.21.15.12:0/3191507514 wait complete. 2023-10-30T12:57:45.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6b0046640 1 Processor -- start 2023-10-30T12:57:45.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6b0046640 1 -- start start 2023-10-30T12:57:45.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6b0046640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe6a80c2570 0x7fe6a80736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6b0046640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a80c29d0 0x7fe6a8159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6b0046640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6a8150920 0x7fe6a81595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6b0046640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a814ebb0 con 0x7fe6a80c29d0 2023-10-30T12:57:45.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6b0046640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a8141aa0 con 0x7fe6a8150920 2023-10-30T12:57:45.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6b0046640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a814fae0 con 0x7fe6a80c2570 2023-10-30T12:57:45.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6ad5e1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a80c29d0 0x7fe6a8159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6ad5e1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a80c29d0 0x7fe6a8159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51146/0 (socket says 172.21.15.12:51146) 2023-10-30T12:57:45.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6ad5e1640 1 -- 172.21.15.12:0/1916905919 learned_addr learned my addr 172.21.15.12:0/1916905919 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:57:45.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6ae5e3640 1 --2- 172.21.15.12:0/1916905919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6a8150920 0x7fe6a81595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6adde2640 1 --2- 172.21.15.12:0/1916905919 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe6a80c2570 0x7fe6a80736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6ad5e1640 1 -- 172.21.15.12:0/1916905919 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe6a80c2570 msgr2=0x7fe6a80736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6ad5e1640 1 --2- 172.21.15.12:0/1916905919 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe6a80c2570 0x7fe6a80736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6ad5e1640 1 -- 172.21.15.12:0/1916905919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6a8150920 msgr2=0x7fe6a81595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6ad5e1640 1 --2- 172.21.15.12:0/1916905919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6a8150920 0x7fe6a81595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6ad5e1640 1 -- 172.21.15.12:0/1916905919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe698009420 con 0x7fe6a80c29d0 2023-10-30T12:57:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6ad5e1640 1 --2- 172.21.15.12:0/1916905919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a80c29d0 0x7fe6a8159070 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe6980ae040 tx=0x7fe698019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe69e7fc640 1 -- 172.21.15.12:0/1916905919 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6980b4840 con 0x7fe6a80c29d0 2023-10-30T12:57:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe69e7fc640 1 -- 172.21.15.12:0/1916905919 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe6980b49c0 con 0x7fe6a80c29d0 2023-10-30T12:57:45.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe69e7fc640 1 -- 172.21.15.12:0/1916905919 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6980b4d90 con 0x7fe6a80c29d0 2023-10-30T12:57:45.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6b0046640 1 -- 172.21.15.12:0/1916905919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a814fdc0 con 0x7fe6a80c29d0 2023-10-30T12:57:45.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.066+0000 7fe6b0046640 1 -- 172.21.15.12:0/1916905919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a8141280 con 0x7fe6a80c29d0 2023-10-30T12:57:45.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.070+0000 7fe6b0046640 1 -- 172.21.15.12:0/1916905919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a804e170 con 0x7fe6a80c29d0 2023-10-30T12:57:45.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.070+0000 7fe69e7fc640 1 -- 172.21.15.12:0/1916905919 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe6980b2040 con 0x7fe6a80c29d0 2023-10-30T12:57:45.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.070+0000 7fe69e7fc640 1 --2- 172.21.15.12:0/1916905919 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe68c0664f0 0x7fe68c0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.070+0000 7fe69e7fc640 1 -- 172.21.15.12:0/1916905919 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fe69811e6a0 con 0x7fe6a80c29d0 2023-10-30T12:57:45.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.070+0000 7fe6adde2640 1 --2- 172.21.15.12:0/1916905919 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe68c0664f0 0x7fe68c0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.070+0000 7fe6adde2640 1 --2- 172.21.15.12:0/1916905919 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe68c0664f0 0x7fe68c0689b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4005890 tx=0x7fe6a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.074+0000 7fe69e7fc640 1 -- 172.21.15.12:0/1916905919 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe6980eb710 con 0x7fe6a80c29d0 2023-10-30T12:57:45.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.234+0000 7fe6b0046640 1 -- 172.21.15.12:0/1916905919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6a806f010 con 0x7fe6a80c29d0 2023-10-30T12:57:45.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.234+0000 7fe69e7fc640 1 -- 172.21.15.12:0/1916905919 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+18079 (secure 0 0 0) 0x7fe6980b0090 con 0x7fe6a80c29d0 2023-10-30T12:57:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-30T12:57:45.242 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":55,"fsid":"97b69bd8-a5ea-4668-9ffd-462651b43fe5","created":"2023-10-30T12:34:40.373137+0000","modified":"2023-10-30T12:54:25.460451+0000","last_up_change":"2023-10-30T12:34:51.438770+0000","last_in_change":"2023-10-30T12:34:42.656278+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:34:53.017612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:35:03.216053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:35:04.597162+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":46,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce150b4b-3d29-49ec-9e06-bf801656e271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6809","nonce":2303962477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6811","nonce":2303962477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6815","nonce":2303962477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2303962477},{"type":"v1","addr":"172.21.15.12:6813","nonce":2303962477}]},"public_addr":"172.21.15.12:6809/2303962477","cluster_addr":"172.21.15.12:6811/2303962477","heartbeat_back_addr":"172.21.15.12:6815/2303962477","heartbeat_front_addr":"172.21.15.12:6813/2303962477","state":["exists","up"]},{"osd":1,"uuid":"b1ca990f-7ae6-42e1-9a39-94e0fd820e40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6817","nonce":2335476277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6819","nonce":2335476277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6823","nonce":2335476277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2335476277},{"type":"v1","addr":"172.21.15.12:6821","nonce":2335476277}]},"public_addr":"172.21.15.12:6817/2335476277","cluster_addr":"172.21.15.12:6819/2335476277","heartbeat_back_addr":"172.21.15.12:6823/2335476277","heartbeat_front_addr":"172.21.15.12:6821/2335476277","state":["exists","up"]},{"osd":2,"uuid":"2f0d49df-1f4f-47a5-9dc4-a253753a371f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6801","nonce":3034215139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6803","nonce":3034215139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6807","nonce":3034215139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3034215139},{"type":"v1","addr":"172.21.15.12:6805","nonce":3034215139}]},"public_addr":"172.21.15.12:6801/3034215139","cluster_addr":"172.21.15.12:6803/3034215139","heartbeat_back_addr":"172.21.15.12:6807/3034215139","heartbeat_front_addr":"172.21.15.12:6805/3034215139","state":["exists","up"]},{"osd":3,"uuid":"739f6118-6ac3-4a60-9da5-1a0ee98f9518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6825","nonce":3921928345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6827","nonce":3921928345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6831","nonce":3921928345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3921928345},{"type":"v1","addr":"172.21.15.12:6829","nonce":3921928345}]},"public_addr":"172.21.15.12:6825/3921928345","cluster_addr":"172.21.15.12:6827/3921928345","heartbeat_back_addr":"172.21.15.12:6831/3921928345","heartbeat_front_addr":"172.21.15.12:6829/3921928345","state":["exists","up"]},{"osd":4,"uuid":"11ac0f0b-1ebc-41f9-afa0-90cbf86b73db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6809","nonce":3824067880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6811","nonce":3824067880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6816","nonce":3824067880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3824067880},{"type":"v1","addr":"172.21.15.154:6813","nonce":3824067880}]},"public_addr":"172.21.15.154:6809/3824067880","cluster_addr":"172.21.15.154:6811/3824067880","heartbeat_back_addr":"172.21.15.154:6816/3824067880","heartbeat_front_addr":"172.21.15.154:6813/3824067880","state":["exists","up"]},{"osd":5,"uuid":"310a87c6-f25b-4349-a5f3-8130e2a53210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6801","nonce":2056450204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6803","nonce":2056450204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6807","nonce":2056450204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2056450204},{"type":"v1","addr":"172.21.15.154:6805","nonce":2056450204}]},"public_addr":"172.21.15.154:6801/2056450204","cluster_addr":"172.21.15.154:6803/2056450204","heartbeat_back_addr":"172.21.15.154:6807/2056450204","heartbeat_front_addr":"172.21.15.154:6805/2056450204","state":["exists","up"]},{"osd":6,"uuid":"ff1a4b95-1e38-47f5-bcd7-fb53346c63dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6819","nonce":1102176766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6823","nonce":1102176766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6829","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6831","nonce":1102176766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6825","nonce":1102176766},{"type":"v1","addr":"172.21.15.154:6827","nonce":1102176766}]},"public_addr":"172.21.15.154:6819/1102176766","cluster_addr":"172.21.15.154:6823/1102176766","heartbeat_back_addr":"172.21.15.154:6831/1102176766","heartbeat_front_addr":"172.21.15.154:6827/1102176766","state":["exists","up"]},{"osd":7,"uuid":"e6cbc67e-29b3-48a6-ba42-310011cafb14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6818","nonce":219299884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6822","nonce":219299884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6830","nonce":219299884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":219299884},{"type":"v1","addr":"172.21.15.154:6826","nonce":219299884}]},"public_addr":"172.21.15.154:6818/219299884","cluster_addr":"172.21.15.154:6822/219299884","heartbeat_back_addr":"172.21.15.154:6830/219299884","heartbeat_front_addr":"172.21.15.154:6826/219299884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.294743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.551181+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.249961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.391979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.171648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.149607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.259413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:34:48.287229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6836/2204547612":"2023-10-31T12:54:25.460390+0000","172.21.15.12:6837/2752088514":"2023-10-31T12:53:20.432410+0000","172.21.15.12:6834/2752088514":"2023-10-31T12:53:20.432410+0000","172.21.15.154:6839/2204547612":"2023-10-31T12:54:25.460390+0000","172.21.15.12:6836/2219276371":"2023-10-31T12:50:50.379953+0000","172.21.15.12:6833/2219276371":"2023-10-31T12:50:50.379953+0000","172.21.15.12:6835/559649789":"2023-10-31T12:49:35.359754+0000","172.21.15.12:6834/3836803994":"2023-10-31T12:37:55.139871+0000","172.21.15.154:6836/1517672481":"2023-10-31T12:43:13.483926+0000","172.21.15.12:6832/2676506149":"2023-10-31T12:40:40.195783+0000","172.21.15.154:6839/1517672481":"2023-10-31T12:43:13.483926+0000","172.21.15.12:6834/3124135109":"2023-10-31T12:46:40.313866+0000","172.21.15.12:6837/3836803994":"2023-10-31T12:37:55.139871+0000","172.21.15.12:6836/291392212":"2023-10-31T12:44:35.272569+0000","172.21.15.12:6837/3124135109":"2023-10-31T12:46:40.313866+0000","172.21.15.12:6833/291392212":"2023-10-31T12:44:35.272569+0000","172.21.15.154:6836/1635719967":"2023-10-31T12:47:45.331652+0000","172.21.15.12:6835/2676506149":"2023-10-31T12:40:40.195783+0000","172.21.15.154:6839/1635719967":"2023-10-31T12:47:45.331652+0000","172.21.15.12:6832/559649789":"2023-10-31T12:49:35.359754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:57:45.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.238+0000 7fe6b0046640 1 -- 172.21.15.12:0/1916905919 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe68c0664f0 msgr2=0x7fe68c0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.238+0000 7fe6b0046640 1 --2- 172.21.15.12:0/1916905919 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe68c0664f0 0x7fe68c0689b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4005890 tx=0x7fe6a4009040 comp rx=0 tx=0).stop 2023-10-30T12:57:45.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.238+0000 7fe6b0046640 1 -- 172.21.15.12:0/1916905919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a80c29d0 msgr2=0x7fe6a8159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.238+0000 7fe6b0046640 1 --2- 172.21.15.12:0/1916905919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a80c29d0 0x7fe6a8159070 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe6980ae040 tx=0x7fe698019220 comp rx=0 tx=0).stop 2023-10-30T12:57:45.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.238+0000 7fe6b0046640 1 -- 172.21.15.12:0/1916905919 shutdown_connections 2023-10-30T12:57:45.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.238+0000 7fe6b0046640 1 --2- 172.21.15.12:0/1916905919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6a8150920 0x7fe6a81595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.238+0000 7fe6b0046640 1 --2- 172.21.15.12:0/1916905919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a80c29d0 0x7fe6a8159070 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.238+0000 7fe6b0046640 1 --2- 172.21.15.12:0/1916905919 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe68c0664f0 0x7fe68c0689b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.238+0000 7fe6b0046640 1 --2- 172.21.15.12:0/1916905919 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe6a80c2570 0x7fe6a80736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.238+0000 7fe6b0046640 1 -- 172.21.15.12:0/1916905919 >> 172.21.15.12:0/1916905919 conn(0x7fe6a806dfd0 msgr2=0x7fe6a8142f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:45.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.238+0000 7fe6b0046640 1 -- 172.21.15.12:0/1916905919 shutdown_connections 2023-10-30T12:57:45.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.238+0000 7fe6b0046640 1 -- 172.21.15.12:0/1916905919 wait complete. 2023-10-30T12:57:45.263 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-30T12:57:45.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.402+0000 7fe75f617640 1 Processor -- start 2023-10-30T12:57:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.402+0000 7fe75f617640 1 -- start start 2023-10-30T12:57:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.402+0000 7fe75f617640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe758153040 0x7fe7580b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.402+0000 7fe75f617640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe758159930 con 0x7fe7581559f0 2023-10-30T12:57:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.402+0000 7fe75f617640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe758156200 con 0x7fe758155e20 2023-10-30T12:57:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.402+0000 7fe75f617640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7580c0130 con 0x7fe758153040 2023-10-30T12:57:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.406+0000 7fe75d3b3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe758153040 0x7fe7580b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.406+0000 7fe75cbb2640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe7581559f0 0x7fe7580c6100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42496/0 (socket says 172.21.15.12:42496) 2023-10-30T12:57:45.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.406+0000 7fe75cbb2640 1 -- 172.21.15.12:0/2990156580 learned_addr learned my addr 172.21.15.12:0/2990156580 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:57:45.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.406+0000 7fe75d3b3640 1 -- 172.21.15.12:0/2990156580 >> v1:172.21.15.154:6789/0 conn(0x7fe758155e20 legacy=0x7fe758160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.406+0000 7fe75d3b3640 1 -- 172.21.15.12:0/2990156580 >> v1:172.21.15.12:6789/0 conn(0x7fe7581559f0 legacy=0x7fe7580c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.406+0000 7fe74ffff640 1 -- 172.21.15.12:0/2990156580 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984197754 0 0) 0x7fe758159930 con 0x7fe7581559f0 2023-10-30T12:57:45.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.406+0000 7fe75d3b3640 1 -- 172.21.15.12:0/2990156580 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7580c7260 con 0x7fe758153040 2023-10-30T12:57:45.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.406+0000 7fe75d3b3640 1 --2- 172.21.15.12:0/2990156580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe758153040 0x7fe7580b0e30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe7540097c0 tx=0x7fe75401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff7ee5d3ca73f92 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.406+0000 7fe74ffff640 1 -- 172.21.15.12:0/2990156580 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7540a8970 con 0x7fe758153040 2023-10-30T12:57:45.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.406+0000 7fe74ffff640 1 -- 172.21.15.12:0/2990156580 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe7540a8ad0 con 0x7fe758153040 2023-10-30T12:57:45.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.406+0000 7fe74ffff640 1 -- 172.21.15.12:0/2990156580 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7540a8e80 con 0x7fe758153040 2023-10-30T12:57:45.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75f617640 1 -- 172.21.15.12:0/2990156580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe758153040 msgr2=0x7fe7580b0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75f617640 1 --2- 172.21.15.12:0/2990156580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe758153040 0x7fe7580b0e30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe7540097c0 tx=0x7fe75401ec70 comp rx=0 tx=0).stop 2023-10-30T12:57:45.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75f617640 1 -- 172.21.15.12:0/2990156580 shutdown_connections 2023-10-30T12:57:45.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75f617640 1 --2- 172.21.15.12:0/2990156580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe758153040 0x7fe7580b0e30 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75f617640 1 -- 172.21.15.12:0/2990156580 >> 172.21.15.12:0/2990156580 conn(0x7fe75806d9a0 msgr2=0x7fe75806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75f617640 1 -- 172.21.15.12:0/2990156580 shutdown_connections 2023-10-30T12:57:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75f617640 1 -- 172.21.15.12:0/2990156580 wait complete. 2023-10-30T12:57:45.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75f617640 1 Processor -- start 2023-10-30T12:57:45.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75f617640 1 -- start start 2023-10-30T12:57:45.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75f617640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe758153040 0x7fe758158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75f617640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7581559f0 0x7fe758147140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75f617640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe758155e20 0x7fe7581ed580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75f617640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7581ece50 con 0x7fe758153040 2023-10-30T12:57:45.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75f617640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe758154680 con 0x7fe7581559f0 2023-10-30T12:57:45.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75f617640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7581ed0f0 con 0x7fe758155e20 2023-10-30T12:57:45.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75d3b3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe758153040 0x7fe758158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75d3b3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe758153040 0x7fe758158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51154/0 (socket says 172.21.15.12:51154) 2023-10-30T12:57:45.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75d3b3640 1 -- 172.21.15.12:0/46495473 learned_addr learned my addr 172.21.15.12:0/46495473 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:57:45.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75cbb2640 1 --2- 172.21.15.12:0/46495473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7581559f0 0x7fe758147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.410+0000 7fe75dbb4640 1 --2- 172.21.15.12:0/46495473 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe758155e20 0x7fe7581ed580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.414+0000 7fe75d3b3640 1 -- 172.21.15.12:0/46495473 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe758155e20 msgr2=0x7fe7581ed580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.414+0000 7fe75d3b3640 1 --2- 172.21.15.12:0/46495473 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe758155e20 0x7fe7581ed580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.414+0000 7fe75d3b3640 1 -- 172.21.15.12:0/46495473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7581559f0 msgr2=0x7fe758147140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.414+0000 7fe75d3b3640 1 --2- 172.21.15.12:0/46495473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7581559f0 0x7fe758147140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.414+0000 7fe75d3b3640 1 -- 172.21.15.12:0/46495473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe754009420 con 0x7fe758153040 2023-10-30T12:57:45.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.414+0000 7fe75d3b3640 1 --2- 172.21.15.12:0/46495473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe758153040 0x7fe758158780 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe7540b3040 tx=0x7fe75401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.414+0000 7fe74dffb640 1 -- 172.21.15.12:0/46495473 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7540b9630 con 0x7fe758153040 2023-10-30T12:57:45.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.414+0000 7fe74dffb640 1 -- 172.21.15.12:0/46495473 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe7540b97b0 con 0x7fe758153040 2023-10-30T12:57:45.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.414+0000 7fe74dffb640 1 -- 172.21.15.12:0/46495473 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7540b9b80 con 0x7fe758153040 2023-10-30T12:57:45.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.414+0000 7fe75f617640 1 -- 172.21.15.12:0/46495473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7581f07e0 con 0x7fe758153040 2023-10-30T12:57:45.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.414+0000 7fe75f617640 1 -- 172.21.15.12:0/46495473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7581f0da0 con 0x7fe758153040 2023-10-30T12:57:45.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.414+0000 7fe75f617640 1 -- 172.21.15.12:0/46495473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe758074250 con 0x7fe758153040 2023-10-30T12:57:45.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.414+0000 7fe74dffb640 1 -- 172.21.15.12:0/46495473 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe7540b7040 con 0x7fe758153040 2023-10-30T12:57:45.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.414+0000 7fe74dffb640 1 --2- 172.21.15.12:0/46495473 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe73c0664d0 0x7fe73c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.418+0000 7fe74dffb640 1 -- 172.21.15.12:0/46495473 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fe754122300 con 0x7fe758153040 2023-10-30T12:57:45.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.418+0000 7fe75cbb2640 1 --2- 172.21.15.12:0/46495473 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe73c0664d0 0x7fe73c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.418+0000 7fe75cbb2640 1 --2- 172.21.15.12:0/46495473 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe73c0664d0 0x7fe73c068990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe7580c5620 tx=0x7fe748007240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.422+0000 7fe74dffb640 1 -- 172.21.15.12:0/46495473 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe7540ef3a0 con 0x7fe758153040 2023-10-30T12:57:45.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.614+0000 7fe75f617640 1 -- 172.21.15.12:0/46495473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe75804e170 con 0x7fe758153040 2023-10-30T12:57:45.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.614+0000 7fe74dffb640 1 -- 172.21.15.12:0/46495473 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v108) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe7540b5020 con 0x7fe758153040 2023-10-30T12:57:45.620 INFO:teuthology.orchestra.run.smithi012.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-30T12:57:45.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.618+0000 7fe75f617640 1 -- 172.21.15.12:0/46495473 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe73c0664d0 msgr2=0x7fe73c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.618+0000 7fe75f617640 1 --2- 172.21.15.12:0/46495473 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe73c0664d0 0x7fe73c068990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe7580c5620 tx=0x7fe748007240 comp rx=0 tx=0).stop 2023-10-30T12:57:45.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.618+0000 7fe75f617640 1 -- 172.21.15.12:0/46495473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe758153040 msgr2=0x7fe758158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.618+0000 7fe75f617640 1 --2- 172.21.15.12:0/46495473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe758153040 0x7fe758158780 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe7540b3040 tx=0x7fe75401ba10 comp rx=0 tx=0).stop 2023-10-30T12:57:45.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.618+0000 7fe75f617640 1 -- 172.21.15.12:0/46495473 shutdown_connections 2023-10-30T12:57:45.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.618+0000 7fe75f617640 1 --2- 172.21.15.12:0/46495473 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7fe73c0664d0 0x7fe73c068990 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.618+0000 7fe75f617640 1 --2- 172.21.15.12:0/46495473 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe758155e20 0x7fe7581ed580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.618+0000 7fe75f617640 1 --2- 172.21.15.12:0/46495473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7581559f0 0x7fe758147140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.618+0000 7fe75f617640 1 --2- 172.21.15.12:0/46495473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe758153040 0x7fe758158780 secure :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe7540b3040 tx=0x7fe75401ba10 comp rx=0 tx=0).stop 2023-10-30T12:57:45.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.618+0000 7fe75f617640 1 -- 172.21.15.12:0/46495473 >> 172.21.15.12:0/46495473 conn(0x7fe75806d9a0 msgr2=0x7fe7580714d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:45.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.618+0000 7fe75f617640 1 -- 172.21.15.12:0/46495473 shutdown_connections 2023-10-30T12:57:45.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.618+0000 7fe75f617640 1 -- 172.21.15.12:0/46495473 wait complete. 2023-10-30T12:57:45.651 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-30T12:57:45.702 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2023-10-30T12:57:45.704 INFO:tasks.ceph.mds.c.smithi012.stdout:starting mds.c at 2023-10-30T12:57:45.705 INFO:tasks.ceph.mds.e.smithi012.stdout:starting mds.e at 2023-10-30T12:57:45.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.802+0000 7f6c45646640 1 Processor -- start 2023-10-30T12:57:45.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.802+0000 7f6c45646640 1 -- start start 2023-10-30T12:57:45.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c45646640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6c40153980 0x7f6c400caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c45646640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c40159930 0x7f6c400cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c45646640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6c400cf1a0 0x7f6c400cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c45646640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c4015a920 con 0x7f6c400cf1a0 2023-10-30T12:57:45.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c45646640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c400cb930 con 0x7f6c40153980 2023-10-30T12:57:45.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c45646640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c40159dc0 con 0x7f6c40159930 2023-10-30T12:57:45.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c3f7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6c400cf1a0 0x7f6c400cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c3f7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6c400cf1a0 0x7f6c400cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51200/0 (socket says 172.21.15.12:51200) 2023-10-30T12:57:45.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c3f7fe640 1 -- 172.21.15.12:0/1623975003 learned_addr learned my addr 172.21.15.12:0/1623975003 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:57:45.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c3effd640 1 --2- 172.21.15.12:0/1623975003 >> v2:172.21.15.154:3300/0 conn(0x7f6c40153980 0x7f6c400caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c37fff640 1 --2- 172.21.15.12:0/1623975003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c40159930 0x7f6c400cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c3f7fe640 1 -- 172.21.15.12:0/1623975003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c40159930 msgr2=0x7f6c400cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:45.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c3f7fe640 1 --2- 172.21.15.12:0/1623975003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c40159930 0x7f6c400cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c3f7fe640 1 -- 172.21.15.12:0/1623975003 >> v2:172.21.15.154:3300/0 conn(0x7f6c40153980 msgr2=0x7f6c400caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:45.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c3f7fe640 1 --2- 172.21.15.12:0/1623975003 >> v2:172.21.15.154:3300/0 conn(0x7f6c40153980 0x7f6c400caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.806+0000 7f6c3f7fe640 1 -- 172.21.15.12:0/1623975003 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4015a080 con 0x7f6c400cf1a0 2023-10-30T12:57:45.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c3f7fe640 1 --2- 172.21.15.12:0/1623975003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c400cf1a0 0x7f6c400cf630 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6c400cfd90 tx=0x7f6c3801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f755792d3bbb1f7e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c3e7fc640 1 -- 172.21.15.12:0/1623975003 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c380a5820 con 0x7f6c400cf1a0 2023-10-30T12:57:45.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c3e7fc640 1 -- 172.21.15.12:0/1623975003 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6c380a5980 con 0x7f6c400cf1a0 2023-10-30T12:57:45.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c3e7fc640 1 -- 172.21.15.12:0/1623975003 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c380a5d30 con 0x7f6c400cf1a0 2023-10-30T12:57:45.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c45646640 1 -- 172.21.15.12:0/1623975003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c400cf1a0 msgr2=0x7f6c400cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c45646640 1 --2- 172.21.15.12:0/1623975003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c400cf1a0 0x7f6c400cf630 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6c400cfd90 tx=0x7f6c3801c9a0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c45646640 1 -- 172.21.15.12:0/1623975003 shutdown_connections 2023-10-30T12:57:45.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c45646640 1 --2- 172.21.15.12:0/1623975003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c400cf1a0 0x7f6c400cf630 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c45646640 1 --2- 172.21.15.12:0/1623975003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c40159930 0x7f6c400cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c45646640 1 --2- 172.21.15.12:0/1623975003 >> v2:172.21.15.154:3300/0 conn(0x7f6c40153980 0x7f6c400caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c45646640 1 -- 172.21.15.12:0/1623975003 >> 172.21.15.12:0/1623975003 conn(0x7f6c4006c0f0 msgr2=0x7f6c4006c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:45.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c45646640 1 -- 172.21.15.12:0/1623975003 shutdown_connections 2023-10-30T12:57:45.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c45646640 1 -- 172.21.15.12:0/1623975003 wait complete. 2023-10-30T12:57:45.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c45646640 1 Processor -- start 2023-10-30T12:57:45.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c45646640 1 -- start start 2023-10-30T12:57:45.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.810+0000 7f6c45646640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c400cf1a0 0x7f6c40152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c45646640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c40153980 0x7f6c400cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c45646640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c400cd9d0 0x7f6c40146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c45646640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4015bba0 con 0x7f6c40153980 2023-10-30T12:57:45.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c45646640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c40149870 con 0x7f6c400cd9d0 2023-10-30T12:57:45.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c45646640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4015ae70 con 0x7f6c400cf1a0 2023-10-30T12:57:45.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c37fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c40153980 0x7f6c400cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c37fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c40153980 0x7f6c400cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51208/0 (socket says 172.21.15.12:51208) 2023-10-30T12:57:45.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c37fff640 1 -- 172.21.15.12:0/2387312663 learned_addr learned my addr 172.21.15.12:0/2387312663 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:57:45.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c3effd640 1 --2- 172.21.15.12:0/2387312663 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c400cf1a0 0x7f6c40152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c3f7fe640 1 --2- 172.21.15.12:0/2387312663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c400cd9d0 0x7f6c40146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c37fff640 1 -- 172.21.15.12:0/2387312663 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c400cf1a0 msgr2=0x7f6c40152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c37fff640 1 --2- 172.21.15.12:0/2387312663 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c400cf1a0 0x7f6c40152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c37fff640 1 -- 172.21.15.12:0/2387312663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c400cd9d0 msgr2=0x7f6c40146900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c37fff640 1 --2- 172.21.15.12:0/2387312663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c400cd9d0 0x7f6c40146900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c37fff640 1 -- 172.21.15.12:0/2387312663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c38009b30 con 0x7f6c40153980 2023-10-30T12:57:45.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c37fff640 1 --2- 172.21.15.12:0/2387312663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c40153980 0x7f6c400cd490 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800e870 tx=0x7f6c2800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c377fe640 1 -- 172.21.15.12:0/2387312663 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28009510 con 0x7f6c40153980 2023-10-30T12:57:45.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c377fe640 1 -- 172.21.15.12:0/2387312663 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6c2800eec0 con 0x7f6c40153980 2023-10-30T12:57:45.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c377fe640 1 -- 172.21.15.12:0/2387312663 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2800cb60 con 0x7f6c40153980 2023-10-30T12:57:45.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c45646640 1 -- 172.21.15.12:0/2387312663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4015ab90 con 0x7f6c40153980 2023-10-30T12:57:45.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c45646640 1 -- 172.21.15.12:0/2387312663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4015b4e0 con 0x7f6c40153980 2023-10-30T12:57:45.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c45646640 1 -- 172.21.15.12:0/2387312663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4015bce0 con 0x7f6c40153980 2023-10-30T12:57:45.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c377fe640 1 -- 172.21.15.12:0/2387312663 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6c2800ccc0 con 0x7f6c40153980 2023-10-30T12:57:45.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.814+0000 7f6c377fe640 1 --2- 172.21.15.12:0/2387312663 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6c200664a0 0x7f6c20068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.818+0000 7f6c377fe640 1 -- 172.21.15.12:0/2387312663 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6943+0+0 (secure 0 0 0) 0x7f6c280845b0 con 0x7f6c40153980 2023-10-30T12:57:45.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.818+0000 7f6c3effd640 1 --2- 172.21.15.12:0/2387312663 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6c200664a0 0x7f6c20068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.818+0000 7f6c3effd640 1 --2- 172.21.15.12:0/2387312663 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6c200664a0 0x7f6c20068960 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009a70 tx=0x7f6c30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:45.822+0000 7f6c377fe640 1 -- 172.21.15.12:0/2387312663 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6c28051430 con 0x7f6c40153980 2023-10-30T12:57:46.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.010+0000 7f6c45646640 1 -- 172.21.15.12:0/2387312663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6c4004e170 con 0x7f6c40153980 2023-10-30T12:57:46.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.674+0000 7f6c377fe640 1 -- 172.21.15.12:0/2387312663 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v109) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6c28054a50 con 0x7f6c40153980 2023-10-30T12:57:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.678+0000 7f6c45646640 1 -- 172.21.15.12:0/2387312663 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6c200664a0 msgr2=0x7f6c20068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.678+0000 7f6c45646640 1 --2- 172.21.15.12:0/2387312663 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6c200664a0 0x7f6c20068960 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009a70 tx=0x7f6c30008040 comp rx=0 tx=0).stop 2023-10-30T12:57:46.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.678+0000 7f6c45646640 1 -- 172.21.15.12:0/2387312663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c40153980 msgr2=0x7f6c400cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.678+0000 7f6c45646640 1 --2- 172.21.15.12:0/2387312663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c40153980 0x7f6c400cd490 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800e870 tx=0x7f6c2800ec70 comp rx=0 tx=0).stop 2023-10-30T12:57:46.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.678+0000 7f6c45646640 1 -- 172.21.15.12:0/2387312663 shutdown_connections 2023-10-30T12:57:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.678+0000 7f6c45646640 1 --2- 172.21.15.12:0/2387312663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c400cd9d0 0x7f6c40146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.678+0000 7f6c45646640 1 --2- 172.21.15.12:0/2387312663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c40153980 0x7f6c400cd490 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.678+0000 7f6c45646640 1 --2- 172.21.15.12:0/2387312663 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f6c200664a0 0x7f6c20068960 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.678+0000 7f6c45646640 1 --2- 172.21.15.12:0/2387312663 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c400cf1a0 0x7f6c40152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.678+0000 7f6c45646640 1 -- 172.21.15.12:0/2387312663 >> 172.21.15.12:0/2387312663 conn(0x7f6c4006c0f0 msgr2=0x7f6c40158940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:46.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.678+0000 7f6c45646640 1 -- 172.21.15.12:0/2387312663 shutdown_connections 2023-10-30T12:57:46.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.678+0000 7f6c45646640 1 -- 172.21.15.12:0/2387312663 wait complete. 2023-10-30T12:57:46.706 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-30T12:57:46.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.894+0000 7f206a7c5640 1 Processor -- start 2023-10-30T12:57:46.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.894+0000 7f206a7c5640 1 -- start start 2023-10-30T12:57:46.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.894+0000 7f206a7c5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20640c6eb0 0x7f20640c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.894+0000 7f206a7c5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20640c6a50 con 0x7f2064156ed0 2023-10-30T12:57:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.894+0000 7f206a7c5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20640c3750 con 0x7f20640c2ca0 2023-10-30T12:57:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.894+0000 7f206a7c5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20640c6630 con 0x7f20640c6eb0 2023-10-30T12:57:46.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.894+0000 7f2063fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2064156ed0 0x7f20640c43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42546/0 (socket says 172.21.15.12:42546) 2023-10-30T12:57:46.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.894+0000 7f2063fff640 1 -- 172.21.15.12:0/2477962021 learned_addr learned my addr 172.21.15.12:0/2477962021 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:57:46.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.898+0000 7f20637fe640 1 --2- 172.21.15.12:0/2477962021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20640c6eb0 0x7f20640c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.898+0000 7f2062ffd640 1 -- 172.21.15.12:0/2477962021 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764339722 0 0) 0x7f20640c6a50 con 0x7f2064156ed0 2023-10-30T12:57:46.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.898+0000 7f2062ffd640 1 -- 172.21.15.12:0/2477962021 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f204c003530 con 0x7f2064156ed0 2023-10-30T12:57:46.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.898+0000 7f2062ffd640 1 -- 172.21.15.12:0/2477962021 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216415351 0 0) 0x7f20640c3750 con 0x7f20640c2ca0 2023-10-30T12:57:46.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.898+0000 7f2062ffd640 1 -- 172.21.15.12:0/2477962021 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20640c6a50 con 0x7f20640c2ca0 2023-10-30T12:57:46.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.898+0000 7f2062ffd640 1 -- 172.21.15.12:0/2477962021 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3931530461 0 0) 0x7f204c003530 con 0x7f2064156ed0 2023-10-30T12:57:46.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.898+0000 7f20637fe640 1 -- 172.21.15.12:0/2477962021 >> v1:172.21.15.154:6789/0 conn(0x7f20640c2ca0 legacy=0x7f206406f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.898+0000 7f20637fe640 1 -- 172.21.15.12:0/2477962021 >> v1:172.21.15.12:6789/0 conn(0x7f2064156ed0 legacy=0x7f20640c43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.898+0000 7f20637fe640 1 -- 172.21.15.12:0/2477962021 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20641572b0 con 0x7f20640c6eb0 2023-10-30T12:57:46.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.898+0000 7f20637fe640 1 --2- 172.21.15.12:0/2477962021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20640c6eb0 0x7f20640c2760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f20580097f0 tx=0x7f205801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e63cbf0823540e9d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.898+0000 7f2062ffd640 1 -- 172.21.15.12:0/2477962021 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054001680 con 0x7f20640c6eb0 2023-10-30T12:57:46.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f2062ffd640 1 -- 172.21.15.12:0/2477962021 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f20580a7050 con 0x7f20640c6eb0 2023-10-30T12:57:46.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f2062ffd640 1 -- 172.21.15.12:0/2477962021 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20580ab600 con 0x7f20640c6eb0 2023-10-30T12:57:46.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f206a7c5640 1 -- 172.21.15.12:0/2477962021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20640c6eb0 msgr2=0x7f20640c2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f206a7c5640 1 --2- 172.21.15.12:0/2477962021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20640c6eb0 0x7f20640c2760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f20580097f0 tx=0x7f205801c3b0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f206a7c5640 1 -- 172.21.15.12:0/2477962021 shutdown_connections 2023-10-30T12:57:46.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f206a7c5640 1 --2- 172.21.15.12:0/2477962021 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f20640c6eb0 0x7f20640c2760 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f206a7c5640 1 -- 172.21.15.12:0/2477962021 >> 172.21.15.12:0/2477962021 conn(0x7f20640ccf80 msgr2=0x7f20640cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:46.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f206a7c5640 1 -- 172.21.15.12:0/2477962021 shutdown_connections 2023-10-30T12:57:46.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f206a7c5640 1 -- 172.21.15.12:0/2477962021 wait complete. 2023-10-30T12:57:46.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f206a7c5640 1 Processor -- start 2023-10-30T12:57:46.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f206a7c5640 1 -- start start 2023-10-30T12:57:46.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f206a7c5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20640c2ca0 0x7f206414e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f206a7c5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20640c6eb0 0x7f2064148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f206a7c5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2064156ed0 0x7f2064143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f206a7c5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2064143910 con 0x7f20640c2ca0 2023-10-30T12:57:46.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f206a7c5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2064148ca0 con 0x7f20640c6eb0 2023-10-30T12:57:46.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.902+0000 7f206a7c5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20641f24e0 con 0x7f2064156ed0 2023-10-30T12:57:46.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f2063fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20640c2ca0 0x7f206414e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f2063fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20640c2ca0 0x7f206414e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51216/0 (socket says 172.21.15.12:51216) 2023-10-30T12:57:46.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f2063fff640 1 -- 172.21.15.12:0/2896820350 learned_addr learned my addr 172.21.15.12:0/2896820350 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:57:46.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f20637fe640 1 --2- 172.21.15.12:0/2896820350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20640c6eb0 0x7f2064148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f2068d62640 1 --2- 172.21.15.12:0/2896820350 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2064156ed0 0x7f2064143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f2063fff640 1 -- 172.21.15.12:0/2896820350 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2064156ed0 msgr2=0x7f2064143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f2063fff640 1 --2- 172.21.15.12:0/2896820350 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2064156ed0 0x7f2064143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f2063fff640 1 -- 172.21.15.12:0/2896820350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20640c6eb0 msgr2=0x7f2064148090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f2063fff640 1 --2- 172.21.15.12:0/2896820350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20640c6eb0 0x7f2064148090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f2063fff640 1 -- 172.21.15.12:0/2896820350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2058009420 con 0x7f20640c2ca0 2023-10-30T12:57:46.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f2063fff640 1 --2- 172.21.15.12:0/2896820350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20640c2ca0 0x7f206414e990 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f205400c700 tx=0x7f205400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f2060ff9640 1 -- 172.21.15.12:0/2896820350 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054005ae0 con 0x7f20640c2ca0 2023-10-30T12:57:46.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f2060ff9640 1 -- 172.21.15.12:0/2896820350 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2054005c40 con 0x7f20640c2ca0 2023-10-30T12:57:46.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f2060ff9640 1 -- 172.21.15.12:0/2896820350 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054007290 con 0x7f20640c2ca0 2023-10-30T12:57:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f206a7c5640 1 -- 172.21.15.12:0/2896820350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2064148f40 con 0x7f20640c2ca0 2023-10-30T12:57:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f206a7c5640 1 -- 172.21.15.12:0/2896820350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20641f27c0 con 0x7f20640c2ca0 2023-10-30T12:57:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f206a7c5640 1 -- 172.21.15.12:0/2896820350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206404e170 con 0x7f20640c2ca0 2023-10-30T12:57:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f2060ff9640 1 -- 172.21.15.12:0/2896820350 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f20540073f0 con 0x7f20640c2ca0 2023-10-30T12:57:46.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.906+0000 7f2060ff9640 1 --2- 172.21.15.12:0/2896820350 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20500664d0 0x7f2050068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.910+0000 7f2060ff9640 1 -- 172.21.15.12:0/2896820350 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6943+0+0 (secure 0 0 0) 0x7f2054083b70 con 0x7f20640c2ca0 2023-10-30T12:57:46.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.910+0000 7f20637fe640 1 --2- 172.21.15.12:0/2896820350 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20500664d0 0x7f2050068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.910+0000 7f20637fe640 1 --2- 172.21.15.12:0/2896820350 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20500664d0 0x7f2050068990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f20580ad000 tx=0x7f20580a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:46.914+0000 7f2060ff9640 1 -- 172.21.15.12:0/2896820350 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f20540509f0 con 0x7f20640c2ca0 2023-10-30T12:57:47.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.074+0000 7f206a7c5640 1 -- 172.21.15.12:0/2896820350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f20640c9570 con 0x7f20640c2ca0 2023-10-30T12:57:47.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.686+0000 7f2060ff9640 1 -- 172.21.15.12:0/2896820350 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2054017020 con 0x7f20640c2ca0 2023-10-30T12:57:47.691 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' removed 2023-10-30T12:57:47.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.686+0000 7f206a7c5640 1 -- 172.21.15.12:0/2896820350 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20500664d0 msgr2=0x7f2050068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:47.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.686+0000 7f206a7c5640 1 --2- 172.21.15.12:0/2896820350 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20500664d0 0x7f2050068990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f20580ad000 tx=0x7f20580a6000 comp rx=0 tx=0).stop 2023-10-30T12:57:47.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.686+0000 7f206a7c5640 1 -- 172.21.15.12:0/2896820350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20640c2ca0 msgr2=0x7f206414e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:47.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.686+0000 7f206a7c5640 1 --2- 172.21.15.12:0/2896820350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20640c2ca0 0x7f206414e990 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f205400c700 tx=0x7f205400cbc0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.686+0000 7f206a7c5640 1 -- 172.21.15.12:0/2896820350 shutdown_connections 2023-10-30T12:57:47.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.686+0000 7f206a7c5640 1 --2- 172.21.15.12:0/2896820350 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f20500664d0 0x7f2050068990 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.686+0000 7f206a7c5640 1 --2- 172.21.15.12:0/2896820350 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2064156ed0 0x7f2064143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.686+0000 7f206a7c5640 1 --2- 172.21.15.12:0/2896820350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20640c6eb0 0x7f2064148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.686+0000 7f206a7c5640 1 --2- 172.21.15.12:0/2896820350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20640c2ca0 0x7f206414e990 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.686+0000 7f206a7c5640 1 -- 172.21.15.12:0/2896820350 >> 172.21.15.12:0/2896820350 conn(0x7f20640ccf80 msgr2=0x7f206414eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:47.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.690+0000 7f206a7c5640 1 -- 172.21.15.12:0/2896820350 shutdown_connections 2023-10-30T12:57:47.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.690+0000 7f206a7c5640 1 -- 172.21.15.12:0/2896820350 wait complete. 2023-10-30T12:57:47.727 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-30T12:57:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.886+0000 7f5a2586b640 1 Processor -- start 2023-10-30T12:57:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.886+0000 7f5a2586b640 1 -- start start 2023-10-30T12:57:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a2586b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a200cb3c0 0x7f5a200c4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a2586b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a200c4540 con 0x7f5a2015d9d0 2023-10-30T12:57:47.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a2586b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a200cb810 con 0x7f5a201536d0 2023-10-30T12:57:47.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a2586b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2015a210 con 0x7f5a200cb3c0 2023-10-30T12:57:47.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1e7fc640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5a2015d9d0 0x7f5a200cf090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42562/0 (socket says 172.21.15.12:42562) 2023-10-30T12:57:47.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1e7fc640 1 -- 172.21.15.12:0/3583545144 learned_addr learned my addr 172.21.15.12:0/3583545144 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-30T12:57:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1f7fe640 1 --2- 172.21.15.12:0/3583545144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a200cb3c0 0x7f5a200c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:47.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2982786381 0 0) 0x7f5a200c4540 con 0x7f5a2015d9d0 2023-10-30T12:57:47.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a08003530 con 0x7f5a2015d9d0 2023-10-30T12:57:47.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048865334 0 0) 0x7f5a200cb810 con 0x7f5a201536d0 2023-10-30T12:57:47.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a200c4540 con 0x7f5a201536d0 2023-10-30T12:57:47.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2588027784 0 0) 0x7f5a08003530 con 0x7f5a2015d9d0 2023-10-30T12:57:47.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a200cb3c0 msgr2=0x7f5a200c4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:47.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 --2- 172.21.15.12:0/3583545144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a200cb3c0 0x7f5a200c4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 >> v1:172.21.15.154:6789/0 conn(0x7f5a201536d0 legacy=0x7f5a20159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2015a4f0 con 0x7f5a2015d9d0 2023-10-30T12:57:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2630567990 0 0) 0x7f5a14001600 con 0x7f5a2015d9d0 2023-10-30T12:57:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 >> v1:172.21.15.12:6789/0 conn(0x7f5a2015d9d0 legacy=0x7f5a200cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:47.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 --2- 172.21.15.12:0/3583545144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a08003b40 0x7f5a08003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:47.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 --2- 172.21.15.12:0/3583545144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a080047d0 0x7f5a08004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:47.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 --2- 172.21.15.12:0/3583545144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a08005190 0x7f5a0800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2015a210 con 0x7f5a080047d0 2023-10-30T12:57:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0800bfb0 con 0x7f5a08003b40 2023-10-30T12:57:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0800c170 con 0x7f5a08005190 2023-10-30T12:57:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1f7fe640 1 --2- 172.21.15.12:0/3583545144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a200cb3c0 0x7f5a200c4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:57:47.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1effd640 1 --2- 172.21.15.12:0/3583545144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a08003b40 0x7f5a08003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:47.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1e7fc640 1 --2- 172.21.15.12:0/3583545144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a080047d0 0x7f5a08004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:47.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1f7fe640 1 --2- 172.21.15.12:0/3583545144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a08005190 0x7f5a0800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1e7fc640 1 -- 172.21.15.12:0/3583545144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a08005190 msgr2=0x7f5a0800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1e7fc640 1 --2- 172.21.15.12:0/3583545144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a08005190 0x7f5a0800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1e7fc640 1 -- 172.21.15.12:0/3583545144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a08003b40 msgr2=0x7f5a08003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1e7fc640 1 --2- 172.21.15.12:0/3583545144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a08003b40 0x7f5a08003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.890+0000 7f5a1e7fc640 1 -- 172.21.15.12:0/3583545144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a08003530 con 0x7f5a080047d0 2023-10-30T12:57:47.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.894+0000 7f5a1e7fc640 1 --2- 172.21.15.12:0/3583545144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a080047d0 0x7f5a08004c50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5a14007f20 tx=0x7f5a1401ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af824b246845d9a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:47.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.894+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14005ae0 con 0x7f5a080047d0 2023-10-30T12:57:47.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.894+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5a14005c40 con 0x7f5a080047d0 2023-10-30T12:57:47.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.894+0000 7f5a1dffb640 1 -- 172.21.15.12:0/3583545144 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a140a5770 con 0x7f5a080047d0 2023-10-30T12:57:47.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a2586b640 1 -- 172.21.15.12:0/3583545144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a080047d0 msgr2=0x7f5a08004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:47.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a2586b640 1 --2- 172.21.15.12:0/3583545144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a080047d0 0x7f5a08004c50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5a14007f20 tx=0x7f5a1401ca80 comp rx=0 tx=0).stop 2023-10-30T12:57:47.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a1f7fe640 1 -- 172.21.15.12:0/3583545144 reap_dead start 2023-10-30T12:57:47.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a2586b640 1 -- 172.21.15.12:0/3583545144 shutdown_connections 2023-10-30T12:57:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a2586b640 1 -- 172.21.15.12:0/3583545144 >> 172.21.15.12:0/3583545144 conn(0x7f5a2006bf70 msgr2=0x7f5a2006c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a2586b640 1 -- 172.21.15.12:0/3583545144 shutdown_connections 2023-10-30T12:57:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a2586b640 1 -- 172.21.15.12:0/3583545144 wait complete. 2023-10-30T12:57:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a2586b640 1 Processor -- start 2023-10-30T12:57:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a2586b640 1 -- start start 2023-10-30T12:57:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a2586b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a2015c500 0x7f5a200b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a2586b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a200cd5e0 0x7f5a20146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a2586b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2015a920 0x7f5a20146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a2586b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2015aee0 con 0x7f5a2015a920 2023-10-30T12:57:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a2586b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a200ce090 con 0x7f5a2015c500 2023-10-30T12:57:47.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a2586b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a200cdad0 con 0x7f5a200cd5e0 2023-10-30T12:57:47.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a1f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2015a920 0x7f5a20146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:47.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a1f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2015a920 0x7f5a20146650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51228/0 (socket says 172.21.15.12:51228) 2023-10-30T12:57:47.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a1f7fe640 1 -- 172.21.15.12:0/3337387668 learned_addr learned my addr 172.21.15.12:0/3337387668 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-30T12:57:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a1effd640 1 --2- 172.21.15.12:0/3337387668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a2015c500 0x7f5a200b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.898+0000 7f5a1e7fc640 1 --2- 172.21.15.12:0/3337387668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a200cd5e0 0x7f5a20146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a1f7fe640 1 -- 172.21.15.12:0/3337387668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a200cd5e0 msgr2=0x7f5a20146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a1f7fe640 1 --2- 172.21.15.12:0/3337387668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a200cd5e0 0x7f5a20146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a1f7fe640 1 -- 172.21.15.12:0/3337387668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a2015c500 msgr2=0x7f5a200b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a1f7fe640 1 --2- 172.21.15.12:0/3337387668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a2015c500 0x7f5a200b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a1f7fe640 1 -- 172.21.15.12:0/3337387668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a200cddb0 con 0x7f5a2015a920 2023-10-30T12:57:47.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a1f7fe640 1 --2- 172.21.15.12:0/3337387668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2015a920 0x7f5a20146650 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5a200b30c0 tx=0x7f5a1800f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:47.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a07fff640 1 -- 172.21.15.12:0/3337387668 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1800e040 con 0x7f5a2015a920 2023-10-30T12:57:47.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a07fff640 1 -- 172.21.15.12:0/3337387668 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5a1800f040 con 0x7f5a2015a920 2023-10-30T12:57:47.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a07fff640 1 -- 172.21.15.12:0/3337387668 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18003c00 con 0x7f5a2015a920 2023-10-30T12:57:47.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a2586b640 1 -- 172.21.15.12:0/3337387668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a200b32d0 con 0x7f5a2015a920 2023-10-30T12:57:47.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a2586b640 1 -- 172.21.15.12:0/3337387668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a200b3890 con 0x7f5a2015a920 2023-10-30T12:57:47.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a2586b640 1 -- 172.21.15.12:0/3337387668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a200b3a80 con 0x7f5a2015a920 2023-10-30T12:57:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a07fff640 1 -- 172.21.15.12:0/3337387668 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5a18004030 con 0x7f5a2015a920 2023-10-30T12:57:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a07fff640 1 --2- 172.21.15.12:0/3337387668 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5a0c0664a0 0x7f5a0c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a07fff640 1 -- 172.21.15.12:0/3337387668 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f5a18083fb0 con 0x7f5a2015a920 2023-10-30T12:57:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.902+0000 7f5a1effd640 1 --2- 172.21.15.12:0/3337387668 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5a0c0664a0 0x7f5a0c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.906+0000 7f5a1effd640 1 --2- 172.21.15.12:0/3337387668 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5a0c0664a0 0x7f5a0c068960 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5a10005630 tx=0x7f5a10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:47.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:47.906+0000 7f5a07fff640 1 -- 172.21.15.12:0/3337387668 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5a18051100 con 0x7f5a2015a920 2023-10-30T12:57:48.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:48.066+0000 7f5a2586b640 1 -- 172.21.15.12:0/3337387668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5a2004e170 con 0x7f5a2015a920 2023-10-30T12:57:48.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:48.686+0000 7f5a07fff640 1 -- 172.21.15.12:0/3337387668 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5a18054720 con 0x7f5a2015a920 2023-10-30T12:57:48.693 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' removed 2023-10-30T12:57:48.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:48.690+0000 7f5a2586b640 1 -- 172.21.15.12:0/3337387668 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5a0c0664a0 msgr2=0x7f5a0c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:48.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:48.690+0000 7f5a2586b640 1 --2- 172.21.15.12:0/3337387668 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5a0c0664a0 0x7f5a0c068960 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5a10005630 tx=0x7f5a10009040 comp rx=0 tx=0).stop 2023-10-30T12:57:48.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:48.690+0000 7f5a2586b640 1 -- 172.21.15.12:0/3337387668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2015a920 msgr2=0x7f5a20146650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:48.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:48.690+0000 7f5a2586b640 1 --2- 172.21.15.12:0/3337387668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2015a920 0x7f5a20146650 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5a200b30c0 tx=0x7f5a1800f7b0 comp rx=0 tx=0).stop 2023-10-30T12:57:48.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:48.690+0000 7f5a2586b640 1 -- 172.21.15.12:0/3337387668 shutdown_connections 2023-10-30T12:57:48.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:48.690+0000 7f5a2586b640 1 --2- 172.21.15.12:0/3337387668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2015a920 0x7f5a20146650 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:48.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:48.690+0000 7f5a2586b640 1 --2- 172.21.15.12:0/3337387668 >> [v2:172.21.15.154:6832/1474604059,v1:172.21.15.154:6833/1474604059] conn(0x7f5a0c0664a0 0x7f5a0c068960 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:48.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:48.690+0000 7f5a2586b640 1 --2- 172.21.15.12:0/3337387668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a200cd5e0 0x7f5a20146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:48.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:48.690+0000 7f5a2586b640 1 --2- 172.21.15.12:0/3337387668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a2015c500 0x7f5a200b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:48.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:48.690+0000 7f5a2586b640 1 -- 172.21.15.12:0/3337387668 >> 172.21.15.12:0/3337387668 conn(0x7f5a2006bf70 msgr2=0x7f5a201e2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:48.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:48.690+0000 7f5a2586b640 1 -- 172.21.15.12:0/3337387668 shutdown_connections 2023-10-30T12:57:48.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-30T12:57:48.690+0000 7f5a2586b640 1 -- 172.21.15.12:0/3337387668 wait complete. 2023-10-30T12:57:48.734 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-30T12:57:48.735 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-30T12:57:48.735 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:48.758 INFO:tasks.ceph.mds.a:Stopped 2023-10-30T12:57:48.759 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-30T12:57:48.759 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:48.826 INFO:tasks.ceph.mds.c:Stopped 2023-10-30T12:57:48.826 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-10-30T12:57:48.827 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:48.890 INFO:tasks.ceph.mds.e:Stopped 2023-10-30T12:57:48.891 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-30T12:57:48.891 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:48.906 INFO:tasks.ceph.mds.b:Stopped 2023-10-30T12:57:48.906 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-30T12:57:48.907 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:48.958 INFO:tasks.ceph.mds.d:Stopped 2023-10-30T12:57:48.959 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-10-30T12:57:48.959 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:49.020 INFO:tasks.ceph.mds.f:Stopped 2023-10-30T12:57:49.021 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-30T12:57:49.021 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-30T12:57:49.022 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:49.121 INFO:tasks.ceph.osd.0:Stopped 2023-10-30T12:57:49.122 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-30T12:57:49.122 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:49.221 INFO:tasks.ceph.osd.1:Stopped 2023-10-30T12:57:49.222 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-30T12:57:49.222 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:49.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~0s 2023-10-30T12:57:49.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-30T12:57:49.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-30T12:57:49.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~0s 2023-10-30T12:57:49.315 INFO:tasks.ceph.osd.2:Stopped 2023-10-30T12:57:49.315 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-30T12:57:49.316 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:49.408 INFO:tasks.ceph.osd.3:Stopped 2023-10-30T12:57:49.409 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-30T12:57:49.409 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:49.489 INFO:tasks.ceph.osd.4:Stopped 2023-10-30T12:57:49.490 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-30T12:57:49.490 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:49.575 INFO:tasks.ceph.osd.5:Stopped 2023-10-30T12:57:49.576 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-30T12:57:49.576 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:54.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-10-30T12:57:54.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e has been restored 2023-10-30T12:57:54.830 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-30T12:57:54.830 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-30T12:57:54.830 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f has been restored 2023-10-30T12:57:55.678 INFO:tasks.ceph.osd.6:Stopped 2023-10-30T12:57:55.679 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-30T12:57:55.679 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:55.771 INFO:tasks.ceph.osd.7:Stopped 2023-10-30T12:57:55.771 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-30T12:57:55.772 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-30T12:57:55.772 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:55.826 INFO:tasks.ceph.mgr.y:Stopped 2023-10-30T12:57:55.827 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-30T12:57:55.828 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:55.857 INFO:tasks.ceph.mgr.x:Stopped 2023-10-30T12:57:55.857 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-30T12:57:55.858 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-30T12:57:55.858 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:55.879 INFO:tasks.ceph.mon.a:Stopped 2023-10-30T12:57:55.879 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-30T12:57:55.880 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:55.917 INFO:tasks.ceph.mon.b:Stopped 2023-10-30T12:57:55.918 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-30T12:57:55.918 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:57:55.974 INFO:tasks.ceph.mon.c:Stopped 2023-10-30T12:57:55.974 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-30T12:57:55.975 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v 'Replacing daemon mds' | head -n 1 2023-10-30T12:57:56.017 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi012.front.sepia.ceph.com 2023-10-30T12:57:56.017 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-30T12:57:56.664 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi012.front.sepia.ceph.com 2023-10-30T12:57:56.664 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-30T12:57:56.901 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi012.front.sepia.ceph.com 2023-10-30T12:57:56.902 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-30T12:57:57.072 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi012.front.sepia.ceph.com 2023-10-30T12:57:57.073 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-30T12:57:57.233 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-30T12:57:57.234 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-30T12:57:57.471 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-30T12:57:57.472 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-30T12:57:57.666 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-30T12:57:57.666 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-30T12:57:57.848 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-30T12:57:57.849 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-30T12:57:58.047 INFO:tasks.ceph:Archiving mon data... 2023-10-30T12:57:58.048 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026/data/mon.a.tgz 2023-10-30T12:57:58.049 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2023-10-30T12:57:58.058 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.IBbEU5nqCP 2023-10-30T12:57:58.059 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.IBbEU5nqCP 2023-10-30T12:57:58.195 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.IBbEU5nqCP 2023-10-30T12:57:58.278 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.IBbEU5nqCP is 344KB 2023-10-30T12:57:58.343 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.IBbEU5nqCP 2023-10-30T12:57:58.352 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026/data/mon.b.tgz 2023-10-30T12:57:58.353 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2023-10-30T12:57:58.359 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.PQXLwH3tQt 2023-10-30T12:57:58.359 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.PQXLwH3tQt 2023-10-30T12:57:58.483 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.PQXLwH3tQt 2023-10-30T12:57:58.548 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.PQXLwH3tQt is 346KB 2023-10-30T12:57:58.610 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.PQXLwH3tQt 2023-10-30T12:57:58.616 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026/data/mon.c.tgz 2023-10-30T12:57:58.616 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2023-10-30T12:57:58.664 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.JMH1Ms9y2Y 2023-10-30T12:57:58.665 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.JMH1Ms9y2Y 2023-10-30T12:57:58.790 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.JMH1Ms9y2Y 2023-10-30T12:57:58.852 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.JMH1Ms9y2Y is 346KB 2023-10-30T12:57:58.910 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.JMH1Ms9y2Y 2023-10-30T12:57:58.917 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-30T12:57:58.919 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-30T12:57:58.922 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-30T12:57:58.972 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-30T12:57:58.972 DEBUG:teuthology.orchestra.run.smithi012:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-30T12:57:58.988 DEBUG:teuthology.orchestra.run.smithi154:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-30T12:57:59.024 INFO:teuthology.orchestra.run.smithi012.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-30T12:57:59.033 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-30T12:57:59.037 INFO:tasks.ceph:Archiving crash dumps... 2023-10-30T12:57:59.038 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026/remote/smithi012/crash 2023-10-30T12:57:59.040 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-30T12:57:59.098 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026/remote/smithi154/crash 2023-10-30T12:57:59.100 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-30T12:57:59.112 INFO:tasks.ceph:Shutting down logrotate 2023-10-30T12:57:59.113 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:57:59.148 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:57:59.205 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-30T12:57:59.227 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-30T12:57:59.238 INFO:tasks.ceph:Compressing logs... 2023-10-30T12:57:59.238 DEBUG:teuthology.orchestra.run.smithi012:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-30T12:57:59.272 DEBUG:teuthology.orchestra.run.smithi154:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-30T12:57:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44209.log 2023-10-30T12:57:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54400.log 2023-10-30T12:57:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50519.log 2023-10-30T12:57:59.292 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44209.log.gz 2023-10-30T12:57:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49435.log 2023-10-30T12:57:59.293 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54400.log.gz 2023-10-30T12:57:59.294 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52808.log 2023-10-30T12:57:59.294 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50519.log.gz 2023-10-30T12:57:59.295 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47002.log 2023-10-30T12:57:59.295 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49435.log.gz 2023-10-30T12:57:59.296 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52808.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52808.log.gz 2023-10-30T12:57:59.296 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48064.log 2023-10-30T12:57:59.298 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54504.log 2023-10-30T12:57:59.298 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47002.log.gz 2023-10-30T12:57:59.298 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48064.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48064.log.gz 2023-10-30T12:57:59.298 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44792.log 2023-10-30T12:57:59.299 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54504.log: gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.54504.log.gz 2023-10-30T12:57:59.300 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53722.log 2023-10-30T12:57:59.300 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44792.log: gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.44198.log -- replaced with /var/log/ceph/ceph-client.admin.44792.log.gz 2023-10-30T12:57:59.300 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-30T12:57:59.301 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53722.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53722.log.gz 2023-10-30T12:57:59.302 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51134.log 2023-10-30T12:57:59.303 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44198.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44198.log.gz -5 2023-10-30T12:57:59.303 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52120.log 2023-10-30T12:57:59.303 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53833.log 2023-10-30T12:57:59.303 INFO:teuthology.orchestra.run.smithi012.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51134.log.gz 2023-10-30T12:57:59.304 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52120.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52120.log.gz 2023-10-30T12:57:59.305 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-30T12:57:59.306 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49647.log 2023-10-30T12:57:59.307 INFO:teuthology.orchestra.run.smithi012.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53833.log.gz 2023-10-30T12:57:59.307 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54124.log 2023-10-30T12:57:59.309 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38803.log 2023-10-30T12:57:59.309 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-30T12:57:59.310 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38803.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-30T12:57:59.310 INFO:teuthology.orchestra.run.smithi154.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38803.log.gz 2023-10-30T12:57:59.310 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38827.log 2023-10-30T12:57:59.311 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38705.log 2023-10-30T12:57:59.311 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38827.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38827.log.gz 2023-10-30T12:57:59.311 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-30T12:57:59.311 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-30T12:57:59.312 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.38705.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2023-10-30T12:57:59.312 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-30T12:57:59.312 INFO:teuthology.orchestra.run.smithi154.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38705.log.gz 2023-10-30T12:57:59.313 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-30T12:57:59.313 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38729.log 2023-10-30T12:57:59.316 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38681.log 2023-10-30T12:57:59.317 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.38729.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38729.log.gz 2023-10-30T12:57:59.317 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-30T12:57:59.317 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38681.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38681.log.gz 2023-10-30T12:57:59.318 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-30T12:57:59.318 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.audit.log: 94.2% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2023-10-30T12:57:59.318 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38657.log 2023-10-30T12:57:59.319 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mds.d.log: 94.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-30T12:57:59.319 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38779.log 2023-10-30T12:57:59.323 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38657.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38657.log.gz 2023-10-30T12:57:59.325 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-30T12:57:59.325 INFO:teuthology.orchestra.run.smithi154.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-30T12:57:59.325 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38779.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-30T12:57:59.326 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mds.b.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38779.log.gz 2023-10-30T12:57:59.326 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38755.log 2023-10-30T12:57:59.329 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mon.b.log: 94.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-30T12:57:59.329 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-30T12:57:59.330 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38755.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38755.log.gz 2023-10-30T12:57:59.333 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49647.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49647.log.gz 2023-10-30T12:57:59.333 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45171.log 2023-10-30T12:57:59.333 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46898.log 2023-10-30T12:57:59.334 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54124.log.gz 2023-10-30T12:57:59.334 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45171.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.45171.log.gz 2023-10-30T12:57:59.334 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53197.log 2023-10-30T12:57:59.335 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51805.log 2023-10-30T12:57:59.335 INFO:teuthology.orchestra.run.smithi012.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46898.log.gz 2023-10-30T12:57:59.335 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53197.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53197.log.gz 2023-10-30T12:57:59.335 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52706.log 2023-10-30T12:57:59.336 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43983.log 2023-10-30T12:57:59.336 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51805.log.gz 2023-10-30T12:57:59.336 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52706.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52706.log.gz 2023-10-30T12:57:59.337 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.33121.log 2023-10-30T12:57:59.337 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44002.log 2023-10-30T12:57:59.337 INFO:teuthology.orchestra.run.smithi012.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43983.log.gz 2023-10-30T12:57:59.337 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph.tmp-client.admin.33121.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.33121.log.gz 2023-10-30T12:57:59.338 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54224.log 2023-10-30T12:57:59.338 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52303.log 2023-10-30T12:57:59.338 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44002.log.gz 2023-10-30T12:57:59.339 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54224.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.54224.log.gz --verbose 2023-10-30T12:57:59.339 INFO:teuthology.orchestra.run.smithi012.stderr: -- /var/log/ceph/ceph-client.admin.54712.log 2023-10-30T12:57:59.339 INFO:teuthology.orchestra.run.smithi012.stderr: 95.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz/var/log/ceph/ceph-client.admin.52303.log: 2023-10-30T12:57:59.339 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44000.log 2023-10-30T12:57:59.340 INFO:teuthology.orchestra.run.smithi012.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52303.log.gz 2023-10-30T12:57:59.340 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45431.log 2023-10-30T12:57:59.340 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54712.log.gz 2023-10-30T12:57:59.341 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44000.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44000.log.gz 2023-10-30T12:57:59.341 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49566.log 2023-10-30T12:57:59.341 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45431.log: gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.45002.log -- replaced with /var/log/ceph/ceph-client.admin.45431.log.gz 2023-10-30T12:57:59.341 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-30T12:57:59.342 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49566.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49566.log.gz 2023-10-30T12:57:59.342 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44947.log 2023-10-30T12:57:59.342 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45002.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45002.log.gz 2023-10-30T12:57:59.343 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49594.log 2023-10-30T12:57:59.343 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51483.log 2023-10-30T12:57:59.343 INFO:teuthology.orchestra.run.smithi012.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44947.log.gz 2023-10-30T12:57:59.343 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49594.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49594.log.gz 2023-10-30T12:57:59.344 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48494.log 2023-10-30T12:57:59.344 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51483.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51483.log.gz 2023-10-30T12:57:59.344 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-30T12:57:59.344 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48494.log: gzip -5 --verbose -- 87.6% /var/log/ceph/ceph-client.admin.53885.log -- replaced with /var/log/ceph/ceph-client.admin.48494.log.gz 2023-10-30T12:57:59.345 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-30T12:57:59.345 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54979.log 2023-10-30T12:57:59.345 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44392.log 2023-10-30T12:57:59.345 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53885.log.gz 2023-10-30T12:57:59.346 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54979.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54979.log.gz 2023-10-30T12:57:59.346 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48442.log 2023-10-30T12:57:59.346 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44392.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.53142.log -- replaced with /var/log/ceph/ceph-client.admin.44392.log.gz 2023-10-30T12:57:59.347 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-30T12:57:59.347 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48442.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48442.log.gzgzip 2023-10-30T12:57:59.347 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54582.log 2023-10-30T12:57:59.354 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49754.log 2023-10-30T12:57:59.354 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53142.log.gz 2023-10-30T12:57:59.354 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54582.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.54582.log.gz --verbose 2023-10-30T12:57:59.355 INFO:teuthology.orchestra.run.smithi012.stderr: -- /var/log/ceph/ceph-client.admin.52873.log 2023-10-30T12:57:59.355 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49754.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49754.log.gzgzip 2023-10-30T12:57:59.355 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43093.log 2023-10-30T12:57:59.356 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52873.log: gzip -5 88.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.52873.log.gz -- 2023-10-30T12:57:59.356 INFO:teuthology.orchestra.run.smithi012.stderr: /var/log/ceph/ceph-client.admin.52654.log 2023-10-30T12:57:59.356 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48416.log 2023-10-30T12:57:59.356 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43093.log.gz 2023-10-30T12:57:59.357 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52654.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52654.log.gz 2023-10-30T12:57:59.357 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48038.log 2023-10-30T12:57:59.357 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48416.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48416.log.gz 2023-10-30T12:57:59.358 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46950.log 2023-10-30T12:57:59.358 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49295.log 2023-10-30T12:57:59.358 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48038.log.gz 2023-10-30T12:57:59.358 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46950.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46950.log.gz -5 2023-10-30T12:57:59.359 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50016.log 2023-10-30T12:57:59.359 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54764.log 2023-10-30T12:57:59.359 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49295.log.gz 2023-10-30T12:57:59.360 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50016.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50016.log.gz 2023-10-30T12:57:59.360 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53515.log 2023-10-30T12:57:59.360 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48196.log 89.1% 2023-10-30T12:57:59.360 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54764.log.gz 2023-10-30T12:57:59.361 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53515.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53515.log.gz 2023-10-30T12:57:59.361 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48572.log 2023-10-30T12:57:59.361 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49514.log 2023-10-30T12:57:59.361 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48196.log.gz 2023-10-30T12:57:59.362 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48572.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48572.log.gz -5 2023-10-30T12:57:59.362 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50467.log 2023-10-30T12:57:59.362 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49514.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49514.log.gzgzip 2023-10-30T12:57:59.363 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52015.log 2023-10-30T12:57:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52732.log 2023-10-30T12:57:59.363 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50467.log.gz 2023-10-30T12:57:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45249.log 2023-10-30T12:57:59.364 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52015.log.gz 2023-10-30T12:57:59.364 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51911.log 2023-10-30T12:57:59.364 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52732.log.gz 2023-10-30T12:57:59.364 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45249.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45249.log.gz 2023-10-30T12:57:59.365 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54868.log 2023-10-30T12:57:59.365 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51911.log: gzip -5 --verbose 88.3% -- -- replaced with /var/log/ceph/ceph-client.admin.51911.log.gz 2023-10-30T12:57:59.365 INFO:teuthology.orchestra.run.smithi012.stderr: /var/log/ceph/ceph-client.admin.52574.log 2023-10-30T12:57:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54868.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54868.log.gz 2023-10-30T12:57:59.371 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53275.log 2023-10-30T12:57:59.371 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52574.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52574.log.gz 2023-10-30T12:57:59.371 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53223.log 2023-10-30T12:57:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54634.log 2023-10-30T12:57:59.372 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53275.log.gz 2023-10-30T12:57:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44418.log 2023-10-30T12:57:59.372 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53223.log.gz 2023-10-30T12:57:59.373 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54634.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54634.log.gz 2023-10-30T12:57:59.373 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49806.log 2023-10-30T12:57:59.373 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44418.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.44418.log.gz 2023-10-30T12:57:59.374 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54738.log 2023-10-30T12:57:59.374 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49806.log: gzip -5 --verbose 88.2% -- -- replaced with /var/log/ceph/ceph-client.admin.49806.log.gz 2023-10-30T12:57:59.374 INFO:teuthology.orchestra.run.smithi012.stderr: /var/log/ceph/ceph-client.admin.50120.log 2023-10-30T12:57:59.374 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43788.log 2023-10-30T12:57:59.375 INFO:teuthology.orchestra.run.smithi012.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54738.log.gz 2023-10-30T12:57:59.375 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51245.log 2023-10-30T12:57:59.375 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50120.log.gz 2023-10-30T12:57:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43788.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43788.log.gz 2023-10-30T12:57:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51727.log 2023-10-30T12:57:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45561.log 2023-10-30T12:57:59.376 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51245.log.gz 2023-10-30T12:57:59.377 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51727.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51727.log.gz 2023-10-30T12:57:59.377 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51271.log 2023-10-30T12:57:59.377 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45561.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-30T12:57:59.377 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45561.log.gz 2023-10-30T12:57:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51271.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51271.log.gz 2023-10-30T12:57:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52275.log 2023-10-30T12:57:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53031.log 2023-10-30T12:57:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55109.log 2023-10-30T12:57:59.379 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52275.log.gz 2023-10-30T12:57:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44740.log 2023-10-30T12:57:59.383 INFO:teuthology.orchestra.run.smithi012.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53031.log.gz 2023-10-30T12:57:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55109.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55109.log.gz 2023-10-30T12:57:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53459.log 2023-10-30T12:57:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53355.log 2023-10-30T12:57:59.384 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44740.log.gz 2023-10-30T12:57:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53459.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53459.log.gz 2023-10-30T12:57:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43202.log 2023-10-30T12:57:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49702.log 2023-10-30T12:57:59.385 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53355.log.gz 2023-10-30T12:57:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43202.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43202.log.gz 2023-10-30T12:57:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50658.log 2023-10-30T12:57:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53966.log 2023-10-30T12:57:59.386 INFO:teuthology.orchestra.run.smithi012.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49702.log.gz 2023-10-30T12:57:59.387 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50658.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50658.log.gz 2023-10-30T12:57:59.387 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48255.log 2023-10-30T12:57:59.387 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46729.log 2023-10-30T12:57:59.387 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53966.log.gz 2023-10-30T12:57:59.388 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48255.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.48255.log.gz 2023-10-30T12:57:59.388 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-30T12:57:59.390 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49380.log 2023-10-30T12:57:59.391 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46729.log.gz 2023-10-30T12:57:59.391 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54046.log 2023-10-30T12:57:59.391 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49380.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49380.log.gz 2023-10-30T12:57:59.391 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50493.log 2023-10-30T12:57:59.392 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44531.log 2023-10-30T12:57:59.392 INFO:teuthology.orchestra.run.smithi012.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54046.log.gz 2023-10-30T12:57:59.392 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50493.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50493.log.gz 2023-10-30T12:57:59.392 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55031.log 2023-10-30T12:57:59.393 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51160.log 2023-10-30T12:57:59.393 INFO:teuthology.orchestra.run.smithi012.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44531.log.gz 2023-10-30T12:57:59.393 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55031.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55031.log.gz 2023-10-30T12:57:59.395 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52953.log 2023-10-30T12:57:59.395 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53171.log 2023-10-30T12:57:59.396 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51160.log.gz 2023-10-30T12:57:59.396 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52953.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52953.log.gz 2023-10-30T12:57:59.396 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44583.log 2023-10-30T12:57:59.396 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52899.log 2023-10-30T12:57:59.397 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53171.log.gz 2023-10-30T12:57:59.398 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44583.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44583.log.gz 2023-10-30T12:57:59.398 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47986.log 2023-10-30T12:57:59.398 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43507.log 2023-10-30T12:57:59.399 INFO:teuthology.orchestra.run.smithi012.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52899.log.gz 2023-10-30T12:57:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47986.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47986.log.gz 2023-10-30T12:57:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47028.log 2023-10-30T12:57:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43507.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43507.log.gz 2023-10-30T12:57:59.401 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50282.log 2023-10-30T12:57:59.401 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52067.log 2023-10-30T12:57:59.402 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47028.log.gz 2023-10-30T12:57:59.402 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50282.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50282.log.gz 2023-10-30T12:57:59.403 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43481.log 2023-10-30T12:57:59.404 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52067.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52067.log.gz 2023-10-30T12:57:59.404 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50791.log 2023-10-30T12:57:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43481.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43481.log.gz 2023-10-30T12:57:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49621.log 2023-10-30T12:57:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46599.log 2023-10-30T12:57:59.408 INFO:teuthology.orchestra.run.smithi012.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50791.log.gz 2023-10-30T12:57:59.408 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49621.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49621.log.gz 2023-10-30T12:57:59.409 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54426.log 2023-10-30T12:57:59.410 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46599.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46599.log.gz 2023-10-30T12:57:59.410 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50199.log 2023-10-30T12:57:59.411 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54426.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54426.log.gz 2023-10-30T12:57:59.412 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50256.log 2023-10-30T12:57:59.412 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50684.log 2023-10-30T12:57:59.413 INFO:teuthology.orchestra.run.smithi012.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50199.log.gz 2023-10-30T12:57:59.414 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50256.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50256.log.gz 2023-10-30T12:57:59.414 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51186.log 2023-10-30T12:57:59.415 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44211.log 2023-10-30T12:57:59.416 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50684.log.gz 2023-10-30T12:57:59.416 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51186.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51186.log.gz 2023-10-30T12:57:59.417 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43716.log 2023-10-30T12:57:59.418 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52223.log 2023-10-30T12:57:59.419 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44211.log.gz 2023-10-30T12:57:59.419 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43716.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43716.log.gz 2023-10-30T12:57:59.420 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43286.log 2023-10-30T12:57:59.420 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53485.log 2023-10-30T12:57:59.421 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52223.log.gz 2023-10-30T12:57:59.423 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43286.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43286.log.gz 2023-10-30T12:57:59.424 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45125.log 2023-10-30T12:57:59.424 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53485.log: gzip -5 --verbose -- 89.3% /var/log/ceph/ceph-client.admin.44635.log 2023-10-30T12:57:59.425 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53485.log.gz 2023-10-30T12:57:59.425 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.0.45125.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45125.log.gz 2023-10-30T12:57:59.426 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50545.log 2023-10-30T12:57:59.426 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49136.log 2023-10-30T12:57:59.427 INFO:teuthology.orchestra.run.smithi012.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44635.log.gz 2023-10-30T12:57:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50545.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50545.log.gz 2023-10-30T12:57:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51080.log 2023-10-30T12:57:59.429 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49136.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49136.log.gz 2023-10-30T12:57:59.430 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44818.log 2023-10-30T12:57:59.430 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48170.log 2023-10-30T12:57:59.431 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51080.log.gz 2023-10-30T12:57:59.432 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44818.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44818.log.gz 2023-10-30T12:57:59.432 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44208.log 2023-10-30T12:57:59.433 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48170.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48170.log.gz 2023-10-30T12:57:59.434 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46625.log 2023-10-30T12:57:59.434 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43057.log 2023-10-30T12:57:59.435 INFO:teuthology.orchestra.run.smithi012.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44208.log.gz 2023-10-30T12:57:59.436 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46625.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46625.log.gz 2023-10-30T12:57:59.437 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46547.log 2023-10-30T12:57:59.437 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43057.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43057.log.gz 2023-10-30T12:57:59.438 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44004.log 2023-10-30T12:57:59.439 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53617.log 2023-10-30T12:57:59.440 INFO:teuthology.orchestra.run.smithi012.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46547.log.gz 2023-10-30T12:57:59.440 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44004.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44004.log.gz 2023-10-30T12:57:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45509.log 2023-10-30T12:57:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48468.log 2023-10-30T12:57:59.442 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53617.log.gz 2023-10-30T12:57:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45509.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45509.log.gz 2023-10-30T12:57:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53381.log 2023-10-30T12:57:59.444 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48468.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48468.log.gz 2023-10-30T12:57:59.445 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51831.log 2023-10-30T12:57:59.446 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55007.log 2023-10-30T12:57:59.447 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51831.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53381.log.gz 2023-10-30T12:57:59.447 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51831.log.gz 2023-10-30T12:57:59.448 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47083.log 2023-10-30T12:57:59.448 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55007.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55007.log.gz 2023-10-30T12:57:59.449 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43742.log 2023-10-30T12:57:59.450 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54922.log 2023-10-30T12:57:59.451 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47083.log.gz 2023-10-30T12:57:59.451 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43742.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43742.log.gz 2023-10-30T12:57:59.452 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-30T12:57:59.453 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54922.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54922.log.gz 2023-10-30T12:57:59.453 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49938.log 2023-10-30T12:57:59.454 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51753.log 2023-10-30T12:57:59.456 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49938.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49938.log.gz 2023-10-30T12:57:59.456 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48144.log 2023-10-30T12:57:59.457 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51753.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51753.log.gz 2023-10-30T12:57:59.458 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50818.log 2023-10-30T12:57:59.459 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48144.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48144.log.gz 2023-10-30T12:57:59.459 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53561.log 2023-10-30T12:57:59.461 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43000.log 2023-10-30T12:57:59.461 INFO:teuthology.orchestra.run.smithi012.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50818.log.gz 2023-10-30T12:57:59.462 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43429.log 2023-10-30T12:57:59.462 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53561.log.gz 2023-10-30T12:57:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52518.log 2023-10-30T12:57:59.464 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43000.log.gz 2023-10-30T12:57:59.466 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49884.log 2023-10-30T12:57:59.466 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43429.log.gz 2023-10-30T12:57:59.467 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45327.log 2023-10-30T12:57:59.467 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52518.log.gz 2023-10-30T12:57:59.468 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49884.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49884.log.gz 2023-10-30T12:57:59.468 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47130.log 2023-10-30T12:57:59.470 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52195.log 2023-10-30T12:57:59.470 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45327.log.gz 2023-10-30T12:57:59.471 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47130.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47130.log.gz 2023-10-30T12:57:59.471 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43234.log 2023-10-30T12:57:59.472 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55161.log 2023-10-30T12:57:59.473 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52195.log.gz 2023-10-30T12:57:59.474 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43234.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43234.log.gz 2023-10-30T12:57:59.474 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45145.log 2023-10-30T12:57:59.475 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44187.log 2023-10-30T12:57:59.476 INFO:teuthology.orchestra.run.smithi012.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55161.log.gz 2023-10-30T12:57:59.476 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45145.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45145.log.gz 2023-10-30T12:57:59.477 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44661.log 2023-10-30T12:57:59.477 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44187.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44187.log.gz 2023-10-30T12:57:59.478 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45197.log 2023-10-30T12:57:59.479 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44188.log 2023-10-30T12:57:59.479 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44661.log.gz 2023-10-30T12:57:59.480 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45197.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45197.log.gz 2023-10-30T12:57:59.481 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54478.log 2023-10-30T12:57:59.481 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53914.log 2023-10-30T12:57:59.482 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44188.log.gz 2023-10-30T12:57:59.483 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54478.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54478.log.gz 2023-10-30T12:57:59.483 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51701.log 2023-10-30T12:57:59.485 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50713.log 2023-10-30T12:57:59.485 INFO:teuthology.orchestra.run.smithi012.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53914.log.gz 2023-10-30T12:57:59.486 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51701.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51701.log.gz 2023-10-30T12:57:59.486 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53090.log 2023-10-30T12:57:59.487 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49990.log 2023-10-30T12:57:59.487 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50713.log.gz 2023-10-30T12:57:59.488 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53090.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53090.log.gz 2023-10-30T12:57:59.488 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49328.log 2023-10-30T12:57:59.490 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46841.log 2023-10-30T12:57:59.490 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49990.log.gz 2023-10-30T12:57:59.491 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49328.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49328.log.gz 2023-10-30T12:57:59.491 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48229.log 2023-10-30T12:57:59.492 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43338.log 2023-10-30T12:57:59.493 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46841.log.gz 2023-10-30T12:57:59.493 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48229.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48229.log.gz 2023-10-30T12:57:59.494 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45379.log 2023-10-30T12:57:59.495 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49354.log 2023-10-30T12:57:59.495 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43338.log.gz 2023-10-30T12:57:59.496 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45379.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45379.log.gz 2023-10-30T12:57:59.496 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-30T12:57:59.497 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50632.log 2023-10-30T12:57:59.498 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.1.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49354.log.gz 2023-10-30T12:57:59.499 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47054.log 2023-10-30T12:57:59.500 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50632.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.50632.log.gz -5 2023-10-30T12:57:59.500 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49676.log 2023-10-30T12:57:59.502 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53303.log 2023-10-30T12:57:59.502 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47054.log.gz 2023-10-30T12:57:59.503 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49676.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49676.log.gz 2023-10-30T12:57:59.504 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53057.log 2023-10-30T12:57:59.504 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53303.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53303.log.gz 2023-10-30T12:57:59.505 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51963.log 2023-10-30T12:57:59.506 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53057.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53057.log.gz 2023-10-30T12:57:59.507 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44212.log 2023-10-30T12:57:59.508 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51963.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51963.log.gz 2023-10-30T12:57:59.508 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51405.log 2023-10-30T12:57:59.510 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44212.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44212.log.gz -5 2023-10-30T12:57:59.510 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44202.log 2023-10-30T12:57:59.511 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51405.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.51405.log.gz -5 2023-10-30T12:57:59.511 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54686.log 2023-10-30T12:57:59.512 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44202.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44202.log.gz 2023-10-30T12:57:59.513 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45223.log 2023-10-30T12:57:59.514 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54686.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54686.log.gz 2023-10-30T12:57:59.514 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43455.log 2023-10-30T12:57:59.515 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45223.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45223.log.gz 2023-10-30T12:57:59.516 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52249.log 2023-10-30T12:57:59.517 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52355.log 2023-10-30T12:57:59.518 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43455.log.gz 2023-10-30T12:57:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52249.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52249.log.gz 2023-10-30T12:57:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46703.log 2023-10-30T12:57:59.521 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52355.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52355.log.gz 2023-10-30T12:57:59.521 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50389.log 2023-10-30T12:57:59.522 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46703.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46703.log.gz 2023-10-30T12:57:59.522 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55135.log 2023-10-30T12:57:59.524 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53774.log 2023-10-30T12:57:59.524 INFO:teuthology.orchestra.run.smithi012.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50389.log.gz 2023-10-30T12:57:59.525 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50599.log 2023-10-30T12:57:59.526 INFO:teuthology.orchestra.run.smithi012.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55135.log.gz 2023-10-30T12:57:59.526 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53774.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53774.log.gz 2023-10-30T12:57:59.527 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53859.log 2023-10-30T12:57:59.530 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43690.log 2023-10-30T12:57:59.530 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50599.log.gz 2023-10-30T12:57:59.531 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53859.log: gzip -5 --verbose -- 87.6% /var/log/ceph/ceph-client.admin.43260.log 2023-10-30T12:57:59.531 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53859.log.gz 2023-10-30T12:57:59.532 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43690.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.52492.log 2023-10-30T12:57:59.532 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43690.log.gz 2023-10-30T12:57:59.533 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52041.log 2023-10-30T12:57:59.533 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43260.log.gz 2023-10-30T12:57:59.534 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52492.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52492.log.gz 2023-10-30T12:57:59.535 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51618.log 2023-10-30T12:57:59.536 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52041.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52041.log.gz -5 2023-10-30T12:57:59.537 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48012.log 2023-10-30T12:57:59.538 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51618.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51618.log.gz 2023-10-30T12:57:59.539 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48012.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48012.log.gz 2023-10-30T12:57:59.540 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51989.log 2023-10-30T12:57:59.541 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54660.log 2023-10-30T12:57:59.543 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45535.log 2023-10-30T12:57:59.543 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51989.log.gz 2023-10-30T12:57:59.544 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.4.log: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-30T12:57:59.545 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52626.log 2023-10-30T12:57:59.545 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54660.log.gz 2023-10-30T12:57:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52329.log 2023-10-30T12:57:59.546 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45535.log.gz 2023-10-30T12:57:59.547 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49964.log 2023-10-30T12:57:59.548 INFO:teuthology.orchestra.run.smithi012.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52626.log.gz 2023-10-30T12:57:59.548 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52329.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52329.log.gz 2023-10-30T12:57:59.550 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49964.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49964.log.gz 2023-10-30T12:57:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51297.log 2023-10-30T12:57:59.552 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43995.log 2023-10-30T12:57:59.554 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54842.log 2023-10-30T12:57:59.554 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51297.log.gz 2023-10-30T12:57:59.555 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43995.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43995.log.gz 2023-10-30T12:57:59.557 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54842.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54842.log.gz 2023-10-30T12:57:59.557 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52979.log 2023-10-30T12:57:59.558 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45587.log 2023-10-30T12:57:59.560 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54250.log 2023-10-30T12:57:59.560 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52979.log.gz 2023-10-30T12:57:59.561 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43312.log 2023-10-30T12:57:59.562 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45587.log.gz 2023-10-30T12:57:59.563 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54250.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54250.log.gz -5 2023-10-30T12:57:59.563 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49110.log 2023-10-30T12:57:59.564 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48310.log 2023-10-30T12:57:59.565 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43312.log.gz 2023-10-30T12:57:59.566 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49110.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49110.log.gz 2023-10-30T12:57:59.566 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54894.log 2023-10-30T12:57:59.568 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48310.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48310.log.gz 2023-10-30T12:57:59.568 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43559.log 2023-10-30T12:57:59.569 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51431.log 2023-10-30T12:57:59.569 INFO:teuthology.orchestra.run.smithi012.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54894.log.gz 2023-10-30T12:57:59.571 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43559.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-30T12:57:59.571 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43559.log.gz 2023-10-30T12:57:59.572 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49188.log 2023-10-30T12:57:59.572 INFO:teuthology.orchestra.run.smithi012.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51431.log.gz 2023-10-30T12:57:59.575 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.49188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53994.log 2023-10-30T12:57:59.575 INFO:teuthology.orchestra.run.smithi012.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49188.log.gz 2023-10-30T12:57:59.576 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43396.log 2023-10-30T12:57:59.578 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53994.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53994.log.gz 2023-10-30T12:57:59.578 INFO:teuthology.orchestra.run.smithi012.stderr: 86.9% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-30T12:57:59.579 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49488.log 2023-10-30T12:57:59.579 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43396.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43396.log.gz 2023-10-30T12:57:59.580 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43781.log 2023-10-30T12:57:59.581 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54020.log 2023-10-30T12:57:59.582 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49488.log.gz 2023-10-30T12:57:59.583 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52094.log 2023-10-30T12:57:59.583 INFO:teuthology.orchestra.run.smithi012.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43781.log.gz 2023-10-30T12:57:59.584 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54020.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54020.log.gz -5 2023-10-30T12:57:59.584 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50042.log 2023-10-30T12:57:59.586 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52762.log 2023-10-30T12:57:59.586 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52094.log.gz 2023-10-30T12:57:59.587 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50042.log: gzip -5 88.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.50042.log.gz -- 2023-10-30T12:57:59.587 INFO:teuthology.orchestra.run.smithi012.stderr: /var/log/ceph/ceph-client.admin.52600.log 2023-10-30T12:57:59.588 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53249.log 2023-10-30T12:57:59.589 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52762.log.gz 2023-10-30T12:57:59.590 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52600.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52600.log.gz 2023-10-30T12:57:59.590 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53644.log 2023-10-30T12:57:59.591 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53249.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53249.log.gz 2023-10-30T12:57:59.592 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43780.log 2023-10-30T12:57:59.593 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51212.log 2023-10-30T12:57:59.593 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53644.log.gz 2023-10-30T12:57:59.594 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46651.log 2023-10-30T12:57:59.595 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43780.log.gz 2023-10-30T12:57:59.596 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51212.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.0.45116.log 2023-10-30T12:57:59.596 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51212.log.gz 2023-10-30T12:57:59.597 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51326.log 2023-10-30T12:57:59.598 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46651.log.gz 2023-10-30T12:57:59.598 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.0.45116.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45116.log.gz 2023-10-30T12:57:59.599 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50441.log 2023-10-30T12:57:59.600 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51326.log: gzip -5 88.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.51326.log.gz -- 2023-10-30T12:57:59.601 INFO:teuthology.orchestra.run.smithi012.stderr: /var/log/ceph/ceph-client.admin.49832.log 2023-10-30T12:57:59.602 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53696.log 88.3% 2023-10-30T12:57:59.602 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50441.log.gz 2023-10-30T12:57:59.603 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50068.log 2023-10-30T12:57:59.603 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49832.log.gz 2023-10-30T12:57:59.605 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49540.log 2023-10-30T12:57:59.605 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53696.log.gz 2023-10-30T12:57:59.607 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50068.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50068.log.gz 2023-10-30T12:57:59.607 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53407.log 2023-10-30T12:57:59.608 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49540.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49540.log.gz 2023-10-30T12:57:59.608 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43368.log 2023-10-30T12:57:59.610 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53407.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53407.log.gz 2023-10-30T12:57:59.610 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54608.log 2023-10-30T12:57:59.612 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52381.log 2023-10-30T12:57:59.612 INFO:teuthology.orchestra.run.smithi012.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43368.log.gz 2023-10-30T12:57:59.613 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54608.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54608.log.gz 2023-10-30T12:57:59.613 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51779.log 2023-10-30T12:57:59.615 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43533.log 2023-10-30T12:57:59.615 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52381.log.gz 2023-10-30T12:57:59.616 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53670.log 2023-10-30T12:57:59.616 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51779.log.gz 2023-10-30T12:57:59.618 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54530.log 2023-10-30T12:57:59.618 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43533.log.gz 2023-10-30T12:57:59.619 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53670.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53670.log.gz 2023-10-30T12:57:59.619 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49780.log 2023-10-30T12:57:59.621 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55244.log 2023-10-30T12:57:59.621 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54530.log.gz 2023-10-30T12:57:59.622 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54816.log 2023-10-30T12:57:59.623 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49780.log.gz 2023-10-30T12:57:59.624 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51537.log 2023-10-30T12:57:59.624 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55244.log.gz 2023-10-30T12:57:59.625 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54816.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54816.log.gz 2023-10-30T12:57:59.625 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44505.log 2023-10-30T12:57:59.627 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51537.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51537.log.gz 2023-10-30T12:57:59.627 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55270.log 2023-10-30T12:57:59.628 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44505.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44505.log.gz 2023-10-30T12:57:59.629 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48336.log 2023-10-30T12:57:59.630 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55270.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55270.log.gz -5 2023-10-30T12:57:59.631 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52925.log 2023-10-30T12:57:59.632 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38206.log 2023-10-30T12:57:59.632 INFO:teuthology.orchestra.run.smithi012.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48336.log.gz 2023-10-30T12:57:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43664.log 2023-10-30T12:57:59.634 INFO:teuthology.orchestra.run.smithi012.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52925.log.gz 2023-10-30T12:57:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38206.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38206.log.gz 2023-10-30T12:57:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45301.log 2023-10-30T12:57:59.636 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44003.log 2023-10-30T12:57:59.637 INFO:teuthology.orchestra.run.smithi012.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43664.log.gz 2023-10-30T12:57:59.638 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50357.log 2023-10-30T12:57:59.638 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45301.log.gz 2023-10-30T12:57:59.639 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44003.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44003.log.gz 2023-10-30T12:57:59.640 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50357.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50357.log.gz 2023-10-30T12:57:59.641 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46573.log 2023-10-30T12:57:59.642 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54374.log 2023-10-30T12:57:59.644 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48090.log 2023-10-30T12:57:59.644 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46573.log.gz 2023-10-30T12:57:59.646 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49243.log 2023-10-30T12:57:59.646 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54374.log.gz 2023-10-30T12:57:59.647 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48090.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48090.log.gz -5 2023-10-30T12:57:59.647 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53005.log 2023-10-30T12:57:59.649 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50844.log 2023-10-30T12:57:59.649 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49243.log.gz 2023-10-30T12:57:59.650 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54181.log 2023-10-30T12:57:59.650 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53005.log.gz 2023-10-30T12:57:59.652 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54328.log 2023-10-30T12:57:59.652 INFO:teuthology.orchestra.run.smithi012.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50844.log.gz 2023-10-30T12:57:59.654 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54181.log: gzip -5 --verbose 87.7% -- -- replaced with /var/log/ceph/ceph-client.admin.54181.log.gz /var/log/ceph/ceph-client.admin.51675.log 2023-10-30T12:57:59.654 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-30T12:57:59.655 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54328.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.54328.log.gz -5 2023-10-30T12:57:59.655 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53940.log 2023-10-30T12:57:59.656 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45405.log 2023-10-30T12:57:59.656 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51675.log.gz 2023-10-30T12:57:59.657 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53940.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53940.log.gz 2023-10-30T12:57:59.658 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38083.log 2023-10-30T12:57:59.660 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45405.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45405.log.gz -5 2023-10-30T12:57:59.660 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49912.log 2023-10-30T12:57:59.661 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43612.log 2023-10-30T12:57:59.661 INFO:teuthology.orchestra.run.smithi012.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38083.log.gz 2023-10-30T12:57:59.662 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51511.log 2023-10-30T12:57:59.663 INFO:teuthology.orchestra.run.smithi012.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49912.log.gz 2023-10-30T12:57:59.664 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43612.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43612.log.gz 2023-10-30T12:57:59.664 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-30T12:57:59.665 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51511.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51511.log.gz 2023-10-30T12:57:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph.audit.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-30T12:57:59.669 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53591.log 2023-10-30T12:57:59.670 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-30T12:57:59.672 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48389.log 2023-10-30T12:57:59.672 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53591.log.gz 2023-10-30T12:57:59.676 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.48389.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48389.log.gz 2023-10-30T12:57:59.676 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44713.log 2023-10-30T12:57:59.677 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43791.log 2023-10-30T12:57:59.678 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44713.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44713.log.gz 2023-10-30T12:57:59.680 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43791.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43791.log.gz 2023-10-30T12:57:59.680 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50094.log 2023-10-30T12:57:59.682 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43638.log 2023-10-30T12:57:59.683 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50094.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50094.log.gz 2023-10-30T12:57:59.684 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43638.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43638.log.gz 2023-10-30T12:57:59.685 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-30T12:57:59.687 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51859.log 2023-10-30T12:57:59.690 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.51859.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51859.log.gz 2023-10-30T12:57:59.690 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49858.log 2023-10-30T12:57:59.693 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49858.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49858.log.gz 2023-10-30T12:57:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51590.log 2023-10-30T12:57:59.697 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51590.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51590.log.gz 2023-10-30T12:57:59.697 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50147.log 2023-10-30T12:57:59.700 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50147.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50147.log.gz 2023-10-30T12:57:59.701 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48116.log 2023-10-30T12:57:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48116.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48116.log.gz 2023-10-30T12:57:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51379.log 2023-10-30T12:57:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51379.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51379.log.gz 2023-10-30T12:57:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50173.log 2023-10-30T12:57:59.710 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50173.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50173.log.gz 2023-10-30T12:57:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45353.log 2023-10-30T12:57:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45353.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45353.log.gz 2023-10-30T12:57:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43989.log 2023-10-30T12:57:59.717 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43989.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43989.log.gz 2023-10-30T12:57:59.718 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53329.log 2023-10-30T12:57:59.721 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53329.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53329.log.gz 2023-10-30T12:57:59.721 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54302.log 2023-10-30T12:57:59.724 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54302.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54302.log.gz 2023-10-30T12:57:59.724 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50573.log 2023-10-30T12:57:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50573.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50573.log.gz 2023-10-30T12:57:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53433.log 2023-10-30T12:57:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53433.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53433.log.gz 2023-10-30T12:57:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54790.log 2023-10-30T12:57:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54790.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54790.log.gz 2023-10-30T12:57:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52466.log 2023-10-30T12:57:59.738 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52466.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52466.log.gz 2023-10-30T12:57:59.738 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46976.log 2023-10-30T12:57:59.745 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46976.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46976.log.gz 2023-10-30T12:57:59.745 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49409.log 2023-10-30T12:57:59.745 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49409.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49409.log.gz 2023-10-30T12:57:59.746 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54556.log 2023-10-30T12:57:59.748 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54556.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54556.log.gz 2023-10-30T12:57:59.749 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49728.log 2023-10-30T12:57:59.751 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49728.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49728.log.gz 2023-10-30T12:57:59.752 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54276.log 2023-10-30T12:57:59.755 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54276.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54276.log.gz 2023-10-30T12:57:59.755 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54452.log 2023-10-30T12:57:59.758 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54452.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54452.log.gz 2023-10-30T12:57:59.758 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51649.log 2023-10-30T12:57:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51649.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51649.log.gz 2023-10-30T12:57:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50896.log 2023-10-30T12:57:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53748.log 2023-10-30T12:57:59.765 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50896.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50896.log.gz 2023-10-30T12:57:59.767 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53748.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53748.log.gz 2023-10-30T12:57:59.768 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50924.log 2023-10-30T12:57:59.769 INFO:teuthology.orchestra.run.smithi154.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-30T12:57:59.771 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50924.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50924.log.gz 2023-10-30T12:57:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52433.log 2023-10-30T12:57:59.775 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52433.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52433.log.gz 2023-10-30T12:57:59.775 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50739.log 2023-10-30T12:57:59.778 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50739.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50739.log.gz 2023-10-30T12:57:59.779 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55057.log 2023-10-30T12:57:59.782 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55057.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55057.log.gz 2023-10-30T12:57:59.782 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44609.log 2023-10-30T12:57:59.785 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44609.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44609.log.gz 2023-10-30T12:57:59.786 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45275.log 2023-10-30T12:57:59.789 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45275.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45275.log.gz 2023-10-30T12:57:59.789 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46924.log 2023-10-30T12:57:59.792 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46924.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46924.log.gz 2023-10-30T12:57:59.793 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43586.log 2023-10-30T12:57:59.795 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43586.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43586.log.gz 2023-10-30T12:57:59.796 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44470.log 2023-10-30T12:57:59.799 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44470.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44470.log.gz 2023-10-30T12:57:59.799 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44897.log 2023-10-30T12:57:59.803 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44897.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44897.log.gz 2023-10-30T12:57:59.805 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54072.log 2023-10-30T12:57:59.806 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54072.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54072.log.gz 2023-10-30T12:57:59.807 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50870.log 2023-10-30T12:57:59.810 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50870.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50870.log.gz 2023-10-30T12:57:59.810 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50415.log 2023-10-30T12:57:59.813 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50415.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50415.log.gz 2023-10-30T12:57:59.813 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49461.log 2023-10-30T12:57:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49461.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49461.log.gz 2023-10-30T12:57:59.817 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51028.log 2023-10-30T12:57:59.820 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51028.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51028.log.gz 2023-10-30T12:57:59.820 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51000.log 2023-10-30T12:57:59.824 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51000.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51000.log.gz 2023-10-30T12:57:59.824 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53800.log 2023-10-30T12:57:59.827 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53800.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53800.log.gz 2023-10-30T12:57:59.827 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49215.log 2023-10-30T12:57:59.830 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49215.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49215.log.gz 2023-10-30T12:57:59.830 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46815.log 2023-10-30T12:57:59.831 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-10-30T12:57:59.834 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46815.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46815.log.gz 2023-10-30T12:57:59.834 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50765.log 2023-10-30T12:57:59.837 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50765.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50765.log.gz 2023-10-30T12:57:59.838 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50953.log 2023-10-30T12:57:59.841 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50953.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50953.log.gz 2023-10-30T12:57:59.841 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45483.log 2023-10-30T12:57:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45483.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45483.log.gz 2023-10-30T12:57:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51885.log 2023-10-30T12:57:59.847 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51885.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51885.log.gz 2023-10-30T12:57:59.848 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46755.log 2023-10-30T12:57:59.852 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46755.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46755.log.gz 2023-10-30T12:57:59.852 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50228.log 2023-10-30T12:57:59.854 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50228.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50228.log.gz 2023-10-30T12:57:59.854 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48363.log 2023-10-30T12:57:59.857 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54152.log 2023-10-30T12:57:59.858 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48363.log.gz 2023-10-30T12:57:59.860 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43792.log 2023-10-30T12:57:59.862 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54152.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54152.log.gz 2023-10-30T12:57:59.864 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43792.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43792.log.gz 2023-10-30T12:57:59.865 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46789.log 2023-10-30T12:57:59.868 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46789.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46789.log.gz 2023-10-30T12:57:59.869 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54949.log 2023-10-30T12:57:59.872 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54949.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54949.log.gz 2023-10-30T12:57:59.872 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37971.log 2023-10-30T12:57:59.875 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37971.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37971.log.gz 2023-10-30T12:57:59.876 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44444.log 2023-10-30T12:57:59.879 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44444.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44444.log.gz 2023-10-30T12:57:59.879 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51937.log 2023-10-30T12:57:59.882 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43795.log 2023-10-30T12:57:59.882 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51937.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51937.log.gz 2023-10-30T12:57:59.885 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43795.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43795.log.gz 2023-10-30T12:57:59.885 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46677.log 2023-10-30T12:57:59.888 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46677.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46677.log.gz 2023-10-30T12:57:59.888 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48281.log 2023-10-30T12:57:59.891 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48281.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48281.log.gz 2023-10-30T12:57:59.892 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51054.log 2023-10-30T12:57:59.895 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51054.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51054.log.gz 2023-10-30T12:57:59.895 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55083.log 2023-10-30T12:57:59.898 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55083.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55083.log.gz 2023-10-30T12:57:59.899 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49269.log 2023-10-30T12:57:59.901 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43990.log 2023-10-30T12:57:59.902 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49269.log.gz 2023-10-30T12:57:59.903 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50311.log 2023-10-30T12:57:59.905 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43990.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43990.log.gz 2023-10-30T12:57:59.908 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50311.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50311.log.gz 2023-10-30T12:57:59.908 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44687.log 2023-10-30T12:57:59.911 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44687.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44687.log.gz 2023-10-30T12:57:59.912 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51457.log 2023-10-30T12:57:59.914 INFO:teuthology.orchestra.run.smithi154.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-30T12:57:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51457.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51457.log.gz 2023-10-30T12:57:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52149.log 2023-10-30T12:57:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52149.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52149.log.gz 2023-10-30T12:57:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51563.log 2023-10-30T12:57:59.921 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51563.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51563.log.gz 2023-10-30T12:57:59.922 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43796.log 2023-10-30T12:57:59.925 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43796.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43796.log.gz 2023-10-30T12:57:59.925 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43779.log 2023-10-30T12:57:59.928 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43779.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43779.log.gz 2023-10-30T12:57:59.929 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53116.log 2023-10-30T12:57:59.931 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53116.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53116.log.gz 2023-10-30T12:57:59.932 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43176.log 2023-10-30T12:57:59.935 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43176.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43176.log.gz 2023-10-30T12:57:59.935 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2023-10-30T12:57:59.941 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52548.log 2023-10-30T12:57:59.946 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44766.log 2023-10-30T12:57:59.948 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52548.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52548.log.gz 2023-10-30T12:57:59.951 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44766.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44766.log.gz 2023-10-30T12:57:59.952 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46870.log 2023-10-30T12:57:59.953 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48526.log 2023-10-30T12:57:59.955 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46870.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46870.log.gz 2023-10-30T12:57:59.959 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48526.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48526.log.gz 2023-10-30T12:57:59.960 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54098.log 2023-10-30T12:57:59.965 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51352.log 2023-10-30T12:57:59.967 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54098.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54098.log.gz 2023-10-30T12:57:59.971 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51352.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51352.log.gz 2023-10-30T12:57:59.972 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44557.log 2023-10-30T12:57:59.973 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44844.log 2023-10-30T12:57:59.975 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44557.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44557.log.gz 2023-10-30T12:57:59.980 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44844.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44844.log.gz 2023-10-30T12:57:59.980 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52680.log 2023-10-30T12:57:59.982 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52407.log 2023-10-30T12:57:59.983 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52680.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52680.log.gz 2023-10-30T12:57:59.988 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52407.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52407.log.gz 2023-10-30T12:57:59.989 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45457.log 2023-10-30T12:57:59.994 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51106.log 2023-10-30T12:57:59.995 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45457.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45457.log.gz 2023-10-30T12:57:59.998 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49162.log 2023-10-30T12:58:00.001 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51106.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51106.log.gz 2023-10-30T12:58:00.003 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49162.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49162.log.gz 2023-10-30T12:58:00.710 INFO:teuthology.orchestra.run.smithi012.stderr: 90.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-30T12:58:02.425 INFO:teuthology.orchestra.run.smithi154.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-30T12:58:02.913 INFO:teuthology.orchestra.run.smithi012.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-30T12:58:03.144 INFO:teuthology.orchestra.run.smithi154.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-30T12:58:03.595 INFO:teuthology.orchestra.run.smithi012.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-30T12:58:03.838 INFO:teuthology.orchestra.run.smithi154.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-30T12:58:04.720 INFO:teuthology.orchestra.run.smithi154.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-30T12:58:04.791 INFO:teuthology.orchestra.run.smithi012.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-30T12:58:05.122 INFO:teuthology.orchestra.run.smithi012.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-30T12:58:10.061 INFO:teuthology.orchestra.run.smithi154.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-30T12:58:10.062 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-10-30T12:58:10.062 INFO:teuthology.orchestra.run.smithi154.stderr:real 0m10.779s 2023-10-30T12:58:10.062 INFO:teuthology.orchestra.run.smithi154.stderr:user 0m27.929s 2023-10-30T12:58:10.062 INFO:teuthology.orchestra.run.smithi154.stderr:sys 0m1.046s 2023-10-30T12:58:18.198 INFO:teuthology.orchestra.run.smithi012.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-30T12:58:28.767 INFO:teuthology.orchestra.run.smithi012.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-30T12:58:35.726 INFO:teuthology.orchestra.run.smithi012.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2023-10-30T12:58:35.728 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-30T12:58:35.728 INFO:teuthology.orchestra.run.smithi012.stderr:real 0m36.453s 2023-10-30T12:58:35.729 INFO:teuthology.orchestra.run.smithi012.stderr:user 1m36.662s 2023-10-30T12:58:35.729 INFO:teuthology.orchestra.run.smithi012.stderr:sys 0m7.663s 2023-10-30T12:58:35.730 INFO:tasks.ceph:Archiving logs... 2023-10-30T12:58:35.731 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026/remote/smithi012/log 2023-10-30T12:58:35.732 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-30T12:58:46.129 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026/remote/smithi154/log 2023-10-30T12:58:46.130 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-30T12:58:48.978 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-30T12:58:48.987 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-30T12:58:48.988 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-30T12:58:48.992 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-30T12:58:49.070 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-30T12:58:49.080 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-30T12:58:49.080 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-30T12:58:49.084 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-30T12:58:49.097 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2023-10-30T12:58:49.098 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2023-10-30T12:58:49.098 INFO:teuthology.orchestra.run.smithi012.stdout:*hv01.front.sepi 44.4.53.2 2 u 5 64 377 0.078 -1.054 0.405 2023-10-30T12:58:49.098 INFO:teuthology.orchestra.run.smithi012.stdout:+hv02.front.sepi 63.231.80.2 3 u 8 64 377 0.078 +2.443 0.391 2023-10-30T12:58:49.099 INFO:teuthology.orchestra.run.smithi012.stdout:+hv03.front.sepi 74.6.168.72 3 u 9 64 377 0.078 +0.621 0.475 2023-10-30T12:58:49.099 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2023-10-30T12:58:49.101 INFO:teuthology.orchestra.run.smithi154.stdout: remote refid st t when poll reach delay offset jitter 2023-10-30T12:58:49.101 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================== 2023-10-30T12:58:49.101 INFO:teuthology.orchestra.run.smithi154.stdout:*hv01.front.sepi 44.4.53.2 2 u 5 64 377 0.096 -11.766 0.693 2023-10-30T12:58:49.101 INFO:teuthology.orchestra.run.smithi154.stdout:+hv02.front.sepi 63.231.80.2 3 u 2 64 377 0.107 -7.243 1.019 2023-10-30T12:58:49.102 INFO:teuthology.orchestra.run.smithi154.stdout:+hv03.front.sepi 74.6.168.72 3 u 14 64 377 0.096 -10.215 0.709 2023-10-30T12:58:49.102 INFO:teuthology.orchestra.run.smithi154.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2023-10-30T12:58:49.102 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-30T12:58:49.112 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-30T12:58:49.112 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-30T12:58:49.138 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-30T12:58:49.171 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-30T12:58:49.220 INFO:teuthology.task.internal:Duration was 2173.420011 seconds 2023-10-30T12:58:49.220 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-30T12:58:49.255 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-30T12:58:49.256 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-30T12:58:49.260 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-30T12:58:49.367 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-30T12:58:49.367 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2023-10-30T12:58:49.368 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-30T12:58:49.391 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2023-10-30T12:58:49.392 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-30T12:58:49.401 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-30T12:58:49.402 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-30T12:58:49.440 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-30T12:58:49.475 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-30T12:58:49.476 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-30T12:58:49.524 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-30T12:58:49.601 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-30T12:58:49.612 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-30T12:58:49.613 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-30T12:58:49.634 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-30T12:58:49.649 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-30T12:58:49.659 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-30T12:58:49.680 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-30T12:58:49.695 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2023-10-30T12:58:49.702 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2023-10-30T12:58:49.807 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-30T12:58:49.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:58:49.816 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-30T12:58:49.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:58:49.854 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-30T12:58:49.865 INFO:teuthology.task.internal:Transferring archived files... 2023-10-30T12:58:49.866 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026/remote/smithi012 2023-10-30T12:58:49.867 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-30T12:58:49.894 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026/remote/smithi154 2023-10-30T12:58:49.895 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-30T12:58:49.915 INFO:teuthology.task.internal:Removing archive directory... 2023-10-30T12:58:49.916 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-30T12:58:49.935 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-30T12:58:49.960 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-30T12:58:49.972 INFO:teuthology.task.internal:Not uploading archives. 2023-10-30T12:58:49.972 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-30T12:58:49.997 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-30T12:58:49.997 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-30T12:58:50.001 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-30T12:58:50.007 INFO:teuthology.orchestra.run.smithi012.stdout: 657970 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 30 12:58 /home/ubuntu/cephtest 2023-10-30T12:58:50.014 INFO:teuthology.orchestra.run.smithi154.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 30 12:58 /home/ubuntu/cephtest 2023-10-30T12:58:50.015 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-30T12:58:50.039 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-30T12:58:50.106 INFO:teuthology.nuke:Checking targets against current locks 2023-10-30T12:58:50.132 DEBUG:teuthology.nuke:shortname: smithi012 2023-10-30T12:58:50.133 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-30T12:58:50.156 DEBUG:teuthology.nuke:shortname: smithi154 2023-10-30T12:58:50.156 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-30T12:58:50.160 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-30 12:13:00.703267', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOIXUlZNg24Ie/pDwa4MO4k4bufpZa63YdkTdS0B3ACISQbEG5xzJxPppwdXu1r1e/5mohao6G9qZoXXF4DvJ6k='} 2023-10-30T12:58:50.192 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441026', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-30 12:13:00.705372', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFC+TrPsAKqCG2WeGXqVschBg9bQSR3yot22TUIQ+OSTc5On7+Srgg955m9m+XBsGGUGsXdNNBTJIM5PraPhF/4='} 2023-10-30T12:58:50.207 INFO:teuthology.orchestra.console.smithi012:Power off 2023-10-30T12:58:50.208 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-30T12:58:50.232 DEBUG:teuthology.orchestra.console.smithi012:power off output: Chassis Power Control: Down/Off 2023-10-30T12:58:50.232 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T12:58:50.247 INFO:teuthology.orchestra.console.smithi154:Power off 2023-10-30T12:58:50.248 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-30T12:58:50.258 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2023-10-30T12:58:50.292 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off 2023-10-30T12:58:50.292 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T12:58:50.315 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2023-10-30T12:58:54.260 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T12:58:54.316 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T12:58:54.371 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2023-10-30T12:58:54.426 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2023-10-30T12:58:58.376 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T12:58:58.427 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T12:58:58.486 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2023-10-30T12:58:58.537 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2023-10-30T12:58:58.638 INFO:teuthology.orchestra.console.smithi154:Power off completed 2023-10-30T12:59:02.488 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T12:59:02.598 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is off 2023-10-30T12:59:02.700 INFO:teuthology.orchestra.console.smithi012:Power off completed 2023-10-30T12:59:02.806 INFO:teuthology.run:Summary data: client.0-kernel-sha1: eacf00cc51e204f401b4d5d4f84825a68f373c74 description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} duration: 2173.420011281967 failure_reason: 'Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi012 with status 128: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=314390cf30fb4062bd25fa01f7bab155 status: fail success: false 2023-10-30T12:59:02.807 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-30T12:59:02.903 INFO:teuthology.run:FAIL